Track Down Stealth Fileless
Injection-based Nginx
Backdoor in the Attack
Peter Syu, Jr-Wei Huang

Speaker's Bio

Peter Syu

Jr-Wei Huang

Security Researcher@ TeamT5

Product Developer@ TeamT5

His research mainly focuses on
incident response and malware analysis.

His research interests are in the areaof system security and threat hunting.

2

AGENDA
01

Overview of the Attack

02

Malware Analysis

03

Other Nginx-based Backdoor

04

Detection

05

Conclusions

AGENDA
01

Overview of the Attack

02

Malware Analysis

03

Other Nginx-based Backdoor

04

Detection

05

Conclusions

The story starts from...
◆

We found a suspicious string in a server’s debug log, and we decided tofollow up
echo
CmVjaG8gL1Rk
NldGb0FBQVRt
MXJSR0FnQWhB
UllBQUFCMEwr
V2o0bGYvVk81
ZEFCY0wzQVNB
VURJbTdoMTlH
L1B5eXpQczdQ
ZzM4T... | base64-d | bash
5Overview

PwnKitCVE-2021-4034

00000000-hot.bak

Log erased tool

f92b4c07.xz

8cf39813-hot.bakcron injector

e170bf30-hot.bak

6

Overviewe170bf30-hot.bak

2.
Inject NginxSpy tocron injector

Nginx master

Nginx injector
Nginx master process
NginxSpy
1. Inject Nginx injector to cron

cron process

NginxStealth3.
Inject NginxStealth toNginx worker

Nginx worker process7

Nginx Overview
Open Source Software◆ HTTP server for various purpose◆ Static content delivery◆ Reverse proxy◆ Load balancer◆ Cache server◆ etc…
◆ Nginx has one master process andone or more worker processes◆

source: http://www.aosabook.org/en/nginx.html#fig.nginx.arch8

Nginx is module-oriented
Nginx modules come in different forms:◆ core modules◆ event modules◆ phase handlers◆ filters◆ upstreams◆ load balancers◆ Nginx added support for dynamic
modules in version 1.9.11◆

source : https://zq99299.github.io/note-architect/ztc/06/08.html

9

AGENDA
01

Overview of the Attack

02

Malware Analysis

03

Other Nginx-based-Backdoor

04

Detection

05

Conclusions

In-Depth Analysis of Malware
Fileless injection technique◆ NginxStealth◆ Filtering specific connection◆ Evading logging mechanism◆ NginxSpy◆ Backdoor communication◆ Backdoor functionality◆

11Fileless Injection Technique

Overview of process injection
Linux reflective code injection◆ Allows an attacker to inject a library into a victim from memory◆ Reflective code injection is not as commonly used in Linux◆

Syscall: memfd_create
memfd = syscall(SYS_memfd_create, "", MFD_CLOEXEC);
write(memfd,elf_buf,file_size);sprintf(memFilePath,"/proc/self/fd/%d",memfd);
execve(memFilePath, NULL,NULL);
Zombie Ant Farm: Practical Tips for Playing Hide and Seek with Linux EDRs
Blackhat-USA 201913

Detection – memfd_create◆

Examining the fd under proc can identify the presence of memfd usage

14

Process injector in this attack
A unique approach of reflective code injection on Linux◆ Based on ptrace to modify memory◆ Utilize the concept of ELF application loader◆ Support x64 process

Injector

◆

.text

.data
shellcode
backdoor

15

Process injector in this attack◆

In the beginning, injector copies the mmap/munmap function into
victim process
.text

.text
mmap func

.data
.data
shellcode
backdoor

Injectorcron

16Process injector in this attack◆

Afterwards, the attacker has the ability to open new segments with any
desired permissions.
.text
mmap func

.data

New

cron

17

Process injector in this attack◆

Move shellcode and backdoor into cron process
.text

.text
mmap func

.data
.data
shellcode
backdoor

Injectorshellcode
backdoorcron

18Process injector in this attack◆

The injector calls the shellcode as a function through SETREGISTER in
ptrace.
.text
mmap func.datavoid * shellcode (dlopen_mode_func,
libc_dlsym_func,
backdoor_addr,
backdoor_length){ ...}

shellcode
backdoorcron

19Process injector in this attack
Shellcode == ELF loader
shellcode
backdoorcron

1. preload API function
2. map LD_LOAD segments
3. Dynamic link library
4. Address relocate
5. Call init function

20

ELF Loader◆

The shellcode loads the data into the segments by reading thestructures of the backdoor ELF and loads the required dependencies.
shellcodeshellcodeshellcode

backdoor

backdoor

backdoor

cron

.text

.text

.data

.data

Dependence

Dependence

cron

cron

21

ELF Loader◆

Since the backdoor library’s image will exist in an arbitrary address,
loader should repair the code that depends on the address.
shellcode
backdoor

.text
.data
Dependence

cron

22

ELF Loader◆

The shellcode reads the init array and executes the recorded functions
from within it.

Backdoor

Shellcode23Detection – ELF Loader
The maps file doesn’t show any connections between the running process
and the backdoor.
◆ In such a situation, blue team could focus on pinpoint the consistency
between the native ELF and the loaded module.
◆

24

NginxStealthSome Nginx structures
ngx_cycle : ngx_cycle_t◆ An event cycle object◆ ngx_http_module : ngx_http_module_t◆ Defines the module context of an HTTP
module.
◆ ngx_http_core_module : ngx_module_t◆ A required structure used to define somebasic module hooks.
◆

Source:https://github.com/nginx/nginx/blob/master/src/http/ngx_http_config.h#definengx_http_cycle_get_module_main_conf(cycle, module) \(cycle->conf_ctx[ngx_http_module.index] ? \((ngx_http_conf_ctx_t *)cycle->
conf_ctx[ngx_http_module.index])\->main_conf[module.ctx_index]: \
NULL)

26

#define
ngx_http_cycle_get_module_main_conf(cycle, module) \(cycle->conf_ctx[ngx_http_module.index] ? \((ngx_http_conf_ctx_t *)cycle->
conf_ctx[ngx_http_module.index])\->main_conf[module.ctx_index]: \
NULL)

27
Source: https://www.codedump.info/post/20190103-nginx-config-parse/

Hook phase_engine’s handler
Make sure checker is not 0◆ Replace handler with hooked function◆

struct ngx_http_core_main_conf_t{
ngx_array_t servers;
ngx_http_phase_engine_t phase_engine;.
.
.
ngx_http_phase_t phases[11];};typedef struct {ngx_http_phase_handler_t *handlers;
ngx_uint_t server_rewrite_index;
ngx_uint_t location_rewrite_index;} ngx_http_phase_engine_t;
struct ngx_http_phase_handler_s {ngx_http_phase_handler_pt checker;
ngx_http_handler_pt handler;
ngx_uint_t next;};28

Nginx phases
Each HTTP request passes through asequence of phases◆ The same phase has the same checker
function, and the handler stores the module
processing function◆ Nginx will not use the handler directly, but
implement a specific checker function foreach phase, and call the handler in the
checker◆ Map the handler in phases to thephase_engine◆ ngx_http_init_phase_handlers◆

NGX_HTTP_POST_READ_PHASE
NGX_HTTP_SERVER_REWRITE_PHASE
NGX_HTTP_FIND_CONFIG_PHASE

NGX_HTTP_REWRITE_PHASE
NGX_HTTP_POST_REWRITE_PHASE
NGX_HTTP_PREACCESS_PHASE
NGX_HTTP_ACCESS_PHASE

NGX_HTTP_POST_ACCESS_PHASE
NGX_HTTP_TRY_FILES_PHASE
NGX_HTTP_CONTENT_PHASE
NGX_HTTP_LOG_PHASE
29

Request Flow
Host: localhost
Accept-Encoding: gzip,encoding

HTTP
Requestngx_http_init_request
POST /index.php HTTP/1.1

Ngx_http_process_request_line

ngx_http_process_request

ngx_http_read_request_header

ngx_http_handler

ngx_http_parse_header_line

ngx_http_core_run_phases

Ngx_http_process_request_line

ngx_http_read_request_header

ngx_http_parse_request_line

Parse header name and find thecorresponding handler in
ngx_http_headers_in
Call the handler to process theheaderHeader filter
Body filter
ngx_http_write_filter

End

30

Hook Logging Mechanism
Make sure NGX_HTTP_LOG_PHASE.handler.nelts is not 0◆ Replace elts with hooked function◆ Saved the old elts◆

struct ngx_http_core_main_conf_t{
ngx_array_t servers;
ngx_http_phase_engine_t phase_engine;.
.
.
ngx_http_phase_t phases[11];};struct ngx_http_phase_t{
ngx_array_t handlers;};

struct ngx_array_t{
void *elts;
ngx_uint_t nelts;
size_t size;
ngx_uint_t nalloc;
ngx_pool_t *pool;};31

Nginx Logging◆

ngx_http_log_request

32

Filtering Specific Connection
Filter request with User-Agent :360 Safe Browser 2.0◆ If not, pass the request to original
handler◆

GET / HTTP/1.0
User-Agent: 360 Safe Browser 2.0

33

Pass the fd to NginxSpy
Set the log_level to NGX_LOG_STDERR◆ Connect to /var/run/nginx.sock◆ Send the original connection’s fd to
NginxSpy◆34

Evading Logging Mechanism
Filter request with User-Agent :360 Safe Browser 2.0◆ If yes, do nothing == no logging◆◆

If not, pass the request to original
logging handler

GET / HTTP/1.0
User-Agent: 360 Safe Browser 2.0

35

NginxSpy

Recv the fd
Remove old unix domain socket andbind a new one◆ /var/run/nginx.sock◆ Check the message from NginxStealth◆ Get the fd from nginx’s connection◆ Pass the fd to backdoor thread◆37

Key Exchange Mode #1
Infected Nginx Server

Actor
Send the request withmalicious user agent
Send the magic

Check the magic

Send the RC4 Key

GET / HTTP/1.0
User-Agent: 360 Safe Browser 2.0\xe5\xcf\xff\x02\xb1\x69\x20\x69\x75\x15\x99\x45\x20\x2e\x0a\x10\xe3\x10\xb3\x70\x8d\x6d\xab\x54\x52\x79\xf0\x1b\x78\x5e\xd1\x46\xe5\xcf\xff\x02\xb1\x69\x20\x69\x75\x15\x99\x45\x20\x2e\x0a\x10\xe3\x10\xb3\x70\x8d\x6d\xab\x54\x52\x79\xf0\x1b\x78\x5e\xd1\x46

32 byte RC4 Key

Check the user agent
Check whether magic ==hardcoded magic
Send the hardcoded
magic back
Recv the RC4 Key
38

Key Exchange Mode #2
Infected Nginx Server

Actor
Send the request withmalicious user agent
Check the magic

Send the magic

Recv the RC4 Key

GET / HTTP/1.0
User-Agent: 360 Safe Browser 2.0\xe5\xcf\xff\x02\xb1\x69\x20\x69\x75\x15\x99\x45\x20\x2e\x0a\x10\xe3\x10\xb3\x70\x8d\x6d\xab\x54\x52\x79\xf0\x1b\x78\x5e\xd1\x46\xe5\xcf\xff\x02\xb1\x69\x20\x69\x75\x15\x99\x45\x20\x2e\x0a\x10\xe3\x10\xb3\x70\x8d\x6d\xab\x54\x52\x79\xf0\x1b\x78\x5e\xd1\x46

32 byte RC4 Key

Check the user agent
Send the hardcodedmagic
Check whether magic ==hardcoded magic
Generate random 32
byte as RC4 Key
39

NginxSpy Commands
CommandID

Description

0

Show current directory (getcwd)

1

Show version

2

Kill cron process

3

Kill cron process and inject it with recv payload

4

Show current user’s info
(getuid, geteuid, getpwuid)5

Get login information (setutxent)

6

Get login information (read wtmp)

7Enum processes

8

Change directory

9

Enum specific directory

10

Read /proc/modules

11

Delete file or directory (remove)

12

Read data from file

13

Read data from file14

Write data to file40

Backdoor Communication : HTTP
GET / HTTP/1.0
User-Agent: 360 Safe Browser 2.0

1. Send the request withmalicious user agent

2. Pass the fd to NGINXSPY

TCP socket

Actor

HTTP

\xe5\xcf\xff\x02\xb1\x69\x20\x69\x75\x15\x99\x45\x20\x2e\x0a\x10\xe3\x10\xb3\x70\x8d\x6d\xab\x54\x52\x79\xf0\x1b\x78\x5e\xd1\x46

4. Recv magic in TCP socket

TCP socket

Infected Nginx Server
3. Send the hardcoded magic
back in TCP socket without
any encapsulation

41

Backdoor Communication : HTTPSGET / HTTP/1.0
User-Agent: 360 Safe Browser 2.0

1. Send the request withmalicious user agent

2. Pass the fd to NGINXSPY

TCP socket

TLS / SSL

HTTP

\xe5\xcf\xff\x02\xb1\x69\x20\x69\x75\x15\x99\x45\x20\x2e\x0a\x10\xe3\x10\xb3\x70\x8d\x6d\xab\x54\x52\x79\xf0\x1b\x78\x5e\xd1\x46

Actor

TCP socket

TLS / SSL

HTTP

\xe5\xcf\xff\x02\xb1\x69\x20\x69\x75\x15\x99\x45\x20\x2e\x0a\x10\xe3\x10\xb3\x70\x8d\x6d\xab\x54\x52\x79\xf0\x1b\x78\x5e\xd1\x46

4. Recv magic in TCP socket

TCP socket

Infected Nginx Server

3. Send the hardcoded magic
back in TCP socket without
any encapsulation
42

DEMO

AGENDA
01

Overview of the Attack

02

Malware Analysis

03

Other Nginx-based Backdoor

04

Detection

05

Conclusions

Attack Vector

Existing Nginx-based backdoor◆ Recompile Nginx main program◆ Compiling Nginx module◆ NginxStealth and NginxSpy(Injection-based)◆ Inject into Nginx process◆

NGINX
Executable

45

Recompile Nginx main program
Compile & Installsudo ./configure --add-module=<module
path>

NGINX
Sourcesudo makesudo cp -f objs/nginx/path/to/nginx/sbin/nginx

NGINX
Build System

NGINX
Executable

ModuleSource46

Compiling Nginx module
Compile & Installsudo ./configure --add-dynamicmodule=<module path>sudo make modules
sudo cp
objs/ngx_http_check_headers_module.so<nginx_module_path>
load_module<path_to_your_module>/ngx_http_check_headers_module.so;# Modify nginx.conf

NGINX
SourceNGINX
Executable
NGINX
Build System

ModuleSourceDynamic
ModuleNginx Version: 1.9.11

47

Other Nginx-based Backdoor
Name

Recompile Nginx
programCompile Nginx
moduleInjection

Abuse Point

pwnginxO

X

X

ngx_http_top_header_filter

NginxExecute

O

O

X

ngx_http_core_loc_conf_t -> handlerXngx_http_core_main_conf_t-> phases[NGX_HTTP_ACCESS_PHASE]-> handlers

vgo0/nginxbackdoor

NginxSpy /NginxStealth

O

X

O

X

O

1. ngx_http_core_main_conf_t ->phase_engine -> handlers>hanlder
2. ngx_http_core_main_conf_t-> phases[NGX_HTTP_LOG_PHASE]-> handlers -> elts
48

Challenge
Injection-based Nginx backdoor◆ Without modifying nginx.conf◆ Without recompiling nginx program◆ No malware on disk
◆ There is no any log in access.log◆ For a webshell, there is no webpage in web folder◆ Only need the right user-agent◆

49

AGENDA
01

Overview of the Attack

02

Malware Analysis

03

Other Nginx-based Backdoor

04

Detection

05

Conclusions

Detection opportunity◆◆

Recompile Nginx main program◆ Check the modify time of Nginx◆ Check whether symbol table existed
Compiling Nginx module◆ Check the loaded dynamic module

51

Current Detection on
NginxStealth & NginxSpy
Check the modify time of
Nginx : failed◆ Check whether symbol table
existed : failed◆

Suspicious!!!

Check the loaded dynamic
module : failed◆ Check Nginx’s memory maps◆

52

Using all we know...
Check Nginx’s memory maps
◆ Abuse Point
1. ngx_http_core_main_conf_t-> phase_engine -> handlers-> handler
2. ngx_http_core_main_conf_t->
phases[NGX_HTTP_LOG_PH
ASE] -> handlers -> elts◆

Suspicious!!!

53

NginxCheck◆◆◆We developed a Nginx module, which is modified fromhttps://github.com/vgo0/nginx-backdoor
This module list the address :◆ checker and handler in
ngx_http_core_main_conf_t -> phase_engine -> handlers◆ elts in
ngx_http_core_main_conf_t-> phases[NGX_HTTP_LOG_PHASE] -> handlers
You can download from : https://github.com/p25072004/NginxCheck

54

Compile & Installation◆

Tested on Nginx version: Nginx/1.18.0
CompileInstallsudo ./configure --add-dynamicmodule=<module path># for self-compiled version

sudo cp objs/ngx_http_check_headers_module.so<nginx_module_path>sudo ./configure --add-dynamicmodule=<module path> --with-compat# for compiled version
sudo make modules# Compile module

load_module<path_to_your_module>/ngx_http_check_headers_module.so;# Modify nginx.confsudo nginx -s reload# Reload the configuration file without
disconnecting current established connections
55

How to use
Usagecurl -H "check: on" <c2 address>
Original

HookedNginx Maps

Suspicious!!!

56

GDB
◆

checker and handler in
ngx_http_core_main_conf_t -> phase_engine -> handlers

set $main = (ngx_http_core_main_conf_t *)*(*ngx_cycle->conf_ctx[ngx_http_module.index] +ngx_http_core_module->ctx_index)
set $s = 0
set $start = ($main->phase_engine->handlers)
while(*$start->checker!=0)
p *(ngx_http_phase_handler_t *)$start
set $s=$s+1
set $start = ($main->phase_engine->handlers+$s*1)
end

57

GDB
◆

elts inngx_http_core_main_conf_t -> phases[NGX_HTTP_LOG_PHASE] -> handlers

set $main = (ngx_http_core_main_conf_t *)*(*ngx_cycle->conf_ctx[ngx_http_module.index] +ngx_http_core_module->ctx_index)
set $log_handler = $main->phases[NGX_HTTP_LOG_PHASE].handlers.elts
set $n = $main->phases[NGX_HTTP_LOG_PHASE].handlers.nelts
set $i = 0
while($i<$n)
p *(ngx_http_handler_pt *)$log_handler
set $i = $i + 1
set $log_handler = $main->phases[NGX_HTTP_LOG_PHASE].handlers.elts + $i*8
end

58

DEMO

AGENDA
01

Overview of the Attack

02

Malware Analysis

03

Other Nginx-based Backdoor

04

Detection

05

Conclusions

Conclusions
New attack vector for Nginx-based backdoor: Injection◆ We have discovered two notable functions in this backdoor◆ Filter specific connection◆ Evading logging mechanism◆ We provide detection for this kind of backdoor◆ Target Areas : Taiwan
◆

61

IoC
◆

◆

◆

Dropper (e170bf30-hot.bak)◆ 94b6eec17f9aa71781a4d3aec91d7982
NginxStealth◆ 30084d79e5f027c0e070ffbcb4f2bf2e
NginxSpy◆ 06cafab4820fe2cccd8623178f2bf712

62

Reference◆

https://www.it145.com/9/184072.html◆

http://nginx.org/en/docs/dev/development_guide.html◆

http://static.kancloud.cn/kancloud/master-nginx-develop/51838◆

https://www.cnblogs.com/yjf512/archive/2012/04/01/2428385.html◆

https://friday.plus/archives/nginx%E4%B9%8Bhttp%E6%A8%A1%E5%9D%97◆

https://friday.plus/archives/nginx%E5%9F%BA%E7%A1%80%E6%9E%B6%E6%9E%84◆

https://openresty.org/download/agentzh-nginx-tutorials-en.pdf◆

https://www.nginx.com/wp-content/uploads/2018/11/NGINX-Conf-2018-slides_Soshnikovmodules.pdf#fromHistory◆

https://www.nginx.com/blog/dynamic-modules-development/◆

https://developer.aliyun.com/article/761260◆

https://blog.csdn.net/linux_vae/article/details/70610755◆

https://blog.csdn.net/ddazz0621/article/details/121221975

63

THANK YOU!
Any Questions?