ngx_mruby vs lua-nginx-module memo


## Configuration, Environment
### Machine

- Ubuntu14.04 on VMWare 
- Intel(R) Core(TM) i7-4770K CPU @ 3.50GHz 4core
- Memory 8GB

### Software version

- [ngx_mruby GitHub latest master](https://github.com/matsumoto-r/ngx_mruby/commit/df050067427fc8448a3c8589403fdfe336b82136) 
  - df050067427fc8448a3c8589403fdfe336b82136
- [lua-nginx-module GitHub latest master](https://github.com/openresty/lua-nginx-module/commit/bc120f9a2279a384e4335bedf765e844b4d22adb) 
  - bc120f9a2279a384e4335bedf765e844b4d22adb
- nginx 1.7.6
- Didn't tune up kernel parameters

#### OS

```
$ uname -a
Linux ubuntu14-04-64 3.13.0-24-generic #47-Ubuntu SMP Fri May 2 23:30:00 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
```

### nginx binary
```bash
$ ldd build/nginx/sbin/nginx
        linux-vdso.so.1 =>  (0x00007fff28946000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007feca7446000)
        libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007feca720d000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007feca6f06000)
        libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007feca6b2c000)
        libhiredis.so.0.10 => /usr/lib/x86_64-linux-gnu/libhiredis.so.0.10 (0x00007feca6922000)
        libluajit-5.1.so.2 => /usr/lib/x86_64-linux-gnu/libluajit-5.1.so.2 (0x00007feca66b2000)
        libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007feca6474000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007feca625b000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007feca5e94000)
        /lib64/ld-linux-x86-64.so.2 (0x00007feca7677000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007feca5c90000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007feca5a7a000)
```

### nginx build options
```bash
$ ./build/nginx/sbin/nginx -V
nginx version: nginx/1.7.6
built by gcc 4.8.2 (Ubuntu 4.8.2-19ubuntu1) 
configure arguments: --add-module=/home/matsumotory/DEV/ngx_mruby --add-module=/home/matsumotory/DEV/ngx_mruby/dependence/ngx_devel_kit --prefix=/home/matsumotory/DEV/ngx_mruby/build/nginx --add-module=/home/matsumotory/DEV/lua-nginx-module
```

### nginx.conf
```nginx:nginx.conf
worker_processes auto;

events {
    worker_connections 1024;
    multi_accept on;
}

daemon off;

http {
    include mime.types;
    server {
        listen 58080;
        server_name  localhost;
        root /home/matsumotory/DEV/ngx_mruby/build/nginx/html/;

        location /mruby {
            mruby_content_handler_code "Nginx.echo 'hello ngx_mruby world.'";
        }

        location /lua {
            content_by_lua "ngx.say('Hello ngx_lualu world.')";
        }

        location /static {
            # create static/inex.html
        }
    }
}

```

### Benchmark Commands
```bash
$ ab -c 100 -n 1000000 -k http://127.0.0.1:58080/mruby
$ ab -c 100 -n 1000000 -k http://127.0.0.1:58080/lua
$ ab -c 100 -n 1000000 -k http://127.0.0.1:58080/static/index.html
```

## Benchmark Results
### ngx_mruby

```
This is ApacheBench, Version 2.3 <$Revision: 1528965 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 100000 requests
Completed 200000 requests
Completed 300000 requests
Completed 400000 requests
Completed 500000 requests
Completed 600000 requests
Completed 700000 requests
Completed 800000 requests
Completed 900000 requests
Completed 1000000 requests
Finished 1000000 requests


Server Software:        nginx/1.7.6
Server Hostname:        127.0.0.1
Server Port:            58080

Document Path:          /mruby
Document Length:        23 bytes

Concurrency Level:      100
Time taken for tests:   11.708 seconds
Complete requests:      1000000
Failed requests:        0
Keep-Alive requests:    990044
Total transferred:      143950220 bytes
HTML transferred:       23000000 bytes
Requests per second:    85414.56 [#/sec] (mean)
Time per request:       1.171 [ms] (mean)
Time per request:       0.012 [ms] (mean, across all concurrent requests)
Transfer rate:          12007.27 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.2      0      28
Processing:     0    1   1.5      1     111
Waiting:        0    1   1.5      1     111
Total:          0    1   1.6      1     114

Percentage of the requests served within a certain time (ms)
  50%      1
  66%      1
  75%      1
  80%      2
  90%      2
  95%      2
  98%      4
  99%      5
 100%    114 (longest request)
```

### lua-nginx-module
```
This is ApacheBench, Version 2.3 <$Revision: 1528965 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 100000 requests
Completed 200000 requests
Completed 300000 requests
Completed 400000 requests
Completed 500000 requests
Completed 600000 requests
Completed 700000 requests
Completed 800000 requests
Completed 900000 requests
Completed 1000000 requests
Finished 1000000 requests


Server Software:        nginx/1.7.6
Server Hostname:        127.0.0.1
Server Port:            58080

Document Path:          /lua
Document Length:        23 bytes

Concurrency Level:      100
Time taken for tests:   15.227 seconds
Complete requests:      1000000
Failed requests:        0
Keep-Alive requests:    990053
Total transferred:      169950265 bytes
HTML transferred:       23000000 bytes
Requests per second:    65674.42 [#/sec] (mean)
Time per request:       1.523 [ms] (mean)
Time per request:       0.015 [ms] (mean, across all concurrent requests)
Transfer rate:          10899.79 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.3      0      16
Processing:     0    2   2.5      1     143
Waiting:        0    1   2.5      1     143
Total:          0    2   2.5      1     143

Percentage of the requests served within a certain time (ms)
  50%      1
  66%      2
  75%      2
  80%      2
  90%      3
  95%      3
  98%      5
  99%      8
 100%    143 (longest request)

```

### Static Content
```
This is ApacheBench, Version 2.3 <$Revision: 1528965 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 100000 requests
Completed 200000 requests
Completed 300000 requests
Completed 400000 requests
Completed 500000 requests
Completed 600000 requests
Completed 700000 requests
Completed 800000 requests
Completed 900000 requests
Completed 1000000 requests
Finished 1000000 requests


Server Software:        nginx/1.7.6
Server Hostname:        127.0.0.1
Server Port:            58080

Document Path:          /static/hello.html
Document Length:        22 bytes

Concurrency Level:      100
Time taken for tests:   16.687 seconds
Complete requests:      1000000
Failed requests:        0
Keep-Alive requests:    990049
Total transferred:      256950245 bytes
HTML transferred:       22000000 bytes
Requests per second:    59927.44 [#/sec] (mean)
Time per request:       1.669 [ms] (mean)
Time per request:       0.017 [ms] (mean, across all concurrent requests)
Transfer rate:          15037.47 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.3      0      58
Processing:     0    2   3.8      1     449
Waiting:        0    2   3.8      1     449
Total:          0    2   3.9      1     449

Percentage of the requests served within a certain time (ms)
  50%      1
  66%      2
  75%      2
  80%      2
  90%      3
  95%      4
  98%      8
  99%     12
 100%    449 (longest request)

```

one mistake; response data byte is different from others. If you write the following into nginx.conf, more faster, maybe.

```
accept_mutex_delay 100ms;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_requests 100000;
```

## Thoughts

name|result
----|-----
ngx_mruby|Requests per second:    85414.56 [#/sec] (mean)
lua-nginx-module|Requests per second:    65674.42 [#/sec] (mean)
static contents|Requests per second:    59927.44 [#/sec] (mean)

It's a just hello world benchmark, so this benchmark is hobby. But, very interesting.