mschwartz/SilkJS — Issue #4: Benchmarks
Author: X4
Created: 2011-12-11 03:27:53+00:00
State: open

Hi Mr. Schwartz

I've read your announcement by luck http://www.sencha.com/forum/showthread.php?160128-Announcing-SilkJS
and found your description of your benchmark results curious. It would make sense to share your:

Machine Specs (+cores)
Kernel parameters (if any)
NIC Bandwidth
File size of test-file (100Byte, 1KB, 512KB 1MB)

I can recommend weighttp. ab is single-threaded and utilizes only one core/cpu.
Your server doesn't scale linearly, thus varying req/s dependent on req# and concurrence level is normal.
Enabling keep-alive also further improves results.

I get about 4.8k to 5k req/s on a 1.3 DualCore :) I know it's weak, but hey I wanted to share my results you.
$: ab -t 30 -c 50 -k http://localhost:9090/anchor.png
...
Server Software: SILK
Server Hostname: localhost
Server Port: 9090

Document Path: /anchor.png
Document Length: 523 bytes

Concurrency Level: 50
Time taken for tests: 10.402 seconds
Complete requests: 50000
Failed requests: 0
Write errors: 0
Keep-Alive requests: 50000
Total transferred: 37700000 bytes
HTML transferred: 26150000 bytes
Requests per second: 4806.58 [#/sec] (mean)
Time per request: 10.402 [ms] (mean)
Time per request: 0.208 [ms] (mean, across all concurrent requests)
Transfer rate: 3539.22 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 50.3 0 3005
Processing: 0 10 9.5 8 176
Waiting: 0 10 9.5 8 176
Total: 0 10 52.0 8 3094

Percentage of the requests served within a certain time (ms)
50% 8
66% 12
75% 15
80% 17
90% 21
95% 25
98% 30
99% 32
100% 3094 (longest request)

$: weighttp -n 100000 -c 100 -t 2 -k "http://localhost:9090/anchor.png"
...
finished in 19 sec, 787 millisec and 667 microsec, 5053 req/s, 3721 kbyte/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored
traffic: 75400000 bytes total, 23100000 bytes http, 52300000 bytes data

I think using 250 workers is naive, because the time lost for context-switches is enormous, it's better to map threads to cpu's.
On a 6Core XEON processor for example you can use up to 10 pthreads after that you won't notice an improvement, but a slow decrease in performance.

Cheers :)