influxdb/influxdb — pull request #488 by Dieterbe (2014-04-30 22:45:14+00:00)
Title: Graphite fixes2

this is the continuation of #444 and is not ready for merging.
it's rebased on the current master, and I also added a new patch to have no unneeded series objects with single points, but keep points together in series objects.

(bench) results:

new before (master):
```
[dieter.plaetinck@dfvimeographite1 tomselleck]$ head -n 1000 generate_proto1_metrics.txt | ./tomselleck -addr :2003 -workers 10 -requests 100 -maxprocs 2
2014/04/30 16:52:20 {:2003 10 100 false false} 1000

Tom Selleck Reports:

Address to hit: :2003 (:2003)
* 10 workers
* 100 reqs per worker

Connects: 1000
ConnectsErrors: 0
BytesWritten: 43909000
BytesRead: 0
BytesReadErrors: 0
EmptyResponses: 1000
UnexpectedLengthResponses: 0

Breakdown by percentile:
Pctl Best Wrst
50% 54.532us 414.797us
60% 415.117us 478.386us
70% 479.768us 536.902us
80% 537.814us 610.429us
90% 610.577us 661.686us
95% 662.758us 686.713us
96% 688.04us 704.953us
97% 707.746us 736.22us
98% 764.471us 889us
99% 955.593us 2.752421ms
All 54.532us 2.752421ms

Total time: 39.342382ms
Reqs per sec: 25417.881
Avg req time: 371.619us
Median req time: 370.65us
```
=> a bit faster indeed than the previous 'before' (master)
and the only errors in the log are `Got to the end of the write buffer without getting to the last written request.` which is fine (see #444)

the batched writes performance is still really bad, comparable to my previous patchset.
(this is just the rebase, without the extra commit that groups points in series objects)
```
[dieter.plaetinck@dfvimeographite1 tomselleck]$ head -n 1000 generate_proto1_metrics.txt | ./tomselleck -addr :2003 -workers 10 -requests 100 -maxprocs 2
2014/04/30 16:55:49 {:2003 10 100 false false} 1000

Tom Selleck Reports:

Address to hit: :2003 (:2003)
* 10 workers
* 100 reqs per worker

Connects: 1000
ConnectsErrors: 0
BytesWritten: 43909000
BytesRead: 0
BytesReadErrors: 0
EmptyResponses: 1000
UnexpectedLengthResponses: 0

Breakdown by percentile:
Pctl Best Wrst
50% 52.124us 427.819us
60% 427.84us 488.815us
70% 490.456us 552.538us
80% 553.025us 668.197us
90% 668.89us2.999761678s
95%2.999770136s2.999971186s
96%2.999992863s3.000133372s
97%3.000142783s3.000283211s
98%3.000284337s 3.00039835s
99% 3.00041448s3.000847178s
All 52.124us3.000847178s

Total time: 18.036579762s
Reqs per sec: 55.443
Avg req time: 165.355223ms
Median req time: 391.971us
```
the last commit is not finished yet, i got a panic and couldn't figure it out yet.

```
[04/30/14 18:23:06] [INFO] Starting Http Api server on port 8086
[2014/04/30 18:23:06 EDT] [INFO] (server.(*Server).ListenAndServe:131) Starting Http Api server on port 8086
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x8 pc=0x4994f1]

goroutine 23 [running]:
runtime.panic(0x9e67a0, 0x121abc8)
/usr/lib64/golang/src/pkg/runtime/panic.c:266 +0xb6
coordinator.(*CoordinatorImpl).CommitSeriesData(0xc2100b8700, 0xc2100aa550, 0x8, 0xc21163e000, 0x6f4, ...)
/home/users/dieter/influxdb/src/coordinator/coordinator.go:583 +0x161
coordinator.(*CoordinatorImpl).WriteSeriesData(0xc2100b8700, 0x7ff8f06dbad0, 0xc21006f500, 0xc2100aa550, 0x8, ...)
/home/users/dieter/influxdb/src/coordinator/coordinator.go:474 +0x20f
api/graphite.func·002(0xc2100b9720)
/home/users/dieter/influxdb/src/api/graphite/api.go:139 +0x30d
api/graphite.(*Server).committer(0xc210046720)
/home/users/dieter/influxdb/src/api/graphite/api.go:176 +0x40d
created by api/graphite.(*Server).ListenAndServe
/home/users/dieter/influxdb/src/api/graphite/api.go:92 +0x1fd
```