organize test datasets better
addDest (WIP)
Merge pull request #37 from pwielgolaski/master

changes to enable REST interface
better testEndpoint wait/sync stuff, cleaner test code

we can now cleanly wait (and err on timeout) until
data seen or num accepts
also clean up code a bit, DRY etc
add a benchmark
Merge branch 'master' of github.com:rcrowley/carbon-relay-ng
make it better
bugfix: avoid busy loop
make tests allocate much less memory

it was getting in the way of doing proper analysis
new attempt/approach at verifying all received records with less allocations

as you can see, i couldn't make it work (esp at step 3 it gets tricky)
so disabled for now.  anyway with the previous approach all these checks
were fine, so it'll still be fine, and we check the num received anyway.
but we should try to reintroduce these checks somehow
this implementation should do less/smaller memory allocations

actually it's not the case, so i'm reverting it straight away.
but i feel like it's good to have this in the git history for reference

tested by sending and receiving 100k messages

BEFORE

    5002398987 ns/op
ok  github.com/graphite-ng/carbon-relay-ng  5.011s

AFTER

after:
    15001749641 ns/op
ok  github.com/graphite-ng/carbon-relay-ng 5.015s

BEFORE MEM
(pprof) top40
Total: 11.8 MB
     4.6  38.7%  38.7%      4.6  38.7% github.com/graphite-ng/carbon-relay-ng.NewKeepSafe
     4.5  38.0%  76.6%      4.5  38.0% github.com/graphite-ng/carbon-relay-ng.(*TestEndpoint).handle
     2.3  19.1%  95.8%      2.3  19.1% github.com/graphite-ng/carbon-relay-ng.func·009
     0.5   4.2% 100.0%      0.5   4.2% allocg
     0.0   0.0% 100.0%      0.5   4.2% _rt0_go
     0.0   0.0% 100.0%      4.6  38.7% github.com/graphite-ng/carbon-relay-ng.(*Destination).updateConn
     0.0   0.0% 100.0%      4.6  38.7% github.com/graphite-ng/carbon-relay-ng.NewConn
     0.0   0.0% 100.0%      0.5   4.2% mcommoninit
     0.0   0.0% 100.0%     11.3  95.8% runtime.gosched0
     0.0   0.0% 100.0%      0.5   4.2% runtime.malg
     0.0   0.0% 100.0%      0.5   4.2% runtime.mpreinit
     0.0   0.0% 100.0%      0.5   4.2% runtime.schedinit

AFTER MEM
(pprof) top100
Total: 18.3 MB
     6.0  32.8%  32.8%      6.0  32.8% container/list.(*List).insertValue
     5.0  27.4%  60.2%      5.0  27.4% github.com/graphite-ng/carbon-relay-ng.(*TestEndpoint).handle
     3.5  19.2%  79.4%      3.5  19.2% runtime.convT2E
     2.3  12.4%  91.8%      2.3  12.4% github.com/graphite-ng/carbon-relay-ng.func·010
     0.5   2.7%  94.5%      0.5   2.7% allocg
     0.5   2.7%  97.3%      0.5   2.7% hash/crc32.makeTable
     0.5   2.7% 100.0%      0.5   2.7% time.loadZoneData

BEFORE CPU:
(pprof) top100
Total: 259 samples
      50  19.3%  19.3%       50  19.3% runtime.futex
      14   5.4%  24.7%       14   5.4% runtime.xchg
      12   4.6%  29.3%       30  11.6% runtime.mallocgc
      10   3.9%  33.2%       31  12.0% selectgo
       9   3.5%  36.7%        9   3.5% runtime.usleep
       7   2.7%  39.4%        7   2.7% scanblock
       5   1.9%  41.3%        5   1.9% runtime.memclr
       5   1.9%  43.2%        5   1.9% stopm
       4   1.5%  44.8%       12   4.6% cnew
       4   1.5%  46.3%        7   2.7% fmt.(*pp).printArg
       4   1.5%  47.9%        4   1.5% runtime.MSpan_Sweep
       4   1.5%  49.4%        4   1.5% runtime.epollwait
       4   1.5%  51.0%        9   3.5% runtime.lock

AFTER CPU:

      55  19.6%  19.6%       55  19.6% runtime.futex
      13   4.6%  24.2%       13   4.6% flushptrbuf
      13   4.6%  28.8%       31  11.0% runtime.mallocgc
      12   4.3%  33.1%       30  10.7% selectgo
       9   3.2%  36.3%        9   3.2% scanblock
       8   2.8%  39.1%        8   2.8% runtime.MSpan_Sweep
       8   2.8%  42.0%        8   2.8% runtime.usleep
       7   2.5%  44.5%        7   2.5% runtime.xchg
       6   2.1%  46.6%        6   2.1% ExternalCode
       6   2.1%  48.8%        6   2.1% schedule
       5   1.8%  50.5%       29  10.3% github.com/graphite-ng/carbon-relay-ng.(*Conn).HandleData
       5   1.8%  52.3%       40  14.2% github.com/graphite-ng/carbon-relay-ng.(*Destination).relay
Revert "this implementation should do less/smaller memory allocations"

This reverts commit ce58c37b14bb11af20ad20c4816c2e61b5b876e1.
easily benchmark various amounts. see #38
make sure we can see the reconnect in step 3
configurable loglevel
fix data race

we were passing on slices pointing to the array that we were overwriting

note: in the future, instead of allocating all these small bufs, maybe
we should use a large byte array containing all the packets, or
something. we could compute the index if they are equal length
more useful output
better logging
make it so that nothing in the pipeline modifies the buf array

so that in tests we can use the same datastructure as source input
and to compare the output against, though we should add a test
that the code doesn't modify it.
also this can save us some precious reallocations and seems to
make the code a bit faster
finally an approach that's light on test memory allocations

and still lets us keep packets to verify what we received
fix "SeenThisOrFatal" checks