Use McReply instead of mc_msg_t (fixed)

Summary: D1538298 with fixed memory leak

Test Plan: unit tests

Reviewed By: brianp@fb.com

Subscribers: trunkagent, zhuohuang, alikhtarov

FB internal diff: D1550394

Tasks: 4889426
Remove dead test

Summary: title

Test Plan: build mcrouter

Reviewed By: alikhtarov@fb.com

Subscribers: alikhtarov

FB internal diff: D1559985
Log TKO samples

Summary: Framework for logging in interanl code base every time we mark/unmark box as TKO.

Test Plan: Unit tests

Reviewed By: sdmich@fb.com

Subscribers: sameen, ipozgaj, alikhtarov

FB internal diff: D1541598

Tasks: 5014297
Fix test_metaget_failover

Summary: fails for me every now and then

Test Plan: run mcrouter unit tests

Reviewed By: alikhtarov@fb.com

Subscribers: alikhtarov

FB internal diff: D1562615
Return zero exit code for --help

Summary:
there is no point in returning "Unrecoverable error" exit code when someone wants to check command line options.
This fixes issue #1 :-) https://github.com/facebook/mcrouter/issues/1

Test Plan: ./mcrouter --help; echo $?

Reviewed By: alikhtarov@fb.com

Subscribers: alikhtarov

FB internal diff: D1560464
Log standalone options and pid to startup_options

Summary: we miss those options from startup_options. Also I find it really useful to easily find process id of mcrouter instance.

Test Plan: mcrouter unit tests, run mcrouter and check startup_options

Reviewed By: alikhtarov@fb.com

Subscribers: alikhtarov

FB internal diff: D1561278

Tasks: 5147288
Fix clang build

Test Plan: build it

Reviewed By: pavlo@fb.com

Subscribers: mathieubaudet, alikhtarov

FB internal diff: D1564726

Tasks: 5192045
Make mc_res_connect_timeout a hard error

Summary: If we end up shortening mc_res_connect_timeout from the max request time mc_res_connect_timeout should be soft so that retransmits don't hard TKO a box.
hort of that, if we can't even establish a connection in the time allocated for a request, the server is likely dead.

Test Plan: unit tests

Reviewed By: pavlo@fb.com

Subscribers: trunkagent, sameen, ipozgaj, alikhtarov

FB internal diff: D1506290
Add name to McOperation

Summary: refactoring

Test Plan: run mcrouter unit tests

Reviewed By: andrii@fb.com

Subscribers: alikhtarov

FB internal diff: D1548911
Remove rtt & dynamic_stats stats

Summary: this diff entirely removes dynamic_stats & rtt stats (rtt, rtt_min, rtt_max). Instead of timers uses avgLatency from ProxyDestination. Also refactoring of ProxyDestination to make it more C++-like and reuse code.

Test Plan:
mcrouter unit tests; run mcrouter and send "stats servers" command. Result:

[pavlo@dev1763 ~/fbcode] echo "stats servers" | nc 0 5000 | grep found
STAT [2401:db00:0010:9071:face:0000:001d:0000]:11302:TCP:ascii-1-0 avg_latency_us:524.667 new:1 up:3 found:3
STAT [2401:db00:0010:903d:face:0000:0026:0000]:11302:TCP:ascii-1-0 avg_latency_us:488.766 up:4 notfound:5
STAT [2401:db00:0010:903f:face:0000:0019:0000]:11302:TCP:ascii-1-0 avg_latency_us:496.254 up:4 found:14 notfound:3

Benchmarks:

This diff:
Best time (no results > 0): 25ms
Worst time (all results > 0): 35ms
Average time (5 results > 0): 27ms

Previous modification of this diff:
Best time (no results > 0): 35ms
Worst time (all results > 0): 400ms
Average time (5 results > 0): 100ms

Without this diff:
Best time (no results > 0): 40ms
Worst time (all results > 0): 500ms
Average time (5 results > 0): 130ms

Reviewed By: alikhtarov@fb.com

Subscribers: trunkagent, alikhtarov

FB internal diff: D1562458
Better consts, no need to have any order

Summary: currently we use array for consts and can not use any macro there. It is not as flexible as allow everything: random order, simpler syntax and free mix of consts/macros.

Test Plan: mcrouter unit tests

Reviewed By: alikhtarov@fb.com

Subscribers: alikhtarov

FB internal diff: D1553292
Add shard_splits

Summary:
this diff introduces ShardSplitRoute which:
on get sends request to random shard split, based on key hash.
on other requests uses AllInitial logic to send requests to all shard splits

to send request to different destination we append "aa", "ba", "ca", ..., "zz" to shard id

when "shard_splits" field is broken (invalid values, not an object, etc.) we simply ignore it and write error to log.

Test Plan: mcrouter unit tests; added unit tests for this specific feature

Reviewed By: alikhtarov@fb.com

Subscribers: vikas, alikhtarov

FB internal diff: D1552979

Tasks: 4876664
Minor refactoring

Summary: fixed couple of odd things I encountered in our code base: unused FiberManager call, test with router-less mode, useless pointer.

Test Plan: mcrouter unit tests

Reviewed By: alikhtarov@fb.com

Subscribers: alikhtarov

FB internal diff: D1560778
Zero-copy for umbrella

Summary:
title
This alone is not a zero-copy yet, will need to rebase on top of all other zero-copy diffs before we will be able to test performance of this.
But it should be ok for review, since it doesn't conflict with other parts.

Test Plan: tests

Reviewed By: brianp@fb.com

Subscribers: trunkagent, alikhtarov

FB internal diff: D1538327

Tasks: 4889426
Fix opensource build

Summary: title

Test Plan: tests

Reviewed By: alikhtarov@fb.com

Subscribers: alikhtarov

FB internal diff: D1569967
Remove shard_map support

Summary: It is deprecated and not used anymore. We have better options like hash weights, etc.

Test Plan: run mcrouter unit tests, build everything

Reviewed By: bwatling@fb.com

Subscribers: trunkagent, vikas, alikhtarov

FB internal diff: D1569491

Tasks: 5182268
Use folly macro to detect address sanitizer

Summary: This works for both gcc and clang

Test Plan:
build and ran
Ran and examined __mcrouter__.options(fibers_stack_size)

Reviewed By: meyering@fb.com

Subscribers: trunkagent, alikhtarov

FB internal diff: D1570079

Tasks: 5192045
Make fibers tests more robust

Summary: This version works with ASAN.

Test Plan: ran test with ASAN enabled and disabled

Reviewed By: meyering@fb.com

Subscribers: alikhtarov

FB internal diff: D1570107

Tasks: 5192045
Misc leak fixes detected by LSAN

Summary:
Note: `countedfd_t` should really be replaced with `std::shared_ptr<folly::File>`, but
it's a larger refactoring (in particular, writelog_entry_t should be refactored not to allocate
buffer storage with malloc at the end of the struct).

Test Plan: ran test with ASAN and without

Reviewed By: pavlo@fb.com

Subscribers: trunkagent, alikhtarov

FB internal diff: D1570151

Tasks: 5192045
Fix fbi tests memory leaks

Summary: LSAN detected

Test Plan: unit tests

Reviewed By: pavlo@fb.com

Subscribers: alikhtarov

FB internal diff: D1571237

Tasks: 5192045