Thrift security fix

Summary:
When a message is sent during security negotiation, it is
stored into a queue and replayed back after negotiation is complete.
However, since negotiation happens always with compact protocol, the
messages are serialized as compact. If the user-selected protocol is
binary, then the compact message is replayed back with a header saying
it's binary.

This diff sets the protocol to compact just before a handshake message
is sent and restores it to the original protocol just after that.

(hphp_dbg_clang_contbuild_push_blocking is taking almost 3 hours)

Test Plan: Added test

Reviewed By: mshneer@fb.com

Subscribers: trunkagent, alandau, bmatheny, mshneer

FB internal diff: D1689893

Tasks: 5382924

Signature: t1:1689893:1416363669:7526e4f454a2b15db0555690b99c20f02d5398ea
Change exit(-1) to exit(EXIT_FAILURE) in thrift/

Summary: Changing exit(-1) to exit(EXIT_FAILURE) across fbcode

Test Plan: Compiled the project

Reviewed By: davejwatson@fb.com

Subscribers: alandau, bmatheny, mshneer

FB internal diff: D1694566

Tasks: 4073156

Signature: t1:1694566:1416512540:f818f27f2c7f43669162728b7e99ffe3ec005355
cpp2 SimpleJSONProtocol: better exceptions

Summary: always throw TProtocolExceptions instead of whatever folly throws

Test Plan: g-unittest

Reviewed By: haijunz@fb.com

Subscribers: trunkagent, alandau, bmatheny, mshneer

FB internal diff: D1689561

Signature: t1:1689561:1416423577:bcacac104360100743c822ff28de639a046914fa
AsyncTransportHandler

Summary: pulled this out of D1618704. It's one implementation of the handler that sits at the bottom of a channel pipeline and actually does the talking to the transport

Test Plan: will try to test in isolation. using in thrift diff

Reviewed By: davejwatson@fb.com

Subscribers: njormrod, folly-diffs@, trunkagent, fugalh, alandau, bmatheny, mshneer

FB internal diff: D1628202

Signature: t1:1628202:1416529657:8211a62225d68a7958214c483e06636c039c2831
AsyncTimeoutSet lives in proxygen now

Summary:
This is deprecated in favor of HHWheelTimer now. In the meantime,
take the code into proxygen since we are the last major user.

Test Plan: unit tests

Reviewed By: davejwatson@fb.com

Subscribers: hphp-diffs@, ps, kparton, jdperlow, fugalh, cgheorghe, trunkagent, doug, fbcode-common-diffs@, davejwatson, andrewcox, atlas2-eng@, alandau, bmatheny, alikhtarov, mshneer

FB internal diff: D1677319

Tasks: 5485692

Signature: t1:1677319:1415833221:a4e97ecf1ca20537d8d5248a03d692929263af09
CompactProtocol: serialization perf - use inline stack

Summary:
Use inline stack if possible instead of allocating on the heap.
Similar to D1672045 but for writes.

Test Plan:
```
------ old ------   ------ new ------
Benchmark                                time/iter  iters/s  time/iter  iters/s        win
==========================================================================================
CompactProtocol_write_Empty               123.10ns    8.12M    92.19ns   10.85M     33.62%
CompactProtocol_write_SmallInt            140.60ns    7.11M   109.05ns    9.17M     28.97%
CompactProtocol_write_BigInt              172.44ns    5.80M   144.68ns    6.91M     19.14%
CompactProtocol_write_SmallString         154.23ns    6.48M   125.45ns    7.97M     22.99%
CompactProtocol_write_BigString           405.69ns    2.46M   368.60ns    2.71M     10.16%
CompactProtocol_write_BigBinary           303.58ns    3.29M   276.57ns    3.62M     10.03%
CompactProtocol_write_LargeBinary         312.93ns    3.20M   290.44ns    3.44M      7.50%
CompactProtocol_write_Mixed               206.38ns    4.85M   176.04ns    5.68M     17.11%
CompactProtocol_write_SmallListInt        235.18ns    4.25M   208.11ns    4.81M     13.18%
CompactProtocol_write_BigListInt           99.76us   10.02K    99.08us   10.09K      0.70%
CompactProtocol_write_BigListMixed          1.03ms   974.08   997.80us    1.00K      2.66%
CompactProtocol_write_LargeListMixed      113.49ms     8.81   111.85ms     8.94      1.48%
```

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, alandau, bmatheny, mshneer

FB internal diff: D1698243

Signature: t1:1698243:1416614287:d3ed533057a85aff121989857e7f0ec61ebb5e8f
Kill the SASL retries logic in ServiceRouter

Summary:
SASL retries, because it was piggy-backing on ServiceRouter retries
mechanism, was subject to retries being throttled in case of high error rate.

This caused the SASL Handshake Timeouts to surface as user facing errors as
fallback to insecure was suppressed in the first attempt.

See https://our.intern.facebook.com/intern/tasks/?t=5611513 for more details.

The plan forward is to use fb303 health checks in SR (since they try to
establish a secure connection), and prioritize those connections in the pool
which happened over secure thrift.

Test Plan:
servicerouter and thrift unit tests.
Ran the thrift loadgen.

Reviewed By: hitesh@fb.com

Subscribers: trunkagent, mcduff, hitesh, alandau, bmatheny, mshneer, andreib, shamdor

FB internal diff: D1695845

Tasks: 5611513

Signature: t1:1695845:1416854394:0a979368faf30af9073420f9faf7b7c47f75dbe1
Add portability implementations of pthread timed locks

Summary:
MacOSX pthreads doesn't have these methods.
Provide a slightly awkward, but functional implementation.

Test Plan: Part of getting HHVM to build on MacOSX

Reviewed By: paulbiss@fb.com

Subscribers: trunkagent, alandau, bmatheny, mshneer

FB internal diff: D1686641

Signature: t1:1686641:1416333720:213f581032fc78ccdf7a2c1a1d914a41b2315792
Make py.asyncio a thing

Summary: This pushes @haijunz's asyncio forward to introduce the py.asyncio namespace. There's no new tests yet since there's not a Client implementation yet.

Test Plan: fbconfig -r thrift/test/py && fbmake runtests passes; there's one unrelated failure with the cpp_server that I'm going to fix independently.

Reviewed By: haijunz@fb.com

Subscribers: trunkagent, simpkins, ruibalp, agallagher, alandau, bmatheny, mshneer, haijunz, moshez

FB internal diff: D1698740

Tasks: 5674845

Signature: t1:1698740:1416848529:3aa426b5cfadbccfbc63d94ee2a9dc2bbd6d29f5
fix misleading error "SASL required by client but server closed connection"

Summary:
This can be logged to scuba any time the connection closes
during the sasl handshake, so say that.

Test Plan:
Hack the code to LOG() what is sent to scuba, and run
_build/dev/experimental/andreib/clients/test_client --port=3572 --ip=2401:db00:10:905e:face:0:2e:0 --thrift_security=permitted --thrift_security_service_tier=host
_build/dev/experimental/andreib/clients/test_client --port=3572 --ip=2401:db00:10:905e:face:0:2e:0 --thrift_security=required --thrift_security_service_tier=host

Reviewed By: sandeepkk@fb.com

Subscribers: trunkagent, alandau, bmatheny, mshneer

FB internal diff: D1696337

Tasks: 5658350

Signature: t1:1696337:1416535836:22c4ee7bdaa32a9bdefbde4c3fb606d53ab4ed9e
Add libnuma-dev

Summary: As title

Test Plan: oss build

Reviewed By: alandau@fb.com

Subscribers: alandau, mshneer

FB internal diff: D1700952

Tasks: 5678991

Signature: t1:1700952:1416861916:00642d97cdbe2eaf57d68443e00d401d649a085f
Add "usetrait" option to Thrift PHP generator

Summary: Currently it's not possible to add methods to a Thrift struct class in PHP, necessitating utility classes, etc.  It would be much better design if we could just have the Thrift struct class optionally use a trait (which will then be manually added and edited as necessary to provie the desired functions).  That's what this diff does.

Test Plan: Ran compiler against a thrift file with the option enabled.  Generated code looked good.

Reviewed By: ckwalsh@fb.com

Subscribers: trunkagent, alandau, mshneer

FB internal diff: D1687451

Signature: t1:1687451:1416939918:a66c889bc7e0fb8257420e43449858c7ce18d051
CompactWriter: varint improvements

Summary:
- make i32ToZigzag and 64 inlinable
- unroll writeVarint
- separate writeVarint fast and slow path

Test Plan:
```
------ old ------   ------ new ------
Benchmark                                time/iter  iters/s  time/iter  iters/s        win
==========================================================================================
CompactProtocol_write_Empty                92.25ns   10.84M    90.90ns   11.00M      1.48%
CompactProtocol_write_SmallInt            108.94ns    9.18M   108.22ns    9.24M      0.65%
CompactProtocol_write_BigInt              143.65ns    6.96M   118.00ns    8.47M     21.70%
CompactProtocol_write_SmallString         124.89ns    8.01M   122.40ns    8.17M      2.00%
CompactProtocol_write_BigString           362.07ns    2.76M   361.81ns    2.76M      0.00%
CompactProtocol_write_BigBinary           275.05ns    3.64M   272.73ns    3.67M      0.82%
CompactProtocol_write_LargeBinary         289.10ns    3.46M   274.89ns    3.64M      5.20%
CompactProtocol_write_Mixed               175.97ns    5.68M   166.67ns    6.00M      5.63%
CompactProtocol_write_SmallListInt        206.88ns    4.83M   185.48ns    5.39M     11.59%
CompactProtocol_write_BigListInt           98.42us   10.16K    77.42us   12.92K     27.17%
CompactProtocol_write_BigListMixed        994.65us    1.01K   869.78us    1.15K     13.86%
CompactProtocol_write_LargeListMixed      111.89ms     8.94    99.14ms    10.09     12.86%
```

Reviewed By: davejwatson@fb.com

Subscribers: trunkagent, alandau, bmatheny, mshneer

FB internal diff: D1702704

Signature: t1:1702704:1417025039:a2c53f530f740b23ad6babcfcbb7c694dae5be1c
New, more Pythonic API for generating Thrift servers; unit tests for py.asyncio

Summary: While this brings us to a point where we can at least test regressions, the Client implementation that is generated for py.asyncio is still weak since both the send_ and recv_ methods are currently blocking (the API should be reimplemented as coroutines). This will come in a future change.

Test Plan: run tests in thrift/test/py.asyncio

Reviewed By: fried@fb.com

Subscribers: trunkagent, ruibalp, alandau, bmatheny, mshneer, moshez

FB internal diff: D1710698

Tasks: 5674845

Signature: t1:1710698:1417473516:7400c08d9ec5e57c971e0e39bb6ec217cede5013
Krb5CredentialsCacheManager understands DIR ccache

Summary:
If default ccache is 'DIR:PATH' and client principal is client_princ
ccmanager will try to read/write ccache from/to PATH/client_princ

If client_princ is not known on read, first principal from keytab will
be used.

Test Plan:
Manually tested(for both FILE and DIR) that:
1. Credentials are read from right ccache.
2. If client principal mismatch it will not be read.
3. If client principal mismatch and ccache is not about to expire it will not be
overwritten.
4. If client principal mismatch and ccache is about to expire it will be overwritten.
5. If client principal is not in keytab ccache will not be overwritten.

Reviewed By: mhorowitz@fb.com

Subscribers: trunkagent, alandau, bmatheny, mshneer, acampi, sandeepkk

FB internal diff: D1679764

Tasks: 5250753

Signature: t1:1679764:1416969124:2b959c0d97e67ea899e2c212f69238074dec4c39
Add ability to specify a timeout

Summary: I realized that Thrift2 has a 5 second default by default. We should expose `setTimeout` so that services written in JS can control this value. We rely on the smc tier setting for JSService to timeout on the clientside (the thrift client side = PHP) and we are ok with requests to JSService taking longer to complete (resources might take a while to be loaded, especially in sandboxes after a rebase etc.). This is also necessary for doing data fetching via GraphQL in JSService which may take more than 5 seconds overall (create queries + fetch data + render all html).

Test Plan: In JSService, set the timeout to `10` ms and see how every call to it expires. Set the timeout to 0 or a high number and see how the requests succeed.

Reviewed By: coops@fb.com

Subscribers: alandau, bmatheny, laney, cpojer, mshneer, glh, josh

FB internal diff: D1711883

Signature: t1:1711883:1417548373:8b7838302af03251685e1a18a6c7c0d9ac31a6d1
Code quality improvement in TAsyncioServer

Summary: Addressed suggestions in code review by @jackdied.

Test Plan: runtests in thrift/test/py.asyncio still pass.

Reviewed By: jackdied@fb.com

Subscribers: ruibalp, alandau, bmatheny, mshneer, jackdied, fried

FB internal diff: D1711820

Tasks: 5674845

Signature: t1:1711820:1417538140:a4cdc67877f61463fa0ce046dad61d8db6b13a11
nit - remove unneeded keychain.service

Summary:
this is a nit to reduce the surface area of the keychain.service
usages

Test Plan: run the tutorial server + client, see the reversed string. amazing!

Reviewed By: andrei.bajenov@fb.com

Subscribers: ruibalp, alandau, mshneer

FB internal diff: D1703655

Signature: t1:1703655:1417567110:6748ec67e4988107ff6e3c605d7c80bba3ebfae9
Modifying thrift/java/swift to be able to compile

Summary:
Fixed and cleaned up pom files to make example compile.
(and updated parent fb pom version)

Diffing out first in fbcode, so differences are visible,
then will just copy it to fbjava without changes.

Test Plan:
mvn clean install

Only complaint left were duplicate org.apache.commons.logging classes,
ignored that.

Reviewed By: dilipj@fb.com

Subscribers: istocniks, dilipj, andrewcox, alandau, mshneer

FB internal diff: D1632377

Signature: t1:1632377:1415258143:cdb91c55ce6613e9b37c95f61abd9f237ed81190
Required enums with a default shouldn't be nullable

Summary: Ads team was concerned about adding nullthrows everywhere when I made enums nullable, to prepare for first class enums. This makes enum values non-nullable if the field is marked as required (we will throw if an invalid value is read from the wire) and a default is set, so we can guarantee there is a valid value at all times.

Test Plan: Regenerate all thrift in www, look at required enums with values and confirm they are non-nullable.

Reviewed By: mcduff@fb.com

Subscribers: trunkagent, alandau, mshneer

FB internal diff: D1691697

Signature: t1:1691697:1417646106:0b70bac8bce2e520047e8a2e24cac0bd0148d0e3