Make unions work correctly.

Summary: Unions don't work with thrift's hack generator. This makes it work by treating unions basically the same as structs with all optional members.

Test Plan: Not sure how to test this! I see some PHP tests in fbcode. Please advice!

Reviewed By: ckwalsh@fb.com

Subscribers: alandau, campbellmadam

FB internal diff: D1438988

Tasks: 4709436
another fix (old client -> new server)

Summary:
Not sure why it worked before and then broke.
If the server receives a compact message with a binary header, it tries
to deserialize with binary, fails, retries with compact and (the new
thing in this diff) remembers to use compact for the reply.
Previously the reply would have been serialized with the protocol from
the header (which broke old binary clients)

Facebook:
@override-unit-failures
- it thinks hphp_dbg_contbuild_push_blocking is still pending, while
sandcastle says it finished successfully

Test Plan: tests old client -> new server

Reviewed By: haijunz@fb.com

Subscribers: alandau, bmatheny

FB internal diff: D1498610

Tasks: 4914390
limit the number of outstanding secure connects; manage timeouts better

Summary:
Modify the thread mananger to queue connection setups if too
many are pending.  This also adds a callback to indicate when the
connection setup is actually starting, so timeouts can work more
meaningfully when a lot of connections are started at once.

Test Plan:
Test fanout to many servers at once; confirm that all
expected connections succeed.

Reviewed By: andrei.bajenov@fb.com

Subscribers: mcduff, hitesh, mshneer, marccelani, alandau, bmatheny

FB internal diff: D1479164

Tasks: 4793859
DistinctTable

Test Plan: Unit tests

Reviewed By: henryf@fb.com

Subscribers: alandau, bmatheny

FB internal diff: D1496778

Tasks: 4909108
Rename JSONProtocol to SimpleJSONProtocol

Summary: Calling this JSONProtocol is misleading because it is really an instance of the TSimpleJSONProtocol.  I have changed the name to reflect this

Test Plan: fbconfig -r thrift/lib/hs && fbmake runtests_dev

Reviewed By: jon.coens@fb.com

Subscribers: ldbrandy, alandau, bmatheny, smarlow, akr, bnitka, jcoens

FB internal diff: D1498899

Tasks: 4916083
Implement TJSONProtocol

Summary: Implementation of the TJSONProtocol ported from Apache Thrift

Test Plan: fbconfig -r thrift/lib/hs && fbmake runtests_dev

Reviewed By: jon.coens@fb.com

Subscribers: alandau, bmatheny, jcoens

FB internal diff: D1499567

Tasks: 4916083
Use TMemoryBuffer instead of new transport

Summary: I kind of reinvented the wheel originally in the integration tests by writing the MockTransport.  It already existed and is called TMemoryBuffer.  I refactored to use this instead

Test Plan: fbconfig -r thrift/lib/hs && fbmake runtests_dev

Reviewed By: jon.coens@fb.com

Subscribers: alandau, bmatheny, jcoens

FB internal diff: D1498561
Server load % header

Summary: Adds a server load % header to send back to the client (SR).  Currently only sends for overload errors, but we could also send for all requests (assuming it isn't too expensive).

Test Plan:
fbconfig thrift/perf/cpp; fbmake dbg
ThriftServer --max_connections=150 --max_requests=100
loadgen --weight_burn=1 --async -cpp2 --num_threads=10 --async_ops=100

Reviewed By: hitesh@fb.com

Subscribers: fbcode-common-diffs@, davejwatson, andrewcox, sorina, doug, alandau, bmatheny, mcduff

FB internal diff: D1390750

Tasks: 4523025
Avoid unnecessary copy while parsing SSL client hello

Summary:
Using wrapBuffer() to avoid copying the buffer during client hello.
Copying is done only during cases where the callback doesn't contain
the entire message for client hello.

Test Plan:
Existing unit tests for SSL client hello parsing pass:
_bin/thrift/lib/cpp/test/TAsyncSSLSocketTest

Reviewed By: afrind@fb.com

Subscribers: alandau, ssl-diffs@, bmatheny

FB internal diff: D1482374

Tasks: 4555879
Revert "limit the number of outstanding secure connects; manage timeouts better"

Summary:
This reverts commit 613f4672fd53be1e49f9ee44a85daeb90dd471b5.

Its causing crashes in perflab and sandcastle.

Test Plan: automated tests, perflabs no longer crashing

Reviewed By: andrei.bajenov@fb.com

Subscribers: mcduff, marccelani, hitesh, mshneer, alandau, bmatheny

FB internal diff: D1502316

Tasks: 4925245

Blame Revision: D1479164
Throw exception from future methods returning void

Summary:
We were not generating the right code. In the callback of the future
method, we need to check if an exception exists and throw it.

Test Plan: Where are the unit tests for this?

Reviewed By: davejwatson@fb.com

Subscribers: ruibalp, fugalh, alandau

FB internal diff: D1505251

Tasks: 4939669
Use THeaderTransport in BaseThriftClient

Summary: If client set __TRANSPORT__ to THeaderTransport this will end up using the default protocol id of 2 for TCompact, but the default protocol here is TBinaryAccelerated.

Test Plan: See the client and server in t4906615. With this change the client works with THeaderTransport.

Reviewed By: davejwatson@fb.com

Subscribers: alandau, bmatheny, yongshi, acampi, ruibalp

FB internal diff: D1498643

Tasks: 4906615
Fix TCppServer not working with unframed

Summary: TCppServer breaks when client is using unframed because readFrame accepts a message length para but it is always passed 0. It is ignored for framed and header, but needed for unframed.

Test Plan: Tested with a TCppServer and an unframed client. With this fix the unframed client works.

Reviewed By: mhorowitz@fb.com

Subscribers: ruibalp, alandau, bmatheny

FB internal diff: D1503083

Tasks: 4906615
wait to call connectionDestroyed

Summary: don't call connectionDestroyed until after all the requests have compeleted.  stop() marks everything as canceled, but we can't actually *stop* tasks running in the threadManager.  Instead they have shared_ptrs to the connection object.   Move the connectionDestroyed call to the destructor, so that it isn't called until *after* all the requests have finished, including any processoreventhandler events.

Test Plan:
?? dihde do you have an easy repro? I should probably write a unittest

pusheen_kind_of_lazy

Reviewed By: alandau@fb.com

Subscribers: doug, alandau, bmatheny

FB internal diff: D1498831

Tasks: 4915237
rm -rf streaming

Summary: Unused intern project, subsumed by duplex for pub/sub

Test Plan: contbuild

Reviewed By: alandau@fb.com

Subscribers: mcduff, marccelani, hitesh, mshneer, doug, ruibalp, nli, fugalh, alandau, bmatheny

FB internal diff: D1504102
limit the number of outstanding secure connects; manage timeouts better

Summary:
Modify the thread mananger to queue connection setups if too
many are pending.  This also adds a callback to indicate when the
connection setup is actually starting, so timeouts can work more
meaningfully when a lot of connections are started at once.  This is a
resubmit of D1479164 which caused perflab errors (see t4925245).  It
includes changes to stash a copy of saslThreadManager_, and to hold a
mutex when accessing evb_ to guarantee the channel remains available
for the duration of the call.

Test Plan:
Test fanout to many servers at once; confirm that all
expected connections succeed.  Run some perflab experiments as
described in t4925245, make sure they succeed.

Reviewed By: andrei.bajenov@fb.com

Subscribers: trunkagent, bmatheny, alandau, marccelani, mshneer, hitesh, mcduff

FB internal diff: D1503561

Tasks: 4793859
Make unions work correctly in cpp2

Summary: Scoping was off

Test Plan:
Unit tests. Also see the generated code below:

template <class Protocol_>
uint32_t TestUnion_read(Protocol_* iprot, TestUnion* obj) {
uint32_t xfer = 0;
std::string fname;
apache::thrift::protocol::TType ftype;
int16_t fid;

xfer += iprot->readStructBegin(fname);

using apache::thrift::TProtocolException;

xfer += iprot->readFieldBegin(fname, ftype, fid);
if (ftype == apache::thrift::protocol::T_STOP) {
obj->__clear();
} else
switch (fid) {
case 1:
{
if (ftype == apache::thrift::protocol::T_STRING) {
obj->set_string_field();
xfer += iprot->readString(obj->mutable_string_field());
} else {
xfer += iprot->skip(ftype);
}
break;
}
case 2:
{
if (ftype == apache::thrift::protocol::T_I32) {
obj->set_i32_field();
xfer += iprot->readI32(obj->mutable_i32_field());
} else {
xfer += iprot->skip(ftype);
}
break;
}
default:
{
xfer += iprot->skip(ftype);
break;
}
} {
xfer += iprot->readFieldEnd();
xfer += iprot->readFieldBegin(fname, ftype, fid);
xfer += iprot->readFieldEnd();
}
xfer += iprot->readStructEnd();

return xfer;
}

After the fix:

uint32_t TestUnion_read(Protocol_* iprot, TestUnion* obj) {
uint32_t xfer = 0;
std::string fname;
apache::thrift::protocol::TType ftype;
int16_t fid;

xfer += iprot->readStructBegin(fname);

using apache::thrift::TProtocolException;

xfer += iprot->readFieldBegin(fname, ftype, fid);
if (ftype == apache::thrift::protocol::T_STOP) {
obj->__clear();
} else {
switch (fid) {
case 1:
{
if (ftype == apache::thrift::protocol::T_STRING) {
obj->set_string_field();
xfer += iprot->readString(obj->mutable_string_field());
} else {
xfer += iprot->skip(ftype);
}
break;
}
case 2:
{
if (ftype == apache::thrift::protocol::T_I32) {
obj->set_i32_field();
xfer += iprot->readI32(obj->mutable_i32_field());
} else {
xfer += iprot->skip(ftype);
}
break;
}
default:
{
xfer += iprot->skip(ftype);
break;
}
}
xfer += iprot->readFieldEnd();
xfer += iprot->readFieldBegin(fname, ftype, fid);
xfer += iprot->readFieldEnd();
}
xfer += iprot->readStructEnd();

return xfer;
}

Reviewed By: davejwatson@fb.com

Subscribers: ruibalp, alandau

FB internal diff: D1495827
Sparse, unique layouts

Summary:
Recursively de-duplicated, sparse layouts.

De-duplication is done by building layout tree recursively from the bottom up, which enables shallow equality comparison to apply to the entire subtree.

Test Plan: Unit tests

Reviewed By: henryf@fb.com

Subscribers: alandau, bmatheny

FB internal diff: D1496822

Tasks: 4909108
Revert "limit the number of outstanding secure connects; manage timeouts better"

Summary: This reverts commit d53e8be80eee8373d696f083e5082808f370ffc8.

Test Plan: None

Reviewed By: mhorowitz@fb.com

Subscribers: mcduff, marccelani, hitesh, mshneer, alandau, bmatheny

FB internal diff: D1507986
Fix various compiler warnings

Summary:
This allows thrift to be compiled with -Wextra (minus
the extremely noisy unused-parameter) by including components.

Test Plan: compiled with gcc and clang

Reviewed By: alandau@fb.com

Subscribers: alandau, bmatheny, shikong, fma, benyluo, kmdent, ranjeeth, subodh

FB internal diff: D1509844