HIVE-8943: Fix memory limit check for combine nested mapjoins [Spark Branch] (Szehon via Xuefu)

git-svn-id: https://svn.apache.org/repos/asf/hive/branches/spark@1643058 13f79535-47bb-0310-9956-ffa450edef68
HIVE-8991: Fix custom_input_output_format [Spark Branch] (Rui via Xuefu)

git-svn-id: https://svn.apache.org/repos/asf/hive/branches/spark@1643157 13f79535-47bb-0310-9956-ffa450edef68
HIVE-9011: Fix parquet_join.q determinism (Jimmy via Xuefu)
Merged from trunk, r1643125

git-svn-id: https://svn.apache.org/repos/asf/hive/branches/spark@1643159 13f79535-47bb-0310-9956-ffa450edef68
HIVE-9027 : Enable ppd_join4 [Spark Branch] (Chao Sun via Szehon)

git-svn-id: https://svn.apache.org/repos/asf/hive/branches/spark@1643162 13f79535-47bb-0310-9956-ffa450edef68
HIVE-8783: Create some tests that use Spark counter for stats collection [Spark Branch] (Chengxiang via Xuefu)

git-svn-id: https://svn.apache.org/repos/asf/hive/branches/spark@1643294 13f79535-47bb-0310-9956-ffa450edef68
HIVE-9016: SparkCounter display name is not set correctly[Spark Branch] (Chengxiang via Xuefu)

git-svn-id: https://svn.apache.org/repos/asf/hive/branches/spark@1643295 13f79535-47bb-0310-9956-ffa450edef68
HIVE-8992: Fix bucket related test failure: parquet_join.q [Spark Branch] (Jimmy via Xuefu)

git-svn-id: https://svn.apache.org/repos/asf/hive/branches/spark@1643380 13f79535-47bb-0310-9956-ffa450edef68
HIVE-8536: Enable SkewJoinResolver for spark [Spark Branch] (Rui via Xuefu)

git-svn-id: https://svn.apache.org/repos/asf/hive/branches/spark@1643744 13f79535-47bb-0310-9956-ffa450edef68
[spark-client] Netty-based RPC implementation.

This is the RPC layer for the spark-client; it just provides the RPC mechanism
for replacing akka in the spark-client communication. See README.md for
more discussion about the approach.

The client impl is still using akka - that will be changed in a separate commit.
[spark-client] Goodbye akka, hello netty.

Use the netty-based RPC implementation for spark-client. API is still
exactly the same, just the internals have changed. API semantics should
still be the same too, although with the new RPC system there is still
room for improvement.

Main thing to remember right now is that all types sent over the wire
need an empty constructor. Currently, failure to do that will result
in errors to be printed to the logs and RPC channels to close, but
for easier debugging in the future I'm planning to change the RPC
internals a bit so that failure to deserialize the payload causes an
RPC failure, instead of the current behavior.
[spark-client] Use a Promise in JobHandleImpl.

Makes it integrate better with the RPC layer, and avoids some ugly
code in the process.
[spark-client] Set up RPC thread factory, add missing timeout.
[spark-client] Better handle serialization errors.

It's hard to report back serialization errors that happen during an
RPC if the RPC header and payload are serialized as a single object.
So break those down into two different objects; we're sure the internal
RPC header is serializable, so this allows us to better handle the case
when the user payload is not serializable and provide nicer errors,
making it easier to debug things.
Fix buglets.
Fix some serialization issues.
More fixes.

- More types needed serialization tweaks (empty constructors or
  transient fields)
- Fix a race when registering outgoing RPCs.
- Add a TODO to fix some very suspicious code in SparkCounter.
Fix SparkCounters serialization.

This code is a little bit weird; the "Accumulator" instance cannot
be serialized by Kryo's default serializer, so we cannot include
it when sending SparkCounters back to the RSC. But we can't make the
field transient, because apparently it need to be serialized when
sent to actual tasks.

Long term it would be better to separate internal counter usage
from RSC counter views (in the spirit of keeping RSC completely
independent from the rest of Hive). But right now make things work
by copying things.
Configurable channel traffic log level.
Avoid an ugly exception when remote end dies.
Make rpc server instance private in factory.