[SPARK-4920][UI] add version on master and worker page for standalone mode

Author: Zhang, Liye <liye.zhang@intel.com>

Closes #3769 from liyezhang556520/spark-4920_WebVersion and squashes the following commits:

3bb7e0d [Zhang, Liye] add version on master and worker page

(cherry picked from commit 9077e721cd36adfecd50cbd1fd7735d28e5be8b5)
Signed-off-by: Josh Rosen <joshrosen@databricks.com>
[SPARK-4882] Register PythonBroadcast with Kryo so that PySpark works with KryoSerializer

This PR fixes an issue where PySpark broadcast variables caused NullPointerExceptions if KryoSerializer was used.  The fix is to register PythonBroadcast with Kryo so that it's deserialized with a KryoJavaSerializer.

Author: Josh Rosen <joshrosen@databricks.com>

Closes #3831 from JoshRosen/SPARK-4882 and squashes the following commits:

0466c7a [Josh Rosen] Register PythonBroadcast with Kryo.
d5b409f [Josh Rosen] Enable registrationRequired, which would have caught this bug.
069d8a7 [Josh Rosen] Add failing test for SPARK-4882

(cherry picked from commit efa80a531ecd485f6cf0cdc24ffa42ba17eea46d)
Signed-off-by: Josh Rosen <joshrosen@databricks.com>
[SPARK-4908][SQL] Prevent multiple concurrent hive native commands

This is just a quick fix that locks when calling `runHive`.  If we can find a way to avoid the error without a global lock that would be better.

Author: Michael Armbrust <michael@databricks.com>

Closes #3834 from marmbrus/hiveConcurrency and squashes the following commits:

bf25300 [Michael Armbrust] prevent multiple concurrent hive native commands

(cherry picked from commit 480bd1d2edd1de06af607b0cf3ff3c0b16089add)
Signed-off-by: Michael Armbrust <michael@databricks.com>
[SPARK-4386] Improve performance when writing Parquet files

Convert type of RowWriteSupport.attributes to Array.

Analysis of performance for writing very wide tables shows that time is spent predominantly in apply method on  attributes var. Type of attributes previously was LinearSeqOptimized and apply is O(N) which made write O(N squared).

Measurements on 575 column table showed this change made a 6x improvement in write times.

Author: Michael Davies <Michael.BellDavies@gmail.com>

Closes #3843 from MickDavies/SPARK-4386 and squashes the following commits:

892519d [Michael Davies] [SPARK-4386] Improve performance when writing Parquet files

(cherry picked from commit 7425bec320227bf8818dc2844c12d5373d166364)
Signed-off-by: Michael Armbrust <michael@databricks.com>
[SPARK-4813][Streaming] Fix the issue that ContextWaiter didn't handle 'spurious wakeup'

Used `Condition` to rewrite `ContextWaiter` because it provides a convenient API `awaitNanos` for timeout.

Author: zsxwing <zsxwing@gmail.com>

Closes #3661 from zsxwing/SPARK-4813 and squashes the following commits:

52247f5 [zsxwing] Add explicit unit type
be42bcf [zsxwing] Update as per review suggestion
e06bd4f [zsxwing] Fix the issue that ContextWaiter didn't handle 'spurious wakeup'

(cherry picked from commit 6a897829444e2ef273586511f93a40d36e64fb0b)
Signed-off-by: Tathagata Das <tathagata.das1565@gmail.com>