SPARK-1650: Correctly identify maven project version

Better account for various side-effect outputs while executing
"mvn help:evaluate -Dexpression=project.version"

Author: Rahul Singhal <rahul.singhal@guavus.com>

Closes #572 from rahulsinghaliitd/SPARK-1650 and squashes the following commits:

fd6a611 [Rahul Singhal] SPARK-1650: Correctly identify maven project version
(cherry picked from commit 7b2527d74deac9512f8ee9ad6d4b060f05e1ab26)

Signed-off-by: Patrick Wendell <pwendell@gmail.com>
SPARK-1648 Support closing JIRA's as part of merge script.

Adds an automated hook in the merge script that can close the JIRA,
set the fix versions, and leave a comment on the JIRA indicating the
PR in which it was resolved. This ensures that (a) we always close JIRA's
when issues are merged and (b) there is a link to the pull request in every JIRA.

This requires a python library called `jira-client`. We could look at embedding this
library in our project, but it seemed simple enough to just gracefully disable this
feature if it is not installed. It can be installed with `pip install jira-client`.

Author: Patrick Wendell <pwendell@gmail.com>

Closes #570 from pwendell/jira-pr-merge and squashes the following commits:

3022b96 [Patrick Wendell] SPARK-1648 Support closing JIRA's as part of merge script.
(cherry picked from commit fe65beeaab737a1c9922b84452ca661b39075cfa)

Signed-off-by: Patrick Wendell <pwendell@gmail.com>
SPARK-1651: Delete existing deployment directory

Small bug fix to make sure the "spark contents" are copied to the
deployment directory correctly.

Author: Rahul Singhal <rahul.singhal@guavus.com>

Closes #573 from rahulsinghaliitd/SPARK-1651 and squashes the following commits:

402c999 [Rahul Singhal] SPARK-1651: Delete existing deployment directory
(cherry picked from commit eefb90d382747c29d7537630ed5ad2c783bb8263)

Signed-off-by: Patrick Wendell <pwendell@gmail.com>
HOTFIX: Minor patch to merge script.
SPARK-1145: Memory mapping with many small blocks can cause JVM allocation failures

This includes some minor code clean-up as well. The main change is that small files are not memory mapped. There is a nicer way to write that code block using Scala's `Try` but to make it easy to back port and as simple as possible, I opted for the more explicit but less pretty format.

Author: Patrick Wendell <pwendell@gmail.com>

Closes #43 from pwendell/block-iter-logging and squashes the following commits:

1cff512 [Patrick Wendell] Small issue from merge.
49f6c269 [Patrick Wendell] Merge remote-tracking branch 'apache/master' into block-iter-logging
4943351 [Patrick Wendell] Added a test and feedback on mateis review
a637a18 [Patrick Wendell] Review feedback and adding rewind() when reading byte buffers.
b76b95f [Patrick Wendell] Review feedback
4e1514e [Patrick Wendell] Don't memory map for small files
d238b88 [Patrick Wendell] Some logging and clean-up
(cherry picked from commit 6b3c6e5dd8e74435f71ecdb224db532550ef407b)

Signed-off-by: Patrick Wendell <pwendell@gmail.com>
Fix SPARK-1609:  Executor fails to start when Command.extraJavaOptions contains multiple Java options

Author: witgo <witgo@qq.com>

Closes #547 from witgo/SPARK-1609 and squashes the following commits:

deb6a4c [witgo] review commit
91da0bb [witgo] Merge branch 'master' of https://github.com/apache/spark into SPARK-1609
0640852 [witgo] review commit
8f90b22 [witgo] Merge branch 'master' of https://github.com/apache/spark into SPARK-1609
bcf36cb [witgo] Merge branch 'master' of https://github.com/apache/spark into SPARK-1609
1185605 [witgo] fix extraJavaOptions split
f7c0ab7 [witgo] bugfix
86fc4bb [witgo] bugfix
8a265b7 [witgo] Fix SPARK-1609: Executor fails to start when use spark-submit
Update the import package name for TestHive in sbt shell

sbt/sbt hive/console will fail as TestHive changed its package from "org.apache.spark.sql.hive" to "org.apache.spark.sql.hive.test".

Author: Cheng Hao <hao.cheng@intel.com>

Closes #574 from chenghao-intel/hive_console and squashes the following commits:

de14035 [Cheng Hao] Update the import package name for TestHive in sbt shell

(cherry picked from commit ea01affc34a8b816f92a71e0f8f6901397c8a4a3)
Signed-off-by: Reynold Xin <rxin@apache.org>
[SQL]Append some missing types for HiveUDF

Add the missing types

Author: Cheng Hao <hao.cheng@intel.com>

Closes #459 from chenghao-intel/missing_types and squashes the following commits:

21cba2e [Cheng Hao] Append some missing types for HiveUDF

(cherry picked from commit f735884414a15c0c07df60068ee11f9da47eff77)
Signed-off-by: Reynold Xin <rxin@apache.org>