Split HadoopRDD into one for general Hadoop datasets and one tailored to Hadoop files, which is a common case.

This is the first step to avoiding unnecessary Configuration broadcasts per HadoopRDD instantiation.
Add a cache for HadoopRDD metadata needed during computation.

Currently, the cache is in SparkHadoopUtils, since it's conveniently a member of the SparkEnv.
Move Configuration broadcasts to SparkContext.
Send Task results through the block manager when larger than Akka frame size.

This change requires adding an extra failure mode: tasks can complete
successfully, but the result gets lost or flushed from the block manager
before it's been fetched.
Merge remote-tracking branch 'oldsparkme/hadoopRDD-broadcast-change' into hadoop-config-cache
Merge HadoopDatasetRDD into HadoopRDD.
Addressed Matei's code review comments
Merge remote-tracking branch 'upstream/master' into results_through-bm

Conflicts:
	core/src/main/scala/org/apache/spark/scheduler/cluster/ClusterScheduler.scala
	core/src/main/scala/org/apache/spark/scheduler/cluster/ClusterTaskSetManager.scala
	core/src/main/scala/org/apache/spark/scheduler/local/LocalTaskSetManager.scala
Fixed compilation errors and broken test.
Added additional unit test for repeated task failures
Update README
fixed a bug of using wildcard in quotes
ask ivy/sbt to check local maven repo under ~/.m2
SPARK-920 - JSON endpoint URI scheme part (spark://) duplicated
SPARK-921 - Add Application UI URL to ApplicationInfo Json output
Allow users to set the application name for Spark on Yarn
Merge pull request #10 from kayousterhout/results_through-bm

Send Task results through the block manager when larger than Akka frame size (fixes SPARK-669).

This change requires adding an extra failure mode: tasks can complete
successfully, but the result gets lost or flushed from the block manager
before it's been fetched.

This change also moves the deserialization of tasks into a separate thread, so it's no longer part of the DAG scheduler's tight loop. This should improve scheduler throughput, particularly when tasks are sending back large results.

Thanks Josh for writing the original version of this patch!

This is duplicated from the mesos/spark repo: https://github.com/mesos/spark/pull/835
Add default value to usage statement
Merge pull request #28 from tgravescs/sparYarnAppName

Allow users to set the application name for Spark on Yarn
Merge pull request #25 from CruncherBigData/master

Update README: updated the link