Initial work towards scheduler refactoring:

- Replace use of hostPort vs host in Task.preferredLocations with a
  TaskLocation class that contains either an executorId and a host or
  just a host. This is part of a bigger effort to eliminate hostPort
  based data structures and just use executorID, since the hostPort vs
  host stuff is confusing (and not checkable with static typing, leading
  to ugly debug code), and hostPorts are not provided by Mesos.

- Replaced most hostPort-based data structures and fields as above.

- Simplified ClusterTaskSetManager to deal with preferred locations in a
  more concise way and generally be more concise.

- Updated the way ClusterTaskSetManager handles racks: instead of
  enqueueing a task to a separate queue for all the hosts in the rack,
  which would create lots of large queues, have one queue per rack name.

- Removed non-local fallback stuff in ClusterScheduler that tried to
  launch less-local tasks on a node once the local ones were all
  assigned. This change didn't work because many cluster schedulers send
  offers for just one node at a time (even the standalone and YARN ones
  do so as nodes join the cluster one by one). Thus, lots of non-local
  tasks would be assigned even though a node with locality for them
  would be able to receive tasks just a short time later.

- Renamed MapOutputTracker "generations" to "epochs".
More scheduling fixes:

- Added periodic revival of offers in StandaloneSchedulerBackend

- Replaced task scheduling aggression with multi-level delay scheduling
  in ClusterTaskSetManager

- Fixed ZippedRDD preferred locations because they can't currently be
  process-local

- Fixed some uses of hostPort
Comment cleanup (via Kay) and some debug messages
Address some review comments:

- When a resourceOffers() call has multiple offers, force the TaskSets
  to consider them in increasing order of locality levels so that they
  get a chance to launch stuff locally across all offers

- Simplify ClusterScheduler.prioritizeContainers

- Add docs on the new configuration options
Added some comments on threading in scheduler code
Added unit tests for ClusterTaskSetManager, and fix a bug found with
resetting locality level after a non-local launch
Allow subclasses of Product2 in all key-value related classes (ShuffleDependency, PairRDDFunctions, etc).
Added the missing RDD files and cleaned up SparkContext.
Made PairRDDFunctions taking only Tuple2, but made the rest of the shuffle code path working with general Product2.
Added a test for sorting using MutablePair's.
Code review feedback. (added tests for cogroup and substract; added more documentation on MutablePair)
Small fixes to web UI:

- Use SPARK_PUBLIC_DNS environment variable if set (for EC2)
- Use a non-ephemeral port (3030 instead of 33000) by default
- Updated test to use non-ephemeral port too
Merge pull request #847 from rxin/rdd

Allow subclasses of Product2 in all key-value related classes
Allow some wiggle room in UISuite port test and in EC2 ports
Merge pull request #849 from mateiz/web-fixes

Small fixes to web UI
Merge pull request #828 from mateiz/sched-improvements

Scheduler fixes and improvements
Merge branch 'master' of https://github.com/mesos/spark into master-csd

Conflicts:
	core/src/test/scala/spark/scheduler/DAGSchedulerSuite.scala