Made changes to the SparkContext to have a DynamicVariable for setting local properties that can be passed down the stack. Added an implementation of the fair scheduler
Changed the name of the system property to set the allocation xml
Updated/Refactored the Fair Task Scheduler. It does not inherit ClusterScheduler anymore. Rather, ClusterScheduler internally uses TaskSetQueuesManager that handles the scheduling of taskset queues. This is the class that should be extended to support other scheduling policies
Cleaned up the code
Removed some commented code
refactor fair scheduler implementation
1.Chage "pool" properties to be the memeber of ActiveJob
2.Abstract the Schedulable of Pool and TaskSetManager
3.Abstract the FIFO and FS comparator algorithm
4.Miscellaneous changing of class define and construction
Just update typo and comments
1.remove redundant spacing in source code
2.replace get/set functions with val and var defination
change some typo and some spacing
add properties default value null to fix sbt/sbt test errors
Shuffle Performance fix: Use netty embeded OIO file server instead of ConnectionManager
Shuffle Performance Optimization: do not send 0-byte block requests to reduce network messages
change reference from io.Source to scala.io.Source to avoid looking into io.netty package

Signed-off-by: shane-huang <shengsheng.huang@intel.com>
Contiue to update codes for scala code style
1.refactor braces for "class" "if" "while" "for" "match"
2.make code lines less than 100
3.refactor class parameter and extends defination
fix a bug in netty Block Fetcher

Signed-off-by: shane-huang <shengsheng.huang@intel.com>
refactor the Schedulable interface and add unit test for SchedulingAlgorithm
remove TaskSetQueueManager and update code style
first attempt at an RDD to pull data from JDBC sources
prevent mysql driver from pulling entire resultset into memory. explicitly close resultset and statement.
Added a BlockObjectWriter interface in block manager so ShuffleMapTask
doesn't need to build up an array buffer for each shuffle bucket.
Allow changing the serializer on a per shuffle basis.
Allow the specification of a shuffle serializer in the read path (for
local block reads).