Refactor SGD options into a new class.
This refactoring pulls out code shared between SVM, Lasso, LR into
a common GradientDescentOpts class. Some style cleanup as well
Log the launch command for Spark daemons

For debugging and analysis purposes, it's nice to have the exact command
used to launch Spark contained within the logs. This adds the necessary
hooks to make that possible.
Refactor optimizers and create GLMs
This change refactors the structure of GLMs to use mixins which maintain
a similar interface to other ML lib algorithms. This change also creates
an Optimizer trait which allows GLMs to be extended to use other optimization
techniques.
Log the launch command for Spark executors
Move implicit arg to constructor for Java access.
Make output formatting consistent between bash/scala
expose HDFS file system stats via Executor metrics
reformat registerFileSystemStat()
Java examples, tests for KMeans and ALS
- Changes ALS to accept RDD[Rating] instead of (Int, Int, Double) making it
  easier to call from Java
- Renames class methods from `train` to `run` to enable static methods to be
  called from Java.
- Add unit tests which check if both static / class methods can be called.
- Also add examples which port the main() function in ALS, KMeans to the
  examples project.

Couple of minor changes to existing code:
- Add a toJavaRDD method in RDD to convert scala RDD to java RDD easily
- Workaround a bug where using double[] from Java leads to class cast exception in
  KMeans init
Add a test case for random initialization.
Also workaround a bug where double[][] class cast fails
Refactor GLM algorithms and add Java tests
This change adds Java examples and unit tests for all GLM algorithms
to make sure the MLLib interface works from Java. Changes include
- Introduce LabeledPoint and avoid using Doubles in train arguments
- Rename train to run in class methods
- Make the optimizer a member variable of GLM to make sure the builder
  pattern works
Merge branch 'master' of git://github.com/mesos/spark into sgd-cleanup

Conflicts:
	mllib/src/main/scala/spark/mllib/util/MLUtils.scala
For standalone mode, add worker local env setting of SPARK_JAVA_OPTS as default and let application env override default options if applicable

Signed-off-by: shane-huang <shengsheng.huang@intel.com>
Adding Scala version of PageRank example
Fixed 3 bugs that caused UI to crash (including SPARK-810).

One bug caused the UI to crash if you try to look at a job's status
before any of the tasks have finished.

The second bug was a concurrency issue where two different threads
(the scheduling thread and a UI thread) could be reading/updating
the data structures in JobProgressListener concurrently.

The third bug mis-used an Option, also causing the UI to crash
under certain conditions.
Style changes as per Matei's comments
Attempt to fix bug reported in PR 791 : a race condition in ConnectionManager and Connection
Change line size
Remove Java-specific constructor for Rating.
The scala constructor works for native type java types. Modify examples
to match this.
Add setters for optimizer, gradient in SGD.
Also remove java-specific constructor for LabeledPoint.