SPARK-4156 [MLLIB] EM algorithm for GMMs

Implementation of Expectation-Maximization for Gaussian Mixture Models.

This is my maiden contribution to Apache Spark, so I apologize now if I have done anything incorrectly; having said that, this work is my own, and I offer it to the project under the project's open source license.


Agreed.  This can be part of bringing MultivariateGaussian to the public scope.


> On Dec 18, 2014, at 1:43 PM, jkbradley <notifications@github.com> wrote:
> 
> In mllib/src/main/scala/org/apache/spark/mllib/stat/impl/MultivariateGaussian.scala:
> 
> > +
> > +package org.apache.spark.mllib.stat.impl
> > +
> > +import breeze.linalg.{DenseVector => BreezeVector, DenseMatrix => BreezeMatrix}
> > +import breeze.linalg.{Transpose, det, pinv}
> > +
> > +/** 
> > +   * Utility class to implement the density function for multivariate Gaussian distribution.
> > +   * Breeze provides this functionality, but it requires the Apache Commons Math library,
> > +   * so this class is here so-as to not introduce a new dependency in Spark.
> > +   */
> > +private[mllib] class MultivariateGaussian(
> > +    val mu: BreezeVector[Double], 
> > +    val sigma: BreezeMatrix[Double]) extends Serializable {
> > +  private val sigmaInv2 = pinv(sigma) * -0.5
> > +  private val U = math.pow(2.0 * math.Pi, -mu.length / 2.0) * math.pow(det(sigma), -0.5)
> By the way, det and pinv are factorizing the matrix twice. It would be better to do one factorization (like SVD) and then compute the det and inv from it. We can do that in a follow-up PR though.
> 
> —
> Reply to this email directly or view it on GitHub.