apache/spark — pull request #1871 by Ishiihara (2014-08-10 05:01:30+00:00)
Title: [SPARK-2907] [MLlib] Use mutable.HashMap to represent model in Word2Vec

Change list:
1. Used mutable.HashMap to represent syn0Global and syn1Global to reduce shuffle size.
2. Introduced local vocabulary to perform more precise learning rate update.
3. Replace layer1Size with vectorSize to correctly set vector size. Previously, layer1Size was always the default value of vectorSize.

For 100 partitions, using mutable.HashMap reduces shuffle size from 8.1G to 4G.