apache/spark — pull request #3505 by zsxwing (2014-11-28 07:02:47+00:00)
Title: [SPARK-4644][Core] Implement skewed join

Skewed data is not rare. For example, a book recommendation site may have several books which are liked by most of the users. Running ALS on such skewed data will raise a OutOfMemory error, if some book has too many users which cannot be fit into memory. To solve it, we propose a skewed join implementation. [Design Doc](https://issues.apache.org/jira/secure/attachment/12684140/Skewed%20Join%20Design%20Doc.pdf)