amplab/graphx — pull request #61 by ankurdave (2013-11-10 08:25:55+00:00)
Title: Shuffle replicated vertex attributes efficiently in columnar format

Vertex attributes were previously replicated in row format using MessageToPartition. This pull request instead replicates them in column format by grouping together all attributes being sent from a vertex partition to an edge partition.

For PageRank on LiveJournal, this change reduces communication by about 10% compared to #55 (from 1276 to 1156 MB per shuffle), and overall runtime by 20% (from 315 +/- 8.0 s to 252 +/- 3.1 s).

**Depends on #55. Incremental diff: https://github.com/ankurdave/graphx/compare/aggregateNeighbors-variants...pid2vid**

Resolves #31.