graphlab-code/graphlab — pull request #38 by ylow (2013-09-15 03:50:12+00:00)
Title: RPC Performance Enhancements

Major modifications of the RPC internal stack, resulting in more than a 3x performance improvement for asynchronous engine executions.

mpiexec -n 2 -hostfile ~/machines ./pagerank_old --powerlaw=1000000 --engine=async
engine time = 40.1s

mpiexec -n 2 -hostfile ~/machines ./pagerank --powerlaw=1000000 --engine=async
engine time = 11.2s

Synchronous performance is not impacted, and may in fact encounter substantial performance gains as well (unable to test synchronous definitively due to machine asymmetry in our current test machines). Single machine tests suggest that there may be large performance gains here as well.

Summary of changes:
- Message sends now go to thread-local buffers. Send thread performs a sweep to pick up all messages. This provides much greater opportunities for message combining. Sends of small messages are now down to <300 cycles.
- All RPC tuning parameters are now isolated to dc_compile_parameters.hpp
- A fiber-aware buffered exchange has been added, and the synchronous engine now uses that.