mesos/spark — pull request #302 by tdas (2012-11-09 23:49:50+00:00)
Title: Blockmanager fix

Fixed BlockManager deadlock.
1. Changed the lock structure of BlockManager by replacing the 337 coarse-grained locks to use BlockInfo objects as per-block fine-grained locks.
2. Changed the MemoryStore lock structure by making the block putting threads lock on a different object (not the memory store) thus making sure that putting threads minimally contends with the getting threads.
3. Added spark.storage.ThreadingTest to stress test the BlockManager using 5 block producer and 5 block consumer threads.

Tested with 5 producer (consumer) threads each doing 50k puts (gets), took 15 minutes to run, no errors or deadlocks.