I've been researching MPSC queues over the weekend and got some observations:
- Synchronized calls may be elided by the JIT, but this isn't helpful when the head and tail of the queue is in different threads.
- juc.Lock gives good concurrent throughput but mostly because of the thread-parking behavior, which adds platform-dependent latency from 1us (Sparc) to 2ms (Windows) which basically serializes out the benchmark and may give false impressions.
- MPSC queue implementations scale not as well because the constant thrashing on the tail of the queue. I've been experimenting with a striped variant where there are as many MPSC queues as are CPUs and each producer is assigned one queue based on its thread id modulo. The values are then consumed round-robin. This setup seem to keep up the throughput but I haven't battle-tested the setup yet.
- Most unbounded MPSC queues I saw were linked-list-like-based which prevents striped-read optimizations in CPUs due to the nodes being all around the place in memory. When going bounded, ringbuffers work but may deadlock if the consumer thread tries to place an item in the buffer when its full: a very likely event due to schedulePeriodically and recursive scheduling we allow. I've experimented with Disruptor in this regard but couldn't make it work properly and gave poor throughput compared to the other approaches.