A Cool Generic Concurrency Primitive in Rust
Channel: Jon Gjengset
Date: 2020-11-22

A few years ago, I built a concurrent hash map designed specifically to support highly concurrent reads: https://github.com/jonhoo/rust-evmap. It is part of what gives Noria, my PhD thesis work, such great read performance. The concurrency primitive that enables evmap to be so fast isn't really specific to maps though, and it's bothered me for a while that the particular datastructure I used it for in Noria was the only available way to use it.

In this stream, we explore an idea I came up with a few weeks ago to make a generic version of just the concurrency primitive that can then be re-used for other datastructures as well. You can see the outline of the plan here: https://github.com/jonhoo/rust-evmap/issues/45#issuecomment-723706112. This entailed slicing out just the concurrency stuff from evmap, and making it into its own reusable crate, and then reworking evmap to use that library. We got more or less all the way there, and you can see the final result in this PR: https://github.com/jonhoo/rust-evmap/pull/73.

0:00:00 Introduction
0:10:15 How evmap works
0:42:56 The change
0:53:58 Reorganizing the code
0:58:46 Stripping out the concurrency primitive
3:01:01 Biobreak
3:03:42 Make evmap use LeftRight
4:52:38 Fixing up evmap unit tests
5:15:15 Associated types vs. generic traits
5:18:36 Fixing up more evmap unit tests
5:21:47 Fixing up the full test suite
5:37:15 Making ReadHandle take &self
6:02:18 Outro

You can see the unsafe code guidelines comment I made about transmuting a type that contains T to ManuallyDrop T here: https://github.com/rust-lang/unsafe-code-guidelines/issues/35#issuecomment-731656729.

You can watch the live version with comments at https://www.youtube.com/watch?v=pZSDTiVEcxs