neo4j/neo4j — pull request #3508 by tinwelint (2014-11-20 20:15:13+00:00)
Title: Lock-free loading of relationships

Removes node lock previously used during loading of relationships. This
lock would contend with committing transaction and produce latency issues.
The lock prevented cache poisoning issues introduced by race conditions
between committer and reader(s).

To achieve a lock-free solution some guards had to be added for various
corner cases. These are:
- Committers are now aware of ot he fact that, and parries for,
concurrent readers loading and caching relationships committed by the
committer, before the committer has put them into cache itself,
producing duplicates when the committer would add them.
(Issue comes from the fact that store is updated before cache).
- Committers uses knowledge about how relationhips are stored on disk to
be able to tell if any reader has loaded the relationships it has just
written to the store, before updating the cache. This logic sits in a
filter which is consulted before adding relationships to the cache.
- Nodes evicted from cache as part of being upgraded from sparse to dense
now happens as part of updating the node instead of as a separate step
during commit. This to also guard for concurrent readers.
- Simplification and reduction of code and logic of DenseNodeChainPosition
where all individual chain positions are initialized eagerly, so that
the filtering logic (as mentioned above) becomes simpler.
- Ordering of relationship-related records when applying transaction
to store changed to: relationships, relationship groups, nodes so that
additions to relationship chains are seen atomically by
concurrent readers.