libgit2/libgit2 — pull request #2108 by arrbee (2014-02-09 00:57:49+00:00)
Title: Make index iterator thread safe

Making the index fully threadsafe is a large task, but making the index iterator take a snapshot of the index content and use that is a much more manageable project. The goals of this PR are:

1. Make the index iterator use a snapshot of the index content
2. Protect the index entries vector during mutations with a mutex
3. While there are outstanding index iterators, defer freeing index entries until the iterators are freed
4. Make index iterator sort order independent of the native order of the index by resorting the snapshot

I wrote a bunch of the code to test out these ideas and then refactored it into a few more coherent commits along with a certain amount of cleanup stuff and debugging helper stuff. There are still a few things left to do:

- [ ] allow index iterator to re-sort the entries (and get rid of re-sorting cruft in tree to index diff)
- [ ] add threaded tests of concurrent diffs using the index
- [ ] add threaded tests of diff while another thread adds and deletes from the index (making sure deferred deletes come into play)

This PR does **not** make general index usage thread-safe. There are still a ton of places where the code sorts the index and then does a binary search without holding any kind of lock, so another thread could easily swoop in and mutate the index in the middle of the binary search. For that to work, either more `rwlock` work will be required or a better index data structure should be used.