skydb/sky — pull request #136 by boourns (2013-09-10 21:13:30+00:00)
Title: Use read-only LMDB transactions to allow parallel querying per servlet

@benbjohnson

When load testing we noticed SkyDB would take longer and longer to process requests, but not max CPU or disk time.

Turns out LMDB was locking the servlet per query, because our querying was using read-write transactions. I confirmed this by adding a 20s sleep halfway through `RunQuery` and confirmed that if I ran 2 queries in parallel, the first would take 20s and the second would take ~40s.

To use read-only transactions in parallel we also need to turn off LMDB's usage of thread local storage, since goroutines will be executed in the same kernel thread.

Not sure if we're missing anywhere else we should set readonly to true - factors db perhaps?

Also, I think this means that until today we're not using skydb for reading and writing at the exact same time. Not sure if issues will arise once we're doing that.