LRUCache optimisation

This improves LRUCache performance pretty substantially (particularly for the multithreaded case) while keeping the same basic algorithm as it had before. I'm afraid the bulk of the work is in a single commit, because the rather wandering path I took to get to this point didn't make sense as individual commits. I have however added a lot of documentation to the private members which hopefully does a pretty decent job of explaining what's going on - so it probably makes more sense just to read the final result rather than look at the diffs.
We have some uses like ComputationCache which the cost is constant 1 and it seems wasteful always representing cost with a size_t. I suggest two changes: make it also a templated argument, so we could represent as a single byte and move this line to closer to the "char status" line, so the compiler can potentially pack this two fields in the same word and have smaller sizeof(CacheEntry)