Add the concept of cache heuristics.

This provides a solution to #32 and #18