Move the cache into the context.

This simplifies the code base in a number of commits since the context can *contain* the cahce and this makes the state way simpler to handle in the long run.