add some caching code which seems to give 15-20% speedup in offline mode