Improve performance again

I hate slow code so fix the performance by caching the whether values
and avoid round trips to mongod this way. 

This gets runtime down to ~100 seconds