Stop memory leaks by having expiring memoization.

The memoization values each have an associated timer, which will
expire it as a certain time since the last update.