Remove memoization of display metrics

This will keep a static reference to the context stored in an atom: Very
bad idea!