Fix a considerable memory leak

Considerable meaning "we were keeping around a copy of everything
we've ever taken a hash of."