Remove left over parts of caching

This caused the memory leak over time.

Finally closes #14 for good.