Move zero inside the thunk.

This means we don't spuriously zero the tensor we're about to reuse.
Cleanup caching code