Cache only the state of the sampler