buffer: fix caching of fetch function in sampler_prepare