samplers: do not run prepare() on buffer set