fix a bug when processing workers that support sample_sets