Antidistillation Sampling
Authors: name: Yash Savani; name: Asher Trockman; name: Zhili Feng; name: Avi Schwarzschild; name: Alexander Robey; name: Marc Finzi; name: J. Zico Kolter
Published: 2025-04-17 17:54:14

Abstract: Frontier models that generate extended reasoning traces inadvertently produce
rich token sequences that can facilitate model distillation. Recognizing this
vulnerability, model owners may seek sampling strategies that limit the
effectiveness of distillation without compromising model performance.
Antidistillation sampling provides exactly this capability. By
strategically modifying a model's next-token probability distribution,
antidistillation sampling poisons reasoning traces, rendering them
significantly less effective for distillation while preserving the model's
practical utility. For further details, see https://antidistillation.com.