Fix glitch where players would keep the colored hats