Remove votes on challenge deletion