Fix the bot telling you you haven't voted after retracting abstain

Reported by @Vgr255 on IRC.