Fix logic error

Last moment change made it so that mute only worked on people with the
permission! This fixes that.