Remove unnecessary code at scoreboard removal