Update scoreboard and bar for spectators too