Attempt to track, and remove scoreboards at reconnect