Fix issues with lingering scoreboards