Only count down movesToGo if we're not in sudden death