Only match phase on seek if playing.