Do not reset the focus surface pointer if it found a valid one