Clean up control over active view in removeKeyboardControl