Fix: Only act when InputManager asks *us* to