Track mouse and keyboard focuse separately. Make joy depend on key.

This more accurately models how one would expect the joystick to work
on Linux.

Fixes #402.