Don't lock up when a client isn't paying attention

Also fix the mouse to account for this... development.
Smoother mouse movement all around, though it may make things slightlys
lower by spamming lots of mouse packets. Just write good clients and
everything will be fine...