Only change the ignoreKeys variable once

Currently we call ignoreKeys() on every player, which then all set
the ignoreKeys variable on their own keyhandler, which is always the
same instance.