Fix removeKeyListener to remove only the one client-key pair

Instead of all listeners to the key.

Also rename key listener disconnect to remove to match add.