Tweak to fix premature set change regarding release event delays.

Previously, a set change on release would occur as
the last assigned key was being held down. This tweak
ensures that the joystick will stay on a set until
all key delays have been processed.