Add separate key buffer for shortcuts to fix misbehaving panning with keyboard