Repository: autokey/autokey

Issue #132: AutoKey and xterm
Created: 2018-04-10T02:34:28Z
System: Ubuntu 16.04
Here is a simple script.
keyboard.send_keys("<up>")
It works perfectly in every other application.
However it does not work in xterm.
Using the showkey -a and pressing the key that the script is bound to doesn't spit back anything.
Does anyone know what's going on here?

Duplicate issue #195: Feature request: Autokey and terminal
Created: 2018-10-09T08:57:53Z
Terminator or Gnome terminal ignore synthetic keypress events. As a result keyboard.send_keys does not work with these terminals. This is of course not Autokey's fault, but it reduces its usefulness (for me). I have no way to judge if this is possible, but it would be great if there would be some way to get Autokey to work with the terminal.

Closing comment (2018-10-09T13:53:08Z): I’ll start this with a bit of a narrative…
Current Interface
Currently, we use the XRecord interface for reading and sending key presses. If you create key strokes using this interface, they get emitted as synthetic key presses, which are, as you know, ignored by some applications.
A bit about input loops
AutoKey both reads and writes on the same abstraction/API layer, which conveniently has the benefit of not getting any feedback loops: AutoKey won’t receive it’s own output as input.
But to get AutoKey to work with applications ignoring key events marked as synthetic, we need to go a layer or two lower to the actual hardware. Doing so will introduce those loops. If you start emulating a physical keyboard, AutoKey will receive it’s own generated output as input again.
Currently, having a script like keyboard.send_keys("<ctrl>+u") and binding it to hotkey <ctrl>+u works as expected. But writing a (hypothetical) script like emulated_physical_keyboard.send_keys("<ctrl>+u"), and again binding it to hotkey <ctrl>+u will cause an infinite loop. The script will re-trigger itself over and over again, until you kill the running AutoKey process. The same is true for abbreviations. If a script prints it’s own abbreviation (and a suitable trigger character, if any), it will trigger itself. So keep this in mind when experimenting with the code posted below.
The roadmap to a better future (TM)
In the current state, AutoKey is tightly bound to XRecord. The interface needs to be re-written to properly fix this for both scripts and hotkeys. Here is a rough roadmap document containing what I planned on doing about it. I suspect it to take at least one or two full-time weeks to get the current interface code to a state where it is easily possible to plug in other interfaces and output generation systems.
Working around XRecord limitations in Scripts
In the meantime we can work around those restrictions directly in AutoKey scripts. It won’t work for Phrases, but you can always use a script instead of a phrase.
Going down the API stack
There is one promising API, which allows us to easily emulate physical keyboards: The uinput kernel module can create arbitrary virtual devices and has nice Python bindings (Github: https://github.com/tuomasjjrasanen/python-uinput).
Using uinput to type text
To use it, install python-uinput from PyPI
Their documentation is quite good, so I won’t re-phrase it. Just one thing: After device creation, X11 needs some time to react to the new device, so just using on-demand device creation will require massive sleeping, otherwise X11 will swallow the sent text. So it is advisable to create the device once and then re-use it during the whole run-time.
Setting up the keyboard
Use a script like this to setup the device once. I called it uinput_setup and that’s what the code using it will assume when calling it. It is executed once during autokey runtime and will create the device for you.
The script still lacks some features, esp. it can’t type whitespace characters. And typing using it has issues: It collides with physically held modifier keys (which can’t be prevented easily). That’s why rewriting the interface in AutoKey is kind of a hard task…
# uinput setup code

import string
import atexit
import inspect
import time
from typing import List, Tuple

import uinput

UInputKey = Tuple[int, int]


def setup():
    """Perform uinput device creation."""
    device = _generate_device()
    store.set_global_value("uinput_keyboard", device)
    store.set_global_value("uinput_send_keys", _generate_send_keys_function(device))
    # Device creation is performed. Now sleep a whole second to give X11 time to recognize
    # the new device and start listening on events. Without this, the first keystrokes done
    # using the new device will be lost.
    time.sleep(1)


def _generate_device() -> uinput.Device:
    """Create a uinput Device and attach a destruction hook to clean up on autokey exit."""
    supported_keys = _get_supported_device_keys()
    device = uinput.Device(supported_keys.values())
    atexit.register(device.destroy)  # Automatically call device destruction on autokey exit
    return device
    

def _get_supported_device_keys() -> List[UInputKey]:
    """Use module inspection to generate a list of device keys for usage."""
    supported_keys = {
        name: key for name, key in inspect.getmembers(uinput) 
        if name.startswith("KEY_") and len(name) == 5
    }  # Only contains ASCII letters and digits: [A-Z] and [0-9]
    supported_keys["KEY_LEFTSHIFT"] = uinput.KEY_LEFTSHIFT  # Support Shift
    # TODO: Add other modifier keys, keypad keys, other keys not labeled with [A-Z] or [0-9]
    return supported_keys


def _generate_send_keys_function(device: uinput.Device):
    supported_keys = _get_supported_device_keys()
    key_table = {name[4:]: key for name, key in supported_keys.items() if len(name) == 5}

    # Define an inner function. The device reference is used from the outer scope and acts
    # like a constant or built-in.
    def send_keys(key_string: str):
        """
        Use uinput to type the given key_string.
        This function currently does not support any special AutoKey syntax, like '<meta>+u' or '<code70>'.
        It supports typing simple text only.
        """
        for char in key_string:
            try:
                if char.isupper():
                    device.emit_combo((uinput.KEY_LEFTSHIFT, key_table[char]))
                else:
                    # Convert the lower case character to upper case for table lookup,
                    # because the keys in key_table are upper case letters
                    device.emit_click(key_table[char.upper()])
            except KeyError:  # Just ignore unknown keys
                pass

    return send_keys  # Return the function object


if store.get_global_value("uinput_keyboard") is None:
    setup()


Then use it like this:
# Type using uinput
import time

# Setup uinput
if store.get_global_value("uinput_keyboard") is None:
    engine.run_script("uinput_setup")

send_keys = store.get_global_value("uinput_send_keys")

# If this script was triggered using a hotkey, you are probably still physically holding down
# a modifier key, like <shift>, or <super>. This collides with the typed text and will cause
# autokey to output keyboard combinations instead of typed text. 
# In this example, if it is bound to <super>+u, it may output <super>+<shift>+t, <super>+e, <super>+s, <super>+t, etc…
# Because <super> is still pressed on your physical keyboard!
# So wait until you released the physical modifier keys.
time.sleep(0.5)

# Now we can type

send_keys("TestText123")


BTW: I tried this and it works in xterm.
And you need autokey version 0.95.3 to run it. Otherwise it will crash spectacularly on save or application exit, and maybe even destroy your user data (scripts or phrases)!
@josephj11 A construct like this revealed the bugs in the global store. I was poking around using uinput. I tried to pass around function objects like shown here, which crashed and actually killed a script on my system.
Duplicate of #132