Repository: qtile/qtile

Issue:
KeyboardLayout widget doesn't work as expected
# Issue description
In `libqtile.widget.keyboardlayout.KeyboardLayout` viewing the current layout is implemented through parsing `setxkbmap` (`setxkbmap -verbose 10 -query`) which is wrong (it doesn't give information about the current layout).
You can check with this example:
```
$ setxkbmap 'us,ru' -option 'grp:caps_toggle'
$ setxkbmap -verbose 10 -query > english # Current layout — English.
$ setxkbmap -verbose 10 -query > русский # Current layout — Russian.
$ diff -Naurp0 english русский # Without output, which means it doesn't change when you change the keyboard layout.
```

`setxkbmap` doesn't let you know the current layout.

Switching layouts also doesn't work as expected. Take a look at an example:
```
# `libqtile.widget.keyboardlayout.KeyboardLayout()`
$ setxkbmap 'us,ru' -option 'grp:caps_toggle,compose:menu' # My setup.
$ setxkbmap us # `libqtile.widget.keyboardlayout.KeyboardLayout` actions when clicking on it.
$ setxkbmap us # `libqtile.widget.keyboardlayout.KeyboardLayout` actions when clicking on it.
# My keyboard layout is now only 'us' I cannot change it back to 'ru' even through the widget, through Caps Lock.

# `libqtile.widget.keyboardlayout.KeyboardLayout(configured_keyboards=['us', 'ru'])`
$ setxkbmap 'us,ru' -option 'grp:caps_toggle,compose:menu' # My setup.
$ setxkbmap ru # `libqtile.widget.keyboardlayout.KeyboardLayout` actions when clicking on it.
$ setxkbmap us # `libqtile.widget.keyboardlayout.KeyboardLayout` actions when clicking on it.
# Thus, I cannot switch the keyboard layout via Caps Lock. Only through the widget even if I add an `option='grp:caps_toggle,compose:menu'` parameter.
```

Also, the keyboard layout doesn't change automatically in the widget if I change it through Caps Lock.

# Qtile version
Since [9bb78e80 ("Adding keyboard layout widget", 2013-08-17)](https://github.com/qtile/qtile/commit/9bb78e80).
Fix and remove unused code in KeyboardLayout widget
#2027 


Discussion:
Can you propose a patch to fix it? I'm not sure I understand all the nuances from your report.
> Can you propose a patch to fix it? I'm not sure I understand all the nuances from your report.

I can implement this using [xkb-switch](https://github.com/grwlf/xkb-switch) or something similar (the easiest way). I think the best option would be to use [python-xlib](https://github.com/python-xlib/python-xlib) but [it doesn't provide the XKB extension](https://github.com/python-xlib/python-xlib/issues/180). I think it's possible to use something like [xkbgroup](https://github.com/hcpl/xkbgroup) but no events are implemented there. I'm new to Python, so I'm not sure if I can implement the third option but I'll try (as a last resort I implement using [xkb-switch](https://github.com/grwlf/xkb-switch) or something similar).
We don't use xlib, we use xcb; xcffib needs some patches to provide that too. An xkb-switch implementation would be fine, given that we're already fork()ing for this widget.
I am using the `KeyboardLayout`-widget and it works for me, the output of `setxkbmap -verbose 10 -query` correctly returns the current layout. All the other commands you mentioned work fine, too. Which version of Qtile are you using?
> I am using the `KeyboardLayout`-widget and it works for me, the output of `setxkbmap -verbose 10 -query` correctly returns the current layout. All the other commands you mentioned work fine, too. Which version of Qtile are you using?

Qtile version: 3701.d6f8a6b2-1 (AUR, qtile-git). I don't think it depends on the Qtile version.
You have output in
```
$ setxkbmap 'us,ru' -option 'grp:caps_toggle'
$ setxkbmap -verbose 10 -query > a # Current layout — English.
$ setxkbmap -verbose 10 -query > ф # Current layout — Russian (to change it click on Caps Lock).
$ diff -Naurp0 a ф
```
?
I have not run the first line, since I have set up my available keymaps in Qtile. But everything else I did run, including things like `setxkbmap gb` to see if everything stays synchronized. And it did - the current layout returned by `setxkbmap -verbose 10 -query` as well as the widget always returned the proper setting.
> I have not run the first line, since I have set up my available keymaps in Qtile. But everything else I did run, including things like `setxkbmap gb` to see if everything stays synchronized. And it did - the current layout returned by `setxkbmap -verbose 10 -query` as well as the widget always returned the proper setting.

Do you have two (or more) keyboard layouts? If so, show the output of `setxkbmap -verbose 10 -query` for your two (or more) keyboard layouts, please.
Can you show the actual output form `setxkbmap -query -verbose 10` instead of a diff?
> Can you show the actual output form `setxkbmap -query -verbose 10` instead of a diff?

```
$ setxkbmap -query -verbose 10
Setting verbose level to 10
locale is C
Trying to load rules file ./rules/evdev...
Trying to load rules file /usr/share/X11/xkb/rules/evdev...
Success.
Applied rules from evdev:
rules:      evdev
model:      pc105
layout:     us,ru
options:    grp:caps_toggle,compose:menu
Trying to build keymap using the following components:
keycodes:   evdev+aliases(qwerty)
types:      complete
compat:     complete
symbols:    pc+us+ru:2+inet(evdev)+capslock(grouplock)+compose(menu)
geometry:   pc(pc105)
rules:      evdev
model:      pc105
layout:     us,ru
options:    grp:caps_toggle,compose:menu
```
On Mon, Dec 21, 2020 at 10:22:00AM -0800, k4leg wrote:
> > Can you show the actual output form `setxkbmap -query -verbose 10` instead of a diff?
> 
> ```
> $ setxkbmap -query -verbose 10
> Setting verbose level to 10
> locale is C
> Trying to load rules file ./rules/evdev...
> Trying to load rules file /usr/share/X11/xkb/rules/evdev...
> Success.
> Applied rules from evdev:
> rules:      evdev
> model:      pc105
> layout:     us,ru
> options:    grp:caps_toggle,compose:menu
> Trying to build keymap using the following components:
> keycodes:   evdev+aliases(qwerty)
> types:      complete
> compat:     complete
> symbols:    pc+us+ru:2+inet(evdev)+capslock(grouplock)+compose(menu)
> geometry:   pc(pc105)
> rules:      evdev
> model:      pc105
> layout:     us,ru
> options:    grp:caps_toggle,compose:menu
> ```

Looks to me like `layout: us,ru` is the problem; I think the widget
assumes there's only one thing rendered there.

> Looks to me like `layout: us,ru` is the problem; I think the widget assumes there's only one thing rendered there.

Yes you are right. Perhaps I did not  explain something well (I do not really know English). By the current keyboard layout I mean: "us", "ru" or "sk" or others (separately);  not "us,ru,sk" (for example). It would also be nice to implement this using events rather than refresh interval. `xkb-switch -W` exactly what is needed (it watches events and prints the current keyboard layout to stdout). But I don't quite understand how to combine this with switching the current keyboard layout (using `xkb-switch -n`).
I think in order to use anything besides some command line utility we'll need xkb support in xcffib; I don't remember exactly the details about why it fails to build right now, but I suspect it's nontrival.

I'd definitely takes a patch that ports the widget to use xkb-switch, since it seems like it doesn't really do anything useful right now.
@k4leg Why not just use two distinct layouts and add a Qtile key to switch between the two?
> @k4leg Why not just use two distinct layouts and add a Qtile key to switch between the two?

It will be too crutch. For me for example, switching the current keyboard layout occurs via Caps Lock which will add difficulties. And it will be implemented without watching events. The lag of the current keyboard layout from the displayed one will not look very nice, even if the refresh rate is low.


Resolving patch:
diff --git a/libqtile/widget/keyboardlayout.py b/libqtile/widget/keyboardlayout.py
--- a/libqtile/widget/keyboardlayout.py
+++ b/libqtile/widget/keyboardlayout.py
@@ -22,116 +22,269 @@
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 # SOFTWARE.
 
-import re
+import asyncio
 from subprocess import CalledProcessError
+from typing import NoReturn, Union
 
 from libqtile.log_utils import logger
 from libqtile.widget import base
 
-kb_layout_regex = re.compile(r'layout:\s+(?P<layout>\w+)')
-kb_variant_regex = re.compile(r'variant:\s+(?P<variant>\w+)')
 
+class NoDependencyFoundError(Exception):
+    pass
 
-class KeyboardLayout(base.InLoopPollText):
-    """Widget for changing and displaying the current keyboard layout
-
-    To use this widget effectively you need to specify keyboard layouts you want to use (using "configured_keyboards")
-    and bind function "next_keyboard" to specific keys in order to change layouts.
-
-    For example:
 
-        Key([mod], "space", lazy.widget["keyboardlayout"].next_keyboard(), desc="Next keyboard layout."),
+class KeyboardLayout(base.InLoopPollText):
+    """
+    Widget for changing and displaying the current keyboard layout group
 
-    It requires setxkbmap to be available in the system.
+    It requires xkb-switch to be available in the system.  setxkbmap is
+    optional dependency (to configure the keyboard locally).
     """
+
     orientations = base.ORIENTATION_HORIZONTAL
     defaults = [
-        ("update_interval", 1, "Update time in seconds."),
-        ("configured_keyboards", ["us"], "A list of predefined keyboard layouts "
-            "represented as strings. For example: "
-            "['us', 'us colemak', 'es', 'fr']."),
-        ("display_map", {}, "Custom display of layout. Key should be in format "
-            "'layout variant'. For example: "
-            "{'us': 'us ', 'lt sgs': 'sgs', 'ru phonetic': 'ru '}"),
-        ("option", None, "string of setxkbmap option. Ex., 'compose:menu,grp_led:scroll'"),
+        (
+            'compact',
+            None,
+            "Specifies the name of the compatibility map component used"
+            " to construct a keyboard layout.",
+        ),
+        (
+            'config_file',
+            None,
+            "Specifies the name of an XKB configuration file which"
+            " describes the keyboard to be used.",
+        ),
+        (
+            'device',
+            None,
+            "Specifies the numeric device id of the input device to be"
+            " updated with the new keyboard layout. If not specified,"
+            " the core keyboard device of the X server is updated.",
+        ),
+        (
+            'display',
+            None,
+            "Specifies the display to be updated with the new keyboard"
+            " layout.",
+        ),
+        (
+            'geometry',
+            None,
+            "Specifies the name of the geometry component used to"
+            " construct a keyboard layout.",
+        ),
+        (
+            'dirs',
+            None,
+            "Adds a directory to the list of directories to be used to"
+            " search for specified layout or rules files.",
+        ),
+        (
+            'keycodes',
+            None,
+            "Specifies the name of the keycodes component used to"
+            " construct a keyboard layout.",
+        ),
+        (
+            'keymap',
+            None,
+            "Specifies the name of the keymap description used to"
+            " construct a keyboard layout.",
+        ),
+        (
+            'layout_groups',
+            None,
+            "Specifies the name of the layout used to determine the"
+            " components which make up the keyboard description. For"
+            " example: ['us', 'ru', 'sk'], 'us,ru,sk'.",
+        ),
+        (
+            'model',
+            None,
+            "Specifies the name of the keyboard model used to determine"
+            " the components which make up the keyboard description."
+            " For example: 'pc101', 'pc105'.",
+        ),
+        (
+            'option',
+            None,
+            "Specifies the name of an option to determine the"
+            " components which make up the keyboard description. For"
+            " example: ['compose:menu', 'grp_led:scroll'],"
+            " 'compose:menu,grp_led:scroll'.",
+        ),
+        (
+            'rules',
+            None,
+            "Specifies the name of the rules file used to resolve the"
+            " requested layout and model to a set of component names.",
+        ),
+        (
+            'symbols',
+            None,
+            "Specifies the name of the symbols component used to"
+            " construct a keyboard layout.",
+        ),
+        (
+            'synch',
+            False,
+            "Force synchronization for X requests.",
+        ),
+        (
+            'types',
+            None,
+            "Specifies the name of the types component used to"
+            " construct a keyboard layout.",
+        ),
+        (
+            'variant',
+            None,
+            "Specifies which variant of the keyboard layout should be"
+            " used to determine the components which make up the"
+            " keyboard description.",
+        ),
     ]
 
-    def __init__(self, **config):
+    def __init__(self, **config) -> None:
+        # Instead of `base.InLoopPollText.__init__(self, **config)` to avoid
+        # adding 'update_interval' to default values.
         base.InLoopPollText.__init__(self, **config)
-        self.add_defaults(KeyboardLayout.defaults)
+        self.add_defaults(self.defaults)
+        self.raise_for_dependencies()
+        self.use_optional_dependencies_if_available()
 
-        self.keyboard = self.configured_keyboards[0]
+        asyncio.ensure_future(self.loop())
+        self.add_callbacks({'Button1': self.next_keyboard_layout_group})
 
-        self.add_callbacks({'Button1': self.next_keyboard})
-
-    def next_keyboard(self):
-        """Set the next layout in the list of configured keyboard layouts as
-        new current layout in use
-
-        If the current keyboard layout is not in the list, it will set as new
-        layout the first one in the list.
+    def raise_for_dependencies(self) -> None:
+        """
+        Raises NoDependencyFoundError if at least one dependency was not
+        found
         """
+        try:
+            self.call_process(['xkb-switch', '-v'])
+        except FileNotFoundError as e:
+            err_msg = "Please, check that xkb-switch is available (%s)"
+            logger.error(err_msg, e)
+            raise NoDependencyFoundError(err_msg % e) from None
 
-        current_keyboard = self.keyboard
-        if current_keyboard in self.configured_keyboards:
-            # iterate the list circularly
-            next_keyboard = self.configured_keyboards[
-                (self.configured_keyboards.index(current_keyboard) + 1) %
-                len(self.configured_keyboards)]
+    def use_optional_dependencies_if_available(self) -> None:
+        """
+        Checks for optional dependencies and uses them if any.  If they
+        aren't present and there is an attempt to use them then a
+        corresponding message is added to the logs.  If they aren't in
+        the system and there is no attempt to use them then nothing will
+        happen.
+        """
+        parameter_names = [i[0] for i in self.defaults]
+        parameter_values = [
+            getattr(self, name) for name in parameter_names
+        ]
+        parameters = {
+            name: value
+            for name, value in zip(parameter_names, parameter_values)
+            if value
+        }
+        try:
+            self.call_process(['setxkbmap', '-version'])
+        except FileNotFoundError:
+            if parameters:
+                logger.warning(
+                    "To use the %s parameter(s) you need setxkbmap to be"
+                    " available in the system.",
+                    ", ".join(parameters)
+                )
         else:
-            next_keyboard = self.configured_keyboards[0]
+            self._use_setxkbmap(parameters)
 
-        self.keyboard = next_keyboard
+    def _use_setxkbmap(self, parameters: dict) -> None:
+        self.dirs: Union[list, tuple, str]
+        self.layout_groups: Union[list, tuple, str]
+        exclude_parameters = ('config_file', 'dirs', 'layout_groups', 'synch')
+        for name, value in parameters.items():
+            if name not in exclude_parameters:
+                if isinstance(value, (list, tuple)):
+                    setattr(self, name, ','.join(value))
+                    value = getattr(self, name)
+                try:
+                    self.call_process(['setxkbmap', f'-{name}', value])
+                except CalledProcessError as e:
+                    logger.error(
+                        "Invalid arguments passed for %s (%s)", name, e
+                    )
+        if self.config_file:
+            try:
+                self.call_process(['setxkbmap', '-config', self.config_file])
+            except CalledProcessError as e:
+                logger.error(
+                    "Invalid arguments passed for config_file (%s)", e
+                )
+        if self.dirs:
+            if isinstance(self.dirs, (list, tuple)):
+                self.dirs = ' -I '.join(self.dirs)
+            if isinstance(self.dirs, str):
+                try:
+                    self.call_process(['setxkbmap', '-I ' + self.dirs])
+                except CalledProcessError as e:
+                    logger.error("Invalid arguments passed for dirs (%s)", e)
+            else:
+                logger.error(
+                    "dirs argument must be list, tuple or string, not %s",
+                    type(self.dirs),
+                )
+        if self.layout_groups:
+            if isinstance(self.layout_groups, (list, tuple)):
+                self.layout_groups = ','.join(self.layout_groups)
+            try:
+                self.call_process(['setxkbmap', '-layout', self.layout_groups])
+            except CalledProcessError as e:
+                logger.error(
+                    "Invalid arguments passed for layout_groups (%s)", e
+                )
+        if self.synch:
+            self.call_process(['setxkbmap', '-synch'])
 
+    async def loop(self) -> NoReturn:
+        """
+        An endless loop that updates the text when the group of the
+        current layout is updated
+        """
         self.tick()
-
-    def poll(self):
-        if self.keyboard in self.display_map.keys():
-            return self.display_map[self.keyboard]
-        return self.keyboard.upper()
-
-    def get_keyboard_layout(self, setxkbmap_output):
-        match_layout = kb_layout_regex.search(setxkbmap_output)
-        match_variant = kb_variant_regex.search(setxkbmap_output)
-
-        if match_layout is None:
-            return 'ERR'
-
-        kb = match_layout.group('layout')
-        if match_variant:
-            kb += " " + match_variant.group('variant')
-        return kb
+        while True:
+            event = await asyncio.create_subprocess_exec('xkb-switch', '-w')
+            await event.wait()
+            self.tick()
 
     @property
-    def keyboard(self):
-        """Return the currently used keyboard layout as a string
+    def current_keyboard_layout_group(self) -> str:
+        """Return the current keyboard layout group
 
-        Examples: "us", "us dvorak".  In case of error returns "unknown".
+        Examples: 'us', 'ru', 'sk'.  In case of error returns 'unknown'.
         """
         try:
-            command = 'setxkbmap -verbose 10 -query'
-            setxkbmap_output = self.call_process(command.split(' '))
-            keyboard = self.get_keyboard_layout(setxkbmap_output)
-            return str(keyboard)
+            return self.call_process(['xkb-switch', '-p']).rstrip()
         except CalledProcessError as e:
-            logger.error('Can not get the keyboard layout: {0}'.format(e))
-        except OSError as e:
-            logger.error('Please, check that xset is available: {0}'.format(e))
-        return "unknown"
-
-    @keyboard.setter
-    def keyboard(self, keyboard):
-        command = ['setxkbmap']
-        command.extend(keyboard.split(" "))
-        if self.option:
-            command.extend(['-option', self.option])
+            logger.error("Can't get the keyboard layout (%s)", e)
+        return 'unknown'
+
+    def next_keyboard_layout_group(self) -> None:
+        """Switch to the next keyboard layout group"""
         try:
-            self.call_process(command)
+            self.call_process(['xkb-switch', '-n'])
         except CalledProcessError as e:
-            logger.error('Can not change the keyboard layout: {0}'.format(e))
-        except OSError as e:
-            logger.error('Please, check that setxkbmap is available: {0}'.format(e))
+            logger.error("Can't set the keyboard layout (%s)", e)
+
+    def poll(self) -> str:
+        """Return the current keyboard layout group"""
+        return self.current_keyboard_layout_group
+
+    def timer_setup(self):
+        # Do not use interval update `base.InLoopPollText.timer_setup`, events
+        # are used instead.
+        pass
 
-    def cmd_next_keyboard(self):
-        """Select next keyboard layout"""
-        self.next_keyboard()
+    def button_press(self, x, y, button):
+        # To avoid unnecessary calling `self.tick`.
+        return base.InLoopPollText.button_press(self, x, y, button)


Test patch:
diff --git a/test/widgets/test_widget_init_configure.py b/test/widgets/test_widget_init_configure.py
--- a/test/widgets/test_widget_init_configure.py
+++ b/test/widgets/test_widget_init_configure.py
@@ -35,7 +35,8 @@
 # To skip a test entirely, list the widget class here
 no_test = [
     widgets.Mirror,  # Mirror requires a reflection object
-    widgets.PulseVolume
+    widgets.PulseVolume,
+    widgets.KeyboardLayout,  # requires xkb-switch to be installed
 ]
 
 ################################################################################