Repository: qtile/qtile

Issue:
All hooks in config will be subscribed and fired twice.
### The issue:

~~This is probably due to configuration testing syntax step.~~

```python
'startup_complete': [<function xstartup_complete at 0x7f2005fc49d0>,
                      <function xstartup_complete at 0x7f2005fc5510>]}
```

Code to reproduce is simple:

in config:

```python
@hook.subscribe.startup_complete
def xstartup_complete():
    ...

logger.warn(pprint.pformat(hook.subscriptions))
```

All hooks are actually being fired twice, not only startup but all for each event.

### Required:

- [X] I have searched past issues to see if this bug has already been reported.


Discussion:
Are you reloading the config file or does this happen when you start qtile for the first time?
this affects only reload afaik

after doing some print-style-debugging:

("CONFIG READED" printed in config namespace)

cmd_restart:

(restart-load-to-check-config-reload-submods-for-config, execve, load_config-load-import. Everything is fine)

```
2022-12-14 13:21:18,312 WARNING libqtile manager.py:cmd_restart():L1121 XXX manager: cmd_restart
2022-12-14 13:21:18,312 WARNING libqtile confreader.py:load():L125 XXX configreader.load()
2022-12-14 13:21:18,312 WARNING libqtile confreader.py:load():L128 XXX configreader reload submods
2022-12-14 13:21:18,334 WARNING libqtile config.py:<module>():L874 XXX: CONFIG READED
2022-12-14 13:21:18,348 WARNING libqtile config.py:<module>():L874 XXX: CONFIG READED
2022-12-14 13:21:18,630 WARNING libqtile manager.py:load_config():L108 XXX manager load_config()
2022-12-14 13:21:18,631 WARNING libqtile confreader.py:load():L125 XXX configreader.load()
2022-12-14 13:21:18,631 WARNING libqtile confreader.py:load():L132 XXX configreader import_module
2022-12-14 13:21:18,655 WARNING libqtile config.py:<module>():L874 XXX: CONFIG READED
```

cmd_reload:

(reload-load-to-check-config-reload-submods-for-config, clear state (incuding hooks),  reload_submods << PROBLEM IS HERE)

```
2022-12-14 13:22:08,703 WARNING libqtile manager.py:cmd_reload_config():L261 XXX manager: cmd_reload_config
2022-12-14 13:22:08,703 WARNING libqtile confreader.py:load():L125 XXX configreader.load()
2022-12-14 13:22:08,703 WARNING libqtile confreader.py:load():L128 XXX configreader reload submods
2022-12-14 13:22:08,725 WARNING libqtile config.py:<module>():L874 XXX: CONFIG READED
2022-12-14 13:22:08,740 WARNING libqtile config.py:<module>():L874 XXX: CONFIG READED
2022-12-14 13:22:08,741 WARNING libqtile manager.py:cmd_reload_config():L273 XXX HOOKS CLEARED =======
2022-12-14 13:22:08,742 WARNING libqtile manager.py:load_config():L108 XXX manager load_config()
2022-12-14 13:22:08,742 WARNING libqtile confreader.py:load():L125 XXX configreader.load()
2022-12-14 13:22:08,742 WARNING libqtile confreader.py:load():L128 XXX configreader reload submods
2022-12-14 13:22:08,763 WARNING libqtile config.py:<module>():L874 XXX: CONFIG READED
2022-12-14 13:22:08,777 WARNING libqtile config.py:<module>():L874 XXX: CONFIG READED
```

Well. Problem is here:

(confreader.py)

```python
class Config:
    (...)
    def load(self):
        (...)
        if name in sys.modules:
            logger.warn('XXX configreader reload submods')
            self._reload_config_submodules(path)
            config = importlib.reload(sys.modules[name])
        (...)
```

`_reload_config_submodules` and `importlib.reload` both read config file.

also it looks like workaround could be to extract hooks logic from `config.py` to `config_hooks.py` and import it from `config.py` :-)
Dirty fixed in my setup by excluding config file itself in `_reload_config_submodules`:

```python
    def _reload_config_submodules(self, path: Path) -> None:
        (...)
        for module in sys.modules.copy().values():
            (...)
            if hasattr(module, "__file__") and module.__file__ is not None:
+               if module.__file__ == self.file_path:
+                   continue
            (...)
```

Result after same test as above during reload:

```
2022-12-14 13:28:55,638 WARNING libqtile manager.py:cmd_reload_config():L261 XXX manager: cmd_reload_config
2022-12-14 13:28:55,638 WARNING libqtile confreader.py:load():L127 XXX configreader.load()
2022-12-14 13:28:55,638 WARNING libqtile confreader.py:load():L130 XXX configreader reload submods
2022-12-14 13:28:55,661 WARNING libqtile config.py:<module>():L874 XXX: CONFIG READED
2022-12-14 13:28:55,663 WARNING libqtile manager.py:cmd_reload_config():L273 XXX HOOKS CLEARED =======
2022-12-14 13:28:55,663 WARNING libqtile manager.py:load_config():L108 XXX manager load_config()
2022-12-14 13:28:55,663 WARNING libqtile confreader.py:load():L127 XXX configreader.load()
2022-12-14 13:28:55,663 WARNING libqtile confreader.py:load():L130 XXX configreader reload submods
2022-12-14 13:28:55,686 WARNING libqtile config.py:<module>():L874 XXX: CONFIG READED
```
Thanks for that. As I recall, the config is actually loaded twice in `reload_config` in lines 279 and 295:
https://github.com/qtile/qtile/blob/fee9527b7f2faa4a19552288756d642ba1cef9bc/libqtile/core/manager.py#L269-L295

If we could mask the hook subscriptions in the first one, would that also work? 

EDIT: hmm, maybe not actually. We've got `hook.clear()` in there. I'll need to look at your solution more closely!!
> I'll need to look at your solution more closely!!

see 2nd from the end paragraph in here https://github.com/qtile/qtile/issues/4064#issuecomment-1350811783 =)

it does not happen in restart/clear start because there is nothing to reload in sys.modules
OK - do you want to submit this as a PR?
> OK - do you want to submit this as a PR?

are you OK with proposed fix?
If I understand correctly...
https://github.com/qtile/qtile/blob/fee9527b7f2faa4a19552288756d642ba1cef9bc/libqtile/confreader.py#L125-L126

The main config gets reloaded during the submodule reload in line 125 but then is also being reloaded in line 126. If that's correct then your solution is good as the _submodule_ reload shouldn't include the parent.
> if that's correct

yes, it is.

i'll submit PR in a few mins

but you will need to be careful in the future if you use `_reload_config_submodules` somewhere else  - `_reload_config_submodules` will do only the thing it says in its name: reload config submodules, but not config itself.
That's fine - it's only used there.

Resolving patch:
diff --git a/libqtile/confreader.py b/libqtile/confreader.py
--- a/libqtile/confreader.py
+++ b/libqtile/confreader.py
@@ -108,6 +108,14 @@ def _reload_config_submodules(self, path: Path) -> None:
             if hasattr(module, "__file__") and module.__file__ is not None:
                 subpath = Path(module.__file__)
 
+                if subpath == path:
+                    # do not reevaluate config itself here, we want only
+                    # reload all submodules. Also we cant reevaluate config
+                    # here, because it will cache all current modules before they
+                    # are reloaded. Thus, config file should be reloaded after
+                    # this routine.
+                    continue
+
                 # Check if the module is in the config folder or subfolder
                 # if so, reload it
                 if folder in subpath.parents:


Test patch:
diff --git a/test/configs/reloading.py b/test/configs/reloading.py
--- a/test/configs/reloading.py
+++ b/test/configs/reloading.py
@@ -88,4 +88,6 @@
     floating_layout = layout.Floating()
     wmname = "TEST"
 
+    qtile.test_data_config_evaluations += 1
+
 groups.append(ScratchPad("S", dropdowns))
diff --git a/test/test_manager.py b/test/test_manager.py
--- a/test/test_manager.py
+++ b/test/test_manager.py
@@ -1167,7 +1167,10 @@ def assert_dd_appeared():
 
     # Reload #1 - with libqtile.qtile.test_data
     manager_nospawn.c.eval("self.test_data = 1")
+    manager_nospawn.c.eval("self.test_data_config_evaluations = 0")
     manager_nospawn.c.reload_config()
+    # should be readed twice (check+read), but no more
+    assert manager_nospawn.c.eval("self.test_data_config_evaluations") == (True, "2")
     assert manager_nospawn.c.eval("len(self.keys_map)") == (True, "2")
     assert manager_nospawn.c.eval("len(self.mouse_map)") == (True, "2")
     assert "".join(manager_nospawn.c.get_groups().keys()) == "123456789S"
@@ -1192,6 +1195,7 @@ def assert_dd_appeared():
 
     # Reload #2 - back to without libqtile.qtile.test_data
     manager_nospawn.c.eval("del self.test_data")
+    manager_nospawn.c.eval("del self.test_data_config_evaluations")
     manager_nospawn.c.reload_config()
     assert manager_nospawn.c.eval("len(self.keys_map)") == (True, "1")
     assert manager_nospawn.c.eval("len(self.mouse_map)") == (True, "1")