Repository: Ulauncher/Ulauncher
Pull request #1461: feature: #273 automatically install extensions
Author: login: gornostal
Created: 2025-05-01T18:05:12Z | Merged: 2025-05-08T16:10:45Z

#273 
Implements automatic dependency installation for extensions.

Uses `pip install ... --target /path` to download dependencies. Then, it passes the deps dir in `PYTHONPATH` when running extensions.


Diff:
diff --git a/docs/extensions/tutorial.rst b/docs/extensions/tutorial.rst
index 14228cba1..f5d5d8313 100644
--- a/docs/extensions/tutorial.rst
+++ b/docs/extensions/tutorial.rst
@@ -218,7 +218,7 @@ ExtensionCustomAction
 
 ActionList
   List of actions
-  
+
 
 Custom Action on Item Enter
 ---------------------------
@@ -265,6 +265,21 @@ Custom Action on Item Enter
   Now this will be rendered when you click on any item
 
 
+Dependencies and requirements.txt file
+--------------------------------------
+
+If your extension requires any python dependencies, you can specify them in a ``requirements.txt`` file.
+Ulauncher will install them automatically when the extension is installed.
+
+Avoid specifying development dependencies such as mypy, pytest, etc. in the ``requirements.txt`` file — users don't need them.
+Instead, you can create a separate ``requirements-dev.txt`` file for your development dependencies.
+
+It's a good idea to specify the version of the dependencies you are using to avoid breaking changes in the future.
+For example, if your extension requires the `requests` library, you can create a `requirements.txt` file with the following content::
+
+  requests==2.32.3
+
+Refer to this `requirements.txt documentation <https://pip.pypa.io/en/stable/reference/requirements-file-format/>`_ for more information.
 
 .. NOTE::
   Please take `a short survey <https://goo.gl/forms/wcIRCTjQXnO0M8Lw2>`_ to help us build greater API and documentation
diff --git a/makefile b/makefile
index ee5df5268..ca540deaa 100644
--- a/makefile
+++ b/makefile
@@ -138,7 +138,7 @@ format: # Auto format the code
 
 .PHONY: prefs docker docs sdist manpage deb nix-run nix-build-dev nix-build
 
-docs: ## Build the API docs
+docs: # Build the API docs
 	@set -euo pipefail
 	cd docs
 	sphinx-build -M html . ./_build
diff --git a/nix/default.nix b/nix/default.nix
index da4726a20..1ff16c4e4 100644
--- a/nix/default.nix
+++ b/nix/default.nix
@@ -119,7 +119,7 @@ let
 
       substituteInPlace \
           ulauncher/modes/extensions/extension_controller.py \
-        --replace-fail '"PYTHONPATH": paths.APPLICATION,' '"PYTHONPATH": ":".join(sys.path),'
+        --replace-fail 'paths.APPLICATION,' '":".join(sys.path),'
 
       substituteInPlace \
           ulauncher.service \
diff --git a/preferences-src/src/components/widgets/ExtensionInstallError.vue b/preferences-src/src/components/widgets/ExtensionInstallError.vue
index 7141a122d..83ae66176 100644
--- a/preferences-src/src/components/widgets/ExtensionInstallError.vue
+++ b/preferences-src/src/components/widgets/ExtensionInstallError.vue
@@ -26,12 +26,19 @@
         >
           A network error occurred: <b>{{ errorMessage }}</b>
           <br><br>Please check that your network is ok, that the repository is not private, and that the extension has all the required files.
-          <br><br>You can also install extensions manually by adding them to 
+          <br><br>You can also install extensions manually by adding them to
           <a
             href
             @click.prevent="openExtensionsDir()"
           >your extension directory</a>.
         </p>
+        <p v-else-if="errorType === 'ExtensionDependenciesRecoverableError'">
+          Failed to install extension dependencies:
+          <br />
+          <code style="white-space: pre-line;">{{ errorMessage }}</code>
+          <br /><br />If nothing seems clearly wrong on your end, consider
+          <a href @click.prevent="openUrlInBrowser(`${extUrl}/issues`)">contacting</a> the extension author and let them know about this problem.
+        </p>
         <p v-else>
           An unexpected error occurred.
           <br>Please copy the technical details and report this problem via
diff --git a/tests/modes/apps/extensions/test_extension_dependencies.py b/tests/modes/apps/extensions/test_extension_dependencies.py
new file mode 100644
index 000000000..4d18ab58e
--- /dev/null
+++ b/tests/modes/apps/extensions/test_extension_dependencies.py
@@ -0,0 +1,46 @@
+import sys
+from unittest.mock import MagicMock, mock_open, patch
+
+import pytest
+
+from ulauncher.modes.extensions.extension_dependencies import ExtensionDependencies
+
+
+@pytest.fixture
+def extension_dependencies() -> ExtensionDependencies:
+    return ExtensionDependencies(ext_id="extension-X", path="/fake/path")
+
+
+@patch("ulauncher.modes.extensions.extension_dependencies.isfile", return_value=True)
+@patch("ulauncher.modes.extensions.extension_dependencies.isdir", return_value=True)
+def test_get_dependencies_path(
+    mock_isdir: MagicMock, mock_isfile: MagicMock, extension_dependencies: ExtensionDependencies  # noqa: ARG001
+) -> None:
+    deps_path = extension_dependencies.get_dependencies_path()
+    assert deps_path == "/fake/path/.dependencies"
+
+
+@patch("ulauncher.modes.extensions.extension_dependencies.isfile", return_value=True)
+@patch("builtins.open", new_callable=mock_open, read_data="some-package==1.0\n")
+@patch("subprocess.run")
+def test_install(
+    mock_subprocess: MagicMock,
+    mock_open_fn: MagicMock,  # noqa: ARG001
+    mock_isfile: MagicMock,  # noqa: ARG001
+    extension_dependencies: ExtensionDependencies,
+) -> None:
+    mock_subprocess.return_value = MagicMock(stdout="Installation successful", returncode=0)
+
+    extension_dependencies.install()
+
+    expected_command = [
+        sys.executable,
+        "-m",
+        "pip",
+        "install",
+        "-r",
+        "/fake/path/requirements.txt",
+        "--target",
+        "/fake/path/.dependencies",
+    ]
+    mock_subprocess.assert_called_once_with(expected_command, check=True, capture_output=True, text=True)
diff --git a/ulauncher/modes/extensions/extension_controller.py b/ulauncher/modes/extensions/extension_controller.py
index 5befcc4e7..09602acc6 100644
--- a/ulauncher/modes/extensions/extension_controller.py
+++ b/ulauncher/modes/extensions/extension_controller.py
@@ -4,14 +4,19 @@
 import json
 import logging
 import sys
+import tempfile
 from datetime import datetime
 from pathlib import Path
-from shutil import rmtree
+from shutil import copytree, rmtree
 from typing import Any, Iterator
 from weakref import WeakValueDictionary
 
 from ulauncher.config import get_options, paths
 from ulauncher.modes.extensions import extension_finder
+from ulauncher.modes.extensions.extension_dependencies import (
+    ExtensionDependencies,
+    ExtensionDependenciesRecoverableError,
+)
 from ulauncher.modes.extensions.extension_manifest import (
     ExtensionIncompatibleRecoverableError,
     ExtensionManifest,
@@ -50,10 +55,6 @@ def __setitem__(self, key: str, value: Any) -> None:  # type: ignore[override]
 extension_runtimes: dict[str, ExtensionRuntime] = {}
 
 
-class ExtensionControllerError(Exception):
-    pass
-
-
 class ExtensionController:
     id: str
     state: ExtensionState
@@ -114,6 +115,10 @@ def get_from_keyword(cls, keyword: str) -> ExtensionController | None:
     def manifest(self) -> ExtensionManifest:
         return ExtensionManifest.load(self.path)
 
+    @property
+    def deps(self) -> ExtensionDependencies:
+        return ExtensionDependencies(self.id, self.path)
+
     @property
     def path(self) -> str:
         if not self._path:
@@ -147,8 +152,17 @@ def save_user_preferences(self, data: Any) -> None:
     def get_normalized_icon_path(self, icon: str | None = None) -> str | None:
         return get_icon_path(icon or self.manifest.icon, base_path=self.path)
 
-    async def download(self, commit_hash: str | None = None, warn_if_overwrite: bool = True) -> None:
+    async def install(self, commit_hash: str | None = None, warn_if_overwrite: bool = True) -> None:
         commit_hash, commit_timestamp = self.remote.download(commit_hash, warn_if_overwrite)
+
+        # install python dependencies from requirements.txt
+        # or remove the downloaded extension files to avoid broken state
+        try:
+            self.deps.install()
+        except ExtensionDependenciesRecoverableError:
+            await self.remove()
+            raise
+
         self.state.save(
             commit_hash=commit_hash,
             commit_time=datetime.fromtimestamp(commit_timestamp).isoformat(),
@@ -188,8 +202,21 @@ async def update(self) -> bool:
             commit_hash[:8],
         )
 
-        await self.stop()
-        await self.download(commit_hash, warn_if_overwrite=False)
+        # Backup extension files. If update fails, restore from backup
+        with tempfile.TemporaryDirectory(prefix="ulauncher_ext_") as backup_dir:
+            # use local variable, because self.path property will call locate() on a non-existing path
+            ext_path = self.path
+            copytree(ext_path, backup_dir, dirs_exist_ok=True)
+
+            try:
+                await self.stop()
+                await self.install(commit_hash, warn_if_overwrite=False)
+            except Exception:
+                logger.exception("Failed to update extension. Restoring from backup.")
+                copytree(backup_dir, ext_path, dirs_exist_ok=True)
+                await self.toggle_enabled(was_running)
+                raise
+
         return await self.toggle_enabled(was_running)
 
     async def check_update(self) -> tuple[bool, str]:
@@ -228,6 +255,7 @@ def error_handler(error_type: str, error_msg: str) -> None:
 
             self.state.save(error_type="", error_message="")  # clear any previous error
 
+            ext_deps = ExtensionDependencies(self.id, self.path)
             cmd = [sys.executable, f"{self.path}/main.py"]
             prefs = {id: pref.value for id, pref in self.user_preferences.items()}
             triggers = {id: t.keyword for id, t in self.manifest.triggers.items() if t.keyword}
@@ -235,7 +263,7 @@ def error_handler(error_type: str, error_msg: str) -> None:
             v2_prefs = {**triggers, **prefs}
             env = {
                 "VERBOSE": str(int(verbose_logging)),
-                "PYTHONPATH": paths.APPLICATION,
+                "PYTHONPATH": ":".join(x for x in [paths.APPLICATION, ext_deps.get_dependencies_path()] if x),
                 "EXTENSION_PREFERENCES": json.dumps(v2_prefs, separators=(",", ":")),
             }
 
diff --git a/ulauncher/modes/extensions/extension_dependencies.py b/ulauncher/modes/extensions/extension_dependencies.py
new file mode 100644
index 000000000..d0220cef5
--- /dev/null
+++ b/ulauncher/modes/extensions/extension_dependencies.py
@@ -0,0 +1,81 @@
+from __future__ import annotations
+
+import logging
+import subprocess
+import sys
+from os.path import isdir, isfile
+
+logger = logging.getLogger()
+
+
+class ExtensionDependenciesRecoverableError(Exception):
+    pass
+
+
+class ExtensionDependencies:
+    """
+    Manages python dependencies of the extension.
+    """
+
+    ext_id: str
+    path: str
+
+    deps_out_subdir: str = ".dependencies"
+
+    def __init__(self, ext_id: str, path: str):
+        self.ext_id = ext_id
+        self.path = path
+
+    def get_dependencies_path(self) -> str | None:
+        """
+        Get the path to the dependencies directory.
+        Returns None if there's no dependencies for the extension.
+        """
+        requirements_file = f"{self.path}/requirements.txt"
+        deps_path = f"{self.path}/{self.deps_out_subdir}"
+        if not isfile(requirements_file) or not isdir(deps_path):
+            return None
+
+        return deps_path
+
+    def install(self) -> None:
+        """
+        Install the dependencies for the extension.
+        Runs `pip install -r extension-X/requirements.txt --target extension-X/deps`
+        """
+        requirements = self._read_requirements()
+
+        if not requirements:
+            return
+
+        command = [
+            sys.executable,
+            "-m",
+            "pip",
+            "install",
+            "-r",
+            f"{self.path}/requirements.txt",
+            "--target",
+            f"{self.path}/{self.deps_out_subdir}",
+        ]
+        command_str = " ".join(command)
+        logger.info("Installing dependencies for %s. Running: %s", self.ext_id, command_str)
+
+        try:
+            result = subprocess.run(command, check=True, capture_output=True, text=True)
+            logger.info("Installation successful. Output: %s", result.stdout)
+        except subprocess.CalledProcessError as e:
+            logger.exception("Error during installation. Output: %s", e.stderr)
+
+            raise ExtensionDependenciesRecoverableError(f"$ {command_str}\n{e.stderr}") from e  # noqa: TRY003, EM102
+
+    def _read_requirements(self) -> str | None:
+        """
+        Read the requirements file and return its content.
+        """
+        requirements_file = f"{self.path}/requirements.txt"
+        if not isfile(requirements_file):
+            return None
+
+        with open(requirements_file) as f:
+            return f.read().strip()
diff --git a/ulauncher/modes/extensions/extension_remote.py b/ulauncher/modes/extensions/extension_remote.py
index c937a94c6..2a7c3c86e 100644
--- a/ulauncher/modes/extensions/extension_remote.py
+++ b/ulauncher/modes/extensions/extension_remote.py
@@ -66,6 +66,7 @@ def __init__(self, url: str):
             self.url = f"{self.protocol}://{self.host}/{self.path}"
 
         except Exception as e:
+            logger.warning("Invalid URL: %s (%s: %s)", url, type(e).__name__, e)
             msg = f"Invalid URL: {url}"
             raise InvalidExtensionRecoverableError(msg) from e
 
@@ -86,6 +87,7 @@ def _get_download_url(self, commit: str) -> str:
 
     def _get_refs(self) -> dict[str, str]:
         refs = {}
+        ref = None
         url = f"{self.url}.git" if self.host in ("github.com", "gitlab.com", "codeberg.org") else self.url
         try:
             if self._use_git:
diff --git a/ulauncher/ui/preferences_server.py b/ulauncher/ui/preferences_server.py
index fe49271f2..dc51583df 100644
--- a/ulauncher/ui/preferences_server.py
+++ b/ulauncher/ui/preferences_server.py
@@ -253,7 +253,7 @@ async def extension_get_all(self, reload: bool) -> dict[str, dict[str, Any]]:
     async def extension_add(self, url: str) -> dict[str, Any]:
         logger.info("Add extension: %s", url)
         controller = ExtensionController.create_from_url(url)
-        await controller.download()
+        await controller.install()
         await controller.stop()
         await controller.start()
         return get_extension_data(controller)


Reviews:
user: login: gornostal
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-01T18:09:26Z
user: login: friday
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-01T20:37:27Z
user: login: friday
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-01T20:40:02Z
user: login: friday
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-01T20:43:09Z
user: login: gornostal
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-03T06:35:27Z
user: login: friday
body: I have some small comments/concerns only. Looks great otherwise, and nice work on the documentation as well.

I didn't actually test it yet.

We're not using venvs here, only pip to a `.dependencies` path that we later add to the extension path. I think this is more disk-space friendly approach than #721 (issue created by the virtualenv author btw), and from what I can see it should work as well for us (right @troycurtisjr?).

I'm also considering how to handle upgrading extensions. I think the current logic would delete the dependencies subdir when you upgrade the extension and then reinstall them after, but if you're on the newest version of the extension, then updating won't update the dependencies? This is a reasonable for the PR, but I do want to add some extension update CLI command.

We should consider the `--upgrade` and `--ignore-installed` flag. I would prefer not using (to enable reusing system dependencies), but I'm not sure it's that can lead to similar issues as I had with my+gobject stubs. And if we don't pass `--upgrade`, then the version installed would not upgrade of course. But if we do it will likely also override system packages.
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-04T22:54:44Z
user: login: troycurtisjr
body: Left a few initial comments, I'll also try running it locally as well tomorrow night.
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-05T01:36:09Z
user: login: nazarewk
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-05T07:16:56Z
user: login: nazarewk
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-05T07:21:53Z
user: login: gornostal
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-05T16:54:50Z
user: login: gornostal
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-05T16:55:14Z
user: login: gornostal
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-05T16:59:13Z
user: login: gornostal
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-05T17:02:13Z
user: login: friday
body: I tested with [the first extension I could find that already had requirements.txt](https://github.com/tuanpham-dev/ulauncher-better-calculator) and it worked. This extension is just a wrapper around py_expression_eval, so if it runs at all and can do computations it works.

The only concern I have now is nix.

Side note: This extension does absolutely nothing our built in calculator doesn't do, so the name is questionable...
state: APPROVED
author_association: MEMBER
submitted_at: 2025-05-05T22:10:28Z
user: login: troycurtisjr
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-06T02:04:51Z
user: login: troycurtisjr
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-06T02:15:42Z
user: login: gornostal
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-06T07:33:53Z
user: login: troycurtisjr
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-08T01:29:55Z
user: login: troycurtisjr
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-08T01:35:22Z
user: login: troycurtisjr
state: APPROVED
author_association: MEMBER
submitted_at: 2025-05-08T01:42:09Z
user: login: gornostal
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-08T16:08:33Z

Review comments:
diff_hunk: @@ -119,7 +119,7 @@ let
 
       substituteInPlace \
           ulauncher/modes/extensions/extension_controller.py \
-        --replace-fail '"PYTHONPATH": paths.APPLICATION,' '"PYTHONPATH": ":".join(sys.path),'
+        --replace-fail 'paths.APPLICATION,' '":".join(sys.path),'
path: nix/default.nix
user: login: gornostal
body: I haven't used nix before.
As I understand it helps setting up a local dev environment? Curious, what was the main challenge that nix solves for you?
created_at: 2025-05-01T18:09:26Z
author_association: MEMBER
diff_hunk: @@ -119,7 +119,7 @@ let
 
       substituteInPlace \
           ulauncher/modes/extensions/extension_controller.py \
-        --replace-fail '"PYTHONPATH": paths.APPLICATION,' '"PYTHONPATH": ":".join(sys.path),'
+        --replace-fail 'paths.APPLICATION,' '":".join(sys.path),'
path: nix/default.nix
user: login: friday
body: It's a package manager: https://www.youtube.com/watch?v=FJVFXsNzYZQ. I don't use it. It was contributed by @nazarewk.
created_at: 2025-05-01T20:37:26Z
author_association: MEMBER
diff_hunk: @@ -119,7 +119,7 @@ let
 
       substituteInPlace \
           ulauncher/modes/extensions/extension_controller.py \
-        --replace-fail '"PYTHONPATH": paths.APPLICATION,' '"PYTHONPATH": ":".join(sys.path),'
+        --replace-fail 'paths.APPLICATION,' '":".join(sys.path),'
path: nix/default.nix
user: login: friday
body: I don't know if this change is the correct thing to do, but if the CI build thinks it is, that's a good sign. If I don't know how to proceed I ask nazarewk or someone else for help (not necessarily before merging the breaking change).
created_at: 2025-05-01T20:40:01Z
author_association: MEMBER
diff_hunk: @@ -86,6 +87,7 @@ def _get_download_url(self, commit: str) -> str:
 
     def _get_refs(self) -> dict[str, str]:
         refs = {}
+        ref = None
path: ulauncher/modes/extensions/extension_remote.py
user: login: friday
body: This looks unused?
created_at: 2025-05-01T20:43:08Z
author_association: MEMBER
diff_hunk: @@ -86,6 +87,7 @@ def _get_download_url(self, commit: str) -> str:
 
     def _get_refs(self) -> dict[str, str]:
         refs = {}
+        ref = None
path: ulauncher/modes/extensions/extension_remote.py
user: login: gornostal
body: This is to fix the "could be unassigned variable" linter error
created_at: 2025-05-03T06:35:27Z
author_association: MEMBER
diff_hunk: @@ -3,7 +3,7 @@ root = true
 [*]
 charset = utf-8
 indent_style = space
-indent_size = 2
+indent_size = 4
path: .editorconfig
user: login: friday
body: `indent_size = 4` is already the case for .py (line 12), while all other languages (.html, .js, .vue) and config files (.toml, .yml, .nix) uses 2. So I think it's better to keep .py as the exception?
created_at: 2025-05-04T20:18:52Z
author_association: MEMBER
diff_hunk: @@ -265,6 +265,21 @@ Custom Action on Item Enter
   Now this will be rendered when you click on any item
 
 
+Dependencies and requirements.txt file
+--------------------------------------
+
+If your extension requires any python dependencies, you can specify them in a ``requirements.txt`` file.
+Ulauncher will install them automatically when the extension is installed.
+
+Avoid specifying development dependencies such as mypy, pytest, etc. in the ``requirements.txt`` file — users don't need them.
+Instead, you can create a separate ``requirements-dev.txt`` file for your development dependencies.
+
+It's a good idea to specify the version of the dependencies you are using to avoid breaking changes in the future.
+For example, if your extension requires the `requests` library, you can create a `requirements.txt` file with the following content::
+
+  requests==2.32.3
path: docs/extensions/tutorial.rst
user: login: friday
body: This of course depends. On one hand the deps would be documented and installed with the specific versions the extensions are known to work with, which is a huge benefit. But they would get outdated more easily, and if the user has a global/system  dependency installed to their system, it would be unlikely to be used. I would say I'm in favor of this recommendation though.
created_at: 2025-05-04T21:15:59Z
author_association: MEMBER
diff_hunk: @@ -86,6 +87,7 @@ def _get_download_url(self, commit: str) -> str:
 
     def _get_refs(self) -> dict[str, str]:
         refs = {}
+        ref = None
path: ulauncher/modes/extensions/extension_remote.py
user: login: friday
body: I see. I didn't see that I used this variable. 

I'm not getting this error though. Is it from pylint? I removed pylint because it was very slow compared to ruff. There are some good stuff pylint has but ruff is missing, but I think too painful to run and also to maintain two sets of rules.
created_at: 2025-05-04T21:47:32Z
author_association: MEMBER
diff_hunk: @@ -119,7 +119,7 @@ let
 
       substituteInPlace \
           ulauncher/modes/extensions/extension_controller.py \
-        --replace-fail '"PYTHONPATH": paths.APPLICATION,' '"PYTHONPATH": ":".join(sys.path),'
+        --replace-fail 'paths.APPLICATION,' '":".join(sys.path),'
path: nix/default.nix
user: login: friday
body: I definitely remember this PYTHONPATH patch was something we talked about when merging it originally. I would want confirmation from some nix user about this.
created_at: 2025-05-04T22:35:37Z
author_association: MEMBER
diff_hunk: @@ -265,6 +265,21 @@ Custom Action on Item Enter
   Now this will be rendered when you click on any item
 
 
+Dependencies and requirements.txt file
+--------------------------------------
+
+If your extension requires any python dependencies, you can specify them in a ``requirements.txt`` file.
+Ulauncher will install them automatically when the extension is installed.
+
+Avoid specifying development dependencies such as mypy, pytest, etc. in the ``requirements.txt`` file — users don't need them.
+Instead, you can create a separate ``requirements-dev.txt`` file for your development dependencies.
+
+It's a good idea to specify the version of the dependencies you are using to avoid breaking changes in the future.
+For example, if your extension requires the `requests` library, you can create a `requirements.txt` file with the following content::
+
+  requests==2.32.3
path: docs/extensions/tutorial.rst
user: login: troycurtisjr
body: I also wonder if whether we should use a custom name in the repo instead of co-opting `requirements.txt`. Generally `requirements.txt` is really used for repeatable builds and not generally runtime requirements. Since there is an established purpose for this file which is just tangentially related to what ulauncher needs from it, it seems best not to reuse it.
created_at: 2025-05-05T01:12:28Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,81 @@
+from __future__ import annotations
+
+import logging
+import subprocess
+import sys
+from os.path import isdir, isfile
+
+logger = logging.getLogger()
+
+
+class ExtensionDependenciesRecoverableError(Exception):
+    pass
+
+
+class ExtensionDependencies:
+    """
+    Manages python dependencies of the extension.
+    """
+
+    ext_id: str
+    path: str
+
+    deps_out_subdir: str = ".dependencies"
+
+    def __init__(self, ext_id: str, path: str):
+        self.ext_id = ext_id
+        self.path = path
+
+    def get_dependencies_path(self) -> str | None:
+        """
+        Get the path to the dependencies directory.
+        Returns None if there's no dependencies for the extension.
+        """
+        requirements_file = f"{self.path}/requirements.txt"
+        deps_path = f"{self.path}/{self.deps_out_subdir}"
+        if not isfile(requirements_file) or not isdir(deps_path):
+            return None
+
+        return deps_path
+
+    def install(self) -> None:
+        """
+        Install the dependencies for the extension.
+        Runs `pip install -r extension-X/requirements.txt --target extension-X/deps`
+        """
+        requirements = self._read_requirements()
+
+        if not requirements:
+            return
+
+        command = [
+            sys.executable,
+            "-m",
+            "pip",
+            "install",
+            "-r",
+            f"{self.path}/requirements.txt",
+            "--target",
+            f"{self.path}/{self.deps_out_subdir}",
+        ]
+        command_str = " ".join(command)
+        logger.info("Installing dependencies for %s. Running: %s", self.ext_id, command_str)
+
+        try:
+            result = subprocess.run(command, check=True, capture_output=True, text=True)
+            logger.info("Installation successful. Output: %s", result.stdout)
+        except subprocess.CalledProcessError as e:
+            logger.exception("Error during installation. Output: %s", e.stderr)
+
+            raise ExtensionDependenciesRecoverableError(f"$ {command_str}\n{e.stderr}") from e  # noqa: TRY003, EM102
+
+    def _read_requirements(self) -> str | None:
path: ulauncher/modes/extensions/extension_dependencies.py
user: login: troycurtisjr
body: I suspect we'll want some protections here. The main one I can think of is an installation of a different ulauncher version than is currently being run. 
created_at: 2025-05-05T01:33:11Z
author_association: MEMBER
diff_hunk: @@ -119,7 +119,7 @@ let
 
       substituteInPlace \
           ulauncher/modes/extensions/extension_controller.py \
-        --replace-fail '"PYTHONPATH": paths.APPLICATION,' '"PYTHONPATH": ":".join(sys.path),'
+        --replace-fail 'paths.APPLICATION,' '":".join(sys.path),'
path: nix/default.nix
user: login: nazarewk
body: I don't remember the exact details, but `paths.APPLICATIONS` assumes Ulauncher lives inside a single directory (probably `~/.local/share`).

In Nix it's not a single directory, but a combination of **many** `/nix/store/` entries joined through PYTHONPATH. As far as I remember, there was nothing there except for `ulauncher` module without this patch.
created_at: 2025-05-05T07:16:49Z
author_association: CONTRIBUTOR
diff_hunk: @@ -119,7 +119,7 @@ let
 
       substituteInPlace \
           ulauncher/modes/extensions/extension_controller.py \
-        --replace-fail '"PYTHONPATH": paths.APPLICATION,' '"PYTHONPATH": ":".join(sys.path),'
+        --replace-fail 'paths.APPLICATION,' '":".join(sys.path),'
path: nix/default.nix
user: login: nazarewk
body: Again I don't remember the details apart from the non-Nix way having a good reason to be there, but this seems to be the whole discussion from when I wanted to "upstream" the change instead of using a patch: https://github.com/Ulauncher/Ulauncher/pull/1306 
created_at: 2025-05-05T07:21:53Z
author_association: CONTRIBUTOR
diff_hunk: @@ -86,6 +87,7 @@ def _get_download_url(self, commit: str) -> str:
 
     def _get_refs(self) -> dict[str, str]:
         refs = {}
+        ref = None
path: ulauncher/modes/extensions/extension_remote.py
user: login: gornostal
body: It was Pyright. It's probably better to keep this change. If the `try` block fails, the `except` block will try to reference an undeclared variable.
created_at: 2025-05-05T16:54:49Z
author_association: MEMBER
diff_hunk: @@ -3,7 +3,7 @@ root = true
 [*]
 charset = utf-8
 indent_style = space
-indent_size = 2
+indent_size = 4
path: .editorconfig
user: login: gornostal
body: You're right. I missed the python section below. Reverted.
created_at: 2025-05-05T16:55:14Z
author_association: MEMBER
diff_hunk: @@ -265,6 +265,21 @@ Custom Action on Item Enter
   Now this will be rendered when you click on any item
 
 
+Dependencies and requirements.txt file
+--------------------------------------
+
+If your extension requires any python dependencies, you can specify them in a ``requirements.txt`` file.
+Ulauncher will install them automatically when the extension is installed.
+
+Avoid specifying development dependencies such as mypy, pytest, etc. in the ``requirements.txt`` file — users don't need them.
+Instead, you can create a separate ``requirements-dev.txt`` file for your development dependencies.
+
+It's a good idea to specify the version of the dependencies you are using to avoid breaking changes in the future.
+For example, if your extension requires the `requests` library, you can create a `requirements.txt` file with the following content::
+
+  requests==2.32.3
path: docs/extensions/tutorial.rst
user: login: gornostal
body: I like `requirements.txt` because it's a standard naming convention in Python. 

Why do you think it's not used for runtime dependencies? I use it like that all the time.
created_at: 2025-05-05T16:59:13Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,81 @@
+from __future__ import annotations
+
+import logging
+import subprocess
+import sys
+from os.path import isdir, isfile
+
+logger = logging.getLogger()
+
+
+class ExtensionDependenciesRecoverableError(Exception):
+    pass
+
+
+class ExtensionDependencies:
+    """
+    Manages python dependencies of the extension.
+    """
+
+    ext_id: str
+    path: str
+
+    deps_out_subdir: str = ".dependencies"
+
+    def __init__(self, ext_id: str, path: str):
+        self.ext_id = ext_id
+        self.path = path
+
+    def get_dependencies_path(self) -> str | None:
+        """
+        Get the path to the dependencies directory.
+        Returns None if there's no dependencies for the extension.
+        """
+        requirements_file = f"{self.path}/requirements.txt"
+        deps_path = f"{self.path}/{self.deps_out_subdir}"
+        if not isfile(requirements_file) or not isdir(deps_path):
+            return None
+
+        return deps_path
+
+    def install(self) -> None:
+        """
+        Install the dependencies for the extension.
+        Runs `pip install -r extension-X/requirements.txt --target extension-X/deps`
+        """
+        requirements = self._read_requirements()
+
+        if not requirements:
+            return
+
+        command = [
+            sys.executable,
+            "-m",
+            "pip",
+            "install",
+            "-r",
+            f"{self.path}/requirements.txt",
+            "--target",
+            f"{self.path}/{self.deps_out_subdir}",
+        ]
+        command_str = " ".join(command)
+        logger.info("Installing dependencies for %s. Running: %s", self.ext_id, command_str)
+
+        try:
+            result = subprocess.run(command, check=True, capture_output=True, text=True)
+            logger.info("Installation successful. Output: %s", result.stdout)
+        except subprocess.CalledProcessError as e:
+            logger.exception("Error during installation. Output: %s", e.stderr)
+
+            raise ExtensionDependenciesRecoverableError(f"$ {command_str}\n{e.stderr}") from e  # noqa: TRY003, EM102
+
+    def _read_requirements(self) -> str | None:
path: ulauncher/modes/extensions/extension_dependencies.py
user: login: gornostal
body: Could you elaborate, please? I'm not sure what edge case you have in mind.
created_at: 2025-05-05T17:02:12Z
author_association: MEMBER
diff_hunk: @@ -265,6 +265,21 @@ Custom Action on Item Enter
   Now this will be rendered when you click on any item
 
 
+Dependencies and requirements.txt file
+--------------------------------------
+
+If your extension requires any python dependencies, you can specify them in a ``requirements.txt`` file.
+Ulauncher will install them automatically when the extension is installed.
+
+Avoid specifying development dependencies such as mypy, pytest, etc. in the ``requirements.txt`` file — users don't need them.
+Instead, you can create a separate ``requirements-dev.txt`` file for your development dependencies.
+
+It's a good idea to specify the version of the dependencies you are using to avoid breaking changes in the future.
+For example, if your extension requires the `requests` library, you can create a `requirements.txt` file with the following content::
+
+  requests==2.32.3
path: docs/extensions/tutorial.rst
user: login: troycurtisjr
body: It is the fact that is a standard naming convention that makes me suggest we should be using a different namespace for ulauncher extension purposes. 

It is true that at its core a `requirements.txt` file is just a list of packages to install and thus can be used for many purposes, including for runtime requirements. However, the typical purposes would be for things as described in [the Pip docs](https://pip.pypa.io/en/latest/user_guide/#requirements-files). Things like repeatable builds with the output of `pip freeze`, forcing specific versions of transitive dependencies, etc. I think it is this variety of existing and established usages that means we should have a dedicated name for ulauncher's purposes. Already the docs mention needing to move dev only deps out into a separate file, implicitly acknowledging that any existing `requirements.txt` file is probably for local development setup. A more standardized form of specifying dependencies would be through package meta-data, but I know we aren't planning on having ulauncher extensions turned into packages distributed on pypi. 

Specifying a distinct name I think gives a few advantages:
1. It clearly communicates it's purpose: to install dependencies when installed by ulauncher
2. It doesn't collide with existing usages of any `requirements.txt`
3. Because of point 2, it provides a cleaner upgrade path since no existing `requirements.txt` will get misinterpreted as being for `ulauncher`
created_at: 2025-05-06T02:04:51Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,81 @@
+from __future__ import annotations
+
+import logging
+import subprocess
+import sys
+from os.path import isdir, isfile
+
+logger = logging.getLogger()
+
+
+class ExtensionDependenciesRecoverableError(Exception):
+    pass
+
+
+class ExtensionDependencies:
+    """
+    Manages python dependencies of the extension.
+    """
+
+    ext_id: str
+    path: str
+
+    deps_out_subdir: str = ".dependencies"
+
+    def __init__(self, ext_id: str, path: str):
+        self.ext_id = ext_id
+        self.path = path
+
+    def get_dependencies_path(self) -> str | None:
+        """
+        Get the path to the dependencies directory.
+        Returns None if there's no dependencies for the extension.
+        """
+        requirements_file = f"{self.path}/requirements.txt"
+        deps_path = f"{self.path}/{self.deps_out_subdir}"
+        if not isfile(requirements_file) or not isdir(deps_path):
+            return None
+
+        return deps_path
+
+    def install(self) -> None:
+        """
+        Install the dependencies for the extension.
+        Runs `pip install -r extension-X/requirements.txt --target extension-X/deps`
+        """
+        requirements = self._read_requirements()
+
+        if not requirements:
+            return
+
+        command = [
+            sys.executable,
+            "-m",
+            "pip",
+            "install",
+            "-r",
+            f"{self.path}/requirements.txt",
+            "--target",
+            f"{self.path}/{self.deps_out_subdir}",
+        ]
+        command_str = " ".join(command)
+        logger.info("Installing dependencies for %s. Running: %s", self.ext_id, command_str)
+
+        try:
+            result = subprocess.run(command, check=True, capture_output=True, text=True)
+            logger.info("Installation successful. Output: %s", result.stdout)
+        except subprocess.CalledProcessError as e:
+            logger.exception("Error during installation. Output: %s", e.stderr)
+
+            raise ExtensionDependenciesRecoverableError(f"$ {command_str}\n{e.stderr}") from e  # noqa: TRY003, EM102
+
+    def _read_requirements(self) -> str | None:
path: ulauncher/modes/extensions/extension_dependencies.py
user: login: troycurtisjr
body: I was actually thinking of ulauncher specifically (i.e. they had something like `ulauncher==5.0.0` or similar), forgetting that ulauncher isn't distributed via PyPi anyway :grin: 

But instead of ulauncher specifically, if the extension specified different versions of ulauncher dependencies, it could break the install. Though I suppose that would simply be breaking the extension itself, and presumably for some version of ulauncher that it actually worked in, they wouldn't have specified known busted packages. And only that particular extension would be busted, not impacting anyone else.

We already have to deal with extension problems being perceived as ulauncher issues, so it probably doesn't change that issue much. Still I'd be tempted to error on specification of packages that ulauncher itself needs, such that only additional packages should be specified by the extension.

I think the messaging interchange is well defined enough that we probably don't have to worry about being overly sensitive to packages being different versions between the ulauncher and extension processes.
created_at: 2025-05-06T02:15:42Z
author_association: MEMBER
diff_hunk: @@ -265,6 +265,21 @@ Custom Action on Item Enter
   Now this will be rendered when you click on any item
 
 
+Dependencies and requirements.txt file
+--------------------------------------
+
+If your extension requires any python dependencies, you can specify them in a ``requirements.txt`` file.
+Ulauncher will install them automatically when the extension is installed.
+
+Avoid specifying development dependencies such as mypy, pytest, etc. in the ``requirements.txt`` file — users don't need them.
+Instead, you can create a separate ``requirements-dev.txt`` file for your development dependencies.
+
+It's a good idea to specify the version of the dependencies you are using to avoid breaking changes in the future.
+For example, if your extension requires the `requests` library, you can create a `requirements.txt` file with the following content::
+
+  requests==2.32.3
path: docs/extensions/tutorial.rst
user: login: gornostal
body: I see your points Troy. I feel different about the purpose of `requirements.txt`, tbh.

I did a small experiment and I learned that the current implementation would "cure" over 80% of the existing extensions that include `requirements.txt`. 

That's my main argument in favor of keeping the naming, besides that it's a convention too.

I downloaded 230 of the most recent extensions, 35 of them had requirements.txt file, and only 5 of 35 failed on `pip install -r requirements.txt`. Sure, some dependencies were development-related (mypy, etc.), but most of them were runtime deps.
created_at: 2025-05-06T07:33:53Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,81 @@
+from __future__ import annotations
+
+import logging
+import subprocess
+import sys
+from os.path import isdir, isfile
+
+logger = logging.getLogger()
+
+
+class ExtensionDependenciesRecoverableError(Exception):
+    pass
+
+
+class ExtensionDependencies:
+    """
+    Manages python dependencies of the extension.
+    """
+
+    ext_id: str
+    path: str
+
+    deps_out_subdir: str = ".dependencies"
+
+    def __init__(self, ext_id: str, path: str):
+        self.ext_id = ext_id
+        self.path = path
+
+    def get_dependencies_path(self) -> str | None:
+        """
+        Get the path to the dependencies directory.
+        Returns None if there's no dependencies for the extension.
+        """
+        requirements_file = f"{self.path}/requirements.txt"
+        deps_path = f"{self.path}/{self.deps_out_subdir}"
+        if not isfile(requirements_file) or not isdir(deps_path):
+            return None
+
+        return deps_path
+
+    def install(self) -> None:
+        """
+        Install the dependencies for the extension.
+        Runs `pip install -r extension-X/requirements.txt --target extension-X/deps`
+        """
+        requirements = self._read_requirements()
+
+        if not requirements:
+            return
+
+        command = [
+            sys.executable,
+            "-m",
+            "pip",
+            "install",
+            "-r",
+            f"{self.path}/requirements.txt",
+            "--target",
+            f"{self.path}/{self.deps_out_subdir}",
+        ]
+        command_str = " ".join(command)
+        logger.info("Installing dependencies for %s. Running: %s", self.ext_id, command_str)
+
+        try:
+            result = subprocess.run(command, check=True, capture_output=True, text=True)
+            logger.info("Installation successful. Output: %s", result.stdout)
+        except subprocess.CalledProcessError as e:
+            logger.exception("Error during installation. Output: %s", e.stderr)
+
+            raise ExtensionDependenciesRecoverableError(f"$ {command_str}\n{e.stderr}") from e  # noqa: TRY003, EM102
+
+    def _read_requirements(self) -> str | None:
path: ulauncher/modes/extensions/extension_dependencies.py
user: login: troycurtisjr
body: Yeah probably just being overly cautious without a clear failure mode.
created_at: 2025-05-08T01:29:55Z
author_association: MEMBER
diff_hunk: @@ -265,6 +265,21 @@ Custom Action on Item Enter
   Now this will be rendered when you click on any item
 
 
+Dependencies and requirements.txt file
+--------------------------------------
+
+If your extension requires any python dependencies, you can specify them in a ``requirements.txt`` file.
+Ulauncher will install them automatically when the extension is installed.
+
+Avoid specifying development dependencies such as mypy, pytest, etc. in the ``requirements.txt`` file — users don't need them.
+Instead, you can create a separate ``requirements-dev.txt`` file for your development dependencies.
+
+It's a good idea to specify the version of the dependencies you are using to avoid breaking changes in the future.
+For example, if your extension requires the `requests` library, you can create a `requirements.txt` file with the following content::
+
+  requests==2.32.3
path: docs/extensions/tutorial.rst
user: login: troycurtisjr
body: Ultimately the worst case is some grumbling and momentary confusion with extensions authors, and then they can just use a differently named file if they need to use it for one of the other purposes. On the user side, there might just be a few extra deps installed that are only needed for dev and shouldn't cause any real issues.
created_at: 2025-05-08T01:35:22Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,81 @@
+from __future__ import annotations
+
+import logging
+import subprocess
+import sys
+from os.path import isdir, isfile
+
+logger = logging.getLogger()
+
+
+class ExtensionDependenciesRecoverableError(Exception):
+    pass
+
+
+class ExtensionDependencies:
+    """
+    Manages python dependencies of the extension.
+    """
+
+    ext_id: str
+    path: str
+
+    deps_out_subdir: str = ".dependencies"
+
+    def __init__(self, ext_id: str, path: str):
+        self.ext_id = ext_id
+        self.path = path
+
+    def get_dependencies_path(self) -> str | None:
+        """
+        Get the path to the dependencies directory.
+        Returns None if there's no dependencies for the extension.
+        """
+        requirements_file = f"{self.path}/requirements.txt"
+        deps_path = f"{self.path}/{self.deps_out_subdir}"
+        if not isfile(requirements_file) or not isdir(deps_path):
+            return None
+
+        return deps_path
+
+    def install(self) -> None:
+        """
+        Install the dependencies for the extension.
+        Runs `pip install -r extension-X/requirements.txt --target extension-X/deps`
+        """
+        requirements = self._read_requirements()
+
+        if not requirements:
+            return
+
+        command = [
+            sys.executable,
+            "-m",
+            "pip",
+            "install",
+            "-r",
+            f"{self.path}/requirements.txt",
+            "--target",
+            f"{self.path}/{self.deps_out_subdir}",
+        ]
+        command_str = " ".join(command)
+        logger.info("Installing dependencies for %s. Running: %s", self.ext_id, command_str)
+
+        try:
+            result = subprocess.run(command, check=True, capture_output=True, text=True)
+            logger.info("Installation successful. Output: %s", result.stdout)
+        except subprocess.CalledProcessError as e:
+            logger.exception("Error during installation. Output: %s", e.stderr)
+
+            raise ExtensionDependenciesRecoverableError(f"$ {command_str}\n{e.stderr}") from e  # noqa: TRY003, EM102
+
+    def _read_requirements(self) -> str | None:
path: ulauncher/modes/extensions/extension_dependencies.py
user: login: gornostal
body: I see what you mean here. 

> if the extension specified different versions of ulauncher dependencies, it could break the install. Though I suppose that would simply be breaking the extension itself

That could definitely happen. It's good that Ulauncher doesn't use too many dependencies. 
created_at: 2025-05-08T16:08:33Z
author_association: MEMBER

PR comments:
user: login: gornostal
created_at: 2025-05-01T18:06:39Z
body: @friday The implementation ended up much simpler than I thought. It almost seems like I missed something.
I won't merge this yet. I'll run some more tests and update the extension docs.

Please let me know if you think anything is missing.
author_association: MEMBER
user: login: friday
created_at: 2025-05-01T20:45:44Z
body: > @friday The implementation ended up much simpler than I thought. It almost seems like I missed something. I won't merge this yet. I'll run some more tests and update the extension docs.
> 
> Please let me know if you think anything is missing.

I'm not at all surprised this was simple. I'll  try to look at the PR more and test it in the weekend.
author_association: MEMBER
user: login: troycurtisjr
created_at: 2025-05-08T01:40:17Z
body: I also tested locally and things seemed to work well. I also waffle on whether we should use the `--upgrade` option or not. It seems like there are good reasons either way. I'd probably lean towards using `--upgrade` and then the requirements.txt could always be updated to constrain versions if some newer version causes breakage.
author_association: MEMBER
user: login: gornostal
created_at: 2025-05-08T16:10:41Z
body: Thanks for taking a look at this! I really appreciate your perspective on potential "what if" scenarios
author_association: MEMBER