Repository: canonical/snapcraft (language: python)

Issue:
Cannot run snapcraft in docker anymore
### Bug Description

I cannot run snapcraft in docker container anymore. It requires snapd to be running.

### To Reproduce

Build a docker container with snapcraft in it and then use that docker container to build a snap

### Environment

Docker with noble based image

### snapcraft.yaml

```yaml
no yaml config needed
```


### Relevant log output

```shell
root@d04e7a25f671:/datax# SNAP_INSTANCE_NAME=snapcraft snapcraft version
snapcraft internal error: SnapCtlError('Call to snapctl failed with error 1: error: cannot communicate with server: Post "http://localhost/v2/snapctl": dial unix /run/snapd-snap.socket: connect: no such file or directory\n')                                                                                                                               
Full execution log: '/root/.local/state/snapcraft/log/snapcraft-20240930-150845.117122.log'                                                                                    
Traceback (most recent call last):
  File "/snap/snapcraft/current/lib/python3.10/site-packages/craft_application/application.py", line 568, in run
    return_code = self._run_inner()
  File "/snap/snapcraft/current/lib/python3.10/site-packages/snapcraft/application.py", line 215, in _run_inner
    return_code = super()._run_inner()
  File "/snap/snapcraft/current/lib/python3.10/site-packages/craft_application/application.py", line 521, in _run_inner
    platform = self.get_arg_or_config(parsed_args, "platform")
  File "/snap/snapcraft/current/lib/python3.10/site-packages/craft_application/application.py", line 511, in get_arg_or_config
    return self.services.config.get(item)
  File "/snap/snapcraft/current/lib/python3.10/site-packages/craft_application/services/config.py", line 175, in get
    value = handler.get_raw(item)
  File "/snap/snapcraft/current/lib/python3.10/site-packages/craft_application/services/config.py", line 103, in get_raw
    return self._snap.get(snap_item)
  File "/snap/snapcraft/current/lib/python3.10/site-packages/snaphelpers/_conf.py", line 125, in get
    options = self.get_options(top_key)
  File "/snap/snapcraft/current/lib/python3.10/site-packages/snaphelpers/_conf.py", line 114, in get_options
    options.fetch()
  File "/snap/snapcraft/current/lib/python3.10/site-packages/snaphelpers/_conf.py", line 86, in fetch
    self._config = self._snapctl.config_get(*self._keys)
  File "/snap/snapcraft/current/lib/python3.10/site-packages/snaphelpers/_ctl.py", line 147, in config_get
    conf = json.loads(self.run("get", "-d", *keys))
  File "/snap/snapcraft/current/lib/python3.10/site-packages/snaphelpers/_ctl.py", line 271, in run
    raise SnapCtlError(process)
snaphelpers._ctl.SnapCtlError: Call to snapctl failed with error 1: error: cannot communicate with server: Post "http://localhost/v2/snapctl": dial unix /run/snapd-snap.socket: connect: no such file or directory
```
```


### Additional context

_No response_


Discussion / hints:
Thanks for the report! The only way I could see this happening is if you're setting the `SNAP` environment variable and the rest of the environment to mimic running in a snap. Could you try unsetting the `SNAP` environment variable in your container and see if it succeeds?

Separately I've created https://github.com/canonical/craft-application/pull/497 upstream so that we behave better in situations like this.
Separately from this - would it be feasible to move your use of snapcraft in containers to use https://github.com/canonical/snapcraft-rocks?
I am running into the same issue.

First, I saw an error stating

`snapcraft internal error: AttributeError("'INSTANCE_NAME'")`

After adding the `SNAP_INSTANCE_NAME` env var, which wasn't required when running Snapcraft 8.3.4, it then gave me the socket error mentioned above. Removing the `SNAP` variable resulted in another error: 

`/usr/bin/python3: can't open file '/bin/snapcraft': [Errno 2] No such file or directory`.

Oddly enough, this actually only happens for the `snapcraft upload` and `snapcraft version` command. I am able to run `snapcraft` and `snapcraft release` without seeing the `INSTANCE_NAME` error.
@lengau I used docker container from https://github.com/canonical/snapcraft-rocks and the build almost worked but it failed when my `override-build` stanza tried to call `craftctl` to set version.
```
2024-10-02 11:40:31.903 :: Traceback (most recent call last):
2024-10-02 11:40:31.903 ::   File "/usr/libexec/snapcraft/craftctl", line 5, in <module>
2024-10-02 11:40:31.903 ::     from craft_parts.ctl import main
2024-10-02 11:40:31.903 :: ModuleNotFoundError: No module named 'craft_parts'
2024-10-02 11:40:32.059 'override-build' in part 'st-matter-adapter' failed with code 1.
2024-10-02 11:40:32.064 Traceback (most recent call last):
2024-10-02 11:40:32.064   File "/lib/python3.12/site-packages/craft_application/services/lifecycle.py", line 270, in run
2024-10-02 11:40:32.064     aex.execute(action, stdout=stream, stderr=stream)
2024-10-02 11:40:32.064   File "/lib/python3.12/site-packages/craft_parts/executor/executor.py", line 324, in execute
2024-10-02 11:40:32.065     self._executor.execute(actions, stdout=stdout, stderr=stderr)
2024-10-02 11:40:32.065   File "/lib/python3.12/site-packages/craft_parts/executor/executor.py", line 136, in execute
2024-10-02 11:40:32.065   File "/lib/python3.12/site-packages/craft_parts/executor/executor.py", line 211, in _run_action
2024-10-02 11:40:32.065     handler.run_action(action, stdout=stdout, stderr=stderr)
2024-10-02 11:40:32.065   File "/lib/python3.12/site-packages/craft_parts/executor/part_handler.py", line 171, in run_action
2024-10-02 11:40:32.065     state = handler(step_info, stdout=stdout, stderr=stderr)
2024-10-02 11:40:32.066   File "/lib/python3.12/site-packages/craft_parts/executor/part_handler.py", line 310, in _run_build
2024-10-02 11:40:32.066     self._run_step(
2024-10-02 11:40:32.066   File "/lib/python3.12/site-packages/craft_parts/executor/part_handler.py", line 485, in _run_step
2024-10-02 11:40:32.066     step_handler.run_scriptlet(
2024-10-02 11:40:32.066   File "/lib/python3.12/site-packages/craft_parts/executor/step_handler.py", line 330, in run_scriptlet
2024-10-02 11:40:32.066     raise errors.ScriptletRunError(
2024-10-02 11:40:32.066 craft_parts.errors.ScriptletRunError: 'override-build' in part 'st-matter-adapter' failed with code 1.
2024-10-02 11:40:32.066 Recommended resolution: Review the scriptlet and make sure it's correct.
2024-10-02 11:40:32.066 Full execution log: '/root/.local/state/snapcraft/log/snapcraft-20241002-113945.009476.log'
```
This will be fixed in Snapcraft 8.4.2, which is scheduled to be in the candidate channel next week. 

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/CRAFT-3530.

> This message was autogenerated


Fix patch:
diff --git a/docs/reference/changelog.rst b/docs/reference/changelog.rst
index fbc344f8ac..61bd5ff297 100644
--- a/docs/reference/changelog.rst
+++ b/docs/reference/changelog.rst
@@ -68,6 +68,58 @@ Changelog
 
   For a complete list of commits, check out the `X.Y.Z`_ release on GitHub.
 
+
+8.4.2 (2024-Oct-07)
+-------------------
+
+Core
+====
+
+* Fix a regression where Snapcraft would fail to run on some architectures due
+  to a ``cryptography`` dependency that attempted to load legacy algorithms
+  (`#5077`_).
+
+* Fix a regression where Snapcraft would fail to run in a container if it was
+  not running as a snap (`#5079`_).
+
+* Fix a bug where parallel installations of Snapcraft would not work if the
+  Snapcraft snap was installed from the store (`#4683`_, `#4927`_).
+
+Plugins
+#######
+
+Python
+""""""
+
+* Fix an issue where the ``python`` plugin would fail to build if the part
+  had no Python scripts.
+
+Remote build
+============
+
+* Fix a bug where the remote builder would ignore the user's response when a
+  build is interrupted and always clean the launchpad project (`#4929`_).
+
+Documentation
+=============
+
+* Update Rust plugin doc with recent changes to the Rust toolchain.
+
+For a complete list of commits, check out the `8.4.2`_ release on GitHub.
+
+
+7.5.7 (2024-Oct-03)
+-------------------
+
+Core
+====
+
+* Fix a bug where parallel installations of Snapcraft would not work if the
+  Snapcraft snap was installed from the store (`#4683`_, `#4927`_).
+
+For a complete list of commits, check out the `7.5.7`_ release on GitHub.
+
+
 8.4.1 (2024-Sep-20)
 -------------------
 
@@ -1187,6 +1239,7 @@ For a complete list of commits, check out the `8.0.0`_ release on GitHub.
 .. _#4517: https://github.com/canonical/snapcraft/issues/4517
 .. _#4520: https://github.com/canonical/snapcraft/issues/4520
 .. _#4547: https://github.com/canonical/snapcraft/issues/4547
+.. _#4683: https://github.com/canonical/snapcraft/issues/4683
 .. _#4685: https://github.com/canonical/snapcraft/issues/4685
 .. _#4735: https://github.com/canonical/snapcraft/issues/4735
 .. _#4744: https://github.com/canonical/snapcraft/issues/4744
@@ -1219,6 +1272,8 @@ For a complete list of commits, check out the `8.0.0`_ release on GitHub.
 .. _#4890: https://github.com/canonical/snapcraft/issues/4890
 .. _#4908: https://github.com/canonical/snapcraft/issues/4908
 .. _#4909: https://github.com/canonical/snapcraft/issues/4909
+.. _#4927: https://github.com/canonical/snapcraft/issues/4927
+.. _#4929: https://github.com/canonical/snapcraft/issues/4929
 .. _#4930: https://github.com/canonical/snapcraft/issues/4930
 .. _#4941: https://github.com/canonical/snapcraft/issues/4941
 .. _#4942: https://github.com/canonical/snapcraft/issues/4942
@@ -1228,8 +1283,11 @@ For a complete list of commits, check out the `8.0.0`_ release on GitHub.
 .. _#4995: https://github.com/canonical/snapcraft/issues/4995
 .. _#5008: https://github.com/canonical/snapcraft/issues/5008
 .. _#5048: https://github.com/canonical/snapcraft/issues/5048
+.. _#5077: https://github.com/canonical/snapcraft/issues/5077
+.. _#5079: https://github.com/canonical/snapcraft/issues/5079
 
 .. _7.5.6: https://github.com/canonical/snapcraft/releases/tag/7.5.6
+.. _7.5.7: https://github.com/canonical/snapcraft/releases/tag/7.5.7
 .. _8.0.0: https://github.com/canonical/snapcraft/releases/tag/8.0.0
 .. _8.0.1: https://github.com/canonical/snapcraft/releases/tag/8.0.1
 .. _8.0.2: https://github.com/canonical/snapcraft/releases/tag/8.0.2
@@ -1257,3 +1315,4 @@ For a complete list of commits, check out the `8.0.0`_ release on GitHub.
 .. _8.3.4: https://github.com/canonical/snapcraft/releases/tag/8.3.4
 .. _8.4.0: https://github.com/canonical/snapcraft/releases/tag/8.4.0
 .. _8.4.1: https://github.com/canonical/snapcraft/releases/tag/8.4.1
+.. _8.4.2: https://github.com/canonical/snapcraft/releases/tag/8.4.2
diff --git a/requirements-devel.txt b/requirements-devel.txt
index f9daa0d898..da45be35cf 100644
--- a/requirements-devel.txt
+++ b/requirements-devel.txt
@@ -24,13 +24,13 @@ click==8.1.7
 codespell==2.3.0
 colorama==0.4.6
 coverage==7.6.1
-craft-application==4.2.4
+craft-application==4.2.6
 craft-archives==2.0.0
 craft-cli==2.7.0
 craft-grammar==2.0.1
-craft-parts==2.1.1
+craft-parts==2.1.2
 craft-platforms==0.1.1
-craft-providers==2.0.3
+craft-providers==2.0.4
 craft-store==3.0.2
 cryptography==43.0.1
 cssutils==2.11.1
@@ -108,7 +108,7 @@ pyftpdlib==1.5.10
 pygit2==1.13.3
 Pygments==2.18.0
 pylint==3.2.7
-pylxd==2.3.4
+pylxd==2.3.5
 pymacaroons==0.13.0
 PyNaCl==1.5.0
 pyparsing==3.1.4
@@ -128,9 +128,9 @@ pyxdg==0.28
 PyYAML==6.0.2
 raven==6.10.0
 regex==2024.7.24
-requests==2.31.0
+requests==2.32.3
 requests-toolbelt==1.0.0
-requests-unixsocket==0.3.0
+requests-unixsocket2==0.4.2
 ruamel.yaml==0.18.6
 ruamel.yaml.clib==0.2.8
 SecretStorage==3.3.3
@@ -182,7 +182,7 @@ types-toml==0.10.8.20240310
 types-urllib3==1.26.25.14
 typing_extensions==4.12.2
 uc-micro-py==1.0.3
-urllib3==1.26.20
+urllib3==2.2.3
 uvicorn==0.30.6
 validators==0.33.0
 venusian==3.1.0
diff --git a/requirements-docs.txt b/requirements-docs.txt
index 06ab3d0070..1c690ec149 100644
--- a/requirements-docs.txt
+++ b/requirements-docs.txt
@@ -19,13 +19,13 @@ chardet==5.2.0
 charset-normalizer==3.3.2
 click==8.1.7
 colorama==0.4.6
-craft-application==4.2.4
+craft-application==4.2.6
 craft-archives==2.0.0
 craft-cli==2.7.0
 craft-grammar==2.0.1
-craft-parts==2.1.1
+craft-parts==2.1.2
 craft-platforms==0.1.1
-craft-providers==2.0.3
+craft-providers==2.0.4
 craft-store==3.0.2
 cryptography==43.0.1
 cssutils==2.11.1
@@ -81,7 +81,7 @@ pydantic_yaml==1.3.0
 pyelftools==0.31
 pygit2==1.13.3
 Pygments==2.18.0
-pylxd==2.3.4
+pylxd==2.3.5
 pymacaroons==0.13.0
 PyNaCl==1.5.0
 pyparsing==3.1.4
@@ -94,9 +94,9 @@ pyxdg==0.28
 PyYAML==6.0.2
 raven==6.10.0
 regex==2024.7.24
-requests==2.31.0
+requests==2.32.3
 requests-toolbelt==1.0.0
-requests-unixsocket==0.3.0
+requests-unixsocket2==0.4.2
 ruamel.yaml==0.18.6
 ruamel.yaml.clib==0.2.8
 SecretStorage==3.3.3
@@ -136,7 +136,7 @@ tinydb==4.8.0
 toml==0.10.2
 typing_extensions==4.12.2
 uc-micro-py==1.0.3
-urllib3==1.26.20
+urllib3==2.2.3
 uvicorn==0.30.6
 validators==0.33.0
 wadllib==1.3.6
diff --git a/requirements.txt b/requirements.txt
index f6eac1072b..d69ae9cef6 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -7,13 +7,13 @@ cffi==1.17.1
 chardet==5.2.0
 charset-normalizer==3.3.2
 click==8.1.7
-craft-application==4.2.4
+craft-application==4.2.6
 craft-archives==2.0.0
 craft-cli==2.7.0
 craft-grammar==2.0.1
-craft-parts==2.1.1
+craft-parts==2.1.2
 craft-platforms==0.1.1
-craft-providers==2.0.3
+craft-providers==2.0.4
 craft-store==3.0.2
 cryptography==43.0.1
 distro==1.9.0
@@ -47,7 +47,7 @@ pydantic_core==2.20.1
 pydantic_yaml==1.3.0
 pyelftools==0.31
 pygit2==1.13.3
-pylxd==2.3.4
+pylxd==2.3.5
 pymacaroons==0.13.0
 PyNaCl==1.5.0
 pyparsing==3.1.4
@@ -58,9 +58,9 @@ pytz==2024.1
 pyxdg==0.28
 PyYAML==6.0.2
 raven==6.10.0
-requests==2.31.0
+requests==2.32.3
 requests-toolbelt==1.0.0
-requests-unixsocket==0.3.0
+requests-unixsocket2==0.4.2
 ruamel.yaml==0.18.6
 ruamel.yaml.clib==0.2.8
 SecretStorage==3.3.3
@@ -72,7 +72,7 @@ tabulate==0.9.0
 tinydb==4.8.0
 toml==0.10.2
 typing_extensions==4.12.2
-urllib3==1.26.20
+urllib3==2.2.3
 validators==0.33.0
 wadllib==1.3.6
 wheel==0.44.0
diff --git a/setup.py b/setup.py
index 351f5a9379..fdfb1be53b 100755
--- a/setup.py
+++ b/setup.py
@@ -98,13 +98,13 @@ def recursive_data_files(directory, install_directory):
     "attrs",
     "catkin-pkg; sys_platform == 'linux'",
     "click",
-    "craft-application>=4.2.4,<5.0.0",
+    "craft-application>=4.2.6,<5.0.0",
     "craft-archives~=2.0",
     "craft-cli~=2.6",
     "craft-grammar>=2.0.1,<3.0.0",
-    "craft-parts~=2.1",
+    "craft-parts>=2.1.2,<3.0.0",
     "craft-platforms~=0.1",
-    "craft-providers>=2.0.3,<3.0.0",
+    "craft-providers>=2.0.4,<3.0.0",
     "craft-store>=3.0.2,<4.0.0",
     "docutils<0.20",  # Frozen until we can update sphinx dependencies.
     "gnupg",
@@ -130,7 +130,7 @@ def recursive_data_files(directory, install_directory):
     "pyyaml",
     "raven",
     "requests-toolbelt",
-    "requests-unixsocket",
+    "requests-unixsocket2",
     "requests",
     "simplejson",
     "snap-helpers",
@@ -138,7 +138,6 @@ def recursive_data_files(directory, install_directory):
     "toml",
     "tinydb",
     "typing-extensions",
-    "urllib3<2",  # requests-unixsocket does not yet work with urllib3 v2.0+
     "validators>=0.28.3",
 ]
 


Test patch:
diff --git a/tests/legacy/fake_servers/snapd.py b/tests/legacy/fake_servers/snapd.py
index 4616b28a32..be5c52a01f 100644
--- a/tests/legacy/fake_servers/snapd.py
+++ b/tests/legacy/fake_servers/snapd.py
@@ -53,7 +53,6 @@ def _handle_snaps(self):
 
     def _handle_snap_file(self, parsed_url):
         self.send_response(200)
-        self.send_header("Content-Length", len(parsed_url))
         self.send_header("Content-type", "text/plain")
         self.end_headers()
         self.wfile.write(parsed_url.encode())
diff --git a/tests/unit/parts/plugins/test_python_plugin.py b/tests/unit/parts/plugins/test_python_plugin.py
index d41f6d295a..5fa6d87337 100644
--- a/tests/unit/parts/plugins/test_python_plugin.py
+++ b/tests/unit/parts/plugins/test_python_plugin.py
@@ -66,7 +66,7 @@ def test_get_build_commands(plugin, new_dir):
         f'PARTS_PYTHON_VENV_INTERP_PATH="{new_dir}/parts/my-part/install/bin/${{PARTS_PYTHON_INTERPRETER}}"',
         f"{new_dir}/parts/my-part/install/bin/pip install  -U pip setuptools wheel",
         f"[ -f setup.py ] || [ -f pyproject.toml ] && {new_dir}/parts/my-part/install/bin/pip install  -U .",
-        f'find "{new_dir}/parts/my-part/install" -type f -executable -print0 | xargs -0 \\\n'
+        f'find "{new_dir}/parts/my-part/install" -type f -executable -print0 | xargs --no-run-if-empty -0 \\\n'
         '    sed -i "1 s|^#\\!${PARTS_PYTHON_VENV_INTERP_PATH}.*$|#!/usr/bin/env ${PARTS_PYTHON_INTERPRETER}|"\n',
         dedent(
             f"""\