Repository: zestsoftware/zest.releaser
Pull request #463: Really build source dist and wheel in isolation.
Author: login: mauritsvanrees
Created: 2025-03-07T16:48:31Z | Merged: 2025-03-08T20:49:45Z

Previously we passed along our own `PYTHONPATH`, but that could lead to an unintended `setuptools` version being used.
See [my comment](https://github.com/plone/meta/pull/268#issuecomment-2703442444) on a `plone.meta` PR.

Also, when I ran `tox -e py39` locally, I got a traceback:

```
...
  File "/Users/maurits/community/zest.releaser/zest/releaser/__init__.py", line 12, in <module>
    __version__ = version("zest.releaser")
  File "/Users/maurits/.pyenv/versions/3.9.21/lib/python3.9/importlib/metadata.py", line 569, in version
    return distribution(distribution_name).version
  File "/Users/maurits/.pyenv/versions/3.9.21/lib/python3.9/importlib/metadata.py", line 542, in distribution
    return Distribution.from_name(distribution_name)
  File "/Users/maurits/.pyenv/versions/3.9.21/lib/python3.9/importlib/metadata.py", line 196, in from_name
    raise PackageNotFoundError(name)
importlib.metadata.PackageNotFoundError: zest.releaser
```

So in the second commit, we try hard to determine our own installed version, but do not fail when this does not work.  Report the version as "unknown" then.  There is no sense in failing to start up just because Python packaging is having a rough day...  Or week...



Diff:
diff --git a/CHANGES.rst b/CHANGES.rst
index 9b43daa0..ed779909 100644
--- a/CHANGES.rst
+++ b/CHANGES.rst
@@ -4,7 +4,12 @@ Changelog for zest.releaser
 9.4.1 (unreleased)
 ------------------
 
-- Nothing changed yet.
+- Worked around python 3.9 version-reporting issue that could prevent startup.
+  [maurits]
+
+- Really build source dist and wheel in isolation.
+  Previously we passed along our own ``PYTHONPATH``, but that could lead to an unintended ``setuptools`` version being used.
+  [maurits]
 
 
 9.4.0 (2025-03-05)
diff --git a/zest/releaser/__init__.py b/zest/releaser/__init__.py
index e1b5ed25..2f1297a1 100644
--- a/zest/releaser/__init__.py
+++ b/zest/releaser/__init__.py
@@ -1,4 +1,5 @@
 from colorama import init
+from importlib.metadata import PackageNotFoundError
 from importlib.metadata import version
 
 
@@ -8,4 +9,13 @@
 # but it looks like it already does that itself.
 init(autoreset=True)
 
-__version__ = version("zest.releaser")
+# Depending on which Python and setuptools version you use, and whether you use
+# an editable install or install from sdist or wheel, or with pip or Buildout,
+# there may be problems getting our own version.  Let's not break on that.
+try:
+    __version__ = version("zest.releaser")
+except PackageNotFoundError:
+    try:
+        __version__ = version("zest-releaser")
+    except PackageNotFoundError:
+        __version__ = "unknown"
diff --git a/zest/releaser/release.py b/zest/releaser/release.py
index d39c91dd..fddd7fab 100644
--- a/zest/releaser/release.py
+++ b/zest/releaser/release.py
@@ -67,10 +67,47 @@ def _project_builder_runner(cmd, cwd=None, extra_environ=None):
     extra_environ will contain for example:
 
     {'PEP517_BUILD_BACKEND': 'setuptools.build_meta:__legacy__'}
+
+    The default runner is `pyproject_hooks.default_subprocess_runner`,
+    which currently does this:
+
+        env = os.environ.copy()
+        if extra_environ:
+            env.update(extra_environ)
+        check_call(cmd, cwd=cwd, env=env)
+
+    So let's do the same, but use our wrapper for only showing interesting lines.
+    Note that originally we simply passed along the extra_environ.  But
+    execute_command then combined this with the current environment and with
+    the PYTHONPATH.  That last part gave unwanted results: an upper bound
+    on the setuptools version in the build-system was ignored.
     """
-    utils.show_interesting_lines(
-        execute_command(cmd, cwd=cwd, extra_environ=extra_environ)
-    )
+    env = os.environ.copy()
+    if extra_environ:
+        env.update(extra_environ)
+    utils.show_interesting_lines(execute_command(cmd, cwd=cwd, env=env))
+
+
+def _build_in_isolated_env(distribution):
+    """Build distribution in isolated env.
+
+    This is our variant of the _build_in_isolated_env function from the
+    `build` module.
+    """
+    if distribution not in {"sdist", "wheel"}:
+        raise ValueError(f"Invalid argument passed: {distribution=}")
+    with DefaultIsolatedEnv(installer="pip") as env:
+        # We use an isolated env, otherwise `build` cannot install packages
+        # needed for the build system, for example `hatchling`.
+        # See https://github.com/zestsoftware/zest.releaser/issues/448
+        builder = ProjectBuilder.from_isolated_env(
+            env,
+            source_dir=".",
+            runner=_project_builder_runner,
+        )
+        env.install(builder.build_system_requires)
+        env.install(builder.get_requires_for_build(distribution))
+        builder.build(distribution, "./dist/")
 
 
 class Releaser(baserelease.Basereleaser):
@@ -148,24 +185,13 @@ def _upload_distributions(self, package):
             "Making a source distribution of a fresh tag checkout (in %s).",
             self.data["tagworkingdir"],
         )
-        with DefaultIsolatedEnv() as env:
-            # We use an isolated env, otherwise `build` cannot install packages
-            # needed for the build system, for example `hatchling`.
-            # See https://github.com/zestsoftware/zest.releaser/issues/448
-            builder = ProjectBuilder.from_isolated_env(
-                env,
-                source_dir=".",
-                runner=_project_builder_runner,
+        _build_in_isolated_env("sdist")
+        if self.zest_releaser_config.create_wheel():
+            logger.info(
+                "Making a wheel of a fresh tag checkout (in %s).",
+                self.data["tagworkingdir"],
             )
-            env.install(builder.build_system_requires)
-            builder.build("sdist", "./dist/")
-            if self.zest_releaser_config.create_wheel():
-                logger.info(
-                    "Making a wheel of a fresh tag checkout (in %s).",
-                    self.data["tagworkingdir"],
-                )
-                env.install(builder.get_requires_for_build("wheel"))
-                builder.build("wheel", "./dist/")
+            _build_in_isolated_env("wheel")
         if not self.zest_releaser_config.upload_pypi():
             logger.info("Upload to PyPI was disabled in the configuration.")
             return
@@ -190,6 +216,10 @@ def _upload_distributions(self, package):
         register = self.zest_releaser_config.register_package()
 
         # If TWINE_REPOSITORY_URL is set, use it.
+        print(f"Current working directory: {os.getcwd()}")
+        print("These files are ready for upload:")
+        for filename in files_in_dist:
+            print(f"- {filename}")
         if self.pypiconfig.twine_repository_url():
             if not self._ask_upload(
                 package, self.pypiconfig.twine_repository_url(), register
diff --git a/zest/releaser/tests/functional-git.txt b/zest/releaser/tests/functional-git.txt
index 559522cf..70f1856a 100644
--- a/zest/releaser/tests/functional-git.txt
+++ b/zest/releaser/tests/functional-git.txt
@@ -94,16 +94,14 @@ known on PyPI:
     tag_svn_revision = false
     <BLANKLINE>
     <BLANKLINE>
-    Showing first few lines...
+    ...
     running sdist
-    running egg_info
-    creating src/tha.example.egg-info
     ...
-    Creating ...
-    removing ...
+    running bdist_wheel
+    ...
     Question: Upload to pypi (y/N)?
     Our reply: yes
-    MOCK twine dispatch upload ...example-0.1.tar.gz
+    MOCK twine dispatch upload ...whl ...example-0.1.tar.gz
 
 (Note: depending in the Python/setuptools version, the filename may contain ``tha.example`` or ``tha_example``.)
 
diff --git a/zest/releaser/tests/functional-with-hooks.txt b/zest/releaser/tests/functional-with-hooks.txt
index 14b87c52..5e038a8b 100644
--- a/zest/releaser/tests/functional-with-hooks.txt
+++ b/zest/releaser/tests/functional-with-hooks.txt
@@ -101,17 +101,18 @@ The release script tags the release and uploads it:
     [zest.releaser]
     ...
     releaser_after_checkout
-    Showing first few lines...
+    ...
     running sdist
-    running egg_info
-    creating src/tha.example.egg-info
     ...
-    Creating ...
-    removing ...
+    running bdist_wheel
+    ...
     releaser_before_upload
+    ...
+    These files are ready for upload:
+    ...
     Question: Upload to pypi (Y/n)?
     Our reply: y
-    MOCK twine dispatch upload ...example-0.1.tar.gz
+    MOCK twine dispatch upload ...whl ...example-0.1.tar.gz
     releaser_after
 
 (Note: depending in the Python/setuptools version, the filename may contain ``tha.example`` or ``tha_example``.)
diff --git a/zest/releaser/tests/pyproject-toml.txt b/zest/releaser/tests/pyproject-toml.txt
index 591bc70c..4c792447 100644
--- a/zest/releaser/tests/pyproject-toml.txt
+++ b/zest/releaser/tests/pyproject-toml.txt
@@ -92,19 +92,15 @@ known on PyPI:
     Our reply: y
     RED Note: ...0.1...
     ...
-    RED HEAD is now at ...
     Preparing release 0.1
-    <BLANKLINE>
-    Showing first few lines...
+    ...
     running sdist
-    running egg_info
-    creating src/tha.example.egg-info
     ...
-    Creating ...
-    removing ...
+    running bdist_wheel
+    ...
     Question: Upload to pypi (y/N)?
     Our reply: yes
-    MOCK twine dispatch upload ...example-0.1.tar.gz
+    MOCK twine dispatch upload ...whl ...example-0.1.tar.gz
 
 (Note: depending in the Python/setuptools version, the filename may contain ``tha.example`` or ``tha_example``.)
 
diff --git a/zest/releaser/utils.py b/zest/releaser/utils.py
index 2d69e8e9..e38a7a03 100644
--- a/zest/releaser/utils.py
+++ b/zest/releaser/utils.py
@@ -647,14 +647,18 @@ def format_command(command):
     return " ".join(args)
 
 
-def _execute_command(command, cwd=None, extra_environ=None):
+def _execute_command(command, cwd=None, extra_environ=None, env=None):
     """Execute a command, returning stdout, plus maybe parts of stderr."""
     # Enforce the command to be a list or arguments.
     assert isinstance(command, (list, tuple))
+    if extra_environ and env:
+        raise ValueError("You cannot pass both 'extra_environ' and 'env'.")
     logger.debug("Running command: '%s'", format_command(command))
-    env = dict(os.environ, PYTHONPATH=os.pathsep.join(sys.path))
-    if extra_environ is not None:
-        env.update(extra_environ)
+    if env is None:
+        env = os.environ.copy()
+        env["PYTHONPATH"] = os.pathsep.join(sys.path)
+        if extra_environ is not None:
+            env.update(extra_environ)
     # By default we show errors, of course.
     show_stderr = True
     if command[0].startswith(sys.executable):
@@ -731,7 +735,7 @@ def get_errors(stderr_output):
 
 
 def execute_command(
-    command, allow_retry=False, fail_message="", cwd=None, extra_environ=None
+    command, allow_retry=False, fail_message="", cwd=None, extra_environ=None, env=None
 ):
     """Run the command and possibly retry it.
 
@@ -740,6 +744,11 @@ def execute_command(
 
     When allow_retry is True, a few things change.
 
+    You can either pass extra_environ options that we then add to a copy of the
+    current OS environment, plus we add the PYTHONPATH.
+    Or you can pass your own env.
+    You can only use one of these two options.
+
     We print interesting lines.  When all is right, this will be the
     first and last few lines, otherwise the full standard output plus
     error output.
@@ -753,7 +762,7 @@ def execute_command(
 
     It might be a warning, but we cannot detect the distinction.
     """
-    result = _execute_command(command, cwd=cwd, extra_environ=extra_environ)
+    result = _execute_command(command, cwd=cwd, extra_environ=extra_environ, env=env)
     if not allow_retry:
         if ERROR_EXIT_CODE in result:
             print(result)


Reviews:
user: login: reinout
body: Looks good!
Only one changelog comment. 

I saw the 3.9 local failure, too, btw. Ignored it as GitHub worked fine. Weird…

Hope everything works again now. 
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-03-08T08:23:09Z
user: login: mauritsvanrees
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-08T20:44:30Z

Review comments:
diff_hunk: @@ -4,7 +4,14 @@ Changelog for zest.releaser
 9.4.1 (unreleased)
 ------------------
 
-- Nothing changed yet.
+- Try hard to determine our own installed version, but do not fail when this does not work.
path: CHANGES.rst
user: login: reinout
body: Too much detail for a changelog entry? The three lines can be summarized as “Worked around python 3.9 version-reporting issue that prevented startup”?
created_at: 2025-03-08T08:20:52Z
author_association: COLLABORATOR
diff_hunk: @@ -4,7 +4,14 @@ Changelog for zest.releaser
 9.4.1 (unreleased)
 ------------------
 
-- Nothing changed yet.
+- Try hard to determine our own installed version, but do not fail when this does not work.
path: CHANGES.rst
user: login: mauritsvanrees
body: Agreed and done.
created_at: 2025-03-08T20:44:30Z
author_association: MEMBER

PR comments:
user: login: mauritsvanrees
created_at: 2025-03-07T23:29:27Z
body: Note to self: would it help on Python 3.9 to always use the extra `importlib-metadata` package instead of `importlib.metadata` from the standard lib?  Just an idea while I was brushing my teeth. :-)
author_association: MEMBER
user: login: reinout
created_at: 2025-03-08T20:55:41Z
body: Released as `9.5.0`.
(Though I did get an error about my branch not being up to date after releasing despite doing a `git pull` beforehand, so I'm looking into that).

I really like the "these are the two files that will be uploaded" message, btw!
author_association: COLLABORATOR
user: login: reinout
created_at: 2025-03-08T20:57:44Z
body: <img width="245" alt="Screenshot 2025-03-08 at 21 56 57" src="https://github.com/user-attachments/assets/5fc9ee58-da9e-4b8b-89aa-1bc14f4a12de" />

We've been fullreleasing at exactly the same time with the same software version, so there were no merge conflicts :-)
author_association: COLLABORATOR