Repository: gerbenoostra/poetry-plugin-mono-repo-deps

Issue:
Missing changes in the pyproject.toml file
Hi,

I have been trying to use your extension, which should - on paper - solve my issue of multiple inter-related packages being developed in a mono-repo.

However, I was unable to publish to PyPi.

My pyproject.toml file contains the following:

```
[tool.poetry]
name = "media-muncher"
version = "1.2.0"
description = "A module to work with media assets, in particular DASH and HLS streams"
authors = ["Fabre Lambeau <fabre.lambeau@broadpeak.tv>"]
license = "MIT"
readme = "README.md"
packages = [{ include = "media_muncher" }]

[tool.poetry.dependencies]
python = "^3.9"
isodate = "^0.6.1"
mpd-inspector = { path = "../../mpd-inspector", develop = true}

[tool.poetry-monorepo.deps]
enabled = true
commands = ["build", "export"]
constraint = "~="
source_types = ["file", "directory"]
only_develop = false

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
```

When I run `poetry build`, I do see the line indicating that the plugin performed the replacement:

```
# Replacing path dependency mpd-inspector @ file:///Users/fabrelambeau/Broadpeak/Code/mpd-inspector in group main with mpd-inspector (>=0.8.0,<0.9.0)
Building media-muncher (1.2.0)
  - Building sdist
  - Built media_muncher-1.2.0.tar.gz
  - Building wheel
  - Built media_muncher-1.2.0-py3-none-any.whl
```

However, when I run `poetry publish`, I get the following error:

```
HTTP Error 400: Can't have direct dependency: media_muncher@ file:///Users/fabrelambeau/Broadpeak/Code/bpkio-python-tools/media-muncher. See https://packaging.python.org/specifications/core-metadata for more information. | b"<html>\n <head>\n  <title>400 Can't have direct dependency: media_muncher@ file:///Users/fabrelambeau/Broadpeak/Code/bpkio-python-tools/media-muncher. See https://packaging.python.org/specifications/core-metadata for more information.\n \n <body>\n  <h1>400 Can't have direct dependency: media_muncher@ file:///Users/fabrelambeau/Broadpeak/Code/bpkio-python-tools/media-muncher. See https://packaging.python.org/specifications/core-metadata for more information.\n  The server could not comply with the request since it is either malformed or otherwise incorrect.<br/><br/>\nCan&#x27;t have direct dependency: media_muncher@ file:///Users/fabrelambeau/Broadpeak/Code/bpkio-python-tools/media-muncher. See https://packaging.python.org/specifications/core-metadata for more information.\n\n\n \n"
```

I had a look at the content of the tar.gz file and although I can see that the replacement was done in the PKG-INFO file, I can also see that it contains the pyproject.toml file unmodified. I am *guessing* that this is the issue.

Discussion:
codecov[bot]: ## [Codecov](https://app.codecov.io/gh/gerbenoostra/poetry-plugin-mono-repo-deps/pull/24?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Gerben+Oostra) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 100.00%. Comparing base [(`6e34350`)](https://app.codecov.io/gh/gerbenoostra/poetry-plugin-mono-repo-deps/commit/6e34350e2997023b0240b13bdf69bb335db2424a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Gerben+Oostra) to head [(`e6444df`)](https://app.codecov.io/gh/gerbenoostra/poetry-plugin-mono-repo-deps/commit/e6444df73694b7168a5a83da6c6c3c70b7b8bda2?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Gerben+Oostra).

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff            @@
##              main       #24   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            2         2           
  Lines          146       190   +44     
  Branches        20        25    +5     
=========================================
+ Hits           146       190   +44     
```

| [Flag](https://app.codecov.io/gh/gerbenoostra/poetry-plugin-mono-repo-deps/pull/24/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Gerben+Oostra) | Coverage Δ | |
|---|---|---|
| [unittests](https://app.codecov.io/gh/gerbenoostra/poetry-plugin-mono-repo-deps/pull/24/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Gerben+Oostra) | `100.00% <100.00%> (ø)` | |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Gerben+Oostra#carryforward-flags-in-the-pull-request-comment) to find out more.

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/gerbenoostra/poetry-plugin-mono-repo-deps/pull/24?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Gerben+Oostra).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Gerben+Oostra).


Patch:
diff --git a/poetry_plugin_mono_repo_deps/plugin.py b/poetry_plugin_mono_repo_deps/plugin.py
index 7c2665c..153cf13 100644
--- a/poetry_plugin_mono_repo_deps/plugin.py
+++ b/poetry_plugin_mono_repo_deps/plugin.py
@@ -5,7 +5,8 @@ from dataclasses import dataclass
 from typing import Any, Dict, List, TypeVar, cast
 
 from cleo.events.console_event import ConsoleEvent
-from cleo.events.console_events import COMMAND
+from cleo.events.console_events import COMMAND, TERMINATE
+from cleo.events.console_terminate_event import ConsoleTerminateEvent
 from cleo.events.event import Event
 from cleo.events.event_dispatcher import EventDispatcher
 from cleo.io.io import IO
@@ -16,6 +17,7 @@ from poetry.plugins.application_plugin import ApplicationPlugin
 from poetry.poetry import Poetry
 from poetry.repositories.lockfile_repository import LockfileRepository
 from poetry.utils.helpers import merge_dicts
+from tomlkit import TOMLDocument
 
 T = TypeVar("T")
 
@@ -83,12 +85,14 @@ def load_config(poetry: Poetry) -> Config | None:
 class MonoRepoDepsPlugin(ApplicationPlugin):
     def __init__(self) -> None:
         super().__init__()
+        self._original_toml_data: TOMLDocument | None = None
 
     def activate(self, application: Application) -> None:
         self._application = application
         dispatcher = application.event_dispatcher
         if dispatcher is not None:
             dispatcher.add_listener(COMMAND, self.handle_command)
+            dispatcher.add_listener(TERMINATE, self.handle_terminate)
         else:  # pragma: no cover
             pass
 
@@ -128,6 +132,8 @@ class MonoRepoDepsPlugin(ApplicationPlugin):
 
         # for build
         self.update_locked_repository(io, config)
+        if command.name != "export":
+            self.update_pyproject_toml(config)
         # for export
         self.update_lock_data(config)
         return None
@@ -160,10 +166,67 @@ class MonoRepoDepsPlugin(ApplicationPlugin):
         """Updates the lockers internal lock data, necessary for commands like `export`"""
         poetry = self._application.poetry
         locked_packages = cast(List[Dict[str, Any]], poetry._locker.lock_data["package"])
+
         for info in locked_packages:
             if is_to_be_replaced_package_lock(config, info):
                 modify_locked_package_to_named(config, info, locked_packages)
 
+    def update_pyproject_toml(self, config: Config) -> None:
+        """Updates the pyproject.toml file, necessary for commands like `build`"""
+        poetry = self._application.poetry
+        pyproject = poetry.pyproject
+
+        toml_data: TOMLDocument = pyproject.data
+        self._original_toml_data = deepcopy(toml_data)
+        poetry_config = pyproject.poetry_config
+        # used to retrieve the current version of the package
+        locked_packages = cast(List[Dict[str, Any]], poetry._locker.lock_data["package"])
+        # update all possible dependency sections in the pyproject.toml
+        update_locked_dependencies(config, poetry_config.get("dependencies", {}), locked_packages)
+        update_locked_dependencies(config, poetry_config.get("dev-dependencies", {}), locked_packages)
+        if "group" in poetry_config:
+            for group_config in poetry_config["group"].values():
+                update_locked_dependencies(config, group_config.get("dependencies", {}), locked_packages)
+        # don't need to assign it back to pyproject.data, as we've modified the data structure in place
+        # writes the modified pyproject to disk, will be restored after the command by `restore_pyproject_toml`
+        pyproject.save()
+
+    def restore_pyproject_toml(self) -> None:
+        """Restores the pyproject.toml file, necessary for commands like `build`"""
+        toml_data = self._original_toml_data
+        if toml_data is None:
+            # we apparently didn't save it
+            return
+        pyproject = self._application.poetry.pyproject
+        pyproject._toml_document = toml_data
+        pyproject.save()
+        self._original_toml_data = None
+
+    def handle_terminate(self, event: Event, _event_name: str, _dispatcher: EventDispatcher) -> None:
+        try:
+            poetry = self._application.poetry
+        except RuntimeError:
+            # should only happen if poetry runs outside poetry folder structure
+            # as we modify poetry lock files, the plugin can only work in poetry folders
+            # and is only relevant for commands that work in poetry folders
+            # thus, either the command is not relevant
+            # or, the command would fail itself
+            return
+
+        config = load_config(poetry)
+        if config is None:  # pragma: no cover
+            return
+
+        event = cast(ConsoleTerminateEvent, event)  # because we listen to TERMINATEs
+        command = event.command
+        if command.name not in config.commands:
+            # Skipped for export, but that's handled by restoration
+            # which only restores if we modified the pyproject.toml file
+            return
+        # for build
+        self.restore_pyproject_toml()
+        return None
+
 
 def is_to_be_replaced_package_lock(config: Config, locked_package_data: dict[str, Any]) -> bool:
     source = locked_package_data.get("source", {})
@@ -240,10 +303,16 @@ def modify_locked_package_to_named(
     if is_to_be_replaced_package_lock(config, info):
         _modify_locked_package_to_named(info)
         # remove path and develop from dependencies of dependencies
-        for dep_name, dep in info.get("dependencies", {}).items():
-            if is_to_be_replaced_dependency_lock(config, dep):
-                dep_version = get_current_locked_version(all_locked_packages, dep_name, "*")
-                _modify_locked_dependency_to_named(dep, dep_version)
+        update_locked_dependencies(config, info.get("dependencies", {}), all_locked_packages)
+
+
+def update_locked_dependencies(
+    config: Config, dependencies: dict[str, Any], all_locked_packages: list[dict[str, Any]]
+) -> None:
+    for dep_name, dep in dependencies.items():
+        if is_to_be_replaced_dependency_lock(config, dep):
+            dep_version = get_current_locked_version(all_locked_packages, dep_name, "*")
+            _modify_locked_dependency_to_named(dep, dep_version)
 
 
 def get_current_locked_version(locked_packaged: list[dict[str, Any]], name: str, default: str) -> str:


Test patch:
diff --git a/tests/test_commands_e2e.py b/tests/test_commands_e2e.py
index 50d1c85..e03c374 100644
--- a/tests/test_commands_e2e.py
+++ b/tests/test_commands_e2e.py
@@ -103,6 +103,37 @@ def test_build_artifact(fixture_simple_a: Path, module_dir: str) -> None:
             pkg_info_content = list(b.decode("utf-8").strip(os.linesep) for b in pkg_info_bytes)
             validate_package_metadata(setup, pkg_info_content)
 
+        pyproject_stream = tar.extractfile(f"{package_name}-0.0.1/pyproject.toml")
+        assert pyproject_stream is not None
+        with pyproject_stream:
+            pyproject_bytes = pyproject_stream.readlines()
+            pyproject_content = list(b.decode("utf-8").strip(os.linesep) for b in pyproject_bytes)
+            validate_pyproject_content(setup, pyproject_content)
+
+    # verifying that we didn't modify the original pyproject.toml
+    with open("pyproject.toml") as f:
+        original_content = f.readlines()
+        setup_disabled = TestSetup(enabled=False, deps=setup.deps, transitive_deps=setup.transitive_deps)
+        validate_pyproject_content(setup_disabled, original_content)
+
+
+def validate_pyproject_content(setup: TestSetup, content: list[str]) -> None:
+    _logger.info("Validating pyproject content:")
+    _logger.info("\n".join(content))
+    # path dependency should be replaced by named dependency
+    for dep in setup.deps or []:
+        dep_line_candidates = [line for line in content if line.strip().startswith(f"{dep.name} = ")]
+        # we expect only one in our test setups
+        # (could fail if we would reuse the lib name, for example as extras identifier)
+        # (therefore we've constructed the pyproject.toml's to not reuse the same name)
+        assert len(dep_line_candidates) == 1
+        dep_line = dep_line_candidates[0]
+        if setup.enabled:
+            assert dep.version in dep_line
+            dep.export_line
+        else:
+            assert dep.version not in dep_line
+
 
 def validate_package_metadata(setup: TestSetup, content: list[str]) -> None:
     _logger.info("Validating package metadata:")