Repository: pdm-project/pdm

Issue:
Dependency Groups from Poetry do not migrate properly to PDM
I tried to migrate a project from Poetry that used [dependency groups](https://python-poetry.org/docs/master/managing-dependencies/#dependency-groups) and the groups were not migrated correctly to PDM. 

- [x] I have searched the issue tracker and believe that this is not a duplicate.

**Make sure you run commands with `-v` flag before pasting the output.**

## Steps to reproduce

1. Create a new directory with nothing in it
1. Add the following `pyproject.toml`

```toml
[build-system]
build-backend = "poetry.core.masonry.api"
requires = ["poetry-core"]

[tool.poetry]
name = "example-project"
version = "2.0.0"
description = ""
authors = ["Amit Parekh <7276308+amitkparekh@users.noreply.github.com>"]
packages = [{ from = "src", include = "example_project" }]

[tool.poetry.dependencies]
python = "^3.11"
torch = "^2.0.1"
torchdata = "^0.6.1"
orjson = "^3.8.5"
pydantic = "^1.10.4"
rich = "*"
loguru = "*"
typer = "*"

[tool.poetry.group.lint.dependencies]
black = "*"
mypy = "*"
pre-commit = "*"
ruff = "*"

[tool.poetry.group.tests.dependencies]
pytest = "*"
pytest-cases = "*"
pytest-cov = "*"
```

2. Run `pdm init` to migrate the project to PDM

```bash
> pdm init -v
pyproject.toml already exists, update it now.
Please enter the Python interpreter to use
0. /Users/amit/.pyenv/shims/python3 (3.11)
1. /usr/local/bin/python3 (3.11)
2. /Users/amit/.pyenv/versions/3.11.4/bin/python3.11 (3.11)
3. /Users/amit/.local/share/rtx/installs/python/3.11.5/bin/python (3.11)
Please select (0): 3
Would you like to create a virtualenv with /Users/amit/.local/share/rtx/installs/python/3.11.5/bin/python? [y/n] (y):
Cleaning existing target directory /Users/amit/Develop/test-prob/.venv
Run command: ['/Users/amit/.local/share/rtx/installs/pdm/2.9.2/venv/bin/python', '-m', 'virtualenv', '/Users/amit/Develop/test-prob/.venv',
'-p', '/Users/amit/.local/share/rtx/installs/python/3.11.5/bin/python', '--prompt=test-prob-3.11']
created virtual environment CPython3.11.5.final.0-64 in 331ms
  creator CPython3Posix(dest=/Users/amit/Develop/test-prob/.venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/amit/Library/Application Support/virtualenv)
    added seed packages: pip==23.2.1, setuptools==68.2.0, wheel==0.41.2
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
Virtualenv is created successfully at /Users/amit/Develop/test-prob/.venv
Is the project a library that is installable?
If yes, we will need to ask a few more questions to include the project name and build backend [y/n] (n):
License(SPDX name) (MIT):
Author name (Amit Parekh):
Author email (7276308+amitkparekh@users.noreply.github.com):
Python requires('*' to allow any) (>=3.11):
Project is initialized successfully
Found following files from other formats that you may import:
0. /Users/amit/Develop/test-prob/pyproject.toml (poetry)
1. don't do anything, I will import later.
Please select: 0
Changes are written to pyproject.toml.
```

<!--Describe the minimized example of how to reproduce the bug-->

## Actual behavior

A `pyproject.toml` is updated to be the following

```toml
[tool.poetry]
name = "example-project"
version = "2.0.0"
description = ""
authors = ["Amit Parekh <7276308+amitkparekh@users.noreply.github.com>"]
packages = [{ from = "src", include = "example_project" }]

[tool.poetry.dependencies]
python = "^3.11"
torch = "^2.0.1"
torchdata = "^0.6.1"
orjson = "^3.8.5"
pydantic = "^1.10.4"
rich = "*"
loguru = "*"
typer = "*"

[tool.poetry.group.lint.dependencies]
black = "*"
mypy = "*"
pre-commit = "*"
ruff = "*"

[tool.poetry.group.tests.dependencies]
pytest = "*"
pytest-cases = "*"
pytest-cov = "*"


[tool.pdm.build]
includes = ["src/example_project"]
[project]
name = "example-project"
version = "2.0.0"
description = ""
authors = [
    {name = "Amit Parekh", email = "7276308+amitkparekh@users.noreply.github.com"},
]
dependencies = [
    "torch<3.0.0,>=2.0.1",
    "torchdata<1.0.0,>=0.6.1",
    "orjson<4.0.0,>=3.8.5",
    "pydantic<2.0.0,>=1.10.4",
    "rich",
    "loguru",
    "typer",
]
requires-python = ">=3.11,<4.0"
readme = "README.md"
license = {text = "MIT"}


[project.group]
[project.group.lint.dependencies]
black = "*"
mypy = "*"
pre-commit = "*"
ruff = "*"

[project.group.tests.dependencies]
pytest = "*"
pytest-cases = "*"
pytest-cov = "*"

[build-system]
requires = ["pdm-backend"]
build-backend = "pdm.backend"
```

<!--A clear and concise description the result of the above steps-->

## Expected behavior

The development dependencies were migrated incorrectly. The `pyproject.toml` should be like this: 

```toml
[tool.poetry]
name = "example-project"
version = "2.0.0"
description = ""
authors = ["Amit Parekh <7276308+amitkparekh@users.noreply.github.com>"]
packages = [{ from = "src", include = "example_project" }]

[tool.poetry.dependencies]
python = "^3.11"
torch = "^2.0.1"
torchdata = "^0.6.1"
orjson = "^3.8.5"
pydantic = "^1.10.4"
rich = "*"
loguru = "*"
typer = "*"

[tool.poetry.group.lint.dependencies]
black = "*"
mypy = "*"
pre-commit = "*"
ruff = "*"

[tool.poetry.group.tests.dependencies]
pytest = "*"
pytest-cases = "*"
pytest-cov = "*"

[tool.pdm.build]
includes = ["src/example_project"]

[tool.pdm.dev-dependencies]
test = ["pytest", "pytest-cases", "pytest-cov"]
lint = ["black", "ruff", "mypy", "pre-commit"]

[project]
name = "example-project"
version = "2.0.0"
description = ""
authors = [
    { name = "Amit Parekh", email = "7276308+amitkparekh@users.noreply.github.com" },
]
dependencies = [
    "torch<3.0.0,>=2.0.1",
    "torchdata<1.0.0,>=0.6.1",
    "orjson<4.0.0,>=3.8.5",
    "pydantic<2.0.0,>=1.10.4",
    "rich",
    "loguru",
    "typer",
]
requires-python = ">=3.11,<4.0"
readme = "README.md"
license = { text = "MIT" }


[build-system]
requires = ["pdm-backend"]
build-backend = "pdm.backend"
```

<!--A clear and concise description of what you expected to happen.-->

## Environment Information

```bash
# Paste the output of `pdm info && pdm info --env` below:
PDM version:
  2.9.2
Python Interpreter:
  /Users/amit/Develop/test-prob/.venv/bin/python (3.11)
Project Root:
  /Users/amit/Develop/test-prob
Local Packages:

{
  "implementation_name": "cpython",
  "implementation_version": "3.11.5",
  "os_name": "posix",
  "platform_machine": "x86_64",
  "platform_release": "22.6.0",
  "platform_system": "Darwin",
  "platform_version": "Darwin Kernel Version 22.6.0: Fri Sep 15 13:39:52 PDT 2023; root:xnu-8796.141.3.700.8~1/RELEASE_X86_64",
  "python_full_version": "3.11.5",
  "platform_python_implementation": "CPython",
  "python_version": "3.11",
  "sys_platform": "darwin"
}

```

I haven't run `pdm install` yet after this step. 

Discussion:
codecov[bot]: ## [Codecov](https://app.codecov.io/gh/pdm-project/pdm/pull/2311?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project) Report
All modified lines are covered by tests :white_check_mark:

| [Files](https://app.codecov.io/gh/pdm-project/pdm/pull/2311?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project) | Coverage Δ | |
|---|---|---|
| [src/pdm/cli/commands/add.py](https://app.codecov.io/gh/pdm-project/pdm/pull/2311?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project#diff-c3JjL3BkbS9jbGkvY29tbWFuZHMvYWRkLnB5) | `87.50% <100.00%> (+0.17%)` | :arrow_up: |
| [src/pdm/formats/poetry.py](https://app.codecov.io/gh/pdm-project/pdm/pull/2311?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project#diff-c3JjL3BkbS9mb3JtYXRzL3BvZXRyeS5weQ==) | `86.09% <100.00%> (+0.47%)` | :arrow_up: |


:loudspeaker: Thoughts on this report? [Let us know!](https://about.codecov.io/pull-request-comment-report/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=pdm-project).


Patch:
diff --git a/news/2285.bugfix.md b/news/2285.bugfix.md
new file mode 100644
index 00000000..cdb3966e
--- /dev/null
+++ b/news/2285.bugfix.md
@@ -0,0 +1,1 @@
+Fix a bug that dependency groups from Poetry 1.2+ do not migrate properly to PDM.
diff --git a/src/pdm/cli/commands/update.py b/src/pdm/cli/commands/update.py
index a3981917..8799f811 100644
--- a/src/pdm/cli/commands/update.py
+++ b/src/pdm/cli/commands/update.py
@@ -126,6 +126,7 @@ class Command(BaseCommand):
         if not packages:
             if prerelease:
                 raise PdmUsageError("--prerelease must be used with packages given")
+            selection.validate()
             for group in selection:
                 updated_deps[group] = all_dependencies[group]
         else:
diff --git a/src/pdm/formats/poetry.py b/src/pdm/formats/poetry.py
index 12487530..79a4d615 100644
--- a/src/pdm/formats/poetry.py
+++ b/src/pdm/formats/poetry.py
@@ -143,9 +143,17 @@ class PoetryMetaConverter(MetaConverter):
 
     @convert_from("dev-dependencies")
     def dev_dependencies(self, value: dict) -> None:
-        self.settings["dev-dependencies"] = {
-            "dev": make_array([r for key, req in value.items() for r in _convert_req(key, req)], True),
-        }
+        self.settings.setdefault("dev-dependencies", {})["dev"] = make_array(
+            [r for key, req in value.items() for r in _convert_req(key, req)], True
+        )
+        raise Unset()
+
+    @convert_from("group")
+    def group_dependencies(self, value: dict[str, dict[str, Any]]) -> None:
+        for name, group in value.items():
+            self.settings.setdefault("dev-dependencies", {})[name] = make_array(
+                [r for key, req in group.get("dependencies", {}).items() for r in _convert_req(key, req)], True
+            )
         raise Unset()
 
     @convert_from()


Test patch:
diff --git a/tests/fixtures/poetry-new.toml b/tests/fixtures/poetry-new.toml
new file mode 100644
index 00000000..607ee757
--- /dev/null
+++ b/tests/fixtures/poetry-new.toml
@@ -0,0 +1,20 @@
+[tool.poetry]
+name = "test-poetry"
+version = "0.1.0"
+description = ""
+authors = ["Frost Ming <me@frostming.com>"]
+readme = "README.md"
+packages = [{include = "test_poetry"}]
+
+[tool.poetry.dependencies]
+python = "^3.8"
+httpx = "*"
+pendulum = "*"
+
+[tool.poetry.group.test.dependencies]
+pytest = "^6.0.0"
+pytest-mock = "*"
+
+[build-system]
+requires = ["poetry-core"]
+build-backend = "poetry.core.masonry.api"
diff --git a/tests/test_formats.py b/tests/test_formats.py
index 204ae531..ea4c1728 100644
--- a/tests/test_formats.py
+++ b/tests/test_formats.py
@@ -87,6 +87,15 @@ def test_convert_poetry(project):
     assert build["excludes"] == ["my_package/excluded.py"]
 
 
+def test_convert_poetry_12(project):
+    golden_file = FIXTURES / "poetry-new.toml"
+    with cd(FIXTURES):
+        result, settings = poetry.convert(project, golden_file, Namespace(dev=False, group=None))
+
+    assert result["dependencies"] == ["httpx", "pendulum"]
+    assert settings["dev-dependencies"]["test"] == ["pytest<7.0.0,>=6.0.0", "pytest-mock"]
+
+
 def test_convert_flit(project):
     golden_file = FIXTURES / "projects/flit-demo/pyproject.toml"
     assert flit.check_fingerprint(project, golden_file)