Repository: pdm-project/pdm

Issue:
`add --dev --group test` ignores `dev` if empty group with that name already exists in optional-dependencies
- [x] I have searched the issue tracker and believe that this is not a duplicate.

## Steps to reproduce

### Tl;DR
**If a package has previously been added as an optional dependency, and then removed and re-added as a dev dependency with the same group name (which is currently empty), then it gets added back to `optional-dependencies` instead of `dev-dependencies`.**

1. Create a new project with `pdm init`, accept defaults.
   - In my case I selected "yes" for 'Do you want to build this project for distribution(such as wheel)?

2. First add `pytest` (for example) as a test dependency: `pdm add --group test pytest`. pyproject.toml will now have this:
   ```toml
   [project.optional-dependencies]
   test = [
       "pytest>=8.3.2",
   ]
   
   [build-system]
   requires = ["setuptools>=61", "wheel"]
   build-backend = "setuptools.build_meta"
   
   [tool.pdm]
   distribution = true
   ```

3. At this point the user (me) realises that instead it should be a [dev-only dependency](https://pdm-project.org/latest/usage/dependency/#add-development-only-dependencies), so I do: `pdm remove -v --group test pytest`. Resulting section in pyproject.toml is still have the empty group 'test':
   ```toml
   [project.optional-dependencies]
   test = [
   ]
   ```

4. Now, I add it as a `dev` dependency using `pdm add -v --dev --group test pytest` but in pyproj it's added back to the old section with the pre-existing group 'test':
   ```toml
   [project.optional-dependencies]
   test = [
       "pytest>=8.3.2",
   ]
   ```

5. Remove it _again_ from the group using  _similar-but-not-same command_ as Ste 3: `pdm remove -v --dev --group test pytest` and **also fully delete the section `[project.optional-dependencies]`** in `pyproject.toml`

6. Again, add it as a `dev` dependency using `pdm add -v --dev --group test pytest` - exactly the same command as in step 4.
   - This time, it goes into a new and correct section with that group name:
   ```toml
   [tool.pdm.dev-dependencies]
   test = [
       "pytest>=8.3.2",
   ]
   ```

## Actual behavior

(Step 4 above.)

## Expected behavior

(Step 6 above.) ➕

The documentation states:
> The same group name MUST NOT appear in both `[tool.pdm.dev-dependencies]` and `[project.optional-dependencies]`.
Which is the **likely cause of the issue**. However, I did remove that dependency and then re-add it using the expected-correct commands. But because the `test` group name already existed as an **empty list** in optional-deps, it just got added there again.

**Potential fixes:**
- Remove empty groups in `*-dependencies` to allow moving packages & groups between optional & dev deps.
  - Otherwise, it requires the user to manually modify the deps sections in `pyproject.toml` - which is fine if they realised that was the cause.
- Or, **Issue a warning** that the group with the same name already exists in the "other-deps" section (whichever _other_ is) and will be added there instead of where the user stated in the command.

## Environment Information

```bash
PS C:\some\folders\pdm-bug> pdm info && pdm info --env
PDM version:
  2.18.1
Python Interpreter:
  C:\some\folders\pdm-bug\.venv\Scripts\python.exe (3.12)
Project Root:
  C:/some/folders/pdm-bug
Local Packages:

{
  "implementation_name": "cpython",
  "implementation_version": "3.12.5",
  "os_name": "nt",
  "platform_machine": "AMD64",
  "platform_release": "10",
  "platform_system": "Windows",
  "platform_version": "10.0.19045",
  "python_full_version": "3.12.5",
  "platform_python_implementation": "CPython",
  "python_version": "3.12",
  "sys_platform": "win32"
}
PS C:\some\folders\pdm-bug>
```



Discussion:
Also, just for further verification, I've now also **reproduced the issue in reverse**: If the group name exists in dev-deps and then is removed & added as regular optional-dep, it still gets added back to dev-deps:

1. After step 6 above, pyproject.toml has:
   ```toml
   [tool.pdm.dev-dependencies]
   test = [
       "pytest>=8.3.2",
   ]
   ```

2. Then execute: `pdm remove -v --dev --group test pytest`:
   ```toml
   [tool.pdm.dev-dependencies]
   test = [
   ]
   ```
3. After `pdm add -v --group test pytest`, it's not in opt-deps
   ```toml
   [tool.pdm.dev-dependencies]
   test = [
       "pytest>=8.3.2",
   ]
   ```

4. pdm-Removing it again; then manually deleting only the `test` group from `dev-dependencies` which is now empty; and re-adding it with `pdm add -v --group test pytest` also works. And notably, it deletes the now-empty `tool.pdm.dev-dependencies` section too.




Patch:
diff --git a/news/3133.bugfix.md b/news/3133.bugfix.md
new file mode 100644
index 0000000000..857f464989
--- /dev/null
+++ b/news/3133.bugfix.md
@@ -0,0 +1,1 @@
+Remove empty groups when removing packages with `pdm remove`.
diff --git a/src/pdm/cli/commands/remove.py b/src/pdm/cli/commands/remove.py
index 49804b73f4..13ae5b5e3a 100644
--- a/src/pdm/cli/commands/remove.py
+++ b/src/pdm/cli/commands/remove.py
@@ -118,6 +118,10 @@ def do_remove(
         if lock_groups and group not in lock_groups:
             project.core.ui.warn(f"Group [success]{group}[/] isn't in lockfile, skipping lock.")
             return
+        # It may remove the whole group, exclude it from lock groups first
+        project_groups = project.iter_groups()
+        if lock_groups is not None:
+            lock_groups = [g for g in lock_groups if g in project_groups]
         do_lock(project, "reuse", dry_run=dry_run, tracked_names=tracked_names, hooks=hooks, groups=lock_groups)
         if sync:
             do_sync(
diff --git a/src/pdm/project/core.py b/src/pdm/project/core.py
index 823ab63879..0a3ba215a8 100644
--- a/src/pdm/project/core.py
+++ b/src/pdm/project/core.py
@@ -556,7 +556,10 @@ def use_pyproject_dependencies(
         def update_dev_dependencies(deps: list[str]) -> None:
             from tomlkit.container import OutOfOrderTableProxy
 
-            settings.setdefault("dev-dependencies", {})[group] = deps
+            if deps:
+                settings.setdefault("dev-dependencies", {})[group] = deps
+            else:
+                settings.setdefault("dev-dependencies", {}).pop(group, None)
             if isinstance(self.pyproject._data["tool"], OutOfOrderTableProxy):
                 # In case of a separate table, we have to remove and re-add it to make the write correct.
                 # This may change the order of tables in the TOML file, but it's the best we can do.
@@ -570,7 +573,9 @@ def update_dev_dependencies(deps: list[str]) -> None:
         deps_setter = [
             (
                 metadata.get("optional-dependencies", {}),
-                lambda x: metadata.setdefault("optional-dependencies", {}).__setitem__(group, x),
+                lambda x: metadata.setdefault("optional-dependencies", {}).__setitem__(group, x)
+                if x
+                else metadata.setdefault("optional-dependencies", {}).pop(group, None),
             ),
             (settings.get("dev-dependencies", {}), update_dev_dependencies),
         ]


Test patch:
diff --git a/tests/cli/test_remove.py b/tests/cli/test_remove.py
index 2dc5204a7e..faa07492c1 100644
--- a/tests/cli/test_remove.py
+++ b/tests/cli/test_remove.py
@@ -69,7 +69,7 @@ def test_remove_package_exist_in_multi_groups(project, working_set, pdm):
     pdm(["add", "requests"], obj=project, strict=True)
     pdm(["add", "--dev", "urllib3"], obj=project, strict=True)
     pdm(["remove", "--dev", "urllib3"], obj=project, strict=True)
-    assert all("urllib3" not in line for line in project.pyproject.settings["dev-dependencies"]["dev"])
+    assert "dev-dependencies" not in project.pyproject.settings
     assert "urllib3" in working_set
     assert "requests" in working_set
 
@@ -103,5 +103,5 @@ def test_remove_group_not_in_lockfile(project, pdm, mocker):
     assert project.lockfile.groups == ["default"]
     locker = mocker.patch.object(actions, "do_lock")
     pdm(["remove", "--group", "tz", "pytz"], obj=project, strict=True)
-    assert not project.pyproject.metadata["optional-dependencies"].get("tz")
+    assert "optional-dependencies" not in project.pyproject.metadata
     locker.assert_not_called()