Repository: kedro-org/kedro
Pull request: Remove setuptools dependency
## Description
Remove setuptools dependency as mentioned in #2350 

## Development notes
<!-- What have you changed, and how has this been tested? -->


## Developer Certificate of Origin
We need all contributions to comply with the [Developer Certificate of Origin (DCO)](https://developercertificate.org/). All commits must be signed off by including a `Signed-off-by` line in the commit message. [See our wiki for guidance](https://github.com/kedro-org/kedro/wiki/Guidelines-for-contributing-developers/).

If your PR is blocked due to unsigned commits, then you must follow the instructions under "Rebase the branch" on the GitHub Checks page for your PR. This will retroactively add the sign-off to all unsigned commits and allow the DCO check to pass.

## Checklist

- [ ] Read the [contributing](https://github.com/kedro-org/kedro/blob/main/CONTRIBUTING.md) guidelines
- [ ] Signed off each commit with a [Developer Certificate of Origin (DCO)](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/managing-repository-settings/managing-the-commit-signoff-policy-for-your-repository)
- [ ] Opened this PR as a 'Draft Pull Request' if it is work-in-progress
- [ ] Updated the documentation to reflect the code changes
- [ ] Added a description of this change in the [`RELEASE.md`](https://github.com/kedro-org/kedro/blob/main/RELEASE.md) file
- [ ] Added tests to cover my changes
- [ ] Checked if this change will affect Kedro-Viz, and if so, communicated that with the Viz team


Problem statement:
Remove setuptools from core dependencies
## Description
I'm always frustrated when `pip install kedro` upgrades the versions of `setuptools`, `wheel` and `pip`.

## Context
Usually I would not expect `pip install` to upgrade these packages unless I explicitly run `python3 -m pip install --upgrade pip setuptools wheel`.

The behaviour is even more surprising as `kedro` does not give a first impression of a package that is related to packaging or dependency management.

## Possible Implementation
- `wheel` and `pip` dependencies will probably be excluded when `pip-tools` is removed from the core dependencies (by deprecating `kedro build-reqs`)
-  `setuptools` can be moved to an optional dependency, `kedro[package]` to support the functionality of `kedro package`

## Possible Alternatives
n/a

Hints from issue discussion:
super reasonable suggestion @ZiruiXuQB - I'm actually amazed this hasn't been suggested before!
I wouldn't move `setuptools` to an optional dependency, but rather remove it altogether. Instead of invoking `setup.py` directly, it's preferable to use `python -m build` (as of PEP 517, PEP 518). However, I suppose then `build` becomes a dependency...

Alternatively, you could just deprecate the `kedro package` command, because all it is is a thin wrapper on top of a `setup.py` invocation. I think I'd recommend this, since Kedro went through the [exercise of evaluating CLI command usage](https://github.com/kedro-org/kedro/issues/1293) and deprecating other thin wrappers; not sure why `kedro package` stats don't show up in that report.

Somewhat relevant (but not a solution): https://github.com/kedro-org/kedro/issues/2280
`kedro package`  is just updated with the packaged config, which may improve the usage(?) @deepyaman 

I am adding this to the inbox to make sure we discussed it in backlog grooming. `kedro[pacakge]` sounds reasonable to me, does `micropkg` also requires this? I haven't checked but probably yes?
Pinging @astrojuanlu 
Now that I've been summoned 👹 let me just note that in the next pip release, if `wheel` is not installed and the project does not have a `pyproject.toml`, `pip` will not try `setup.py install` anymore. See https://github.com/pypa/pip/pull/11874 (and related: https://github.com/pypa/pip/pull/11860)

So, I think completing https://github.com/kedro-org/kedro/milestone/36 is becoming more urgent, and after that we could consider this issue.
Fantastic push astro-👹
This is blocked by migrating to `pyproject.toml` (see https://github.com/kedro-org/kedro/milestone/36)
@astrojuanlu Can we close this already? I think for `kedro` it is move to `pyproject.toml` so these are only build requirements now? 


kedro still has a dependency on pip-tools and setuptools:

https://github.com/kedro-org/kedro/blob/664411d568417d414a8efc848a02b0fb08ce1c80/pyproject.toml#L31-L36

IIUC, when we remove `kedro build-reqs` part of this issue will fix itself.

OTOH, `kedro package` now does not depend on `setuptools` anymore, and `build` does not depend on it either:

https://github.com/pypa/build/blob/381ce789e1662e2e16b0ca4be520cdcf7464f5a3/pyproject.toml#L34

I was under the impression that we needed to wait on #2853 to remove our direct dependency on `setuptools`, but maybe we can proceed already. That would fix the other half of the issue.
Double check `micropkg` though, I recall we use setuptools package discovery. The structure of `kedro micropkg package` could be changed into an src-layout to avoid this, but it would be a breaking change. If that's needed to close this issue though, maybe let's do it ahead of 0.19.

Commit message: Remove setuptools

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

Patch to review:
diff --git a/kedro/framework/cli/micropkg.py b/kedro/framework/cli/micropkg.py
index dcdfc5d99e..09fdfe7ef8 100644
--- a/kedro/framework/cli/micropkg.py
+++ b/kedro/framework/cli/micropkg.py
@@ -21,7 +21,6 @@
 from rope.contrib import generate
 from rope.refactor.move import MoveModule
 from rope.refactor.rename import Rename
-from setuptools.discovery import FlatLayoutPackageFinder
 
 from build.util import project_wheel_metadata
 from kedro.framework.cli.pipeline import (
@@ -227,9 +226,11 @@ def _pull_package(  # noqa: PLR0913
         # However, the rest of the code expects the non-normalized package name,
         # so we have to find it.
         packages = [
-            package
-            for package in FlatLayoutPackageFinder().find(project_root_dir)
-            if "." not in package
+            elem.name
+            for elem in project_root_dir.iterdir()
+            if elem.is_dir()
+            and elem.name != "tests"
+            and (elem / "__init__.py").exists()
         ]
         if len(packages) != 1:
             # Should not happen if user is calling `micropkg pull`
diff --git a/pyproject.toml b/pyproject.toml
index d0f024700a..ee63ae5316 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -31,7 +31,6 @@ dependencies = [
     "PyYAML>=4.2,<7.0",
     "rich>=12.0,<14.0",
     "rope>=0.21,<2.0",  # subject to LGPLv3 license
-    "setuptools>=65.5.1",
     "toml>=0.10.0",
     "toposort>=1.5",  # Needs to be at least 1.5 to be able to raise CircularDependencyError
 ]


Code review comments:
diff_hunk: @@ -227,9 +226,11 @@ def _pull_package(  # noqa: PLR0913
         # However, the rest of the code expects the non-normalized package name,
         # so we have to find it.
         packages = [
-            package
-            for package in FlatLayoutPackageFinder().find(project_root_dir)
-            if "." not in package
+            elem.name
path: kedro/framework/cli/micropkg.py
text: @user1:
`elem` isn't a very descriptive variable name, can you rename it to be clearer? Something like project_item maybe?

@author:
Thanks, fixed!