internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: pypa/setuptools

Issue:
pyproject.toml should be included in sdist by default
An issue came up at work recently where installing [python-daemon](https://pagure.io/python-daemon) with `pip` was hitting the `easy_install` path. I assumed it was because `python-daemon` didn't have a `pyproject.toml`, but it turns out they do - it's just not included in the `Manifest.in`, so it didn't get included in the `sdist`. To verify this I created a super basic example project and it does seem that the default is to exclude `pyproject.toml`.

Given that `pyproject.toml` is one of the most important files to have when executing a build, it should be included in the sdist unless explicitly excluded.


Discussion:

Resolving patch:
diff --git a/setuptools/command/sdist.py b/setuptools/command/sdist.py
--- a/setuptools/command/sdist.py
+++ b/setuptools/command/sdist.py
@@ -121,6 +121,14 @@ def __read_template_hack(self):
     if has_leaky_handle:
         read_template = __read_template_hack
 
+    def _add_defaults_optional(self):
+        if six.PY2:
+            sdist_add_defaults._add_defaults_optional(self)
+        else:
+            super()._add_defaults_optional()
+        if os.path.isfile('pyproject.toml'):
+            self.filelist.append('pyproject.toml')
+
     def _add_defaults_python(self):
         """getting python files"""
         if self.distribution.has_pure_modules():


Test patch:
diff --git a/setuptools/tests/test_sdist.py b/setuptools/tests/test_sdist.py
--- a/setuptools/tests/test_sdist.py
+++ b/setuptools/tests/test_sdist.py
@@ -449,6 +449,20 @@ def test_sdist_with_latin1_encoded_filename(self):
             except UnicodeDecodeError:
                 filename not in cmd.filelist.files
 
+    def test_pyproject_toml_in_sdist(self):
+        """
+        Check if pyproject.toml is included in source distribution if present
+        """
+        open(os.path.join(self.temp_dir, 'pyproject.toml'), 'w').close()
+        dist = Distribution(SETUP_ATTRS)
+        dist.script_name = 'setup.py'
+        cmd = sdist(dist)
+        cmd.ensure_finalized()
+        with quiet():
+            cmd.run()
+        manifest = cmd.filelist.files
+        assert 'pyproject.toml' in manifest
+
 
 def test_default_revctrl():
     """
Repository: pypa/setuptools

Issue:
v40.7.1 omits port in index download
In this commit https://github.com/pypa/setuptools/commit/3a351aa00e2110c3b3df0302b1ed6bcf66a1f8d8 is used host instead of netloc, but host does not contain port. Version 40.6.3 works OK.
v40.7.1 omits port in index download
In this commit https://github.com/pypa/setuptools/commit/3a351aa00e2110c3b3df0302b1ed6bcf66a1f8d8 is used host instead of netloc, but host does not contain port. Version 40.6.3 works OK.


Discussion:
Sorry about the regression. After re-reading the diff, I can see the issue, which was probably brought on by the naming of 'host' when it meant 'netloc'. Fix should be straightforward.
Sorry about the regression. After re-reading the diff, I can see the issue, which was probably brought on by the naming of 'host' when it meant 'netloc'. Fix should be straightforward.

Resolving patch:
diff --git a/setuptools/package_index.py b/setuptools/package_index.py
--- a/setuptools/package_index.py
+++ b/setuptools/package_index.py
@@ -1072,7 +1072,7 @@ def open_with_auth(url, opener=urllib.request.urlopen):
 
     if auth:
         auth = "Basic " + _encode_auth(auth)
-        parts = scheme, parsed.hostname, path, params, query, frag
+        parts = scheme, netloc, path, params, query, frag
         new_url = urllib.parse.urlunparse(parts)
         request = urllib.request.Request(new_url)
         request.add_header("Authorization", auth)


Test patch:
Repository: pypa/setuptools

Issue:
Git and Mercurial revision checkout not working under Windows
The revision checkout routines located in `setuptools/setuptools/package_index.py`, functions _download_git and _download_hg, rely on `(cd %s && <command>)` to execute `<command>` in `%s`. This does not work under Windows, as simply tested by executing `cd .. && echo %cd%`.

Further reading: https://stackoverflow.com/q/55641332


Discussion:

Resolving patch:
diff --git a/setuptools/package_index.py b/setuptools/package_index.py
--- a/setuptools/package_index.py
+++ b/setuptools/package_index.py
@@ -897,7 +897,7 @@ def _download_git(self, url, filename):
 
         if rev is not None:
             self.info("Checking out %s", rev)
-            os.system("(cd %s && git checkout --quiet %s)" % (
+            os.system("git -C %s checkout --quiet %s" % (
                 filename,
                 rev,
             ))
@@ -913,7 +913,7 @@ def _download_hg(self, url, filename):
 
         if rev is not None:
             self.info("Updating to %s", rev)
-            os.system("(cd %s && hg up -C -r %s -q)" % (
+            os.system("hg --cwd %s up -C -r %s -q" % (
                 filename,
                 rev,
             ))


Test patch:
diff --git a/setuptools/tests/test_packageindex.py b/setuptools/tests/test_packageindex.py
--- a/setuptools/tests/test_packageindex.py
+++ b/setuptools/tests/test_packageindex.py
@@ -249,7 +249,7 @@ def test_download_git_with_rev(self, tmpdir):
         first_call_args = os_system_mock.call_args_list[0][0]
         assert first_call_args == (expected,)
 
-        tmpl = '(cd {expected_dir} && git checkout --quiet master)'
+        tmpl = 'git -C {expected_dir} checkout --quiet master'
         expected = tmpl.format(**locals())
         assert os_system_mock.call_args_list[1][0] == (expected,)
         assert result == expected_dir
Repository: pypa/setuptools

Issue:
Python2 is broken after 41.5.0 [Windows]
In PR #1811 a [non-python2 statement was introduced](https://github.com/pypa/setuptools/pull/1811/files#diff-ebddaa1b649ef567c2534a012ba25913R627) (`encoding` argument is not available in Python2):

```python
with open(state_path, 'rt', encoding='utf-8') as state_file:
```

This breaks any installation in Windows py2 that requires to generate a package


Discussion:
Does it work with this patch:
```diff
setuptools/msvc.py | 1 +
 1 file changed, 1 insertion(+)

diff --git i/setuptools/msvc.py w/setuptools/msvc.py
index ffa7053b..2ffe1c81 100644
--- i/setuptools/msvc.py
+++ w/setuptools/msvc.py
@@ -20,6 +20,7 @@
 """
 
 import json
+from io import open
 from os import listdir, pathsep
 from os.path import join, isfile, isdir, dirname
 import sys
```
?
Yes, it will, that's one of the _classic_ workarounds. I'd submit the PR... but I'm not sure how to add a test for that :S
To clarify, the issue only occurs when building a package with a C extension?

Resolving patch:
diff --git a/setuptools/msvc.py b/setuptools/msvc.py
--- a/setuptools/msvc.py
+++ b/setuptools/msvc.py
@@ -20,6 +20,7 @@
 """
 
 import json
+from io import open
 from os import listdir, pathsep
 from os.path import join, isfile, isdir, dirname
 import sys


Test patch:
Repository: pypa/setuptools

Issue:
Deprecated distutils bdist_wininst is going to be removed
I proposed to remove the bdist_winstinst command from distutils in Python 3.9:

* https://bugs.python.org/issue39541
* https://discuss.python.org/t/remove-distutils-bdist-wininst-command/3115
* https://github.com/python/cpython/pull/18329

Problem: setuptools always uses it on all platforms at: setuptools/command/install_scripts.py, line 35:

```
  bw_cmd = self.get_finalized_command("bdist_wininst")
```

See #857 which is a closed duplicated which proposed different options to fix the issue.


Discussion:

Resolving patch:
diff --git a/setuptools/command/install_scripts.py b/setuptools/command/install_scripts.py
--- a/setuptools/command/install_scripts.py
+++ b/setuptools/command/install_scripts.py
@@ -32,8 +32,11 @@ def run(self):
         )
         bs_cmd = self.get_finalized_command('build_scripts')
         exec_param = getattr(bs_cmd, 'executable', None)
-        bw_cmd = self.get_finalized_command("bdist_wininst")
-        is_wininst = getattr(bw_cmd, '_is_running', False)
+        try:
+            bw_cmd = self.get_finalized_command("bdist_wininst")
+            is_wininst = getattr(bw_cmd, '_is_running', False)
+        except ImportError:
+            is_wininst = False
         writer = ei.ScriptWriter
         if is_wininst:
             exec_param = "python.exe"


Test patch:
Repository: pypa/setuptools

Issue:
Removing compatibility for Python 2
In #1458 and the Setuptools 45 release, this project dropped declared support for Python 2, adding a warning when a late version was invoked on Python 2. This warning helped address many of the systemic uses of Setuptools 45+ on Python 2, but there continue to be users (presumably) reporting that they've [encountered the warning](https://github.com/pypa/setuptools/issues?q=is%3Aissue+in%3Atitle+%22incompatible+install%22+).

I say presumably because most of them have submitted a blank template without providing any information.

Since March, these users have been directed to the template via bit.ly, so I have metrics on the number of users encountering and following the link.

![image](https://user-images.githubusercontent.com/308610/81501269-806b5b80-92a5-11ea-9d0a-1189e4c57061.png)

It seems there have been 50-100 clicks per day since Apr 11. I'm guessing bit.ly doesn't give me data older than 30 days.

To put that in perspective, Setuptools received over 45M downloads in the last month, so the number of people that followed that link (3.3k) is 0.007% of the downloads.

Still, that's upwards of 100 people per day whose workflow would be broken until they could fix their environment.

Let's also consider that each of these users encountering this issue are following discouraged if not deprecated workflows and are creating new or updated environments (new since setuptools 45 was released in January).

It seems to me we have two options - support Python 2 until the incidents of users encountering this error message reduces to a trickle (what is that threshold) or bite the bullet and drop support for Python 2.

I'd like to review the outstanding issues relating to this issue, but my inclination is to move forward with dropping support.


Discussion:

Resolving patch:
diff --git a/pkg_resources/py2_warn.py b/pkg_resources/py2_warn.py
--- a/pkg_resources/py2_warn.py
+++ b/pkg_resources/py2_warn.py
@@ -4,18 +4,13 @@
 
 
 msg = textwrap.dedent("""
-    You are running Setuptools on Python 2, which is no longer
-    supported and
-    >>> SETUPTOOLS WILL STOP WORKING <<<
-    in a subsequent release (no sooner than 2020-04-20).
-    Please ensure you are installing
-    Setuptools using pip 9.x or later or pin to `setuptools<45`
-    in your environment.
-    If you have done those things and are still encountering
-    this message, please follow up at
-    https://bit.ly/setuptools-py2-warning.
+    Encountered a version of Setuptools that no longer supports
+    this version of Python. Please head to
+    https://bit.ly/setuptools-py2-warning for support.
     """)
 
-pre = "Setuptools will stop working on Python 2\n"
+pre = "Setuptools no longer works on Python 2\n"
 
-sys.version_info < (3,) and warnings.warn(pre + "*" * 60 + msg + "*" * 60)
+if sys.version_info < (3,):
+    warnings.warn(pre + "*" * 60 + msg + "*" * 60)
+    raise SystemExit(32)


Test patch:
Repository: pypa/setuptools

Issue:
exec_module() missing 1 required positional argument: 'module'
Hi. Over at https://github.com/python-pillow/Pillow/issues/4769, we've started receiving an error in Python 3.9 -

```
TypeError: exec_module() missing 1 required positional argument: 'module'
```

I find that our code passes with setuptools 47.3.1, but fails with 47.3.2. Investigating, I find that https://github.com/pypa/setuptools/pull/2071/files adds `ExtensionFileLoader(__name__,__file__).exec_module()` - calling `exec_module` without any arguments.

Looking at https://docs.python.org/3/library/importlib.html#importlib.machinery.ExtensionFileLoader, I would conclude that yes, `exec_module` does require an argument.

I'm hoping that seems like a suitably obvious bug in setuptools. Let me know if it's not.


Discussion:
@alexhenrie Can you investigate and devise a fix?

Resolving patch:
diff --git a/setuptools/command/bdist_egg.py b/setuptools/command/bdist_egg.py
--- a/setuptools/command/bdist_egg.py
+++ b/setuptools/command/bdist_egg.py
@@ -59,7 +59,7 @@ def __bootstrap__():
             from importlib.machinery import ExtensionFileLoader
             __file__ = pkg_resources.resource_filename(__name__, %r)
             __loader__ = None; del __bootstrap__, __loader__
-            ExtensionFileLoader(__name__,__file__).exec_module()
+            ExtensionFileLoader(__name__,__file__).load_module()
         __bootstrap__()
         """).lstrip()
     with open(pyfile, 'w') as f:
diff --git a/setuptools/command/build_ext.py b/setuptools/command/build_ext.py
--- a/setuptools/command/build_ext.py
+++ b/setuptools/command/build_ext.py
@@ -268,7 +268,7 @@ def write_stub(self, output_dir, ext, compile=False):
                     "     os.chdir(os.path.dirname(__file__))",
                     if_dl("     sys.setdlopenflags(dl.RTLD_NOW)"),
                     "     ExtensionFileLoader(__name__,",
-                    "                         __file__).exec_module()",
+                    "                         __file__).load_module()",
                     "   finally:",
                     if_dl("     sys.setdlopenflags(old_flags)"),
                     "     os.chdir(old_dir)",


Test patch:
Repository: pypa/setuptools

Issue:
Sphinx setup should be stricter
I noticed that some of the docs pages are unreachable when navigating from the main RTD page. In particular, _I know_ that there's `history.rst` that is only accessible if one knows the URL upfront.

I tracked this to https://github.com/pypa/setuptools/pull/2097 which removes entries from the TOC but doesn't reintroduce them in other places.

Sphinx has a few toggles that make it nitpicky about warnings. I think this should be enabled in the CI to prevent such problems in the future. This should catch implicit orphan pages as well as dead references or typos.


Discussion:

Resolving patch:
diff --git a/docs/conf.py b/docs/conf.py
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -146,3 +146,12 @@
         ],
     ),
 }
+
+
+# Be strict about any broken references:
+nitpicky = True
+
+
+# Ref: https://github.com/python-attrs/attrs/pull/571/files\
+#      #diff-85987f48f1258d9ee486e3191495582dR82
+default_role = 'any'


Test patch:
Repository: pypa/setuptools

Issue:
Add mechanism for side-by-side comparison of setup.py and its equivalent setup.cfg
We have many documentation examples that are purely declarative and are either documented as `setup.py` or `setup.cfg`. It would be really awesome if, for each of these, we had the option to have either both versions side-by-side or, even better, in a sort of "tabbed container", like the one in the [code sample at the bottom of this example](https://leetcode.com/articles/median-of-two-sorted-arrays/).

Requirements for this:

1. Cannot *link to* any third-party javascript dependencies. Ideally we wouldn't use any at all, but if you do they must be vendored in the documentation.
2. If javascript is disabled, it has to fall back to something intelligible.

Ideally it would be implemented in pure CSS / HTML if that's at all possible.


Discussion:
Hi I'm just wondering if this is for setuptools' own documentation? because if it is built with sphinx I would think this is about making a sphinx extension that generates such tabbed container ?
Yes, this probably would be for Setuptools' own documentation. Perhaps Sphinx would need to have an extension, or perhaps there's another approach that would suffice (embedded HTML/JS, ...). I've also experienced the constraining environment of Sphinx when all I wanted to do was provide URLs with substitutions using Sphinx variables (settled on it's not possible). I'm open to innovations that achieve the objective.
I'll look into this! 

Seems like there exists a [sphinx-extension](https://github.com/executablebooks/sphinx-tabs) that implements the tabbed container if we go that route. However, it does use JS to achieve this and doesn't account for when JS is disabled (requirement#2). 

I'm thinking about exploring embedded HTML/CSS (and have it be side-by-side like a responsive flexbox) or create a similar tab extension that will account for when JS is disabled.

Resolving patch:
diff --git a/docs/conf.py b/docs/conf.py
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -93,3 +93,6 @@
 html_theme = 'alabaster'
 templates_path = ['_templates']
 html_sidebars = {'index': ['tidelift-sidebar.html']}
+
+# Add support for inline tabs
+extensions += ['sphinx_inline_tabs']


Test patch:
Repository: pypa/setuptools

Issue:
[BUG] Should fail hard when installing packages that want 2to3
### setuptools version

setuptools==58.0.0

### Python version

Python 3.9

### OS

Linux

### Additional environment information

_No response_

### Description

Tried to install a package that uses 2to3 (specifically https://pypi.org/project/demjson/), got a message from pip like:
```
Successfully installed demjson-2.2.3
```
But really it wasn't successful, and then get confusing `SyntaxError` exceptions.



### Expected behavior

I have now worked out that this is because of the change to remove `2to3` from setuptools, which I think is a perfectly reasonable change, and because my setuptools version wasn't pinned. But setuptools is not doing something the package is asking for, and the result is going to be something that does not run (though I guess there are caveats here).

I would have expected to get an error like "This module wants to use 2to3, but this is not supported [link to github issue about removing 2to3 support]". At least, that would certainly have saved me time.

### How to Reproduce

Install `demjson==2.2.3` with `setuptools==58.0.0`. Attempt to import and get a syntax error.

### Output

```console
$ pip install -U setuptools==58.0.0 && pip install demjson==2.2.3 && python -c 'import demjson'
Collecting setuptools==58.0.0
  Using cached setuptools-58.0.0-py3-none-any.whl (816 kB)
Installing collected packages: setuptools
  Attempting uninstall: setuptools
    Found existing installation: setuptools 56.0.0
    Uninstalling setuptools-56.0.0:
      Successfully uninstalled setuptools-56.0.0
Successfully installed setuptools-58.0.0
Collecting demjson==2.2.3
  Using cached demjson-2.2.3.tar.gz (131 kB)
Using legacy 'setup.py install' for demjson, since package 'wheel' is not installed.
Installing collected packages: demjson
    Running setup.py install for demjson ... done
Successfully installed demjson-2.2.3
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/user/example/venv/lib/python3.9/site-packages/demjson.py", line 645
    class json_int( (1L).__class__ ):    # Have to specify base this way to satisfy 2to3
                      ^
SyntaxError: invalid syntax
```

### Code of Conduct

- [X] I agree to follow the PSF Code of Conduct


Discussion:
Good suggestion. I think I agree.

Resolving patch:
diff --git a/setuptools/dist.py b/setuptools/dist.py
--- a/setuptools/dist.py
+++ b/setuptools/dist.py
@@ -289,6 +289,10 @@ def assert_bool(dist, attr, value):
         raise DistutilsSetupError(tmpl.format(attr=attr, value=value))
 
 
+def invalid(dist, attr, value):
+    raise DistutilsSetupError(f"{attr} is invalid.")
+
+
 def check_requirements(dist, attr, value):
     """Verify that install_requires is a valid requirements list"""
     try:


Test patch:
Repository: pypa/setuptools

Issue:
Deprecate setuptools.installer but not setup_requires
> You're going to break entire workflows and distros with this, please reconsider.

I'm tempted to un-deprecate `setup_requires` as a parameter, but to deprecate/remove only the `fetch_build_eggs` part. In other words, continue to allow `setup_requires` to supply requirements for PEP 517 hooks (`get_requires_for_build_sdist` and `get_requires_for_build_wheel`).


_Originally posted by @jaraco in https://github.com/pypa/setuptools/issues/2824#issuecomment-967430754_


Discussion:

Resolving patch:
diff --git a/setuptools/__init__.py b/setuptools/__init__.py
--- a/setuptools/__init__.py
+++ b/setuptools/__init__.py
@@ -4,7 +4,6 @@
 import functools
 import os
 import re
-import warnings
 
 import _distutils_hack.override  # noqa: F401
 
@@ -145,11 +144,6 @@ def finalize_options(self):
     # Honor setup.cfg's options.
     dist.parse_config_files(ignore_option_errors=True)
     if dist.setup_requires:
-        warnings.warn(
-            "setup_requires is deprecated. Supply build "
-            "dependencies using PEP 517 pyproject.toml build-requires.",
-            SetuptoolsDeprecationWarning,
-        )
         dist.fetch_build_eggs(dist.setup_requires)
 
 
diff --git a/setuptools/installer.py b/setuptools/installer.py
--- a/setuptools/installer.py
+++ b/setuptools/installer.py
@@ -3,11 +3,13 @@
 import subprocess
 import sys
 import tempfile
+import warnings
 from distutils import log
 from distutils.errors import DistutilsError
 
 import pkg_resources
 from setuptools.wheel import Wheel
+from ._deprecation_warning import SetuptoolsDeprecationWarning
 
 
 def _fixup_find_links(find_links):
@@ -22,6 +24,11 @@ def fetch_build_egg(dist, req):  # noqa: C901  # is too complex (16)  # FIXME
     """Fetch an egg needed for building.
 
     Use pip/wheel to fetch/build a wheel."""
+    warnings.warn(
+        "setuptools.installer is deprecated. Requirements should "
+        "be satisfied by a PEP 517 installer.",
+        SetuptoolsDeprecationWarning,
+    )
     # Warn if wheel is not available
     try:
         pkg_resources.get_distribution('wheel')


Test patch:
Repository: pypa/setuptools

Issue:
[Docs] GitHub reference in changelog rendered as email
### Summary

In [the changelog for v59.0.0](https://setuptools.pypa.io/en/latest/history.html#v59-0-0), a `distutils` commit is referenced using `pypa/distutils@f1b0a2b` in the RST source. Although it should link to pypa/distutils@f1b0a2b (`https://github.com/pypa/distutils/commit/f1b0a2b`) as GitHub automatically does, it instead renders the URL as `mailto:pypa/distutils@f1b0a2b`, which is incorrect.

### OS / Environment

N/a. This is in the source code.

### Additional Information

I would solve this myself, but I don't know the best solution. I can see that some things like GitHub issues are automatically linked, but I'm not sure if it should be added to the linking code in [`docs/conf.py`](https://github.com/pypa/setuptools/blob/4b980ef4072a817aae0da3643d0fa70c30fcb6cf/docs/conf.py) or just manually made a link.

### Code of Conduct

- [X] I agree to follow the PSF Code of Conduct


Discussion:
Thanks for the report.

I'm not sure either. I'd first check to see if there's any prior art for linking to a revision in distutils. If not, the linker definitions in conf.py might work, so that's worth exploring. Those use `rst.linker`, which isn't particularly easy to understand.

Resolving patch:
diff --git a/docs/conf.py b/docs/conf.py
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -68,6 +68,10 @@
                 pattern=r'pypa/distutils#(?P<distutils>\d+)',
                 url='{GH}/pypa/distutils/issues/{distutils}',
             ),
+            dict(
+                pattern=r'pypa/distutils@(?P<distutils_commit>[\da-f]+)',
+                url='{GH}/pypa/distutils/commit/{distutils_commit}',
+            ),
             dict(
                 pattern=r'^(?m)((?P<scm_version>v?\d+(\.\d+){1,2}))\n[-=]+\n',
                 with_scm='{text}\n{rev[timestamp]:%d %b %Y}\n',


Test patch: