internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: pypa/pip

Issue:
Use Bitwise AND to set permissions in utils/unpacking.py
Originally triaged in https://github.com/pypa/pip/issues/8164

We are using `0o777 - current_umask()` to set file permissions, but we should be using a Bitwise AND  (`0o777 & ~current_umask()`)

https://github.com/pypa/pip/blob/43426eea344c626153efa46f3a9bc6b62d7e8a69/src/pip/_internal/utils/unpacking.py#L148

https://github.com/pypa/pip/blob/43426eea344c626153efa46f3a9bc6b62d7e8a69/src/pip/_internal/utils/unpacking.py#L232

We would also want to move the two `os.chmod` calls into a helper function.



Discussion:
Hi @sbidoul , @pradyunsg 

Do we still want to make the above mentioned changes? If yes, I can go ahead and make a PR for the same.
@deveshks yes, please go ahead.

Resolving patch:
diff --git a/src/pip/_internal/utils/unpacking.py b/src/pip/_internal/utils/unpacking.py
--- a/src/pip/_internal/utils/unpacking.py
+++ b/src/pip/_internal/utils/unpacking.py
@@ -95,6 +95,15 @@ def is_within_directory(directory, target):
     return prefix == abs_directory
 
 
+def set_extracted_file_to_default_mode_plus_executable(path):
+    # type: (Union[str, Text]) -> None
+    """
+    Make file present at path have execute for user/group/world
+    (chmod +x) is no-op on windows per python docs
+    """
+    os.chmod(path, (0o777 & ~current_umask() | 0o111))
+
+
 def unzip_file(filename, location, flatten=True):
     # type: (str, str, bool) -> None
     """
@@ -140,9 +149,7 @@ def unzip_file(filename, location, flatten=True):
                     # if mode and regular file and any execute permissions for
                     # user/group/world?
                     if mode and stat.S_ISREG(mode) and mode & 0o111:
-                        # make dest file have execute for user/group/world
-                        # (chmod +x) no-op on windows per python docs
-                        os.chmod(fn, (0o777 - current_umask() | 0o111))
+                        set_extracted_file_to_default_mode_plus_executable(fn)
     finally:
         zipfp.close()
 
@@ -225,9 +232,7 @@ def untar_file(filename, location):
                 tar.utime(member, path)  # type: ignore
                 # member have any execute permissions for user/group/world?
                 if member.mode & 0o111:
-                    # make dest file have execute for user/group/world
-                    # no-op on windows per python docs
-                    os.chmod(path, (0o777 - current_umask() | 0o111))
+                    set_extracted_file_to_default_mode_plus_executable(path)
     finally:
         tar.close()
 


Test patch:
Repository: pypa/pip

Issue:
change ResolutionImpossible error message to point to /latest/ docs
Followup to  #8371, #8530, #8513, #8660. The `ResolutionImpossible` error currently points to https://pip.pypa.io/en/stable/user_guide/#fixing-conflicting-dependencies but -- at least during pip 20.2 -- it should probably point to https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies which is faster to update.

Per discussion with @pradyunsg  just now in IRC.


Discussion:

Resolving patch:
diff --git a/src/pip/_internal/resolution/resolvelib/factory.py b/src/pip/_internal/resolution/resolvelib/factory.py
--- a/src/pip/_internal/resolution/resolvelib/factory.py
+++ b/src/pip/_internal/resolution/resolvelib/factory.py
@@ -453,7 +453,7 @@ def describe_trigger(parent):
         logger.info(msg)
 
         return DistributionNotFound(
-            "ResolutionImpossible For help visit: "
-            "https://pip.pypa.io/en/stable/user_guide/"
+            "ResolutionImpossible: for help visit "
+            "https://pip.pypa.io/en/latest/user_guide/"
             "#fixing-conflicting-dependencies"
         )


Test patch:
Repository: pypa/pip

Issue:
Remove `--find-links` in `pip freeze`
**What's the problem this feature will solve?**

`pip freeze --find-links=somewhere` will simply output `-f somewhere` at the beginning of the output:
https://github.com/pypa/pip/blob/8ae1bd607e9fa84d6640909dc9998a0c61a10da6/src/pip/_internal/operations/freeze.py#L59-L62

Except for history (this option exists since the `freeze` command and https://github.com/pypa/pip/commit/3d32c196b53bedf28860df4f29d0ed5bc8bec245), there is no reason to special case it and not allow `--index-url`, `--pre` and all the other options allowed in a requirements file.

**Describe the solution you'd like**

Deprecate and remove it.

**Alternative Solutions**

Keep it.

**Additional context**

Deprecating it would ease the transition for #7157



Discussion:

Resolving patch:
diff --git a/src/pip/_internal/commands/freeze.py b/src/pip/_internal/commands/freeze.py
--- a/src/pip/_internal/commands/freeze.py
+++ b/src/pip/_internal/commands/freeze.py
@@ -9,6 +9,7 @@
 from pip._internal.models.format_control import FormatControl
 from pip._internal.operations.freeze import freeze
 from pip._internal.utils.compat import stdlib_pkgs
+from pip._internal.utils.deprecation import deprecated
 from pip._internal.utils.typing import MYPY_CHECK_RUNNING
 
 DEV_PKGS = {'pip', 'setuptools', 'distribute', 'wheel'}
@@ -86,6 +87,14 @@ def run(self, options, args):
 
         cmdoptions.check_list_path_option(options)
 
+        if options.find_links:
+            deprecated(
+                "--find-links option in pip freeze is deprecated.",
+                replacement=None,
+                gone_in="21.2",
+                issue=9069,
+            )
+
         freeze_kwargs = dict(
             requirement=options.requirements,
             find_links=options.find_links,


Test patch:
diff --git a/tests/functional/test_freeze.py b/tests/functional/test_freeze.py
--- a/tests/functional/test_freeze.py
+++ b/tests/functional/test_freeze.py
@@ -358,7 +358,8 @@ def test_freeze_mercurial_clone_srcdir(script, tmpdir):
     _check_output(result.stdout, expected)
 
     result = script.pip(
-        'freeze', '-f', '{repo_dir}#egg=pip_test_package'.format(**locals())
+        'freeze', '-f', '{repo_dir}#egg=pip_test_package'.format(**locals()),
+        expect_stderr=True,
     )
     expected = textwrap.dedent(
         """
Repository: pypa/pip

Issue:
Revert "Redact auth from URL in UpdatingDefaultsHelpFormatter"
Reverts pypa/pip#9163


Discussion:

Resolving patch:
diff --git a/src/pip/_internal/cli/parser.py b/src/pip/_internal/cli/parser.py
--- a/src/pip/_internal/cli/parser.py
+++ b/src/pip/_internal/cli/parser.py
@@ -112,15 +112,23 @@ class UpdatingDefaultsHelpFormatter(PrettyHelpFormatter):
     """
 
     def expand_default(self, option):
-        default_value = None
+        default_values = None
         if self.parser is not None:
             self.parser._update_defaults(self.parser.defaults)
-            default_value = self.parser.defaults.get(option.dest)
+            default_values = self.parser.defaults.get(option.dest)
         help_text = optparse.IndentedHelpFormatter.expand_default(self, option)
 
-        if default_value and option.metavar == 'URL':
-            help_text = help_text.replace(
-                default_value, redact_auth_from_url(default_value))
+        if default_values and option.metavar == 'URL':
+            if isinstance(default_values, string_types):
+                default_values = [default_values]
+
+            # If its not a list, we should abort and just return the help text
+            if not isinstance(default_values, list):
+                default_values = []
+
+            for val in default_values:
+                help_text = help_text.replace(
+                    val, redact_auth_from_url(val))
 
         return help_text
 


Test patch:
diff --git a/tests/functional/test_help.py b/tests/functional/test_help.py
--- a/tests/functional/test_help.py
+++ b/tests/functional/test_help.py
@@ -74,6 +74,17 @@ def test_help_command_redact_auth_from_url(script):
     assert 'secret' not in result.stdout
 
 
+def test_help_command_redact_auth_from_url_with_extra_index_url(script):
+    """
+    Test `help` on various subcommands redact auth from url with extra index url
+    """
+    script.environ['PIP_INDEX_URL'] = 'https://user:secret@example.com'
+    script.environ['PIP_EXTRA_INDEX_URL'] = 'https://user:secret@example2.com'
+    result = script.pip('install', '--help')
+    assert result.returncode == SUCCESS
+    assert 'secret' not in result.stdout
+
+
 def test_help_commands_equally_functional(in_memory_pip):
     """
     Test if `pip help` and 'pip --help' behave the same way.
Repository: pypa/pip

Issue:
Move slow check_manifest out of .pre-commit-config.yaml
The check-manifest pre-commit hook is by far the slowest of the checks, in particular when committing a few files.

When pre-commit is installed locally as a pre-commit hook, it tends to slow down the development flow significantly.

Could we move it out of the default pre-commit config to a regular CI check ?



Discussion:

Resolving patch:
diff --git a/noxfile.py b/noxfile.py
--- a/noxfile.py
+++ b/noxfile.py
@@ -149,6 +149,9 @@ def lint(session):
         args = ["--all-files", "--show-diff-on-failure"]
 
     session.run("pre-commit", "run", *args)
+    session.run(
+        "pre-commit", "run", "-c", ".pre-commit-config-slow.yaml", *args
+    )
 
 
 @nox.session


Test patch:
Repository: pypa/pip

Issue:
Email link not working in docs?
![Screenshot from 2020-12-23 16-07-59](https://user-images.githubusercontent.com/3323703/103016589-2ff5f500-453a-11eb-82a8-9b6978d12c78.png)

I don't think the link to the email is working.


Discussion:
What's wrong with it, and what would be a good way to fix it? 

Resolving patch:
diff --git a/docs/docs_feedback_sphinxext.py b/docs/docs_feedback_sphinxext.py
--- a/docs/docs_feedback_sphinxext.py
+++ b/docs/docs_feedback_sphinxext.py
@@ -141,7 +141,7 @@ def setup(app: Sphinx) -> Dict[str, Union[bool, str]]:
     )
     app.add_config_value(
         'docs_feedback_email',
-        default='Docs UX Team <docs-feedback+ux/pip.pypa.io@pypa.io>',
+        default='Docs UX Team <docs-feedback@pypa.io>',
         rebuild=rebuild_trigger,
     )
     app.add_config_value(
diff --git a/docs/html/conf.py b/docs/html/conf.py
--- a/docs/html/conf.py
+++ b/docs/html/conf.py
@@ -307,7 +307,7 @@ def to_document_name(path, base_dir):
 # NOTE: 'important', 'note', 'tip', 'warning' or 'admonition'.
 docs_feedback_admonition_type = 'important'
 docs_feedback_big_doc_lines = 50  # bigger docs will have a banner on top
-docs_feedback_email = 'Docs UX Team <docs-feedback+ux/pip.pypa.io@pypa.io>'
+docs_feedback_email = 'Docs UX Team <docs-feedback@pypa.io>'
 docs_feedback_excluded_documents = {  # these won't have any banners
     'news', 'reference/index',
 }


Test patch:
Repository: pypa/pip

Issue:
nox -s vendoring -- --upgrade fails on Windows
**Environment**

* pip version: 20.3.3
* Python version: 38.
* OS: Windows

**Description**

Using `nox -s vendoring -- --upgrade` on Windows, causes extra newlines to be added to `vendor.txt`, resulting in the process failing.

**Expected behavior**
The process works on Windows.

**How to Reproduce**
`nox -s vendoring -- --upgrade` on a Windows system.

**Output**

```
>nox -s vendoring -- --upgrade
nox > Running session vendoring
nox > Re-using existing virtual environment at .nox\vendoring.
nox > python -m pip install vendoring>=0.3.0
nox > vendoring update . appdirs
Load configuration... Done!
Updating requirements... Done!
nox > vendoring update . CacheControl
Load configuration... Done!
Updating requirements...
  [(2, ''), (4, ''), (6, ''), (8, ''), (10, ''), (12, ''), (14, ''), (16, ''), (18, ''), (20, ''), (22, ''), (24, ''), (26, ''), (28, ''), (30, ''), (32, ''), (34, ''), (36, ''), (38, ''), (40, ''), (42, ''), (44, ''), (46, '')]
nox > Session vendoring raised exception ValueError('not enough values to unpack (expected 2, got 1)')
Traceback (most recent call last):
  File "c:\users\gustav\.local\pipx\venvs\nox\lib\site-packages\nox\sessions.py", line 549, in execute
    self.func(session)
  File "c:\users\gustav\.local\pipx\venvs\nox\lib\site-packages\nox\_decorators.py", line 53, in __call__
    return self.func(*args, **kwargs)
  File "D:\Work\Projects\pip\noxfile.py", line 182, in vendoring
    for inner_name, inner_version in pinned_requirements(vendor_txt):
  File "D:\Work\Projects\pip\noxfile.py", line 167, in pinned_requirements
    one, two = line.split("==", 1)
ValueError: not enough values to unpack (expected 2, got 1)
nox > Session vendoring failed.
```

I suspect the issue is two-fold: the nox session doesn't handle blank lines in `vendor.txt`, but also something (possibly the vendoring tool?) is writing unnecessary blank lines?


Discussion:
Ping @pradyunsg
What's the value of `line` here?
"". Because we have extra blank lines in the file.
This works on macOS, with current versions of everything, so it could be Windows only, perhaps? (`pipx run nox -s vendoring -- --upgrade`)
It is Windows only, it's a line ending issue as far as I can tell.

Resolving patch:
diff --git a/noxfile.py b/noxfile.py
--- a/noxfile.py
+++ b/noxfile.py
@@ -174,11 +174,14 @@ def vendoring(session):
 
     def pinned_requirements(path):
         # type: (Path) -> Iterator[Tuple[str, str]]
-        for line in path.read_text().splitlines():
-            one, two = line.split("==", 1)
+        for line in path.read_text().splitlines(keepends=False):
+            one, sep, two = line.partition("==")
+            if not sep:
+                continue
             name = one.strip()
-            version = two.split("#")[0].strip()
-            yield name, version
+            version = two.split("#", 1)[0].strip()
+            if name and version:
+                yield name, version
 
     vendor_txt = Path("src/pip/_vendor/vendor.txt")
     for name, old_version in pinned_requirements(vendor_txt):


Test patch:
Repository: pypa/pip

Issue:
Default proceed with uninstall
**What's the problem this feature will solve?**
<!-- What are you trying to do, that you are unable to achieve with pip as it currently stands? -->
When being asked to proceed with uninstall, pressing enter prompts the following response:
```Your response ('') was not one of the expected responses: y, n```

**Describe the solution you'd like**
<!-- Clear and concise description of what you want to happen. -->
I'd like to add a default input in the style of:
```(Y/n)```
...for which pressing 'Enter' would continue with the uninstall.

<!-- Provide examples of real world use cases that this would enable and how it solves the problem described above. -->



Discussion:

Resolving patch:
diff --git a/src/pip/_internal/req/req_uninstall.py b/src/pip/_internal/req/req_uninstall.py
--- a/src/pip/_internal/req/req_uninstall.py
+++ b/src/pip/_internal/req/req_uninstall.py
@@ -438,7 +438,7 @@ def _display(msg, paths):
         if verbose:
             _display('Will actually move:', compress_for_rename(self.paths))
 
-        return ask('Proceed (y/n)? ', ('y', 'n')) == 'y'
+        return ask('Proceed (Y/n)? ', ('y', 'n', '')) != 'n'
 
     def rollback(self):
         # type: () -> None


Test patch:
Repository: pypa/pip

Issue:
Minimum Python version for 21.1 is 3.6.2
Because of the use of typing.NoReturn I think the minimum Python version for 21.1 should be 3.6.2 rather than just 3.6.

Phil


Discussion:
If typing added a name in 3.6.2, that sounds like a breaking change that probably shouldn't have been allowed by CPython 🙁

This appears to have been introduced in #9424. We used `NoReturn` before that, but it was guarded by `if TYPE_CHECKING`, so didn't affect runtime behaviour.

Resolving patch:
diff --git a/src/pip/_internal/req/req_file.py b/src/pip/_internal/req/req_file.py
--- a/src/pip/_internal/req/req_file.py
+++ b/src/pip/_internal/req/req_file.py
@@ -8,17 +8,7 @@
 import shlex
 import urllib.parse
 from optparse import Values
-from typing import (
-    TYPE_CHECKING,
-    Any,
-    Callable,
-    Dict,
-    Iterator,
-    List,
-    NoReturn,
-    Optional,
-    Tuple,
-)
+from typing import TYPE_CHECKING, Any, Callable, Dict, Iterator, List, Optional, Tuple
 
 from pip._internal.cli import cmdoptions
 from pip._internal.exceptions import InstallationError, RequirementsFileParseError
@@ -29,6 +19,10 @@
 from pip._internal.utils.urls import get_url_scheme, url_to_path
 
 if TYPE_CHECKING:
+    # NoReturn introduced in 3.6.2; imported only for type checking to maintain
+    # pip compatibility with older patch versions of Python 3.6
+    from typing import NoReturn
+
     from pip._internal.index.package_finder import PackageFinder
 
 __all__ = ['parse_requirements']
diff --git a/src/pip/_internal/utils/hashes.py b/src/pip/_internal/utils/hashes.py
--- a/src/pip/_internal/utils/hashes.py
+++ b/src/pip/_internal/utils/hashes.py
@@ -1,5 +1,5 @@
 import hashlib
-from typing import TYPE_CHECKING, BinaryIO, Dict, Iterator, List, NoReturn
+from typing import TYPE_CHECKING, BinaryIO, Dict, Iterator, List
 
 from pip._internal.exceptions import HashMismatch, HashMissing, InstallationError
 from pip._internal.utils.misc import read_chunks
@@ -7,6 +7,10 @@
 if TYPE_CHECKING:
     from hashlib import _Hash
 
+    # NoReturn introduced in 3.6.2; imported only for type checking to maintain
+    # pip compatibility with older patch versions of Python 3.6
+    from typing import NoReturn
+
 
 # The recommended hash algo of the moment. Change this whenever the state of
 # the art changes; it won't hurt backward compatibility.


Test patch:
Repository: pypa/pip

Issue:
ValueError('ZIP does not support timestamps before 1980') on GitHub package install
### Description

Still tracking exactly how this is being triggered, but wanted to report early just in case.

Inconsistently getting `ValueError('ZIP does not support timestamps before 1980')` in CI when installing requirements via `git+https` for packages from GitHub.

I see the new zip functionality was introduced via #9689. Stack trace points to `zf.write` in `_create_standalone_pip` function. 

Haven't been able to trigger locally in pip 21.1 yet, only seeing in CI, so wondering if i'm having some kind of network issue. 

Either way, we might want to consider setting `strict_timestamps=False` when using ZipFile for this:
https://docs.python.org/3/library/zipfile.html#zipfile-objects



### Expected behavior

Correctly install requirement from GitHub.

### pip version

21.1

### Python version

3.9.2

### OS

python:3.9.2-buster Docker container

### How to Reproduce

Still trying to nail down a consistent method to reproduce. I suspect something in my CI is contributing to the issue. But exception is triggered while trying to install via `git+https` requirement.



### Output

_No response_

### Code of Conduct

- [X] I agree to follow the [PSF Code of Conduct](https://www.python.org/psf/conduct/).


Discussion:
Maybe the CI has an incorrect system time or delibrately set file system time to a “wrong” value (for reproducilibty for example)? This can never be triggered under normal circumstances since you can’t really have files older than 1980-01-01.

But yeah setting `strict_timestamps=False` should be a good idea either way, we’ll only building a temporary file to run against and don’t really care about timestamps anyway. I’d accept a PR in a heartbeat.

Resolving patch:
diff --git a/src/pip/_internal/build_env.py b/src/pip/_internal/build_env.py
--- a/src/pip/_internal/build_env.py
+++ b/src/pip/_internal/build_env.py
@@ -58,7 +58,10 @@ def _create_standalone_pip() -> Iterator[str]:
 
     with TempDirectory(kind="standalone-pip") as tmp_dir:
         pip_zip = os.path.join(tmp_dir.path, "__env_pip__.zip")
-        with zipfile.ZipFile(pip_zip, "w") as zf:
+        kwargs = {}
+        if sys.version_info >= (3, 8):
+            kwargs["strict_timestamps"] = False
+        with zipfile.ZipFile(pip_zip, "w", **kwargs) as zf:
             for child in source.rglob("*"):
                 zf.write(child, child.relative_to(source.parent).as_posix())
         yield os.path.join(pip_zip, "pip")


Test patch:
Repository: pypa/pip

Issue:
Update quickstart guide to reflect user research
Updates quickstart guide to reflect most common tasks as discovered in our "buy a feature" user research.

Preview: https://pip--9137.org.readthedocs.build/en/9137/quickstart/


Discussion:
@pradyunsg or @uranusjr  this content is ready for review.
If you can let me know if it is _accurate_ I will fix linting and add a news entry.  Thanks!
#9131 (merged recently) also included similar instruction on installing and upgrading pip in the *Installaton* page. Is this duplication intentional, or should we simply redirect users to that page instead?

Current “latest” pip documentation: https://pip.pypa.io/en/latest/installing/
@nlhkabu If you rebase this PR on the current master branch, that'll fix the CI here.



> Is this duplication intentional, or should we simply redirect users to that page instead?

https://www.writethedocs.org/guide/writing/docs-principles/#arid

I think this duplication is A-OK.

Resolving patch:
diff --git a/docs/html/conf.py b/docs/html/conf.py
--- a/docs/html/conf.py
+++ b/docs/html/conf.py
@@ -30,7 +30,7 @@
 
 # General information about the project.
 project = "pip"
-copyright = "2008-2020, PyPA"
+copyright = "The pip developers"
 
 # Find the version and release information.
 # We have a single source of truth for our version number: pip's __init__.py file.


Test patch:
Repository: pypa/pip

Issue:
Use UTF-8 for log file.
**What's the problem this feature will solve?**

* If default text encoding is not UTF-8 (e.g. Windows users), pip might fail to log non-ASCII texts (e.g. non-ASCII paths on Windows).
* When using `PYTHONWARNDEFAULTENCODING=1` (Python 3.10), `dictConfg()` emit EncodingWarning:

```
/home/inada-n/pyenv/versions/3.10.0b2/lib/python3.10/logging/config.py:748: EncodingWarning: 'encoding' argument not specified
  result = factory(**kwargs)
```

**Describe the solution you'd like**

```
diff --git a/src/pip/_internal/utils/logging.py b/src/pip/_internal/utils/logging.py
index 0569b9248..39a18fd6c 100644
--- a/src/pip/_internal/utils/logging.py
+++ b/src/pip/_internal/utils/logging.py
@@ -375,6 +375,7 @@ def setup_logging(verbosity, no_color, user_log_file):
                     "level": "DEBUG",
                     "class": handler_classes["file"],
                     "filename": additional_log_file,
+                    "encoding": "utf-8",
                     "delay": True,
                     "formatter": "indent_with_timestamp",
```

**Alternative Solutions**
<!-- Have you tried to workaround the problem using pip or other tools? Or a different approach to solving this issue? Please elaborate here. -->

**Additional context**
<!-- Add any other context, links, etc. about the feature here. -->



Discussion:

Resolving patch:
diff --git a/src/pip/_internal/utils/logging.py b/src/pip/_internal/utils/logging.py
--- a/src/pip/_internal/utils/logging.py
+++ b/src/pip/_internal/utils/logging.py
@@ -375,6 +375,7 @@ def setup_logging(verbosity, no_color, user_log_file):
                     "level": "DEBUG",
                     "class": handler_classes["file"],
                     "filename": additional_log_file,
+                    "encoding": "utf-8",
                     "delay": True,
                     "formatter": "indent_with_timestamp",
                 },


Test patch: