Repository: pypa/pip (python)

Issue:
Implement protections for externally managed Python environments (PEP 668)
### What's the problem this feature will solve?

See https://peps.python.org/pep-0668/#motivation for the details.

### Describe the solution you'd like

Implement the protections, as described in https://peps.python.org/pep-0668/#specification

### Alternative Solutions

N/A

### Additional context

See https://peps.python.org/pep-0668/. If there's broader issues here, add to the discussion on https://discuss.python.org/t/10302/

### Code of Conduct

- [X] I agree to follow the [PSF Code of Conduct](https://www.python.org/psf/conduct/).

Pull request description:
Check EXTERNALLY-MANAGED in install and uninstall
This implements the PEP 668 logic to 'pip install' and 'pip uninstall'. Are there any other commands that may need it?

This implementation disables the check is any of --prefix, --home, or --target is provided, since those can indicate the command does not actually install into the environment. Note that it is still possible the command is still modifying the environment, but we don't have a way to stop the user *that* determined to break the environment anyway (they can always just use those flags in a virtual environment).

Also not sure how best this can be tested.

Close #11381.

Patch:
diff --git a/.readthedocs-custom-redirects.yml b/.readthedocs-custom-redirects.yml
index d0c072fbf..46c1f819c 100644
--- a/.readthedocs-custom-redirects.yml
+++ b/.readthedocs-custom-redirects.yml
@@ -2,9 +2,9 @@
 # It is related to Read the Docs, but is not a file processed by the platform.
 
 /dev/news-entry-failure: >-
-  https://pip.pypa.io/en/latest/development/contributing/#news-entries
+  https://pip.pypa.io/en/stable/development/contributing/#news-entries
 /errors/resolution-impossible: >-
-  https://pip.pypa.io/en/stable/topics/dependency-resolution/#dealing-with-dependency-conflicts
+  https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
 /surveys/backtracking: >-
   https://forms.gle/LkZP95S4CfqBAU1N6
 /warnings/backtracking: >-
diff --git a/docs/html/installation.md b/docs/html/installation.md
index 036a91397..7db0d47ab 100644
--- a/docs/html/installation.md
+++ b/docs/html/installation.md
@@ -90,7 +90,7 @@ distro community, cloud provider support channels, etc).
 
 ## Upgrading `pip`
 
-Upgrade your `pip` by running:
+Upgrading your `pip` by running:
 
 ```{pip-cli}
 $ pip install --upgrade pip
diff --git a/docs/html/topics/local-project-installs.md b/docs/html/topics/local-project-installs.md
index bd70dbe2f..151035b00 100644
--- a/docs/html/topics/local-project-installs.md
+++ b/docs/html/topics/local-project-installs.md
@@ -34,7 +34,7 @@ Editable installs allow you to install your project without copying any files. I
 With an editable install, you only need to perform a re-installation if you change the project metadata (eg: version, what scripts need to be generated etc). You will still need to run build commands when you need to perform a compilation for non-Python code in the project (eg: C extensions).
 
 ```{caution}
-It is possible to see behaviour differences between regular installs vs editable installs. These differences depend on the build-backend, and you should check the build-backend documentation for the details. In case you distribute the project as a "distribution package", users will see the behaviour of regular installs -- thus, it is important to ensure that regular installs work correctly.
+It is possible to see behaviour differences between regular installs vs editable installs. In case you distribute the project as a "distribution package", users will see the behaviour of regular installs -- thus, it is important to ensure that regular installs work correctly.
 ```
 
 ```{note}
diff --git a/news/11381.feature.rst b/news/11381.feature.rst
new file mode 100644
index 000000000..3df9877b4
--- /dev/null
+++ b/news/11381.feature.rst
@@ -0,0 +1,3 @@
+Implement logic to read the ``EXTERNALLY-MANAGED`` file as specified in PEP 668.
+This allows a downstream Python distributor to prevent users from using pip to
+modify the externally managed environment.
diff --git a/news/11527.bugfix.rst b/news/11527.bugfix.rst
deleted file mode 100644
index 0185a804f..000000000
--- a/news/11527.bugfix.rst
+++ /dev/null
@@ -1,2 +0,0 @@
-Wheel cache behavior is restored to match previous versions, allowing the
-cache to find existing entries.
diff --git a/news/11623.bugfix.rst b/news/11623.bugfix.rst
deleted file mode 100644
index 45b8fe192..000000000
--- a/news/11623.bugfix.rst
+++ /dev/null
@@ -1,1 +0,0 @@
-Fix scripts path in isolated build environment on Debian.
diff --git a/news/11704.bugfix.rst b/news/11704.bugfix.rst
deleted file mode 100644
index 0e7902a25..000000000
--- a/news/11704.bugfix.rst
+++ /dev/null
@@ -1,2 +0,0 @@
-Fix an issue when an already existing in-memory distribution would cause
-exceptions in ``pip install``
diff --git a/src/pip/_internal/build_env.py b/src/pip/_internal/build_env.py
index 24bfa870b..e67b868e8 100644
--- a/src/pip/_internal/build_env.py
+++ b/src/pip/_internal/build_env.py
@@ -8,6 +8,7 @@ import site
 import sys
 import textwrap
 from collections import OrderedDict
+from sysconfig import get_paths
 from types import TracebackType
 from typing import TYPE_CHECKING, Iterable, List, Optional, Set, Tuple, Type
 
@@ -18,7 +19,6 @@ from pip._vendor.packaging.version import Version
 from pip import __file__ as pip_location
 from pip._internal.cli.spinners import open_spinner
 from pip._internal.locations import (
-    get_isolated_environment_bin_path,
     get_isolated_environment_lib_paths,
     get_platlib,
     get_purelib,
@@ -37,7 +37,10 @@ class _Prefix:
     def __init__(self, path: str) -> None:
         self.path = path
         self.setup = False
-        self.bin_dir = get_isolated_environment_bin_path(path)
+        self.bin_dir = get_paths(
+            "nt" if os.name == "nt" else "posix_prefix",
+            vars={"base": path, "platbase": path},
+        )["scripts"]
         self.lib_dirs = get_isolated_environment_lib_paths(path)
 
 
diff --git a/src/pip/_internal/commands/install.py b/src/pip/_internal/commands/install.py
index e081c27d2..accceeaec 100644
--- a/src/pip/_internal/commands/install.py
+++ b/src/pip/_internal/commands/install.py
@@ -41,6 +41,7 @@ from pip._internal.utils.distutils_args import parse_distutils_args
 from pip._internal.utils.filesystem import test_writable_dir
 from pip._internal.utils.logging import getLogger
 from pip._internal.utils.misc import (
+    check_externally_managed,
     ensure_dir,
     get_pip_version,
     protect_pip_from_modification_on_windows,
@@ -284,6 +285,20 @@ class InstallCommand(RequirementCommand):
         if options.use_user_site and options.target_dir is not None:
             raise CommandError("Can not combine '--user' and '--target'")
 
+        # Check whether the environment we're installing into is externally
+        # managed, as specified in PEP 668. Specifying --root, --target, or
+        # --prefix disables the check, since there's no reliable way to locate
+        # the EXTERNALLY-MANAGED file for those cases. An exception is also
+        # made specifically for "--dry-run --report" for convenience.
+        installing_into_current_environment = (
+            not (options.dry_run and options.json_report_file)
+            and options.root_path is None
+            and options.target_dir is None
+            and options.prefix_path is None
+        )
+        if installing_into_current_environment:
+            check_externally_managed()
+
         upgrade_strategy = "to-satisfy-only"
         if options.upgrade:
             upgrade_strategy = options.upgrade_strategy
diff --git a/src/pip/_internal/commands/uninstall.py b/src/pip/_internal/commands/uninstall.py
index dea8077e7..e5a4c8e10 100644
--- a/src/pip/_internal/commands/uninstall.py
+++ b/src/pip/_internal/commands/uninstall.py
@@ -14,7 +14,10 @@ from pip._internal.req.constructors import (
     install_req_from_line,
     install_req_from_parsed_requirement,
 )
-from pip._internal.utils.misc import protect_pip_from_modification_on_windows
+from pip._internal.utils.misc import (
+    check_externally_managed,
+    protect_pip_from_modification_on_windows,
+)
 
 logger = logging.getLogger(__name__)
 
@@ -90,6 +93,8 @@ class UninstallCommand(Command, SessionCommandMixin):
                 f'"pip help {self.name}")'
             )
 
+        check_externally_managed()
+
         protect_pip_from_modification_on_windows(
             modifying_pip="pip" in reqs_to_uninstall
         )
diff --git a/src/pip/_internal/exceptions.py b/src/pip/_internal/exceptions.py
index 2ab1f591f..d28713ff7 100644
--- a/src/pip/_internal/exceptions.py
+++ b/src/pip/_internal/exceptions.py
@@ -6,9 +6,14 @@ subpackage and, thus, should not depend on them.
 """
 
 import configparser
+import contextlib
+import locale
+import logging
+import pathlib
 import re
+import sys
 from itertools import chain, groupby, repeat
-from typing import TYPE_CHECKING, Dict, List, Optional, Union
+from typing import TYPE_CHECKING, Dict, Iterator, List, Optional, Union
 
 from pip._vendor.requests.models import Request, Response
 from pip._vendor.rich.console import Console, ConsoleOptions, RenderResult
@@ -22,6 +27,8 @@ if TYPE_CHECKING:
     from pip._internal.metadata import BaseDistribution
     from pip._internal.req.req_install import InstallRequirement
 
+logger = logging.getLogger(__name__)
+
 
 #
 # Scaffolding
@@ -658,3 +665,81 @@ class ConfigurationFileCouldNotBeLoaded(ConfigurationError):
             assert self.error is not None
             message_part = f".\n{self.error}\n"
         return f"Configuration file {self.reason}{message_part}"
+
+
+_DEFAULT_EXTERNALLY_MANAGED_ERROR = f"""\
+The Python environment under {sys.prefix} is managed externally, and may not be
+manipulated by the user. Please use specific tooling from the distributor of
+the Python installation to interact with this environment instead.
+"""
+
+
+class ExternallyManagedEnvironment(DiagnosticPipError):
+    """The current environment is externally managed.
+
+    This is raised when the current environment is externally managed, as
+    defined by `PEP 668`_. The ``EXTERNALLY-MANAGED`` configuration is checked
+    and displayed when the error is bubbled up to the user.
+
+    :param error: The error message read from ``EXTERNALLY-MANAGED``.
+    """
+
+    reference = "externally-managed-environment"
+
+    def __init__(self, error: Optional[str]) -> None:
+        if error is None:
+            context = Text(_DEFAULT_EXTERNALLY_MANAGED_ERROR)
+        else:
+            context = Text(error)
+        super().__init__(
+            message="This environment is externally managed",
+            context=context,
+            note_stmt=(
+                "If you believe this is a mistake, please contact your "
+                "Python installation or OS distribution provider."
+            ),
+            hint_stmt=Text("See PEP 668 for the detailed specification."),
+        )
+
+    @staticmethod
+    def _iter_externally_managed_error_keys() -> Iterator[str]:
+        # LC_MESSAGES is in POSIX, but not the C standard. The most common
+        # platform that does not implement this category is Windows, where
+        # using other categories for console message localization is equally
+        # unreliable, so we fall back to the locale-less vendor message. This
+        # can always be re-evaluated when a vendor proposes a new alternative.
+        try:
+            category = locale.LC_MESSAGES
+        except AttributeError:
+            lang: Optional[str] = None
+        else:
+            lang, _ = locale.getlocale(category)
+        if lang is not None:
+            yield f"Error-{lang}"
+            for sep in ("-", "_"):
+                before, found, _ = lang.partition(sep)
+                if not found:
+                    continue
+                yield f"Error-{before}"
+        yield "Error"
+
+    @classmethod
+    def from_config(
+        cls,
+        config: Union[pathlib.Path, str],
+    ) -> "ExternallyManagedEnvironment":
+        parser = configparser.ConfigParser(interpolation=None)
+        try:
+            parser.read(config, encoding="utf-8")
+            section = parser["externally-managed"]
+            for key in cls._iter_externally_managed_error_keys():
+                with contextlib.suppress(KeyError):
+                    return cls(section[key])
+        except KeyError:
+            pass
+        except (OSError, UnicodeDecodeError, configparser.ParsingError):
+            from pip._internal.utils._log import VERBOSE
+
+            exc_info = logger.isEnabledFor(VERBOSE)
+            logger.warning("Failed to read %s", config, exc_info=exc_info)
+        return cls(None)
diff --git a/src/pip/_internal/index/package_finder.py b/src/pip/_internal/index/package_finder.py
index b6f8d57e8..9bf247f02 100644
--- a/src/pip/_internal/index/package_finder.py
+++ b/src/pip/_internal/index/package_finder.py
@@ -1,11 +1,14 @@
 """Routines related to PyPI, indexes"""
 
+# The following comment should be removed at some point in the future.
+# mypy: strict-optional=False
+
 import enum
 import functools
 import itertools
 import logging
 import re
-from typing import TYPE_CHECKING, FrozenSet, Iterable, List, Optional, Set, Tuple, Union
+from typing import FrozenSet, Iterable, List, Optional, Set, Tuple, Union
 
 from pip._vendor.packaging import specifiers
 from pip._vendor.packaging.tags import Tag
@@ -36,9 +39,6 @@ from pip._internal.utils.misc import build_netloc
 from pip._internal.utils.packaging import check_requires_python
 from pip._internal.utils.unpacking import SUPPORTED_EXTENSIONS
 
-if TYPE_CHECKING:
-    from pip._vendor.typing_extensions import TypeGuard
-
 __all__ = ["FormatControl", "BestCandidateResult", "PackageFinder"]
 
 
@@ -251,7 +251,7 @@ class LinkEvaluator:
 
 def filter_unallowed_hashes(
     candidates: List[InstallationCandidate],
-    hashes: Optional[Hashes],
+    hashes: Hashes,
     project_name: str,
 ) -> List[InstallationCandidate]:
     """
@@ -540,7 +540,6 @@ class CandidateEvaluator:
                 binary_preference = 1
             if wheel.build_tag is not None:
                 match = re.match(r"^(\d+)(.*)$", wheel.build_tag)
-                assert match is not None, "guaranteed by filename validation"
                 build_tag_groups = match.groups()
                 build_tag = (int(build_tag_groups[0]), build_tag_groups[1])
         else:  # sdist
@@ -943,46 +942,43 @@ class PackageFinder:
                 "No matching distribution found for {}".format(req)
             )
 
-        def _should_install_candidate(
-            candidate: Optional[InstallationCandidate],
-        ) -> "TypeGuard[InstallationCandidate]":
-            if installed_version is None:
-                return True
-            if best_candidate is None:
-                return False
-            return best_candidate.version > installed_version
+        best_installed = False
+        if installed_version and (
+            best_candidate is None or best_candidate.version <= installed_version
+        ):
+            best_installed = True
 
         if not upgrade and installed_version is not None:
-            if _should_install_candidate(best_candidate):
+            if best_installed:
                 logger.debug(
-                    "Existing installed version (%s) satisfies requirement "
-                    "(most up-to-date version is %s)",
+                    "Existing installed version (%s) is most up-to-date and "
+                    "satisfies requirement",
                     installed_version,
-                    best_candidate.version,
                 )
             else:
                 logger.debug(
-                    "Existing installed version (%s) is most up-to-date and "
-                    "satisfies requirement",
+                    "Existing installed version (%s) satisfies requirement "
+                    "(most up-to-date version is %s)",
                     installed_version,
+                    best_candidate.version,
                 )
             return None
 
-        if _should_install_candidate(best_candidate):
+        if best_installed:
+            # We have an existing version, and its the best version
             logger.debug(
-                "Using version %s (newest of versions: %s)",
-                best_candidate.version,
+                "Installed version (%s) is most up-to-date (past versions: %s)",
+                installed_version,
                 _format_versions(best_candidate_result.iter_applicable()),
             )
-            return best_candidate
+            raise BestVersionAlreadyInstalled
 
-        # We have an existing version, and its the best version
         logger.debug(
-            "Installed version (%s) is most up-to-date (past versions: %s)",
-            installed_version,
+            "Using version %s (newest of versions: %s)",
+            best_candidate.version,
             _format_versions(best_candidate_result.iter_applicable()),
         )
-        raise BestVersionAlreadyInstalled
+        return best_candidate
 
 
 def _find_name_version_sep(fragment: str, canonical_name: str) -> int:
diff --git a/src/pip/_internal/locations/__init__.py b/src/pip/_internal/locations/__init__.py
index 496844be1..516bd6078 100644
--- a/src/pip/_internal/locations/__init__.py
+++ b/src/pip/_internal/locations/__init__.py
@@ -25,10 +25,9 @@ __all__ = [
     "USER_CACHE_DIR",
     "get_bin_prefix",
     "get_bin_user",
-    "get_isolated_environment_bin_path",
-    "get_isolated_environment_lib_paths",
     "get_major_minor_version",
     "get_platlib",
+    "get_isolated_environment_lib_paths",
     "get_purelib",
     "get_scheme",
     "get_src_prefix",
@@ -527,7 +526,3 @@ def get_isolated_environment_lib_paths(prefix: str) -> List[str]:
         _log_context(prefix=prefix)
 
     return old_lib_paths
-
-
-def get_isolated_environment_bin_path(prefix: str) -> str:
-    return _sysconfig.get_isolated_environment_paths(prefix)["scripts"]
diff --git a/src/pip/_internal/locations/_sysconfig.py b/src/pip/_internal/locations/_sysconfig.py
index 38e400f94..698215720 100644
--- a/src/pip/_internal/locations/_sysconfig.py
+++ b/src/pip/_internal/locations/_sysconfig.py
@@ -213,13 +213,10 @@ def get_platlib() -> str:
     return sysconfig.get_paths()["platlib"]
 
 
-def get_isolated_environment_paths(prefix: str) -> typing.Dict[str, str]:
-    variables = {"base": prefix, "platbase": prefix}
-    if "venv" in sysconfig.get_scheme_names():
-        return sysconfig.get_paths(vars=variables, scheme="venv")
-    return sysconfig.get_paths(vars=variables)
-
-
 def get_isolated_environment_lib_paths(prefix: str) -> typing.Tuple[str, str]:
-    paths = get_isolated_environment_paths(prefix)
+    vars = {"base": prefix, "platbase": prefix}
+    if "venv" in sysconfig.get_scheme_names():
+        paths = sysconfig.get_paths(vars=vars, scheme="venv")
+    else:
+        paths = sysconfig.get_paths(vars=vars)
     return (paths["purelib"], paths["platlib"])
diff --git a/src/pip/_internal/locations/base.py b/src/pip/_internal/locations/base.py
index 3f9f896e6..3f7de0061 100644
--- a/src/pip/_internal/locations/base.py
+++ b/src/pip/_internal/locations/base.py
@@ -13,7 +13,7 @@ from pip._internal.utils.virtualenv import running_under_virtualenv
 USER_CACHE_DIR = appdirs.user_cache_dir("pip")
 
 # FIXME doesn't account for venv linked to global site-packages
-site_packages: str = sysconfig.get_path("purelib")
+site_packages: typing.Optional[str] = sysconfig.get_path("purelib")
 
 
 def get_major_minor_version() -> str:
diff --git a/src/pip/_internal/models/link.py b/src/pip/_internal/models/link.py
index a1e4d5a08..c7c4b0e9b 100644
--- a/src/pip/_internal/models/link.py
+++ b/src/pip/_internal/models/link.py
@@ -79,9 +79,6 @@ class LinkHash:
         name, value = match.groups()
         return cls(name=name, value=value)
 
-    def as_dict(self) -> Dict[str, str]:
-        return {self.name: self.value}
-
     def as_hashes(self) -> Hashes:
         """Return a Hashes instance which checks only for the current hash."""
         return Hashes({self.name: [self.value]})
@@ -168,6 +165,7 @@ class Link(KeyBasedCompareMixin):
         "requires_python",
         "yanked_reason",
         "dist_info_metadata",
+        "link_hash",
         "cache_link_parsing",
         "egg_fragment",
     ]
@@ -179,6 +177,7 @@ class Link(KeyBasedCompareMixin):
         requires_python: Optional[str] = None,
         yanked_reason: Optional[str] = None,
         dist_info_metadata: Optional[str] = None,
+        link_hash: Optional[LinkHash] = None,
         cache_link_parsing: bool = True,
         hashes: Optional[Mapping[str, str]] = None,
     ) -> None:
@@ -201,11 +200,16 @@ class Link(KeyBasedCompareMixin):
             attribute, if present, in a simple repository HTML link. This may be parsed
             into its own `Link` by `self.metadata_link()`. See PEP 658 for more
             information and the specification.
+        :param link_hash: a checksum for the content the link points to. If not
+            provided, this will be extracted from the link URL, if the URL has
+            any checksum.
         :param cache_link_parsing: A flag that is used elsewhere to determine
-            whether resources retrieved from this link should be cached. PyPI
-            URLs should generally have this set to False, for example.
+                                   whether resources retrieved from this link
+                                   should be cached. PyPI index urls should
+                                   generally have this set to False, for
+                                   example.
         :param hashes: A mapping of hash names to digests to allow us to
-            determine the validity of a download.
+                       determine the validity of a download.
         """
 
         # url can be a UNC windows share
@@ -216,18 +220,13 @@ class Link(KeyBasedCompareMixin):
         # Store the url as a private attribute to prevent accidentally
         # trying to set a new value.
         self._url = url
-
-        link_hash = LinkHash.split_hash_name_and_value(url)
-        hashes_from_link = {} if link_hash is None else link_hash.as_dict()
-        if hashes is None:
-            self._hashes = hashes_from_link
-        else:
-            self._hashes = {**hashes, **hashes_from_link}
+        self._hashes = hashes if hashes is not None else {}
 
         self.comes_from = comes_from
         self.requires_python = requires_python if requires_python else None
         self.yanked_reason = yanked_reason
         self.dist_info_metadata = dist_info_metadata
+        self.link_hash = link_hash or LinkHash.split_hash_name_and_value(self._url)
 
         super().__init__(key=url, defining_class=Link)
 
@@ -402,26 +401,29 @@ class Link(KeyBasedCompareMixin):
         if self.dist_info_metadata is None:
             return None
         metadata_url = f"{self.url_without_fragment}.metadata"
+        link_hash: Optional[LinkHash] = None
         # If data-dist-info-metadata="true" is set, then the metadata file exists,
         # but there is no information about its checksum or anything else.
         if self.dist_info_metadata != "true":
             link_hash = LinkHash.split_hash_name_and_value(self.dist_info_metadata)
-        else:
-            link_hash = None
-        if link_hash is None:
-            return Link(metadata_url)
-        return Link(metadata_url, hashes=link_hash.as_dict())
+        return Link(metadata_url, link_hash=link_hash)
 
-    def as_hashes(self) -> Hashes:
-        return Hashes({k: [v] for k, v in self._hashes.items()})
+    def as_hashes(self) -> Optional[Hashes]:
+        if self.link_hash is not None:
+            return self.link_hash.as_hashes()
+        return None
 
     @property
     def hash(self) -> Optional[str]:
-        return next(iter(self._hashes.values()), None)
+        if self.link_hash is not None:
+            return self.link_hash.value
+        return None
 
     @property
     def hash_name(self) -> Optional[str]:
-        return next(iter(self._hashes), None)
+        if self.link_hash is not None:
+            return self.link_hash.name
+        return None
 
     @property
     def show_url(self) -> str:
@@ -450,15 +452,15 @@ class Link(KeyBasedCompareMixin):
 
     @property
     def has_hash(self) -> bool:
-        return bool(self._hashes)
+        return self.link_hash is not None
 
     def is_hash_allowed(self, hashes: Optional[Hashes]) -> bool:
         """
         Return True if the link has a hash and it is allowed by `hashes`.
         """
-        if hashes is None:
+        if self.link_hash is None:
             return False
-        return any(hashes.is_hash_allowed(k, v) for k, v in self._hashes.items())
+        return self.link_hash.is_hash_allowed(hashes)
 
 
 class _CleanResult(NamedTuple):
diff --git a/src/pip/_internal/operations/check.py b/src/pip/_internal/operations/check.py
index e3bce69b2..fb3ac8b9c 100644
--- a/src/pip/_internal/operations/check.py
+++ b/src/pip/_internal/operations/check.py
@@ -75,7 +75,7 @@ def check_package_set(
             if name not in package_set:
                 missed = True
                 if req.marker is not None:
-                    missed = req.marker.evaluate({"extra": ""})
+                    missed = req.marker.evaluate()
                 if missed:
                     missing_deps.add((name, req))
                 continue
diff --git a/src/pip/_internal/req/req_install.py b/src/pip/_internal/req/req_install.py
index bb38ec09d..4543be34c 100644
--- a/src/pip/_internal/req/req_install.py
+++ b/src/pip/_internal/req/req_install.py
@@ -195,11 +195,7 @@ class InstallRequirement:
         else:
             s = "<InstallRequirement>"
         if self.satisfied_by is not None:
-            if self.satisfied_by.location is not None:
-                location = display_path(self.satisfied_by.location)
-            else:
-                location = "<memory>"
-            s += f" in {location}"
+            s += " in {}".format(display_path(self.satisfied_by.location))
         if self.comes_from:
             if isinstance(self.comes_from, str):
                 comes_from: Optional[str] = self.comes_from
diff --git a/src/pip/_internal/self_outdated_check.py b/src/pip/_internal/self_outdated_check.py
index 41cc42c56..9e2149c52 100644
--- a/src/pip/_internal/self_outdated_check.py
+++ b/src/pip/_internal/self_outdated_check.py
@@ -133,7 +133,7 @@ class UpgradePrompt:
         return Group(
             Text(),
             Text.from_markup(
-                f"{notice} A new release of pip is available: "
+                f"{notice} A new release of pip available: "
                 f"[red]{self.old}[reset] -> [green]{self.new}[reset]"
             ),
             Text.from_markup(
@@ -155,7 +155,7 @@ def was_installed_by_pip(pkg: str) -> bool:
 
 def _get_current_remote_pip_version(
     session: PipSession, options: optparse.Values
-) -> Optional[str]:
+) -> str:
     # Lets use PackageFinder to see what the latest pip version is
     link_collector = LinkCollector.create(
         session,
@@ -176,7 +176,7 @@ def _get_current_remote_pip_version(
     )
     best_candidate = finder.find_best_candidate("pip").best_candidate
     if best_candidate is None:
-        return None
+        return
 
     return str(best_candidate.version)
 
@@ -186,14 +186,11 @@ def _self_version_check_logic(
     state: SelfCheckState,
     current_time: datetime.datetime,
     local_version: DistributionVersion,
-    get_remote_version: Callable[[], Optional[str]],
+    get_remote_version: Callable[[], str],
 ) -> Optional[UpgradePrompt]:
     remote_version_str = state.get(current_time)
     if remote_version_str is None:
         remote_version_str = get_remote_version()
-        if remote_version_str is None:
-            logger.debug("No remote pip version found")
-            return None
         state.set(remote_version_str, current_time)
 
     remote_version = parse_version(remote_version_str)
diff --git a/src/pip/_internal/utils/egg_link.py b/src/pip/_internal/utils/egg_link.py
index eb57ed151..9e0da8d2d 100644
--- a/src/pip/_internal/utils/egg_link.py
+++ b/src/pip/_internal/utils/egg_link.py
@@ -1,7 +1,10 @@
+# The following comment should be removed at some point in the future.
+# mypy: strict-optional=False
+
 import os
 import re
 import sys
-from typing import List, Optional
+from typing import Optional
 
 from pip._internal.locations import site_packages, user_site
 from pip._internal.utils.virtualenv import (
@@ -54,7 +57,7 @@ def egg_link_path_from_location(raw_name: str) -> Optional[str]:
 
     This method will just return the first one found.
     """
-    sites: List[str] = []
+    sites = []
     if running_under_virtualenv():
         sites.append(site_packages)
         if not virtualenv_no_global() and user_site:
diff --git a/src/pip/_internal/utils/misc.py b/src/pip/_internal/utils/misc.py
index a3b04de9f..baa1ba7ea 100644
--- a/src/pip/_internal/utils/misc.py
+++ b/src/pip/_internal/utils/misc.py
@@ -12,6 +12,7 @@ import posixpath
 import shutil
 import stat
 import sys
+import sysconfig
 import urllib.parse
 from io import StringIO
 from itertools import filterfalse, tee, zip_longest
@@ -38,7 +39,7 @@ from pip._vendor.pyproject_hooks import BuildBackendHookCaller
 from pip._vendor.tenacity import retry, stop_after_delay, wait_fixed
 
 from pip import __version__
-from pip._internal.exceptions import CommandError
+from pip._internal.exceptions import CommandError, ExternallyManagedEnvironment
 from pip._internal.locations import get_major_minor_version
 from pip._internal.utils.compat import WINDOWS
 from pip._internal.utils.virtualenv import running_under_virtualenv
@@ -57,10 +58,10 @@ __all__ = [
     "captured_stdout",
     "ensure_dir",
     "remove_auth_from_url",
+    "check_externally_managed",
     "ConfiguredBuildBackendHookCaller",
 ]
 
-
 logger = logging.getLogger(__name__)
 
 T = TypeVar("T")
@@ -581,6 +582,21 @@ def protect_pip_from_modification_on_windows(modifying_pip: bool) -> None:
         )
 
 
+def check_externally_managed() -> None:
+    """Check whether the current environment is externally managed.
+
+    If the ``EXTERNALLY-MANAGED`` config file is found, the current environment
+    is considered externally managed, and an ExternallyManagedEnvironment is
+    raised.
+    """
+    if running_under_virtualenv():
+        return
+    marker = os.path.join(sysconfig.get_path("stdlib"), "EXTERNALLY-MANAGED")
+    if not os.path.isfile(marker):
+        return
+    raise ExternallyManagedEnvironment.from_config(marker)
+
+
 def is_console_interactive() -> bool:
     """Is this console interactive?"""
     return sys.stdin is not None and sys.stdin.isatty()


Test patch:
diff --git a/tests/functional/test_install.py b/tests/functional/test_install.py
index 3fd9329bc..7b07226c9 100644
--- a/tests/functional/test_install.py
+++ b/tests/functional/test_install.py
@@ -2351,45 +2351,3 @@ def test_install_8559_wheel_package_present(
         allow_stderr_warning=False,
     )
     assert DEPRECATION_MSG_PREFIX not in result.stderr
-
-
-@pytest.mark.skipif(
-    sys.version_info < (3, 11),
-    reason="3.11 required to find distributions via importlib metadata",
-)
-def test_install_existing_memory_distribution(script: PipTestEnvironment) -> None:
-    sitecustomize_text = textwrap.dedent(
-        """
-        import sys
-        from importlib.metadata import Distribution, DistributionFinder
-
-
-        EXAMPLE_METADATA = '''Metadata-Version: 2.1
-        Name: example
-        Version: 1.0.0
-
-        '''
-
-        class ExampleDistribution(Distribution):
-            def locate_file(self, path):
-                return path
-
-            def read_text(self, filename):
-                if filename == 'METADATA':
-                    return EXAMPLE_METADATA
-
-
-        class CustomFinder(DistributionFinder):
-            def find_distributions(self, context=None):
-                return [ExampleDistribution()]
-
-
-        sys.meta_path.append(CustomFinder())
-        """
-    )
-    with open(script.site_packages_path / "sitecustomize.py", "w") as sitecustomize:
-        sitecustomize.write(sitecustomize_text)
-
-    result = script.pip("install", "example")
-
-    assert "Requirement already satisfied: example in <memory>" in result.stdout
diff --git a/tests/functional/test_pep668.py b/tests/functional/test_pep668.py
new file mode 100644
index 000000000..1fed85e70
--- /dev/null
+++ b/tests/functional/test_pep668.py
@@ -0,0 +1,81 @@
+import json
+import pathlib
+import textwrap
+from typing import List
+
+import pytest
+
+from tests.lib import PipTestEnvironment, create_basic_wheel_for_package
+from tests.lib.venv import VirtualEnvironment
+
+
+@pytest.fixture()
+def patch_check_externally_managed(virtualenv: VirtualEnvironment) -> None:
+    # Since the tests are run from a virtual environment, and we can't
+    # guarantee access to the actual stdlib location (where EXTERNALLY-MANAGED
+    # needs to go into), we patch the check to always raise a simple message.
+    virtualenv.sitecustomize = textwrap.dedent(
+        """\
+        from pip._internal.exceptions import ExternallyManagedEnvironment
+        from pip._internal.utils import misc
+
+        def check_externally_managed():
+            raise ExternallyManagedEnvironment("I am externally managed")
+
+        misc.check_externally_managed = check_externally_managed
+        """
+    )
+
+
+@pytest.mark.parametrize(
+    "arguments",
+    [
+        pytest.param(["install"], id="install"),
+        pytest.param(["install", "--user"], id="install-user"),
+        pytest.param(["install", "--dry-run"], id="install-dry-run"),
+        pytest.param(["uninstall", "-y"], id="uninstall"),
+    ],
+)
+@pytest.mark.usefixtures("patch_check_externally_managed")
+def test_fails(script: PipTestEnvironment, arguments: List[str]) -> None:
+    result = script.pip(*arguments, "pip", expect_error=True)
+    assert "I am externally managed" in result.stderr
+
+
+@pytest.mark.parametrize(
+    "arguments",
+    [
+        pytest.param(["install", "--root"], id="install-root"),
+        pytest.param(["install", "--prefix"], id="install-prefix"),
+        pytest.param(["install", "--target"], id="install-target"),
+    ],
+)
+@pytest.mark.usefixtures("patch_check_externally_managed")
+def test_allows_if_out_of_environment(
+    script: PipTestEnvironment,
+    arguments: List[str],
+) -> None:
+    wheel = create_basic_wheel_for_package(script, "foo", "1.0")
+    result = script.pip(*arguments, script.scratch_path, wheel.as_uri())
+    assert "Successfully installed foo-1.0" in result.stdout
+    assert "I am externally managed" not in result.stderr
+
+
+@pytest.mark.usefixtures("patch_check_externally_managed")
+def test_allows_install_dry_run(
+    script: PipTestEnvironment,
+    tmp_path: pathlib.Path,
+) -> None:
+    output = tmp_path.joinpath("out.json")
+    wheel = create_basic_wheel_for_package(script, "foo", "1.0")
+    result = script.pip(
+        "install",
+        "--dry-run",
+        f"--report={output.as_posix()}",
+        wheel.as_uri(),
+        expect_stderr=True,
+    )
+    assert "Would install foo-1.0" in result.stdout
+    assert "I am externally managed" not in result.stderr
+    with output.open(encoding="utf8") as f:
+        assert isinstance(json.load(f), dict)
diff --git a/tests/unit/test_cache.py b/tests/unit/test_cache.py
index f27daa266..f1f0141c7 100644
--- a/tests/unit/test_cache.py
+++ b/tests/unit/test_cache.py
@@ -1,7 +1,7 @@
 import os
 from pathlib import Path
 
-from pip._vendor.packaging.tags import Tag, interpreter_name, interpreter_version
+from pip._vendor.packaging.tags import Tag
 
 from pip._internal.cache import WheelCache, _hash_dict
 from pip._internal.models.format_control import FormatControl
@@ -52,48 +52,6 @@ def test_cache_hash() -> None:
     assert h == "f83b32dfa27a426dec08c21bf006065dd003d0aac78e7fc493d9014d"
 
 
-def test_link_to_cache(tmpdir: Path) -> None:
-    """
-    Test that Link.from_json() produces Links with consistent cache
-    locations
-    """
-    wc = WheelCache(os.fspath(tmpdir), FormatControl())
-    # Define our expectations for stable cache path.
-    i_name = interpreter_name()
-    i_version = interpreter_version()
-    key_parts = {
-        "url": "https://files.pythonhosted.org/packages/a6/91/"
-        "86a6eac449ddfae239e93ffc1918cf33fd9bab35c04d1e963b311e347a73/"
-        "netifaces-0.11.0.tar.gz",
-        "sha256": "043a79146eb2907edf439899f262b3dfe41717d34124298ed281139a8b93ca32",
-        "interpreter_name": i_name,
-        "interpreter_version": i_version,
-    }
-    expected_hash = _hash_dict(key_parts)
-    parts = [
-        expected_hash[:2],
-        expected_hash[2:4],
-        expected_hash[4:6],
-        expected_hash[6:],
-    ]
-    pathed_hash = os.path.join(*parts)
-    # Check working from a Link produces the same result.
-    file_data = {
-        "filename": "netifaces-0.11.0.tar.gz",
-        "hashes": {
-            "sha256": key_parts["sha256"],
-        },
-        "requires-python": "",
-        "url": key_parts["url"],
-        "yanked": False,
-    }
-    page_url = "https://pypi.org/simple/netifaces/"
-    link = Link.from_json(file_data=file_data, page_url=page_url)
-    assert link
-    path = wc.get_path_for_link(link)
-    assert pathed_hash in path
-
-
 def test_get_cache_entry(tmpdir: Path) -> None:
     wc = WheelCache(os.fspath(tmpdir), FormatControl())
     persi_link = Link("https://g.c/o/r/persi")
diff --git a/tests/unit/test_collector.py b/tests/unit/test_collector.py
index 47307c00e..55676a4fc 100644
--- a/tests/unit/test_collector.py
+++ b/tests/unit/test_collector.py
@@ -6,7 +6,7 @@ import re
 import uuid
 from pathlib import Path
 from textwrap import dedent
-from typing import Dict, List, Optional, Tuple
+from typing import List, Optional, Tuple
 from unittest import mock
 
 import pytest
@@ -538,7 +538,7 @@ def test_parse_links_json() -> None:
         metadata_link.url
         == "https://example.com/files/holygrail-1.0-py3-none-any.whl.metadata"
     )
-    assert metadata_link._hashes == {"sha512": "aabdd41"}
+    assert metadata_link.link_hash == LinkHash("sha512", "aabdd41")
 
 
 @pytest.mark.parametrize(
@@ -575,41 +575,41 @@ _pkg1_requirement = Requirement("pkg1==1.0")
 
 
 @pytest.mark.parametrize(
-    "anchor_html, expected, hashes",
+    "anchor_html, expected, link_hash",
     [
         # Test not present.
         (
             '<a href="/pkg1-1.0.tar.gz"></a>',
             None,
-            {},
+            None,
         ),
         # Test with value "true".
         (
             '<a href="/pkg1-1.0.tar.gz" data-dist-info-metadata="true"></a>',
             "true",
-            {},
+            None,
         ),
         # Test with a provided hash value.
         (
             '<a href="/pkg1-1.0.tar.gz" data-dist-info-metadata="sha256=aa113592bbe"></a>',  # noqa: E501
             "sha256=aa113592bbe",
-            {},
+            None,
         ),
         # Test with a provided hash value for both the requirement as well as metadata.
         (
             '<a href="/pkg1-1.0.tar.gz#sha512=abc132409cb" data-dist-info-metadata="sha256=aa113592bbe"></a>',  # noqa: E501
             "sha256=aa113592bbe",
-            {"sha512": "abc132409cb"},
+            LinkHash("sha512", "abc132409cb"),
         ),
     ],
 )
 def test_parse_links__dist_info_metadata(
     anchor_html: str,
     expected: Optional[str],
-    hashes: Dict[str, str],
+    link_hash: Optional[LinkHash],
 ) -> None:
     link = _test_parse_links_data_attribute(anchor_html, "dist_info_metadata", expected)
-    assert link._hashes == hashes
+    assert link.link_hash == link_hash
 
 
 def test_parse_links_caches_same_page_by_url() -> None:
diff --git a/tests/unit/test_exceptions.py b/tests/unit/test_exceptions.py
index 8f8224dc8..6510b569e 100644
--- a/tests/unit/test_exceptions.py
+++ b/tests/unit/test_exceptions.py
@@ -1,12 +1,17 @@
 """Tests the presentation style of exceptions."""
 
 import io
+import locale
+import logging
+import pathlib
+import sys
 import textwrap
+from typing import Optional, Tuple
 
 import pytest
 from pip._vendor import rich
 
-from pip._internal.exceptions import DiagnosticPipError
+from pip._internal.exceptions import DiagnosticPipError, ExternallyManagedEnvironment
 
 
 class TestDiagnosticPipErrorCreation:
@@ -472,3 +477,178 @@ class TestDiagnosticPipErrorPresentation_Unicode:
               It broke. :(
             """
         )
+
+
+class TestExternallyManagedEnvironment:
+    default_text = (
+        f"The Python environment under {sys.prefix} is managed externally, "
+        f"and may not be\nmanipulated by the user. Please use specific "
+        f"tooling from the distributor of\nthe Python installation to "
+        f"interact with this environment instead.\n"
+    )
+
+    @pytest.fixture(autouse=True)
+    def patch_locale(self, monkeypatch: pytest.MonkeyPatch) -> None:
+        orig_getlocal = locale.getlocale
+
+        def fake_getlocale(category: int) -> Tuple[Optional[str], Optional[str]]:
+            """Fake getlocale() that always reports zh_Hant for LC_MESSASGES."""
+            result = orig_getlocal(category)
+            if category == getattr(locale, "LC_MESSAGES", None):
+                return "zh_Hant", result[1]
+            return result
+
+        monkeypatch.setattr(locale, "getlocale", fake_getlocale)
+
+    @pytest.fixture()
+    def marker(self, tmp_path: pathlib.Path) -> pathlib.Path:
+        marker = tmp_path.joinpath("EXTERNALLY-MANAGED")
+        marker.touch()
+        return marker
+
+    def test_invalid_config_format(
+        self,
+        caplog: pytest.LogCaptureFixture,
+        marker: pathlib.Path,
+    ) -> None:
+        marker.write_text("invalid", encoding="utf8")
+
+        with caplog.at_level(logging.WARNING, "pip._internal.exceptions"):
+            exc = ExternallyManagedEnvironment.from_config(marker)
+        assert len(caplog.records) == 1
+        assert caplog.records[-1].getMessage() == f"Failed to read {marker}"
+
+        assert str(exc.context) == self.default_text
+
+    @pytest.mark.parametrize(
+        "config",
+        [
+            pytest.param("", id="empty"),
+            pytest.param("[foo]\nblah = blah", id="no-section"),
+            pytest.param("[externally-managed]\nblah = blah", id="no-key"),
+        ],
+    )
+    def test_config_without_key(
+        self,
+        caplog: pytest.LogCaptureFixture,
+        marker: pathlib.Path,
+        config: str,
+    ) -> None:
+        marker.write_text(config, encoding="utf8")
+
+        with caplog.at_level(logging.WARNING, "pip._internal.exceptions"):
+            exc = ExternallyManagedEnvironment.from_config(marker)
+        assert not caplog.records
+        assert str(exc.context) == self.default_text
+
+    @pytest.mark.skipif(
+        sys.platform == "win32",
+        reason="Localization disabled on Windows",
+    )
+    @pytest.mark.parametrize(
+        "config, expected",
+        [
+            pytest.param(
+                """\
+                [externally-managed]
+                Error = 最後
+                Error-en = English
+                Error-zh = 中文
+                Error-zh_Hant = 繁體
+                Error-zh_Hans = 简体
+                """,
+                "繁體",
+                id="full",
+            ),
+            pytest.param(
+                """\
+                [externally-managed]
+                Error = 最後
+                Error-en = English
+                Error-zh = 中文
+                Error-zh_Hans = 简体
+                """,
+                "中文",
+                id="no-variant",
+            ),
+            pytest.param(
+                """\
+                [externally-managed]
+                Error = 最後
+                Error-en = English
+                """,
+                "最後",
+                id="fallback",
+            ),
+        ],
+    )
+    def test_config_canonical(
+        self,
+        caplog: pytest.LogCaptureFixture,
+        marker: pathlib.Path,
+        config: str,
+        expected: str,
+    ) -> None:
+        marker.write_text(
+            textwrap.dedent(config),
+            encoding="utf8",
+        )
+
+        with caplog.at_level(logging.WARNING, "pip._internal.exceptions"):
+            exc = ExternallyManagedEnvironment.from_config(marker)
+        assert not caplog.records
+        assert str(exc.context) == expected
+
+    @pytest.mark.skipif(
+        sys.platform != "win32",
+        reason="Non-Windows should implement localization",
+    )
+    @pytest.mark.parametrize(
+        "config",
+        [
+            pytest.param(
+                """\
+                [externally-managed]
+                Error = 最後
+                Error-en = English
+                Error-zh = 中文
+                Error-zh_Hant = 繁體
+                Error-zh_Hans = 简体
+                """,
+                id="full",
+            ),
+            pytest.param(
+                """\
+                [externally-managed]
+                Error = 最後
+                Error-en = English
+                Error-zh = 中文
+                Error-zh_Hans = 简体
+                """,
+                id="no-variant",
+            ),
+            pytest.param(
+                """\
+                [externally-managed]
+                Error = 最後
+                Error-en = English
+                """,
+                id="fallback",
+            ),
+        ],
+    )
+    def test_config_canonical_no_localization(
+        self,
+        caplog: pytest.LogCaptureFixture,
+        marker: pathlib.Path,
+        config: str,
+    ) -> None:
+        marker.write_text(
+            textwrap.dedent(config),
+            encoding="utf8",
+        )
+
+        with caplog.at_level(logging.WARNING, "pip._internal.exceptions"):
+            exc = ExternallyManagedEnvironment.from_config(marker)
+        assert not caplog.records
+        assert str(exc.context) == "最後"