Repository: GoogleCloudPlatform/artifact-registry-python-tools (language: python)

Issue:
Status code 403 with pip 21.1.x and keyrings.google-artifactregistry-auth 0.0.2
- WSL2/Fedora 33
- Python 3.9.2
- works with pip 21.0.1
- cannot install a package located in my GAR with pip 21.1 or 21.1.1.

```bash
Getting page https://europe-pypi.pkg.dev/xxxxxx-xix9/python/simple/cpe-common/
Found index url https://europe-pypi.pkg.dev/xxxxxx-xix9/python/simple/
Looking up "https://europe-pypi.pkg.dev/xxxxxx-xix9/python/simple/cpe-common/" in the cache
Request header has "max_age" as 0, cache bypassed
Starting new HTTPS connection (1): europe-pypi.pkg.dev:443
https://europe-pypi.pkg.dev:443 "GET /xxxxxx-xix9/python/simple/cpe-common/ HTTP/1.1" 403 0
Status code 403 not in (200, 203, 300, 301)
Could not fetch URL https://europe-pypi.pkg.dev/xxxxxx-xix9/python/simple/cpe-common/: 403 Client Error: Forbidden for url: https://europe-pypi.pkg.dev/xxxxxx-xix9/python/simple/cpe-common/ - skipping
```


```
╰(.venv)─» pip freeze
appdirs==1.4.4
bleach==3.3.0
cachetools==4.2.2
certifi==2020.12.5
cffi==1.14.5
chardet==4.0.0
colorama==0.4.4
cryptography==3.4.7
distlib==0.3.1
docutils==0.17.1
filelock==3.0.12
google-auth==1.30.0
idna==2.10
importlib-metadata==4.0.1
jeepney==0.6.0
keyring==23.0.1
keyrings.google-artifactregistry-auth==0.0.2
packaging==20.9
pkginfo==1.7.0
pluggy==0.13.1
py==1.10.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.20
Pygments==2.9.0
pyparsing==2.4.7
readme-renderer==29.0
requests==2.25.1
requests-toolbelt==0.9.1
rfc3986==1.4.0
rsa==4.7.2
SecretStorage==3.3.1
six==1.15.0
toml==0.10.2
tox==3.23.0
tqdm==4.60.0
twine==3.4.1
urllib3==1.26.4
virtualenv==20.4.4
webencodings==0.5.1
zipp==3.4.1
```

```bash
╰(.venv)─» keyring --list-backends
keyrings.gauth.GooglePyPIAuth (priority: 9)
keyring.backends.chainer.ChainerBackend (priority: -1)
keyring.backends.fail.Keyring (priority: 0)
```



Discussion / hints:
It looks like the keyring is not called at all with pip 21.1.x, because using pip 21.0.1, I can see the logs messages from the keyring that I don't see with pip 21.1.x

```
Getting page https://europe-pypi.pkg.dev/xxxxxx-xix9/python/simple/cpe-common/
Found index url https://europe-pypi.pkg.dev/xxxxxx-xix9/python/simple/
Getting credentials from keyring for https://europe-pypi.pkg.dev/xxxxxx-xix9/python/simple/
Checking None for explicit credentials as part of auth process...
Checking Cloud SDK credentials as part of auth process...
Making request: POST https://oauth2.googleapis.com/token
Starting new HTTPS connection (1): oauth2.googleapis.com:443
https://oauth2.googleapis.com:443 "POST /token HTTP/1.1" 200 None
Found credentials in keyring for europe-pypi.pkg.dev
Looking up "https://europe-pypi.pkg.dev/xxxxxx-xix9/python/simple/cpe-common/" in the cache
Request header has "max_age" as 0, cache bypassed
Starting new HTTPS connection (1): europe-pypi.pkg.dev:443
```
maybe linked to pypa/pip#9870
@megan-kuo @Gridasov : can you confirm something changed very recently and now the GAR returns a 401 instead of a 403 when not authenticated? 
This generated another kind of issue on our side, cf https://github.com/pypa/pip/issues/9870#issuecomment-841076151
Yes, Artifact Registry now returns a 401 instead of a 403 when not authenticated. 

Can you elaborate a bit more on your issue? It is the intended behavior that users should have to set up authentication by installing `keyrings.google-artifactregistry-auth` before installing packages from an Artifact Registry Repository (see this quickstart guide here: [https://cloud.google.com/artifact-registry/docs/python/quickstart]

Also, you can use ar-contact@google.com to reach our team :)
- Given I have a `pip.conf` that refers to a GAR url as an `extra-url`

```
[global]
timeout = 5
extra-index-url = https://europe-pypi.pkg.dev/my-gcp-project/python/simple/
```

- Given I am in a new virtual env that does not have the keyring packages installed yet (keyring==23.0.1 keyrings.google-artifactregistry-auth==0.0.2)
- When I want to install the packages `keyring` and `keyrings.google-artifactregistry-auth==0.0.2`
- Then I am prompted for a Username/Password when pip tries to reach the package repository hosted on the GAR

If the expected mandatory behaviour is to install the keyrings packages before configuring the pip.conf, what is the way to do it inside `tox` venv? Using a custom `install_command`:

```ini
[testenv]
install_command = {toxinidir}/path/to/tox_custom_install_command.sh {opts} {packages}
```

- `tox_custom_install_command.sh` content:

```bash
#!/usr/bin/env bash

PIP_CONFIG_FILE=/dev/null pip install --upgrade pip==21.1.1
PIP_CONFIG_FILE=/dev/null pip install 'keyring>=23.0.1'
PIP_CONFIG_FILE=/dev/null pip install 'keyrings.google-artifactregistry-auth>=0.0.2'
pip install "$@"

```

? 
This feels more like a hack to me, but I may be wrong :)

Hello, I'm both `tox` and `virtualenv` maintainer. @di summoned me here.

> If the expected mandatory behaviour is to install the keyrings packages before configuring the pip.conf, what is the way to do it inside `tox` venv?

tox under the hood uses virtualenv, not venv. The problem described to me sounds like a valid virtual environment on these machines is one that doesn't just have pip+setuptools+wheel in it but also `keyring` and `keyrings.google-artifactregistry-auth`. Therefore my recommendation is to alter the virtual environment creation accordingly, you can extend virtualenv package seed logic, see https://virtualenv.pypa.io/en/latest/extend.html#seed-mechanism. Likely your best option is to extend the current seeder and seed these extra packages afterward. You can then install this tox plugin within the machine manually. Sadly you can't use https://tox.readthedocs.io/en/latest/config.html#conf-requires because that creates a virtual environment without this plugin being already active (chicken-egg problem).

Your hacky solution works too, but is kinda hacky 😮 

Fix patch:
diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..32f1514
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,4 @@
+dist/
+build/
+.eggs/
+*.egg-info/
diff --git a/README.md b/README.md
index f601e6c..a6ff744 100644
--- a/README.md
+++ b/README.md
@@ -73,3 +73,37 @@ To use the keyring backend:
    * `keyrings.gauth.GooglePyPIAuth (priority: 9)`
    * `keyring.backends.chainer.ChainerBackend (priority: 10)`
    * `keyring.backends.fail.Keyring (priority: 0)`
+
+## Usage with other tools
+
+### Usage with `tox`
+
+The [`tox` tool](https://pypi.org/project/tox/) is a testing and automation tool.
+
+Because the credential helper needs to be installed _before_ any private
+dependencies are installed, it needs to be bootstrapped into the `tox`
+environment via a plugin.
+
+To do this, specify the `keyrings.google-artifactregistry-auth` package via the
+[`requires`](https://tox.readthedocs.io/en/latest/config.html#conf-requires)
+requirement in your `tox.ini` file:
+
+    ```
+    [tox]
+    envlist = py
+    requires = keyrings.google-artifactregistry-auth
+    ```
+
+You can then configure your `tox.ini` file to use the Artifact Registry repo as
+an extra index, and specify both public and private dependencies:
+
+    ```
+    [testenv]
+    setenv =
+        PIP_EXTRA_INDEX_URL = https://[REGION]-pypi.pkg.dev/[PROJECT_ID]/[REPOSITORY]/simple
+    deps =
+        # samplepackage will be installed directly from PyPI
+        samplepackage
+        # mypackage will be installed from the Artifact Registry repository
+        mypackage
+    ```
diff --git a/google_artifactregistry_auth/__init__.py b/google_artifactregistry_auth/__init__.py
new file mode 100644
index 0000000..3b93d0b
--- /dev/null
+++ b/google_artifactregistry_auth/__init__.py
@@ -0,0 +1,1 @@
+__version__ = "0.0.2"
diff --git a/google_artifactregistry_auth/tox_bootstrap.py b/google_artifactregistry_auth/tox_bootstrap.py
new file mode 100644
index 0000000..ad8c8a3
--- /dev/null
+++ b/google_artifactregistry_auth/tox_bootstrap.py
@@ -0,0 +1,17 @@
+import pluggy
+
+import google_artifactregistry_auth
+
+hookimpl = pluggy.HookimplMarker("tox")
+
+
+@hookimpl
+def tox_testenv_install_deps(venv, action):
+    venv._install(
+        [
+            "keyrings.google-artifactregistry-auth=={}".format(
+                google_artifactregistry_auth.__version__
+            )
+        ],
+        action=action,
+    )
diff --git a/pyproject.toml b/pyproject.toml
new file mode 100644
index 0000000..d1f33e8
--- /dev/null
+++ b/pyproject.toml
@@ -0,0 +1,3 @@
+[build-system]
+# Minimum requirements for the build system to execute.
+requires = ["setuptools>=46.4.0", "wheel"]
diff --git a/setup.cfg b/setup.cfg
index cc7f95a..b3c3b89 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -1,7 +1,7 @@
 [metadata]
 license_file = LICENSE
 name = keyrings.google-artifactregistry-auth
-version = 0.0.2
+version = attr: google_artifactregistry_auth.__version__
 author = Megan Kuo
 author_email = megankuo@google.com
 description = Keyring backend for Google Auth tokens
@@ -23,6 +23,7 @@ install_requires =
 	keyring
 	google-auth
 	requests
+	pluggy
 setup_requires = setuptools_scm[toml] >= 3.4.1
 
 [options.extras_require]
@@ -40,4 +41,5 @@ testing =
 [options.entry_points]
 keyring.backends =
 	Google Auth = keyrings.gauth
-
+tox =
+	tox_bootstrap = google_artifactregistry_auth.tox_bootstrap


Test patch:
diff --git a/.github/workflows/test-tox-plugin.yml b/.github/workflows/test-tox-plugin.yml
new file mode 100644
index 0000000..7ca64a2
--- /dev/null
+++ b/.github/workflows/test-tox-plugin.yml
@@ -0,0 +1,17 @@
+on: [push, pull_request]
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: [ '3.6', '3.7', '3.8', '3.9' ]
+    name: Tox plugin - Python ${{ matrix.python-version }}
+    steps:
+      - uses: actions/checkout@v2
+      - name: Setup python
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python-version }}
+      - run: python -m pip install tox
+      - run: python -m pip install -e .
+      - run: python -m tox -c tests/tox-plugin/tox.ini
diff --git a/tests/tox-plugin/setup.py b/tests/tox-plugin/setup.py
new file mode 100644
index 0000000..6068493
--- /dev/null
+++ b/tests/tox-plugin/setup.py
@@ -0,0 +1,3 @@
+from setuptools import setup
+
+setup()
diff --git a/tests/tox-plugin/tox.ini b/tests/tox-plugin/tox.ini
new file mode 100644
index 0000000..c9960c1
--- /dev/null
+++ b/tests/tox-plugin/tox.ini
@@ -0,0 +1,6 @@
+[tox]
+envlist = py
+requires = keyrings.google-artifactregistry-auth
+
+[testenv]
+commands = python -m pip show keyrings.google-artifactregistry-auth