Repository: jupyterhub/jupyterhub

Issue:
302 on token-authenticated API request to not-running server
<!-- Thank you for contributing. These HTML comments will not render in the issue, but you can delete them once you've read them if you prefer! -->

### Bug description
We at DataSpell faced the problem with JupyterHub 2.0. The IDE can't access Jupyter's Contents API any more. 

Take a look at the events happening when a user tries to register a JupyterHub 2.0 server in DataSpell:
* The user presses the button that adds a Jupyter connection, then they provide a valid URL to the server
* The IDE makes a request to the server, reaches out "jupyterhub" in the response headers, and proposes obtaining an API token
* The user successfully gets the token at /hub/api/token in a web browser, and provides the token to the IDE
* The IDE sends a GET request to /user/username/api/contents/ with a header `Authorization: Token ...`
* `PrefixRedirectHandler` responds with HTTP 302, suggesting to go to `/hub/user/username/api/contents/ `
* The IDE follows the redirect, and `UserUrlHandler` redirects to `/hub/login`, even though the Authorization header is provided.

That behaviour was introduced in edfdf672d829dd9dc2d7f285bc5b2a5693291425. Although the commit message states that it was to avoid surprises, I have to admit that it actually was a surprise for us.

### How to reproduce
So, to simplify things, here are steps how to reproduce without DataSpell:
* Start a JupyterHub 2.0.0, spawn a notebook server, obtain an API token
* Execute `curl -vL http://server/user/username/api/contents/ -H 'Authorization: Token ...'`

#### Expected behaviour
The response of the server on a Contents API request is a valid JSON, like it used to be with JupyterHub 1.4.*

#### Actual behaviour
JupyterHub disallows to access Contents API without a cookie which can be obtained only through passing a login form, and it redirects to a login page.

### Your personal set up
<!--
Tell us a little about the system you're using.
Please include information about how you installed,
e.g. are you using a distribution such as zero-to-jupyterhub or the-littlest-jupyterhub.
 -->

 - OS:
   Actually, it doesn't matter, but let it be macOS.
 - Version(s):
   `jupyterhub --version` writes `2.0.0`
    Python 3.10.1 (and also Python 3.9.5, and I bet any other version too).

- <details><summary>Full environment</summary>

    ```
    alembic==1.7.5
    anyio==3.4.0
    appnope==0.1.2
    argon2-cffi==21.3.0
    argon2-cffi-bindings==21.2.0
    async-generator==1.10
    attrs==21.2.0
    Babel==2.9.1
    backcall==0.2.0
    bleach==4.1.0
    certifi==2021.10.8
    certipy==0.1.3
    cffi==1.15.0
    charset-normalizer==2.0.9
    cryptography==36.0.0
    debugpy==1.5.1
    decorator==5.1.0
    defusedxml==0.7.1
    entrypoints==0.3
    greenlet==1.1.2
    idna==3.3
    ipykernel==6.6.0
    ipython==7.30.1
    ipython-genutils==0.2.0
    ipywidgets==7.6.5
    jedi==0.18.1
    Jinja2==3.0.3
    json5==0.9.6
    jsonschema==4.2.1
    jupyter==1.0.0
    jupyter-client==7.1.0
    jupyter-console==6.4.0
    jupyter-core==4.9.1
    jupyter-server==1.13.1
    jupyter-telemetry==0.1.0
    jupyterhub==2.0.0
    jupyterlab==3.2.5
    jupyterlab-pygments==0.1.2
    jupyterlab-server==2.9.0
    jupyterlab-widgets==1.0.2
    Mako==1.1.6
    MarkupSafe==2.0.1
    matplotlib-inline==0.1.3
    mistune==0.8.4
    nbclassic==0.3.4
    nbclient==0.5.9
    nbconvert==6.3.0
    nbformat==5.1.3
    nest-asyncio==1.5.4
    notebook==6.4.6
    numpy==1.21.4
    oauthlib==3.1.1
    packaging==21.3
    pamela==1.0.0
    pandas==1.3.5
    pandocfilters==1.5.0
    parso==0.8.3
    pexpect==4.8.0
    pickleshare==0.7.5
    prometheus-client==0.12.0
    prompt-toolkit==3.0.24
    ptyprocess==0.7.0
    pycparser==2.21
    Pygments==2.10.0
    pyOpenSSL==21.0.0
    pyparsing==3.0.6
    pyrsistent==0.18.0
    python-dateutil==2.8.2
    python-json-logger==2.0.2
    pytz==2021.3
    pyzmq==22.3.0
    qtconsole==5.2.2
    QtPy==1.11.3
    requests==2.26.0
    ruamel.yaml==0.17.17
    Send2Trash==1.8.0
    six==1.16.0
    sniffio==1.2.0
    SQLAlchemy==1.4.28
    terminado==0.12.1
    testpath==0.5.0
    tornado==6.1
    traitlets==5.1.1
    urllib3==1.26.7
    wcwidth==0.2.5
    webencodings==0.5.1
    websocket-client==1.2.3
    widgetsnbextension==3.5.2
    ```
</details>

- <details><summary>Configuration</summary>
    No custom configuration files at all.
    
    ```python
    # jupyterhub_config.py
    ```
    </details>

- <details><summary>Logs</summary>
    The log is mixed up with some debugging prints I added, but believe me I didn't change anything in the logic.

    ```
    /Users/lagunov/src/jetbrains/idea/main/python/jupyter/setup-jupyter-env/build/pythons/py310_jupyter_hub_2/lib/python3.10/site-packages/traitlets/traitlets.py:2562: FutureWarning: --Spawner.args=['--Application.log_level=DEBUG'] for containers is deprecated in traitlets 5.0. You can pass `--Spawner.args item` ... multiple times to add items to a list.
      warn(
    /Users/lagunov/src/jetbrains/idea/main/python/jupyter/setup-jupyter-env/build/pythons/py310_jupyter_hub_2/lib/python3.10/site-packages/traitlets/traitlets.py:2562: FutureWarning: --Spawner.cmd=['jupyterhub-singleuser', '--allow-root'] for containers is deprecated in traitlets 5.0. You can pass `--Spawner.cmd item` ... multiple times to add items to a list.
      warn(
    /Users/lagunov/src/jetbrains/idea/main/python/jupyter/setup-jupyter-env/build/pythons/py310_jupyter_hub_2/lib/python3.10/site-packages/traitlets/traitlets.py:2562: FutureWarning: --Authenticator.admin_users={'lagunov'} for containers is deprecated in traitlets 5.0. You can pass `--Authenticator.admin_users item` ... multiple times to add items to a list.
      warn(
    [W 2021-12-20 19:05:00.383 JupyterHub app:786] JupyterHub.proxy_api_port is deprecated in JupyterHub 0.8, use ConfigurableHTTPProxy.api_url
    [D 2021-12-20 19:05:00.384 JupyterHub application:731] Looking for jupyterhub_config in /private/tmp/hub
    [I 2021-12-20 19:05:00.385 JupyterHub app:2717] Running JupyterHub version 2.0.0
    [I 2021-12-20 19:05:00.385 JupyterHub app:2747] Using Authenticator: jupyterhub.auth.PAMAuthenticator-2.0.0
    [I 2021-12-20 19:05:00.385 JupyterHub app:2747] Using Spawner: jupyterhub.spawner.LocalProcessSpawner-2.0.0
    [I 2021-12-20 19:05:00.385 JupyterHub app:2747] Using Proxy: jupyterhub.proxy.ConfigurableHTTPProxy-2.0.0
    [D 2021-12-20 19:05:00.387 JupyterHub app:2678] Could not load pycurl: No module named 'pycurl'
        pycurl is recommended if you have a large number of users.
    [I 2021-12-20 19:05:00.387 JupyterHub app:1596] Loading cookie_secret from /private/tmp/hub/jupyterhub_cookie_secret
    [D 2021-12-20 19:05:00.388 JupyterHub app:1763] Connecting to db: sqlite:///jupyterhub.sqlite
    [D 2021-12-20 19:05:00.417 JupyterHub orm:955] database schema version found: 833da8570507
    [I 2021-12-20 19:05:00.482 JupyterHub proxy:496] Generating new CONFIGPROXY_AUTH_TOKEN
    [D 2021-12-20 19:05:00.482 JupyterHub app:2007] Loading default roles to database
    [I 2021-12-20 19:05:00.497 JupyterHub app:1913] Not using allowed_users. Any authenticated user will be allowed.
    [D 2021-12-20 19:05:00.503 JupyterHub app:2229] Purging expired APITokens
    [D 2021-12-20 19:05:00.507 JupyterHub app:2229] Purging expired OAuthCodes
    [D 2021-12-20 19:05:00.509 JupyterHub app:2080] Loading predefined roles from config file to database
    [D 2021-12-20 19:05:00.524 JupyterHub app:2375] Initializing spawners
    [D 2021-12-20 19:05:00.527 JupyterHub app:2506] Loaded users:
        
    [I 2021-12-20 19:05:00.527 JupyterHub app:2786] Initialized 0 spawners in 0.004 seconds
    [W 2021-12-20 19:05:00.529 JupyterHub proxy:565] Found proxy pid file: /private/tmp/hub/jupyterhub-proxy.pid
    [W 2021-12-20 19:05:00.530 JupyterHub proxy:577] Proxy no longer running at pid=5806
    [D 2021-12-20 19:05:00.530 JupyterHub proxy:618] Removing proxy pid file jupyterhub-proxy.pid
    [W 2021-12-20 19:05:00.531 JupyterHub proxy:687] Running JupyterHub without SSL.  I hope there is SSL termination happening somewhere else...
    [I 2021-12-20 19:05:00.531 JupyterHub proxy:691] Starting proxy @ http://127.0.0.1:8000/
    [D 2021-12-20 19:05:00.531 JupyterHub proxy:692] Proxy cmd: ['configurable-http-proxy', '--ip', '127.0.0.1', '--port', '8000', '--api-ip', '127.0.0.1', '--api-port', '56323', '--error-target', 'http://127.0.0.1:56322/hub/error']
    [D 2021-12-20 19:05:00.535 JupyterHub proxy:610] Writing proxy pid file: jupyterhub-proxy.pid
    19:05:00.739 [ConfigProxy] [32minfo[39m: Proxying http://127.0.0.1:8000 to (no default)
    19:05:00.741 [ConfigProxy] [32minfo[39m: Proxy API at http://127.0.0.1:56323/api/routes
    [D 2021-12-20 19:05:00.799 JupyterHub proxy:728] Proxy started and appears to be up
    [D 2021-12-20 19:05:00.805 JupyterHub proxy:821] Proxy: Fetching GET http://127.0.0.1:56323/api/routes
    19:05:00.819 [ConfigProxy] [32minfo[39m: 200 GET /api/routes 
    [I 2021-12-20 19:05:00.820 JupyterHub app:3035] Hub API listening on http://127.0.0.1:56322/hub/
    [D 2021-12-20 19:05:00.820 JupyterHub proxy:343] Fetching routes to check
    [D 2021-12-20 19:05:00.821 JupyterHub proxy:821] Proxy: Fetching GET http://127.0.0.1:56323/api/routes
    19:05:00.823 [ConfigProxy] [32minfo[39m: 200 GET /api/routes 
    [D 2021-12-20 19:05:00.823 JupyterHub proxy:346] Checking routes
    [I 2021-12-20 19:05:00.823 JupyterHub proxy:431] Adding route for Hub: / => http://127.0.0.1:56322
    [D 2021-12-20 19:05:00.824 JupyterHub proxy:821] Proxy: Fetching POST http://127.0.0.1:56323/api/routes/
    19:05:00.826 [ConfigProxy] [32minfo[39m: Adding route / -> http://127.0.0.1:56322
    19:05:00.827 [ConfigProxy] [32minfo[39m: Route added / -> http://127.0.0.1:56322
    19:05:00.827 [ConfigProxy] [32minfo[39m: 201 POST /api/routes/ 
    [I 2021-12-20 19:05:00.829 JupyterHub app:3101] JupyterHub is now running at http://127.0.0.1:8000/
    [D 2021-12-20 19:05:00.830 JupyterHub app:2710] It took 0.454 seconds for the Hub to start
    [I 2021-12-20 19:05:24.069 JupyterHub web:2175] *** LAGUNOV DBG: GET / is handled by $<class 'jupyterhub.handlers.base.PrefixRedirectHandler'>
    [I 2021-12-20 19:05:24.077 JupyterHub base:422] LAGUNOV DBG get_current_user self._accept_token_auth=False self._accept_cookie_auth=True
    [I 2021-12-20 19:05:24.078 JupyterHub web:812] LAGUNOV DBG redirect 302 /hub/
    [I 2021-12-20 19:05:24.078 JupyterHub log:189] 302 GET / -> /hub/ (@127.0.0.1) 9.74ms
    [I 2021-12-20 19:05:24.082 JupyterHub web:2175] *** LAGUNOV DBG: GET /hub/ is handled by $<class 'jupyterhub.handlers.pages.RootHandler'>
    [I 2021-12-20 19:05:24.083 JupyterHub base:422] LAGUNOV DBG get_current_user self._accept_token_auth=False self._accept_cookie_auth=True
    [I 2021-12-20 19:05:24.084 JupyterHub web:812] LAGUNOV DBG redirect 302 /hub/login?next=%2Fhub%2F
    [I 2021-12-20 19:05:24.085 JupyterHub log:189] 302 GET /hub/ -> /hub/login?next=%2Fhub%2F (@127.0.0.1) 2.56ms
    [I 2021-12-20 19:05:24.088 JupyterHub web:2175] *** LAGUNOV DBG: GET /hub/login?next=%2Fhub%2F is handled by $<class 'jupyterhub.handlers.login.LoginHandler'>
    [I 2021-12-20 19:05:24.089 JupyterHub base:422] LAGUNOV DBG get_current_user self._accept_token_auth=False self._accept_cookie_auth=True
    [I 2021-12-20 19:05:24.125 JupyterHub log:189] 200 GET /hub/login?next=%2Fhub%2F (@127.0.0.1) 37.64ms
    [I 2021-12-20 19:05:26.166 JupyterHub web:2175] *** LAGUNOV DBG: GET /hub/token is handled by $<class 'jupyterhub.handlers.pages.TokenPageHandler'>
    [I 2021-12-20 19:05:26.167 JupyterHub base:422] LAGUNOV DBG get_current_user self._accept_token_auth=False self._accept_cookie_auth=True
    [W 2021-12-20 19:05:26.167 JupyterHub base:393] Invalid or expired cookie token
    [I 2021-12-20 19:05:26.168 JupyterHub web:812] LAGUNOV DBG redirect 302 /hub/login?next=%2Fhub%2Ftoken
    [I 2021-12-20 19:05:26.169 JupyterHub log:189] 302 GET /hub/token -> /hub/login?next=%2Fhub%2Ftoken (@127.0.0.1) 3.05ms
    [I 2021-12-20 19:05:26.177 JupyterHub web:2175] *** LAGUNOV DBG: GET /hub/login?next=%2Fhub%2Ftoken is handled by $<class 'jupyterhub.handlers.login.LoginHandler'>
    [I 2021-12-20 19:05:26.178 JupyterHub base:422] LAGUNOV DBG get_current_user self._accept_token_auth=False self._accept_cookie_auth=True
    [I 2021-12-20 19:05:26.180 JupyterHub log:189] 200 GET /hub/login?next=%2Fhub%2Ftoken (@127.0.0.1) 2.77ms
    [I 2021-12-20 19:05:26.194 JupyterHub web:2175] *** LAGUNOV DBG: GET /hub/logo is handled by $<class 'jupyterhub.handlers.static.LogoHandler'>
    [D 2021-12-20 19:05:26.203 JupyterHub log:189] 304 GET /hub/logo (@127.0.0.1) 8.04ms
    [I 2021-12-20 19:05:32.714 JupyterHub web:2175] *** LAGUNOV DBG: POST /hub/login?next=%2Fhub%2Ftoken is handled by $<class 'jupyterhub.handlers.login.LoginHandler'>
    [I 2021-12-20 19:05:32.714 JupyterHub base:422] LAGUNOV DBG get_current_user self._accept_token_auth=False self._accept_cookie_auth=True
    [D 2021-12-20 19:05:32.966 JupyterHub base:568] Setting cookie for lagunov: jupyterhub-hub-login
    [D 2021-12-20 19:05:32.966 JupyterHub base:564] Setting cookie jupyterhub-hub-login: {'httponly': True, 'path': '/hub/'}
    [I 2021-12-20 19:05:32.966 JupyterHub base:803] User logged in: lagunov
    [I 2021-12-20 19:05:32.967 JupyterHub web:812] LAGUNOV DBG redirect 302 /hub/token
    [I 2021-12-20 19:05:32.967 JupyterHub log:189] 302 POST /hub/login?next=%2Fhub%2Ftoken -> /hub/token (lagunov@127.0.0.1) 253.34ms
    [I 2021-12-20 19:05:32.974 JupyterHub web:2175] *** LAGUNOV DBG: GET /hub/token is handled by $<class 'jupyterhub.handlers.pages.TokenPageHandler'>
    [I 2021-12-20 19:05:32.975 JupyterHub base:422] LAGUNOV DBG get_current_user self._accept_token_auth=False self._accept_cookie_auth=True
    [D 2021-12-20 19:05:32.996 JupyterHub user:347] Creating <class 'jupyterhub.spawner.LocalProcessSpawner'> for lagunov:
    [I 2021-12-20 19:05:32.998 JupyterHub log:189] 200 GET /hub/token (lagunov@127.0.0.1) 23.94ms
    [I 2021-12-20 19:05:33.050 JupyterHub web:2175] *** LAGUNOV DBG: GET /hub/static/js/token.js?v=20211220190500 is handled by $<class 'jupyterhub.handlers.static.CacheControlStaticFilesHandler'>
    [D 2021-12-20 19:05:33.051 JupyterHub log:189] 200 GET /hub/static/js/token.js?v=20211220190500 (@127.0.0.1) 1.61ms
    [I 2021-12-20 19:05:33.065 JupyterHub web:2175] *** LAGUNOV DBG: GET /hub/static/js/jhapi.js?v=20211220190500 is handled by $<class 'jupyterhub.handlers.static.CacheControlStaticFilesHandler'>
    [D 2021-12-20 19:05:33.066 JupyterHub log:189] 200 GET /hub/static/js/jhapi.js?v=20211220190500 (@127.0.0.1) 1.24ms
    [I 2021-12-20 19:05:33.066 JupyterHub web:2175] *** LAGUNOV DBG: GET /hub/static/components/moment/moment.js?v=20211220190500 is handled by $<class 'jupyterhub.handlers.static.CacheControlStaticFilesHandler'>
    [D 2021-12-20 19:05:33.071 JupyterHub log:189] 200 GET /hub/static/components/moment/moment.js?v=20211220190500 (@127.0.0.1) 4.06ms
    [I 2021-12-20 19:05:33.077 JupyterHub web:2175] *** LAGUNOV DBG: GET /hub/static/js/utils.js?v=20211220190500 is handled by $<class 'jupyterhub.handlers.static.CacheControlStaticFilesHandler'>
    [D 2021-12-20 19:05:33.079 JupyterHub log:189] 200 GET /hub/static/js/utils.js?v=20211220190500 (@127.0.0.1) 2.08ms
    [I 2021-12-20 19:05:34.380 JupyterHub web:2175] *** LAGUNOV DBG: POST /hub/api/users/lagunov/tokens is handled by $<class 'jupyterhub.apihandlers.users.UserTokenListAPIHandler'>
    [I 2021-12-20 19:05:34.381 JupyterHub base:422] LAGUNOV DBG get_current_user self._accept_token_auth=True self._accept_cookie_auth=True
    [D 2021-12-20 19:05:34.389 JupyterHub roles:450] Assigning default role to token
    [I 2021-12-20 19:05:34.394 JupyterHub roles:454] Added role token to token <APIToken('c931...', user='lagunov', client_id='jupyterhub')>
    [I 2021-12-20 19:05:34.397 JupyterHub users:429] User lagunov requested new API token
    [I 2021-12-20 19:05:34.402 JupyterHub log:189] 201 POST /hub/api/users/lagunov/tokens (lagunov@127.0.0.1) 21.45ms
    [I 2021-12-20 19:05:39.676 JupyterHub web:2175] *** LAGUNOV DBG: GET /user/lagunov/api/contents/ is handled by $<class 'jupyterhub.handlers.base.PrefixRedirectHandler'>
    [I 2021-12-20 19:05:39.677 JupyterHub base:422] LAGUNOV DBG get_current_user self._accept_token_auth=False self._accept_cookie_auth=True
    [I 2021-12-20 19:05:39.677 JupyterHub web:812] LAGUNOV DBG redirect 302 /hub/user/lagunov/api/contents/
    [I 2021-12-20 19:05:39.678 JupyterHub log:189] 302 GET /user/lagunov/api/contents/ -> /hub/user/lagunov/api/contents/ (@127.0.0.1) 1.95ms
    [I 2021-12-20 19:05:39.682 JupyterHub web:2175] *** LAGUNOV DBG: GET /hub/user/lagunov/api/contents/ is handled by $<class 'jupyterhub.handlers.base.UserUrlHandler'>
    [I 2021-12-20 19:05:39.683 JupyterHub base:422] LAGUNOV DBG get_current_user self._accept_token_auth=False self._accept_cookie_auth=True
    [I 2021-12-20 19:05:39.683 JupyterHub web:812] LAGUNOV DBG redirect 302 /hub/login?next=%2Fhub%2Fuser%2Flagunov%2Fapi%2Fcontents%2F
    [I 2021-12-20 19:05:39.684 JupyterHub log:189] 302 GET /hub/user/lagunov/api/contents/ -> /hub/login?next=%2Fhub%2Fuser%2Flagunov%2Fapi%2Fcontents%2F (@127.0.0.1) 1.70ms
    ```
    </details>



Discussion:
Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! :hugs:
<br>If you haven't done so already, check out [Jupyter's Code of Conduct](https://github.com/jupyter/governance/blob/master/conduct/code_of_conduct.md). Also, please try to follow the issue template as it helps other other community members to contribute more effectively.
![welcome](https://raw.githubusercontent.com/jupyterhub/.github/master/images/welcome.jpg)
You can meet the other [Jovyans](https://jupyter.readthedocs.io/en/latest/community/content-community.html?highlight=jovyan#what-is-a-jovyan) by joining our [Discourse forum](http://discourse.jupyter.org/). There is also an intro thread there where you can stop by and say Hi! :wave:
<br>Welcome to the Jupyter community! :tada:

A 302 on `/user/lagunov/api/contents/ -> /hub/user/lagunov/api/contents/` means that `lagunov` does not have a running server. You cannot talk to a server that is not running. How do you currently make sure that the server is running before you try to talk to it?

I am surprised this is getting you redirected to `/hub/login`, when it normally would fail with an informative error that the server must be started (it would be 503 in 1.4).

Resolving patch:
diff --git a/jupyterhub/handlers/base.py b/jupyterhub/handlers/base.py
--- a/jupyterhub/handlers/base.py
+++ b/jupyterhub/handlers/base.py
@@ -1381,6 +1381,9 @@ class UserUrlHandler(BaseHandler):
     Note that this only occurs if bob's server is not already running.
     """
 
+    # accept token auth for API requests that are probably to non-running servers
+    _accept_token_auth = True
+
     def _fail_api_request(self, user_name='', server_name=''):
         """Fail an API request to a not-running server"""
         self.log.warning(


Test patch: