Repository: jupyterhub/jupyterhub

Issue:
Jupyter skips spawn form after authentication
<!-- Thank you for contributing. These HTML commments will not render in the issue, but you can delete them once you've read them if you prefer! -->

### Bug description
We have just noticed a change in behaviour, probably coming from https://github.com/jupyterhub/jupyterhub/commit/6f2e409fb9bb4959310f26f2316bc4d4d228acc0

JupyterHub skips spawner now directly after authentication as it seems some params are passed along from auth and callback:

```
302 GET /hub/oauth_callback?code=[secret]&state=[secret] -> /hub/spawn?code=[secret]&state=[secret] (@::ffff:10.128.2.1) 261.56ms
302 GET /hub/spawn?code=[secret]&state=[secret] -> /hub/spawn-pending/vpavlin?code=[secret]&state=[secret] 
```

The `code` seems to trigger the code in the commit linked above.

This is our oauth config: https://github.com/opendatahub-io/jupyterhub-odh/blob/master/.jupyter/jupyterhub_config.py#L86-L143

Is it possible that it is a misconfiguration on our side? Could it be an issue with authenticator?

#### Expected behaviour
Spawner is shown

#### Actual behaviour
Spawner is skipped

### How to reproduce
I don't have a simple reproducer at the moment, but I can try to get one if this turns out to me more complicated

### Your personal set up

 - OpenShift 4.5
- Kubepawner, oauthenticator/openshift

Exclude "code" and "state" query params from next_url
This fixes our issue with JH skipping spawner UI after authentization although no query parameters were provided - `code` and `state` stay in query after auth and cause Spawner to kick off. All the details are in the issue #3247.

Fixes #3247 

cc @kinow - since the `append_query_parameters` is your contribution, can you take a look?




Discussion:
Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! :hugs:
<br>If you haven't done so already, check out [Jupyter's Code of Conduct](https://github.com/jupyter/governance/blob/master/conduct/code_of_conduct.md). Also, please try to follow the issue template as it helps other other community members to contribute more effectively.
![welcome](https://raw.githubusercontent.com/jupyterhub/.github/master/images/welcome.jpg)
You can meet the other [Jovyans](https://jupyter.readthedocs.io/en/latest/community/content-community.html?highlight=jovyan#what-is-a-jovyan) by joining our [Discourse forum](http://discourse.jupyter.org/). There is also an intro thread there where you can stop by and say Hi! :wave:
<br>Welcome to the Jupyter community! :tada:

cc @mroman-redhat
I believe the actual issue might be that the auth flow does not strip the `code` and `state` which are not needed in URL after the auth flow is done - https://github.com/jupyterhub/jupyterhub/blob/master/jupyterhub/services/auth.py#L987
Ok, more digging...seems like the actual culprit of the issue might be https://github.com/jupyterhub/jupyterhub/commit/14378f4cc2da8ced635841547d396d07027f56d7 - this adds the `code` and `state` query parameters which causes the spawner to spawn instead of going to UI
I was able to verify that the `code` and `state` are indeed the problem - when the `pages.py` code ignores them, everything is https://github.com/vpavlin/jupyterhub/blob/bug/remove-auth-args/jupyterhub/handlers/base.py#L662

Is this the right solution? I am not sure if it can potentially break some auth in case the get_next_url is used for something else then the last step of auth
Thanks for submitting your first pull request! You are awesome! :hugs:
<br>If you haven't done so already, check out [Jupyter's Code of Conduct](https://github.com/jupyter/governance/blob/master/conduct/code_of_conduct.md). Also, please make sure you followed the pull request template, as this will help us review your contribution more quickly.
![welcome](https://raw.githubusercontent.com/jupyterhub/.github/master/images/welcome.jpg)
You can meet the other [Jovyans](https://jupyter.readthedocs.io/en/latest/community/content-community.html?highlight=jovyan#what-is-a-jovyan) by joining our [Discourse forum](http://discourse.jupyter.org/). There is also a intro thread there where you can stop by and say Hi! :wave:
<br>Welcome to the Jupyter community! :tada:

HI @kinow thank you for your thoughts!

This is not a problem of spawner per se. This seems to be the combination of OAuth and the changes in spawner (appending query params and picking them up and doing "auto-spawn").

OAuth actually uses the [BaseHandler.get_next_url](https://github.com/jupyterhub/oauthenticator/blob/master/oauthenticator/oauth2.py#L190), so there are potentially 2 places where to implement my change - in the BaseHandler or in the OAuth.

My preference is BaseHandler since if we only remove `code` and `state` for OAuth, we could get inconsistent behaviour.

Also probably worth mentioning - `code` (a temporary code for client to get access token) and `state` (a unique/random id to identify a request is coiming from a user who initiated the authentication) are used in OAuth flow, so they should not be relied upon after the auth is done.

I would also consider reviewing the whole approach with the query params, since it seems to me that it does not actually work with OAuth. What happened when I tried:

1. Access `/hub?xyz=test`
2. Redirect to `/hub/login` (the query param is lost)
3. Perform login
4. Redirect to `/hub/spawn?code=...&state=...` (no `xyz` param and spawner wants to start based on `code` and `state`)

So my preference woudl be to merge this PR to fix the OAuth flow and then we can dig deeper into how we can preserve the query params for OAuth.

WDYT?

Resolving patch:
diff --git a/jupyterhub/handlers/base.py b/jupyterhub/handlers/base.py
--- a/jupyterhub/handlers/base.py
+++ b/jupyterhub/handlers/base.py
@@ -634,6 +634,12 @@ def get_next_url(self, user=None, default=None):
                 next_url,
             )
 
+        # this is where we know if next_url is coming from ?next= param or we are using a default url
+        if next_url:
+            next_url_from_param = True
+        else:
+            next_url_from_param = False
+
         if not next_url:
             # custom default URL, usually passed because user landed on that page but was not logged in
             if default:
@@ -659,7 +665,10 @@ def get_next_url(self, user=None, default=None):
             else:
                 next_url = url_path_join(self.hub.base_url, 'home')
 
-        next_url = self.append_query_parameters(next_url, exclude=['next'])
+        if not next_url_from_param:
+            # when a request made with ?next=... assume all the params have already been encoded
+            # otherwise, preserve params from the current request across the redirect
+            next_url = self.append_query_parameters(next_url, exclude=['next'])
         return next_url
 
     def append_query_parameters(self, url, exclude=None):
diff --git a/jupyterhub/handlers/pages.py b/jupyterhub/handlers/pages.py
--- a/jupyterhub/handlers/pages.py
+++ b/jupyterhub/handlers/pages.py
@@ -44,7 +44,7 @@ def get(self):
         elif user:
             url = self.get_next_url(user)
         else:
-            url = self.settings['login_url']
+            url = url_concat(self.settings["login_url"], dict(next=self.request.uri))
         self.redirect(url)
 
 


Test patch:
diff --git a/jupyterhub/tests/test_pages.py b/jupyterhub/tests/test_pages.py
--- a/jupyterhub/tests/test_pages.py
+++ b/jupyterhub/tests/test_pages.py
@@ -31,7 +31,7 @@ async def test_root_no_auth(app):
     url = ujoin(public_host(app), app.hub.base_url)
     r = await async_requests.get(url)
     r.raise_for_status()
-    assert r.url == ujoin(url, 'login')
+    assert r.url == url_concat(ujoin(url, 'login'), dict(next=app.hub.base_url))
 
 
 async def test_root_auth(app):
@@ -622,9 +622,16 @@ def mock_authenticate(handler, data):
         (False, '//other.domain', '', None),
         (False, '///other.domain/triple', '', None),
         (False, '\\\\other.domain/backslashes', '', None),
-        # params are handled correctly
-        (True, '/hub/admin', 'hub/admin?left=1&right=2', [('left', 1), ('right', 2)]),
-        (False, '/hub/admin', 'hub/admin?left=1&right=2', [('left', 1), ('right', 2)]),
+        # params are handled correctly (ignored if ?next= specified)
+        (
+            True,
+            '/hub/admin?left=1&right=2',
+            'hub/admin?left=1&right=2',
+            {"left": "abc"},
+        ),
+        (False, '/hub/admin', 'hub/admin', [('left', 1), ('right', 2)]),
+        (True, '', '', {"keep": "yes"}),
+        (False, '', '', {"keep": "yes"}),
     ],
 )
 async def test_login_redirect(app, running, next_url, location, params):
@@ -633,10 +640,15 @@ async def test_login_redirect(app, running, next_url, location, params):
     if location:
         location = ujoin(app.base_url, location)
     elif running:
+        # location not specified,
         location = user.url
+        if params:
+            location = url_concat(location, params)
     else:
         # use default url
         location = ujoin(app.base_url, 'hub/spawn')
+        if params:
+            location = url_concat(location, params)
 
     url = 'login'
     if params:
@@ -655,7 +667,73 @@ async def test_login_redirect(app, running, next_url, location, params):
     r = await get_page(url, app, cookies=cookies, allow_redirects=False)
     r.raise_for_status()
     assert r.status_code == 302
-    assert location == r.headers['Location']
+    assert r.headers["Location"] == location
+
+
+@pytest.mark.parametrize(
+    'location, next, extra_params',
+    [
+        (
+            "{base_url}hub/spawn?a=5",
+            None,
+            {"a": "5"},
+        ),  # no ?next= given, preserve params
+        ("/x", "/x", {"a": "5"}),  # ?next=given, params ignored
+        (
+            "/x?b=10",
+            "/x?b=10",
+            {"a": "5"},
+        ),  # ?next=given with params, additional params ignored
+    ],
+)
+async def test_next_url(app, user, location, next, extra_params):
+    params = {}
+    if extra_params:
+        params.update(extra_params)
+    if next:
+        params["next"] = next
+    url = url_concat("/", params)
+    cookies = await app.login_user("monster")
+
+    # location can be a string template
+    location = location.format(base_url=app.base_url)
+
+    r = await get_page(url, app, cookies=cookies, allow_redirects=False)
+    r.raise_for_status()
+    assert r.status_code == 302
+    assert r.headers["Location"] == location
+
+
+async def test_next_url_params_sequence(app, user):
+    """Test each step of / -> login -> spawn
+
+    and whether they preserve url params
+    """
+    params = {"xyz": "5"}
+    # first request: root page, with params, not logged in
+    r = await get_page("/?xyz=5", app, allow_redirects=False)
+    r.raise_for_status()
+    location = r.headers["Location"]
+
+    # next page: login
+    cookies = await app.login_user(user.name)
+    assert location == url_concat(
+        ujoin(app.base_url, "/hub/login"), {"next": ujoin(app.base_url, "/hub/?xyz=5")}
+    )
+    r = await async_requests.get(
+        public_host(app) + location, cookies=cookies, allow_redirects=False
+    )
+    r.raise_for_status()
+    location = r.headers["Location"]
+
+    # after login, redirect back
+    assert location == ujoin(app.base_url, "/hub/?xyz=5")
+    r = await async_requests.get(
+        public_host(app) + location, cookies=cookies, allow_redirects=False
+    )
+    r.raise_for_status()
+    location = r.headers["Location"]
+    assert location == ujoin(app.base_url, "/hub/spawn?xyz=5")
 
 
 async def test_auto_login(app, request):
@@ -669,14 +747,18 @@ def get(self):
     )
     # no auto_login: end up at /hub/login
     r = await async_requests.get(base_url)
-    assert r.url == public_url(app, path='hub/login')
+    assert r.url == url_concat(
+        public_url(app, path="hub/login"), {"next": app.hub.base_url}
+    )
     # enable auto_login: redirect from /hub/login to /hub/dummy
     authenticator = Authenticator(auto_login=True)
     authenticator.login_url = lambda base_url: ujoin(base_url, 'dummy')
 
     with mock.patch.dict(app.tornado_settings, {'authenticator': authenticator}):
         r = await async_requests.get(base_url)
-    assert r.url == public_url(app, path='hub/dummy')
+    assert r.url == url_concat(
+        public_url(app, path="hub/dummy"), {"next": app.hub.base_url}
+    )
 
 
 async def test_auto_login_logout(app):