Repository: cal-itp/benefits

Issue:
Django 4.0.x breaks enrollment overlay
It seems that our recent update from Django `3.2.10` to `4.0` (also `4.0.1`) broke the payment processor card verification / enrollment overlay. This update has not been deployed to `prod` yet, and this breakage is the reason we've been holding that release.

## Background

After a user has verified eligibility, the following steps happen:

1. The user is redirected to the enrollment page
1. The enrollment page uses [AJAX to request the card tokenization script](https://github.com/cal-itp/benefits/blob/dev/benefits/enrollment/templates/enrollment/index.html#L12) from the payment processor 
1. Once the card tokenization script is loaded:
    * an [AJAX request](https://github.com/cal-itp/benefits/blob/dev/benefits/enrollment/templates/enrollment/index.html#L14)
    * to the [Django backend](https://github.com/cal-itp/benefits/blob/dev/benefits/enrollment/views.py#L96)
    * [acquires an access token](https://github.com/cal-itp/benefits/blob/dev/benefits/enrollment/api.py#L203) from the payment processor
1. The access token is [passed as options to a JavaScript function](https://github.com/cal-itp/benefits/blob/dev/benefits/enrollment/templates/enrollment/index.html#L27) in the template, which is called immediately
    * the function name comes from the app config
    * the function implementation is from the payment processor's card tokenization script; it sets up a watcher for a click on the enrollment button
1. The user clicks the enrollment button, the card verification overlay opens
1. The payment processor's script makes an AJAX request to acquire a public key from the payment processor's server
1. The overlay's interface loads completely, showing a collection form
1. The user fills out their payment card details in the collection form, which submits directly to the payment processor
1. After successful submission to the payment processor, the card tokenization script [triggers a callback with the tokenized card](https://github.com/cal-itp/benefits/blob/dev/benefits/enrollment/templates/enrollment/index.html#L34), which is then sent via [HTML form submission](https://github.com/cal-itp/benefits/blob/dev/benefits/enrollment/templates/enrollment/index.html#L42) to the Django backend.
1. The Django backend uses the tokenized card to [enroll the user into a discount group](https://github.com/cal-itp/benefits/blob/dev/benefits/enrollment/api.py#L224)

## The issue

Somewhere in **Steps 3 - 7** is where this seems to break down. The payment processor's request for the public key from their own server (of **Step 5** above) works in `3.2.10` and fails in `4.0.x`.

Here are the headers for this request under `3.2.10` (overlay works):

```
:authority: web.paymentprocessor.com
:method: GET
:path: /api/xxxx/public-keys
:scheme: https
accept: */*
accept-encoding: gzip, deflate, br
accept-language: en-US,en;q=0.9
authorization: Bearer eeeeeeeeeeeeeeeeeeeeeeeeeeee.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
dnt: 1
origin: https://verify.paymentprocessor.com
sec-fetch-dest: empty
sec-fetch-mode: cors
sec-fetch-site: same-site
```

And headers for the same request under `4.0.x` (overlay does not work):

```
:authority: web.paymentprocessor.com
:method: GET
:path: /api/xxxx/public-keys
:scheme: https
accept: */*
accept-encoding: gzip, deflate, br
accept-language: en-US,en;q=0.9
authorization: Bearer undefined
dnt: 1
origin: https://verify.paymentprocessor.com
sec-fetch-dest: empty
sec-fetch-mode: cors
sec-fetch-site: same-site
```

So obviously the `Bearer` token is missing in the `authorization` header in the latter example. But why? This is the same access token acquired in **Step 3** above, and that works under both `3.2.10` and `4.0.x`.

## Guesses?

### `CSRF_TRUSTED_ORIGINS` 

That new setting that we had to fix in #268 was due to a change in `4.0` related to `Origin` header checking for `POST` and other "dangerous" (read: side-effect producing) requests. Except this request isn't coming _to our server_, or _from our server_, but _from the user's browser_, so this seems less relevant. I also tried adding various combinations of the payment processor's URLs into that setting to no avail. I'm pretty sure this isn't the problem.

### `JsonResponse`

Did this change at all in the new Django version? This is how the access token is sent back to the user's browser from Django in **Step 3** above.

## Reproduce

1. Open repo locally in VS Code (not devcontainer)
2. Edit `requirements.txt` to change the Django version: `3.2.10` (works) vs. `4.0` and/or `4.0.1` (does not work)
3. Make sure you have `fixtures/` for the payment processor (and pem data), and your `.env` `DJANGO_INIT_PATH` points to them
4. Inside `.devcontainer/`, run `docker compose build client`
5. Once built, run `docker compose up -d client server`
6. Get the localhost port and browse to the localhost site
7. Run through eligibility with the test EV server credentials
8. On the enrollment page, click the button to open the overlay. Notice if it does or does not load the form and/or show an error message.
9. Press `F12` to view Developer Tools for the overlay, reload the overlay with `F5` _while selected on the Developer Tools window_
10. Look on the Network tab for the call to `public-keys`. Notice it does or does not succeed with headers like shown above.
11. You can also `Rebuild and Reopen` in the devcontainer (after following these steps 1-4) and set a breakpoint to debug
    * the access token calls - from user to Django to payment processor to Django to user - succeed for all the versions


Discussion:
As a workaround to get all the fixes and language updates out the door this week, let's rollback to Django `3.2.10` temporarily while we figure this out.
@afeld see the fixtures here for local configuration for connecting to the payment processor: https://github.com/cal-itp/benefits-secrets/tree/main/benefits/localhost/fixtures

Resolving patch:
diff --git a/benefits/settings.py b/benefits/settings.py
--- a/benefits/settings.py
+++ b/benefits/settings.py
@@ -114,6 +114,10 @@ def _filter_empty(ls):
 
 SECURE_BROWSER_XSS_FILTER = True
 
+# required so that cross-origin pop-ups (like the enrollment overlay) have access to parent window context
+# https://github.com/cal-itp/benefits/pull/793
+SECURE_CROSS_ORIGIN_OPENER_POLICY = "same-origin-allow-popups"
+
 # the NGINX reverse proxy sits in front of the application in deployed environments
 # SSL terminates before getting to Django, and NGINX adds this header to indicate
 # if the original request was secure or not


Test patch: