Repository: jupyterhub/jupyterhub

Issue:
Call to .add on a frozenset in _intersect_expanded_scopes fails
### Bug description

Unexpected frozenset in `jupyterhub.scopes._intersect_expanded_scopes` means that any call (e.g. [here](https://github.com/jupyterhub/jupyterhub/blame/4.0.2/jupyterhub/scopes.py#L269)) to `.add(...)` fails with `AttributeError: 'frozenset' object has no attribute 'add'`.

### How to reproduce

Not sure if there's a specific behavior or configuration that led to this, or of it's just not commonly hit due to a default codepath that results in a mutable `set()`.

With just a brief review of the code I'm not sure how this `.add()` could ever work unless the `.get(..., set())` was usually just returning the default mutable set and not the frozenset when the key is actually found.

The variable `common_servers` which in this case had `.add(...)` called on it triggering the AttributeError derives from a variable originally returned from `parse_scopes`, which returns a `_memoize.FrozenDict`. That class serializes any regular sets into frozensets: https://github.com/jupyterhub/jupyterhub/blob/4.0.2/jupyterhub/_memoize.py#L137-L138

#### Expected behaviour

Scope checking would succeed and the original API request (in this case to `/activity`) would succeed 200 OK.

#### Actual behaviour

500 error due to `jupyterhub.scopes._intersect_expanded_scopes`

### Your personal set up

Ubuntu 20.04
Python 3.8
JupyterHub 4.0.2
TLJH 1.0.0 with DockerSpawner

<details><summary>Full environment</summary>

```
aiohttp==3.8.5
aiosignal==1.3.1
alembic==1.12.0
async-generator==1.10
async-timeout==4.0.3
attrs==23.1.0
backoff==2.2.1
bcrypt==4.0.1
certifi==2023.7.22
certipy==0.1.3
cffi==1.15.1
charset-normalizer==3.2.0
cryptography==41.0.3
docker==6.1.3
dockerspawner==12.1.0
escapism==1.0.1
frozenlist==1.4.0
greenlet==2.0.2
idna==3.4
importlib-metadata==6.8.0
importlib-resources==6.0.1
Jinja2==3.1.2
jsonschema==4.19.0
jsonschema-specifications==2023.7.1
jupyter-telemetry==0.1.0
jupyterhub==4.0.2
jupyterhub-firstuseauthenticator==1.0.0
jupyterhub-idle-culler==1.2.1
jupyterhub-ldapauthenticator==1.3.2
jupyterhub-nativeauthenticator==1.2.0
jupyterhub-systemdspawner==1.0.1
jupyterhub-tmpauthenticator==1.0.0
jupyterhub-traefik-proxy==1.1.0
ldap3==2.9.1
Mako==1.2.4
MarkupSafe==2.1.3
multidict==6.0.4
oauthenticator==16.0.7
oauthlib==3.2.2
onetimepass==1.0.1
packaging==23.1
pamela==1.1.0
passlib==1.7.4
pkg_resources==0.0.0
pkgutil_resolve_name==1.3.10
pluggy==1.3.0
prometheus-client==0.17.1
psycopg2-binary==2.9.7
pyasn1==0.5.0
pycparser==2.21
pycurl==7.45.2
pyOpenSSL==23.2.0
python-dateutil==2.8.2
python-json-logger==2.0.7
referencing==0.30.2
requests==2.31.0
rpds-py==0.10.3
ruamel.yaml==0.17.32
ruamel.yaml.clib==0.2.7
six==1.16.0
SQLAlchemy==2.0.20
the-littlest-jupyterhub @ git+https://github.com/jupyterhub/the-littlest-jupyterhub.git@fc8e19b1b5663f58f0e7b089903d1d1769db06b8
toml==0.10.2
tornado==6.3.3
traitlets==5.10.0
typing_extensions==4.7.1
urllib3==2.0.4
websocket-client==1.6.3
yarl==1.9.2
zipp==3.16.2
```
</details>

<details><summary>Configuration</summary>
Using `DockerSpawner`, but skipping `jupyterhub_config.py` configuration since this happens pretty deep within the `jupyterhub` lib and doesn't seem too related to the configuration. Please let me know if this needs to be added.
</details>

<details><summary>Logs</summary>

```
[E 2023-09-14 14:25:08.361 JupyterHub web:1871] Uncaught exception POST /hub/api/users/u89d3d00/activity (172.17.0.2)
    HTTPServerRequest(protocol='http', host='10.1.45.4:15001', method='POST', uri='/hub/api/users/u89d3d00/activity', version='HTTP/1.1', remote_ip='172.17.0.2')
    Traceback (most recent call last):
      File "/opt/tljh/hub/lib/python3.8/site-packages/tornado/web.py", line 1765, in _execute
        result = await result  # type: ignore
      File "/opt/tljh/hub/lib/python3.8/site-packages/jupyterhub/apihandlers/base.py", line 79, in prepare
        await super().prepare()
      File "/opt/tljh/hub/lib/python3.8/site-packages/jupyterhub/handlers/base.py", line 100, in prepare
        self._resolve_roles_and_scopes()
      File "/opt/tljh/hub/lib/python3.8/site-packages/jupyterhub/handlers/base.py", line 465, in _resolve_roles_and_scopes
        self.expanded_scopes = scopes.get_scopes_for(orm_token)
      File "/opt/tljh/hub/lib/python3.8/site-packages/jupyterhub/scopes.py", line 366, in get_scopes_for
        intersection = _intersect_expanded_scopes(
      File "/opt/tljh/hub/lib/python3.8/site-packages/jupyterhub/_memoize.py", line 96, in cached
        result = func(*args, **kwargs)
      File "/opt/tljh/hub/lib/python3.8/site-packages/jupyterhub/scopes.py", line 269, in _intersect_expanded_scopes
        common_servers.add(server)
    AttributeError: 'frozenset' object has no attribute 'add'
    
[E 2023-09-14 14:25:08.363 JupyterHub log:183] {
      "Authorization": "token [secret]",
      "Content-Type": "application/json",
      "Connection": "close",
      "Host": "10.1.45.4:15001",
      "User-Agent": "Tornado/6.3.2",
      "Content-Length": "115",
      "Accept-Encoding": "gzip"
    }
[E 2023-09-14 14:25:08.363 JupyterHub log:191] 500 POST /hub/api/users/u89d3d00/activity (u89d3d00@172.17.0.2) 99.21ms
```
</details>



Discussion:
Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! :hugs:
<br>If you haven't done so already, check out [Jupyter's Code of Conduct](https://github.com/jupyter/governance/blob/master/conduct/code_of_conduct.md). Also, please try to follow the issue template as it helps other other community members to contribute more effectively.
![welcome](https://raw.githubusercontent.com/jupyterhub/.github/master/images/welcome.jpg)
You can meet the other [Jovyans](https://jupyter.readthedocs.io/en/latest/community/content-community.html?highlight=jovyan#what-is-a-jovyan) by joining our [Discourse forum](http://discourse.jupyter.org/). There is also an intro thread there where you can stop by and say Hi! :wave:
<br>Welcome to the Jupyter community! :tada:

This patch appears to fix the issue:

```diff
diff --git a/jupyterhub/scopes.py b/jupyterhub/scopes.py
index 00170133..5a1ec499 100644
--- a/jupyterhub/scopes.py
+++ b/jupyterhub/scopes.py
@@ -253,8 +253,8 @@ def _intersect_expanded_scopes(scopes_a, scopes_b, db=None):
             }
 
             # resolve hierarchies (group/user/server) in both directions
-            common_servers = common_filters[base].get("server", set())
-            common_users = common_filters[base].get("user", set())
+            common_servers = set(common_filters[base].get("server", set()))
+            common_users = set(common_filters[base].get("user", set()))
 
             for a, b in [(filters_a, filters_b), (filters_b, filters_a)]:
                 if 'server' in a and b.get('server') != a['server']:
```

Resolving patch:
diff --git a/jupyterhub/scopes.py b/jupyterhub/scopes.py
--- a/jupyterhub/scopes.py
+++ b/jupyterhub/scopes.py
@@ -257,8 +257,12 @@ def groups_for_server(server):
             }
 
             # resolve hierarchies (group/user/server) in both directions
-            common_servers = common_filters[base].get("server", set())
-            common_users = common_filters[base].get("user", set())
+            common_servers = initial_common_servers = common_filters[base].get(
+                "server", frozenset()
+            )
+            common_users = initial_common_users = common_filters[base].get(
+                "user", frozenset()
+            )
 
             for a, b in [(filters_a, filters_b), (filters_b, filters_a)]:
                 if 'server' in a and b.get('server') != a['server']:
@@ -270,7 +274,7 @@ def groups_for_server(server):
                         for server in servers:
                             username, _, servername = server.partition("/")
                             if username in b['user']:
-                                common_servers.add(server)
+                                common_servers = common_servers | {server}
 
                     # resolve group/server hierarchy if db available
                     servers = servers.difference(common_servers)
@@ -279,7 +283,7 @@ def groups_for_server(server):
                         for server in servers:
                             server_groups = groups_for_server(server)
                             if server_groups & b['group']:
-                                common_servers.add(server)
+                                common_servers = common_servers | {server}
 
                 # resolve group/user hierarchy if db available and user sets aren't identical
                 if (
@@ -293,14 +297,16 @@ def groups_for_server(server):
                     for username in users:
                         groups = groups_for_user(username)
                         if groups & b["group"]:
-                            common_users.add(username)
+                            common_users = common_users | {username}
 
-            # add server filter if there wasn't one before
-            if common_servers and "server" not in common_filters[base]:
+            # add server filter if it's non-empty
+            # and it changed
+            if common_servers and common_servers != initial_common_servers:
                 common_filters[base]["server"] = common_servers
 
-            # add user filter if it's non-empty and there wasn't one before
-            if common_users and "user" not in common_filters[base]:
+            # add user filter if it's non-empty
+            # and it changed
+            if common_users and common_users != initial_common_users:
                 common_filters[base]["user"] = common_users
 
     intersection = unparse_scopes(common_filters)


Test patch:
diff --git a/jupyterhub/tests/test_scopes.py b/jupyterhub/tests/test_scopes.py
--- a/jupyterhub/tests/test_scopes.py
+++ b/jupyterhub/tests/test_scopes.py
@@ -912,6 +912,22 @@ def test_intersect_expanded_scopes(left, right, expected, should_warn, recwarn):
             ["read:users!user=uy"],
             {"gx": ["ux"], "gy": ["uy"]},
         ),
+        (
+            # make sure the group > user > server hierarchy
+            # is managed
+            ["read:servers!server=ux/server", "read:servers!group=gy"],
+            ["read:servers!server=uy/server", "read:servers!user=ux"],
+            ["read:servers!server=ux/server", "read:servers!server=uy/server"],
+            {"gx": ["ux"], "gy": ["uy"]},
+        ),
+        (
+            # make sure the group > user hierarchy
+            # is managed
+            ["read:servers!user=ux", "read:servers!group=gy"],
+            ["read:servers!user=uy", "read:servers!group=gx"],
+            ["read:servers!user=ux", "read:servers!user=uy"],
+            {"gx": ["ux"], "gy": ["uy"]},
+        ),
     ],
 )
 def test_intersect_groups(request, db, left, right, expected, groups):