Repository: jupyterhub/jupyterhub

Issue:
Database error when hub tries to add existing group
We enabled groups on a hub, and they seemed to be working fine for a little while, but users started to see 500 errors on logging in. One traceback indicated that the hub attempted to add an already existing group to the ORM database. The groups are based on course enrollments which we obtain from our Canvas LMS as users login. Unfortunately I don't have other tracebacks.

Here is the traceback from one such instance:
https://github.com/berkeley-dsep-infra/datahub/issues/3556#issuecomment-1212830090

The deployment is https://github.com/berkeley-dsep-infra/datahub and our authenticator is at https://github.com/berkeley-dsep-infra/datahub/tree/staging/images/hub/canvasauthenticator. Here is a condensed version of our `authenticate` method:
```
    async def authenticate(self, handler, data=None):
        user = await super().authenticate(handler, data)
        courses = await self.get_courses(user['auth_state']['access_token'])
        user['groups'] = self.extract_course_groups(courses)
        return user
```

We've temporarily disabled the assignment to the `groups` key and that stopped the 500 errors.

We [had set](https://github.com/berkeley-dsep-infra/datahub/blob/5096d43940768a3f33dca48b9c87f2c5d7fcbf91/hub/values.yaml#L194) `Authenticator.managed_groups = True` but that is also temporarily disabled.

Our hub is `jupyterhub/k8s-hub:1.1.3-n554.hfa81c67f` and is using a customized KubeSpawner.

Under /hub/admin we can see that 35 of the 37 groups were created during this time have only one user, while two groups have two. Our students tend to cluster in the same classes so I would normally expect higher group memberships. Some people were using the hub but hadn't yet been forced to reauthenticate due to the age of their login. I don't know how two groups managed to get two users added, given the behavior.

The error from the traceback:
```
    sqlalchemy.exc.IntegrityError: (sqlite3.IntegrityError) UNIQUE constraint failed: groups.name
    [SQL: INSERT INTO groups (name) VALUES (?)]
    [parameters: ('canvas::1497305::student',)]
```
occurred with the group `canvas::1497305::student`, however I see no such existing group in the hub.



Discussion:
Is there any chance `user["groups"]` could have the same group name occurring twice? Looking at the code for sync_groups, I think it would try to create a group that doesn't exist once for each occurrence in `user["groups"]`
The alternative is that the `orm.Group.name.in_` query isn't actually working. That would be harder to track down and deal with.

Resolving patch:
diff --git a/jupyterhub/user.py b/jupyterhub/user.py
--- a/jupyterhub/user.py
+++ b/jupyterhub/user.py
@@ -310,19 +310,19 @@ def sync_groups(self, group_names):
             return
 
         # log group changes
-        new_groups = set(group_names).difference(current_groups)
+        added_groups = new_groups.difference(current_groups)
         removed_groups = current_groups.difference(group_names)
-        if new_groups:
-            self.log.info(f"Adding user {self.name} to group(s): {new_groups}")
+        if added_groups:
+            self.log.info(f"Adding user {self.name} to group(s): {added_groups}")
         if removed_groups:
             self.log.info(f"Removing user {self.name} from group(s): {removed_groups}")
 
         if group_names:
             groups = (
-                self.db.query(orm.Group).filter(orm.Group.name.in_(group_names)).all()
+                self.db.query(orm.Group).filter(orm.Group.name.in_(new_groups)).all()
             )
             existing_groups = {g.name for g in groups}
-            for group_name in group_names:
+            for group_name in added_groups:
                 if group_name not in existing_groups:
                     # create groups that don't exist yet
                     self.log.info(
@@ -331,9 +331,9 @@ def sync_groups(self, group_names):
                     group = orm.Group(name=group_name)
                     self.db.add(group)
                     groups.append(group)
-            self.groups = groups
+            self.orm_user.groups = groups
         else:
-            self.groups = []
+            self.orm_user.groups = []
         self.db.commit()
 
     async def save_auth_state(self, auth_state):


Test patch:
diff --git a/jupyterhub/tests/test_user.py b/jupyterhub/tests/test_user.py
--- a/jupyterhub/tests/test_user.py
+++ b/jupyterhub/tests/test_user.py
@@ -29,12 +29,12 @@ async def test_userdict_get(db, attr):
         ["isin1", "isin2"],
         ["isin1"],
         ["notin", "isin1"],
-        ["new-group", "isin1"],
+        ["new-group", "new-group", "isin1"],
         [],
     ],
 )
 def test_sync_groups(app, user, group_names):
-    expected = sorted(group_names)
+    expected = sorted(set(group_names))
     db = app.db
     db.add(orm.Group(name="notin"))
     in_groups = [orm.Group(name="isin1"), orm.Group(name="isin2")]