Repository: apluslms/a-plus

Issue:
Crash in adding deviations due to unique constraint violation
Sometimes, when a teacher is adding deadline deviations, A+ crashes.

Based on the traceback below, this occurs when new deviations are supposed to override existing ones. It looks like the code mixes up creating new and modifying existing records in some cases.

Stripped traceback that highlights the deviation override view and the unique constraint violation when the new deviation is saved:

```
[ERROR/log] Internal Server Error: /<course>/<instance>/teachers/deadline-deviations/override/
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "exercise_deadlineruledeviatio_exercise_id_xyz_uniq"
DETAIL:  Key (exercise_id, submitter_id)=(123, 456) already exists.

The above exception was the direct cause of the following exception:

  File "/a-plus/deviations/viewbase.py", line 141, in form_valid
    new_deviation.save()
django.db.utils.IntegrityError: duplicate key value violates unique constraint "exercise_deadlineruledeviatio_exercise_id_xyz_uniq"
DETAIL:  Key (exercise_id, submitter_id)=(123, 456) already exists.
```

The code clearly tries to separate existing and new deviations:
https://github.com/apluslms/a-plus/blob/8245ca68e99a848ae61180f5b39d8a3a4925c76c/deviations/viewbase.py#L128-L141


Discussion:
@PasiSa FYI this should be fixed soon.
Noted. Wasn't able to reproduce this locally yet, though. Apparently something happens between setting self.existing_deviations in get_common_objects() and using it in form_valid(), but couldn't yet figure it out.
Replacing the quoted code with a solution based on get_or_create() might be good enough?

EDIT: ok, it is not quite that straight forward, due to different kinds of deviations...
I get a 500 for every deviation I make. The deviation however gets created and the crash happens after that. This suggests that there might be an attempt to create it twice.
@oseppala can you give more specifics: which course, which deviation? If I could repeat it, that would help tracking down what and why exactly happens there.
For me I get a 500 every time. Moreover, if I use multiple students the deviation is added only for one of them (assuming that the crash interrupts) and if I try to do it for a whole module only the first exercise in the module gets the extension. At least only the first exercise is listed afterwards. 
This means that if I want to add deviations for 3 students in a module with 4 exercises I have to do everything 12 times. At least that is how it looks for me at the moment.
Yes, we tested this with Otto last evening and I noticed the same. It seems that including whole module is the triggering factor here. I didn't test the multiple students case, but good to know. Thanks!
Some additional observations:

- This can be replicated in minus, but when testing similar case locally using aplus-manual course, everything works fine, and I am not able to make it crash.
- The crash also happens without override, on viewbase.py line 68, which has similar code, but without existing overlapping deviations
It turns out that [get_submitters()](https://github.com/apluslms/a-plus/blob/8245ca68e99a848ae61180f5b39d8a3a4925c76c/deviations/viewbase.py#L46) can return multiple instances of the same user when the DB has user tags, and therefore the same deviation is tried to be added multiple times in the following for loop. In my local docker environment there were no user tags, so I did not initially notice this. If the submitter_tag field in the form is not empty, things seem to work. So the get_submitters() filter needs some fixing.

Resolving patch:
diff --git a/deviations/viewbase.py b/deviations/viewbase.py
--- a/deviations/viewbase.py
+++ b/deviations/viewbase.py
@@ -296,4 +296,4 @@ def get_submitters(form_data: Dict[str, Any]) -> models.QuerySet[UserProfile]:
     return UserProfile.objects.filter(
         models.Q(id__in=form_data.get('submitter', []))
         | models.Q(taggings__tag__in=form_data.get('submitter_tag', []))
-    )
+    ).distinct()


Test patch: