Repository: paperless-ngx/paperless-ngx

Issue:
[BUG] Docker 1.13-dev: Multi-User Setup, can't change anything with to document details
### Description

Hi, I am pretty new to paperless. I wanted to start directly with the multi-user feature for my whole family. So I decided to install the dev-docker image. It runs quote nice, there is just this one thing...
When a document has permission set to owner: Person A and read permission to Person B. I cannot change anything within the document e.g. Name, Permission, Tags, Document type etc.

When I try to do so, there is just this error:
`Error saving document: Http failure response for https://example.com/api/documents/23/: 500 OK`

The change thru the webui won't be saved. When I remove the permission (read-permission to Person B) everything is working thru the webui. When I put the permission back, the error comes back as well.

BTW I can change things without a problem within the Django admin panel.

### Steps to reproduce

1. Have a document
2. Have different users
3. Share this document as an owner to a second person
4. Error pops up when trying to change information on the document
5. remove permission, leave owner
6. Document can be changed
7. give read permission to the second person again
8. same error comes back

### Webserver logs

```bash
[2023-03-31 22:13:36,819] [ERROR] [django.request] Internal Server Error: /api/documents/23/
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/asgiref/sync.py", line 486, in thread_handler
    raise exc_info[1]
  File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 43, in inner
    response = await get_response(request)
  File "/usr/local/lib/python3.9/site-packages/django/core/handlers/base.py", line 253, in _get_response_async
    response = await wrapped_callback(
  File "/usr/local/lib/python3.9/site-packages/asgiref/sync.py", line 448, in __call__
    ret = await asyncio.wait_for(future, timeout=None)
  File "/usr/local/lib/python3.9/asyncio/tasks.py", line 442, in wait_for
    return await fut
  File "/usr/local/lib/python3.9/site-packages/asgiref/current_thread_executor.py", line 22, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.9/site-packages/asgiref/sync.py", line 490, in thread_handler
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/views/decorators/csrf.py", line 55, in wrapped_view
    return view_func(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/viewsets.py", line 125, in view
    return self.dispatch(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
  File "/usr/src/paperless/src/documents/views.py", line 278, in update
    response = super().update(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/mixins.py", line 68, in update
    self.perform_update(serializer)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/mixins.py", line 78, in perform_update
    serializer.save()
  File "/usr/local/lib/python3.9/site-packages/rest_framework/serializers.py", line 207, in save
    self.instance = self.update(self.instance, validated_data)
  File "/usr/src/paperless/src/documents/serialisers.py", line 419, in update
    super().update(instance, validated_data)
  File "/usr/src/paperless/src/documents/serialisers.py", line 210, in update
    self._set_permissions(validated_data["set_permissions"], instance)
  File "/usr/src/paperless/src/documents/serialisers.py", line 149, in _set_permissions
    set_permissions_for_object(permissions, object)
  File "/usr/src/paperless/src/documents/permissions.py", line 70, in set_permissions_for_object
    if len(users_to_remove) > 0:
  File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 376, in __len__
    self._fetch_all()
  File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 1867, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 87, in __iter__
    results = compiler.execute_sql(
  File "/usr/local/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1385, in execute_sql
    sql, params = self.as_sql()
  File "/usr/local/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 603, in as_sql
    raise NotSupportedError(
django.db.utils.NotSupportedError: difference is not supported on this database backend.
```


### Browser logs

_No response_

### Paperless-ngx version

1.13.0-dev

### Host OS

Docker@Ubuntu 22.04

### Installation method

Docker - official image

### Browser

Safari

### Configuration changes

_No response_

### Other

_No response_


Discussion:
What database engine are you using?

Resolving patch:
diff --git a/src/documents/permissions.py b/src/documents/permissions.py
--- a/src/documents/permissions.py
+++ b/src/documents/permissions.py
@@ -66,7 +66,7 @@ def set_permissions_for_object(permissions, object):
             with_group_users=False,
         )
         if len(users_to_add) > 0 and len(users_to_remove) > 0:
-            users_to_remove = users_to_remove.difference(users_to_add)
+            users_to_remove = users_to_remove.exclude(id__in=users_to_add)
         if len(users_to_remove) > 0:
             for user in users_to_remove:
                 remove_perm(permission, user, object)
@@ -87,7 +87,7 @@ def set_permissions_for_object(permissions, object):
             permission,
         )
         if len(groups_to_add) > 0 and len(groups_to_remove) > 0:
-            groups_to_remove = groups_to_remove.difference(groups_to_add)
+            groups_to_remove = groups_to_remove.exclude(id__in=groups_to_add)
         if len(groups_to_remove) > 0:
             for group in groups_to_remove:
                 remove_perm(permission, group, object)


Test patch: