Repository: paperless-ngx/paperless-ngx

Issue:
[BUG] Errors after mass changing tags when filename includes tags
### Description

Hi,

today I imported a couple hundred of documents and mass changed their tags. As my file naming is including tags that also resulted in a mass rename of files. I noticed many(!) log entries from the webserver container telling me that a file 'has gone'. I suspect it beeing the renaming due to the naming scheme and the log entry is mentioning the old name (old tags, not new ones). 

The files are still there and metadata is correct at any time. The logs are just a bit disturbing. Webserver logs below just as an example. I got hundreds of them as mentioned.

There might also be a correlation to the media directory (all of it) beeing stored within a CIFS share. I can't easily test it on a local drive.

**Mount options**: type cifs (rw,relatime,vers=3.1.1,cache=strict,username=dms,uid=1001,noforceuid,gid=1001,noforcegid,addr=192.168.200.50,file_mode=0755,dir_mode=0755,soft,nounix,serverino,mapposix,rsize=4194304,wsize=4194304,bsize=1048576,echo_interval=60,actimeo=1,user=dms)

**PAPERLESS_FILENAME_FORMAT**={document_type}/{correspondent}/{created_year}{created_month}{created_day}-{title}-{tag_list}

As there is no real impact I'd categorize this as rather low priority. If I can add any more info to this I'm happy to do so.

### Steps to reproduce

- Have a naming scheme that includes tags as part of the file name
- Give new tags to a couple hundred files
- Watch webserver logs for a couple of minutes

### Webserver logs

```bash
webserver_1  | [2022-10-11 23:13:32,582] [CRITICAL] [paperless.handlers] Document 1974-11-09 17-scan_20181123_000219: File /usr/src/paperless/media/documents/originals/none/none/19741109-17-scan_20181123_000219-INBOX,Tmp- Backlog,Tmp- Unsorted.pdf has gone.
webserver_1  | [2022-10-11 23:13:32,735] [CRITICAL] [paperless.handlers] Document 1974-11-09 28-scan_20181123_000219: File /usr/src/paperless/media/documents/originals/none/none/19741109-28-scan_20181123_000219-INBOX,Tmp- Backlog,Tmp- Unsorted.pdf has gone.
```


### Paperless-ngx version

1.9.2

### Host OS

Ubuntu 22.04 LTS

### Installation method

Docker - official image

### Browser

n/a

### Configuration changes

PAPERLESS_FILENAME_FORMAT={document_type}/{correspondent}/{created_year}{created_month}{created_day}-{title}-{tag_list}

### Other

_No response_


Discussion:
Quick update: Did the same with only a few files (about 10) and the issue did **not** happen. So only with mass changes.
This seems most likely to be an issue caused by hosting the media on a network share.  I was able to rename ~600 files both from the UI by applying a tag when the format includes the tag list.
I do agree that this is probably related to some async behavior of the CIFS share. I'll play around with this a bit on the weekend. Any idea if that could have some real impact on Paperless? Like orphaned DB entries and the likes or is this likely to be log noise?
Acutally I'm wondering why this happens. If I do understand the code correctly it is the following line giving me the error in the log:

https://github.com/paperless-ngx/paperless-ngx/blob/8feada6907a616c16956646810342dca11e18a02/src/documents/signals/handlers.py#L367-L371

The exception raised just passes, so that's fine. If the move is done (maybe slowly by the CIFS server) then it is correct behaviour as the oldfile is no longer there.

This function is only called in two lines:

https://github.com/paperless-ngx/paperless-ngx/blob/8feada6907a616c16956646810342dca11e18a02/src/documents/signals/handlers.py#L367-L371

Which are part of this function:

https://github.com/paperless-ngx/paperless-ngx/blob/8feada6907a616c16956646810342dca11e18a02/src/documents/signals/handlers.py#L382-L384

What I struggle to understand is why this error happens. 

- Tag changes
- Paperless tries to change the filename
- It gets a success from the OS for that (otherwise there would be a different error)
- The oldfile does not vanish instantly or the new file is not visible instantly (cifs cache, async, whatever)
- Something happens
- Paperless calls validate_move again, this time logging an error

That 'something' is what irritates me. The rename actually happens (before the error is logged). So why is it trying it again at all?

Edit: Sorry that all of my reports seem to be hard to grasp stuff with way too long postings on them as the one about memory consumption ;)
`validate_move` is before any renaming happens.  The idea is to check the from file exists and the to file does not exist before the rename happens.  It's called twice, once for the original document, once for the archive version, if it exists.

So this error log is from the check for the from file existing.  I wonder, did you apply multiple tags to documents at once?  That may fire the signals multiple times.  Though they should be locked out by the FileLock
If memory serves correctly I was adding 2 tags and removing one, yes. Not completly sure if I did the removal in the same step or if that was separate. But you're correct about adding two at the same time to about 1000 documents (checked that again).

All of the files I was working on also do have an archive version.

Quick check of FileLock behaviour did not come up with anything obvious related to CIFS. Only their documentation mentioning that async IO is not supported. But that shouldn't really come into effect here. Leaves me wondering.

Sorry for the question, I'm not really understanding the code here: If there are two Events (signals?) fired for let's say two new tags. Are they queued or is it trying to do them more or less at the same time? Trying to understand a bit about how it is working here so I might be able to dig a bit deeper.

I do understand that this just seems to a nuisance in my specific setup and that the priority is rather low :)
I've played around with this further in unit tests.  In everything I could come up with, the accesses and renames were properly serialized, presumably by the file system lock.  If that glitched briefly during the initial lock, I could see how maybe two renames got going at once, if indeed that is what happened. 
I've managed to reproduce (and fix) this.  The locking happens fine and I don't think CIFS is a problem.  If multiple updates are queued up behind the lock, the moving of file may attempt to happen on stale data from the database.
Thanks for investing the time into an issue I thought to be a corner case specific to my setup!

Can I do anything to help? Like testing a feature branch or something like that?

Resolving patch:
diff --git a/src/documents/consumer.py b/src/documents/consumer.py
--- a/src/documents/consumer.py
+++ b/src/documents/consumer.py
@@ -405,6 +405,7 @@ def progress_callback(current_progress, max_progress):
 
                 # Don't save with the lock active. Saving will cause the file
                 # renaming logic to acquire the lock as well.
+                # This triggers things like file renaming
                 document.save()
 
                 # Delete the file only if it was successfully consumed
@@ -438,6 +439,9 @@ def progress_callback(current_progress, max_progress):
 
         self._send_progress(100, 100, "SUCCESS", MESSAGE_FINISHED, document.id)
 
+        # Return the most up to date fields
+        document.refresh_from_db()
+
         return document
 
     def _store(self, text, date, mime_type) -> Document:
diff --git a/src/documents/signals/handlers.py b/src/documents/signals/handlers.py
--- a/src/documents/signals/handlers.py
+++ b/src/documents/signals/handlers.py
@@ -400,6 +400,13 @@ def update_filename_and_move_files(sender, instance, **kwargs):
 
     with FileLock(settings.MEDIA_LOCK):
         try:
+
+            # If this was waiting for the lock, the filename or archive_filename
+            # of this document may have been updated.  This happens if multiple updates
+            # get queued from the UI for the same document
+            # So freshen up the data before doing anything
+            instance.refresh_from_db()
+
             old_filename = instance.filename
             old_source_path = instance.source_path
 


Test patch:
diff --git a/src/documents/tests/test_consumer.py b/src/documents/tests/test_consumer.py
--- a/src/documents/tests/test_consumer.py
+++ b/src/documents/tests/test_consumer.py
@@ -14,6 +14,7 @@
     import backports.zoneinfo as zoneinfo
 
 from django.conf import settings
+from django.utils import timezone
 from django.test import override_settings
 from django.test import TestCase
 
@@ -326,6 +327,12 @@ def get_test_archive_file(self):
     def testNormalOperation(self):
 
         filename = self.get_test_file()
+
+        # Get the local time, as an aware datetime
+        # Roughly equal to file modification time
+        rough_create_date_local = timezone.localtime(timezone.now())
+
+        # Consume the file
         document = self.consumer.try_consume_file(filename)
 
         self.assertEqual(document.content, "The Text")
@@ -351,7 +358,20 @@ def testNormalOperation(self):
 
         self._assert_first_last_send_progress()
 
-        self.assertEqual(document.created.tzinfo, zoneinfo.ZoneInfo("America/Chicago"))
+        # Convert UTC time from DB to local time
+        document_date_local = timezone.localtime(document.created)
+
+        self.assertEqual(
+            document_date_local.tzinfo,
+            zoneinfo.ZoneInfo("America/Chicago"),
+        )
+        self.assertEqual(document_date_local.tzinfo, rough_create_date_local.tzinfo)
+        self.assertEqual(document_date_local.year, rough_create_date_local.year)
+        self.assertEqual(document_date_local.month, rough_create_date_local.month)
+        self.assertEqual(document_date_local.day, rough_create_date_local.day)
+        self.assertEqual(document_date_local.hour, rough_create_date_local.hour)
+        self.assertEqual(document_date_local.minute, rough_create_date_local.minute)
+        # Skipping seconds and more precise
 
     @override_settings(FILENAME_FORMAT=None)
     def testDeleteMacFiles(self):