Repository: beetbox/beets

Issue:
Option to copy files atomically
### Problem

When you importing a huge flac file to network disk via not so fast network (wifi for example) it requires a big time to complete.

If network disk has an agent that indexes files as soon as it was uploaded — for example default behaviour of Synology — it may start to index file *before* it is fully uploaded.

As result index will be broken.

The easy way to fix it is make inserting atomic by split it to two steps:
1) upload file to library with name like `$realName.$someTmpPostfix`
2) rename from `$realName.$someTmpPostfix` to `$realName`



Discussion:
Hello! Yes, I can see this being useful as an option. Doing so would need to take some care to ensure that the temporary filename is unlikely to be confused for a complete music file.
@sampsyo I guess the name `[name].[somerandomstring].beets` should be good ;)
> If network disk has an agent that indexes files as soon as it was uploaded — for example default behaviour of Synology — it may start to index file _before_ it is fully uploaded.
> 
> As result index will be broken.

TBH, that sounds like a bug with the indexing service. Shouldn't it detect when the file is subsequently modified (on completion of the upload) and re-index? Is this really a race condition that beet should workaround?
@wisp3rwind this bug was reported to Synology also ;)

But it is a good practice to make upload atomic via temp file into the same directory.

For example it is the best practice when you're wirking with maildir.
I think it's common to prefix temporary files with a `.` to hide them from the file explorer (and other tools) whilst it is still being copied. e.g. `.[name].[somerandomstring].beets` - rsync does this.
@jackwilsdon agreed!

@sampsyo any hope that it can be done? If you suggest places in the code where I need to add it, I can do it. But I scary that I may broke something :)
A good place to start looking into it would be the utility functions that implement moving and copying, such as [beets.util.move](https://github.com/beetbox/beets/blob/361cabe023f42e5487445779d2fa581798ea77ea/beets/util/__init__.py#L477).
@sampsyo ok, maybe I'll do it in next _few days_

But if you can do it before, I'll be very appreciated :)
I'd love to confirm that #4060 fixed all issues with synology indexing.

Resolving patch:
diff --git a/beets/util/__init__.py b/beets/util/__init__.py
--- a/beets/util/__init__.py
+++ b/beets/util/__init__.py
@@ -19,6 +19,7 @@
 import errno
 import locale
 import re
+import tempfile
 import shutil
 import fnmatch
 import functools
@@ -478,6 +479,11 @@ def move(path, dest, replace=False):
     instead, in which case metadata will *not* be preserved. Paths are
     translated to system paths.
     """
+    if os.path.isdir(path):
+        raise FilesystemError(u'source is directory', 'move', (path, dest))
+    if os.path.isdir(dest):
+        raise FilesystemError(u'destination is directory', 'move',
+                              (path, dest))
     if samefile(path, dest):
         return
     path = syspath(path)
@@ -487,15 +493,23 @@ def move(path, dest, replace=False):
 
     # First, try renaming the file.
     try:
-        os.rename(path, dest)
+        os.replace(path, dest)
     except OSError:
-        # Otherwise, copy and delete the original.
+        tmp = tempfile.mktemp(suffix='.beets',
+                              prefix=py3_path(b'.' + os.path.basename(dest)),
+                              dir=py3_path(os.path.dirname(dest)))
+        tmp = syspath(tmp)
         try:
-            shutil.copyfile(path, dest)
+            shutil.copyfile(path, tmp)
+            os.replace(tmp, dest)
+            tmp = None
             os.remove(path)
         except OSError as exc:
             raise FilesystemError(exc, 'move', (path, dest),
                                   traceback.format_exc())
+        finally:
+            if tmp is not None:
+                os.remove(tmp)
 
 
 def link(path, dest, replace=False):


Test patch: