Package: atomicwrites
Atomic file writes.fromatomicwritesimportatomic_writewithatomic_write('foo.txt',overwrite=True)asf:f.write('Hello world.')# "foo.txt" doesn't exist yet.# Now it does.SeeAPI documentationfor more
low-level interfaces.Features that distinguish it from other similar libraries (seeAlternatives and Credit):Race-free assertion that the target file doesn’t yet exist. This can be
controlled with theoverwriteparameter.Windows support, although not well-tested. The MSDN resources are not very
explicit about which operations are atomic. I’m basing my assumptions offa
commentbyDoug Cook, who appears
to be a Microsoft employee:Question: Is MoveFileEx atomic if the existing and new
files are both on the same drive?The simple answer is “usually, but in some cases it will silently fall-back
to a non-atomic method, so don’t count on it”.The implementation of MoveFileEx looks something like this: […]The problem is if the rename fails, you might end up with a CopyFile, which
is definitely not atomic.If you really need atomic-or-nothing, you can try calling
NtSetInformationFile, which is unsupported but is much more likely to be
atomic.Simple high-level API that wraps a very flexible class-based API.Consistent error handling across platforms.How it worksIt uses a temporary file in the same directory as the given path. This ensures
that the temporary file resides on the same filesystem.The temporary file will then be atomically moved to the target location: On
POSIX, it will userenameif files should be overwritten, otherwise a
combination oflinkandunlink. On Windows, it usesMoveFileExthrough
stdlib’sctypeswith the appropriate flags.Note that withlinkandunlink, there’s a timewindow where the file
might be available under two entries in the filesystem: The name of the
temporary file, and the name of the target file.Also note that the permissions of the target file may change this way. In some
situations achmodcan be issued without any concurrency problems, but
since that is not always the case, this library doesn’t do it by itself.fsyncOn POSIX,fsyncis invoked on the temporary file after it is written (to
flush file content and metadata), and on the parent directory after the file is
moved (to flush filename).fsyncdoes not take care of disks’ internal buffers, but there don’t seem
to be any standard POSIX APIs for that. On OS X,fcntlis used withF_FULLFSYNCinstead offsyncfor that reason.On Windows,_commitis used, but there are no guarantees about disk internal buffers.Alternatives and CreditAtomicwrites is directly inspired by the following libraries (and shares a
minimal amount of code):The Trac project’sutility functions,
also used inWerkzeugandmitsuhiko/python-atomicfile. The idea to usectypesinstead ofPyWin32originated there.abarnert/fatomic. Windows support
(based onPyWin32) was originally taken from there.Other alternatives to atomicwrites include:sashka/atomicfile. Originally I
considered using that, but at the time it was lacking a lot of features I
needed (Windows support, overwrite-parameter, overriding behavior through
subclassing).TheBoltons library collectionfeatures a class for atomic file writes, which seems to have a very similaroverwriteparameter. It is lacking Windows support though.LicenseLicensed under the MIT, seeLICENSE.