POSIX filesystem semantics are one of the things where POSIX just plain got stuff wrong. And it isn't helped by the fact that a lot of people want to game benchmarks by slightly lying about their durability compliance, so you really have to jump through hoops to make sure you actually achieve durability.

I do agree with you that we need a better way to interact with the filesystem with regards to integrity and durability, although I'm not sure we entirely agree on how that would look. The idea of multiple modes makes a lot of sense:

* File-atomic mode. This is I believe trivial to implement in the filesystem layer for all filesystems, and the basic idea of this mode has existed for decades. When a reader opens a file for reading, it will never see any other writes to the file. A writer will only update the file when it closes the file [1], at which point any new reader will see only the new file created. The code is intrinsically safe in the face of multiple processes interacting with the file, and is probably the semantics most people would prefer in that situation.

* Append-only transactional files. Here, you can't random-access write into the file (but you can random-access read), only write at the end or truncate the file. A writer designates the text to append to the file as atomic blocks: the reader will only atomically see or not see the block [2]. If the file is truncated, all readers see the original contents of the file until they close.

* Raw files. Don't pretend that a file is a stream of bytes. Instead, expose it as a set of blocks that can be atomically updated (including atomically adding or removing blocks from the file at different places). I don't know filesystem semantics to give any good details here, but my understanding is that databases basically try to get these semantics today, and that getting good guarantees on fully random-access read/write semantics is effectively impossible anyways.

There does feel to me to be a bit of a hole here, where you basically get no multiprocess interactions via files unless you completely change how your code works, but I'm not sure it's entirely feasible to have a middle ground here. You can probably get close enough for most needs with a way to be notified and reopen the file in file-atomic mode, and anything where that's not sufficient probably needs you to go to raw files to really get the guarantees you want.

In addition to the basic file I/O issues, there also needs to be a way to be more transactional with directories, I think. Using paths as the basis for filesystem issues is already opening up programmers to time-of-check-time-of-use attacks today, and moving to a file descriptor-based approach for directory manipulation would solve that while opening up the possibility for better transactional support on the directory level.

The other issue is durability. Most applications in the first two modes would probably be fine with an optional durability: the result of an unexpected power outage would be a file that is out of date, but not corrupt. The filesystem could provide an optional callback on commit that returns when it is durability committed, which would handle those cases where you do actually need to make sure that the data will be committed on unexpected power outage. And the simple semantics of the first two modes means that providing durability reliably should be easy for filesystems.

[1] This also suggests that there should be a way to abort the write.

[2] You can also see how a file-atomic reader can interact with an append-only writer: the filesystem layer needs to remember the size the reader first saw and pretend that's the EOF, but otherwise there's no issue. And append-only readers will act as a file-atomic reader with respect to a file-atomic writer. The interactions make sense, that's a good sign for the model!