Keep in mind that (conventional) micro-kernels are not the same as exokernels.

FUSE is fun, I've written my own filesystems with it, but it's basically a micro-kernel idea, not an exokernel one. (L4 is also great! But I don't think it qualifies as an exokernel?)

https://pdos.csail.mit.edu/6.828/2019/lec/faq-exokernel.txt explains a lot about exokernels that's not mentioned in the papers.

Exokernels never caught on, at least not under that name. The closest equivalent in widespread use today are actually hypervisors for running virtual machines. (Especially if you are running a so called 'unikernel' on them.)

About filesystems: if you just want the kinds of abstractions that conventional filesystems already give you, you won't get too much out of using an exokernel. (As you mention, perhaps you can get a bit of extra performance?) From the FAQ I linked above:

> Q: In what kind of applications is an exokernel operating system preferable?
There are naturally tradeoffs with the extra flexibility provided e.g. it is
easier to make a mistake in user code.

> A: An exokernel is most attractive to an application that needs to do
something that is possible with an exokernel, but not possible with
other kernels. The main area in which the 1995 exokernel paper increased
flexibility was virtual memory. It turns out there are a bunch of neat
techniques applications can use if they have low-level access to virtual
memory mappings; the Appel and Li paper (citation [5]) discusses some of
them. Examples include distributed shared memory and certain garbage
collection tricks. Many operating systems in 1995 didn't give enough
low-level access to virtual memory mappings to implement such
techniques, but the exokernel did. The exokernel authors wrote a later
paper (in SOSP 1997) that describes some examples in much more depth,
including a web server that uses a customized file system layout to
provide very high performance.

The HN submission we are nominally discussing here is also about memory, so that might be applicable.

An example for filesystems I could envision: direct low-level hardware access to an SSD's internals for a database. Databases don't really care about files, and might also want to deal with SSD's peculiar writing processes in a way that's different from the abstractions typical file systems give you.

> Perhaps this is stating the obvious, but I don't think you can get much fault isolation with a pure library filesystem; if all the processes participating in the filesystem are faulty then there's no way to protect the filesystem from fatal corruption from faults. You might be able to reduce the presumed-correct filesystem core process to something like a simplified Kafka: a process that grants other processes read-only access to an append-only log and accepts properly delimited and identified blocks of data from them to append to it.

That might be possible, but wouldn't really be faster than letting a kernel handle it, I'd guess? (But it would perhaps be more flexible to develop, since it's all userland.) You can also take inspiration from how eBPF allows you to upload user level logic into the Linux kernel and run them securely. Instead of uploading them into the kernel, you could also upload them into your filesystem service, I guess?

Some of the original exokernel papers had some more interesting ideas sketched out.

> I know you have a very substantial amount of experience with this as a result of Varnish and your involvement with Fastly. What do you think?

I'm afraid you are mixing me up with someone else?