Neat to see this made it to HN! Congrats.

So, my major concern with OrbitDB is the use of in-memory indexes to hashes. It's a very naive approach for multiple reasons. It just doesn't scale well. Node.js handles these particularly poorly, and develops memory leaks in my own experiments with AOILs (append-only immutable logs) inspired by Secure Scuttlebutt.

What I mean is, take a look at this code:
https://github.com/orbitdb/orbit-db-kvstore/blob/master/src/...

And this:
https://github.com/orbitdb/orbit-db-docstore/blob/master/src...

I'd suggest perhaps building that in SnapDB:

https://github.com/ClickSimply/snap-db

Though, there are some really hard limits to file locks / mutexes that are needed for a decent persistent K/V store. Since, pretty much any database is going to need an index, and an index is just a K/V store. So, in essence, if your platform can't be relied on to implement a good embedded K/V store, it can't be relied on to create anything other than toy databases.

I considered adding SnapDB to OrbitDB, but it would've required a lot of refactoring to support asynchronous lookup. I eventually realized I could do a lot of what I wanted to accomplish without the elaborate CRDTs like OrbitDB employs to keep an ordered log of information.

I would also add that, although it should be obvious, just in case it isn't, AvionDB now builds on OrbitDB which itself builds on IPFS. So, that's a big consideration for those looking to work with this.

(The below is mostly from personal experience and not terribly relevant to AvionDB and OrbitDB)

I've been meaning to do a more formal post-mortem on the project, but haven't gotten anything fully organized yet. I discovered all this after creating a project a while back for what was essentially a P2P image gallery built with IPFS and Electron. I wrote it in TypeScript, React (with Hooks), Redux Toolkit (which makes Redux a lot less insufferable than it used to be, my god), and SnapDB and OrbitDB (but not together). I also had go-ipfs running in a process managed by the Electron client, and included it with the build.

As a result, the app image binary would generally be in excess of 500MB. And this was just our 0.1.0, we hadn't gotten to the really good features yet. The performance was decent in the UI, but the app itself couldn't scale very well past 1000 images. go-ipfs was the real limiting factor at the time; it would just fail in really bizarre ways, and I'd have to not only write code for restarting the go-ipfs process in case it died, but also retry requests that didn't go through. I suspected it might've been due to me having pubsub enabled, but my ability to debug that was hamstrung by the fact that OrbitDB hard fails without pubsub enabled. I haven't tried it with go-ipfs 0.5.0, I think pubsub is enabled by default with that now, and I've already moved on from that implementation.

My biggest concern in terms of performance aside from go-ipfs (which was disappointing and unexpected), was the fact that due to the design, there would be multiple copies of the same data in-memory everywhere: the in-memory indexes used by OrbitDB, the data in Redux, the data in the DOM, etc. I'm able to effectively integrate all of those using a Rust desktop binary. I'm using Iced for the UI, and I've moved on from bundling IPFS entirely. It allows me to build desktop and WASM binaries from the same codebase.

It's possible I could probably make the Electron app work better at this point, but there are still the fundamental limitations I pointed out earlier.

For me, content-addressable hashes and IPNS signatures are perfectly fine. I value performance (CPU, RAM, and disk) over anything else, primarily because I'm developing P2P apps, and without a server, the nodes have to do all the work itself, including the UI. That's why I moved off of TypeScript and into Rust. PubSub would be nice, and OrbitDB has a decent implementation of Lamport Clocks and CRDTs for decentralized ordering of events, but it's just often overkill for most things that could just be content-addressable.

---

Just to be clear, I've made a lot of claims here, and I'm hardly an expert on any one of these things, so if I've gotten any of this wrong, I'd really like to know more. Also, the code I wrote earlier isn't available to the public. If I have any say in the matter, it will be locked away within an enormous warehouse, Indiana Jones style, hopefully never to see the light of day again.