Since it seems we hit the max thread limit (I can't reply to your reply to me), I'll post my reply here, quoting your post as best as I can.

>> I wasn't planning to; it's a once-off cost - the user experience while using any software isn't degraded by the installation time, is it?

> This whole topic started with you claiming that you can even beat Everything in that regard

Nope.

I never claimed that I can beat Everything in "reading the metadata when the app starts". I claimed that I can match the startup and search performance of Everything.

Those are two different claims, and the latter is obviously possible if the application performs queries by querying a daemon that is always running with an in-memory index.

> Remember, your response to:

>> A huge part of Everything's speed comes from reading the master file table that other people mentioned, so you would need a way to quickly read file table entries on linux.

> Was

>> Not a problem. And no, I'm not talking about inotify either, and I'll additionally index the contents of (text) files as well with a negligible additional performance hit. It can be done as fast as, or faster than, `Everything`.

It's "not a problem because the index will already be in in-ram and available before the user launches the app". You read that to mean "not a problem because there is a fast way to read file metadata on startup".

I think that there's a difference in that. My proposal is to never have the app need to read anything on startup (other than configuration settings).

> And btw. indexing content will obviously only put you even further behind. The cost is not negligible.

How will it put me further behind? I did say that it will only be done during software installation, right?

>> It's a daemon. if it isn't running while the user is using their desktop system, then it's not working or the user has turned it off.

>> My desktop system currently has 2.5m files. There are maybe a dozen files which will be modified during a maintenance-mode bootup, which has happened exactly zero times in the last decade.

I have many users, myself included, who use things like shared filesystems, which get modified by multiple systems. And like I already said, modern Linux systems also perform all of their updates in such a maintainance mode. So your app will give thousands of false positives or miss thousands of files completely on those systems.

There's two things there:

1. Shared filesystems - I don't care about this because Everything doesn't care about this being performant: In Everything, as far as I understand it, the user manually indexes network shares.

2. Maintenance modes won't give you thousands of false positives; at most you're looking at a diff of maybe dozens of index entries, if that.

> Sigh... So you're also not going to solve the second issue. I mean I clearly asked you these questions multiple times and I tried to make it clear, that this is where the problem is, to save you and me time, and still you kept it a secret up until now that you're not even attempting to fix those problems.

I didn't keep it a secret - I made it clear that a daemon will hold the index, and the app will talk to it, and that the index will be built once during software installation.

> So I'll have to take back my claim: Under these circumstances I can't guarantee that you'll make a lot of donations, because your app won't do anything special compared to others.

Well, it will be a few orders of magnitude faster to start up than checking for filesystem changes on startup, no?

>> For a Linux desktop file finding utility, monitoring all file writes, moves and deletes pretty much puts you ahead of any game in town right now, right?

> Well kind off, but it's not particularly difficult to solve that issue. The dev versions of FSeaerch already can do that.

If you don't mind me asking, how do you do it? Because inotify is out of the question if you want to monitor 2.5m files. Even for just the home directory you will run the risk of exhausting file descriptors by using inotify.

>> Issue 1 - Initial index creation: I will create the index during the s/ware installation process and never create it again unless it is missing. To speed the creation during installation, I will use the mlocate.db file if it is found.

> So you're doing exactly what everyone else is doing.

All the existing utilities create the index only during installation?

> You can also ingore the mlocate index, because it doesn't contain enough information (size, date modified, ... aren't indexed by it so you'd need to stat all of those files anyway).

>> Issue 2 - Files that are changed/moved/removed when daemon is turned off: I don't really care, mostly. Those files a) have such a small probability of both existing and being of interest to the desktop user that lottery jackpots have a higher chance of happening to the user

> Like I already said, you're ignoring the hard and important problem. That's fine, but you suggested otherwise and now you're again doing nothing out of the ordinary.

Which hard and important problem? That changes made in maintenance mode aren't seen?

>> I believe that this is enough to satisfy my original claim[1] of " "similar in performance and query capabilities""[2].

> Well, it depends, you're not going to beat Everthing in the areas me and others care and in an attempt to get anywhere near that, you're trading accuracy for speed.

Going from 100.0000000% accurate to 99.9999999% accurate is hardly "sacrificing accuracy for speed", considering that you're still in the statistical rounding error group.

> That's fine, but this is nothing new or special, so I'm not really interested in that.

"Faster than existing Linux tools" would, actually, be something new and novel. "Faster than Everything in some specific areas" almost certainly counts, especially when accuracy is within error bars.

I have one last batch of questions, after which I will simply shut up and get to coding something. I kinda hope that you will answer these questions.

A major feature of Everything when people wax on about its speed is how quickly new entries in the filesystem show up in the applications query results.

Even while the results is open, the user can see files that were added since the last keystroke.

1. How does FSearch handle this common and obvious use-case?

2. What's the newest filesystem change you can expect to see when performing a query in FSearch? Is it "the last change made prior to the application startup"? Is it "The last change made prior to the query"? Is it "The last change made since we walked the filesystem"?

3. What's the p99 for startup time in FSearch? The p99 for query results of N (where N is a suitably large number)?

4. You mentioned "areas that you and others care about". Can you briefly list the areas, other than complete and 100% accuracy during maintenance mode. All I know about is what Everthing users appear to care about, and they simply aren't caring about USB memory sticks, cameras plugged in, network drives, maintenance mod diffs, etc. They do appear to care that it is responsive.