I have an unusually poor understanding of this subject, so I know I am likely to be wrong and am seeking your correction. I just don't get it.

Naively, I was shocked that you suggest classifying usr/dict/words as a test after training - to me, this seems like classifying some specific exhaustive sample catalogue of microbes in the Center for Disease Control's building for -- anything related to microbes that was trained on the real world. I expect that of course it would pass the test (be a false positive for) a hospital, as a leprosy ward, as a middle school, as a sewage processing plant, as a factory, as a kitchen, as an African steppe, as the ISS space station, as a suburb, as anything. If it's exhaustive, it'll have whatever is in those places! And if it's a catalogue, you've reduced the frequency information that could differentiate these.

Like, how could I expect an exhaustive list of unique words with their count reduced to 1 to have any ability to be classified in any way based on a training set based on words and frequencies? It just seems like a shocking suggestion because it artificially reduces to a count of 1 the exact feature you are using, and then to make sure you can't even use the presence or absence of anything, it includes 1 of everything.

I was thinking, it is like learning to classify a Romance language (French, Italian, Spanish etc) based on the frequency of the letters A-Z based on documents in those languages, and then "testing" that classification on the text "abcdefghijklmnopqrstuvwxyz". If we trained on frequency, why would we expect useful output after reducing frequency to 1? Actually since a dictionary is exhaustive, it is like including the German ß, not in other languages, the French ç, not in other languages - and then being surprised that it passes the test for German (based on letters frequency) and for French. You've completely removed the information you trained on, then added examples for anything that can be used.

This seems really unrelated to document length.

A further example I thought of is suppose you were training something to classify professions, from a database of classified CV's/resumes, to decide if someone is a programmer, cook, athlete, etc. If I fed it a list of every keyword from my entire database, one per line, then the answer to 'is this an embedded systems programmer' is yes, is this front-end developer the answer is 'yes' and if your database is big enough, is this a Historian specialized is medieval metallurgy the answer is 'yes.' Why would any of these be considered false positives since I've just put them as skills!

So I realize I am trying to be intuitive and likely am very naive, but I am totally confused why you suggest usr/dict/words.

You seem to say it is about length, but my a-z text example is just 26 characters (plus special ones) whereas the training data could be thousands of pages in each language.

I simply don't follow at all, which is frustrating because I have such a poor understanding to begin with.

I will appreciate any correction!

Sorry this ended up a bit long. I tried to include 2 or 3 practical examples so you can better judge my understanding and maybe correct it. Thank you.