Common word usage Suppose that for whatever reason, one is interested in the word usage of a piece of text, perhaps from an article such as this one, downloaded from the Net. One might want to know what word is most frequently used while ignoring all the non-words, like variable names in source code or other random bits of junk. Perhaps a ranking of word frequency is required. First we begin by breaking up the sentences in the text file so that there is no more than one word per line. This tool translates files one character at a time.
Unmatched characters are unaffected.
This accomplishes the first step of isolating words, one per line. Next we need to group the similar words together. Sorting the lines of the file will suffice. It reads two text streams which must be pre-sorted by the key field which it uses to match up lines from either stream. What is useful about natural joins is that rows of data that have no matching counterpart in the other text stream are not output.