Package: pnu-ngc
Installationpip installpnu-ngcngc(1)NAMEngc - n-grams countSYNOPSISngc[-b|--block]
[-c|--convert ARGS]
[-d|--discard ARGS]
[-l|--length ARG]
[-p|--partial ARG]
[-q|--quiet]
[-s|--summary]
[-t|--text]
[-w|--word]
[--debug]
[--help|-?]
[--version]
[--]
[filename ...]DESCRIPTIONThengcutility is used for counting the number of occurrences and computing the frequency ofn-gramsincryptanalysis.For n=1, the n-gram is simply a letter or character. For n=2, the n-gram is called a bigram or digraph. And so on...The-loption is for setting the length of the n-gram (the default is 1 character), and the-boption for using a fixed-window instead of the default sliding one ("ABCD" giving "AB" and "CD" instead of "AB", "BC" and "CD").The-coption is used to perform some prior conversions on your input data: Unicode characters removal (especially accented ones), upper to lower case conversions (or the reverse), extra spaces removal (this last one is performed after other conversions).You can also use the-doption to discard selected categories of characters (for example if you only want to keep letters).If you want to check your input data after these transformations, you can use the-toption to print it.If you want to print only this, you can use the-qoption.And if you want to print some stats on the remaining characters, you can print a summary with the-soption.
This summary also includes thecoincidence indexof your input text.Finally, you can use the-woption to process your input word by word instead of line by line.
If you selected the fixed-window-boption, you can decide what to do with partial blocks with the-poption:
between keeping them as-is, discarding them, or filling them with spaces in order to have only n-grams of the same length.Thengcutility processes all the indicated file names as one file.
If none are provided, it processes the standard input, thus behaving as a filter.OPTIONSOptionsUse-b|--blockUse fixed- instead of sliding-windows blocks-c|--convert ARGSConvert text input. A combination of:a/ Unicode characters to ASCII (remove accents)l/ Upper case letters to loweru/ Lower case letters to uppers/ Spaces-like characters to 1 spaceWarning:landucan't be used at the same time-d|--discard ARGSDiscard characters. A combination of:U/ Unicode charactersu/ Upper case lettersl/ Lower case lettersL/ All lettersc/ Connection symbols (apostrophe and hyphen)d/ Digitsp/ Punctuation (.,;:?!)o/ Other printable symbolss/ Spaces (space, tab, return, formfeed, vtab)n/ Non printable control characters-l|--length ARGLength of the n-gram. Defaults to 1-p|--partial ARGSWhat to do with partial blocks? One among:d/ Discardk/ Keep as-is (default)j/ Keep but right-justify with spaces-q|--quietDon't show occurrences and frequency by n-gram-s|--summaryShow a summary of what was processed-t|--textShow modified text input-w|--wordRespect Word boundaries (delimited by spaces)--debugEnable debug mode--help|-?Print usage and a short help message and exit--versionPrint version and exit--Options processing terminatorENVIRONMENTThe NGC_DEBUG environment variable can also be set to any value to enable debug mode.EXIT STATUSThengcutility exits 0 on success, and >0 if an error occurs.SEE ALSOwc(1),caesar(1),Frequency analysis,Index of coincidenceSTANDARDSThengcutility is not a standard UNIX/POSIX command.It tries to follow thePEP 8style guide forPythoncode.HISTORYThis utility was made forThe PNU project, while playing with a reimplementation of thecaesar(1)utility.LICENSEThis utility is available under the3-clause BSD license.AUTHORSHubert Tournier