Package: pyspellchecker
Pure Python Spell Checking based onPeter
Norvig’sblog post on setting
up a simple spell checking algorithm.It uses aLevenshtein Distancealgorithm to find permutations within an edit distance of 2 from the
original word. It then compares all permutations (insertions, deletions,
replacements, and transpositions) to known words in a word frequency
list. Those words that are found more often in the frequency list aremore likelythe correct results.pyspellcheckersupports multiple languages including English, Spanish,
German, French, Portuguese, Arabic and Basque. For information on how the dictionaries were
created and how they can be updated and improved, please see theDictionary Creation and Updatingsection of the readme!pyspellcheckersupportsPython 3pyspellcheckerallows for the setting of the Levenshtein Distance (up to two) to check.
For longer words, it is highly recommended to use a distance of 1 and not the
default 2. See the quickstart to find how one can change the distance parameter.InstallationThe easiest method to install is using pip:pipinstallpyspellcheckerTo build from source:gitclonehttps://github.com/barrust/pyspellchecker.gitcdpyspellcheckerpython-mbuildForpython 2.7support, installrelease 0.5.6but note that no future updates will supportpython 2.pipinstallpyspellchecker==0.5.6QuickstartAfter installation, usingpyspellcheckershould be fairly straight
forward:fromspellcheckerimportSpellCheckerspell=SpellChecker()# find those words that may be misspelledmisspelled=spell.unknown(['something','is','hapenning','here'])forwordinmisspelled:# Get the one `most likely` answerprint(spell.correction(word))# Get a list of `likely` optionsprint(spell.candidates(word))If the Word Frequency list is not to your liking, you can add additional
text to generate a more appropriate list for your use case.fromspellcheckerimportSpellCheckerspell=SpellChecker()# loads default word frequency listspell.word_frequency.load_text_file('./my_free_text_doc.txt')# if I just want to make sure some words are not flagged as misspelledspell.word_frequency.load_words(['microsoft','apple','google'])spell.known(['microsoft','google'])# will return both now!If the words that you wish to check are long, it is recommended to reduce thedistanceto 1. This can be accomplished either when initializing the spell
check class or after the fact.fromspellcheckerimportSpellCheckerspell=SpellChecker(distance=1)# set at initialization# do some work on longer wordsspell.distance=2# set the distance parameter back to the defaultNon-English Dictionariespyspellcheckersupports several default dictionaries as part of the default
package. Each is simple to use when initializing the dictionary:fromspellcheckerimportSpellCheckerenglish=SpellChecker()# the default is English (language='en')spanish=SpellChecker(language='es')# use the Spanish Dictionaryrussian=SpellChecker(language='ru')# use the Russian Dictionaryarabic=SpellChecker(language='ar')# use the Arabic DictionaryThe currently supported dictionaries are:English       - ‘en’Spanish       - ‘es’French        - ‘fr’Portuguese    - ‘pt’German        - ‘de’Italian       - ‘it’Russian       - ‘ru’Arabic        - ‘ar’Basque        - ‘eu’Latvian       - ‘lv’Dutch         - ‘nl’Dictionary Creation and UpdatingThe creation of the dictionaries is, unfortunately, not an exact science. I have provided a script that, given a text file of sentences (in this case fromOpenSubtitles) it will generate a word frequency list based on the words found within the text. The script then attempts to*clean up*the word frequency by, for example, removing words with invalid characters (usually from other languages), removing low count terms (misspellings?) and attempts to enforce rules as available (no more than one accent per word in Spanish). Then it removes words from a list of known words that are to be removed. It then adds words into the dictionary that are known to be missing or were removed for being too low frequency.The script can be found here:scripts/build_dictionary.py`. The original word frequency list parsed from OpenSubtitles can be found in the`scripts/data/`folder along with each language’sincludeandexcludetext files.Any help in updating and maintaining the dictionaries would be greatly desired. To do this, adiscussioncould be started on GitHub or pull requests to update the include and exclude files could be added.Additional MethodsOn-line documentationis available; below contains the cliff-notes version of some of the available functions:correction(word): Returns the most probable result for the
misspelled wordcandidates(word): Returns a set of possible candidates for the
misspelled wordknown([words]): Returns those words that are in the word frequency
listunknown([words]): Returns those words that are not in the frequency
listword_probability(word): The frequency of the given word out of all
words in the frequency listThe following are less likely to be needed by the user but are available:edit_distance_1(word): Returns a set of all strings at a Levenshtein
Distance of one based on the alphabet of the selected languageedit_distance_2(word): Returns a set of all strings at a Levenshtein
Distance of two based on the alphabet of the selected languageCreditsPeter Norvigblog post on setting up a simple spell checking algorithmP Lison and J Tiedemann, 2016, OpenSubtitles2016: Extracting Large Parallel Corpora from Movie and TV Subtitles. In Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC 2016)