Package: neuspell
NeuSpell: A Neural Spelling Correction ToolkitContentsInstallation & Quick StartIntroductionPretrained modelsDemo SetupDatasetsApplicationsAdditional RequirementsUpdatesApril 2021:neuspellis now available through pip. To install, simply dopip install neuspell.March, 2021: Code-base reformatted. Addressed some bug fixes.November, 2020: Neuspell'sBERTpretrained model is now available as part of huggingface models
asmurali1996/bert-base-cased-spell-correction. We provide an example code snippet
at./scripts/huggingfacefor curious practitioners.September, 2020: This work is accepted at EMNLP 2020 (system demonstrations)Installationgitclonehttps://github.com/neuspell/neuspell;cdneuspell
pipinstall-e.To install extra requirements,pipinstall-rextras-requirements.txtor individually as (NOTE: Forzsh, use ".[elmo]" and ".[spacy]")pipinstall-e.[elmo]pipinstall-e.[spacy]Additionally,spacy modelscan be downloaded as:python-mspacydownloaden_core_web_smFollowAdditional Requirementsfor installing non-neural spell checkers-AspellandJamspell.Then, download pretrained models followingPretrained modelsHere is a quick-start code snippet (command line usage). (Seetest.pyfor more usage
patterns)""" select spell checkers """fromneuspellimportBertChecker""" load spell checkers """checker=BertChecker()checker.from_pretrained()""" spell correction """checker.correct("I luk foward to receving your reply")# → "I look forward to receiving your reply"checker.correct_strings(["I luk foward to receving your reply",])# → ["I look forward to receiving your reply"]checker.correct_from_file(src="noisy_texts.txt")# → "Found 450 mistakes in 322 lines, total_lines=350"""" evaluation of models """checker.evaluate(clean_file="bea60k.txt",corrupt_file="bea60k.noise.txt")# → data size: 63044# → total inference time for this data is: 998.13 secs# → total token count: 1032061# → confusion table: corr2corr:940937, corr2incorr:21060,#                    incorr2corr:55889, incorr2incorr:14175# → accuracy is 96.58%# → word correction rate is 79.76%""" fine-tuning on domain specific dataset """checker.finetune(clean_file="sample_clean.txt",corrupt_file="sample_corrupt.txt")# Once the model is fine-tuned, you can use the saved model checkpoint path#   to load and infer by calling `checker.from_pretrained(...)` as aboveAlternatively, once can also select and load a spell checker differently as follows:fromneuspellimportSclstmCheckerchecker=SclstmChecker()checker=checker.add_("elmo",at="input")# elmo or bert, input or outputchecker.from_pretrained()checker.finetune(clean_file="./data/traintest/test.bea322",corrupt_file="./data/traintest/test.bea322.noise")IntroductionNeuSpell is an open-source toolkit for context sensitive spelling correction in English. This toolkit comprises of 10
spell checkers, with evaluations on naturally occurring mis-spellings from multiple (publicly available) sources. To
make neural models for spell checking context dependent, (i) we train neural models using spelling errors in context,
synthetically constructed by reverse engineering isolated mis-spellings; and  (ii) use richer representations of the
context.This toolkit enables NLP practitioners to use our proposed and existing spelling correction systems, both via a
simple unified command line, as well as a web interface. Among many potential applications, we demonstrate the utility
of our spell-checkers in combating adversarial misspellings.Demo available athttp://neuspell.github.io/List of neural models in the toolkit:CNN-LSTMSC-LSTMNested-LSTMBERTSC-LSTM plus ELMO (at input)SC-LSTM plus ELMO (at output)SC-LSTM plus BERT (at input)SC-LSTM plus BERT (at output)This pipeline corresponds to the `SC-LSTM plus ELMO (at input)` model.PerformancesSpellCheckerWordCorrectionRateTime persentence(in milliseconds)Aspell48.77.3*Jamspell68.92.6*CNN-LSTM75.84.2SC-LSTM76.72.8Nested-LSTM77.36.4BERT79.17.1SC-LSTM plus ELMO (at input)79.815.8SC-LSTM plus ELMO (at output)78.516.3SC-LSTM plus BERT (at input)77.06.7SC-LSTM plus BERT (at output)76.07.2Performance of different correctors in the NeuSpell toolkit on theBEA-60Kdataset with real-world spelling
mistakes. ∗ indicates evaluation on a CPU (for others we use a GeForce RTX 2080 Ti GPU).Pretrained modelsCheckpointsRun the following to download checkpoints of all neural modelscd data/checkpoints
python download_checkpoints.pySeedata/checkpoints/README.mdfor more details. You can alternatively choose to download only selected models'
checkpoints.Demo SetupIn order to setup a demo, follow these steps:DoInstallationDownloadcheckpointsStart a flask server atneuspell/flask-serverby runningCUDA_VISIBLE_DEVICES=0 python app.py(on GPU) orpython app.py(on CPU)DatasetsDownload datasetsRun the following to download datasetscd data/traintest
python download_datafiles.pySeedata/traintest/README.mdfor more details.Synthetic Training Dataset CreationThe toolkit offers 4 kinds of noising strategies to generate synthetic parallel training data to train neural models for
spell correction.RANDOMWord ReplacementProbabilistic ReplacementA combination ofWord ReplacementandProbabilistic ReplacementTrain files are dubbed with names.random,.word,.prob,.probwordrespectively. For each
strategy, we noise ∼20% of the tokens in the clean corpus. We use 1.6 Million sentences from
theOne billion word benchmarkdataset as our clean corpus.Potential applications for practitionersDefenses against adversarial attacks in NLPexample implementation available in folder./applications/Adversarial-MisspellingsImproving OCR text correction systemsImproving grammatical error correction systemsImproving Intent/Domain classifiers in conversational AISpell Checking in Collaboration and Productivity toolsAdditional requirementRequirements forAspellchecker:wget https://files.pythonhosted.org/packages/53/30/d995126fe8c4800f7a9b31aa0e7e5b2896f5f84db4b7513df746b2a286da/aspell-python-py3-1.15.tar.bz2
tar -C . -xvf aspell-python-py3-1.15.tar.bz2
cd aspell-python-py3-1.15
python setup.py installRequirements forJamspellchecker:sudo apt-get install -y swig3.0
wget -P ./ https://github.com/bakwc/JamSpell-models/raw/master/en.tar.gz
tar xf ./en.tar.gz --directory ./Citation@inproceedings{jayanthi-etal-2020-neuspell,
    title = "{N}eu{S}pell: A Neural Spelling Correction Toolkit",
    author = "Jayanthi, Sai Muralidhar  and
      Pruthi, Danish  and
      Neubig, Graham",
    booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations",
    month = oct,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/2020.emnlp-demos.21",
    doi = "10.18653/v1/2020.emnlp-demos.21",
    pages = "158--164",
    abstract = "We introduce NeuSpell, an open-source toolkit for spelling correction in English. Our toolkit comprises ten different models, and benchmarks them on naturally occurring misspellings from multiple sources. We find that many systems do not adequately leverage the context around the misspelt token. To remedy this, (i) we train neural models using spelling errors in context, synthetically constructed by reverse engineering isolated misspellings; and (ii) use richer representations of the context. By training on our synthetic examples, correction rates improve by 9{\%} (absolute) compared to the case when models are trained on randomly sampled character perturbations. Using richer contextual representations boosts the correction rate by another 3{\%}. Our toolkit enables practitioners to use our proposed and existing spelling correction systems, both via a simple unified command line, as well as a web interface. Among many potential applications, we demonstrate the utility of our spell-checkers in combating adversarial misspellings. The toolkit can be accessed at neuspell.github.io.",
}