Package: nlpaug
nlpaugThis python library helps you with augmenting nlp for your machine learning projects. Visit this introduction to understand aboutData Augmentation in NLP.Augmenteris the basic element of augmentation whileFlowis a pipeline to orchestra multi augmenter together.FeaturesGenerate synthetic data for improving model performance without manual effortSimple, easy-to-use and lightweight library. Augment data in 3 lines of codePlug and play to any machine leanring/ neural network frameworks (e.g. scikit-learn, PyTorch, TensorFlow)Support textual and audio inputTextual Data Augmentation ExampleAcoustic Data Augmentation ExampleSectionDescriptionQuick DemoHow to use this libraryAugmenterIntroduce all available augmentation methodsInstallationHow to install this libraryRecent ChangesLatest enhancementExtension ReadingMore real life examples or researchsReferenceReference of external resources such as data or modelQuick DemoQuick ExampleExample of Augmentation for Textual InputsExample of Augmentation for Multilingual Textual InputsExample of Augmentation for Spectrogram InputsExample of Augmentation for Audio InputsExample of Orchestra Multiple AugmentersExample of Showing Augmentation HistoryHow to trainTF-IDF modelHow to trainLAMBADA modelHow to createcustom augmentationAPI DocumentationAugmenterAugmenterTargetAugmenterActionDescriptionTextualCharacterKeyboardAugsubstituteSimulate keyboard distance errorTextualOcrAugsubstituteSimulate OCR engine errorTextualRandomAuginsert, substitute, swap, deleteApply augmentation randomlyTextualWordAntonymAugsubstituteSubstitute opposite meaning word according to WordNet antonymTextualContextualWordEmbsAuginsert, substituteFeeding surroundings word toBERT, DistilBERT,RoBERTaorXLNetlanguage model to find out the most suitlabe word for augmentationTextualRandomWordAugswap, crop, deleteApply augmentation randomlyTextualSpellingAugsubstituteSubstitute word according to spelling mistake dictionaryTextualSplitAugsplitSplit one word to two words randomlyTextualSynonymAugsubstituteSubstitute similar word according to WordNet/ PPDB synonymTextualTfIdfAuginsert, substituteUse TF-IDF to find out how word should be augmentedTextualWordEmbsAuginsert, substituteLeverageword2vec,GloVeorfasttextembeddings to apply augmentationTextualBackTranslationAugsubstituteLeverage two translation models for augmentationTextualReservedAugsubstituteReplace reserved wordsTextualSentenceContextualWordEmbsForSentenceAuginsertInsert sentence according toXLNet,GPT2or DistilGPT2 predictionTextualAbstSummAugsubstituteSummarize article by abstractive summarization methodTextualLambadaAugsubstituteUsing language model to generate text and then using classification model to retain high quality resultsSignalAudioCropAugdeleteDelete audio's segmentSignalLoudnessAugsubstituteAdjust audio's volumeSignalMaskAugsubstituteMask audio's segmentSignalNoiseAugsubstituteInject noiseSignalPitchAugsubstituteAdjust audio's pitchSignalShiftAugsubstituteShift time dimension forward/ backwardSignalSpeedAugsubstituteAdjust audio's speedSignalVtlpAugsubstituteChange vocal tractSignalNormalizeAugsubstituteNormalize audioSignalPolarityInverseAugsubstituteSwap positive and negative for audioSignalSpectrogramFrequencyMaskingAugsubstituteSet block of values to zero according to frequency dimensionSignalTimeMaskingAugsubstituteSet block of values to zero according to time dimensionSignalLoudnessAugsubstituteAdjust volumeFlowAugmenterAugmenterDescriptionPipelineSequentialApply list of augmentation functions sequentiallyPipelineSometimesApply some augmentation functions randomlyInstallationThe library supports python 3.5+ in linux and window platform.To install the library:pipinstallnumpyrequestsnlpaugor install the latest version (include BETA features) from github directlypipinstallnumpygit+https://github.com/makcedward/nlpaug.gitor install over condacondainstall-cmakcedwardnlpaugIf you use BackTranslationAug, ContextualWordEmbsAug, ContextualWordEmbsForSentenceAug and AbstSummAug, installing the following dependencies as wellpipinstalltorch>=1.6.0transformers>=4.11.3sentencepieceIf you use LambadaAug, installing the following dependencies as wellpipinstallsimpletransformers>=0.61.10If you use AntonymAug, SynonymAug, installing the following dependencies as wellpipinstallnltk>=3.4.5If you use WordEmbsAug (word2vec, glove or fasttext), downloading pre-trained model first and installing the following dependencies as wellfromnlpaug.util.file.downloadimportDownloadUtil
DownloadUtil.download_word2vec(dest_dir='.')# Download word2vec modelDownloadUtil.download_glove(model_name='glove.6B',dest_dir='.')# Download GloVe modelDownloadUtil.download_fasttext(model_name='wiki-news-300d-1M',dest_dir='.')# Download fasttext modelpipinstallgensim>=4.1.2If you use SynonymAug (PPDB), downloading file from the following URI. You may not able to run the augmenter if you get PPDB file from other websitehttp://paraphrase.org/#/downloadIf you use PitchAug, SpeedAug and VtlpAug, installing the following dependencies as wellpipinstalllibrosa>=0.9.1matplotlibRecent Changes1.1.11 Jul 6, 2022Return list of outputFix download utilFix lambda label misalignmentAdd language pack reference link for SynonymAugSeechangelogfor more details.Extension ReadingData Augmentation library for TextDoes your NLP model able to prevent adversarial attack?How does Data Noising Help to Improve your NLP Model?Data Augmentation library for Speech RecognitionData Augmentation library for AudioUnsupervied Data AugmentationA Visual Survey of Data Augmentation in NLPReferenceThis library uses data (e.g. capturing from internet), research (e.g. following augmenter idea), model (e.g. using pre-trained model) Seedata sourcefor more details.Citation@misc{ma2019nlpaug,
  title={NLP Augmentation},
  author={Edward Ma},
  howpublished={https://github.com/makcedward/nlpaug},
  year={2019}}This package is cited by many books, workshop and academic research papers (70+). Here are some of examples and you may visithereto get the full list.Workshops cited nlpaugS. Vajjala.NLP without a readymade labeled datasetatToronto Machine Learning Summit, 2021. 2021Book cited nlpaugS. Vajjala, B. Majumder, A. Gupta and H. Surana.Practical Natural Language Processing: A Comprehensive Guide to Building Real-World NLP Systems. 2020A. Bartoli and A. Fusiello.Computer Vision–ECCV 2020 Workshops. 2020L. Werra, L. Tunstall, and T. WolfNatural Language Processing with Transformers. 2022Research paper cited nlpaugGoogle: M. Raghu and  E. Schmidt.A Survey of Deep Learning for Scientific Discovery. 2020Sirius XM: E. Jing, K. Schneck, D. Egan and S. A. Waterman.Identifying Introductions in Podcast Episodes from Automatically Generated Transcripts. 2021Salesforce Research: B. Newman, P. K. Choubey and N. Rajani.P-adapters: Robustly Extracting Factual Information from Language Modesl with Diverse Prompts. 2021Salesforce Research: L. Xue, M. Gao, Z. Chen, C. Xiong and R. Xu.Robustness Evaluation of Transformer-based Form Field Extractors via Form Attacks. 2021Contributionssakares saengkaewBinoy DalalEmrecan Çelik