Package: that-nlp-library
Welcome to that-nlp-libraryInstallpipinstallthat_nlp_libraryIt is advised that you manually install torch (with your compatible cuda
version if you GPU). Typically it’spip3installtorch--index-urlhttps://download.pytorch.org/whl/cu118VisitPytorch pagefor more informationHigh-Level OverviewSupervised LearningFor supervised learning, the main pipeline contains 2 parts:Text Data Controller:TextDataController(for text processing)Here is a list of processings that you can use (in order). You also can
skip any processing if you want to.Here is an example of the Text Controller for a classification task
(predictDivision Name), without any text preprocessing. The code will
also tokenize your text field.tdc=TextDataController.from_csv('sample_data/Womens_Clothing_Reviews.csv',main_text='Review Text',label_names='Division Name',sup_types='classification',)tokenizer=RobertaTokenizer.from_pretrained('roberta-base')tdc.process_and_tokenize(tokenizer,max_length=100,shuffle_trn=True)And here is an example when all processings are appliedfromundertheseaimporttext_normalizeimportnlpaug.augmenter.charasnac# define the augmentation functiondefnlp_aug(x,aug=None):results=aug.augment(x)ifnotisinstance(x,list):returnresults[0]returnresultsaug=nac.KeyboardAug(aug_char_max=3,aug_char_p=0.1,aug_word_p=0.07)nearby_aug_func=partial(nlp_aug,aug=aug)# initialize the TextDataControllertdc=TextDataController.from_csv(dset,main_text='Review Text',# metadatasmetadatas='Title',# labellabel_names='Division Name',sup_types='classification',label_tfm_dict={'Division Name':lambdax:xifx!='Initmates'else'Intimates'},# row filterfilter_dict={'Review Text':lambdax:xisnotNone,'Division Name':lambdax:xisnotNone,},# text transformationcontent_transformation=[text_normalize,str.lower],# validation splitval_ratio=0.2,stratify_cols=['Division Name'],# upsamplingupsampling_list=[('Division Name',lambdax:x=='Intimates')]# text augmentationcontent_augmentations=nearby_aug_func)tokenizer=RobertaTokenizer.from_pretrained('roberta-base')tdc.process_and_tokenize(tokenizer,max_length=100,shuffle_trn=True)For an in-depth tutorial on Text Controller for Supervised Learning
(TextDataController),
please visithereThis library also astreamed version of Text Controller(TextDataControllerStreaming),
allowing you to work with data without having it entirely on your hard
drive. You can still perform all the processings in the non-streamed
version, except forTrain/Validation split(which means you have to
define your validation set beforehand), andUpsampling.For more details onstreaming, visithere.If you are curious on the time and space efficiency between streamed and
non-streamed version, visit the benchmarkhereModel andModelControllerThe library can perform the following:Classification (simple
tutorial)RegressionMultilabel
classificationMultiheads,
where each head can be either classification or regression“Multihead” is when your model needs to predict multiple outputs at
once, for example, given a sentence (e.g. a review on an e-commerce
site), you have to predict what category the sentence is about, and
the sentiment of the sentence, and maybe the rating of the sentence.For the above example, this is a 3-head problem: classification (for
category), classification (for sentiment), and regression (for
rating from 1 to 5)For 2-head classification where there’s hierarchical relationship
between the first output and the second output (e.g. the first output
is level 1 clothing category, and the second output is the level 2
clothing subcategory), you can utilize two specific approaches for
this use-case: training withconditional
probability,
or withdeep hierarchical
classificationDecoupling of Text Controller and Model ControllerIn this library, you can either useTextDataControlleronly to handle all the text processings, and have the final
processed-HuggingFace-DatasetDict returned to you. But if you have your
own processed DatasetDict, you can skip the text controller and use only
theModelControllerfor training your data. There’s a quick tutorial on this decouplinghereLanguage ModelingFor language modeling, the main pipeline also contains 2 partsText Data Controlelr for Language Model:TextDataLMControllerSimilarly toTextDatController,TextDataLMControlleralso provide a list of processings (except forLabel Processing,UpsamplingandText Augmentation). The controller also allow
tokenization line-by-line or by token concatenation. Visit the tutorialhereThere’s also a streamed version
(TextDataLMControllerStreaming)Language Model Controller:ModelLMControllerThe library can train amasked language
modeling(BERT, roBERTa …) or acausal language
model(GPT) either from scratch or from existing pretrained language models.Hidden States ExtractionThe library also allow you toextract the hidden
statesof your choice, for further analysisDocumentationVisithttps://anhquan0412.github.io/that-nlp-library/