Package: textbook
The framework is designed withBERTin mind and currently support seven commonsense reasoning datsets(alphanli,hellaswag,physicaliqa,socialiqa,codah,cosmosqa, andcommonsenseqa). It can be also applied to other datasets with few line of codes.ArchitectureDependencyDownload raw datasetsUsageTemplateRendererBatchToolLoad a dataset with pandasCreate a multitask dataset with multiple datasetsImpletement a New template or renndererContactArchitectureDependencycondainstallav-cconda-forgepipinstall-rrequirements.txt
pipinstall--editable.# orpipinstalltextbookDownload raw datasets./fetch.shIt downloadsalphanli,hellaswag,physicaliqa,socialiqa,codah,cosmosqa, andcommonsenseqafrom AWS indata_cache.
In case you want to use something-something, pelase download the dataset from 20bn's website.UsageTemplateThe goal of a template is to transform raw text into a intermediate datum where abstractive information are provided for later use.Ideally, the template should do the following things:constructtext: a list of list. The outer list is ideal for multichoice situations and inner list if for each input pairs/triplets(e.g context, question, and choice);constructlabel: an integer representing a zero-indexed label for the truth, orNone;constructtoken_type_idandattention: abstractive representation of the segment id and attention. In the following example of anli, bothtoken_type_idandattentionhave three digits, each for the three components of each row of the text.constructimage: any forms of image id/path you want to read later.One example of anli is as follows:# rawcase={"story_id":"58090d3f-8a91-4c89-83ef-2b4994de9d241","obs1":"Ron started his new job as a landscaper today.","obs2":"Ron is immediately fired for insubordination.","hyp1":"Ron ignores his bosses's orders and called him an idiot.","hyp2":"Ron's boss called him an idiot.","label":"1"}# target intermediate datumtarget={'text':[['Ron started his new job as a landscaper today.',"Ron ignores his bosses's orders and called him an idiot.",'Ron is immediately fired for insubordination.'],['Ron started his new job as a landscaper today.',"Ron's boss called him an idiot.",'Ron is immediately fired for insubordination.']],'label':0,'image':None,'token_type_id':[0,1,0],'attention':[1,1,1]}LABEL2INT={"anli":{"1":0,"2":1,},}asserttemplate_anli(case,LABEL2INT['anli'])==targetRendererRenderer transformer your intermediate datum into a fully blown datum. Each renderer takes care of different part of the datum. For example,renderer_textrenders the text intoinput_idand generate all token-basedattentionandtoken_type_id, whilerenderer_videorenders theimagepath to animagetensor. renderers are passed to the dataset constructer in a list, therefore are execute sequentially.BatchToolWe provided a BatchTool where MLM or padding can be used easily, you can check the doc for the class for more information.Load a dataset with pandasfromtransformersimportBertTokenizerfromtextbookimportMultiModalDataset,template_anli,renderer_text,BatchTool,TokenBasedSamplerfromtorch.utils.dataimportDataset,DataLoaderfromtextbookimportLABEL2INTimportpandasaspdtokenizer=BertTokenizer.from_pretrained('bert-base-cased')d1=MultiModalDataset(df=pd.read_json("data_cache/alphanli/train.jsonl",lines=True),template=lambdax:template_anli(x,LABEL2INT['anli']),renderers=[lambdax:renderer_text(x,tokenizer)],)bt1=BatchTool(tokenizer,source="anli")i1=DataLoader(d1,batch_sampler=TokenBasedSampler(d1,batch_size=64),collate_fn=bt1.collate_fn)Create a multitask dataset with multiple datasetsfromtransformersimportBertTokenizerfromtextbookimport*importpandasaspdtokenizer=BertTokenizer.from_pretrained('bert-base-cased')# add additional tokens for each task as special `cls_token`tokenizer.add_special_tokens({"additional_special_tokens":["[ANLI]","[HELLASWAG]"]})d1=MultiModalDataset(df=pd.read_json("data_cache/alphanli/train.jsonl",lines=True),template=lambdax:template_anli(x,LABEL2INT['anli']),renderers=[lambdax:renderer_text(x,tokenizer,"[ANLI]")],)bt1=BatchTool(tokenizer,source="anli")i1=DataLoader(d1,batch_sampler=TokenBasedSampler(d1,batch_size=64),collate_fn=bt1.collate_fn)d2=MultiModalDataset(df=pd.read_json("data_cache/hellaswag/train.jsonl",lines=True),template=lambdax:template_hellaswag(x,LABEL2INT['hellaswag']),renderers=[lambdax:renderer_text(x,tokenizer,"[HELLASWAG]")],)bt2=BatchTool(tokenizer,source="hellaswag")i2=DataLoader(d2,batch_sampler=TokenBasedSampler(d1,batch_size=64),collate_fn=bt2.collate_fn)d=MultiTaskDataset([i1,i2],shuffle=False)#! batch size must be 1 for multitaskdataset, because we already batched in each sub dataset.forbatchinDataLoader(d,batch_size=1,collate_fn=BatchTool.uncollate_fn):pass# {#     "source": "anli" or "hellaswag",#     "labels": ...,#     "input_ids": ...,#     "attentions": ...,#     "token_type_ids": ...,#     "images": ...,# }Impletement a New template or renndererIt is advised to follow the following conventions but you can do whatever you like since you can calllambdaanywhere.deftemplate_xxx(raw_datum,*args,**kwargs):passdefrenderer_xxx(intermediate_datum,*args,**kwargs):passe.g. For Quora question pairs dataset:deftemplate_qqp(raw_datum,label2int={"0":0,"1":1},):result={"text":[[datum['question1'],datum['question2']]],"image":None,"label":Noneif'is_duplicate'notindatumordatum['is_duplicate']isNoneelselabel2int[str(datum['i_duplicate'])],"token_type_id":[0,1],"attention":[1,1],}returnresultContactAuthor: Chenghao MouEmail:mouchenghao@gmail.com