Package: senda
sendasendais a small python package for fine-tuning transformers for
sentiment analysis (and text classification tasks in general).sendabuilds on the excellenttransformers.TrainerAPI (all credit goes to theHuggingfaceteam).Installation guidesendacan be installed fromPyPIwithpip install sendaIf you want the development version then install directly fromGitHub.How to useYou can usesendato fine-tuneanytransformer foranytext classification task inanylanguage.Here we will go through how to usesendafor fine-tuning a transformer for detecting the polarity ('positive', 'neutral' or 'negative')
of Danish Tweets. For training we use more than 5,000 Danish Tweets kindly annotated
and hosted by theAlexandra Institute(thanks!).First, load Danish Tweets annotated with polarity.fromsendaimportget_danish_tweetsdf_train,df_eval,df_test=get_danish_tweets()Note, that the datasets must be DataFrames containing the columns 'text' and 'label', e.g.df_traintextlabelCepos:Vibørdiskutere,hvordanviafviklerj...neutralAvis:FCKøbenhavnogBrøndbyIFiduelomSte...neutral@PeterThorup@IntactDenmarkNej-endnuikke-...positivThatwasprettyclose.TheresaMayfortsætter...neutralSåerdernyFacebook-sidetilminnyeforretn...positiv......@MtnTeit@aeldresagenHeltenig.Vimåbareik...negativ@PrmMortensen@Marchen_Neel@larsloekke@oeste...negativHvordansikrerviØKONOMISKRENTABLEREGIONALE...neutral@JanEJoergensen@24syv@DanskDf1995@Spolitik...negativ@Fonoudi6erenIkkeenig!Synesvivargodtmed...positivNext, instantiate and set up the model.fromsendaimportModel,compute_metricsfromtransformersimportEarlyStoppingCallbackm=Model(train_dataset=df_train,eval_dataset=df_eval,transformer="Maltehb/danish-bert-botxo",labels=['negativ','neutral','positiv'],tokenize_args={'padding':True,'truncation':True,'max_length':512},training_args={"output_dir":'./results',"num_train_epochs":4,"per_device_train_batch_size":8,"evaluation_strategy":"steps","eval_steps":100,"logging_steps":100,"learning_rate":2e-05,"weight_decay":0.01,"per_device_eval_batch_size":32,"warmup_steps":100,"seed":42,"load_best_model_at_end":True,},trainer_args={'compute_metrics':compute_metrics,'callbacks':[EarlyStoppingCallback(early_stopping_patience=4)],})Now, all there is left is to initialize the model (including thetransformers.Trainer) and train it:# initialize Trainerm.init()# run trainingm.train()The model can then be evaluated on the test set:m.evaluate(df_test){'eval_loss':0.5771588683128357,'eval_accuracy':0.7664399092970522,'eval_f1':0.7290485787279956,'eval_runtime':4.2016,'eval_samples_per_second':104.959}Predict new observations:text="Sikke en dejlig dag det er i dag"# in English: 'What a lovely day'm.predict(text)PredictionOutput(predictions=array([[-1.2986785,-0.31318122,1.2002046]],dtype=float32),label_ids=array([0]),metrics={'test_loss':2.7630457878112793,'test_accuracy':0.0,'test_f1':0.0,'test_runtime':0.07,'test_samples_per_second':14.281})m.predict(text,return_labels=True)['positiv']sendamodel available on HuggingfaceAs you see, the model above achieves an accuracy of 0.77 and a macro-averaged F1-score of 0.73 on a small test data set, thatAlexandra Instituteprovides.The model is published onHuggingface.Here is how to download and use the model with PyTorch:fromtransformersimportAutoTokenizer,AutoModelForSequenceClassification,pipelinetokenizer=AutoTokenizer.from_pretrained("pin/senda")model=AutoModelForSequenceClassification.from_pretrained("pin/senda")# create 'senda' sentiment analysis pipelinesenda_pipeline=pipeline('sentiment-analysis',model=model,tokenizer=tokenizer)senda_pipeline("Sikke en dejlig dag det er i dag")[{'label':'positiv','score':0.7678486704826355}]The model can most certainly be improved, and we encourage all NLP-enthusiasts to train a better model - you can use thesendapackage to do this.Backgroundsendais developed as a part ofEkstra Bladet’s activities on Platform Intelligence in News (PIN). PIN is an industrial research project that is carried out in collaboration between theTechnical University of Denmark,University of CopenhagenandCopenhagen Business Schoolwith funding fromInnovation Fund Denmark. The project runs from 2020-2023 and develops recommender systems and natural language processing systems geared for news publishing, some of which are open sourced likesenda.Shout-outsThanks toAlexandra Institutefor doing all of the heavy lifting by annotatingDanish tweets(and publishing them).ContactWe hope, that you will findsendauseful.Please direct any questions and feedbacks tous!If you want to contribute (which we encourage you to), open aPR.If you encounter a bug or want to suggest an enhancement, pleaseopen an issue.