Package: pysentimiento
pysentimiento: A Python toolkit for Sentiment Analysis and Social NLP tasksA Transformer-based library for SocialNLP tasks.Currently supports:TaskLanguagesSentiment Analysises, en, it, ptHate Speech Detectiones, en, it, ptIrony Detectiones, en, it, ptEmotion Analysises, en, it, ptNER & POS tagginges, enContextualized Hate Speech DetectionesTargeted Sentiment AnalysisesJust dopip install pysentimientoand start using it:Getting Startedfrompysentimientoimportcreate_analyzeranalyzer=create_analyzer(task="sentiment",lang="es")analyzer.predict("Qué gran jugador es Messi")# returns AnalyzerOutput(output=POS, probas={POS: 0.998, NEG: 0.002, NEU: 0.000})analyzer.predict("Esto es pésimo")# returns AnalyzerOutput(output=NEG, probas={NEG: 0.999, POS: 0.001, NEU: 0.000})analyzer.predict("Qué es esto?")# returns AnalyzerOutput(output=NEU, probas={NEU: 0.993, NEG: 0.005, POS: 0.002})analyzer.predict("jejeje no te creo mucho")# AnalyzerOutput(output=NEG, probas={NEG: 0.587, NEU: 0.408, POS: 0.005})"""Emotion Analysis in English"""emotion_analyzer=create_analyzer(task="emotion",lang="en")emotion_analyzer.predict("yayyy")# returns AnalyzerOutput(output=joy, probas={joy: 0.723, others: 0.198, surprise: 0.038, disgust: 0.011, sadness: 0.011, fear: 0.010, anger: 0.009})emotion_analyzer.predict("fuck off")# returns AnalyzerOutput(output=anger, probas={anger: 0.798, surprise: 0.055, fear: 0.040, disgust: 0.036, joy: 0.028, others: 0.023, sadness: 0.019})"""Hate Speech (misogyny & racism)"""hate_speech_analyzer=create_analyzer(task="hate_speech",lang="es")hate_speech_analyzer.predict("Esto es una mierda pero no es odio")# returns AnalyzerOutput(output=[], probas={hateful: 0.022, targeted: 0.009, aggressive: 0.018})hate_speech_analyzer.predict("Esto es odio porque los inmigrantes deben ser aniquilados")# returns AnalyzerOutput(output=['hateful'], probas={hateful: 0.835, targeted: 0.008, aggressive: 0.476})hate_speech_analyzer.predict("Vaya guarra barata y de poca monta es XXXX!")# returns AnalyzerOutput(output=['hateful', 'targeted', 'aggressive'], probas={hateful: 0.987, targeted: 0.978, aggressive: 0.969})SeeTASKSfor more details on the supported tasks and languages, and also for reported performance for each benchmarked model.Also, check these notebooks with examples of how to usepysentimientofor each language:Spanish + EnglishItalianPortugueseContextualized Hate Speech - SpanishPreprocessingpysentimientofeatures a tweet preprocessor specially suited for tweet classification with transformer-based models.frompysentimiento.preprocessingimportpreprocess_tweet# Replaces user handles and URLs by special tokenspreprocess_tweet("@perezjotaeme debería cambiar esto http://bit.ly/sarasa")# "@usuario debería cambiar esto url"# Shortens repeated characterspreprocess_tweet("no entiendo naaaaaaaadaaaaaaaa",shorten=2)# "no entiendo naadaa"# Normalizes laughterspreprocess_tweet("jajajajaajjajaajajaja no lo puedo creer ajajaj")# "jaja no lo puedo creer jaja"# Handles hashtagspreprocess_tweet("esto es #UnaGenialidad")# "esto es una genialidad"# Handles emojispreprocess_tweet("🎉🎉",lang="en")# 'emoji party popper emoji emoji party popper emoji'Instructions for developersClone and installgit clone https://github.com/pysentimiento/pysentimiento
pip install poetry
poetry shell
poetry installRun script to train modelsCheckTRAIN.mdfor further information on how to train your modelsNote: you need access to the datasets, which are not public for the time being. Send us an email to get access to them.Upload models to Huggingface's Model HubCheck"Model sharing and upload"instructions inhuggingfacedocs.Licensepysentimientois an open-source library. However, please be aware that models are trained with third-party datasets and are subject to their respective licenses, many of which are for non-commercial useTASS Dataset license(License for Sentiment Analysis in Spanish, Emotion Analysis in Spanish & English)SEMEval 2017 Dataset license(Sentiment Analysis in English)LinCE Datasets(License for NER & POS tagging)Suggestions and bugfixesPlease use the repositoryissue trackerto point out bugs and make suggestions (new models, use another datasets, some other languages, etc)CitationIf you usepysentimientoin your work, please citethis paper@misc{perez2021pysentimiento,title={pysentimiento: A Python Toolkit for Sentiment Analysis and SocialNLP tasks},author={Juan Manuel Pérez and Juan Carlos Giudici and Franco Luque},year={2021},eprint={2106.09462},archivePrefix={arXiv},primaryClass={cs.CL}}Also, pleace cite related pre-trained models and datasets for the specific models you use. CheckREFERENCESfor details.