Package: ciderpolarity
CIDER Python PackagePreprint Available:https://arxiv.org/abs/2307.07864CIDER (Context Informed Dictionary and sEntiment Reasoner) is a Python library used to improve domain-specific sentiment analysis.It generates, filters, and substitutes polarities intoVADER.
The approach taken to generate polarities is taken fromSocialSent.ContentsInstallationOverviewExamplesAlternative ScalesInstallationBefore you begin, ensure you have met the following requirements:You have installed Python 3.7 or later.You have a Windows/Linux/Mac machine.To install CIDER, follow these steps:pipinstallciderpolarityOverviewThe easiest way to use the package is as follows:fromciderpolarityimportCIDER# download test data from: https://github.com/jcy204/ciderPolarity/tree/main/tests/test_data.csv# data input is either a one column csv file where each row is a text entry, or list of stringsinput_data='test_data.csv'output_folder='/path/to/output/folder/'cdr=CIDER(input_data,output_folder)results=cdr.fit_transform()This trains the model, creating a customised VADER classifier, before classifying the provided input using the model. A ficticious example output is as follows:results=[['Really hate this heat. Just want AC',{"neg":0.6,"neu":0.4,"pos":0.0,"compound":-0.6}],['I love an icecream in this heat!',{"neg":0.0,"neu":0.5,"pos":0.5,"compound":0.6}],['I’m melting - terrible weather!',{"neg":0.7,"neu":0.3,"pos":0.0,"compound":-0.7}],['Very dehydrated in this heat',{"neg":0.5,"neu":0.4,"pos":0.0,"compound":-0.5}],...['this sunny weather is great',{"neg":0.0,"neu":0.2,"pos":0.8,"compound":0.7}],['Oh my icecream is melting',{"neg":0.3,"neu":0.4,"pos":0.3,"compound":0.0}],['My AC is broken! 🥵',{"neg":0.6,"neu":0.4,"pos":0.0,"compound":-0.6}]]ExamplesSome alternative ways to use the library are as follows:Applying CIDER to a list of strings, adding custom seed words, custom stopwords, and tuning various parameters:POS_seeds={'lovely':1,'excellent':2,'fortunate':4,'excited':1,'loves':2,'♥':1,'🙂':2}NEG_seeds={'bad':1,'horrible':2,'hate':4,'crappy':1,'sad':2,'bitch':1,'hates':2}input_data=['list of strings']output='/path/to/output/test_outputs/'cdr_example=CIDER(input_data,# input dataoutput,# output pathiterations=100,# number of iterations for bootstrapped label propagationstopwords=['i','it','the'],# custom stopwords, alternativly set as 'default' for the nltk setkeep=['code','python'],# words to force into the final lexiconno_below=5,# exclude words that occur fewer times than thismax_polarities_returned=3000,# maximum number of words returnedpos_seeds=POS_seeds,# positive seeds with custom weightingneg_seeds=NEG_seeds,# negative seeds with custom weightingverbose=False)# whether to print progress or notIf the model only requires training, the following can be executed:cdr_example.fit()And the resulting polarities (before filtering and scaling) can be viewed:Generating SeedwordsWhilst CIDER has built in seed words (foundhere), custom seed words can be generated and suggested. The following shows how this is carried out:Pos,Neg=cdr_example.generate_seeds(['good','brilliant','love'],['bad','terrible','hate'],n=20,sentiment=True)Which looks at strongly polarised words which occur both often, are close to one seed set, and distant from the opposing seed set.The following returns all words in the data, alongside their seed word suitability.df=cdr_example.generate_seeds(['good','brilliant','love'],['bad','terrible','hate'],return_all=True,sentiment=True)Alternative ScalesCIDER is not limited to sentiment. By initiating the model with alternative sets of seed words, non-intuitive linguistic scales can be produced. For instance:fromciderpolarityimportCIDERinput_data='test_data.csv'output_folder='/path/to/output/folder/'cdr=CIDER(input_data,output_folder,predefined_seeds='gender')cdr.fit()The above creates a linguistic scale based off of proximity to gendered seed words (i.e. 'he', 'him', 'brother' and 'she', 'her', 'sister').
Below shows a sample output for this scale when applied to all of the tweets from the UK in 2020.