Package: arabica
ArabicaPython package for exploratory text data analysisText data is often recorded as a time series with significant variability over time. Some examples of time-series text data include social media conversations, product reviews, research metadata, central bankersâ€™ communication, and newspaper headlines. Arabica makes exploratory analysis of these datasets simple by providing:Descriptive n-gram analysis: n-gram frequenciesTime-series n-gram analysis: n-gram frequencies over a periodText visualization: n-gram heatmap, line plot, word cloudSentiment analysis: VADER sentiment classifierFinancial sentiment analysis: with FinVADERStructural breaks identification: Jenks Optimization MethodIt automatically cleans data from punctuation on input. It can also apply all or a selected combination of the following cleaning operations:Remove digits from the textRemove the standard list(s) of stopwordsRemove an additional list of stop wordsArabica works withtextsof languages based on the Latin alphabet, usescleantextfor punctuation cleaning, and enables stop words removal for languages in theNLTKcorpus of stopwords.It reads dates in:US-style:MM/DD/YYYY(2013-12-31, Feb-09-2009, 2013-12-31 11:46:17, etc.)European-style:DD/MM/YYYY(2013-31-12, 09-Feb-2009, 2013-31-12 11:46:17, etc.) date and datetime formats.InstallationArabica requiresNLTK- stop words removal,cleantext- text cleaning,wordcloud- word cloud visualization,plotnine- heatmaps and line graphs,matplotlib- word clouds and graphical operations,vaderSentiment- sentiment analysis,finvader- financial sentiment analysis,
andjenskpyfor breakpoint identification.To install using pip, use:pip install arabicaUsageImport the library:fromarabicaimportarabica_freqfromarabicaimportcappuccinofromarabicaimportcoffee_breakChoose a method:arabica_freqenables a specific set of cleaning operations (lower casing, numbers, common stop words, and additional stop words
removal) and returns a dataframe with aggregated unigrams, bigrams, and trigrams frequencies over a period.defarabica_freq(text:str,# Texttime:str,# Timedate_format:str,# Date format: 'eur' - European, 'us' - Americantime_freq:str,# Aggregation period: 'Y'/'M'/'D', if no aggregation: 'ungroup'max_words:int,# Maximum of most frequent n-grams displayed for each periodstopwords:[],# Languages for stop wordsskip:[],# Remove additional stop wordsnumbers:bool=False,# Remove numberslower_case:bool=False# Lowercase text)cappuccinoenables cleaning operations (lower casing, numbers, common stop words, and additional stop words
removal) and provides plots for descriptive (word cloud) and time-series (heatmap, line plot) visualization.defcappuccino(text:str,# Texttime:str,# Timedate_format:str,# Date format: 'eur' - European, 'us' - Americanplot:str,# Chart type: 'wordcloud'/'heatmap'/'line'ngram:int,# N-gram size, 1 = unigram, 2 = bigram, 3 = trigramtime_freq:str,# Aggregation period: 'Y'/'M', if no aggregation: 'ungroup'max_words:int,# Maximum of most frequent n-grams displayed for each periodstopwords:[],# Languages for stop wordsskip:[],# Remove additional stop wordsnumbers:bool=False,# Remove numberslower_case:bool=False# Lowercase textcoffee_breakprovides sentiment analysis and breakpoint identification in aggregated time series of sentiment. The implemented models are:VADERis a lexicon and rule-based sentiment classifier attuned explicitly to general language expressed in social mediaFinVADERimproves VADER's classification accuracy on financial texts, including two financial lexiconsBreak points in the time series are identified with theFisher-Jenks algorithm(Jenks, 1977. Optimal data classification for choropleth maps).defcoffee_break(text:str,# Texttime:str,# Timedate_format:str,# Date format: 'eur' - European, 'us' - Americanmodel:str,# Sentiment classifier, 'vader' - general language, 'finvader' - financial textskip:[],# Remove additional stop wordspreprocess:bool=False,# Clean data from numbers and punctuationtime_freq:str,# Aggregation period: 'Y'/'M'n_breaks:int# Number of breakpoints: min. 2)Documentation, examples and tutorialsRead thedocumentationFor more examples of coding, read these  tutorials:General use:Sentiment Analysis and Structural Breaks in Time-Series Text DatahereVisualization Module in Arabica Speeds Up Text Data ExplorationhereText as Time Series: Arabica 1.0 Brings New Features for Exploratory Text Data AnalysishereApplications:Business Intelligence:Customer Satisfaction Measurement with N-gram and Sentiment AnalysishereResearch meta-data analysis:Research Article Meta-data Description Made Quick and EasyhereTwitter tweets analysisPlease visitherefor any questions, issues, bugs, and suggestions.