Text Mining Tweets on Religion. Topic Modeling, Sentiment Analysis &... | by Hurmet Noka | Towards Data Science

Many are those claiming that the light of scientific and technological revolution will diminish the presence of religion in human life. Yet, it is 2020, and one thing seems inevitable, religion is still here and is here to stay! That being said, it is worth investigating the public opinion towards religion in modern times.
This article presents a comprehensive guide of using Python to extract, pre-process, and analyze tweets about religion. The analysis is about implementing Topic Modeling (LDA), Sentiment Analysis (Gensim), and Hate Speech Detection (HateSonar) models. The step-by-step tutorial is presented below alongside the code and results.
Credits to Dr. Johannes Schneider and Joshua Handali MSc for their supervision during this work at University of Liechtenstein. The complete code can be found here on GitHub.
To get the tweets, we use a public python script, which enables capturing old tweets, thus bypassing the limitation of the 7-days period of Twitter API. The script is free and can be found here on GitHub. All you need to do is adjust the searching filters and run the program. For our study, we extract tweets containing the phrase “religion is.” To reduce the bias of certain isolated events affecting the feeling towards religion (i.e., Charlie Hebdo attacks), we extend the timeframe to about five years. We extracted 1000 tweets/month starting from January 2015 until October 2019. This results in about 57'351 tweets that are then loaded into a dataframe, ready for pre-processing.
Below is the whole analysis process in a schematic format
These are the necessary packages to perform pre-processing phase.
import pandas as pdimport numpy as npimport re# Plottingimport seaborn as snsimport matplotlib.pyplot as plt# Gensimimport gensimfrom gensim.utils import simple_preprocess# NLTKimport nltkfrom nltk.corpus import stopwordsfrom collections import Counterfrom wordcloud import WordCloudimport warnings warnings.filterwarnings("ignore", category=DeprecationWarning)%matplotlib inline
Data Loading
df = pd.read_csv('full_data.csv', index_col=[0])df.head()
Before any further step, we remove the duplicates as it is common in twitter that people copy paste different quotes and retweet something. After duplicate removal, we get a total of 53'939 unique tweets.
# DROP DUPLICATESdf.drop_duplicates(subset=['tweet_text'], keep='first',inplace=True)df.shapeOut: (53939, 1)
Next we generate some general descriptive statistics:
Twitter enables including usernames within tweets through the symbol “@.” These do not possess any value for our analysis; hence they are removed from the dataset using a function.
# REMOVE '@USER'def remove_users(tweet, pattern1, pattern2):    r = re.findall(pattern1, tweet)    for i in r:        tweet = re.sub(i, '', tweet)      r = re.findall(pattern2, tweet)    for i in r:        tweet = re.sub(i, '', tweet)    return tweetdf['tidy_tweet'] = np.vectorize(remove_users)(df['tweet_text'],     "@ [\w]*", "@[\w]*")
Normalization, a method which converts all tweets into lowercase, so that “Token” and “token” are not considered two different words.
# NORMALIZATIONdf['tidy_tweet'] = df['tidy_tweet'].str.lower()
Same as with usernames, hashtags also are considered not of significant value for topic modeling analysis, in particular, therefore, are removed.
Above are the top 25 hashtags of the dataset, we note that #islam is the most used hashtag, indicating it is the most discussed religion in particular. Hashtags are removed using the function:
# REMOVE HASHTAGSdef remove_hashtags(tweet, pattern1, pattern2):    r = re.findall(pattern1, tweet)    for i in r:        tweet = re.sub(i, '', tweet)        r = re.findall(pattern2, tweet)    for i in r:        tweet = re.sub(i, '', tweet)    return tweetdf['tidy_tweet'] = np.vectorize(remove_hashtags)(df['tidy_tweet'], "# [\w]*", "#[\w]*")
Next in line to be removed are URL’s:
# REMOVE LINKSdef remove_links(tweet):    tweet_no_link = re.sub(r"http\S+", "", tweet)    return tweet_no_linkdf['tidy_tweet'] = np.vectorize(remove_links)(df['tidy_tweet'])
Using the same function as with links, we also remove collection words, those words used for filtering the tweets in the first place, in this case: ‘religion,’ ‘religious.’
Afterward, it is necessary to remove numbers, punctuation (only for topic modeling), and special characters (@,&,#,%, etc.).
# REMOVE Punctuations, Numbers, and Special Charactersdf['tidy_tweet'] = df['tidy_tweet'].str.replace("[^a-zA-Z#]", " ")
Then words with less than three characters (short words) are removed from the dataset, simplifying feature extraction for the analysis.
# REMOVE SHORT WORDSdf['tidy_tweet'] = df['tidy_tweet'].apply(lambda x:' '.join([w for w in x.split() if len(w)>3]))
An essential step of pre-processing is known as Tokenization. It is the process where the text is split according to whitespaces, and every word and punctuation is saved as a separate token. We perform this step using the simple_preprocess method from Gensim.
# TOKENIZATIONdef tokenize(tweet):    for word in tweet:        yield(gensim.utils.simple_preprocess(str(word), deacc=True))  df['tidy_tweet_tokens'] = list(tokenize(df['tidy_tweet']))
Next, we remove stopwords that have no analytic value, usually articles, prepositions, or pronouns, for instance, ‘a,’ ‘and,’ ‘the,’ etc. The default list can be adjusted and extended as desired. We added some new words to the predefined list of Natural Language Toolkit (NLTK), which contains 179 words.
# Prepare Stop Wordsstop_words = stopwords.words('english')stop_words.extend(['from', 'https', 'twitter', 'religions',     'pic','twitt',])# REMOVE STOPWORDSdef remove_stopwords(tweets):    return [[word for word in simple_preprocess(str(tweet)) if word    not in stop_words] for tweet in tweets]df['tokens_no_stop'] = remove_stopwords(df['tidy_tweet_tokens'])
In the last step of pre-processing, we remove tweets with less than three tokens; this results in a total of 49'458 tweets to be considered further on the analysis phase. For topic modeling and also sentiment analysis, documents with less than three tokens are not suitable to generate enough information.
# REMOVE TWEETS LESS THAN 3 TOKENSdf['length'] = df['tokens_no_stop'].apply(len)df = df.drop(df[df['length']<3].index)df = df.drop(['length'], axis=1)df.shapedf.reset_index(drop=True, inplace=True)
Now for an illustrative purpose to identify and visualize the most used words of the dataset, we generate a WordCloud of 200 most frequent words.
The preprocessing phase effects are illustrated below, where the length of cleaned tweets is reduced considerably. As shown, most tweets have less than ten tokens after preprocessing, unlike in original tweets where most tweets have about 20 words. On the second set of graphs, the effects are more visible as the tweet length went from about 150 for most of the tweets to about 50 characters after the cleaning stage. This phase is critical as it reduces the dimensionality and results in significantly valuable tokens for the models, which is explained in the next sections.
Lastly we save the pre-processed dataframe as a pickle, which is then used for Topic Modeling phase.
df.to_pickle('pre-processed.pkl')
These are the required packages to implement LDA (Latent Dirichlet Allocation) algorithm.
# IMPORTSimport pandas as pdimport numpy as npimport networkx as nximport itertoolsimport collectionsimport spacyfrom pprint import pprint# Plottingimport matplotlib.pyplot as pltimport seaborn as snsimport pyLDAvisimport pyLDAvis.gensim# Gensimimport gensimimport gensim.corpora as corporafrom gensim.utils import simple_preprocessfrom gensim.models import CoherenceModelfrom gensim.models.wrappers import LdaMallet# NLTKfrom nltk import bigramsfrom nltk.stem import PorterStemmersns.set(font_scale=1.5)sns.set_style("whitegrid")import warnings warnings.filterwarnings("ignore", category=DeprecationWarning)%matplotlib inline
We are not going to dive into an explanation of how LDA works; details can be found in the original paper by [1]. The core idea of the LDA algorithm can be understood as a generative process where documents are defined by a probability distribution over a set of topics T, and a probability distribution of discrete words, in turn, establishes each topic. Having said that, we load the pre-processed data:
df = pd.read_pickle('pre-processed.pkl')df.head()
After data is loaded, we proceed by adding Bigrams and Trigrams. A sequence of words which often occurs together when expressing a particular meaning. A sequence of N words is known as N-Grams, as theoretically N can be of any length; the most common are pairs of words (Bi-grams) and a sequence of three words (Tri-grams). First, we need to tokenize the no_stop_joined column and convert it to a list holding the tokens for each tweet; we name this list data_words as shown below:
# TOKENIZEdef sent_to_words(sentences):    for sentence in sentences:        yield(gensim.utils.simple_preprocess(str(sentence), deacc=True))  # deacc=True removes punctuationsdata_words = list(sent_to_words(data))
Now we are ready to add bigrams and trigrams to our corpus.
# Build the bigram and trigram modelbigram = gensim.models.Phrases(data_words, min_count=10, threshold=100)trigram = gensim.models.Phrases(bigram[data_words], threshold=100)# Faster way to get a sentence clubbed as a bigrambigram_mod = gensim.models.phrases.Phraser(bigram)trigram_mod = gensim.models.phrases.Phraser(trigram)def make_bigrams(texts):    return [bigram_mod[doc] for doc in texts]def make_trigrams(texts):    return [trigram_mod[bigram_mod[doc]] for doc in texts]# Form Bigramsdata_words_bigrams = make_bigrams(data_words)
The figure below gives a visualization of bigrams in our dataset. We can notice combinations of islam-peace-peaceful, jesus-christ, science-climate-change etc.
The next step is lemmatization, an essential step for many applications of text mining. Lemmatization takes into consideration the context and converts the word to its base form; for instance, the term “hugging” is converted to “hug” and “best” to “good.” For the lemmatization task, the package used is spaCy, an open-source library with many pre-built models for natural language processing.
# LEMMATIZATIONdef lemmatization(tweets, allowed_postags=['NOUN', 'ADJ', 'VERB', 'ADV']):    """https://spacy.io/api/annotation"""    tweets_out = []    for sent in tweets:        doc = nlp(" ".join(sent))         tweets_out.append([token.lemma_ for token in doc if token.pos_ in allowed_postags])    return tweets_out# Initialize spacy 'en' model, keeping only tagger component# python3 -m spacy download ennlp = spacy.load('en', disable=['parser', 'ner'])# Lemmatization keeping only noun, adj, vb, advdf['lemmatized'] = pd.Series(lemmatization(data_words_bigrams, allowed_postags=['NOUN', 'ADJ', 'VERB', 'ADV']))
After this step, we dropped duplicates once more, as short tweets composed of very few tokens when lemmatized may lead to duplicate rows. After this step, we have 48'013 unique rows.
Another useful technique is word stemming, which is the process of transforming a word into its root form. Unlike lemmatization, which was mentioned earlier, stemming is a more aggressive approach as suffix cuttings often result in non-meaningful English words. For instance, the word “animals” would be lemmatized as “animal,” but the Porter Stemmer yields “anim.” We decided to implement both to help in dimensionality reduction.
# STEMMINGstemmer = PorterStemmer()df['stemmed'] = df['lemmatized'].apply(lambda x : [stemmer.stem(y) for y in x])
Before building the LDA model, we have to create two main inputs: the dictionary and the corpus, which are created using functions from the Gensim package.
# Create Dictionaryid2word_stemmed = corpora.Dictionary(df['stemmed'])IN: print(id2word_stemmed)OUT: Dictionary(26748 unique tokens: ['also', 'bless', 'blood', 'deed', 'fact']...)\# Create Corpustweets_stemmed = df['stemmed']IN: df['stemmed'][1]OUT: ['piti', 'antonym', 'disast', 'human', 'live', 'piti']
Here is how corpus looks like, it has a length of 48'013:
Gensim assigns a unique Id to each word, and then the corpus is represented as a tuple (word_id, word_frequency).
# Term Document Frequencycorpus_stemmed = [id2word_stemmed.doc2bow(tweet) for tweet in tweets_stemmed]
For instance, the following unprocessed tweet:
‘@ wagner_claire Religion and pity are antonyms. One is disaster, other is humanity. We can live without religion but not pity. Yet we hug ...’
After pre-processing in the corpus will be presented as a list of tuples:
[(11, 1), (12, 1), (13, 1), (14, 1), (15, 2)]
Now it is the moment that we initialize with number of topics k=10, which then will be subject of adjustments.
# Build LDA modellda_model_stemmed = gensim.models.ldamodel.LdaModel(corpus=corpus_stemmed,                                           id2word=id2word_stemmed,                                           num_topics=10,                                            random_state=100,                                           update_every=1,                                           chunksize=100,                                           passes=15,                                           alpha='auto',                                           per_word_topics=True)
After this step, we may directly generate the topics or search for the optimal model, using coherence score as a measure for each model having a different number of topics.
# OPTIMAL MODELdef compute_coherence_values(dictionary, corpus, texts, limit, start=2, step=3):        coherence_values = []    model_list = []    for num_topics in range(start, limit, step):        model = gensim.models.wrappers.LdaMallet(mallet_path, corpus=corpus_stemmed, num_topics=num_topics, id2word=id2word_stemmed)        model_list.append(model)        coherencemodel = CoherenceModel(model=model, texts=texts, dictionary=dictionary, coherence='c_v')        coherence_values.append(coherencemodel.get_coherence())return model_list, coherence_valuesmodel_list, coherence_values = compute_coherence_values(dictionary=id2word_stemmed, corpus=corpus_stemmed, texts=df['stemmed'], start=2, limit=26, step=3)
Below is illustrated the coherence score for each model generated, the optimal model we choose has k=8 topics.
After building the model and running the optimal model we yeild the following topics:
Interpretation:
Topic [0] Religion & Politics is self-evidently represented by terms like politics, control, government, nation, trump, etc., which illustrate how religion is a sensitive aspect of its role related to politics.
Topic [1] Christianity is constituted with tweets which contain Christianity as the subject of the discussions and its representative key-terms such as church, catholic, christian etc., such topic is expected for an English speaking audience mainly belonging to western world, having Christianity as the leading religion.
Topic [2] Religion & Science is concerned with the never ending debate between religion and science, a debate intensified in the modern world which is led by a technological revolution.
Topic [3] Religious Doctrine , [6] Personal Belief and [7] Diverse Opinions seem to be more closer to each other with slight differences, based on the keywords.
Topic [5] Islam is a collection of tweets with Islam as the main subject of discussion. As seen by the number of hashtags and the word-frequency statistics, Islam is highly discussed on social media for its different aspects, which trigger diverse reactions among people.
Below is given the distribution of tweets among topics, we can notice that the first three topics are more dominant:
For this part the required packages are as follows:
# IMPORTSimport pandas as pdimport numpy as npimport seaborn as snsimport matplotlib.pyplot as pltimport spacyimport refrom pprint import pprintimport nltknltk.download('vader_lexicon')from nltk.sentiment.vader import SentimentIntensityAnalyzerfrom nltk.corpus import stopwordsfrom collections import Counterfrom wordcloud import WordCloudimport warnings warnings.filterwarnings("ignore", category=DeprecationWarning)%matplotlib inline
VADER stands for Valence Aware Dictionary for sEntiment Reasoning and was developed by [2] as a rule-based model for sentiment analysis. Detailed information how this model was developed and its features can be found on original paper (see References). It takes into consideration punctuation, capitalization, degree modifiers, conjunctions, Tri-gram preceding when assigning sentiment values: negative, positive, neutral. These features make VADER sentiment analyzer achieve remarkable results when classifying social media texts like tweets and make it a suitable tool to conduct our analysis.
The pre-processing phase is slightly different from the one employed for Topic Modeling because of the features that VADER sentiment analyzer emodies, the steps taken are listed below, whereas the code is the same as in the pre-processing phase. Also you can access all the code here.
VADER MODEL
# Create an object of Vader Sentiment Analyzervader_analyzer = SentimentIntensityAnalyzer()negative = []neutral = []positive = []compound = []def sentiment_scores(df, negative, neutral, positive, compound):    for i in df['tweet_text_p']:        sentiment_dict = vader_analyzer.polarity_scores(i)        negative.append(sentiment_dict['neg'])        neutral.append(sentiment_dict['neu'])        positive.append(sentiment_dict['pos'])        compound.append(sentiment_dict['compound'])# Function calling sentiment_scores(df, negative, neutral, positive, compound)# Prepare columns to add the scores laterdf["negative"] = negativedf["neutral"] = neutraldf["positive"] = positivedf["compound"] = compound# Fill the overall sentiment with encoding:# (-1)Negative, (0)Neutral, (1)Positivesentiment = []for i in df['compound']:    if i >= 0.05 :         sentiment.append(1)      elif i <= - 0.05 :         sentiment.append(-1)             else :         sentiment.append(0)df['sentiment'] = sentiment
After applying the model to our data we have the following sentiment scores assigned to each tweet. The last column sentiment indicates the final classification (1 - positive, 0 - neutral, -1- negative).
Figure below, gives the overall classification result of tweets to the sentiment class, where 42.2% (20'512) of tweets are classified as positive. 41.1% (19'957) were classified as negative, and the remaining 16.7% (8'109) of tweets are considered neutral (neither positive or negative). The public opinion seems to be balanced in terms of the sentiments on discussions about religion on social media.
Moreover we can have a look at the negative and positive tweets using word-cloud to illustrate the dominant words for each category.
Most frequent words are as expected domain-related words such as Islam, Muslim, Christian, Christianity, whereas some of the sentiment-related words are: love, peace, support, great, respect, good, etc., which naturally imply a positive sentiment score of the tweet.
In case of negative tweets, beside domain-related words, some of the words carrying negative sentiment are: kill, hate, attack, stupid, violence, problem, evil, fuck, shit, etc..
As the opinions are shown to be almost equally split regarding positive and negative sentiment, the positive tweets beside the slight numerical advantage also have a small negative difference in terms of the compound score, as shown in the figure below. Positive tweets have a mean of 0.525122, whereas the negative tweets a mean of -0.541433.
When it comes to specific religions, most of the discussions are oriented towards Islam with the highest number of tweets, 679 tweets after pre-processing steps, followed by Christianity with 127 tweets and then the other popular religions like Hinduism, Buddhism, and Judaism.
Islam is the most discussed religion on social media. Hence, we had a closer look into the sentiments towards Islam particularly. A sentiment analysis on tweets which talk about Islam yielded the following results:
Tweets were classified using the same method (VADER), and analysis resulted in 44.2% (295) of tweets to be positive against 38.7% (258) negative tweets, while 17.0% (113) are considered neutral.
Table below illustrates the most frequent words on positive and negative tweets about Islam accordingly.
Now we present top 10 most positive tweets:
The above-shown tweets are highly rated as positive, where the 10th tweet has a compound score of 0.9810, and the 1st tweet is rated with a compound score of 0.989. The classifier has produced correct labels, as the tweets are clearly positive and do not hide underlying patters of irony or sarcasm that undermine the meaning of the sentences. It is interesting to see how out of 10, only 4 of these tweets mention the word God, whereas the word Love is mentioned 16 times on 7 out of 10 most positive tweets. One tweet is about Islam, and two are about Christianity. The rest are general opinions on religion focusing on the relationship with others, namely loving each other as the core religious principle one should follow. Most of the tweets (6/10) have a negative score of 0.000; other tweets have a very minimal above zero negative score because of the words: dogma, race, unsolicited, etc. The tweet with the highest individual positive score is the 4th with a score of 0.683 because it mentions the word peace seven times, which increases the positivity of the sentence while it does not contain negation words.
Same, here are the top 10 most negative tweets and their respective scores:
The above-shown tweets are very highly (in a negative sense) rated as negative where the 10th tweet has a compound score of -0.982, and the 1st most negative tweet is rated with a compound score of -0.988. It is evident that the classifier has produced correct labels, as the tweets are negative, expressing refusal, dislike, and negative feelings towards religion or a religious group i.e., Muslims. Some of the dominant words and their respective frequency on this table are death (8), Islam (6), rape (4), etc. The most addressed religion is Islam, where eight tweets out of 10 focus on Islam when expressing their negative emotions, while one was about Christianity and another for the Buddhism/Hinduism in India.
The model used to classify tweets which discuss about religion is a pre-trained model based on the work of [3]. This model is considered useful for our task, to investigate the presence of hate speech on tweets about religion, and to measure it. It will contribute to a general understanding of the religious landscape in an online environment.
The required packages for this task are:
# IMPORTSimport pandas as pdimport numpy as npimport seaborn as snsimport matplotlib.pyplot as pltfrom hatesonar import Sonar # This is the hate speech detection libraryimport warningswarnings.filterwarnings('ignore')
The process starts by loading the data in a dataframe format on the notebook. The data consist of about 48’500 tweets that are not pre-processed as the model is trained to process the input to extract the necessary features. Some pre-processing steps used in the model are: remove hashtags, remove links, remove mentions, tokenization, and stemming using Porter Stemmer.
SONAR Model
Now we apply this classification model to our data:
# Create an object of Sonar Hate Speech Detectionsonar = Sonar()Class = []hate = []offensive = []neither = []def hate_speech_classifier(df, Class, hate, offensive, neither): for i in df['tweet_text']:        sonar_dict = sonar.ping(text=i)        Class.append(list(sonar_dict.values())[1]) hate.append(list(list(sonar_dict.values())[2][0].values())[1]) offensive.append(list(list(sonar_dict.values())[2][1].values())[1]) neither.append(list(list(sonar_dict.values())[2][2].values())[1])# Function calling hate_speech_classifier(df, Class, hate, offensive, neither)# Prepare columns to add the scores laterdf["Class"] = Classdf["hate"] = hatedf["offensive"] = offensivedf["neither"] = neither
The resulted dataframe looks like this:
After the model is applied, the dataset consisting of 48’528 tweets is split into three categories Hate Speech, Offensive Language, and Neither (indicating neither hateful nor offensive). The first chart in figure below gives the overall results of tweets distribution across these categories. Hateful and offensive tweets combined resulted in a total of 3’802 tweets or 7.83% of the dataset. Thus, leaving the non-problematic tweets to a total of 44’726 or 92.16% of the dataset. Whereas in the second chart is shown the distribution among the so-called problematic tweets, Hate speech results in a total of 232 Tweets (6.10% of problematic tweets) as Offensive Language dominates this category with a total of 3’570 (93.89%).
It is interesting to see how the sentiment feature correlates with the hate speech classification results. Figure below shows the distribution of tweets per category of hate speech, offensive language, and neither, based on the sentiment, being negative, positive, or neutral. As expected hate speech is occurred more in negative tweets, numerically speaking 60% (145 out of 232) tweets that contain hate speech are negative. In the case of offensive tweets, the proportions are similar to hate speech, as customarily expected the majority of tweets: 58.8% (2089 out of 3570 tweets) contain negative sentiment. From the rest, 1106 or 30.98% are positive, and 375 or 10.50% of tweets classified as offensive are neutral by sentiment.
Figure below illustrates the distribution of hateful tweets based on their compound sentiment score. The left side corresponding to negative tweets is more populated (60% of tweets), which makes sense that a tweet using hate speech is negative in sentiment.
Now we present top 10 most hateful tweets:
Out of 232 tweets classified as hate speech, the above-presented tweets have the highest hate score, the 10th tweet has a hate score of 0.676, and the 1st tweet is rated with a hate score of 0.870. From a general manual analysis, we see that these tweets are dominated by a presence of offensive words like tweet number [4]. The model is trained to overcome the thin border between offensive language and hate speech by focusing on a vocabulary of hateful words. The most dominant aspect of hate is race. Eight out of 10 most hateful tweets contain a reference to white people, using words like white feminist, white racist Americans, white folks, white Nationalism, white European descent, white Men, and white boys club. Tweet number [2] expresses hate against Muslims calling their religion dated, sexist, stupid, while tweets number [7] and [8] contain references to Jews. Tweet number [6] is hate speech with a racist motive with reference to black people using the slang word niggas.
Similarly we present the most offensive tweets:
Offensive language scores dominate the ratings of the list as it varies from 96.9% to 98.9% for the first tweet. Only tweet number [10] has references to Islam, and tweet number [5] uses the word Hebrew but with no clear evidence of any reference to Judaism. Offensive tweets are populated vastly by offensive words of sexual reference such as bitch, which appears in 8 out of 10 tweets of the list. Tweets [3, 5, 6] contain strong sex-related words, while only tweets [1, 2] use the phrase “is my religion” in a figurative way.
After cleaning and pre-processing steps, the data are fetched to the LDA algorithm to perform topic modeling. This phase aimed to investigate the topics related to religion that the public is concerned with, thus investigating how the multidimensionality of religion is materialized on online discussions. Then VADER sentiment analyzer tool was used for sentiment analysis of these tweets to uncover the sentiment of the public towards religion, whether it is negative, positive, or neutral. Lastly, SONAR, a hate speech detection tool, was used to investigate and measure the presence of hate speech and offensive language on discussions about religion.
The results empirically showed that the most relevant topics of public discussion related to religion are religion & politics, religion & science, Christianity, and Islam. Sentiment analysis showed that the public is divided when it comes to their sentiment towards religion, 42.2% (20'512) of tweets are classified as positive. 41.1% (19'957) were classified as negative with the rest classified as neutral. These results can be used to argue on the level of secularization and religion affiliation that the modern society has embraced. Hate speech detection tool presented evidence that indeed there is presence of hate speech and offensive language expressed on social media with religion as a motivation. A total of 3'802 tweets or 7.83% of the dataset resulted in containing either hate speech (232 tweets) or offensive language (3'570 tweets). This shows that religion is a sphere where the public opinion is divided and may motivate hate and offensive language in between their discussions. Thus, social media platforms should make more effort to prevent such abusive behavior of their users in their online discussions.
THANK YOU
References
[1] Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003, 01). Latent Dirichlet Allocation. (J. Lafferty, Ed.) Journal of Machine Learning Research, 993–1022.
[2] Hutto, C. J., & Gilbert, E. (2014). VADER: A Parsimonious Rule-based Model for Sentiment Analysis of Social Media Text. Eighth international AAAI conference on weblogs and social media. Michigan: AAAI Press.
[3] Davidson, T., Warmsley, D., Macy, M., & Weber, I. (2017). Automated Hate Speech Detection and the Problem of Offensive Language. Eleventh International AAAI Conference on Web and Social Media (pp. 512–515). Montreal: AAAI Press.