An NLP Analysis: What is Happening About the Presidents on Twitter?
By: Kurt F.
Tags: Politics, Twitter, Data Visualization, Data Science, NLP

Data Exploration (EDA Process)

1- Visualizing numeric data (‘length’) as frequency:

2- To see how many tweets have been posted in which language:

3- Min/max tweets:

To view the longest and shortest tweets, we want to see the max and min length values using describe ():

min:7 & max=459 character

min = ‘Merkel!’

max = ‘Merkel: Germany’s …

4-Dividing data into clusters, positively and negatively:

This division process will have optimized the data for future studies. We want to have a ready-made data set when we want to view WordCloud or positive / negative comments separately. So let’s first divide them:

Positive dat set: top 5 tweets

Negative dat set: top 5 tweets

5- Positive and negative as a string:

Next, we will try to combine negative and positive sentences into a single statement. For this we will first use tolist () and then join ().

Negative sentences as one string

Positive sentences as one string

6-WordCloud

Projecting a multi-language data set in WordCloud may not seem meaningful. We will add new words to the words in StopWords to make this a little bit better. For example, when we call stopwords for german:

Now let’s do the additions mentioned. Since the most English tweets are posted after German tweets, let’s add the stopwords in English. There may be different ways to do this. I will prefer to display the stopwords for english first and then update my stopwords list by adding these words. In that case:

And now I select all of them and update them with the update () function. However, for stopwords words like “why’s” that have a single quotation mark in the word again, I have to change all quotes to be binary. Otherwise we will get an error.

Thus, the number of words in total stopwords increased from 232 to 404.

Apart from the unnecessary words, Merkel is already mentioned in every tweet. And it doesn’t make much sense to actually see this in WordCloud. So we added “Angela, Frau and Merkel” to this group of words as above. Now we can focus on other phrases.

Negative WordCloud

If we wonder how many times some words occur in this WordCloud, we can use a code like the one below. For example, I would like to learn how many times the words ‘Adolf’, ‘Hitler’ and ‘Nazi’ are used in tweets about Merkel according to our data set:

Next, we will consider displaying positive tweets in a WordCloud. But here, unlike the above, we will make a Merkel’s mask appear in it. For this, the np.array() values of the image are checked first. And we look at whether the values need to be optimized or not.

In this case, it must be ensured that 0 values are 250 feet. We will use transform for this. The function we will use to transform our mask should be:

Congratulations! Now we can show WordCloud in the image.

wc = WordCloud(stopwords=stopwords, width=3000, height=2000, background_color="black", max_words=1000, mask=transformed_merkel_mask,

contour_width=3, contour_color='firebrick').generate(positive_sentences_as_one_string) plt.figure(figsize=(15,10))

plt.imshow(wc)

7- Language Group

In pandas, we can evaluate together one or more data we select between columns by grouping method. Now let’s apply this to the language.

What we want to visualize now is to better understand the average length of negative tweets posted in which language. So let’s see the mean of the group with mean ():

We have not applied any exceptions here. For example, in some languages, only 1 posted tweet is included in this data set. Therefore, graphical values of “bosnian, catalan, crotian, czech, slovenian, finnish” are not very determinant since a single tweet determines the average.

To visualize the most positive / negative tweets sent according to the languages:

plot the top 5 languages for “positive”

plot the top 5 languages for “negative”

8- Issue genereal distrubition

We will now observe the results of the labeling among the negative tweets by subject. First of all, if we consider negative tweets categorically:

Now let’s look at what percentages of tweets labeled “conspiracy theory — insult — political criticism”, by language. Getting all the code here will be a bit confusing. So you can reach this by following the link for the code I will share at the end of the article. If we look at the graph of the percentage stack for the negative issue:

Here we have taken 6 languages that have more data. Because this way we can have a more balanced average.

Let’s examine in which languages the positive content tweets are more intense with ‘Do Not Plot’ According to this:

Thus, we completed the graphical evaluation process according to the subjects.

Now NLP is coming …