Exploratory Data Analysis on Anime Data
By: Vinayak Nayak
Tags: Seaborn, Data Science, Data Visualization, Anime, Data Analysis

Now looking at this we can see that actually speaking, comedy is the most common genre unlike hentai which was supposedly the most common before we did the preprocessing and plotting.

Barcharts

When we have a categorical variable and a numerical variable, we can resort to using barcharts. Depending on whether the categorical column is on x-axis or on y-axis, the chart is synonymously called vertical bar-chart or horizontal bar-chart respectively.

Let’s study rating and episode(numerical variable) with respect to the type(categorical variable). This is a good use-case to study from the perspective of a barchart.

Utils.barplot(data, 'type', 'rating')

Utils.barplot(data, 'type', 'episodes')

Episodes as a function of type

Rating as a function of type

We can see that on an average, the rating is pretty much between 5 and 6 for any type of anime.

However, for TV, the number of episodes seems to be on the higher side of around 35 episodes per anime on an average. For movies, specials and OVA, it is around 1 or 2 and for ONA, it’s around 5 episodes per anime respectively.

Piecharts

To visualize the distribution of counts of a categorical variable, we can use piechart as an alternative to countplot to better understand the relative proportions in a percentage-wise manner. Let’s see the pies of types and genres.

from collections import Counter

typedata = Counter(data.type) all_genres = []

for item in data.genre:

item = item.strip().split(', ')

all_genres.extend(item) genredata = Counter(all_genres) Utils.pieplot(typedata, "type")

Utils.pieplot(genredata, "genre")

Image by Vinayak

Image by Vinayak

Looking at the charts above, we can see that when the number of levels in a category is high, the pie-chart is not a good way to visualize the data because the data cannot be aesthetically and semantically analyzed well. Going for the traditional countplot is a good idea then.

However, for a relatively small number of levels, it is quite effective in communicating a percentage split which colloquially strikes close to the understanding of many folks. We can see that TV, OVA and Movies are predominant in the anime industry whereas Specials, ONA and music anime are relatively less in number.

Boxplot and Violinplots

Boxplot

A histogram is a good way to study the distribution of data however it’s sometimes extraneous and most of histogram’s information could be conveyed more concisely with another plot called boxplot or a box & whisker plot. It is a compressed version of histogram where you come to know about the median and quartiles and you can use the same for flagging potential outliers. A detailed explanation of how to use the boxplot can be found in this article.

Basically we’re interested in finding out quartiles and confirming that most of the data lies within the IQR and the whiskers. Those that lie outside these range are potential outliers, they need to be investigated and if found alright, they will be considered in future analysis, else these records are blocked.

Interpretation of a boxplot

So, let’s visualize the boxplot of ratings and see how it varies with the type variable as well.

Image by Vinayak

We can see that there are a substantial amount of outliers especially on the lower side for TV, OVA and specials. This calls for attention at those points which are far outside the normal rating margins. Why were they rated the way they were? Were the ratings forged, did some negative publicity impact these set of anime, were they rated by enough number of people or was the rating obtained only on the basis of 1 or 2 reviews for those anime?

This plot thus helps us identify the records which could be anomalous and we need to further dig in to make sure those anomalies are genuine and make a call on whether to keep those for our analysis or not.

One more thing to note is that sometimes the range of the y-variable can vary exponentially. In such cases, taking a log transform before plotting the boxplot could really help make things more interpretable.

fig, ax = plt.subplots(1, 2, figsize = (15, 6)) Utils.boxplot(data, "type", "members", ax[0], log = False)

Utils.boxplot(data, "type", "members", ax[1], log = True) plt.savefig('./img/boxplot_members_type')

plt.close() fig, ax = plt.subplots(1, 2, figsize = (15, 6)) Utils.boxplot(data, "type", "episodes", ax[0], log = False)

Utils.boxplot(data, "type", "episodes", ax[1], log = True) plt.savefig('./img/boxplot_episodes_type')

plt.close()

Image by Vinayak

Image by Vinayak

We can see that for the plain figure, it’s very difficult to interpret the boxplot since most of the data is concentrated in such a short interval and the spread of data is huge. In order to mitigate this, we can take a log transform and then study the distribution.

We can see that with respect to members, there’s a few potential outliers in OVA and music and when it comes to episodes, there’s several issues with most of the categories.

A movie must have only 1 episode, any more calls for suspicion. The graph says there are some such datapoints. Probably it was mislabelled Movie when it should’ve been TV or vice versa. Let’s find out.

Image by Vinayak

Although I haven’t personally seen all these, I have seen Byousoku 5 centimeter. It is a movie with 3 short stories. That is probably why the number of episodes are 3 in this case which in my opinion is considerable.

However, Goman-hiki and Kami Usagi Rope are controversial with 100 and 14 episodes respectively for a movie premiere. They are questionably TV series but since they have an underlying story and are very short episodes, collectively they could be clubbed as a movie.

Now, it’s the discretion of the data scientist here whether to consider these records as movies, or series or to drop them completely. No approach is wrong it’s just the thought process of an individual and his way of reasoning to justify the exclusion/inclusion/alteration of records that count here.

Violinplot

Imagine the best of histogram and boxplot coming together — that’s what violinplots are. The kernel density estimate that we saw in histograms is overlaid on top of the box and whiskers plot. Not only this gives us the median, IQR and whiskers but also an overview of the nature of distribution all in one single plot.

Although it’s so information dense, the aesthetics and a lack of simplicity are what limit their use in the real world; however, for presenting information to people who’re aware of this plot and what it represents, I personally can’t think of a better alternative than a violinplot.

Let’s make a violinplot for the categories for ratings.

Image by Vinayak

We can observe clearly the spread and the potential outliers. Although movies have most dispersed ratings, they don’t have a lot of outliers whereas in OVA, we can see that a substantial portion of data extends beyond the whiskers which suggest presence of several outliers.

fig, ax = plt.subplots(1, 2, figsize = (15, 6)) Utils.violinplot(data, "type", "members", ax[0], log = False)

Utils.violinplot(data, "type", "members", ax[1], log = True) plt.savefig('./img/violinplot_members_type')

plt.close() fig, ax = plt.subplots(1, 2, figsize = (15, 6)) Utils.violinplot(data, "type", "episodes", ax[0], log = False)

Utils.violinplot(data, "type", "episodes", ax[1], log = True) plt.savefig('./img/violinplot_episodes_type')

plt.close()

Image by Vinayak

Image by Vinayak

From the violinplot of members, we can see in addition to most of the data being contained within IQR, the concentration of data is not only around the mean but dispersed quite a bit across the range.

On the other hand, when we compare the violin plot of episodes wrt type, most of the types have ridiculously some records with ridiculously large number of episodes for that particular type of anime (as we saw above from the box plot).

Scatterplots

When we need to study the variation between two continuous variables, we resort to using scatterplots. They help us understand the correlation between these two variables i.e. if one variable increases, does the other increase, decrease or stay unaffected.

They serve a good visual check to gauge the performance of linear regression model or rather set expectations for the performance of a regression model before actually building one. Seaborn provides a good functionality of overlaying a regression line on top of the scatterplot to address this point.

Let’s see the correlation of ratings as a function of members, presumably I expect that they should be directly proportional since highly rated anime will have more members following the anime and vice versa.

Utils.scatterplot(data, "members", "rating")

Image by Vinayak

As we can see the bulk of datapoints concentrated in 0 to 400000 members show a positive relationship i.e. increase in one causes increase in another.

Keep in mind that the y axis, in reality is constrained in between 0 to 10. There can’t be a score greater than 10, this is why although a lot of people love some anime, they will be forced to give a rating not more than 10, even if they liked it a lot. In absence of this constraint, I predict that the line and the points in higher 100000s of members would have been still more close to one another.

Let’s do one more, let’s see the relationship between number of episodes and the rating of a show.

Utils.scatterplot(data, "episodes", "rating")

Image by Vinayak

It does seem that there’s a positive correlation between episodes and rating but it’s not as strong as observed between members and ratings. Also, the 95% CI band is wider for this model which means, the regression line isn’t as good a fit as the one between members and ratings.

This is how we can use scatterplots to gain an understanding of how two quantities vary with respect to one another.

Heatmap

Another useful visual is that of a heatmap. When we have two categorical variables and a numerical measure associated with each pair of those two categorical variables, heatmaps are a very good tool to visualise the same. We could see how many anime belong to a particular genre and a particular type and build a heatmap of the counts of same or a heatmap of the ratings of the same and show it aside as a colour scale.

Let’s first look at making a heatmap of counts of type and genre of anime to get a feel for the heatmaps.