Data Science Approach
By: Nash Ali
Tags: Arrow, DC Comics, TV Series, Data Science, Programming

Stephen Amell as The Green Arrow (Image Credit: CW)

Since its inception in 2012, CW’s Arrow has become one of the most popular and revolutionary television series in the world. Arrow kickstarted CW’s DC Universe, dubbed as Arrowverse, that shares story continuity with The Flash, Supergirl, Legends of Tomorrow, Batwoman, and Vixen.

Based on one of the most popular characters in DC Comics, the show has revolutionized modern television superhero series with its faithfulness to its source materials, grounded action sequences, blended timelines, combining neo-noir with superhero fiction, and drawing inspiration from Christopher Nolan’s The Dark Knight trilogy. Arrow has made its lead actor, Stephen Amell, one of the highest-paid and popular actors on television.

Amell at the Heroes & Villains Convention 2016; Amell has received universal praise for his performance with critics highlighting his ability to display a range of emotions on screen and developing a complex character

Now, it’s time to use the Data Science approach on the show to learn how good it is and what people are saying about it.

Disclaimer: The R scripts used here can be advanced for some readers. You must know the basics of R. You may have an understanding of Twitter API to understand it and work with it.

Notes to Readers with NO Programming Knowledge: Please, feel free to ignore the R scripts and other programming aspects. Plots with descriptions and reports are for everyone.

Resources

To learn or refresh your knowledge of R, DataCamp is extremely helpful. You just need to create a free account.

The free online textbook Introduction to Data Analysis and Prediction Algorithms with R is also really helpful.

R tips

Use Microsoft R Open if possible. It comes with the multi-threaded math library (MKL) and checkpoint package.

But don’t hesitate to switch to regular R if you run into trouble. Microsoft R Open can have some bizarre problems in my personal experiences. Don’t waste too much time on fixing them. Switching to regular R often solves the problem.

Install regular R via CRAN (Instructions for Ubuntu). Install checkpointto ensure reproducibility (it is not a Microsoft R Open exclusive.)

Use RStudio and make good use of its Console window. Some people hold strong feelings against R because of some of its uniqueness compared to other major programming languages. A lot of the confusion can be resolved with simple queries in the Console. Not sure whether the vector index starts from zero or one? c(1,2,3)[1] tells you it’s one. Querying 1:10 tells you the result includes 10 (unlike Python).

Sores on Rotten Tomatoes

Rotten Tomatoes is one of the most popular review-aggregation websites for film and television. The Arrow recently concluded with its eighth season being the final season. For each season, Rotten Tomatoes has a Tomatometer (avg. scores of critics) and an audience score of Arrow. It’s all public and easy to retrieve.

library(plotly) Rotten_Tomatoes_Performence <- c("Tomatometer", "Audience Score")

Season_1 <- c(85, 87)

Season_2 <- c(95, 91)

Season_3 <- c(89, 72)

Season_4 <- c(85, 43)

Season_5 <- c(88, 73)

Season_6 <- c(64, 55)

Season_7 <- c(88, 65)

Season_8 <- c(91, 85)

data <- data.frame(Rotten_Tomatoes_Performence, Season_1, Season_2, Season_3, Season_4, Season_5, Season_6, Season_7, Season_8) fig <- plot_ly(data, x = ~Rotten_Tomatoes_Performence, y = ~Season_1, type = 'bar', name = 'Season 1')

fig <- fig %>% add_trace(y = ~Season_2, name = 'Season 2')

fig <- fig %>% add_trace(y = ~Season_3, name = 'Season 3')

fig <- fig %>% add_trace(y = ~Season_4, name = 'Season 4')

fig <- fig %>% add_trace(y = ~Season_5, name = 'Season 5')

fig <- fig %>% add_trace(y = ~Season_6, name = 'Season 6')

fig <- fig %>% add_trace(y = ~Season_7, name = 'Season 7')

fig <- fig %>% add_trace(y = ~Season_8, name = 'Season 8')

fig <- fig %>% layout(yaxis = list(title = 'Scores'), barmode = 'group')

fig <- fig %>% layout(title = "Arrow on Rotten Tomatoes")

fig

The series has been highly popular amongst the critics, but we can see that season 6 has a low rating but the scores improved as season 7 continued. On the other hand, we can see the audience gave season 4 a low rating but since then, the scores from the audience have improved too.

Manu Bennett as Slade Wilson/Deathstroke, the main antagonist of Season 2 which received critical acclaim. Bennett’s Deathstroke is considered by many as one of the best villains on television. Image Credit: CW

Viewership

Let’s create a plot and compare the average viewership for each season. The data was retrieved from multiple sources from 2012–2020 for each season.

library(plotly)

Season <- c(1, 2, 3, 4, 5, 6, 7, 8)

AVG_Viewers_in_millions <- c(3.68, 3.28, 3.52, 2.90, 2.21, 1.76, 1.58, 1.52) data <- data.frame(Season, AVG_Viewers_in_millions) fig <- plot_ly(data, x = Season , y = AVG_Viewers_in_millions, type = 'scatter', mode = 'lines')

fig <- fig %>% layout(title = "Average Viewers for Each Season of Arrow", xaxis = list(title = 'Seasons'), yaxis = list (title = 'AVG Viewer(Millions)'))

fig

The average viewership dramatically dropped as the series went along. However, one important factor to look at is that after each season ended, it became available on Netflix. It’s possible the audience would rather wait for the season to become available on Netflix instead of watching it live on CW.

Ratings by Demographic

IMDb is an online database that contains information on movies, television series, home videos, video games, and streaming content online. It also includes cast, production crew, personal biographies, plot summaries, trivia, ratings, and fan and critical reviews.

Demographic (age group & sex) of voters in IMDb for a specific show or movie is always public.

library(plotly) # Ratings of Arrow by Sex

age_group <- c("<18", "18-29", "30-44", "45+")

male_rating <- c(7.6, 7.7, 7.4, 7.2)

female_rating <- c(7.8, 7.8, 7.5, 7.6)

data <- data.frame(age_group, male_rating, female_rating) fig <- plot_ly(data, x = ~age_group, y = ~male_rating, type = 'bar', name = 'Males Rating')

fig <- fig %>% add_trace(y = ~female_rating, name = 'Female Rating')

fig <- fig %>% layout(yaxis = list(title = 'Score (1-10)'), barmode = 'group')

fig <- fig %>% layout(title = "Rating of Arrow on IMDb")

fig #Votings of Arrrow by Sex

age_group <- c("<18", "18-29", "30-44", "45+")

male_votes <- c(345, 90036, 85361, 19341)

female_votes <- c(93, 23507, 23471, 6655)

data <- data.frame(age_group, male_votes, female_votes)

fig <- plot_ly(data, x = ~age_group, y = ~male_votes, type = 'bar', name = 'Males Rating')

fig <- fig %>% add_trace(y = ~female_votes, name = 'Female Rating')

fig <- fig %>% layout(yaxis = list(title = 'Votes'), barmode = 'group')

fig <- fig %>% layout(title = "IMDb Votes for Arrow")

fig

The lead actors of Arrow at Comic-Con International 2013. Left to right: Katie Cassidy, Stephen Amell, and Emily Bett Rickards.

What People are Saying on Twitter?

Now, we get the data on our own. We will analyze the tweets about CW’s Arrow. We will analyze the recent 10,000 public tweets with #Arrow.

Word Cloud and Frequency

Getting the data and distribution of tweets

First of all, follow the instructions of this article to obtain your own API key and access token, and install twitteR package:

You need these four variables:

consumer_key <- "FILL HERE"

consumer_secret <- "FILL HERE"

access_token <- "FILL HERE"

access_secret <- "FILL HERE"

The main access point for this post is searchTwitter. It will search for 10,000 recent public tweets in English with “#Arrow.”

setup_twitter_oauth(consumer_key, consumer_secret, access_token, access_secret) cw.tweets<- searchTwitter("#arrow",n=10000,lang="en") tweet=cw.tweets[[1]]

tweet$getScreenName()

tweet$getText()

cw.text=laply(cw.tweets,function(t)t$getText())

length(cw.text) #performing data cleaning and store in csv.file

cw.text = gsub("(RT|via)((?:\\b\\W*@\\w+)+)", "", cw.text)

cw.text = gsub("@\\w+", "", cw.text)

cw.text = gsub("[[:punct:]]", "", cw.text)

cw.text = gsub("[[:digit:]]", "", cw.text)

cw.text = gsub("http\\w+", "", cw.text)

cw.text = gsub("[ \t]{2,}", "", cw.text)

cw.text = gsub("^\\s+|\\s+$", "", cw.text)

cw.text <- str_replace_all(cw.text," "," ")

cw.text <- str_replace_all(cw.text,"#[a-z,A-Z]*","")

cw.text <- str_replace_all(cw.text,"@[a-z,A-Z]*","")

cw.text <- tolower(cw.text) write(cw.text, "arrow.csv",ncolumn=1)

We now have a CSV file with all the cleaned up tweets of Arrow. Now, we make a word-cloud with it.