The Bechdel Test: Analyzing gender disparity in Hollywood | by Natassha Selvaraj | Towards Data Science

What is the Bechdel test?
The Bechdel test is named after cartoonist Alison Bechdel, who introduced the idea in a comic strip in the year 1985. To pass the test, a story needs to have:
At least two women
The women need to talk to each other
They need to talk to each other about something other than a man
I first heard about the Bechdel test when watching Jane the Virgin, and it caught my interest. To me, it seemed like the standards of the test were too low, and should be ridiculously easy for movies to pass.
However, a simple Google search told me otherwise. Apparently, there were a large number of movies that did not fulfill the requirements to pass the Bechdel test, including:
Slumdog Millionaire
The Blind Side
Tarzan
The Imitation Game
The Social Network
After doing some reading on the Bechdel test, I was struck with inspiration. I decided to conduct data analysis on movie datasets to check whether or not they passed the test!
The Data Question
My aim was to collect data from various different sources, and answer the following questions about the Bechdel test:
Have the Bechdel scores of movies improved over the years?Do movies with higher IMDB ratings have higher Bechdel scores?Do movies with female directors have higher Bechdel scores?Does the budget of a movie have any impact on its Bechdel score?Do movies with higher Bechdel scores generate a larger revenue?
Have the Bechdel scores of movies improved over the years?
Do movies with higher IMDB ratings have higher Bechdel scores?
Do movies with female directors have higher Bechdel scores?
Does the budget of a movie have any impact on its Bechdel score?
Do movies with higher Bechdel scores generate a larger revenue?
Data Collection
I collected data from three different sources: bechdeltest.com, Kaggle, and this movies dataset.
Data Analysis
I will use Python to perform all data analysis and visualization.
First, I used data from bechdeltest.com, which was pretty easy to grab because of their API.
# importsimport urllib, jsonimport pandas as pddf = pd.read_json('http://bechdeltest.com/api/v1/getAllMovies')
Now, I will take a look at the head of the data frame:
df.head()
There are five variables in the data frame:
rating: Bechdel Score of the movie from 0 to 3. A Bechdel score lower than 3 means the movie failed the Bechdel test, and a Bechdel score of 3 means the movie passed.
imdbid: The movie’s IMDB number
title: Movie title
id: Unique movie ID
year: Year the movie was released
Taking a look at the column ‘year’, you can see that there are movies present from the 19th century, and a lot of those movies have a Bechdel score of 0.
I am going to create a new data frame with movies released after the year 1967, and will be using this new data frame for the rest of my analysis.
dfNew = df[df['year']>=1967]
Taking a look at the new data frame:
dfNew.head()
I am now going to rename the column ‘rating’ to ‘Bechdel Score’, to make things clearer for the rest of the analysis.
dfNew.rename(columns={'rating':'Bechdel Score'}, inplace=True)
Now, I am going to convert the ‘year’ column into a datetime object.
dfNew['year'] = pd.to_datetime(dfNew['year'], format='%Y')
Next, I will change the Bechdel Scores to categorical variables.
dfNew['Bechdel Score'] = dfNew['Bechdel Score'].astype('category',copy=False)
Finally, the data is prepared and can be used to do some data visualization.
I will first create a count plot using Seaborn, to visualize the Bechdel scores.
sns.countplot(x='Bechdel Score',data=dfNew)
Most movies after the year 1967 seem to have higher Bechdel scores, but do they pass the Bechdel test?
li = []for i in dfNew['Bechdel Score']:    if(i<3):        li.append(0)    else:        li.append(1)dfNew['pass_test'] = lidfNew
Now, there is a dataframe called pass test, with a value of 1 for all the movies that pass the Bechdel test, and a value of 0 for the movies that don’t.
I will visualize this using Seaborn.
sns.countplot(x='pass_test',data=dfNew)
There are more movies that have passed the Bechdel test as opposed to the ones that haven’t, since the year 1967.
However, this difference is not as significant as I expected it to be. There still seems to be a very large number of movies that didn’t pass the test.
Now, I will take a look at how the Bechdel scores changes with time.
Has there been an improvement in the representation of women in the film industry? Are things still the same? Did they get worse?
To do this, I will use the Plotnine library.
from plotnine import *(ggplot(dfNew)+geom_point(aes('year',color=dfNew['Bechdel Score']),stat='count',show_legend=False)+geom_line(aes('year',color=dfNew['Bechdel Score']),stat='count',show_legend=False))
In the 1970s and 1980s, there were a few movies that passed the Bechdel test. During this time, there was no significant difference between movies that passed the test and movies that didn’t.
The years following the 1980s and 1990s saw a spike in the number of movies that passed the Bechdel test, and there is massive improvement going into the 21st century.
Now, I want to visualize the relationship between IMDB rating and the Bechdel scores. Are movies with higher Bechdel scores more likely to have higher IMDB ratings, or is it the other way round?
To do so, I will use a different dataset called movies.csv, and merge it with my existing data frame.
imdb = pd.read_csv('movies.csv')imdbNew = imdb[['title','rating']]dfNew = pd.merge(dfNew, imdbNew, how='left', left_on=['title'], right_on=['title'])dfNew.head()
The ‘rating’ column in the dataframe represents the IMBD movie ratings. This column has a few null values, and these rows will be dropped.
I will then create a new data frame with only the year, Bechdel score, and IMDB rating, so I can visualize the relationship between the three.
# Dropping rows with null values:dfNew = dfNew.dropna()dfNew = dfNew.drop('id',axis=1)# Creating a new dataframe with only year, Bechdel scores, and imdb rating:new = dfNew.groupby(['year','Bechdel Score']).agg({'rating':'mean'}).reset_index()new.head()
Perfect! Now, I will visualize this relationship:
# Plot year against IMDB rating and Bechdel Score:ggplot(new,aes(x='year',y='rating',color='Bechdel Score'))+ geom_point()+geom_smooth()+scale_y_continuous(name="imdb rating")+labs( colour='Bechdel Score' )
It appears as though movies that pass the Bechdel test have significantly lower IMDB ratings compared to movies that don’t, which was pretty surprising to me.
Now, I will try to visualize the relationship between the gender of the director and Bechdel scores. I assume that movies with female directors are more likely to have higher Bechdel scores, which I will try to plot here.
To do so, I will use a different movie dataset from Kaggle. I will then use a gender prediction library to predict the gender of the director from their name, and merge this with the previous dataframe.
import gender_guesser.detector as genlatest = pd.read_csv('movielatest.csv',encoding = "latin")dfLatest = latest[['name','director']]dfLatest.rename(columns={'name':'title'}, inplace=True)dfLatest = pd.merge(dfNew, dfLatest, how='left', left_on=['title'], right_on=['title'])dfLatest = dfLatest.dropna()dfLatest.head()
The newly created data frame now has an additional variable in it; director. I will now try to predict the gender of the director given their first name, and append it to the data frame.
# Predicting gender of director from first name:d = gen.Detector()genders = []firstNames = dfLatest['director'].str.split().str.get(0)for i in firstNames[0:len(firstNames)]:    if d.get_gender(i) == 'male':        genders.append('male')    elif d.get_gender(i) == 'female':        genders.append('female')    else:        genders.append('unknown')dfLatest['gender'] = gendersdfLatest = dfLatest[dfLatest['gender'] != 'unknown']# Encode the variable gender into a new dataframe:dfLatest['Male'] = dfLatest['gender'].map( {'male':1, 'female':0} )dfLatest.head()
The data frame has two additional variables; gender and male. Gender represents the gender of the director. The variable male has a value of 1 if the director is male, and 0 if she is female.
Now that I have this information, I will go ahead and visualize some more trends in the data.
I will create a count plot of the variable gender, to visualize the number of males to females in the dataset:
sns.countplot(x='gender',data=dfLatest)
It appears as though almost all the movies in the dataset have male directors. Again, this was something I expected, since Hollywood is known to employ a very small number of female directors.
Next, I will visualize the gender of the director with the Bechdel score, to see if movies with female directors have a higher score.
sns.countplot(x='Bechdel Score',hue='gender',data=dfLatest)
ggplot(aes(x = 'year', y = 'Bechdel Score',color='gender'), data = dfLatest)+geom_point()
Almost all the movies with female directors seem to pass the Bechdel test!
Next, I will take a look at the variable budget, to see if there is any kind of correlation between the budget of a movie and it’s Bechdel score.
dfLatest['budget']=latest['budget']ggplot(aes(x='year', y='budget',color='Bechdel Score'), data = dfLatest)+geom_point()+geom_smooth()
One thing that can be seen here is a significant increase in a movie’s budget with time. Also, movies that pass the Bechdel test seem to have a slightly higher budget than movies that score a zero, but there is no strong correlation that can be observed here.
Now, I will visualize the relationship between budget and gender of the director:
ggplot(aes(x = 'year', y = 'budget',color='gender'), data = dfLatest)+geom_point()+geom_smooth()
The gender of the director does not appear to have any kind of relationship or impact on the budget of a movie.
Visualizing the relationship between a movie’s budget, Bechdel score, and gender of director:
sns.violinplot(x='Bechdel Score',y='budget',hue='Male',data=dfLatest)
Next, I will visualize the genre of a movie and Bechdel scores, and any changes with time.
dfLatest['genre'] = latest['genre']ggplot(aes(x = 'year', y = 'genre',color='Bechdel Score'), data = dfLatest)+geom_point()
Finally, I will take a look at the movie grossings, along with year, Bechdel scores, and gender:
dfLatest['gross'] = latest['gross']# Movie grossing with Bechdel score and gender:sns.violinplot(x='Bechdel Score',y='gross',hue='Male',data=dfLatest)
# Movie grossing with year and gender:ggplot(aes(x = 'year', y = 'gross',color='gender'), data = dfLatest)+geom_point()
And that’s it! I managed to visualize relationships and answer all the data questions I set out to find answers to.
Some interesting data findings include:
Movies that fail the Bechdel test tend to have higher IMDB ratings.Over the years, there has been an increasing number of movies that pass the Bechdel test.Movies with female directors have higher Bechdel scores.There is no clear correlation between the budget or revenue of a movie and its Bechdel score.
Movies that fail the Bechdel test tend to have higher IMDB ratings.
Over the years, there has been an increasing number of movies that pass the Bechdel test.
Movies with female directors have higher Bechdel scores.
There is no clear correlation between the budget or revenue of a movie and its Bechdel score.
Of course, this analysis was only done with the data I gathered from three places, and might not represent the entire population of movies out there.
Furthermore, there may have been other variables present that affected the outcome of this analysis, and it might be a good idea to experiment with data from a couple of other places before coming to a conclusion.
Finally, I would like to mention that the Bechdel test is not necessarily the best benchmark to measure female representation in movies. It does not take into consideration how well written a female character is, neither does it measure meaningful depth of character.
However, it is one of the most well-known metrics used to expose gender bias and is the only test we have this kind of data on.
The datasets and complete code can be found here.