Exploring TED Talks Dataset with Python
By: Nabeel H. Rizvi
Tags: Data Science, Kaggle, Dataset, Data, Exploratory Data Analysis

This blog will be exploring the contents of the famous TED Talks Dataset from Kaggle. It will present the results of our EDA techniques that we applied to prepare our data for a further advanced analysis. Another article on more in-depth analysis will be published soon.

Note: this blog represents the work of a group of 5 college students from LUMS who are doing a Data Science project.

After going through the attributes in our data, we decided to look for answers for the following questions:

The frequency of TED Talks over years and months

The most popular themes in TED talks

TED talks that received the most popularity or provoked the most discussion

Factors affecting a TED talk’s popularity

Understanding the Data

The dataset contains information on all recordings of TED talks uploaded to the official TED.com website until September 2017.

Data Cleaning

There were a total of 2550 rows with 6 null values, all existing in the speaker_occupation column. We drop the null values since there are only 6. We also dropped the name column since it just restates the contents of main_speaker and title columns. Lastly, we needed to convert the formats of film_date and published_date into datetime datatype in Pandas.

Exploratory Data Analysis

Although, almost all were used, the following were the main key features we made most use of in our EDA:

film_date — contains the date time stamp for the TED talk

comments — the number of first level comments made on the talk

title — of the TED talk

views — the number of views

languages — the number of different languages the talk is available in

duration — of the talk in seconds

event — that the talk took place in.

tags — the themes associated with the talk

ratings — a dictionary representing the frequency of certain positive words given to the talk by reviewers

Frequency of TED talks

Frequency of TED Talks across the years

The data shows that TED talks grew increasingly popular during the 2000s and then their frequency settled to around 250 per year after 2010. A significantly lower count of under 100 for 2017 is misleading since many 2017 TED talks hadn’t been uploaded at the point this data was collected. It’s hard to predict whether the count is bound to go higher or lower after 2017.

Frequency of TED Talks over months

The relationship between months and frequency of TED talks is hard to capture since it does not follow any specific trend. There is a significant amount of variation though; with February accounting for as many as 600 TED Talks while January and August both accounting for less than 40 each.

Popular themes

Word Cloud of the most frequent tags

This word cloud gives a fair representation of what most TED talks about. The most common themes are those pertaining to global politics, technology and society, communication and personal growth. Then some less mainstream but arguably equally important themes are also often discussed, albeit with less frequency. These themes are those like humanity, environment, love, healthcare, religion and psychology.

Let’s also have a look at the TED talks which garnered the most views and those which garnered the most discussion.