Kickstarter Projects Walk-Through — Simple Data Exploration in Python | by Franz B. | Towards Data Science

This article analyses a dataset of roughly 380.000 Kickstarter projects. I will lead you through a simple data exploration with Python to reveal interesting insights in Kickstarter projects and what attributes are important when it comes to examining the success (or failure) of a certain project. In general, we will have a look at basic data visualisation and feature engineering with Python.
Kickstarter is one of the most renown platforms for promoting (mostly) creative, smart and visionary ideas and concepts. Each project seeks funding that the Kickstarter-Crowd can provide if they support the idea and would like to see it succeed. However, there are many projects that fail, even though the initial idea might have been convincing. Why is that? This article tries to dive into attributes related to each project and to reveal patterns, insights and anything of interest related to Kickstarter projects.
The dataset contains 378.661 projects from Kickstarter and twelve initial attributes related to each project. Let’s have a look at what information we got about each project:
namePretty obvious, the name of the respective project. E.g. “Daily Brew Coffee”main_categorythe main category the project falls in. E.g. “Poetry”, ”Food”, “Music” and many morecategorya more precise description of the main category. Basically, a subgroup of main_category (see 2.). E.g. “Drinks” which would be a subgroup of “Food” from the main_category attribute.currencythe currency of the project (e.g. USD or GBP)launchedthe launch date for the project. This will be important when we analyze timeframes later ondeadlinethe deadline for the project. Just as the launch date, the deadline will become important just as the launch dateusd_pledged_realamount of USD the project realised at the deadlineusd_goal_realamount of USD the project asked for initiallybackersthe number of supporters that actually invested in the projectcountrycountry of origin of the projectIDproject IDstateWas the project successful at the end of the day? state is a categorical variable divided into the levels successful, failed, live, cancelled, undefined and suspended. For the sake of clarity, we will only look at whether a project was successful or failed (hence, we will remove all projects that are not classified as one of the two). Projects that failed or were successful make up around 88% of all projects.
namePretty obvious, the name of the respective project. E.g. “Daily Brew Coffee”
main_categorythe main category the project falls in. E.g. “Poetry”, ”Food”, “Music” and many more
categorya more precise description of the main category. Basically, a subgroup of main_category (see 2.). E.g. “Drinks” which would be a subgroup of “Food” from the main_category attribute.
currencythe currency of the project (e.g. USD or GBP)
launchedthe launch date for the project. This will be important when we analyze timeframes later on
deadlinethe deadline for the project. Just as the launch date, the deadline will become important just as the launch date
usd_pledged_realamount of USD the project realised at the deadline
usd_goal_realamount of USD the project asked for initially
backersthe number of supporters that actually invested in the project
countrycountry of origin of the project
IDproject ID
stateWas the project successful at the end of the day? state is a categorical variable divided into the levels successful, failed, live, cancelled, undefined and suspended. For the sake of clarity, we will only look at whether a project was successful or failed (hence, we will remove all projects that are not classified as one of the two). Projects that failed or were successful make up around 88% of all projects.
Let’s quickly remove all projects that do not have failed or successful as their state.
#return only successful and failed projects. This makes things more clear later ondata_kick = data_kick.loc[data_kick['state'].isin(            ['successful', 'failed'])]
Let’s first have a look at the datatype of all attributes to get a better understanding of the data and see if everything’s in order.
data_kick.info()ID                  378661 non-null int64name                378657 non-null objectcategory            378661 non-null objectmain_category       378661 non-null objectcurrency            378661 non-null objectdeadline            378661 non-null objectgoal                378661 non-null float64launched            378661 non-null objectpledged             378661 non-null float64state               378661 non-null objectbackers             378661 non-null int64country             378661 non-null objectusd pledged         374864 non-null float64usd_pledged_real    378661 non-null float64usd_goal_real       378661 non-null float64
It can be seen that all attributes tend to be in the exact data format — either floats (hence numeric) or objects (hence categorical). The only two attributes that are not in the desired format are our two time attributes: launched and deadline. We will take care of these in a bit. But let’s get an understanding of the dataset first by analysing the amount pledged vs. the initial goal the project had.
Let’s have a look at the relationship between a project’s goal and the actual amount pledged. It is quite obvious that a project is labelled as successful ifamount pledged ≥ goal and unsuccessful if amount pledged < goal. The following graph visualizes both the goal and amount pledged for each project and the individual state of the project (only failed and successful projects are displayed).
#define colors (darkgreen for successful projects and darkred for failed onescolors = ('darkgreen','darkred')#create a plot using seaborn, adjust data to millionsax = sns.scatterplot(data_kick.usd_pledged_real/1e6, data_kick.usd_goal_real/1e6, hue=data_kick.state, palette=colors)#add blue line to better visualize the border between failed and successful projectssns.lineplot(x=(0,50), y=(0,50), color='darkblue')#set the axes from -1 to their maximum (-1 looks better than 0 actually)ax.set(ylim=(-1,None), xlim=(-1,None))#set labels and titleax.set(xlabel='Amount Pledged in Millions', ylabel='Goal in Millions', title= 'Goal vs. Pledged')
Intuitively, projects that pledged more than/as much as their goal are successful and displayed green and failed projects are marked red. Have a look at the blue line intersecting the origin: successful projects are always underneath the line while failed projects are above it. This makes sense, doesn’t it? By the way, you can observe that the project with the highest amount pledged collected roughly around 20 Million USD and the project with the highest goal was looking for more than 160 Million USD (and failed). Time for a closer look.
Let’s zoom in a little to get a better understanding by changing both axes to a range from zero to 20 Million:
#we are using the same code as above expect for the axis limits, that are set from 0 Millions (-1 in the code) to 20 Millionsax.set(ylim=(-1,20), xlim=(-1,20)
This zoom-in allows a more intuitive approach: The blue line is now intersecting the origin at a 45° angle since we adjusted the axes accordingly to the same range. As mentioned before, every data point (hence project) underneath the line succeeded, whereas every data point above it failed.
Interestingly, the graph suggests that unsuccessful projects usually fail without coming even close to their goal, meaning that they “do not move horizontally towards the blue line but stay at x~0”. This leads to the conclusion that Kickstarter is an “all or nothing” platform. If you don’t make it, you probably didn’t even come close. On the other hand, many successful projects exceed their stated goal by far and pledge a multiple of their initial goal.
How did the relationship between the amount pledged and the goal develop over time? Let us look at the development of goals and pledges over the entire time frame grouped by years. We use the following code to get the accumulated amount for both goals and pledges for each year.
#we choose the desired grouping intervall ('launched year') and the two variables pledge and goal that we are interested in and sum them up over the years.data_kick_groupby_year = data_kick.groupby('launched_year')['usd_pledged_real', 'usd_goal_real'].sum()#let's use this variable to plot the data over time:ax = sns.lineplot(x=data_kick_groupby_year_sum.index, y= data_kick_groupby_year_sum.usd_pledged_real/1e9, linewidth= 4, label= 'USD Pledged', color= 'darkgreen')sns.lineplot(x=data_kick_groupby_year_sum.index, y= data_kick_groupby_year_sum.usd_goal_real/1e9, linewidth= 4,label='USD Goal', color= 'darkred')#set labels and titleax.set(xlabel='Launch Year', ylabel='USD in Billion', title='Comparison Pledged vs Goal over Time')
We obtain this graph:
The graph shows both the accumulated amount of goals and pledges for all projects of the respective year. E.g., in 2013, the sum of all financial goals of all launched projects was a bit higher than 1 Billion USD.
It can be seen that the accumulated amount of the total goal sharply increases in 2013 and has its peak in 2015 before it slowly decreases. On the other hand, the accumulated amount of pledges for each year steadily decreases but rather linearly. It actually decreased slightly from 2015 to 2017.
The reason for the sharp increase in the goal value is quite obvious, isn’t it? The number of projects dramatically increased as can be seen here:
data_kick['launched_year'].value_counts()2015    652722014    593062016    492922017    434192013    411012012    384782011    240482010     95772009     1179
More projects, more goals. However, this is just one component contributing to the increase in accumulated goals. Not only more projects launched over the years, but the average goal per project also increased over the years as well, as shown here:
#we group again. This time we take the average though:data_kick_groupby_year_avg = data_kick.groupby('launched_year')['usd_pledged_real', 'usd_goal_real'].mean()print(data_kick_groupby_year_avg)launched_year  usd_pledged_real  usd_goal_real   2009                2375.535335    6799.5074302010                3041.302897   13270.0626472011                4256.161398    9715.9570622012                8247.532875   17862.7388382013               11326.171820   24946.1649292014                8466.032884   47346.9420482015               10058.201943   70094.5137352016               12847.626664   52879.1357522017               13564.594251   39073.175276
The average goal increases over time (from ca. 10,000 USD in 2011 to ca. 53,000 USD in 2016 followed by a sharp drop in 2017). Hence, not only has the number of projects increased, the average goal per project increased as well. The combination of these factors boosted the accumulated goal graph over time. Now, enough of time. Let’s have a look at names!
Another aspect that might influence the outcome of a project is its name. We would expect shorter, concise names to be more appealing to (potential) investors than long and blurry phrases.
For example, a name like “ Circa Vitae requests crowd fundage to record our full length album... TO VINYL!!!” is less catchy than “Day X” (the first project indeed failed and the latter succeeded). Does the data support this hypothesis? Let’s feature engineer a little and create a new column containing the string length of a project’s name.
data_kick['name_length'] = data_kick['name'].str.len()
We now saved the string length of each project’s name in name_length. Let’s have a look what the column looks like.
data_kick.loc[:,['name','name_length']].head(5)name                                                  name_length0                    The Songs of Adelaide & Abullah         31.01      Greeting From Earth: ZGAC Arts Capsule For ET         45.02                                     Where is Hank?         14.03  ToshiCapital Rekordz Needs Help to Complete Album         49.04  Community Film Project: The Art of Neighborhoo...         58.0
Let’s create a scatterplot to show the relationship between name_length and the amount usd_pledged_real.
#create scatterplot for pledged amount and name lengthax = sns.scatterplot(data_kick.usd_pledged_real/1e6, data_kick.name_length, hue=data_kick.state, palette=colors)#set labels accordinglyax.set(xlabel='Amount Pledged in Millions', ylabel='Character Length of Project Name', title= 'The Importance of Choosing the Right Name')
Interestingly, if a project’s name is exceeding 60 characters, the pledged amount is distributed around 0 and (more or less) all projects seem to fail. Let us have a look at the goal of these projects compared to the <60 projects and check the following hypothesis:
Could it be that >60 character projects do have explicitly high goal values that they aim to collect? To check this hypothesis let’s look at the average goal for name_length≥60 vs. goal for name_length<60.
Let’s have a look at this subgroup of projects that have more than 60 characters in terms of their goal size and their project state:
#create a variable for all data points above a name length of 60data_kick_60_up = data_kick.loc[data_kick.name_length >= 60, ['state','usd_goal_real']]#create a variable for the data points below a name length of 60data_kick_60_down = data_kick.loc[data_kick.name_length < 60, ['state','usd_goal_real']]#compare the median of both groups. We see that the above 60 group actually has a higher median than the below 60 group.print(data_kick_60_up['usd_goal_real'].median() > data_kick_60_down['usd_goal_real'].median())True
We can see that the median (and mean as well) is higher for the ≥60 group regarding their goal (Boolean returns “True”). Hence, we cannot conclude that projects with a name length of ≥60 simply have lower goal values. On the contrary, they actually have higher goals compared to the projects that have names that are <60 characters (on average). This might account for their failure. Isn’t that something!
As mentioned earlier, the most intuitive explanation for the “success-segmentation” based on the project’s name length is that long names have the tendency to sound unprofessional, not on point and not as committed (obviously this is just a personal opinion). Hence, they pledge less and fail.
As mentioned before, all variables are of the right type — except for the time variable(s). Luckily, Pandas allows us to quickly change date variables to actual dates:
data_kick['launched'] = pd.to_datetime(data_kick['launched'],format='%Y-%m-%d %H:%M:%S')data_kick['deadline'] = pd.to_datetime(data_kick['deadline'],format='%Y-%m-%d %H:%M:%S')
If you check data_kick.info() now, you will see that launched and deadline have the right data type. Now that we have converted the time variables to the correct format, let’s have a closer look at them. In particular, we have two interesting time columns: The exact date when the project was launched and secondly the deadline for each project. Now what?
Maybe the date the project was launched influences its success, maybe the time between the launch and deadline also affect the outcome of the project. One could argue that the longer the span between launch and deadline, the more time there is to get noticed and collect money. Hence, the longer the span, the higher the chances for success. In addition, the month (or even hour) the project was launched might influence its outcome. But does the data support these hypotheses? Let’s have a look at the launch times first.
We can extract the respective hour when the project was launched by the following code:
data_kick['launched_hour'] =                           data_kick.launched.apply(lambda x: x.hour)data_kick['launched_month'] =                          data_kick.launched.apply(lambda x: x.month)
We just created two new columns containing the month and the hour of the project’s launch respectively. Let us see if these new attributes contain patterns that help us understand whether a project is successful or not. Let’s visualize the hour of the day and the success of the project:
#extract launched hour and state for all states 'successful' and append the same for all states 'failed'.data_kick = data_kick.loc[data_kick.state == 'successful',['launched_hour', 'state']].append(data_kick.loc[data_kick.state == 'failed',   ['launched_hour', 'state']])#plot the dataax = sns.countplot(x=extracted_levels_hour.launched_hour,               hue = extracted_levels_hour.state)#set labels and titleax.set(xlabel='Launched Hour', ylabel='Number of Projects', title= 'Hour the Project was Launched')
There are a few interesting things to observe here:
In general, there are more projects launched either in the early morning (between 0 and 6 am) and more projects launched in the late afternoon/evening (between 4 and 23 pm) compared to the time between 7 am and 3 pm.The ratio between successful and failed projects varies over the time of the day. We can see that the ratio between failed and successful projects is almost 1 (a bit less actually) for projects launched between 2 and 3 pm. In contrary, the ratio for projects launched after 8 pm is much worse (in other words: by far more projects failed than succeeded).
In general, there are more projects launched either in the early morning (between 0 and 6 am) and more projects launched in the late afternoon/evening (between 4 and 23 pm) compared to the time between 7 am and 3 pm.
The ratio between successful and failed projects varies over the time of the day. We can see that the ratio between failed and successful projects is almost 1 (a bit less actually) for projects launched between 2 and 3 pm. In contrary, the ratio for projects launched after 8 pm is much worse (in other words: by far more projects failed than succeeded).
Let’s have a look whether the month the project was launched reveals anything about its state:
#plot the datasns.countplot(x=data_kick.launched_hour,               hue = data_kick.state)#set labels and titleax.set(xlabel='Month of the Year', ylabel='Number of Projects', title= 'Month the Project was Launched')
Compared to the hour of the day, the months do not seem to reveal any surprising insights. However, note that there are by far fewer projects launched in December compared to all other months.
Finally, let us look at the span between launched and deadline. We can obtain this span easily by subtracting launched from deadline (this is a useful feature of the date_time transformation we have done earlier). In the code below, we take the difference between launched and deadline and extract the days immediately (‘timedelta64[D]’):
data_kick['project_time'] =  (data_kick['deadline'] -data_kick['launched']).astype('timedelta64[D]').astype(int)data_kick['project_time'].head(5)0    581    592    443    294    55
You can see that our new column project_time measures the span between launch and deadline in days. For example, the first project in the dataset ([0]) was on Kickstarter for almost two months (58 days). Before we visualize the span for a project, let us bin the results. The maximum of data_kick[‘project_time’] is at 91, so instead of getting 91 different results, we could bin the data by week to get a more intuitive overview. Let us first write a function that allows us to effectively bin data:
def discretizer(data, binning_values, labels):    """    Input: data series that should be discretized    bininningvalues: enter numerical array of binning cuts    labels: enter string array of names for binsOutput: gives you a list of binned data    """    data = pd.cut(data, bins=binning_values, labels=labels)    return data
Let’s parse in project_time and bin it by weeks:
#save the binned data as new columndata_kick["project_time_weeks"] =#call the discretizer funtiondiscretizer(#first thing we pass is the column we want to be discretized, project_time in this casedata_kick['project_time'],#next, we create a list with the different binning intervals (starting at 0 and then going in steps of 7 until the maximum value of project_time (+7)list(range(0,max(data_kick['project_time'])+7,7)),#last but not least, we need to label the bins. We number them by creating a list from 1 (first week) to the last week depending on the maximum value of project_timelist(range(1,max(list(range(0,max(data_kick['project_time'])//7+2))))))
So what does the new column, project_time_weeks look like?
#print first 5 rows of both columns:data_kick[['project_time', 'project_time_weeks']].head(5)#resulting in:project_time  project_time_weeks          34                  5          19                  3          29                  5          27                  4          14                  2
We can see that the days were successfully binned by weeks. E.g. 19 days are converted to week 3. Now, finally, let’s have a luck whether the time between launch and deadline affects the success of a project:
#plot the datasns.countplot(x=data_kick.project_time_discret,               hue = data_reduced.state, palette=colors)
We instantly notice that most projects are put on Kickstarter for about 5 weeks and almost all projects are live between 1 and 9 weeks (so a bit more than 2 months).
The proportion between successful and failed projects varies over time: Projects with a short run time (1–4 weeks) succeed relatively more often compared to projects that are launched with a medium to long run time (5–13 weeks). This insight could support the hypothesis that project are not more likely to succeed the longer they are live as one might have assumed beforehand (-> the longer a project is live, the more people have the chance to see it over time, the more likely it will succeed. This does not seem to be true).
Now we take a look at the role of backers that support each project. In general, we would expect that a high number of backers contribute positively to the likelihood of success for a specific project. But is this really the case?
Let us visualize the number of backers, amount pledged and the state of the project.
ax = sns.scatterplot(x=data_kick.backers, y=data_kick.usd_pledged_real/1e6, hue=data_kick.state, palette=colors)#we set the xlim to 100,000 backers to get a better overviewax.set(ylim=(-1,None), xlim=(-1,100000))#set labels and titleax.set(xlabel='Number of Backers', ylabel='USD Pledged in Million', title= 'Backer vs Pledge')
The code results in the following graph:
This result isn’t surprising, is it? We can clearly see that the number of backers correlates with the success of a project. Most projects that fail also have a (relatively) smaller number of backers. However, let’s go further: Not only the absolute number of backers per project is interesting but the average amount spent by each backer per project. In order to get this attribute, we need to divide the amount pledged by the number of actual backers. Let’s do that quickly.
#divide usd_pledged_real by number of backers (note that a Zerodivisionerror will occur if we divide by zero resulting in a NaN. We replace these with 0data_kick['backer_ratio'] = (data_kick['usd_pledged_real']//data_kick['backers']).fillna(0)#set label and titleax.set(xlabel='Pledge/Backer Ratio (funding per backer)', ylabel='USD Pledged in Million', title= 'Backer Ratio vs Pledge')
Looking at the backer ratio (hence the sum of the amount pledged divided by the total number of backers) we can see that most successful projects have a relatively low backer ratio, meaning that each backer contributes a relatively small amount. According to the graph, this amount varies between shortly above 0 USD and up to around 700 USD. We can see that most projects that have relatively high backer ratios (<2,000 USD) failed.
Now that we have looked at time frames, names, backers and many other details, it is time to stop our little walkthrough here. There are still plenty of possible insights and patterns to reveal but I leave this up to you ;-)
6. Conclusion
This introductory post about basic Data Visualization in Python revealed some interesting facts about Kickstarter projects. Let’s summarize:
Projects that fail, most of the times fail miserably. They “cannot move towards the blue line” as discussed in part 3. It’s all in or nothing ;-).The number and average goal of projects increased sharply over time (at least until 2015) whereas the provided pledges only slowly increase.Names play a big role. In case you want to launch a project, make sure you keep its name precise and short. “Tenacious Exclusive Scott Tolleson Uncle Argh NOIR Mini Qee” may sound funny (it actually doesn’t) but won’t get you anywhere.Try to launch your project between 12 and 4 pm and do NOT launch in December (Ok, I am not 100% serious about that one).
Projects that fail, most of the times fail miserably. They “cannot move towards the blue line” as discussed in part 3. It’s all in or nothing ;-).
The number and average goal of projects increased sharply over time (at least until 2015) whereas the provided pledges only slowly increase.
Names play a big role. In case you want to launch a project, make sure you keep its name precise and short. “Tenacious Exclusive Scott Tolleson Uncle Argh NOIR Mini Qee” may sound funny (it actually doesn’t) but won’t get you anywhere.
Try to launch your project between 12 and 4 pm and do NOT launch in December (Ok, I am not 100% serious about that one).
I hope you enjoyed this read. I appreciate any kind of feedback, (constructive) criticism or other recommendations. Thanks!