3 Ways to Extract Features from Dates with Python
By: Khuyen Tran
Tags: Data Analysis, Data Science, Time Series Analysis, Time Series Forecasting, Python

3 Ways to Extract Features from Dates with Python

Photo by Maddi Bazzocco on Unsplash

Motivation

While working with time series, the values of your datasets might be affected by holidays, which day of the week, and how many days in a month. Is there a way that you can extract these features from your date feature with one line of code?

Yes! In this article, I will show you how to extract the three features mentioned above from the date column using two different Python libraries.

Import and Process Data

We will use the daily female births dataset. This dataset describes the number of daily female births in California in 1959.

Download the data using:

Datapane randomly assigns the time so you can ignore the time in the table. Now we visualize our data

df.set_index('Date').plot()

Extract Weekday

Different days of the week might affect the number of births. Thus, we will add weekday as a feature of our data

Extract Holidays

Holidays can also affect the number of births. Parents might avoid bearing their children on holidays such as New Year’s Day or Christmas. In this blog, Christmas Day is observed to be the day on which the fewest babies were born.

The easiest way to extract the holiday based on the date is to use a Python library called holidays. Install holidays using

pip install holidays

Holidays library provides the holidays of many countries. We can also specify which state or province in the country that we want to extract the holidays from.

Since our data describes the number of daily female births in California, we will find the holidays in California. After getting a dictionary of holidays, we simply use holiday_dict.get(date) to get the holiday of that date.

Timestamp('1959-01-01 00:00:00') "New Year's Day"

We can also peak at all of the holidays available

datetime.date(1959, 1, 1): "New Year's Day",

datetime.date(1959, 2, 22): "Washington's Birthday",

datetime.date(1959, 5, 30): 'Memorial Day',

datetime.date(1959, 7, 4): 'Independence Day',

datetime.date(1959, 7, 3): 'Independence Day (Observed)',

datetime.date(1959, 9, 7): 'Labor Day',

datetime.date(1959, 10, 12): 'Columbus Day',

datetime.date(1959, 11, 11): 'Veterans Day',

datetime.date(1959, 11, 26): 'Thanksgiving',

datetime.date(1959, 12, 25): 'Christmas Day'}

Try to call the dates that are not available in the dictionary

None

If there are some dates that you find missing in this dictionary, you can also create custom Holiday objects.

Let’s use this library to create a holiday feature for our dataset.

Awesome! Now we have a dataset with the holidays feature. In order to compare the number of births between holidays and normal days, we group the data by holidays and take the median of all of the births that are on the same date.

Visualize our data

The number of births on New Year’s Day is the lowest. It seems like the number of births on other holidays is quite high compared to the number of births on normals dates.

However, this data is just within one year so it is not enough for us to determine the real effect of holidays on the number of births. We might find a lower number of births on other holidays compared to normal dates if we have a dataset of 3 or 4 years.

Extract Days in a Month

Parents might also avoid bearing their children on February 29th because it just occurs every four years.

If we want to compare the number of births between different months, the number of dates in a month might also affect the number of births. Thus, it might be important to take the number of dates in a month into account when analyzing our time series dataset.

A really quick way to extract the number of days in a particular month and year is to use calendar.monthrange. This is a built-in Python library so we don’t need to install the library to use it.

The monthrange() method is used to get the weekday of the first day of the month and number of days in a month, for the specified year and month.

Weekday of first day of the month: 5

Number of days in month: 29

The library seems to work well since we know that there are 29 days in February 2020. We will use this to create a days_in_a_month column for our dataset

Awesome! Now we can compare the average number of births per month and the number of days per month

There is a peak in the average number of births in September. Maybe it is because there are more days in September compared to other months? You probably already know the answer but let’s confirm with a bar plot of days in a month

Not really. There are just 30 days in September. There are more days in months such as January, March, May, etc.

Even though we don’t see the direct correlation here, the combination of the number of days in a month and other features can give us some correlation with the number of births.

Conclusion

Congratulations! You have learned how to create more features for your time series dataset using the date. I hope these ideas will help you come up with creative ways to work with time series dataset.

The source code of this article can be found here.

I like to write about basic data science concepts and play with different algorithms and data science tools. You could connect with me on LinkedIn and Twitter.

Star this repo if you want to check out the codes for all of the articles I have written. Follow me on Medium to stay informed with my latest data science articles like these: