Feature Engineering on Date-Time Data | by Pararawendy Indarjo | Towards Data Science

According to Wikipedia, feature engineering refers to the process of using domain knowledge to extract features from raw data via data mining techniques. These features can then be used to improve the performance of machine learning algorithms.
Feature engineering does not necessarily have to be fancy though. One simple, yet prevalent, use case of feature engineering is in time-series data. The importance of feature engineering in this realm is due to the fact that (raw) time-series data usually only contains one single column to represent the time attribute, namely date-time (or timestamp).
Regarding this date-time data, feature engineering can be seen as extracting useful information from such data as standalone (distinct) features. For example, from a date-time data of “2020–07–01 10:21:05”, we might want to extract the following features from it:
Month: 7Day of month: 1Day name: Wednesday (2020–07–01 was Wednesday)Hour: 10
Month: 7
Day of month: 1
Day name: Wednesday (2020–07–01 was Wednesday)
Hour: 10
Extracting such kinds of features from date-time data is precisely the objective of the current article. Afterwards, we will incorporate our engineered features as predictors of a gradient boosting regression model. Specifically, we will forecast metro interstate traffic volume.
This article will cover the following.
A step-by-step guide to extract the below features from a date-time column.
MonthDay of monthDay nameHourDaypart (morning, afternoon, etc)Weekend flag (1 if weekend, else 0)
Month
Day of month
Day name
Hour
Daypart (morning, afternoon, etc)
Weekend flag (1 if weekend, else 0)
How to incorporate those features in a Gradient Boosting regression model to forecast metro interstate traffic volume.
Throughout the article, we use Metro Interstate Traffic Volume Data Set, which can be found in the UCI Machine Learning Repository here.
Citing its abstract, the data is about hourly Minneapolis-St Paul, MN traffic volume for westbound I-94, which includes weather and holiday features from 2012–2018. This 48204 rows data contains the following attributes.
holiday: Categorical US National holidays plus regional holiday, Minnesota State Fairtemp: Numeric Average temp in kelvinrain_1h: Numeric Amount in mm of rain that occurred in the hoursnow_1h: Numeric Amount in mm of snow that occurred in the hourclouds_all: Numeric Percentage of cloud coverweather_main: Categorical Short textual description of the current weatherweather_description: Categorical Longer textual description of the current weatherdate_time: DateTime Hour of the data collected in local CST timetraffic_volume:Numeric Hourly I-94 ATR 301 reported westbound traffic volume (the target)
holiday: Categorical US National holidays plus regional holiday, Minnesota State Fair
temp: Numeric Average temp in kelvin
rain_1h: Numeric Amount in mm of rain that occurred in the hour
snow_1h: Numeric Amount in mm of snow that occurred in the hour
clouds_all: Numeric Percentage of cloud cover
weather_main: Categorical Short textual description of the current weather
weather_description: Categorical Longer textual description of the current weather
date_time: DateTime Hour of the data collected in local CST time
traffic_volume:Numeric Hourly I-94 ATR 301 reported westbound traffic volume (the target)
Let’s load the data.
# import librariesimport pandas as pdimport numpy as npimport matplotlib.pyplot as plt# load the dataraw = pd.read_csv('Metro_Interstate_Traffic_Volume.csv')# display first five rowsraw.head()# display details for each columnraw.info()
From the output of the info method in the above, we know the date_time column is still in object type. So we need to convert it to datetime type.
# convert date_time column to datetime typeraw.date_time = pd.to_datetime(raw.date_time)
From the output of info method in the above, we know there are categorical features other than the date_time column. But due to the main topic of this article, we will focus on feature engineering our date_time column.
Month
It turns out that Pandas has many handy methods to work with datetime typed data. To extract time/date components, all we need to do is calling pd.Series.dt attributes family. pd.Series.dt.month is the one we need to extract the month component. This will yield a Series of the digit of the month component (e.g. 1 for January, 10 for October) in int64 format.
# extract month featuremonths = raw.date_time.dt.month
Day of month
Quite similar as before, we just need to call pd.Series.dt.day. For example, a date-time of 2012–10–27 09:00:00 would be resulted in 27 using this attribute.
# extract day of month featureday_of_months = raw.date_time.dt.day
Hour
This one is also trivial. The attribute pd.Series.dt.hour will result in a Series of hour digits, ranging from 0 to 23.
# extract hour featurehours = raw.date_time.dt.hour
Day name
This one is getting interesting. Our goal is to extract the day name for each date-time in the raw.date_time Series. It consists of two steps. First is to extract the day name literal using pd.Series.dt.day_name() method. Afterwards, we need to one-hot encode the results from the first step using pd.get_dummies() method.
# first: extract the day name literalto_one_hot = raw.date_time.dt.day_name()# second: one hot encode to 7 columnsdays = pd.get_dummies(to_one_hot)#display datadays
Daypart
In this part, we will create a grouping based on the hour digits. On a high level, we want to have six groups representing each daypart. They are Dawn (02.00 — 05.59), Morning (06.00 —09.59), Noon (10.00–13.59), Afternoon (14.00–17.59), Evening (18.00–21.59), and Midnight (22.00–01.59 on Day+1).
To this end, we create an identifying function that we later use to feed an apply method of a Series. Afterwards, we perform one-hot encoding on the resulted dayparts.
# daypart functiondef daypart(hour):    if hour in [2,3,4,5]:        return "dawn"    elif hour in [6,7,8,9]:        return "morning"    elif hour in [10,11,12,13]:        return "noon"    elif hour in [14,15,16,17]:        return "afternoon"    elif hour in [18,19,20,21]:        return "evening"    else: return "midnight"# utilize it along with apply methodraw_dayparts = hours.apply(daypart)# one hot encodingdayparts = pd.get_dummies(raw_dayparts)# re-arrange columns for conveniencedayparts = dayparts[['dawn','morning','noon','afternoon','evening','midnight']]#display datadayparts
Weekend flag
The final feature we engineer from the date_time column is is_weekend. This column indicates whether the given date-time is in the weekend (Saturday or Sunday) or not. To proceed with this objective, we will make use of our previous pd.Series.dt.day_name() method and apply a simple lambda function on top of it.
# is_weekend flag day_names = raw.date_time.dt.day_name()is_weekend = day_names.apply(lambda x : 1 if x in ['Saturday','Sunday'] else 0)
Holiday flag & weather
Lucky on us, the data also contains public holiday information. The information is granular since it mentions the name of each public holiday. Nevertheless, I assumed that there is no significant gain for encoding each of these holidays. Therefore, let’s just create a binary feature indicating whether or not the corresponding date is a holiday.
# is_holiday flagis_holiday = raw.holiday.apply(lambda x : 0 if x == "None" else 1)
The last categorical feature we need to take care of is the weather column (my assumption strikes again here, I do not include weather_description feature). As you might guess, we just one-hot encode the feature as follows.
# one-hot encode weatherweathers = pd.get_dummies(raw.weather_main)#display dataweathers
The final data
Hurray! We finally have our final — ready-to-train — data! Let’s create a brand new data frame called features that consists of all features, both numerical (we put as-is from the raw data) and categorical (our engineered ones).
# features table#first step: include features with single column naturefeatures = pd.DataFrame({    'temp' : raw.temp,    'rain_1h' : raw.rain_1h,    'snow_1h' : raw.snow_1h,    'clouds_all' : raw.clouds_all,    'month' : months,    'day_of_month' : day_of_months,    'hour' : hours,    'is_holiday' : is_holiday,    'is_weekend' : is_weekend})#second step: concat with one-hot encode typed featuresfeatures = pd.concat([features, days, dayparts, weathers], axis = 1)# target columntarget = raw.traffic_volume
Before we feed the data to our model, we need to split the data (training and test data). Notice below we do not shuffle our data, this is due to the time-series nature of the data.
#split data into training and test dataX_train, X_test, y_train, y_test = train_test_split(features, target, test_size=0.1, shuffle = False)
Now we are ready to build our model to forecast metro interstate traffic volume. In this work, we will use the Gradient Boosting regression model.
The details of the model are beyond the scope of this article but on a high level, the gradient boosting model belongs to ensemble model family which employs gradient descent algorithm to minimize errors in sequential (additive) weak learner models (decision trees).
Model training
Let’s instantiate and train the model on the training data!
from sklearn import datasets, ensemble# define the model parametersparams = {'n_estimators': 500,          'max_depth': 4,          'min_samples_split': 5,          'learning_rate': 0.01,          'loss': 'ls'}# instantiate and train the modelgb_reg = ensemble.GradientBoostingRegressor(**params)gb_reg.fit(X_train, y_train)
Just wait a little while until the training converged.
Model evaluation
To evaluate the model, we use two metrics: MAPE (mean absolute percentage error) and R2 score. We will compute these metrics on the test data.
# define MAPE functiondef mape(true, predicted):            inside_sum = np.abs(predicted - true) / true    return round(100 * np.sum(inside_sum ) / inside_sum.size,2)# import r2 scorefrom sklearn.metrics import r2_score# evaluate the metricsy_true = y_testy_pred = gb_reg.predict(X_test)#print(f"GB model MSE is {round(mean_squared_error(y_true, y_pred),2)}")print(f"GB model MAPE is {mape(y_true, y_pred)} %")print(f"GB model R2 is {round(r2_score(y_true, y_pred)* 100 , 2)} %")
We can see that our model is quite decent in performance. Our MAPE is less than 15%, while R2 score is a little over 95%.
Graphical results
To comprehend our model performance visually, let’s have some plot!
Due to the length of our test data (4820 data points), we just plot the actual vs model-predicted values on the last 100 data points. Moreover, we also include another model (called gb_reg_lite in the plotting code below) which does not incorporate date-time engineered features as its predictors (it only contains non-date-time column as features, including temp, weather, etc).
fig, ax = plt.subplots(figsize = (12,6))index_ordered = raw.date_time.astype('str').tolist()[-len(X_test):][-100:]ax.set_xlabel('Date')ax.set_ylabel('Traffic Volume') # the actual valuesax.plot(index_ordered, y_test[-100:].to_numpy(), color='k', ls='-', label = 'actual')# predictions of model with engineered featuresax.plot(index_ordered, gb_reg.predict(X_test)[-100:], color='b', ls='--', label = 'predicted; with date-time features')# predictions of model without engineered featuresax.plot(index_ordered, gb_reg_lite.predict(X_test_lite)[-100:], color='r', ls='--', label = 'predicted; w/o date-time features')every_nth = 5for n, label in enumerate(ax.xaxis.get_ticklabels()):    if n % every_nth != 0:        label.set_visible(False)ax.tick_params(axis='x', labelrotation= 90)plt.legend()plt.title('Actual vs predicted on the last 100 data points')plt.draw()
The figure supports our previous findings on good evaluation metrics the model attained, as the blue dashed line approximates the with the black solid line closely. That is, our gradient boosting model can forecast the metro traffic decently.
Meanwhile, we see that the model which does not use the date-time engineered features falls apart in performance (red dashed line). Why this occurs? Because the target (transportation traffic) would indeed depend on the features we just created. Transportation traffic tends to lower in weekend days, but spikes during rush hours (see the plot above). Thus, we would miss these sound predictors if we do not perform feature engineering on the date-time column!
Congratulations for you who have managed reading this far!
Now for a short recap. In this article, we learned how to perform feature engineering on date-time data. Afterwards, we incorporated the engineered features to build a powerful gradient boosting regression model, to forecast metro traffic volume.
Finally, thanks for reading and let’s connect with me on LinkedIn!