Forecasting with web traffic data | by Sheenal Srivastava | Towards Data Science

Typically, when you have access to time series data, in this case, we are going to look at an example where we have one week’s worth of web traffic data, it is instinctive to apply a time series model. However, this does not always need to be the case and I will demonstrate why further in this blog post.
In this post, you will learn how to:
Set up your analytics dataset to forecast an event in the future — i.e. whether or not a web user will become a lead the next day
Create additional features to assist in the prediction
Identify and plot feature importance
Utilize a powerful classification technique for time series forecasting
Data Problem: Predict whether a website user will become a lead the next day?
Dataset: A dataset of a 1 week sample of website traffic was provided consisting of three variables: user_id, event (“view”, “search”, or “lead”) and timestamp (date and time)
Proposed method: In order to predict the likelihood of whether a user will submit a lead the next day, we need to understand the user’s previous day’s or previous several days’ activities as well as aggregated activity on the website. For this we need to create features.
library(data.table)library(dplyr)library(padr)library(xgboost)library(Matrix)library(RcppRoll)library(zoo)library(readr)library(sqldf)data_interactions<-read_csv("C:\\Users\\srivastavas\\Downloads\\data-interactions\\data-interactions.csv")
Once the dataset is loaded, it always helps to view the dataset to ensure it has been read in correctly. As shown below, we can see that the dataset consists of four columns: the type of event (“view”, “search” or “lead”) , the user id (unique identifier), timestamp (time and date when user accessed the website) and just the date.
# A tibble: 6 x 4  event  userid                               utc_timestamp       date        <fct>  <chr>                                <dttm>              <date>    1 view   4c02df98-fee3-43dd-8582-4192180a52c8 2019-01-17 03:49:36 2019-01-172 search bcc02079-d9d6-461a-bb7a-6ad679dc3ddb 2019-01-19 22:33:14 2019-01-193 search 4443b108-11aa-44e8-9934-a9c75e7a5b2e 2019-01-20 06:54:42 2019-01-204 search a8513b28-c94b-49b4-9260-07f9ce8a24aa 2019-01-17 23:33:44 2019-01-175 search 261eb70e-af13-4d0c-a2d8-7c2095586b2d 2019-01-20 01:28:01 2019-01-206 search 7d670d0a-f815-4ce1-955a-f026f478ebb5 2019-01-17 22:42:44 2019-01-17
Time variables and unique identifiers are what I like to call “junk” variables. They do not assist the model in prediction. However, time is an important feature in our case as we want to know whether a user will become a lead the next day. An individual user id is also important. So, how can we still use this information.
This brings me to the next part of modelling — feature engineering.
As a user may view or search a website one or more times a day, it makes sense to aggregate this information to reduce the number of records per user but also to be able to create a daily dataset — i.e. one record per day per user. This can be done using R’s dplyr package or sqldf (writing SQL code in R).
In the below code snippet, I have grouped the dataset by date and userid to calculate the number of views, number of searches and number of leads per day per user.
sql_str<-"select userid, date       , SUM(CASE WHEN event = 'leads' THEN 1 ELSE 0 END) AS num_leads       , SUM(CASE WHEN event = 'view' THEN 1 ELSE 0 END) AS num_views       , SUM(CASE WHEN event = 'search' THEN 1 ELSE 0 END) AS num_searches       FROM unique_data_interactions       GROUP BY userid, date"df1<-sqldf(sql_str,stringsAsFactors = FALSE)
Next, in order to use the date variable meaningfully, we can create time-related variables such as day when website was accessed, hour when it was accessed, month of access and week of access. Since, we only have one week’s worth of day, only day of access (i.e. Sun vs. Mon) may be useful. The below code can be used to create time-related variables.
df1$day<-as.numeric(as.factor(weekdays(as.Date(df1$date))))
Now, because we want to predict whether a user will become a lead the next day, we want to know about the user’s previous activity. For instance, did the user view the website the previous day, or the day before, three days before? Did they also search the website previously?
To use previous activity in order to predict future activity, we need to create lag variables. As we want to predict whether a user will become a lead the next day, we use the previous day’s activity to predict the next day’s activity.
Firstly, you need to order the data by date. Then, you want to group the data by userid as you want to predict whether a given user will become a lead the next day.
To predict whether a user becomes a lead the next day, we need to know whether the customer was a lead the previous day (lag num_leads by 1), whether they viewed and searched the website the previous day (lag num_views and num_searches by 1), and total number of views and searches to current date (cum_sum_views and cum_views_searches).
df1<-df1[order(df1$date),]#Create lag variablesdf2<-df1 %>%  group_by(userid) %>%  mutate(lag_leads_1 = lag(num_leads,1) #lag target variable by 1        , cum_sum_views = cumsum(num_views)        , cum_sum_searches = cumsum(num_searches)         , lag_views_1 = lag(num_views,1) #lag num_views by 1 day         , lag_searches_1 = lag(num_searches,1)#lag num_searches by 1 day          )
In temporal data, it is common to create variables such as rolling mean averages, rolling median averages, and various lag variables, but since we have access to only one week’s worth of data, these variables do not seem reasonable.
Now, we can only predict whether a user becomes a lead the next day if we have previous day information of the user. For instance, if the user only shows up in the dataset at the end of the week (the 7th day), it is difficult to test our model accuracy for this user as we do not have information on whether this user became a lead. Hence, such records were removed from the dataset.
#remove users who don't have previous day records as can't predict next day lead for themdf3<-df2[!is.na(df2$lag_leads_1),]
The second thing to do is to then decide on a model. As the problem is to determine whether or not a user a becomes a lead the next day, it is a binary problem and also a classification problem — i.e. user becomes a lead = Success = “1”, user does not become a lead = Failure = “0”. To model this problem, we need to feature engineer this target variable as follows.
df3$lead_flag<-ifelse(df3$num_leads>=1, 1, 0)
We have removed duplicate records, created a daily dataset with one record per user per day, and created time-dependent and our target variable (what we want to predict).
The next step is to now decide how we are going to train and test our model. In most cases, it makes sense to split the data at random — i.e. a random subset of rows become the training set and the remainder become the test set. This way your model is trained on a dataset on which it learns and then tested for accuracy on data it has never seen.
However, when you have temporal data, you are predicting for a time in the future, so the dataset should be split based on time, i.e. first 75% of the dataset is used for training (i.e. Monday to Thursday) and the remainder is used for test (Friday to Sunday). As our dataset is for only one week, this split does not seem reasonable.
In our example, we want to predict whether a particular user will become a lead. So, it makes sense to split the dataset by userids, where the model is trained on a particular set of users and then tested for accuracy on users it has never encountered. To split the data in this way, the following code was written.
#Subset data into training and test sets - split on different usersuser_ids<-as.data.frame(unique(df3$userid))names(user_ids)<-"user_ids"train_user_ids<-sample_frac(user_ids,0.75)names(train_user_ids)<-"user_ids"train <-df3[df3$userid %in% train_user_ids$user_ids,]test <- anti_join(df3, train, by = c("userid" = "userid"))
Once you have your training set and test, you can now start modelling! :)
For binary classification, you need a classification model. Tree-based methods are commonly used in binary classification. A popular tree-based method is xgboost that learns from each new tree (iteration) that is created by trying to reduce the percentage of error. Unlike a random forest, where the final result is an ensemble of trees or an average weighted tree, the final xgboost decision tree is one that has the lowest error rate out of all its rounds.
Both randomforest and xgboost are known to overfit data as they try to improve on previous iterations; however, it is up to the modeller to determine whether the variables provided to the model are independent (not correlated), informative and not biasing the results in any way.
The xgboost model in R requires a matrix and defined parameters as shown in the below code snippet.
The following steps are required for an xgboost model:
Define the target variable — “label”
Remove NAs
Create a matrix with variables you want to include in the model (contrasts.arg is used when you have nominal/categorical variables in your dataset)
Set parameters for your model — binary logisitic function used as it’s a classification problem, don’t want a very deep tree (causes over-fitting so have only specified four levels; 2 and 3 are usually good when you have a small set of variables)
set up modellabel <- train$lead_flag#returns object unchanged if there are NA valuesprevious_na_action<-options('na.action')options(na.action='na.pass')#build matrix input for the modeltrainMatrix<- sparse.model.matrix(~num_views+num_searches+day+lag_leads_1+cum_sum_views+cum_sum_searches                                  +lag_views_1+lag_searches_1                                  , data = train,                                   , contrasts.arg = c('day')                                  , sparse = FALSE, sci = FALSE)options(na.action=previous_na_action$na.action)#create input for xgboosttrainDMatrix <- xgb.DMatrix(data=trainMatrix, label = label)#set parameters of model# Define the parameters for binomial classificationparams <- list(booster = "gbtree",                objective = "binary:logistic",                eta=0.3,                gamma=0,                max_depth=4,                min_child_weight=1,                subsample=1,                colsample_bytree=1)
While your model is training, you can test its accuracy using a method called cross-validation. In cross-validation, you run multiple different training test splits and then average the results, instead of relying entirely on a single particular training set. The most common type of cross-validation is k-fold cross-validation most commonly with K set to 5 or 10. For example, to do five-fold cross-validation, the original dataset is partitioned into five parts of equal or close to equal size. Each of these parts is called a “fold”. Then a series of five models is trained one per fold. The first model: Model one, is trained using folds 2 through 5 as the training set and evaluated using fold 1 as the test set. The second model: Model 2, is trained using Folds 1, 3, 4, and 5 as the training set, and evaluated using Fold 2 as the test set, and so on. When this process is done, we have five accuracy values, one per fold. It is common to take the average of the accuracy scores.
You can set up your model for cross-fold validation as follows. Here we have specified five rounds of cross-validation, told the model to run 1000 iterations but to stop early (at the 20th round) if it no longer shows any improvement in accuracy.
The final line tells us at which iteration did the model produce a tree with lowest error. The output in our case is the 40th iteration.
xgbcv <- xgb.cv( params = params, data = trainDMatrix, nrounds = 1000,                   nfold = 5, showsd = T, stratified = T,                   print_every_n = 10, early_stopping_rounds = 20, maximize = F)num_iterations = xgbcv$best_iteration
Now, we can use this information to run our model on the training set and get it to stop at the 40th iteration. We have also told our model to use the built-in error metric to evaluate the tree as well as the logloss metric.
model <-xgb.train(data=trainDMatrix                  , params=params                  , nrounds = num_iterations                  , maximize = FALSE                  , eval.metric = "error", eval.metric = "logloss")
A feature importance plot tells us how the variables in the model were used to predict whether or not a user will become a lead the next day.
Here, we can see that the number of views and number of searches on the day are the most important predictors in predicting whether or not a customer will become a lead on the same day. We can also see that despite creating lag and cumulative variables, they are not very useful. However, this was expected as the dataset covers a very short timeframe of one week.
importance<-xgb.importance(feature_names=colnames(trainDMatrix), model=model)xgb.ggplot.importance(importance_matrix=importance)
The next step is to test for model accuracy. This requires setting up the test dataset as a matrix and running the command “predict” on the test set using the trained model.
To test for model accuracy, here we compare how many of the lead_indicator values were classified correctly as “0” or “1” by the model. The “pred” dataset is recoded as the model outputs probabilities. Any probability greater than 0.5 is converted to “1” (is a lead) and any value less than 0.5 is converted to “0” (not a lead). We sum all the instances where the predicted value (“prediction”) is equal to the original value in the test set (testMatrix$label), and divide it by the size of the test set.
Here, we get an accuracy rate of 97.53%, or an error rate of 0.03, which means that 97.53% of all users in the test set were classified correctly as whether or not they will become a lead the next day.
testMatrix<- sparse.model.matrix(~num_views+num_searches+day+lag_leads_1+cum_sum_views+cum_sum_searches                                  +lag_views_1+lag_searches_1                                  , data = test,                                   , contrasts.arg = c('day')                                  , sparse = FALSE, sci = FALSE)options(na.action=previous_na_action$na.action)pred<-predict(model,testMatrix)prediction <- as.numeric(pred > 0.5)print(head(prediction))err <- mean(as.numeric(pred > 0.5) != testMatrix$label)print(paste("test-error=", err)) #average error is 0.03 very low so  model is goodresult = sum(prediction==test$lead_flag)/length(test$lead_flag)print(paste("Final Accuracy =",sprintf("%1.2f%%", 100*result))) #97.53%
If you like me can’t believe your eyes, it is helpful to combine the test set with the predictions to see the information side by side as follows. In the below image, you can see that for the 15th of January 2019, all users did not become leads on this day and the prediction is “0” (non-lead) for these users.
If our model had shown reduced accuracy, we can play with the xgboost parameters such as tree depth, number of rounds, minimum number of rows used for tree branch splitting. However, in this case, with such good accuracy, this was not required.
Hopefully, you now have a good understanding of how to model time-series data of a classification nature! Any questions, please comment below.
The full code file can be found here: