How to forecast trip duration using BigQuery ML
By: Fangyi Yu
Tags: Machine Learning, Google Cloud Platform, Sql, Regression, Bigquery

Google Cloud Platform

How to forecast trip duration using BigQuery ML

Photo by Lucian Alexe from Unsplash

This tutorial will go through the challenge lab in Google Cloud Qwiklab: Create ML Models with BigQuery ML: Challenge Lab.

We will use a real-life dataset shared on Google Cloud Public Dataset: austin_bikeshare to help select new bicycle models for public bike share systems. This lab aims at predicting average trip durations for different bike schemes. Two schemes are provided using different assumptions of features selected. You’ll compare which one has a better performance.

Step 1: Create a dataset to store your machine learning models

After logging into the Google Cloud Platform using the assigned credential, open BigQuery and create a dataset in your project.

Create a dataset in BigQuery (image by the author)

Step 2: Create a forecasting BigQuery machine learning model

We’ll incorporate the starting station name, the hour the trip started, the weekday of the trip, and the address of the start station in the first model and use 2018 data only to train the first model.

Model1 training:

# Create a Linear regression model CREATE or REPLACE MODEL austin_bikeshare.bikeshare_model OPTIONS (model_type='linear_reg', labels=['duration_minutes']) AS # Data preprocessing and feature selection SELECT start_station_name, EXTRACT(HOUR FROM start_time) AS hourofstart, EXTRACT(DAYOFWEEK FROM start_time) AS day_of_week, address AS location, duration_minutes FROM `bigquery-public-data.austin_bikeshare.bikeshare_trips` AS trips # Join to get the address of the start station JOIN `bigquery-public-data.austin_bikeshare.bikeshare_stations` AS stations ON trips.start_station_name=stations.name WHERE EXTRACT(YEAR FROM start_time) = 2018 AND duration_minutes > 0

Step 3: Create the second machine learning model

The features in the second model are relatively simpler. We’ll incorporate the starting station name, the hour the trip started, and the bike share subscriber type in the second model and use 2018 data only to train this model.

Model 2 training:

# Create a Linear regression model CREATE or REPLACE MODEL austin_bikeshare.bikeshare_model2 OPTIONS (model_type='linear_reg', labels=['duration_minutes']) AS # Data preprocessing and feature selection SELECT start_station_name, subscriber_type, EXTRACT(HOUR FROM start_time) AS hourofstart, duration_minutes FROM `bigquery-public-data.austin_bikeshare.bikeshare_trips` WHERE EXTRACT(YEAR FROM start_time) = 2018 AND duration_minutes > 0

Step 4: Evaluate the two machine learning models

Both Mean Absolute Error and Root Mean Squared Error are used for evaluation.

Model1 Evaluation:

# Choose evaluation metrics SELECT SQRT(mean_squared_error) AS rmse, mean_absolute_error AS mae FROM ML.EVALUATE(MODEL austin_bikeshare.bikeshare_model, ( # Prepare the testing dataset with the same features as the training set using for 2019 SELECT start_station_name, EXTRACT(HOUR FROM start_time) AS hourofstart, EXTRACT(DAYOFWEEK FROM start_time) AS day_of_week, address AS location, duration_minutes FROM `bigquery-public-data.austin_bikeshare.bikeshare_trips` AS trips JOIN `bigquery-public-data.austin_bikeshare.bikeshare_stations` AS stations ON trips.start_station_name=stations.name WHERE EXTRACT(YEAR FROM start_time) = 2019))

Evaluation for the first model (image by the author)

Model2 Evaluation:

# Choose evaluation metrics SELECT SQRT(mean_squared_error) AS rmse, mean_absolute_error AS mae FROM ML.EVALUATE(MODEL austin_bikeshare.bikeshare_model2, ( # Prepare the testing dataset with the same features as the training set using for 2019 SELECT start_station_name, subscriber_type, EXTRACT(HOUR FROM start_time) AS hourofstart, duration_minutes # No need to join anther table since all the features needed are in one table FROM `bigquery-public-data.austin_bikeshare.bikeshare_trips` WHERE EXTRACT(YEAR FROM start_time) = 2019))

Evaluation for the second model (image by the author)

From the comparison, we can find that the first model is more accurate.

Step 5: Use the second model to predict the average trip durations

We are going to use the second model to predict average trip length for trips from the busiest bike-sharing station in 2019, where the subscriber type is Single Trip .