Super-fast Machine Learning to Production with BigQuery ML
By: David Martins
Tags: Machine Learning, Technology, Google Cloud Platform, Bigquery, Data Science

Super-fast Machine Learning to Production with BigQuery ML

How to use Bigquery ML to deploy your models in no time, and focus on what really matters. David Martins Apr 9·9 min read

Photo by Patrick Federi on Unsplash

A few months ago at Agorapulse, we kickstarted two new Machine Learning based projects. Since we’d been using BigQuery for almost 2 years, we had to give BQML a shot. Here is what we learned, and what you could learn from it!

The biggest challenges might not be what you think

As a Machine Learning engineer, you usually think about How to create the best performing model, What algorithm to use, or How to optimize the model to go from a 94% accuracy to 96%.

But if your company is at the beginning of its Machine Learning journey, it’s probably not the most important part of the process. At least not yet.

— It surely is if you’re training a self-driving car and dealing with human lives, but not for the rest of us.

First, you’ll have to face 2 under-mentioned challenges in Machine Learning.

1. Adoption & Culture Shift

In this article published by Harvard Business Review in 2019, authors say “Technology isn’t the biggest challenge. Culture is.”

When your Marketing or Product team asks for a specific insight or analysis, you can freely focus on the technical part (eg: data analysis). Since you’re replying to a question they already have, adoption shouldn’t be a problem.

But Machine Learning, for people not used to it, is perceived as a “black box”: What’s under the hood? And what can it do for me?

When launching a machine learning project, you will probably have back and forth with the users, to help on the adoption, improve interpretability, or iterate on the algorithm itself to improve the result. Adopting ML takes time.

In one of our surveys nearly 90% of the companies that had engaged in successful scaling practices had spent more than half of their analytics budgets on activities that drove adoption, such as workflow redesign, communication, and training.

— Harvard Business Review — Building the AI-Powered Organization

2. Deployment & Usability

Jupyter Notebook is only the beginning. Although most machine learning courses seem to focus on what happens inside the notebook, your company will never be able to use your work if you stop there.

In our case, we wanted to make our predictions available in some of the tools used by our teams (CRM, Sales tools, etc…), in our dashboards, our data warehouse, and our core-product database. Also, we needed to automate the ML workflows that train, test, deploy, and execute our models.

Still some way to go after Jupyter Notebook!

BigqueryML comes into play

BigQuery ML is a tool to create, execute, and server machine learning models directly in BigQuery, by using standard SQL queries.

We’ve been using BigQuery for almost 2 years, and it never stopped improving. Especially with BigQuery ML, which has been releasing tons of features and models in the past years.

It was definitely worth a try. Here are our findings!

Pros of BigQuery ML

BQML is working with SQL. You don’t have to use multiple languages, and everybody used to interact with a SQL database should be able to use it.

You can serve your models directly from BigQuery. No need to manage a workflow that’ll feed, train, and serve an external system. It’s saving a lot of ML Ops work.

At Agorapulse, we’re using Cloud Composer (GCP’s Apache Airflow) to manage our SQL workflows, along with Updating and Serving BQML Models.

At Agorapulse, we’re using Cloud Composer (GCP’s Apache Airflow) to manage our SQL workflows, along with Updating and Serving BQML Models. It’s quick and easy to test. You don’t have to extract data to you local machine to give it a try. Neither to be an expert machine learning engineer in order to execute a model and get good results.

Documentation is really good. Everything is detailed, and it contains tons of examples: https://cloud.google.com/bigquery-ml/docs

Cons of BigQuery ML

BQML can be a bit of a black box. It’s hard to know what it does under the hood — hopefully, they have inspection functions to get some insights. Which makes it hard to optimize your dataset before injecting it into Bigquery ML, or to fine-tune your models using the various options.

It’s less flexible. If you’re an experienced data scientist used to work with Python (or R) and its tons of libraries, you’ll probably feel limited. There are so many ML models out there that are not available on BQML, and tons of libraries to optimize your models. It’s a trade-off since it’ll take way more time to come up with and deploy something on your own.

Note that you can still create a model with TensorFlow, then import and serve it with BigqueryML

Note that you can still create a model with TensorFlow, then import and serve it with BigqueryML Obviously, it has its limitations. Your data have to be inside BigQuery. And you can only serve your results by batch: it’s not adapted, for instance, if you wanna expose an API endpoint to execute a model on the fly. Even if you can export a model.

BigQuery ML is evolving so fast, and what I found out to be cons could be greatly improved in the coming months.

Is it worth it?

Well, for us who were already using BigQuery, and wanted to focus on the business value as soon as possible: we surely had to experiment with BQML!

How does it work?

Let’s take as an example a real project that we’ve worked on at Agorapulse. For the sake of the example, we’ll use a simplified dataset and a basic BQML model (no complex options and fine-tuning).

To give some necessary context, Agorapulse is a Saas tool that helps businesses manage their social media across Twitter, LinkedIn, Instagram, Facebook, and YouTube. Based on Machine Learning, we wanted to know if a company that tested our SaaS during the free trial period was likely to convert into a paying customer.

Create your model

As for every ML project, you’ll start by preparing your data. It might often be the most time-consuming part of the process. In our case, we are using Airflow (Cloud Composer) to create the data-prep workflows.

Here is the simple dataset that we’ll use:

For every Company trying our product, we have the number of people using our Saas, the language used inside the product, the country, how many social profiles were connected, and a boolean (0 or 1) to tell if the company tested some of our main features: publishing on a social network, managing your inbox, creating/exporting reports about your social accounts.

Finally, the value to predict: has the company converted or not?

Then, you’ll have to choose a model. As written in the documentation, BQML supports the following types of models:

Linear regression

Binary and Multi-Class Logistic Regression

K-means clustering

Matrix Factorization

Time series

Boosted Tree (XGBoost based models)

Deep Neural Network (DNN)

AutoML Tables

TensorFlow model importing

Several models would be interesting to test to solve our example problem, including Logistic Regression, Booster Tree, and even DNN for classification.

Let's continue with a Binary Logistic Regression.

CREATE OR REPLACE MODEL `medium.trial_to_paid_prediction_model`

OPTIONS

(

model_type='LOGISTIC_REG',

input_label_cols=['converted'],

auto_class_weights=TRUE

)

AS

SELECT

*

FROM

`medium.trial_to_paid_prediction_dataset`

That’s all it takes to create our model!

A few takeaways

The only mandatory options are model_type, to choose a model. And input_label_cols to select the value to predict — Although if you name the column to predict ‘label’ in your dataset, it does the same job.

auto_class_weights is used to balance class labels using weights. It’s defaulting to false, but since we have a very unbalanced dataset (6% of the companies have converted), the default could lead the model to always predict false — which would actually result in an “accuracy” of 94% 😉

Dozens of options are available to optimize your model. You can even preprocess your data during the model creation using transform.

Inspections functions, such as ML.FEATURE_INFO or ML.TRAINING_INFO will give you more information about your model and its features.

or will give you more information about your model and its features. BQML deals with non-numerical values with a simple one-hot encoding.

You can see how it handles the different features by using:

SELECT *

FROM ML.WEIGHTS(MODEL `medium.trial_to_paid_prediction_model`)

Evaluate your Model

Now, you can access the evaluation metrics by using:

SELECT

*

FROM

ML.EVALUATE(MODEL `medium.trial_to_paid_prediction_model`)

Or if you want to specify a threshold or some input data:

SELECT

*

FROM

ML.EVALUATE (MODEL `medium.trial_to_paid_prediction_model`,

(

SELECT

*

FROM

`medium.trial_to_paid_prediction_dataset`

),

STRUCT(0.55 AS threshold)

)

Depending on the model you choose, the evaluation metrics might change.

At first glance, the result looks pretty good for a model created in a few SQL commands! And keep in mind that in this example in which we only used 7 features to feed our model.

A few takeaways

Evaluation metrics by themselves aren’t enough to solve a problem. It depends on the problem itself: in our case, if we want to identify the top 5% of free trial users with the highest probability to convert and feed it to our sales team, we might increase our threshold, which will mechanically increase the precision and lower the recall.

If what we want is to identify those with even the slightest chance to convert, we could lower our threshold, which will lead to lower the precision and increase the recall.

If what we want is to identify those with even the slightest chance to convert, we could lower our threshold, which will lead to lower the precision and increase the recall. Depending on the model you use, the evaluation metrics might be different. In the case of logistic regression, you can also check the confusion matrix using ML.CONFUSION_MATRIX

Execute your Model

Now that your model is ready, it’s time to execute and serve it!

Once again with BigQuery ML, you simply need one command:

SELECT *

FROM

ML.PREDICT(MODEL `medium.trial_to_paid_prediction_model`,

(

SELECT

*

FROM

`medium.trial_to_paid_prediction_new_data`

),

STRUCT(0.55 AS threshold) #Optional - Default to 0.5

)

And that’s it! Here is how BigQuery displays the result:

The output uses BigQuery Arrays. Use the UNNEST() function to access the nested values.

Since we used a logistic regression model, BigQuery ML gives us the predicted values based on the threshold we choose. But also the probabilities for each value, that you also might wanna use.

Serve your model to the world

We’re ready to serve the model to the world. And since there is technically no extra step to “deploy” it, you can focus on what matters the most: empowering your teams with ML and providing value to the business.

At Agorapulse, we use Airflow to execute our ML workflows, which consist of:

Periodically retraining our models with the most recent data

Execute our models daily on new data, and send the results in the different places where we need it — CRM, Sales tools, etc…

Conclusion

If like us, you’re already using BigQuery, and want to kickstart your first machine learning project, no doubt that you should experiment with BQML.

You’ll have your first model up and running in days instead of weeks, and be able to quickly focus on what’s important at that stage: showing your team the value of ML, working on the necessary cultural shift for its adoption, and develop the technical workflows to include ML in your “systems”.

Once ML is adopted and the first results are here, you’ll still have plenty of time to improve the model itself, and maybe at some point, without BQML.

At the time I write this article, we’re now focussing on the next steps: Driving adoption by our teams of the few Machine Learning based project that we launched. Creating a virtuous circle to improve our models based on feedback. Providing every tool and team with the results of our models. Triggering automation based on those results — and a lot more to come.