Category: noteboook
Question: How to use Kaggle Notebooks with Google Cloud Services (GCS)?
Answer: Kaggle currently has integrations with the Google Cloud Storage, BigQuery, and AutoML products. To enable these integrations, click on the “Add-ons” menu in the notebook editor and select “Google Cloud Services”. Once on the “Google Cloud Services” page, you will need to attach your account to your notebook and select which of the integrations you want to enable. After enabling these integrations, you will be provided with a code snippet that can be copied and pasted into your notebook.

> *Some of these services incur charges to attached GCP accounts. Please review pricing for each of the following products before you begin to use them in your notebook.*

Each line of this code snippet corresponds to a different Google Cloud Services Integration where |PROJECT_ID| should be an existing Google Cloud Project. Per AutoML docs (linked below), AutoML currently requires that the location (|COMPUTE_REGION|) must be `us-central1` for your GCS Bucket.

For more information on how to use these services, please refer to [Google Cloud Documentation](https://cloud.google.com/docs/) or any of the specific product documentation.

## BigQuery

[**BQ Documentation**](https://cloud.google.com/bigquery/docs/), [**BQML Documentation**](https://cloud.google.com/bigquery-ml/docs/bigqueryml-intro)

Google BigQuery is a fully managed, petabyte scale, low-cost analytics data warehouse. There is no management required for users—instead, users can focus solely on analyzing data through queries and BigQuery ML to find meaningful insights in a pay-as-you-go billing model.

Google BigQuery can be accessed using Kaggle’s free-tier account to query [public data](https://console.cloud.google.com/marketplace/browse?filter=solution-type:dataset&_ga=2.188761902.446093747.1583860775-118720642.1583860775) but requires a [billing-enabled](https://cloud.google.com/billing/docs/how-to/modify-project) GCP account to query any data that isn’t publicly released by BigQuery. You should carefully review the prices of BigQuery before trying the integration in Kaggle Notebooks, as it can be easy to incur charges.

```python
# Set your own project id here
PROJECT_ID = 'your-google-cloud-project'
from google.cloud import bigquery
bigquery_client = bigquery.Client(project=PROJECT_ID)
```

For a more in-depth walkthrough of using the integration, please refer to the following notebooks:
- [BigQuery in Kaggle Notebooks](https://www.kaggle.com/jessicali9530/tutorial-how-to-use-bigquery-in-kaggle-kernels)
- [BigQuery Machine Learning Tutorial](https://www.kaggle.com/rtatman/bigquery-machine-learning-tutorial)

## Google Cloud Storage (GCS)

[**GCS Documentation**](https://cloud.google.com/storage/docs/)

Google Cloud Storage allows for storage and retrieval of data at any time across the globe. Users are able to use the storage space for any type of data and only pay for used storage space (per GB per month).

Google Cloud Storage is a paid service and requires a [billing-enabled](https://cloud.google.com/billing/docs/how-to/modify-project) GCP account. You should carefully review the prices of GCS before trying the integration in Kaggle Notebooks, as it can be easy to incur charges.

```python
# Set your own project id here
PROJECT_ID = 'your-google-cloud-project'
from google.cloud import storage
storage_client = storage.Client(project=PROJECT_ID)
```

For a more in-depth walkthrough of using the integration, please refer to the following notebooks:
- [Moving Data to/from GCS](https://www.kaggle.com/paultimothymooney/how-to-move-data-from-kaggle-to-gcs-and-back)

## AutoML

[**AutoML Documentation**](https://cloud.google.com/automl/docs/)

Google AutoML is a suite of products that enables users to train custom machine learning models for tasks on structured data, vision, and language. It is currently in Beta, so you may encounter usability frictions or known issues. We welcome all feedback from the community. User feedback will help us improve documentation and be shared directly with the AutoML team to help improve the product.

Google AutoML is a paid service and requires a [billing-enabled](https://cloud.google.com/billing/docs/how-to/modify-project) GCP account. You should carefully review the prices of AutoML before trying the integration in Kaggle Notebooks, as it can be easy to incur charges. You can see the pricing for each of the offerings in beta [here](https://cloud.google.com/products/#product-launch-stages):

```python
# Set your own project id and compute region here
PROJECT_ID = 'your-google-cloud-project'
COMPUTE_REGION = 'us-central1' # must be `us-central1` to use AutoML (see docs)
from google.cloud import automl_v1beta1 as automl
automl_client = automl.AutoMlClient()
project_location = automl_client.location_path(PROJECT_ID, COMPUTE_REGION)
```

For a more in-depth walkthrough of using the integration, please refer to the following notebooks:
- [AutoML Tables Tutorial](https://www.kaggle.com/devvret/automl-tables-tutorial-notebook)

## Google Cloud AI Notebooks

If you run into compute constraints while using notebooks on Kaggle, you can consider upgrading to Google Cloud AI Notebooks. These notebooks run under your project in Google Cloud and can be configured to use your choice of virtual machine, accelerators, and run without limits.

To export your notebook to Google Cloud, you can go to the **File** menu and select "Upgrade to Google Cloud AI Notebooks" from within the Notebooks Editor. You can also upgrade a notebook from the Viewer by clicking on the three-dot menu on the top right.

For a more detailed description of how to export your Kaggle Notebooks to Google Cloud AI Notebooks, check out the announcement post [here](https://www.kaggle.com/product-feedback/159602).