How to import Google Sheets data into a Pandas DataFrame using Google’s API v4 (2020)
By: Erik Yan
Tags: Google, Python Programming, Python, Data Science, Data

Importing Google Sheet data to a Pandas DataFrame

Google Sheets is a useful way to share data and collaborate remotely. But transferring the data to environments such as Python on a regular basis can be burdensome. This post will cover how to set up the latest Google Sheets API, v4 as of June 2020, for Python. We’ll also cover how to extract data from a Google Sheet range (or even an entire sheet) into a Pandas data frame.

Before you start:

Before you start you’ll need the following:

Python 2.6 or greater (Python 3 recommended)

Pip/pip3 package management tool (comes standard with Python 2 >= 2.7.9 or Python 3 >= 3.4)

A Google Account (and Google Sheet containing your data-of-interest).

API Setup:

First, you’ll need to enable the Google Sheets API on your Gmail account, where the Google Sheet is stored. Login to your Gmail account and visit the Google Sheets API QuickStart Guide for Python. You’ll see a blue “Enable Google Sheets API” button. Click on the button (marked as [1] in the image below):

Image by author

Select “Desktop App” from the dropdown menu (marked as [2] in the image below) and click “Create” (marked as [3] in the image below). This will create a client configuration, which we’ll need to set up the initial connection via the API:

Image by author

Click the blue “DOWNLOAD CLIENT CONFIGURATION” button (marked as [4] in the image below).

Image by author

You should now have a file named “credentials.json” downloaded. You’ll need to move this file to your working directory.

Install Google’s Client Library:

Next we’ll need to install Google’s Client Library using pip:

pip install — upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib

NOTE: Rather than running the sample Python code Google provides in their guide, we’ll be using a modified version of their script.

Generating our API user tokens:

First, we’ll set up our gsheet_api_check() function. It looks for an existing token.pickle file (which stores our user access and refresh tokens). If no token.pickle file is found, the function will prompt you to log into your Google Gmail account. The credentials.json must be present in your working directory to initiate token.pickle creation/refresh. The function will specify and return the credentials we’ll use to make API calls:

import pickle

import os.path

from google_auth_oauthlib.flow import InstalledAppFlow

from google.auth.transport.requests import Request def gsheet_api_check(SCOPES):

creds = None

if os.path.exists('token.pickle'):

with open('token.pickle', 'rb') as token:

creds = pickle.load(token) if not creds or not creds.valid:

if creds and creds.expired and creds.refresh_token:

creds.refresh(Request())

else:

flow = InstalledAppFlow.from_client_secrets_file(

'credentials.json', SCOPES)

creds = flow.run_local_server(port=0) with open('token.pickle', 'wb') as token:

pickle.dump(creds, token) return creds

NOTE: After completing the initial credentials setup we can discard gsheet_api_check() and load our tokens directly during future API calls. Alternatively, it may be beneficial to keep gsheet_api_check() because the function checks if our tokens are expired and/or missing. If the tokens are expired/missing, gsheet_api_check() will then initiate a refresh of the user tokens (by prompting you to re-login to your Google account). You can decide whether you want to continue using gsheet_api_check() or load the tokens directly.

Making the API call:

Next we’ll define a function that makes the API call and pulls the data we want from Google Sheets. The pull_sheet_data() function establishes the API call and pulls the data we want. If no data is found the function will print “No data found.”, otherwise it will confirm the data has been retrieved by printing “COMPLETE: Data Copied” and return our data:

from googleapiclient.discovery import build def pull_sheet_data(SCOPES,SPREADSHEET_ID,RANGE_NAME):

creds = gsheet_api_check(SCOPES)

service = build('sheets', 'v4', credentials=creds)

sheet = service.spreadsheets()

result = sheet.values().get(

spreadsheetId=SPREADSHEET_ID,

range=RANGE_NAME).execute()

values = result.get('values', [])



if not values:

print('No data found.')

else:

rows = sheet.values().get(spreadsheetId=SPREADSHEET_ID,

range=RANGE_NAME).execute()

data = rows.get('values')

print("COMPLETE: Data copied")

return data

NOTE: pull_sheet_data() can be modified to define other API tasks such as appending data to a google sheet, updating existing data, or creating new spreadsheets.

Running our API call and saving the data as a Pandas DataFrame:

Next we will need two pieces of information. First, we need to find and copy the ID of the spreadsheet-of-interest. This can be found in the URL of your Google Spreadsheet (marked as [5] in the image below):

Image by author

Second, we’ll need the name of the spreadsheet tab from which we’ll pull the data from. Alternatively, you can explicitly define the range of cells you want to retrieve if you wish to pull specific sections of data from the spreadsheet (examples provided below):

#Pulls data from the entire spreadsheet tab.

RANGE_NAME = 'spreadsheet_tab_name!' or #Pulls data only from the specified range of cells.

RANGE_NAME = 'spreadsheet_tab_name!A2:C6'

Finally, we’ll bring all of our code together by specifying our pull_sheet_data() parameters, running the functions, and storing the retrieved data into a Pandas DataFrame. Be sure to replace ‘spreadsheet_url_ID ’ with the spreadsheet ID you copied, and replace ‘spreadsheet_tab_name!’ with your spreadsheet tab name (and range if necessary):

import pandas as pd

SPREADSHEET_ID = 'spreadsheet_url_ID'

RANGE_NAME = 'spreadsheet_tab_name!' SCOPES = [' https://www.googleapis.com/auth/spreadsheets' SPREADSHEET_ID = 'spreadsheet_url_ID'RANGE_NAME = 'spreadsheet_tab_name!' data = pull_sheet_data(SCOPES,SPREADSHEET_ID,RANGE_NAME)

df = pd.DataFrame(data[1:], columns=data[0]) df

And now you’re ready to explore your retrieved data within Python, all without manually downloading or importing the dataset!