How to import Google Sheets data into a Pandas DataFrame using Google’s API v4 (2020) | by Erik Yan | Towards Data Science

Google Sheets is a useful way to share data and collaborate remotely. But transferring the data to environments such as Python on a regular basis can be burdensome. This post will cover how to set up the latest Google Sheets API, v4 as of June 2020, for Python. We’ll also cover how to extract data from a Google Sheet range (or even an entire sheet) into a Pandas data frame.
Before you start you’ll need the following:
Python 2.6 or greater (Python 3 recommended)
Pip/pip3 package management tool (comes standard with Python 2 >= 2.7.9 or Python 3 >= 3.4)
A Google Account (and Google Sheet containing your data-of-interest).
First, you’ll need to enable the Google Sheets API on your Gmail account, where the Google Sheet is stored. Login to your Gmail account and visit the Google Sheets API QuickStart Guide for Python. You’ll see a blue “Enable Google Sheets API” button. Click on the button (marked as [1] in the image below):
Select “Desktop App” from the dropdown menu (marked as [2] in the image below) and click “Create” (marked as [3] in the image below). This will create a client configuration, which we’ll need to set up the initial connection via the API:
Click the blue “DOWNLOAD CLIENT CONFIGURATION” button (marked as [4] in the image below).
You should now have a file named “credentials.json” downloaded. You’ll need to move this file to your working directory.
Next we’ll need to install Google’s Client Library using pip:
pip install — upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib
NOTE: Rather than running the sample Python code Google provides in their guide, we’ll be using a modified version of their script.
First, we’ll set up our gsheet_api_check() function. It looks for an existing token.pickle file (which stores our user access and refresh tokens). If no token.pickle file is found, the function will prompt you to log into your Google Gmail account. The credentials.json must be present in your working directory to initiate token.pickle creation/refresh. The function generates the credentials we’ll use to make API calls:
import pickleimport os.pathfrom google_auth_oauthlib.flow import InstalledAppFlowfrom google.auth.transport.requests import Requestdef gsheet_api_check(SCOPES):    creds = None    if os.path.exists('token.pickle'):        with open('token.pickle', 'rb') as token:            creds = pickle.load(token)    if not creds or not creds.valid:        if creds and creds.expired and creds.refresh_token:            creds.refresh(Request())        else:            flow = InstalledAppFlow.from_client_secrets_file(                'credentials.json', SCOPES)            creds = flow.run_local_server(port=0)        with open('token.pickle', 'wb') as token:            pickle.dump(creds, token)    return creds
NOTE: After completing the initial credentials setup we can discard gsheet_api_check() and load our tokens directly during future API calls. Alternatively, it may be beneficial to keep gsheet_api_check() because the function checks if our tokens are expired and/or missing. If the tokens are expired/missing, gsheet_api_check() will then initiate a refresh of the user tokens (by prompting you to re-login to your Google account). You can decide whether you want to continue using gsheet_api_check() or load the tokens directly.
Next we’ll define a function that makes the API call and pulls the data we want from Google Sheets. The pull_sheet_data() function establishes the API call and pulls the data we want. If no data is found the function will print “No data found.”, otherwise it will confirm the data has been retrieved by printing “COMPLETE: Data Copied” and return our data:
from googleapiclient.discovery import builddef pull_sheet_data(SCOPES,SPREADSHEET_ID,DATA_TO_PULL):    creds = gsheet_api_check(SCOPES)    service = build('sheets', 'v4', credentials=creds)    sheet = service.spreadsheets()    result = sheet.values().get(        spreadsheetId=SPREADSHEET_ID,        range=DATA_TO_PULL).execute()    values = result.get('values', [])        if not values:        print('No data found.')    else:        rows = sheet.values().get(spreadsheetId=SPREADSHEET_ID,                                  range=DATA_TO_PULL).execute()        data = rows.get('values')        print("COMPLETE: Data copied")        return data
NOTE: pull_sheet_data() can be modified to define other API tasks such as appending data to a google sheet, updating existing data, or creating new spreadsheets.
Next we will need two pieces of information. First, we need to find and copy the ID of the spreadsheet-of-interest. This can be found in the URL of your Google Spreadsheet (marked as [5] in the image below):
Second, we’ll need the name of the spreadsheet tab from which we’ll pull the data from. Alternatively, you can explicitly define the range of cells you want to retrieve if you wish to pull specific sections of data from the spreadsheet (examples provided below):
#Pulls data from the entire spreadsheet tab.DATA_TO_PULL = 'spreadsheet_tab_name'or#Pulls data only from the specified range of cells.DATA_TO_PULL = 'spreadsheet_tab_name!A2:C6'
Finally, we’ll bring all of our code together by specifying the pull_sheet_data() parameters, running the functions, and then storing the retrieved data into a Pandas DataFrame. Be sure to replace ‘spreadsheet_url_ID ’ with the spreadsheet ID you copied, and replace ‘spreadsheet_tab_name!’ with your spreadsheet tab name (and range if necessary):
import pandas as pdSCOPES = ['https://www.googleapis.com/auth/spreadsheets']SPREADSHEET_ID = 'spreadsheet_url_ID'DATA_TO_PULL = 'spreadsheet_tab_name'data = pull_sheet_data(SCOPES,SPREADSHEET_ID,DATA_TO_PULL)df = pd.DataFrame(data[1:], columns=data[0])df
And now you’re ready to explore your retrieved data within Python, all without manually downloading or importing the dataset!