Package: gspread-pandas
author: Diego FernandezLinks:DocumentationSource codeShort video tutorialAttention!Looking for maintainerI don’t really use this package anymore and with my current lifestyle I don’t
have much time to maintain this project. The library as-is only supports
gspread version <6. Updating this to support v6 might involve a fair bit of work
which I unfortunately cant do. I’m happy to work with anyone who might want to
maintain, or I can still accept PRs.OverviewA package to easily open an instance of a Google spreadsheet and
interact with worksheets through Pandas DataFrames. It enables you to
easily pull data from Google spreadsheets into DataFrames as well as
push data into spreadsheets from DataFrames. It leveragesgspreadin the backend for
most of the heavylifting, but it has a lot of added functionality
to handle things specific to working with DataFrames as well as
some extra nice to have features.The target audience are Data Analysts and Data Scientists, but it can also
be used by Data Engineers or anyone trying to automate workflows with Google
Sheets and Pandas.Some key goals/features:Be easy to use interactively, with good docstrings and auto-completionNicely handle headers and indexes (including multi-level headers and merged cells)Run on Jupyter, headless server, and/or scriptsAllow storing different user credentials or using Service AccountsAutomatically handle token refreshesEnable handling of frozen rows and columnsEnable filling in all merged cells when pulling dataNicely handle large data sets and auto-retriesEnable creation of filtersHandle retries when exceeding 100 second user quotaWhen pushing DataFrames with MultiIndex columns, allow merging or flattening headersAbility to handle Spreadsheet permissionsAbility to specifyValueInputOptionandValueRenderOptionfor specific columnsInstallation / UsageTo install use pip:$pipinstallgspread-pandasOr clone the repo:$gitclonehttps://github.com/aiguofer/gspread-pandas.git$pythonsetup.pyinstallBefore using, you will need to download Google client credentials for
your app.Client CredentialsTo allow a script to use Google Drive API we need to authenticate our
self towards Google. To do so, we need to create a project, describing
the tool and generate credentials. Please use your web browser and go toGoogle consoleand :ChooseCreate Projectin popup menu on the top.A dialog box appears, so give your project a name and click onCreatebutton.On the left-side menu click onAPI Manager.A table of available APIs is shown. SwitchDrive APIand click onEnable APIbutton. Do the same forSheets API. Other APIs might
be switched off, for our purpose.On the left-side menu click onCredentials.In sectionOAuth consent screenselect your email address and
give your product a name. Then click onSavebutton.In sectionCredentialsclick onAdd credentialsand switchOAuth client ID(if you want to use your own account or enable
the use of multiple accounts) orService account key(if you prefer
to have a service account interacting with spreadsheets).If you selectOAuth client ID:SelectApplication typeitem asDesktop appand give it a name.Click onCreatebutton.Click onDownload JSONicon on the right side of createdOAuth client IDsand store the downloaded file on your file system.If you selectService account keyClick onService accountdropdown and selectNew service accountGive it aService account nameand ignore theRoledropdown
(unless you know you need this for something else, it’s not necessary for
working with spreadsheets)Note theService account IDas you might need to give that user
permission to interact with your spreadsheetsLeaveKey typeasJSONClickCreateand store the downloaded file on your file system.Please be aware, the file contains your private credentials, so take
care of the file in the same way you care of your private SSH key;
Move the downloaded JSON to~/.config/gspread_pandas/google_secret.json(or you can configure the directory and file name by directly callinggspread_pandas.conf.get_configThanks to similar projectdf2gspreadfor this great
description of how to get the client credentials.You can read more about it in theconfiguration docsincluding how to change the default behavior.Exampleimportpandasaspdfromgspread_pandasimportSpread,Clientfile_name="http://stats.idre.ucla.edu/stat/data/binary.csv"df=pd.read_csv(file_name)# 'Example Spreadsheet' needs to already exist and your user must have access to itspread=Spread('Example Spreadsheet')# This will ask to authenticate if you haven't done so before# Display available worksheetsspread.sheets# Save DataFrame to worksheet 'New Test Sheet', create it first if it doesn't existspread.df_to_sheet(df,index=False,sheet='New Test Sheet',start='A2',replace=True)spread.update_cells('A1','B1',['Created by:',spread.email])print(spread)# <gspread_pandas.client.Spread - User: '<example_user>@gmail.com', Spread: 'Example Spreadsheet', Sheet: 'New Test Sheet'># You can now first instanciate a Client separately and query folders and# instanciate other Spread objects by passing in the Clientclient=Client()# Assumming you have a dir called 'example dir' with sheets in itavailable_sheets=client.find_spreadsheet_files_in_folders('example dir')spreads=[]forsheetinavailable_sheets.get('example dir',[]):spreads.append(Spread(sheet['id'],client=client))TroubleshootingEOFError in RodeoIf you’re trying to usegspread_pandasfrom withinRodeoyou might get anEOFError: EOF when reading a lineerror when trying to pass in the verification
code. The workaround for this is to first verify your account in a regular shell.
Since you’re just doing this to get your Oauth token, the spreadsheet doesn’t need
to be valid. Just run this in shell:python-c"from gspread_pandas import Spread; Spread('<user_key>','')"Then follow the instructions to create and store the OAuth creds.This action would increase the number of cells in the workbook above the limit of 10000000 cells.IMO, Google sheets is not the right tool for large datasets. However, there’s probably good reaons
you might have to use it in such cases. When uploading a large DataFrame, you might run into this
error.By default,Spread.df_to_sheetwill add rows and/or columns needed to accomodate the DataFrame.
Since a new sheet contains a fairly large number of columns, if you’re uploading a DF with lots of
rows you might exceed the max number of cells in a worksheet even if your data does not. In order
to fix this you have 2 options:The easiest is to passreplace=True, which will first resize the worksheet and clear out all values.Another option is to first resize to 1x1 usingSpread.sheet.resize(1, 1)and then dodf_to_sheetThere’s a strange caveat with resizing, so going to 1x1 first is recommended (replace=Truealready does this). To read more seethis issue