Package: importio_gsei
IMPORT.IO GOOGLE SHEETS EXTRACTOR INTEGRATIONThis repository includes a script solution that uses URLs taken from aGoogle sheet and copies to an existing Import.io Extractor, then lastlyexecutes the Extractor to collect data from the supplied URLs.The command script is implemented using Python, which can be executed onany operating system that supports a Python version 3.5 or later.InstallationThe command requires the installation of Python 3.5 or later with someadditional third-party packages that can be installed using pip.$ pip install importio_gseiConfigurationBefore the integration can be used you need to setup the Google SheetsAPI and Import.io key.Import.io AuthenticationThe Import.io API needs to be made available via an environment variablethat can be configured as follows:$ export IMPORT_IO_API_KEY=<api key>Google Sheets API SetupGenerate CredentialsNOTE: You must have Google Account to use the Google Sheets API, you cansign up here.1.  Use this wizard to create or select a project in the GoogleDevelopers Console and automatically turn on the API. ClickCONTINUE, then GO TO CREDENTIALS.2.  On the ADD CREDENTIALS TO YOUR PROJECT page, click theCancel button.3.  At the top of the page, select the OAuth consent screen tab.4.  Select an Email address, enter a Product name if not already set,and click the Save button.5.  Select the CREDENTIALS tab, click the CREATE CREDENTIALS button andselect OAUTH CLIENT ID.6.  Select the application type OTHER, enter the name "Google SheetsExtractor Integration", and click the Create button.7.  Click OK to dismiss the resulting dialog.8.  Click the file_download (Download JSON) button to the right of theclient ID.9.  Move this file to a known directory on your computer and rename toclient-secret.jsonAuthorize Access1.  Authorize access to the Google Sheets API by running the followingcommand:$ gextractor authorize -f client-secret.json -a 'Google Sheets Extractor Integration'2.  This will launch a browser and display an authorization page(see below).3.  Click on the ALLOW button.4.  Close the browser window.[Google Authorization Page]OperationThe command has 7 different sub-commands that are invoked similiarly asfollows:$ gsextractor <sub-command>where sub-command is one of the following:-   _authorize_ - Performs the Oauth handshake to allow permit access toGoogle Sheets API-   _copy-urls_ - Copies URLs from a specified Google sheet todesignated Extractor.-   _extractor-start_ - Initiates a crawl-run of an Extractor.-   _extractor-status_ - Provides a status of the crawl-run(s) ofan Extractor.-   _extractor-urls_ - Displays the list of URLs associated withan Extractor.-   _extract_ - Performs the complete extraction operation from copyingthe URLs from the Google sheet to running the Extractor to extractdata from web pages.-   _sheet-urls_ - Displays a list of the URLs in a Google sheet giventhe sheet id and range.Help can be displayed which shows the available sub-commands as follows:gsextractor -husage: gsextractor [-h] [-v]{authorize,copy-urls,extract,extractor-start,extractor-status,extractor-urls,sheet-urls}...Import.io Google Sheets Extractor Integrationpositional arguments:{authorize,copy-urls,extract,extractor-start,extractor-status,extractor-urls,sheet-urls}commandsauthorize           Configures authentication Google Sheets APIcopy-urls           Copies URLs from google sheet to an extractorextract             Runs the full extraction processextractor-start     Starts an extractorextractor-status    Displays the status of recent craw runsextractor-urls      Displays the URLs from an extractorsheet-urls          Displays the URLs from a google sheetoptional arguments:-h, --help            show this help message and exit-v, --version         show program's version number and exitDetail help on each of the commands is provided by the -h with thecorresponding sub-command:$ gsextractor <sub-command> -hThe version of the program can be displayed by running the command withthe -v option:$ gsextractor <sub-command> -v0.4.0authConfigures OAuth$ gsextractor auth -c <path to credentials file>copy-urlsCopy the URLs in the Google Sheet to the Extractor URLs$ gsextractor copy-urls -i <spreasdheet_id> -r <spreadsheet_range> -e <extractor_id>NOTE: _spreadsheet_range_ needs to be surrounded by single quotes toprevent a range of theextractor-startInitiate a crawl-run on a specific Extractor$ gsextractor extractor-start -e <extractor_id>extractor-statusDisplay the status of a crawl-run(s) on a specific Extractor$ gsextractor extractor-status  -e <extractor id>extractor-urlsDisplays the URLs associated with a specific Extractor$ gsextractor extractor-urls  -e <extractor id>extractRuns the complete operation of copying URLs from a Google sheet to anExtractor, and starting the Extractor$ gsextractor extract -i <spreasdheet_id> -r <spreadsheet_range> -e <extractor_id>sheet-urlsDisplays the URLs from a specifice Google sheet and range$ gsextractor sheeturls -i <spreasdheet_id> -r <spreadsheet_range>Programmatic ExecutionThe same operations as listed above can performed programmatically fromPython similar to the following example:from importio_gsei import GsExtractorUrlsg = GsExtractorUrls()g.extractor_start(extractor_id)copy_urls()from importio_gsei import GsExtractorUrlsg = GsExtractorUrls()g.copy_urls(spread_sheet_id, spread_sheet_range, extractor_id)extractor_start()from importio_gsei GsExtractorUrlsg = GsExtractorUrls()g.extractor_start(extractor_id)extractor_status()from importio_gsei import GsExtractorUrlsg = GsExtractorUrls()crawl_runs = g.extractor_status(extractor_id)for crawl_run in crawl_runs:print(crawl_run)extractor_urls()from importio_gsei import GsExtractorUrlsg = GsExtractorUrls()urls = g.extractor_urls(extractor_id)for url in urls:print(url)extractfrom importio_gsei import GsExtractorUrlsg = GsExtractorUrls()g.extract(spread_sheet_id, spread_sheet_range, extractor_id)sheet_urlsfrom importio_gsei import GsExtractorUrlsg = GsExtractorUrls()urls = g.extract(spread_sheet_id, spread_sheet_range)for url in urls:print(url)