Package: ducktables
DuckTables: Python Functions for DuckDBDuckTables is Python library that provides out of the box functions for theDuckDB PyTablesextension.InstallationpipinstallducktablesAuthenticationDuckTables interacts with a number of different services, and each has their own authentication and configuration mechanism. Note that you only have to configure the services that you plan on using.AWSFollow theauth instructionsfor Boto3, the library that ducktables uses to interface with AWS apis. Note that if you're already using boto3 or the AWS CLI, your existing authentication configuration will be utilized, and you don't need to do anything.GithubWhen starting your DuckDB client (such as the DuckDB cli), set an environment variable namedGITHUB_ACCESS_TOKEN. From the command line, this will look like:GITHUB_ACCESS_TOKEN=<token>duckdb-unsignedTo obtain an API token, log into github.com, and create onehere.OpenAITo use OpenAI functions you will need a valid API key and organization id. If you're logged in, you can create an API keyhere, and your organization id can be foundhere.To specify these values, you'll need to set two environment variables when running DuckDB:OPENAI_ORG_IDandOPENAI_API_KEY. This may look something like:OPENAI_ORG_ID=<orid>OPENAI_API_KEY=<apikey>duckdb-unsignedGoogleGoogle related functions assume you have a json applications credentials file on disk where you're running DuckDB. The user or service account assoicated with the file will need to have the necessary permissions to interact with services associated with any functions you may use.To specify the path to the credentials file, set an evironment variable namedGOOGLE_APPLICATION_CREDENTIALSwhen running DuckDB. That may look something like:GOOGLE_APPLICATION_CREDENTIALS=~/.gcloud/some-file.jsonduckdb-unsignedExample QueriesThese queries assume you've already installed thePyTablesextension in a DuckDB session. See the extensionsinstallation guideif you have not.AWSNote that all queries will assume to run in the region you have specified in your configuration file. Set theAWS_DEFAULT_REGIONto override this behavior.EC2 InstancesReturns the results of aDescribeInstancesAPI call.SELECT*FROMpytable('ducktables.aws:ec2_instances');S3 BucketsReturns the results of aListBucketsAPI call.SELECT*FROMpytable('ducktables.aws:s3_buckets');S3 ObjectsThis only includes metadata about the objects themselves, not their contents. See thehttpfsDuckDB extension if you're interested in loading data from objects on S3.Note that the second argument, the prefix path, is optional and can be omitted.SELECT*FROMpytable('ducktables.aws:s3_objects','bucket-name','foo/bar/prefix');GithubRepositoriesEnumerates all repositories for the named user or organization.SELECT*FROMpytable('ducktables.githb:repos_for','duckdb');WorkflowsEnumerate each Github Actions Workflow associated with a repositorySELECT*FROMpytable('ducktables.github:workflows','MarkRoddy/duckdb-pytables')Workflow RunsList each execution of every Github Actions Workflow run on a repository.SELECT*FROMpytable('ducktables.github:workflow_runs','MarkRoddy/duckdb-pytables')Open AIChatGPTGiven a prompt and a number of desired responses, will generate a table with the model's response.SELECT*FROMpytable('chatgpt:prompt','Write a limerick poem about how much you love SQL',2));GoogleSheetsPulls data from a Google Sheet. Note that you'll either need user account authorization, or if you use a service account, the
account will need read permissions on the sheet in question if it is private.This example query pulls from a Google provided example spreadsheet. You can see the sheethere. Note that despite this sheet being public, you will still need Google Cloud authorization configured on your local machine.SELECT*FROMpytable('google:sheet','1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms','Class Data!A2:F31',columns={'name':'VARCHAR','gender':'VARCHAR','class_level':'VARCHAR','state':'VARCHAR','major':'VARCHAR','extracurricular':'VARCHAR'});AnalyticsThis query has three required parameters. The ViewID you wish to query, and the start/end date range for your query. Seethis postfor an explanation of finding your view ID. Start and end dates should be strings in a 'YYYY-MM-DD' format. Note you can alos override the set of Dimensions and Metrics, but this requires understanding how to adjust thecolumnsargument to account for the change in schema that will occur as a result.SELECT*FROMpytable('ducktables.google.analytics:metrics','<view-id>','<start-date>','<end-date>',columns={-- Dimensions'date':'VARCHAR','hour':'VARCHAR','pagePath':'VARCHAR','source':'VARCHAR','medium':'VARCHAR',-- Metrics'sessions':'INT','users':'INT','pageviews':'INT','avgSessionDuration':'FLOAT','bounceRate':'FLOAT'});