Package: qabot
qabotQuery local or remote files with natural language queries powered by
OpenAI'sgptandduckdb🦆.Can query Wikidata, local and remote files.InstallationInstall withpipx:pipx install qabotCommand Line Usage$EXPORTOPENAI_API_KEY=sk-...
$EXPORTQABOT_MODEL_NAME=gpt-4
$qabot-w-q"How many Hospitals are there located in Beijing"Query:HowmanyHospitalsaretherelocatedinBeijing
Thereare39hospitalslocatedinBeijing.
Totaltokens1749approximatecostinUSD:0.05562Python Usagefromqabotimportask_wikidata,ask_fileprint(ask_wikidata("How many hospitals are there in New Zealand?"))print(ask_file("How many men were aboard the titanic?",'data/titanic.csv'))Output:There are 54 hospitals in New Zealand.
There were 577 male passengers on the Titanic.FeaturesWorks on local CSV files:remote CSV files:$ qabot -f https://duckdb.org/data/holdings.csv -q "Tell me how many Apple holdings I currently have"
 🦆 Creating local DuckDB database...
 🦆 Loading data...
create view 'holdings' as select * from 'https://duckdb.org/data/holdings.csv';
 🚀 Sending query to LLM
 🧑 Tell me how many Apple holdings I currently have


 🤖 You currently have 32.23 shares of Apple.


This information was obtained by summing up all the Apple ('APPL') shares in the holdings table.

SELECT SUM(shares) as total_shares FROM holdings WHERE ticker = 'APPL'Even on (public) data stored in S3:You can even load data from disk/URL via the natural language query:Load the file 'data/titanic.csv' into a table called 'raw_passengers'.
Create a view of the raw passengers table for just the male passengers. What
was the average fare for surviving male passengers?~/Dev/qabot> qabot -q "Load the file 'data/titanic.csv' into a table called 'raw_passengers'. Create a view of the raw passengers table for just the male passengers. What was the average fare for surviving male passengers?" -v
 🦆 Creating local DuckDB database...
 🤖 Using model: gpt-4-1106-preview. Max LLM/function iterations before answer 20
 🚀 Sending query to LLM
 🧑 Load the file 'data/titanic.csv' into a table called 'raw_passengers'. Create a view of the raw passengers table for just the male passengers. What was the    
average fare for surviving male passengers?
 🤖 load_data
{'files': ['data/titanic.csv']}
 🦆 Imported with SQL:
["create table 'titanic' as select * from 'data/titanic.csv';"]
 🤖 execute_sql
{'query': "CREATE VIEW male_passengers AS SELECT * FROM titanic WHERE Sex = 'male';"}
 🦆 No output
 🤖 execute_sql
{'query': 'SELECT AVG(Fare) as average_fare FROM male_passengers WHERE Survived = 1;'}
 🦆 average_fare
40.82148440366974
 🦆 {"summary": "The average fare for surviving male passengers was approximately $40.82.", "detail": "The average fare for surviving male passengers was
calculated by creating a view called `male_passengers` to filter only the male passengers from the `titanic` table, and then running a query to calculate the      
average fare for male passengers who survived. The calculated average fare is approximately $40.82.", "query": "CREATE VIEW male_passengers AS SELECT * FROM       
titanic WHERE Sex = 'male';\nSELECT AVG(Fare) as average_fare FROM male_passengers WHERE Survived = 1;"}


 🚀 Question:
 🧑 Load the file 'data/titanic.csv' into a table called 'raw_passengers'. Create a view of the raw passengers table for just the male passengers. What was the    
average fare for surviving male passengers?
 🤖 The average fare for surviving male passengers was approximately $40.82.


The average fare for surviving male passengers was calculated by creating a view called `male_passengers` to filter only the male passengers from the `titanic`    
table, and then running a query to calculate the average fare for male passengers who survived. The calculated average fare is approximately $40.82.

CREATE VIEW male_passengers AS SELECT * FROM titanic WHERE Sex = 'male';
SELECT AVG(Fare) as average_fare FROM male_passengers WHERE Survived = 1;QuickstartYou need to set theOPENAI_API_KEYenvironment variable to your OpenAI API key,
which you can get fromhere.Install theqabotcommand line tool using pip/pipx:$pipinstall-UqabotThen run theqabotcommand with either local files (-f my-file.csv) or-wto query wikidata.ExamplesLocal CSV file/s$qabot-q"how many passengers survived by gender?"-fdata/titanic.csv
🦆Loadingdatafromfiles...
Loadingdata/titanic.csvintotabletitanic...

Query:howmanypassengerssurvivedbygender?
Result:
Therewere233femalepassengersand109malepassengerswhosurvived.🚀anyfurtherquestions?[y/n](y):y🚀Query:whatwasthelargestfamilywhodidnotsurvive?Query:whatwasthelargestfamilywhodidnotsurvive?
Result:
ThelargestfamilywhodidnotsurvivewastheSagefamily,with8members.🚀anyfurtherquestions?[y/n](y):nQuery WikiDataUse the-wflag to query wikidata. For best results use agpt-4or similar model.$EXPORTQABOT_MODEL_NAME=gpt-4
$qabot-w-q"How many Hospitals are there located in Beijing"Intermediate steps and database queriesUse the-vflag to see the intermediate steps and database queries.
Sometimes it takes a long route to get to the answer, but it's interesting to see how it gets there.qabot -f data/titanic.csv -q "how many passengers survived by gender?" -vData accessed via http/s3Use the-f <url>flag to load data from a url, e.g. a csv file on s3:$qabot-fs3://covid19-lake/enigma-jhu-timeseries/csv/jhu_csse_covid_19_timeseries_merged.csv-q"how many confirmed cases of covid are there?"-v
🦆Loadingdatafromfiles...
createtablejhu_csse_covid_19_timeseries_mergedasselect*from's3://covid19-lake/enigma-jhu-timeseries/csv/jhu_csse_covid_19_timeseries_merged.csv';Result:264308334confirmedcasesIdeasstreaming mode to output results as they come intoken limitsSupervisor agent - assess whether a query is "safe" to run, could ask for user confirmation to run anything that gets flagged.Often we can zero-shot the question and get a single query out - perhaps we try this before the MKL chaintest each zeroshot agent individuallyGenerate and pass back assumptions made to the userAdd an optional "clarify" tool to the chain that asks the user to clarify the questionCreate a query checker tool that checks if the query looks valid and/or safeInject AWS credentials into duckdb so we can access private resources in S3Automatic publishing to pypi. Look athttps://blog.pypi.org/posts/2023-04-20-introducing-trusted-publishers/