Package: pbq
PBQpython BiqQuery driver for easy accessInstallingTo start using this package run:$pipinstallpbqFor development use, and local testing run:$pythonsetup.pyinstallDependencybq Command-Line ToolInstall bq cli -- the package is running bq cli in the backgroundhttps://cloud.google.com/sdk/docs/after installation on a new terminal write:$bqinit
$gcloudauthapplication-defaultloginif you don't want to install bq cli you can run the package with the
service account keyall you need to do it to define an environment variable like this:$exportGOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/[FILE_NAME].json"UsageImport this package to your project and you are ready to gofrompbqimportPBQfrompbqimportQueryPBQThis class can have generic functions for big query manipulation and
save queries to tables.some usages:Download table to CSVfrompbqimportPBQfrompbqimportQuery# read sql file and format the query from file to string without parametersquery=Query.read_file('queries/query.sql')# init the query builderpbq=PBQ(query)#run the query and get csvpbq.to_csv('file.csv',save_query=True,**{'table':'table','dataset':'dataset'})when setting save_query=True it means that you want to save your query
to a table, and you need to send dictionary with the table name and
dataset name.Download table to pandas DataFramefrompbqimportPBQfrompbqimportQuery# read sql file and format the query from file to string without parametersquery=Query.read_file('queries/query.sql')# init the query builderpbq=PBQ(query)#run the query and get dataframepbq.to_dataframe()Simply run your queryfrompbqimportPBQfrompbqimportQuery# read sql file and format the query from file to string without parametersquery=Query.read_file('queries/query.sql')# init the query builderpbq=PBQ(query)#run the query if you want to write insert query or alter table for example.pbq.run_query()Save query to a tablefrompbqimportPBQfrompbqimportQuery# read sql file and format the query from file to string without parametersquery=Query('select * from table')# init the query builderpbq=PBQ(query)pbq.save_to_table('table_name','dataset_name','project_name',replace=True,partition='20190910')When setting replace=True it will override the table as long as the
table is not partitioned, if partitioned it will overwrite the
partition.partition format isYYYYMMDDupload DataFrame to a tableimportpandasaspdfrompbqimportQuery,PBQdf=pd.DataFrame()PBQ.save_dataframe_to_table(df,'table','dataset','project_id',partition='20191013',replace=False)QueryThis class will format your query, check validation and return the price
of the querysome usages:generate the query object with a simple queryfrompbqimportQueryquery=Query("select * from table")get query pricefrompbqimportQueryquery=Query("select * from table")print("the query price:",query.price)# the query price: 0.312validate queryfrompbqimportQueryquery=Query("select * from table")ifnotquery.validate():raiseRuntimeError("table not valid")query with parametersfrompbqimportQueryquery=Query("select * from table where user_id={user_id}",parameters={'user_id':123})print(query.query)# select * from table where user_id=123read the query from file with parametersfrompbqimportQueryquery=Query.read_file('file_path.sql',parameters={'user_id':123})print(query.query)# select * from table where user_id=123Free software: MIT licenseDocumentation:https://pbq.readthedocs.io.History0.1.0 (2019-10-03)First release on PyPI.