Package: bitdotio
bit.io Python SDK & Command Line ToolInstallationIn order to support different environments, we have a few ways to install the bitdotio
package with or without thepsycopg2dependency.If you already havepsycopg2installed, you can install the default bitdotio package:pip install bitdotioIf you already have Postgres installed, you can install with the psycopg2 dependency:pip install 'bitdotio[psycopg2]'If you do not have or cannot install Postgres, you can install with the psycopg2-binary dependency:pip install 'bitdotio[psycopg2-binary]'Versioningpython-bitdotiousessemantic versioning.bitdotiomoduleUsageThebitdotiomodule consists of abitdotioSDK object which provides helpful
utilities for creating and managing pre-configured connections to bit.io databases
viapsycopg2, as well as easy methods for accessing the functionality exposed by thebit.io developer APIOnce you havebitdotioinstalled all you need is your API key to start working with bit.io.You can get your API key by logging into bit.io and openingthe "Connect" tabon a database page.Example usageSee API reference at the bottom of this document for a full list of methods provided by
the SDK.importosimporttimefrombitdotioimportbitdotio# Load bit.io API key from environment variableBITIO_KEY=os.environ.get("BITIO_KEY")# Instantiate the bit.io SDK objectb=bitdotio(BITIO_KEY)# Create a new public database on bit.iodb_metadata=b.create_database("my-db",is_private=False)db_name=db_metadata["name"]# Use a pooled cursor to insert some datawithb.pooled_cursor(db_name)ascur:cur.execute("CREATE TABLE person (name text, age integer);")people=[("Alice",45),("Bob",84),("Catherine",19),("Davis",22),("Emily",34)]forname,ageinpeople:cur.execute("INSERT INTO person (name, age) VALUES (%s,%s)",(name,age))# Execute a query via the HTTP APIprint(b.query(db_name,"SELECT * FROM person",data_format="objects"))# Create a new table "city" by importing a file.withopen("city.csv","rb")asf:job_status=b.create_import_job("my-username/my-db","city",file=f)# Poll the status of the job until donewhileTrue:job_status=b.get_import_job(job_status["id"])ifjob_status["state"]=="DONE":breakifjob_status["state"]=="FAILED":# If the import failed for whatever reason, raise an exception with the# error info.error_id=job_status["error_id"]error_type=job_status["error_type"]error_details=job_status["error_details"]raiseException(f"Import failed. error_id={error_id}error_type={error_type}, error_details={error_details}")# Wait a little before trying againtime.sleep(0.2)Making queriesConnecting directlyThebitdotioSDK object provides the methodspooled_connection,pooled_cursor, andget_connectionfor connecting directly to your database (see below for discusson on
when to use each of these). The connections/cursors returned by these methods arepsycopg2connections and cursors, which are fully compatible with thePython DB-API. Full documentation onpsycopg2can
be foundhere.It is preferable to use direct connections in the majority of cases, since they have
superior performance and enable more features compared to thequerymethod. In
particular we strongly recommend using direct connections in programs that are
long-running, require transaction management, or transfer large quantities of data.ThequerymethodThequeryruns queries via the bit.io HTTP API, and therefore will have worse
performance and feature support than direct connections. Importantly, each query run
via thequerymethod is run in a single transaction. Thequerymethod is recommended
in situations where installing a database driver is undesirable or impossible, when
queries are being run very infrequently, or in very short-lived contexts such as one-off
scripts or serverless backends.Connection pooling and managementThe recommended way to obtain a direct connection to a bit.io database is via a
connection pool. In order to support scaling to zero, bit.io automatically closes idle
connections after a period of time, and puts databases into a dormant state when there
are no live connections. If you are designing a long-running application, you should
make sure that your database access pattern is resilient to connection closures and
database shutdowns. The best way to do this is via a connection pool. Acquiring
connections from a connection pool allows connection re-use, and handles reconnects in
the event that a connection is dropped. ThebitdotioSDK object internally manages a
thread-safe connection pool per-database that the caller connects to, and provides two
helpful context mangers for acquiring connections from a pool:pooled_connection: A context manager providing apsycopg2connection to the
given database acquired from a connection pool. When exiting the context created by
this context manager, the connection is returned to the pool if still open, or
discarded if closed. It is recommended to use this method when executing multiple
transacions, or in situations where explicit transaction management is required.pooled_cursor: A context manager providing apsycopg2cursor created from a
connection acquired from a connection pool. When exiting the context created by this
context manager, the cursor's tansaction will be committed, and its connection will be
returned to the connection pool. It is recommended to use this method in situations
where a single transaction is required without any complex handling logic. For
example, performing a sequence ofSELECTqueries, or performing a single insertion
or update.There may be situations in which a self-managed, unpooled connection is needed. For
example, if the client needs to persist state onto the connection's database session using
theSETcommand. For such situations, the SDK object provides theget_connectionmethod.For more information on all of the above, refer to thepsycopg2documentation onconnections,cursors,
andpools.Data imports and exportsThebitdotioSDK object provides helper methods to facilitate importing and exporting
data from your bit.io database.To import data into a table on your bit.io database from a file locally or on the web
you can use thecreate_import_jobmethod. To export data from a query or a table in
your bit.io database you can use thecreate_export_jobmethod.These methods have somewhat symmetrical workflows in that they both kick off jobs on
the bit.io backend which execute asynchronously, and have companion methods
(get_import_job, andget_export_job) to check on the status of a running job.At a high level, the procedure for doing a data import looks like:Callcreate_import_joband get back the job statusUse the job status info to pollget_import_jobuntil the job status is reported
asDONEorFAILEDIfDONE, the data has been successfully imported and your table is ready to queryIfFAILED, the job status object will contain metadata describing what went wrong.Similarly, a data export looks like:Callcreate_export_joband get back the job statusUse the job status info to pollget_export_jobuntil the job status is reported
asDONEorFAILEDIfDONE, the exported data will be available to download at thedownload_urlincluded in the job status info.IfFAILED, the job status object will contain metadata describing what went wrong.MultiprocessingThebitdotioSDK object is not safe to share between multiple processes. If you are
programming in a multiprocess environment, ensure that abitdotioSDK object is
created per-process, not pre-fork.bitCLIInstalling thebitdotiomodule also installs the command line toolbitwhich lets
you interact with bit.io from scripts or the command line. This is installed next to
your python binary.You'll want to grab your API key from your bit.io account - to get the key, log into to
bit.io, go to any database, and retrieve it from the "Connect" tab.Usage: bit [OPTIONS] COMMAND [ARGS]...

Options:
  -k, --key TEXT  Your bit.io API key, available when you click the connect
                  button in bit.io
  --help          Show this message and exit.

Commands:
  db
  queryAll of the commands return JSON.You can supply your API key either via the-k/--keyargument, or by setting the it to
theBITIO_KEYenvironment variable. The latter option keeps the key from showing up
in, eg, apscommand, and allows secret injection for systems like Kubernetes.ExamplesList databasesBITIO_KEY=<yourkey>bitdblistThis is the same as:bit-k<your-key>dblistRun a querybit-k<your-key>query-d"username/dbname"-q"SELECT * FROM my_table"API Referencebitdotio(access_token, min_conn=0, max_conn=100)Returns an instance of the bit.io SDK object.Args:access_token (str): Your bit.io API keyKwargs:min_conn (int): The minimum number of live connections per database connection pool.max_conn (int): The maximum number of live connections per database connection pool.Returns:A bit.io SDK instancebitdotio.pooled_connection(db_name)Creates a context manager which provides a connection from the connection pool for the
given database name. When the context manager is exited, the connection's transaction
will be committed, and the connection will be returned to the connection pool.Args:db_name (str): The name of the database to connect to.Returns:Context manager yielding apsycopg2.connection.bitdotio.pooled_cursor(db_name)Creates a context manager which provides a cursor from the connection pool for the
given database name. When the context manager is exited, the cursor's transaction will
be committed, and its connection will be returned to the connection pool.Args:db_name (str): The name of the database to connect to.Returns:Context manager yielding apsycopg2.connection.bitdotio.get_connection(db_name)Returns an unmanaged, unpooled connection to the given database.Args:db_name (str): The name of the database to connect to.Returns:Apsycopg2.connectionobject configured for the given database.bitdotio.query(db_name, query_str, data_format=None)Execute a query via the bit.io HTTP APIArgs:db_name (str): Name of the database to execute a query against.query_str (str): Query to execute.Kwargs:data_format (Optional[str]):Returns:Dictionary describing query results.bitdotio.list_databases()List metadata pertaining to databases the requester is an owner or collaborator of.Returns:List of dictionaries describing database metadata.bitdotio.create_database(name, is_private=True, storage_limit_bytes=None)Args:name (str): Name of the database being created (excluding the owner's username)Kwargs:is_private (bool): Whether or not the database is set to private. Databases default
to private.storage_limit_bytes (Optional[int]): Maximum storage for the database in bytes.
Limits and defaults are enforced based on billing plan regardless of what is set here.Returns:Dict containing metadata about the newly created database.bitdotio.get_database(db_name)Get metadata about a single database.Args:db_name (str): Name of the database to fetch metadata for.Returns:Dict containing metadata about the given database.bitdotio.update_database(db_name, name=None, is_private=None, storage_limit_bytes=None)Update metadata parameters for a given database.Args:db_name (str): Name of the database to update.Kwargs:name (Optional[str]): New name for the database (excluding the owner's username).is_private (Optional[str]): Whether or not the database is set to private.storage_limit_bytes (Optional[int]): Maximum storage for the database in bytes.
Limits and defaults are enforced based on billing plan regardless of what is set here.Returns:Up to date metadata about the updated database.bitdotio.delete_database(db_name)Delete a database. After deletion the database's name will be unusable for up to 30
days. If you need to reuse it sooner, please contact bit.io support.Args:db_name (str): Name of the database to deleteReturns:None.bitdotio.create_import_job(db_name, table_name, schema_name=None, infer_header=None, file=None, file_url=None)Start a data import job from a file or a URL. Supported filetypes CSV, JSON, XLS/XLSX,
and SQLite.Args:db_name (str): Name of the database to import data intotable_name (str): Name of the table to import data intoschema_name (Optional[str]): Name schema in which the target table resides. Not
required if the table is in thepublicschema.infer_header (Optional[str]): If relevant to the given filetype, indicates how the
first row of the data should be interpreted. Should be one ofauto,first_row, orno_header. Ifauto, we will attempt to determine automatically if there is a
header or not. Iffirst_row, the first row of the data will be used as the header.
Ifno_header, the first row of the data will be interpreted as data.file (Optional[file-like]): A file-like object to upload. If this is passed,file_urlmust not be passsed.file_url (Optional[str]): A URL to import a file from. The URL should point to a
supported filetype on the web. If this is passed,filemust not be passed.Returns:A dict describing the status of the import job. The dict contains the fieldsid, andstatus_url. The value ofidcan be passed toget_import_jobto get the updated
status of the import job.status_urlcan also be requested directly to get the same.
Also included is thestatefield, which indicates the current status of the job.
Possible values ofstateareRECEIVED,PROCESSING,DONE, andFAILED.bitdotio.get_import_job(import_id)Retrieves the status and other metadata about a given data import job. See the docs forcreate_import_jobfor more details on import job statuses and metadata fields.Args:import_id (str): The ID of the import job to retrieve info for.Returns:A dict describing the status and metadata of an import job.bitdotio.create_export_job(db_name, query_string=None, table_name=None, schema_name=None, file_name=None, export_format=None)Start a data export job from a query or a full table. Supported export filetypes are
CSV, JSON, XLS, and Parquet.Args:db_name (str): Name of the database to export data fromquery_string (Optional[str]): A query to export results for. Providing this option
will run the query on your database. If this is passed,table_nameandschema_namemust not be passed.table_name (Optional[str]): Name of the table to export data from. If this is
passed,query_stringmust not be passed.schema_name (Optional[str]): Name schema in which the exported table resides. Not
required if the table is in thepublicschema.file_name (Optional[str]): Name of the exported file.export_format (str): File format to export data to. Accepted values arecsv,json,xls, andparquet. Defaults tocsv.Returns:A dict describing the status of the export job. The dict contains the fieldsid, andstatus_url. The value ofidcan be passed toget_export_jobto get the updated
status of the export job.status_urlcan also be requested directly to get the same.
Also included is thestatefield, which indicates the current status of the job.
Possible values ofstateareRECEIVED,PROCESSING,DONE, andFAILED. When
the status of the job reachesDONE, the fielddownload_urlwill have a non-null
value, and the exported file can be downloaded from it.bitdotio.get_export_job(export_id)Retrieves the status and other metadata about a given data export job. See the docs forcreate_export_jobfor more details on export job statuses and metadata fields.Args:export_id (str): The ID of the export job to retrieve info for.Returns:A dict describing the status and metadata of an export job.bitdotio.list_service_accounts()List metadata pertaining to service accounts the requester has created.Returns:List of dictionaries describing service account metadata.bitdotio.get_service_account(service_account_id)Get metadata about a single service accountArgs:service_account_id (str): ID of the service accountReturns:Dictionary describing service account metadata.bitdotio.create_service_account_key(service_account_id)Create a new API key/database password for a given service accountArgs:service_account_id (str): ID of the service accountReturns:Dictionary containing newly created credentialsbitdotio.revoke_service_account_keys(service_account_id)Revoke all API keys/database passwords for the given service accountArgs:service_account_id (str): ID of the service accountReturns: Nonebitdotio.create_key()Create a new API key/database password with the same permissions as the requesterReturns:Dictionary containing newly created credentialsbitdotio.revoke_keys(api_key=None)Revoke API keys/database passwords. Ifapi_keyis given, only that API key will
be revoked. Otherwise, all API keys for the requester will be revoked.Kwargs:api_key (Optional[str]): API key to revokeReturns: None