Package: daredata-snowconn
SnowConnThis repository is a wrapper around thesnowflake SQLAlchemylibrary. It manages the creation of connections and provides a few convenience functions that should be good enough
to cover most use cases yet be flexible enough to allow additional wrappers to be written around to serve more specific
use cases for different teams.InstallationTo install latest version released to pypi with pip:pipinstalldaredata_snowconnTo install the latest version directly from the repo:pipinstall'git+ssh://git@github.com/DareData/snowconn.git@master#egg=snowconn'ConnectionEverything is implemented in a singleSnowConnclass. To import it is always the same:fromsnowconnimportSnowConn(1) Connection using your own personal credsSet the environment variables:SF_USERNAME - SnowFlake username.SF_PASSWORD - SnowFlake password.SF_ROLE - the role you want to use.SF_ACCOUNT - SnowFlakeaccount identifier, has the format<organization_name>-<account_name>.Then simply do:conn=SnowConn.connect()That's it you are connected! You can connect to a specific schema/database using another role or particular computing warehouse with the following:conn=SnowConn.connect('db_prod','public',role=my_role,warehouse=my_warehouse)APINow that you're connected, there are a few low-level functions that you can use to programatically interact with
the snowflake tables that you have access to.execute_simpleThe exc_simple function is used for when you have a single statement to execute and the result set can fit into memory. It
takes a single argument which a string of the SQL statement that you with to execute.execute_stringIf you have multiple sql statements in a single string that you want to execute or the resultset is larger than
will fit into memory, this is the function that you want to use. It returns a list of cursors that are a result
of each of the statements that are contained in the string. Seeherefor the full documentation.execute_fileIf you have the contents of an sql file that you want to execute, you can use this function. For example:
This also returns a list of cursors the same asexecute_stringdoes. In fact, this function is nothing more than a very
simple wrapper aroundexecute_string.read_dfUse this function to read the results of a query into a dataframe. Note that pandas is NOT a dependency of this repo so
if you want to use it you must satisfy this dependency yourself.It takes one sql string as an argument and returns a dataframe.write_dfUse this to write a dataframe to Snowflake. This is a very thin wrapper around the pandasDataFrame.to_sql()function.Unfortunately, it doesn't play nice with dictionaries and arrays so the use cases are quite limited. Hopefully
we will improve upon this in the future.get_current_roleReturns the current role.closeUse this to cleanly close all connections that have ever been associated with this instance of SnowConn. If you don't
use this your process will hang for a while without saying anything before it actually exits.Accessing the connection objects directlyThese functions are mostly wrappers around 2 connection libraries:The snowflake python connectorThe snowflake SQLAlchemy libraryShould you need to use either of these yourself, you can ask for the connections yourself with the following
functions:get_raw_connectionThis will return the instance of a snowflake connector which is documentedhere. It is a good choice if you have very simple needs and for some reason none
of the functions in the rest of this repo are serving your needs.get_alchemy_engineThis is the result ofcreate_engine()which was called duringconnect()orcredsman_connect(). It does not represent an active connection to the database
but rather acts as a factory for connections.This is useful for using the most commonly abstracted things in other libraries such as dashboards, pandas, etc.
However, like SQLAlchemy in general, despite being very widely supported and feature-complete, it is not the simplest
API so it should probably not be your first choice unless you know exactly that you need it.get_connectionThis returns the result of the creating the sqlalchemy engine and then callingconnect()on it. Unlike the result
ofget_alchemy_enginethis represents an active connection to Snowflake and this has a session associated with it.You can see the object documentationhere