Package: dbhelpers
dbhelpersDatabase helpers and utilities.This is not an ORM, is a set of useful utilities to work with raw queries using the Python Database API Specification.InstallationThe easiest way to install dbhelpers is with pip:$ pip install dbhelpersBackendsThe following backends are supported by default:PostgreSQL: with thepsycopg2adapter:Psycopg2ConnectionMySQL: with theMySQLdbadapter:MySQLdbConnectionSQLite3: with the default adapter of Python:Sqlite3ConnectionYou can extend the functionality of dbhelpers making new connection classes for your custom backends. See the usage section for more information.UsageConnection classesUse a default connection class for your db backend:fromdbhelpersimportPsycopg2Connection# Simple connectionconn=Psycopg2Connection(db='mydb',user='myuser',passwd='mypass').connect()(...)conn.close()# Or using a context manager:withPsycopg2Connection(db='mydb',user='myuser',passwd='mypass')asconn:cursor=conn.cursor()...Or create a custom connection class with your default parameters:fromdbhelpersimportMySQLdbConnectionclasscustomconn(MySQLdbConnection):default_user='myuser'default_passwd='mypass'default_host='localhost'default_port=13306default_extra_kwargs={'charset':'utf8mb4'}withcustomconn('mydb')asconn:cursor=conn.cursor()....Also you can make a connection class for a custom backend inheriting the abstract classBaseConnectionand overriding the methodconnect.fromdbhelpers.connectionsimportBaseConnectionclassMyCustomBackendConnection(BaseConnection):default_port=9876defconnect(self):"""Returns a new connection object."""returncustomadapter.connect(database=self.db,user=self.user,password=self.passwd,host=self.host,port=self.port,**self.extra_kwargs)HelpersThe package include some useful utilities to work with database cursors.Cursor as a context manager:The cursor is executed inside awithblock. When the block ends the cursor is closed. Also does aconnection.commit()when the block ends ifcommit=True(True by default).fromdbhelpersimportcm_cursor# With autocommitwithcustomconn('mydb')asconn:withcm_cursor(conn)ascursor:cursor.execute("INSERT INTO mytable (id, status) VALUES (23, 'info')")# Disable autocommitwithcustomconn('mydb')asconn:withcm_cursor(conn,commit=False)ascursor:(...)Ifcommit=True(default) and an exception is thrown inside thewithblock,cm_cursorcalls theconn.rollback()method instead ofconn.commit()In Python 2.7 and 3.x you can get the connection object and the cursor object of the context managers in a single with statment:withcustomconn('mydb')asconn,cm_cursor(conn)ascursor:# Do something ...Fetchiterfetchitercan be used as a generator for large recordsets:fromdbhelpersimportfetchiterwithcustomconn('mydb')asconn:withcm_cursor(conn)ascursor:cursor.execute("SELECT * FROM bigtable")forrowinfetchiter(cursor):# Do somethingThefetchiterfunction does not copy all rows in memory, do sucessive calls in blocks to retrieve all data. The default block size is 1000.Thecursor.fetchall()method can fill the process memory easily if there are a lot of register to return.fetchiterdo calls tocursor.fetchmany()iteratively until there are no more data  to return. Thefetchiterfunction behaves like an iterator.You can get the whole blocks or change the size of the block:withcustomconn('mydb')asconn:withcm_cursor(conn)ascursor:cursor.execute("SELECT * FROM bigtable")forblockinfetchiter(cursor,size=50,batch=True):# Do something, block is a tuple with 50 rowsPostgreSQL server cursorAlso,fetchiterallows work with PostgreSQL server cursors previously declared.Instead of the standardfetchiterbehavior, which do a query to a server, the server calculates the whole recordset, andfetchiterretrieve the results iteratively to avoid fill the process memory, a server cursor runs the pseudo-iterator on a Postgres server and calculates the partial recordset in blocks iteratively.See more about PostgreSQL cursors in thePostgreSQL documentation.fromdbhelpersimportfetchiterwithcustomconn('mydb')asconn:withcm_cursor(conn)ascursor:cursor.execute("DECLARE C CURSOR FOR SELECT * FROM bigtable")forrowinfetchiter(cursor,server_cursor='C'):# Do somethingcursor.execute("CLOSE C")fetchitercan return the server cursor results as the above example (as an interator or as a block), an you can change the block size. The default block size is 1000.Rows as NamedTuplesfetchone_nt,fetchmany_nt,fetchall_ntfetchiter_ntreturns the rows as NamedTuples:fromdbhelpersimportfetchone_nt,fetchmany_nt,fetchall_ntwithcustomconn('mydb')asconn:withcm_cursor(conn)ascursor:cursor.execute("SELECT id, status FROM mytable WHERE id = 23")row=fetchone_nt(cursor)# Now, row is a NamedTuple with each column mapped as an attribute:# >>> row.id# 32# >>> row.status# 'warning'