Package: pg-simple
Thepg_simplemodule
provides a simple yet efficient layer overpsycopg2providing Python
API for common SQL functions, explicit and implicit transactions
management and database connection pooling for single and multi-threaded
applications.pg_simpleis not intended to provide ORM-like functionality, rather
to make it easier to interact with the PostgreSQL database from python
code for direct SQL access using convenient wrapper methods. The module
wraps the excellentpsycopg2library and most of the functionality
is provided by this behind the scenes.Thepg_simplemodule provides:Simplified handling of database connections/cursorConnection pool for single or multithreaded accessPython API to wrap basic SQL functionality: select, update, delete,
join et alQuery results as python namedtuple and dict objects (usingpsycopg2.extras.NamedTupleCursorandpsycopg2.extras.DictCursorrespectively)Debug logging supportInstallationWithpiporeasy_install:pip install pg_simpleor:easy_install pg_simpleor from the source:python setup.py install30 Seconds Quick-start GuideStep 1: Initialize a connection pool manager usingpg_simple.config_pool()Step 2: Create a database connection and cursor by instantiating apg_simple.PgSimpleobjectHere’s a pseudo-example to illustrate the basic concepts:importpg_simpleconnection_pool=pg_simple.config_pool(dsn='dbname=my_db user=my_username ...')withpg_simple.PgSimple(connection_pool)asdb:db.insert('table_name',data={'column':123,'another_column':'blah blah'})db.commit()withpg_simple.PgSimple(connection_pool)asdb1:rows=db1.fetchall('table_name')Connection pool managementInitialize the connection poolimportpg_simpleconnection_pool=pg_simple.config_pool(max_conn=250,expiration=60,# idle timeout = 60 secondshost='localhost',port=5432,database='pg_simple',user='postgres',password='secret')or, usingdsn:connection_pool=pg_simple.config_pool(max_conn=250,expiration=60,dsn='dbname=database_name user=postgres password=secret')or, usingdb_url:connection_pool=pg_simple.config_pool(max_conn=250,expiration=60,db_url='postgres://username:password@hostname:numeric_port/database')The above snippets will create a connection pool capable of
accommodating a maximum of 250 concurrent database connections. Once
that limit is reached and the pool does not contain any idle
connections, all subsequent new connection request will result in aPoolErrorexception (until the pool gets refilled with idle
connections).Take caution to properly clean up allpg_simple.PgSimpleobjects
after use (wrap the object inside python try-finally block orwithstatement). Once the object is released, it will quietly return the
internal database connction to the idle pool. Failure to disposePgSimpleproperly may result in pool exhaustion error.Configure multiple connection poolsTo generate different connection pools simply define each connection:connection_pool_1=pg_simple.config_pool(max_conn=250,expiration=60,dsn='dbname=database_name_1 user=postgres1 password=secret1')connection_pool_2=pg_simple.config_pool(max_conn=250,expiration=60,dsn='dbname=database_name_2 user=postgres2 password=secret2')After that you can use each connection pool object to generate connections to the databases as you would with only one connection.
You can define as many of connection pool objects as your systems can handle and also both types (SimpleConnectionPoolandThreadedConnectionPool) at the same time.Configure connection pool for thread-safe accessThe defaultSimpleConnectionPoolpool manager is not thread-safe. To
utilize the connection pool in multi-threaded apps, use theThreadedConnectionPool:connection_pool=pg_simple.config_pool(max_conn=250,expiration=60,pool_manager=ThreadedConnectionPool,dsn='...')Disable connection poolingTo disable connection pooling completely, set thedisable_poolingparameter to True:connection_pool=pg_simple.config_pool(disable_pooling=True,dsn='...')All database requests on this pool will create new connections on the
fly, and all connections returned to the pool (upon disposal ofPgSimpleobject or by explicitly invokingpool.put_conn()) will
be discarded immediately.Garbage collect stale connectionsTo explicitly purge the pool of stale database connections (whose
duration of stay in the pool exceeds theexpirationtimeout), invoke
thepool.purge_expired_connections()method:connection_pool.purge_expired_connections()Note that the pool is automatically scavenged for stale connections when
an idle connection is returned to the pool (using thepool.put_conn()method).Basic UsageConnecting to the posgtresql serverThe following snippet will connect to the posgtresql server and allocate
a cursor:importsysimportpg_simpledb=pg_simple.PgSimple(connection_pool,log=sys.stdout,log_fmt=lambdax:'>>%s'%(xifisinstance(x,str)elsex.query),nt_cursor=True)By defaultPgSimplegenerates result sets ascollections.namedtupleobjects (usingpsycopg2.extras.NamedTupleCursor). If you want to access the
retrieved records using an interface similar to the Python dictionaries
(usingpsycopg2.extras.DictCursor), set thent_cursorparameter
toFalse:db=pg_simple.PgSimple(connection_pool,nt_cursor=False)Raw SQL execution>>>db.execute('SELECT tablename FROM pg_tables WHERE schemaname=%sand tablename=%s',['public','books'])<cursorobjectat0x102352a50;closed:0>Dropping and creating tablesdb.drop('books')db.create('books','''
"id" SERIAL NOT NULL,
"type" VARCHAR(20) NOT NULL,
"name" VARCHAR(40) NOT NULL,
"price" MONEY NOT NULL,
"published" DATE NOT NULL,
"modified" TIMESTAMP(6) NOT NULL DEFAULT now()
''')db.execute('''ALTER TABLE "books" ADD CONSTRAINT "books_pkey" PRIMARY KEY ("id")''')db.commit()Emptying a table or set of tablesdb.truncate('tbl1')db.truncate('tbl2, tbl3',restart_identity=True,cascade=True)db.commit()Inserting rowsforiinrange(1,10):db.insert("books",{"genre":"fiction","name":"Book Name vol.%d"%i,"price":1.23*i,"published":"%d-%d-1"%(2000+i,i)})db.commit()Updating rowswithpg_simple.PgSimple(connection_pool)asdb1:db1.update('books',data={'name':'An expensive book','price':998.997,'genre':'non-fiction','modified':'NOW()'},where=('published =%s',[datetime.date(2001,1,1)]))db1.commit()Deleting rowsdb.delete('books',where=('published >=%s',[datetime.date(2005,1,31)]))db.commit()Inserting/updating/deleting rows with return valuerow=db.insert("books",{"type":"fiction","name":"Book with ID","price":123.45,"published":"1997-01-31"},returning='id')print(row.id)rows=db.update('books',data={'name':'Another expensive book','price':500.50,'modified':'NOW()'},where=('published =%s',[datetime.date(2006,6,1)]),returning='modified')print(rows[0].modified)rows=db.delete('books',where=('published >=%s',[datetime.date(2005,1,31)]),returning='name')forrinrows:print(r.name)Fetching a single recordbook=db.fetchone('books',fields=['name','published'],where=('published =%s',[datetime.date(2002,2,1)]))print(book.name+'was published on '+book[1])Fetching multiple recordsbooks=db.fetchall('books',fields=['name AS n','genre AS g'],where=('published BETWEEN%sAND%s',[datetime.date(2005,2,1),datetime.date(2009,2,1)]),order=['published','DESC'],limit=5,offset=2)forbookinbooks:print(book.n+'belongs to '+book[1])Explicit database transaction managementwithpg_simple.PgSimple(connection_pool)as_db:try:_db.execute('Some SQL statement')_db.commit()except:_db.rollback()Implicit database transaction managementwithpg_simple.PgSimple(connection_pool)as_db:_db.execute('Some SQL statement')_db.commit()The above transaction will be rolled back automatically should something
goes awry.