van.pg 2.0
Tools to programmatically manage PostgreSQL clusters as Python test fixtures.
License: BSD

Test PostgreSQL Databases
=========================

Easy creation of PostgreSQL databases (and clusters) for unit testing.

Dirty Databases
---------------

Test databases take a long time to create. In general you need to be a little
careful when you decide to delete/recreate a test database fixture.

Also, there seems to be no robust way in PostgreSQL of figuring out if a
database was committed to or not.

So van.pg has no choice but to place the responsibility on the you to notify
it when a database is dirty.  If this isn't done properly, test isolation will
be compromised.  It's not ideal, but the best we can do.

One exception is if you consistently use the ``transaction`` package
(http://pypi.python.org/pypi/transaction) to manage database commits.  In this
case you can ask for the resource to be dirtied whenever a transaction is
committed.

Integration with ``testresources``
----------------------------------

The typical way to use these fixtures is via ``testresources``
(http://pypi.python.org/pypi/testresources/):

    >>> from testresources import ResourcedTestCase
    >>> from van.pg import DatabaseManager
    >>> import psycopg2

    >>> def init_db(db):
    ...     conn = psycopg2.connect(host=db.host, database=db.database)
    ...     cur = conn.cursor()
    ...     cur.execute("CREATE TABLE foo (bar INTEGER);")
    ...     conn.commit()
    ...     conn.close()

    >>> class MyTest(ResourcedTestCase):
    ... 
    ...     resources = [('db', DatabaseManager(initialize_sql=init_db))]
    ... 
    ...     def runTest(self):
    ...         conn = psycopg2.connect(host=self.db.host, database=self.db.database)
    ...         cur = conn.cursor()
    ...         cur.execute("INSERT INTO foo VALUES (1);")
    ...         conn.commit()
    ...         cur = conn.cursor()
    ...         cur.execute("SELECT * FROM foo")
    ...         self.assertEqual(cur.fetchall(), [(1, )])
    ...         # NOTE: must close connections or dropping databases fails
    ...         conn.close()
    ...         self.db.dirtied() # we changed the DB, so it needs re-loading
        
Actually run the test:

    >>> from unittest import TextTestRunner
    >>> import sys
    >>> runner = TextTestRunner(stream=sys.stdout)
    >>> runner.run(MyTest()) # doctest: +ELLIPSIS
    .
    ...
    OK
    ...

Using template databases
------------------------

If you need to recreate the same database many times, it can be faster to let
PostgreSQL copy the database from a template database. You can do this by having one DatabaseManager serve as the template for another:

    >>> template_db = DatabaseManager(initialize_sql=init_db)
    >>> class MyTest2(MyTest):
    ...     resources = [('db', DatabaseManager(template=template_db))]

    >>> runner.run(MyTest2()) # doctest: +ELLIPSIS
    .
    ...
    OK
    ...

``transaction`` integration
---------------------------

If the keyword argumen ``dirty_on_commit`` is True, a DatabaseManager will mark
the database as dirtied after every successfull commit made through the
``transaction`` module. This means each test which dirties the database does not
have to manually notify it.

    >>> man = DatabaseManager(dirty_on_commit=True)

If you use this feature, you need to depend on the transaction
(http://pypi.python.org/pypi/transaction) package yourself.

Using an existing database
--------------------------

By default, van.pg creates a new PostgreSQL cluster in a temporary directory
and launches a PostgreSQL daemon. This works most of the time, but is not very
fast.

If you have an already running PostgreSQL cluster, you can tell van.pg to use
it by setting the environment variable VAN_PG_HOST. For example, to run
van.pg's tests against a local PostgreSQL server with it's sockets in
/tmp/pgcluster do:

    $ VAN_PG_HOST=/tmp/pgcluster python setup.py test

WARNING: any databases starting with test_db in the target database are likely
to be dropped.

Closing Connections
-------------------

Be careful to properly close all connections to the database once your test is
done with it. PostgreSQL doesn't allow dropping databases while there are open
connections. This will cause van.pg to error when trying to drop the test
database.

Programatically creating a cluster
----------------------------------

At a lower level, you can also programmatically manipulate your own PostgreSQL cluster.

Initialize the Cluster:
 
    >>> from van.pg import Cluster
    >>> cluster = Cluster()
    >>> cluster.initdb()

Which creates a database in a temporary directory:

    >>> import os
    >>> dbdir = cluster.dbdir
    >>> 'PG_VERSION' in os.listdir(dbdir)
    True
    
Start it:

    >>> cluster.start()

Create/Test a database:

    >>> dbname = cluster.createdb()
    

We can connect to the database:

    >>> import psycopg2
    >>> conn = psycopg2.connect(database=dbname, host=cluster.dbdir)
    >>> cur = conn.cursor()

Twiddle the database to make sure we can do the basics:
    
    >>> cur.execute("CREATE TABLE x (y int)")
    >>> cur.execute("INSERT INTO x VALUES (1)")
    >>> conn.commit()
    >>> cur.execute("SELECT * from x")
    >>> cur.fetchall()[0][0]
    1

Stop the cluster daemon:

    >>> conn.close()
    >>> cluster.stop()
    
Start it again:

    >>> cluster.start()
    >>> conn = psycopg2.connect(database=dbname, host=cluster.dbdir)
    >>> cur = conn.cursor()
    >>> cur.execute("SELECT * from x")
    >>> cur.fetchall()[0][0]
    1

And cleanup:

    >>> conn.close()
    >>> cluster.cleanup()
    >>> cluster.dbdir is None
    True
    >>> os.path.exists(dbdir)
    False

Development
-----------

Development takes place on GitHub:

    http://github.com/jinty/van.pg

CHANGES
=======

2.0 (unreleased)
----------------

* Support Python 3.2.

* Drop Python 2.5 support.

* Add tox.ini for testing against multiple python versions.

* Run PostgreSQL as a subprocess rather than as a daemon (via pg_ctl).

* Re-organize code to improve reuse and test coverage.