Package: cuttlepool
CuttlePool is a general purpose, thread-safe resource pooling implementation
for use with long lived resources and/or resources that are expensive to
instantiate. It’s key features are:Pool overflowCreates additional resources if the pool capacity has been reached and
will remove the overflow when demand for resources decreases.Resource harvestingAny resources that haven’t been returned to the pool and are no longer
referenced by anything outside the pool are returned to the pool. This helps
prevent pool depletion when resources aren’t explicitly returned to the
pool and the resource wrapper is garbage collected.Resource queuingIf all else fails and no resource can be immediately found or made, the
pool will wait a specified amount of time for a resource to be returned
to the pool before raising an exception.How-to GuideUsing CuttlePool requires subclassing aCuttlePoolobject with a user
defined methodnormalize_resource()andping(). The example below usessqlite3connections as a resource, but CuttlePool is not limited to
connection drivers.>>> import sqlite3
>>> from cuttlepool import CuttlePool
>>> class SQLitePool(CuttlePool):
...     def normalize_resource(self, resource):
...         resource.row_factory = None
...     def ping(self, resource):
...         try:
...             rv = resource.execute('SELECT 1').fetchall()
...             return (1,) in rv
...         except sqlite3.Error:
...             return False
>>> pool = SQLitePool(factory=sqlite3.connect, database='ricks_lab')Let’s break this down line by line.First, thesqlite3module is imported.sqlite3.connectwill be the
underlying resource factory.CuttlePoolis imported and subclassed. Thenormalize_resource()method takes a resource, in this case asqlite3.Connectioninstance created
bysqlite3.connect, as a parameter and changes it’s properties. This is
important because a resource can be modified while it’s outside of the pool and
any modifications made during that time will persist; this can have unintended
consequences when the resource is later retrieved from the pool.Next theping()method is implemented, which also takes a resource, the
same asnormalize_resource(), as a parameter.ping()ensures the
resource is functional; in this case, it checks that thesqlite3.Connectioninstance is open. If the resource is functional,ping()returnsTrueelse it returnsFalse. In the above example, a simple statement is executed
and if the expected result is returned, it means the resource is open andTrueis returned. The implementation of this method is really dependent on
the resource created by the pool and may not even be necessary.Finally an instance ofSQLitePoolis made. Thesqlite3.connectmethod is
passed to the instance along with the database name.TheCuttlePoolobject and as a result theSQLitePoolobject accepts any
parameters that the underlying resource factory accepts as keyword arguments.
There are three other parameters the pool object accepts that are unrelated to
the resource factory.capacitysets the max number of resources the pool
will hold at any given time.overflowsets the max number of additional
resources the pool will create when depleted. All overflow resources will be
removed from the pool if the pool is at capacity.timeoutsets the amount
of time in seconds the pool will wait for a resource to become free if the pool
is depleted when a request for a resource is made.A resource from the pool can be treated the same way as an instance created by
the resource factory passed to the pool. In our example a resource can be used
just like asqlite3.Connectioninstance.>>> con = pool.get_resource()
>>> cur = con.cursor()
>>> cur.execute(('INSERT INTO garage (invention_name, state) '
...              'VALUES (%s, %s)'), ('Space Cruiser', 'damaged'))
>>> con.commit()
>>> cur.close()
>>> con.close()Callingclose()on the resource returns it to the pool instead of closing
it. It is not necessary to callclose()though. The pool tracks resources
so any unreferenced resources will be collected and returned to the pool. It is
still a good idea to callclose()though, since explicit is better than
implicit.NoteOnceclose()is called on the resource object, it renders the
object useless. The resource object received from the pool is a wrapper
around the actual resource object and callingclose()on it returns
the resource to the pool and removes it from the wrapper effectively
leaving it an empty shell to be garbage collected.To automatically “close” resources,get_resource()can be used in awithstatement.>>> with pool.get_resource() as con:
...     cur = con.cursor()
...     cur.execute(('INSERT INTO garage (invention_name, state) '
...                  'VALUES (%s, %s)'), ('Space Cruiser', 'damaged'))
...     con.commit()
...     cur.close()APIThe API can be found atread the docs.FAQHow do I install it?pip install cuttlepoolContributingIt’s highly recommended to develop in a virtualenv.Fork the repository.Clone the repository:git clone https://github.com/<your_username>/cuttlepool.gitInstall the package in editable mode:cd cuttlepool
pip install -e .[dev]Now you’re set. See the next section for running tests.Running the testsTests can be run with the commandpytest.Where can I get help?If you haven’t read the How-to guide above, please do that first. Otherwise,
check theissue tracker.
Your issue may be addressed there and if it isn’t please file an issue :)