puppetlabs/puppetdb — pull request #770 by kbarber (2013-12-05 01:42:02+00:00)
Title: WIP: PDB-68 Differential edges

This patch changes edge behaviour so that instead of continually
inserting new content, and then providing a GC periodically it
instead does the inserts/deletes for a particular certname.

While this is a slight step backwards for providing deduplication
this is better then what we have today. Before this patch every new
catalog would have to insert a whole set of new edges, the old ones
to be purged later.

Instead now we read the current edges for the certname in question,
and then do a conditional insert/delete on any differences - doing
nothing if the rows already exist. This means that if only 1 edge
was added since the last catalog update, only 1 insert occurs.

To facilitate this we have had to modify the schema for the 'edges'
table so a migration has been provided for this. Now the edges
table is referenced directly to the certnames table. In the future
once we apply differential handling for resources we can rehome it
back to a catalog, but since today we don't support more then 1
catalog logically to a certname the only downside is the larger
foreign key (certname is a txt field versus the catalog_id it used
to have).

Tests have been added to ensure concurrency retries work, and that
only the necessary SQL is performed.

As concurrency is now a greater issue we have found a bug with
the postgresql driver and BoneCP whereby during serialization exceptions
(which are just a natural part of concurrency) the transactions
are not closed after the 'test' SQL made to the database by BoneCP.

The simple work-around is to basically explicitly do a begin and
commit around our own SQL statement.

Signed-off-by: Ken Barber