puppetlabs/puppetdb — pull request #1065 by wkalt (2014-08-22 18:47:43+00:00)
Title: (PDB-810) Load test structured facts indices.

This patch addresses some performance issues for structured facts

- changes a coalesce that was causing the value_integer and value_float indices
to be disregarded. They are verified to work.

- deletes the index on fact path depth, since it was not getting used.

- creates a trgm index on fact_values column value_string, so that our trgm
indexes are now on fact_paths and value_string. This greatly speeds
up queries on value_string and only moderately affects the table size. See
https://gist.githubusercontent.com/wkalt/42792c53a5dc9f0840c4/raw/29f261316bc7e39d888250401f0b1f1c8c28a87b/gistfile1.txt
for some comparisons on a 2000 node dataset with 100 facts per node.

- put a DISTINCT clause in our garbage collection on fact_paths. This incurs a
minor performance cost in small datasets, but yields a tremendous benefit in
large and highly duplicated sets by reducing the load on the accompanying IN.

- changes stringified numerical comparisons on the lower endpoints to use
bigints instead of ints.

- Changes the /v4 endpoint to make the equality
operator type sensitive, i.e a value query for "24" will not match 24 like it
does on the lower endpoints. Inequality operators will also fail on strings.
On the old endpoints the old behavior is preserved.

- adds some rough load testing fuctions to benchmark.clj

- all structured facts indices have been verified to work.