drsnyder/poky — pull request #20 by drsnyder (2013-08-28 21:35:46+00:00)
Title: Benchmarking jdbc-mget

This is a WIP to provide a basic set of tools for benchmarking `jdbc-mget` and more specifically the poky database configuration & tuning and the `mget` stored procedure.

To run, create some test data:

```
./scripts/get-benchmark-data.sh

$COUNT > tmp/benchmark-in.csv
```

This creates a CSV file of the form:

```
bucket,key,2012-10-04 17:23:06+00
```

The clojure code then reads in one or more files of this form and hashes the `$COUNT` `(key, timestamp)` tuples by the bucket. These tuples are then partitioned by a given request size during the simulation. This is to simulate production `mget` set sizes.

Once the tuples have been imported from the data files they are then organized int `(bucket {:key :modified_at} ...)` lists that can then be easily used as parameters to `jdbc-mget`. Before running the benchmarks, the set of lists is randomized to simulate more real-world access patterns.

Here is a sample benchmark run from within clojure:

```clojure
(def newpoky
(bench/bench-jdbc-mget (connection (system/store S))
["tmp/benchmark-in.csv"]
30
:check #(assert (not (empty? %)))))
```