korpling/ANNIS — pull request #62 by he-sk (2012-11-09 18:24:02+00:00)
Title: New shell command to benchmark queries in a file

A new shell command `benchmarkFile $FILENAME` runs every query found in `$FILENAME` and reports the number of matches and the required time in a tab-delimited table. The queries are run in the order they appear in the file. This allows testing of stable workloads on multiple systems.

`$FILENAME` should contain queries as follows:

count tok
count node
...

Currently, only `count` queries are supported that run on the entire database, i.e., every corpus. The command returns the following information:

count	tok	399	78	1	1
count	node	996	49	2	1

The output is tab-delimited. The columns are:

1. The query function, i.e., `count`.
2. The query.
3. The number of matches.
4. The time required to execute the query.
5. The query index which counts the number of unique queries, starting with 1.
6. The query repetition which counts for each unique query the number of evaluations.

If there is a timeout, parse error, or other error the string `ERROR` is printed in the third column.

The command can also be called from the command line, e.g., `annis.sh 'benchmarkFile /path/to/file'`.

Another example of a benchmark file:

count tok
count node
count node
count tok
count tok &
count "der"
count tok

The output is:

bash $ annis.sh 'benchmarkFile test.aql'
running command 'benchmarkFile test.aql'
count	tok	399	78	1	1
count	node	996	49	2	1
count	node	996	60	2	2
count	tok	399	54	1	2
count	tok &	ERROR	0	3	1
count	"der"	9	43	4	1
count	tok	399	55	1	3
Time: 642 ms
bash $

In this example, the query `tok` is run three times, the query `node` is run twice, and `"der"` is run once. The query `tok &` triggers a parse error.