fio: fix job clone mem leak

In the loop to create clones at the bottom of add_job the function
get_new_job clones the thread_data, just to occaisonally get the
allocated pointers for filename and files overwritten a few lines later.

The dup files also duplicates the name strings so the references to
these are lost by the setting to null.

This patch fixes takes care of that and frees the memory before
discarding the pointers (found via valgrind).

Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
fio: allow general repeatability

This patch adds a "allrandrepeat" option similar to the existing
randrepeat.  But other than the former option it does not only affect
the random I/O pattern, but also all remaining users of randomness as
well.

By that e.g. testcases using nrfiles, filesize ranges, blocksizesplit
ranges and so on will stay repeatable across (euqally parametrized) runs
as well.  To maintain compatibility the default is off.

Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>

Update by me to fix missing thread_options.h and cconv.c change.

Signed-off-by: Jens Axboe <axboe@fb.com>
fio: allow milliseconds on all time specifiers

This patch allows all time specifiers to be specified down to
milliseconds.  Default will stay seconds for compatibility with old
configs.

It also adds documentation to the existing time units day, hour and
minute.

Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
fio: provide an option for a startdelay range

This patch allows the specification of start delay as range.  With a
range each thread will chose an individual startdelay out of the range.

That solves an issue of startdelay being the same for each numjob clone
and that way spreads all kind of activities e.g. that all clones with
mixed r/w jobs switch r/w at the same time.  Also all kind of other
"thundering herd" issues can be softened by some time spread due to this
option.

Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
fio: add multi directory support

This patch adds support for ':' seperated multiple directories at the
directory config statement in order to achieve an automatic distribution
of job clones (numjob) across directories.

That way people can distribute a load across these directories (usually
mount points of disks) automatically - changing numjob will be
sufficient to get all job clones evenly (optimal if dirs % numjobs = 0,
otherwise as good as possible) distributed at all times.

To avoid confused users old config Files will behave like they always
did, old fio binaries using new config files won't abort but just use
the first specified dir. If one specifies an explcit (non generated)
filename the distribution to many directories is also deactivated.

It also fixes an issue of events seeming out of order like when running
with --debug=file seeing the "..." message meaning "I created the
clones" prior to the last clone activities.  Now the clones are called
with index N-1 .. 1, zero being the base thread as before.

Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
fio: allow to combine terse output with any selected output type

This patch adds the option --append-terse to be able to request a
combination of any given selected output format AND terse output based
on the same data.

This will help all users that want to parse the terse data for further
use, but need to look into the logs every now and then which then should
be readable.

Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
fio: flush log files on test end

Ensure proper flushing of all logs at the end of tests.

Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
fio: allow 0 as compress percentage

Allow 0 as value for option compress_percentage which can be useful for
certain deduplication and compression based storage back ends.

Signed-off-by: Christian Ehrhardt <ehrhardt@linux.vnet.ibm.com>
Signed-off-by: Jens Axboe <axboe@fb.com>
server: bump protocol version

We have new options added.

Signed-off-by: Jens Axboe <axboe@fb.com>