Adding genfio tool : an fio job creator

Some fio users need to create multiple fio jobs to compare storage
device's performance on different workload.

genfio is made for creating series of fio jobs based on the following
parameters:
- a disk serie
- a block size serie
- the time each job shall run (default is 300 seconds)
- the expected IO patterns to perform (read,write,randread,randwrite)
- a parameter to define if disks shall be tested one after the other or
  simultaneously

Based on all this parameters, a single fio script file will be computed
to process all this workloads.

Example:
[user@localhost]: genfio -d sdb,sdc,sdd,sde -a -b 4k,128k,1m -r 100 -a
Generating localhost-4k,128k,1m-all-read,write,randread,randwrite-sdb,sdc,sdd,sde.fio
Estimated Time = 6000 seconds : 1 hour 40 minutes

If the user run "fio
localhost-4k,128k,1m-all-read,write,randread,randwrite-sdb,sdc,sdd,sde.fio",
it will run:
- a sequential bench on /dev/sdb /dev/sdc /dev/sdd /dev/sde
    block size = 4k with read,write,randread,randwrite tests
    ETA ~ 4 tests * 4 disks * 100 seconds

- a sequential bench on /dev/sdb /dev/sdc /dev/sdd /dev/sde
    block size = 128k with read,write,randread,randwrite tests
    ETA ~ 4 tests * 4 disks * 100 seconds

- a sequential bench on /dev/sdb /dev/sdc /dev/sdd /dev/sde
    block size = 1m with read,write,randread,randwrite tests
    ETA ~ 4 tests * 4 disks * 100 seconds

- a parallel bench on /dev/sdb /dev/sdc /dev/sdd /dev/sde
    block size = 4k with read,write,randread,randwrite tests
    ETA ~ 4 tests * 100 seconds

- a parallel bench on /dev/sdb /dev/sdc /dev/sdd /dev/sde
    block size = 128k with read,write,randread,randwrite tests
    ETA ~ 4 tests * 100 seconds

- a parallel bench on /dev/sdb /dev/sdc /dev/sdd /dev/sde
    block size = 1m with read,write,randread,randwrite tests
    ETA ~ 4 tests * 100 seconds

For each job of this fio script, bandwidth & iops logs file will be
generated allow further investigations or plotting.
Adding fio2gnuplot tool

fio2gnuplot aims at plotting fio output files into gnuplot traces.
It does work pretty differently from fio_generate_plots as :
- it plots simultaneous traces in 3D graphs
- it plots various histogram to represent min/max/average/stddev of the
  series
- it uses a pattern matching mecanism to select the input files
- it have a post-mortem analysis mode to compute global values across
  multiple log files
- it saves metadata to report what traces was related to what file
- it generates automatically picture's topic

fio2gnuplot aims at plotting a huge number of simultaneous fio traces in
a understable way. It got tested with up to 197 simultaneous disks traces.

The 3D rendering helps at understanding the "surface" of the benchmark
and answer is fraction of seconds about the homogenity of the results.

The histograms rendering helps at understanding the average values, the
standard deviations between hosts or storage device. By using the
pattern matching, it's really easy to select a set of files to generate
a dedicated graph.

Example:
fio2gnuplot.py -p 'm[1-3]sw[1-3]-*read*para-*iops*' -g
 Selected m1sw1-128k-sdb-read-para.results_iops.log
 Selected m1sw1-128k-sdc-read-para.results_iops.log
 Selected m1sw1-128k-sdd-read-para.results_iops.log
 Selected m1sw1-128k-sde-read-para.results_iops.log
 Selected m2sw1-128k-sdb-read-para.results_iops.log
 Selected m2sw1-128k-sdc-read-para.results_iops.log
 Selected m2sw1-128k-sdd-read-para.results_iops.log
 Selected m2sw1-128k-sde-read-para.results_iops.log
 Running gnuplot Rendering

The previous example shows how to match some iops log file from fio.
In this example, thoses files were generated by genfio.

As a result, the following files will be generated by gnuplot :
m[1-3]sw[1-3]-read-para-iops-3D.png
m[1-3]sw[1-3]-read-para-iops.max.png
m[1-3]sw[1-3]-read-para-iops.png
m[1-3]sw[1-3]-read-para-iops.average.png
m[1-3]sw[1-3]-read-para-iops.min.png
m[1-3]sw[1-3]-read-para-iops.stddev.png

Thoses files will graph all the selected files (traces) at the same time
with various goals (min/max/stddev/average/3D)
genfio: Adding runtime=0 support

If the runtime is set to 0, it will ask fio to get until the end of the
device. This could be useful in some cases.
genfio: Adding cached_based option

By default fio is still run in direct mode which is safer for many
benchmarking approach. In some case, you might need to enable the
cached-based IOs.

The "-c" option is now here for enabling this.
genfio: Claryfing default options

Let's show the help if no disks are given as parameter.

This patch does also clarify the default values of each parameter.
That clearly helps understanding what's the default settings if no
option used.
genfio: Adding prefix to fio filename

This patch add a prefix (-x) option to add context to fio jobs.
When running many many jobs it's pretty hard to remember which job was
made for.

Adding a prefix to the filename can give some interesting information
like the tested hardware or the day we run it, or a option you test
etc...

If the prefix owns a / in it, it will be considered as a directory and
will create it. The fio jobs will be stored into this directory.
genfio: Fixing empty ETA while runtime > 0

Commit 920e0b66756814f7d01bcbbaf5cf0fd05213628d introduced the runtime=0
support but did considered the amount of minutes in addition of rounded
hours to consider if the ETA message can be shown or not.

This is perfectly wrong since it should have been the total amount of
seconds to consider. If runtime=0, the amount of seconds to run cannot
be computed as = 0.

The present code was wrong as if runtime was rounded on hours it did
show the message that ETA cannot be computed.
fio2gnuplot: Adding 2D graphs for each fio trace

When tracing multiple files, the 3D graph is useful to understand the
behavior of a group of disks but makes difficult to understand how a
particular disk behave during the test.

This patch does add three 2D plots for each fio trace :
- the 2D Raw tracing
This plot is the exact plotting of the raw values reported by fio.
It's very precise but hard to read when having many many traces

- the 2D Smooth tracing
This plot is a smooth version of the raw tracing. It tries to keep as
much as possible a tracing close to the raw one but makes it more
readable by aggregating points.

- the 2D Trend tracing
This plot is trying to setup a trend of the global performance. It
aggregates many points and hide peaks. It only consider at giving a very
easy to understand plotting on how the bench was performing.
It's much more easier to read but could be misleading if raw or smooth
tracing didn't got read before.
fio2gnuplot: Adding sample files

This commit does add some sample log files and makefile to use
fio2gnuplot. This is very useful for understanding, developping or testing fio2gnuplot.

A simple "make" will do the job for you.
fio2gnuplot: Don't graph 3D if not enough data

If the matching pattern doesn't get more than a single file, 3D plotting
doesn't have any meaning and confuse gnuplot.

So let's disable the 3D plotting if a single file matched the pattern.
fio2gnuplot: Adjusting y axis plot (2D)

It's better let gnuplot adjusting the tics on the Y axis unless that
becomes unreadble. Too many ticks where present on simple graphs.

This was a useless option, shouldn't have been here from the beginning.
genfio: Warn users if read occurs before write

The default mode was to generate read pattern _before_ the write ones.
On some storage device that handle read on untouch sectors by answering
zero from the cache, the result is totally invalid and cheat about the
real read capabilities of the device.

This patch does change the default mode to emit writes before reads.
It also add a warning message if user is setting up some modes by itself
and generate a read_before_write pattern.

That's only a warning message that last a few seconds but could avoid
running a wrong benchmark.

This commit occurs after a real case of invalid performances due to read
emitted to an untouched block volume.

Typical output will be:
 root@vm1-1: genfio -d vdb -s -b 4k,128k,4m -m read,randread
 ###############################################################
 # Warning : You are reading data while never wrote them before#
 # On some storage devices, this could lead to invalid results #
 #                                                             #
 # Press Ctrl-C to adjust pattern order if you have doubts     #
 # Or Wait 5 seconds before the file will be created           #
 ###############################################################
 Generating vm1-1-4k,128k,4m-sequential-read,randread-vdb.fio
 Estimated Time = 1800 seconds : 0 hour 30 minutes
genfio: Adding iodepth support

It's useful to be able to specify the iodepth while running fio with
libaio.

By default, we keep iodepth=1 to keep the same as fio's default.
genfio: Enforce time_based if runtime > 0

If the device is short and the bandwidth pretty high, we can exit
_before_ runtime is achieved. To avoid this phenomena, let's use
time_based directive that will ensure that no run will be shorten.