Fix problem with compare algorithm

The logic was missing removed tests except for
1st one.

Improved diff unit test to verify number of diffs
per type (removed_tests, new_tests, value_changed)

Change-Id: I45fd77dbc31a255e0bdc33c6b8b2ebd1c9f088ed
Modify sla failure rate

Parameter "max_failure_percent" of sla section becomes depricated.
Now use
...
  sla:
    failure_rate:
      min: ...
      max: ...

Change-Id: I5e1e4f280084263a483008ba3922be75eab5b6e8
Improve unit tests for UserGenerator

Our experiments with unit tests for UserGenerator, especially those for
UserGenerator.create_tenants(), have revealed that it is the excessive use
of mocks:

    ...
    context = {"admin": {"endpoint": mock.MagicMock()},
               "task": mock.MagicMock()}
    ...

that leads to race conditions:
    http://paste.openstack.org/show/137037/

Here, we fix the mocking of the task object that caused these race conditions
and unify the UTs code.

We also modify the UserGenerator so that it uses the more efficient collections.deque
instead of ordinary lists.

Change-Id: I8696287ecf9bc8bb44f67f70c81c6c93fdb8acbe
Closes-Bug: #1394158
Improvements for HTML output of rally verify results

Changes:
  * Move rally/verification/verifiers/tempest/report_templates/main.mako
    to rally/ui/templates/verification/report.mako
  * Reimplement rally/ui/templates/verification/report.mako
    for cross-project layout and styling, refactor JS code
  * Refactoring in rally.verification.verifiers.tempest.json2html
  * Remove unnecessary `setUpClass' from skipped tests names

Change-Id: Ib403dd8bb49ec0c98f812171e737a01414229979
Add force_delete to safely delete created VMs

When Nova allows soft-delete operation, currently, rally cannot
deal with it and throws the exception when delete all the created
VMs.

Reproducing: (Nova enables soft-delete)
1. Run task: boot-from-volume-and-delete.json
2. It creates several VMs and volumes.
3. It deletes VMs.
4. VM states change to soft-deleted, but volume state is still in-use
because VM is not truly deleted.
5. It throws an exception saying that volume cannot be deleted.
6. The task is failed.

This fix adds force_delete when nova calls delete-server.
You can specify 'force_delete' in task configuration to determine
whether to use default delete action or force_delete action.

Change-Id: I56ac5c039b525a8b05960dfbfd3ff0cacec0aef6
Closes-Bug: #1394119
Merge "Add force_delete to safely delete created VMs"
Merge "Improve unit tests for UserGenerator"
Fix typo in rally tempest config generator

Change-Id: I067793488160d2ecb76aa8353ab3517b5952ffe7
Closes-Bug: #1396515
Merge "Fix typo in rally tempest config generator"
Add aliases for debug levels in rally.log

In Rally, we wrap the standard logging module with our own rally.log
module. This leads to ugle statements like logging.logging.INFO in some
places of Rally, because the logging levels are defined in the standard
logging module, which is, in turn, imported in rally.log.

Change-Id: I89be56592fe1a542e21e105ead17aafbc4b2319f
Add a servers context and a cinder stress scenario

Based on https://github.com/WaltHP/cinder-stress/commits/master/qaStressTest.py
this scenario creates n volumes and performs the following sequence using
those volumes:

1. Create snapshots
2. Attach volumes
3. Detach volumes
4. Delete snapshots
5. Delete volumes

The servers context setup instantiates n servers and removes them on cleanup.

Co-Authored-By: Cesar Mojica <cesar.mojica@hp.com>
Co-Authored-By: Anthony Lee <anthony.mic.lee@hp.com>
Change-Id: I8e32cbef57f12079e78a8f7ae73f2af4835d2cb5
Engine changed to not crash on context creation

engine.run method changed to catch all exception during
context creation and log them instead of crashing.

Closes-bug:#1396274

Change-Id: I2ac96b8f9d1d478378a0f4c1b5bbabc37abb6a08
Merge "Fix problem with compare algorithm"
Merge "Add aliases for debug levels in rally.log"
Merge "Engine changed to not crash on context creation"
Merge "Add a servers context and a cinder stress scenario"
Adds Nova vm live migration benchmark scenarios

 -NovaServers.boot_and_live_migrate_server

Implements blueprint nova-benchmark-scenarios-for-vm-migration

Change-Id: Ib92bf0640c5bdff03162c0cc735df85912295466
Introudce benchmark.context() decorator

Before this patch we have to specify:

__ctx_name__, __ctx_order__ and __ctx_hidden__ by hands as attributes
of context class.

This thing produced few issues:

1) It looks ugly
2) It is absolutelly unclear. So you don't know what they mean, and
   there is no easy way to find information about it. So now it's
   much simpler e.g. you should just take a look at decorator
   implementation
3) This was source of bugs, cause it was not tested by syntax
   of language => so we have to have mad tests to check that
   all __ctx_name__, __ctx_order__ were specified

* Use decorator instead of direct setting attributes

* Rename attributes:
  __ctx_name__ -> _ctx_name
  __ctx_order__ -> _ctx_order
  __ctx_hidden__ -> _ctx_hidden

  This was done cause of unification with other parts of code

* Improve a bit "servers" get uuid of image and flavor only once

* Impove order of context

* Make secgroup and allow_ssh not hidden context. There is no
  need to hide them, and probably somebody would like to use
  them.

* Docs are updated

Change-Id: Ib089b398ba0c7f54d5246eb6ba29f5bbcfd2deee
Add scenarios for security group

New scenarios:
 - create and delete security groups
 - create_and_list_secgroups

Co-Authored-By: Ilya Shakhat <ishakhat@mirantis.com>

Change-Id: If9b25260c8d57dc52d7fb64abbaac0601a0eb4ed
Add benchmark overview page to html report

The overview page shows all scenarios and their summarized results
in single table, so it is easier to understand the whole result
and compare specific scenarios.

Also, there are some minor features and fixes in html report.

Changes:
  * Benchmark overview page
  * Task input file is available now
  * Scenario full duration is added to overview
  * Scenario duration is renamed to `Load duration'
  * Overview is sortable
  * Fix: control from browser history: back/forward buttons now work
  * Fix: durations are reset for iterations with errors
  * AngularJS version is updated to release 1.3.3
  * task sla_check output: column `success' with options True/False
    is renamed to `status' with options PASS/FAIL

Change-Id: I0eb7af01432c9c10e4ec55720bb53417478a5789