Distinguish over-quota for volume size and number.

Fixes LP 1020634

Ensure that exceeding the allowed number of volumes is not
mis-represented in log messages and exception handling as
excessive space usage.

Change-Id: I71ec995c77bc447bfc9221084b057bd8d69a4513
fix metadata file injection with xen

The metadata parameter was misordered in commit 1463839f

Fixes Bug: 1022036
Change-Id: Ica0e2f51b6852c8572bdc3c995cc8c2588b9a25c
Adds diagnostics command for the libvirt driver.

This code provides an implementation of the get_diagnostics
API for the libvirt driver. This API is invoked by the
'nova diagnostic' command.

There is no existing formal specification for what data items
to expose from the 'get_diagnostics' API, so the XenAPI driver
is used as an informal guide. The Xen implementation currently
produces the following output:

+----------------+-----------------+
|    Property    |      Value      |
+----------------+-----------------+
| cpu0           | 4.3627          |
| memory         | 1171088064.0000 |
| memory_target  | 1171088064.0000 |
| vbd_xvda_read  | 0.0             |
| vbd_xvda_write | 0.0             |
| vif_0_rx       | 3223.6870       |
| vif_0_tx       | 0.0             |
| vif_1_rx       | 104.4955        |
| vif_1_tx       | 0.0             |
+----------------+-----------------+

The new libvirt implementation will return a similar set of stats
for guest CPUs, memory, disks and network interfaces, resulting
in the following output:

+------------------+------------+
| Property         | Value      |
+------------------+------------+
| cpu0_time        | 2870000000 |
| memory           | 524288     |
| vda_errors       | -1         |
| vda_read         | 262144     |
| vda_read_req     | 112        |
| vda_write        | 5606400    |
| vda_write_req    | 376        |
| vnet0_rx         | 63343      |
| vnet0_rx_drop    | 0          |
| vnet0_rx_errors  | 0          |
| vnet0_rx_packets | 431        |
| vnet0_tx         | 4905       |
| vnet0_tx_drop    | 0          |
| vnet0_tx_errors  | 0          |
| vnet0_tx_packets | 45         |
+------------------+------------+

The implementation has only been tested with the libvirt KVM
driver.

In a future a formal specification for the required data
format should be written and all virt drivers made to comply.

Finally, the fakelibvirt driver has been updated in order to
support the methods required by the get_diagnostics method.

For those interested, below is the formula to extract the
CPU usage based on the CPU time. I've added it here for reference
since it's not used at the moment and it took quite a while
to locate it in the libvirt documentation.

  usage = 100 * (current_cpu_time - last_cpu_time) /
          (measurement_interval * (10 ** 9))

Fixes bug 986200.
Change-Id: I92da392f43d0840ed31cfd88f54570ea5830bb4a
Split xenapi agent code out to nova.virt.xenapi.agent

This refactoring makes the split between driver methods and internal
methods clearer. It also makes all agent methods private instead of
making some private, some public and some both.

Change-Id: I6c6b405942e47a85816e10224d4e6d986a8b06dc
Refactor instance_usage_audit. Add audit tasklog.

The instance usage audit cronjob that generates periodic
compute.instance.exists notifications is not particularly scalable.
It is run on one server and takes longer as the number of instances grows.

This change moves the generation of those events to a periodic task in
the compute manager. It also adds an api extension that can be used
by administrators to check for errors generating these events.

Change-Id: I856d3d0c73c34e570112f1345d306308ef20a9ae
Clarifying which vm_utils functions are private.

This clarification should aid in future refactorings by giving a hint of
which methods are not referenced outside of the file (except for test
stubs).

Change-Id: I574b2caa25d54969a1627fa69f101b2480152b08
Refactors more snapshot code into vm_utils.

Simplifies vmops a bit while moving logic into vmutils where it better
matches how we've seperated concerns.

Change-Id: Ic6d7559903d0f809f4c90577f4a8c4a0857cae90
Merge "fix metadata file injection with xen"
Use compute_api.get_all in affinity filters.

Updates the affinity filters so they make a single compute API
call to lookup instance host information rather than single
lookups for each UUID.

This resolves a potential performance issue which can cause a
scheduler to hang while processing requests which contain large numbers
of UUID's in the scheduler_hints.

Fixes LP Bug #1017795.

Change-Id: I30f434faf109058573ee41c4a6abce2e48939e8d
Remove unused find_data_files function in setup.py

find_data_files was copied in a while back to work around setuptools
deficiencies. It has since made its way into distribute, and we are using it
from there - thus do not need a copy in our tree anymore.

Change-Id: I8bc76a201c325571601ac8da292ebd5690668065
Merge "Use compute_api.get_all in affinity filters."
Merge "Split xenapi agent code out to nova.virt.xenapi.agent"
Adds network labels to the fixed ips in usages

Change-Id: Ib5990a7f5d7869c862ec3dd3299772e421574ae6
Merge "Refactor instance_usage_audit. Add audit tasklog."
Merge "Adds network labels to the fixed ips in usages"
Merge "Clarifying which vm_utils functions are private."
Re-add nova.virt.driver import to xenapi driver

A recent commit 5dd9e6150c0bebaef53e23fff7e55a17ef694537 removed the
import when it was still used. Re-add the import back so the driver
works again.

Change-Id: I1487a1a171c0d35c2f6752b08620f745ba1df385
Merge "Distinguish over-quota for volume size and number."
Merge "Re-add nova.virt.driver import to xenapi driver"
allows setting accessIPvs to null via update call

fixes bug 1022700

Change-Id: I4b37508ebec1ea23d8dfd91c41ecbd96b2d701bc