Distinguish over-quota for volume size and number.

Fixes LP 1020634

Ensure that exceeding the allowed number of volumes is not
mis-represented in log messages and exception handling as
excessive space usage.

Change-Id: I71ec995c77bc447bfc9221084b057bd8d69a4513
Adds diagnostics command for the libvirt driver.

This code provides an implementation of the get_diagnostics
API for the libvirt driver. This API is invoked by the
'nova diagnostic' command.

There is no existing formal specification for what data items
to expose from the 'get_diagnostics' API, so the XenAPI driver
is used as an informal guide. The Xen implementation currently
produces the following output:

+----------------+-----------------+
|    Property    |      Value      |
+----------------+-----------------+
| cpu0           | 4.3627          |
| memory         | 1171088064.0000 |
| memory_target  | 1171088064.0000 |
| vbd_xvda_read  | 0.0             |
| vbd_xvda_write | 0.0             |
| vif_0_rx       | 3223.6870       |
| vif_0_tx       | 0.0             |
| vif_1_rx       | 104.4955        |
| vif_1_tx       | 0.0             |
+----------------+-----------------+

The new libvirt implementation will return a similar set of stats
for guest CPUs, memory, disks and network interfaces, resulting
in the following output:

+------------------+------------+
| Property         | Value      |
+------------------+------------+
| cpu0_time        | 2870000000 |
| memory           | 524288     |
| vda_errors       | -1         |
| vda_read         | 262144     |
| vda_read_req     | 112        |
| vda_write        | 5606400    |
| vda_write_req    | 376        |
| vnet0_rx         | 63343      |
| vnet0_rx_drop    | 0          |
| vnet0_rx_errors  | 0          |
| vnet0_rx_packets | 431        |
| vnet0_tx         | 4905       |
| vnet0_tx_drop    | 0          |
| vnet0_tx_errors  | 0          |
| vnet0_tx_packets | 45         |
+------------------+------------+

The implementation has only been tested with the libvirt KVM
driver.

In a future a formal specification for the required data
format should be written and all virt drivers made to comply.

Finally, the fakelibvirt driver has been updated in order to
support the methods required by the get_diagnostics method.

For those interested, below is the formula to extract the
CPU usage based on the CPU time. I've added it here for reference
since it's not used at the moment and it took quite a while
to locate it in the libvirt documentation.

  usage = 100 * (current_cpu_time - last_cpu_time) /
          (measurement_interval * (10 ** 9))

Fixes bug 986200.
Change-Id: I92da392f43d0840ed31cfd88f54570ea5830bb4a
Don't block instance delete on missing block device volume

It's easy to create an undeletable instance by specifying an invalid
volume id in the block device mapping. The instance errors, but
subsequent delete fails because the volume is missing. We already
ignore the DiskNotFound case during delete, so this adds a case for
VolumeNotFound.

Change-Id: Ibc516ee4452d088cddf10bbeb315697ce900e5c5
Split xenapi agent code out to nova.virt.xenapi.agent

This refactoring makes the split between driver methods and internal
methods clearer. It also makes all agent methods private instead of
making some private, some public and some both.

Change-Id: I6c6b405942e47a85816e10224d4e6d986a8b06dc
Refactor instance_usage_audit. Add audit tasklog.

The instance usage audit cronjob that generates periodic
compute.instance.exists notifications is not particularly scalable.
It is run on one server and takes longer as the number of instances grows.

This change moves the generation of those events to a periodic task in
the compute manager. It also adds an api extension that can be used
by administrators to check for errors generating these events.

Change-Id: I856d3d0c73c34e570112f1345d306308ef20a9ae
Clarifying which vm_utils functions are private.

This clarification should aid in future refactorings by giving a hint of
which methods are not referenced outside of the file (except for test
stubs).

Change-Id: I574b2caa25d54969a1627fa69f101b2480152b08
Refactors more snapshot code into vm_utils.

Simplifies vmops a bit while moving logic into vmutils where it better
matches how we've seperated concerns.

Change-Id: Ic6d7559903d0f809f4c90577f4a8c4a0857cae90
Merge "Split xenapi agent code out to nova.virt.xenapi.agent"
Adds network labels to the fixed ips in usages

Change-Id: Ib5990a7f5d7869c862ec3dd3299772e421574ae6
Merge "Refactor instance_usage_audit. Add audit tasklog."
Merge "Adds network labels to the fixed ips in usages"
Merge "Clarifying which vm_utils functions are private."
Re-add nova.virt.driver import to xenapi driver

A recent commit 5dd9e6150c0bebaef53e23fff7e55a17ef694537 removed the
import when it was still used. Re-add the import back so the driver
works again.

Change-Id: I1487a1a171c0d35c2f6752b08620f745ba1df385
Merge "Distinguish over-quota for volume size and number."
Merge "Re-add nova.virt.driver import to xenapi driver"
allows setting accessIPvs to null via update call

fixes bug 1022700

Change-Id: I4b37508ebec1ea23d8dfd91c41ecbd96b2d701bc
Merge "Refactors more snapshot code into vm_utils."
Merge "allows setting accessIPvs to null via update call"
Imports base64 to fix xen file injection

Adds base64 to agent.py as its required for file injection to work.

Fixes bug 1023698

Change-Id: I231b6f5e031106471aeff38769e8fc79dcd45b45
Remove deprecated auth-related db code

* Remove user_*, auth_token_* and project_* db api
  functions (except project_get_networks)
* Remove User, Project, AuthToken, UserProjectRoleAssociation,
  UserRoleAssociation, and UserProjectAssociation db models
* Add two migrations:
  * 109 removes the project_id fkey from dns_domains
  * 110 drops tables that correspond to the deleted models
* Related to bp remove-deprecated-auth

Change-Id: Iaacae0e5c0050e94b92e16ab5d8129eadb56c5a7