Make metadata content match the requested version of the metadata API.

Currently the same metadata is returned no matter what metadata
version is requested. Metadata is currently implemented to version
2009-04-04 (apart from existing TODOs) as defined here:
http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/instancedata-data-categories.html

This change adds version checking conditionals throughout
get_ec2_metadata and only includes the metadata if the requested version
should contain it.

When there is a difference between Amazon API implementation and documentation
I've gone with replicating the implementation. So far differences include:
- undocumented hostname introduced in 1.0, deprecated in favour of
  local-hostname
- kernel-id and ramdisk-id seem have to been implemented in 2007-12-15,
  not 2008-02-01 as documented

get_ec2_metadata has also been reordered so that data is added oldest to
newest by API version. In a future change I may attempt to add
unimplemented API versions (2011-01-01, 2012-06-01).

Change-Id: Ibf6e6e3b9807f29dccd31006515df550bf9bf57b
Distinguish over-quota for volume size and number.

Fixes LP 1020634

Ensure that exceeding the allowed number of volumes is not
mis-represented in log messages and exception handling as
excessive space usage.

Change-Id: I71ec995c77bc447bfc9221084b057bd8d69a4513
Properly track VBD and VDI connections in xenapi fake

The fake xenapi module correctly destroys VBDs when destroying a VM
but it won't destroy VBDs when destroying a VDI. If an exception
occurs during a spawn when running test_xenapi, then another exception
can occur which obscures the original (and likely more important)
exception. This can make troubleshooting test suite failures more
difficult.

Change-Id: I19aad7f39b0707893a04784faef90815f7b0effb
Set the default CPU mode to 'host-model' for Libvirt KVM/QEMU guests

Historically Nova has not set any CPU model for guests started with
the libvirt driver. This means they are all using the per-hyervisor
default settings for CPU model. With KVM/QEMU guests the model was
traditionally a very conserative choice which exposed minimal features.
This is significantly limits the performance of applications. Further
the model has changed over time, so the exact default model is
unpredictable. Switching Nova to use the host CPU model by default
should improve the out of the box performance & give a known
setup.

This does not impact migration compatibility, since the migration
code is already doing comparison checks against the source and
destination host CPU model, regardless of the actual model used
in the guest. In the future the migration code should be tweaked
so that it actually compares the current guest CPU model, against
the target host CPU model, which would potentially broaden the
migration compatibility pool.

With this patch there is a new libvirt_cpu_mode="none" which
can be used to explicitly prevent any CPU model setting in the
instance XML. The default value None, will now default to
"none" for LXC/UML/etc, and "host-model" for QEMU/KVM

Fixes: bug #1003373
Implements: blueprint libvirt-xml-cpu-model
Change-Id: I5b96e4532b6a960e1608dd6e19ae9d194379fb6a
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Switch to common notifiers.

- Added notifier files to openstack-common.conf, merged from common.
- Import notifiers from common rather than nova.
- Removed nova-specific notifier code.
- Update some driver paths to refer to the new location within nova.

Should be a no-op, as the common notification code is freshly copied from Nova.

Change-Id: Icdf892bc3826b683fc74f22ead00939beda2859f
fix metadata file injection with xen

The metadata parameter was misordered in commit 1463839f

Fixes Bug: 1022036
Change-Id: Ica0e2f51b6852c8572bdc3c995cc8c2588b9a25c
Update common setup code to latest.

This gets us up to date with common/setup.py and replaces custom nova
autodoc generation with the port of that code found in common.

Change-Id: I2a1c5d2c0fdcf40dbea50cc123b537adb068cdc2
Removed a bunch of cruft files.

Ohai. We dont't uses any of theses anymores. Let's just make buh-bye.

Change-Id: Ib81c92ab9196552150d81d158befb0747966965d
Janitorial:  Catch rpc up with a change in common

Common rpc now uses jsonutils instead of json; this
patch moves that change into nova.

Change-Id: Iba45e473ae875b0973dbe6f752602b58c7980578
Properly name openstack.common.exception.

Now it's 'nova.openstack.common.exception', thanks to a more modern
update.py in common.

Change-Id: I4b3caf9014f866da3a5bbf07ab194fc9910159b0
Propagate setup.py change from common.

This pulls in the following patch:

commit 8a967b16dc2ffd60bc19bee09f45e665ddcc9a81
Author: Monty Taylor <mordred@inaugust.com>
Date:   Mon Jul 2 17:19:06 2012 -0400

    Fix up openstack-common build.

    Ironically enough, even though the code to manage this is in
    openstack-common, openstack-common wasn't using it. This patch fixes doc
    generation, as well as setuptools-git integration and versioning.

Change-Id: Ibb2041b131c67aba963e2dd81dcdfd105e1f9b33
Avoid lazy-loading errors on instance_type

When instances are launched with a source group, the instance_type
is not lazy_loaded in properly.

The issue is intermittent because if another query has already loaded
the instance with a joined instance_type or the instance has not
yet been expunged from the session, then the load succeeds.

Therefore this failure only appears under heavy load. Adding an
explicit joindload fixes the issue.

Change-Id: I450c598c31c416c7be341cbb8cd14e33c6bb2aec
Remove deprecated auth code

* Remove nova.auth.manager -- AuthManager, User, Role, Project, etc.
* Remove nova.auth.dbdriver
* Remove nova.auth.ldapdriver
* Remove nova.auth.signer
* Remove arbitrary scripts and schemas in nova/auth/
* Remove nova/auth/novarc.template
* Remove or update affected tests
* Related to bp remove-deprecated-auth

Change-Id: Ide0fefd0ddf79ae1b3bb74cb242c2893575839e7
Fix snapshots tests failing
bug 1022670

When running snapshot test, FLAGS.instances_path is defined as '',
with snapshot_directory='/snapshots' which will fail to to permissions errors.

Modified class LibvirtConnTestCase to use libvirt_snapshots_directory=''

Change-Id: I930e4f30ddffcf66c58312e92f0ae368e770e438
Expose over-quota exceptions via native API.

Fixes bug LP 1021373.

Previously an over-quota condition would be exposed via the EC2 API,
but hidden in the corresponding call via the native API (in the sense
of the exception detail being replaced with a generic 500 Server Error
response).

We now report any NovaException declared to be safe. In this patch,
the set of safe exception types includes any subclass of QuotaError,
but in subsequent patches the net should be widened to include
exceptions that do not expose senstive information.

Change-Id: I3cc36337c7e67cf487ca49de646c437c217ae538
When deleting an instance, avoid freakout if iscsi device is gone

It's fairly easy to get into a state where an instance is not running,
the iscsi infrastructure has been torn down, and the instance cannot
be deleted because the iscsi device cannot be successfully removed.
This patch ignores the associated return value for that scenario and
marches forward in the teardown process.

Fixes bug 1022520

Change-Id: Ib28790892eebe341ac10a92250cf872605fefe9b
associate_floating_ip an ip already in use

Fixes bug 10014860 to be compliant w/ AWS EC2 spec
instead of return exception.FloatingIpAssociated,
disassociate from old fixed_ip and associate to
new fixed_ip.

Change-Id: I54ab3492092940dbb01f2787cc329a0162698568
Adds diagnostics command for the libvirt driver.

This code provides an implementation of the get_diagnostics
API for the libvirt driver. This API is invoked by the
'nova diagnostic' command.

There is no existing formal specification for what data items
to expose from the 'get_diagnostics' API, so the XenAPI driver
is used as an informal guide. The Xen implementation currently
produces the following output:

+----------------+-----------------+
|    Property    |      Value      |
+----------------+-----------------+
| cpu0           | 4.3627          |
| memory         | 1171088064.0000 |
| memory_target  | 1171088064.0000 |
| vbd_xvda_read  | 0.0             |
| vbd_xvda_write | 0.0             |
| vif_0_rx       | 3223.6870       |
| vif_0_tx       | 0.0             |
| vif_1_rx       | 104.4955        |
| vif_1_tx       | 0.0             |
+----------------+-----------------+

The new libvirt implementation will return a similar set of stats
for guest CPUs, memory, disks and network interfaces, resulting
in the following output:

+------------------+------------+
| Property         | Value      |
+------------------+------------+
| cpu0_time        | 2870000000 |
| memory           | 524288     |
| vda_errors       | -1         |
| vda_read         | 262144     |
| vda_read_req     | 112        |
| vda_write        | 5606400    |
| vda_write_req    | 376        |
| vnet0_rx         | 63343      |
| vnet0_rx_drop    | 0          |
| vnet0_rx_errors  | 0          |
| vnet0_rx_packets | 431        |
| vnet0_tx         | 4905       |
| vnet0_tx_drop    | 0          |
| vnet0_tx_errors  | 0          |
| vnet0_tx_packets | 45         |
+------------------+------------+

The implementation has only been tested with the libvirt KVM
driver.

In a future a formal specification for the required data
format should be written and all virt drivers made to comply.

Finally, the fakelibvirt driver has been updated in order to
support the methods required by the get_diagnostics method.

For those interested, below is the formula to extract the
CPU usage based on the CPU time. I've added it here for reference
since it's not used at the moment and it took quite a while
to locate it in the libvirt documentation.

  usage = 100 * (current_cpu_time - last_cpu_time) /
          (measurement_interval * (10 ** 9))

Fixes bug 986200.
Change-Id: I92da392f43d0840ed31cfd88f54570ea5830bb4a
Don't block instance delete on missing block device volume

It's easy to create an undeletable instance by specifying an invalid
volume id in the block device mapping. The instance errors, but
subsequent delete fails because the volume is missing. We already
ignore the DiskNotFound case during delete, so this adds a case for
VolumeNotFound.

Change-Id: Ibc516ee4452d088cddf10bbeb315697ce900e5c5
refactor vmwareapi_conn => vmwareapi/driver

Part of bp:virt-driver-cleanup

Make the vmwareapi driver consistent in naming as a driver
instead of a connection.

Get all the vmware driver parts into the same directory

Change-Id: I7c7e7c391086cf2fc32e8cb85d557005d417042f