Adds XML namespace example for disk config extension

Adds XML namespace and specifying of the disk config value
to the disk config extensions api sample test data for server
creation. This is being done so the api samples used by the
documentation contain the example of this usage.

DocImpact

Change-Id: Icd7aa3743f71dd1cb8575605e5f1ce5da6bd37bf
Closes-Bug: 1189817
all_tenants=0 should not return instances from all tenants

Fixes set up of server list filters so specifying all_tenants=0
or all_tenants=False when requesting a list of servers does not
result in instances from all tenants being returned.

Change-Id: I72575e4e2cf9019cfd5bf3b07ada53cadf2001e2
Closes-Bug: #1240347
Move get_all_tenants policy enforcement to API

Cleanup to move the enforcement of the get_all_tenants policy
enforcement to the API level. The all_tenants search option in
compute/api.py is only used to trigger policy enforcement while
the logic required by all_tenants (not specifying a project_id or
user_id filter) already resides in the servers API. The flag does
not need to be passed to the db level as it is ignored at that
level anyway.

Partially implements blueprint nova-v3-api

Change-Id: I5200c8912596ffbd32c5f7c6d74939782ff70260
Enforce compute:update policy in V3 API

Now that compute/api.py:update() is no longer being called for instance
updates the policy check is no longer enforced.  This adds policy
checking back in at the V3 API level so that the compute:update policy will
continue to function as it did before.

This was fixed in I365ad8ac2845d914deefb841c314a50f4869dba0 for the V2
API.

Change-Id: I9caa3e10fe029569c6fd467e7b86103dd6590f9b
Closes-Bug: 1242955
Sync log from oslo

5fef5a8105 allow keeping of existing loggers with fileConfig
fb8075a19d Add amqp=WARN,qpid=WARN to default_log_levels
4654a5d247 Don't override default value for eventlet.wsgi.server logging
41ddd7a975 _get_log_file_path explictly return, when logfile/logdire unset
e12e6b45f8 Make openstack.common.log Python 3 compatible
572cfecb4d Make Messages unicode before hitting logging
3243cdfbe9 python3: handle module moves in log

DocImpact

Change-Id: I7bd940a9a2dd4af3d876f2acc650f18675ca177b
Clean BDM when snapshoting volume-backed instances

When snapshoting a volume-backed instance, its block device mapping is
copied entirely into the image properties of the resulting snapshot.

This may lead to some errors when booting from that snapshot because the
fields 'id', 'insance_uuid' and 'connection_info' must be generated for
each instance instead of reusing the ones from the original instance.
For example, sharing the connection_info will cause the new instance to
try to connect to the original volume instead of creating a new one.

This patch cleans all the database-specific plus the connection_info
fields from the block device mapping when creating the snapshot.

Change-Id: I4190e9519d641c61b00b5d86c3885f2ae4fa86f3
Partial-Bug: #1246327
Process image BDM earlier to avoid duplicates

If the block device mapping from the image properties are not handled at
the same time as the ones defined from the API, we might make the false
assumption that the instance has no root and failing or, in the case
we're booting from an image that only defines a BDM and has no disk,
creating a local disk BDM as root and ending up with two root devices
which is forbidden.

This patch moves the handling of the image block device mappings to the
same method were we check the ones provided throught the API. This
allows nova to decide wether the instance needs a local disk from an
image if no root device is defined in any of the block device mappings.

Change-Id: Ide95357895ab4dd1338ab5ee3ec25294af1d010b
Closes-Bug: #1246327
Refactor UnexpectedTaskStateError for handling of deleting instances

To avoid having to repeatedly unfold UnexpectedTaskStateErrors and
do seperate handling of the 'deleting' task state from other states,
everywhere this exception is handled, introduce a separate exception
UnexpectedDeletingTaskStateError that derives from the above.

This way, most handling of InstanceNotFound to handle instances gone
during some action can be made to also include this new exception.

Change-Id: Ia833c2b2361f32c7e33c84ec14d3af6231119a99
Related-Bug: #1223843
Remove unused local variable in test_compute

Local variable 'volume' is assigned but never used in test_compute.py

Change-Id: If4630e6ee45b03bd743e42aad018ec31984091be
libvirt: pass instance to a log() call in the standard way

If we pass the instance in the correct way, then we get nicely
formatted logs which are easier for admins to use.

Change-Id: I27ab372a3388e2aa02491da07c616807bcbb14ef
Shrink the exception handling range

On evacuate action API, the exception handling for getting API
parameters is wide. This patch shrinks it to necessary range for
code cleanup.
In addition, this patch adds some negative tests for the lack of
"onSharedStorage" parameter.

Change-Id: Iccfa9d455758af432b5bcbb33842034f07e8fdc3
Fix type mismatch errors in NetworkTestCase

A few tests of NetworkTestCase use string literals as
network_id values. This works great when running tests
on both SQLite and MySQL, but fails on PostgreSQL, which
is very string when checking data types.

Blueprint: db-api-tests-on-all-backends

Change-Id: I090e42ff045379ac404e7bc5c91fbc82a52920b0
Reverse the quota reservation in revert_resize

If concurrent revert_resize requests happen, the second request
will fail because of save failure. In such situation, we should
revert the reservation.

Change-Id: I90f4189ec385ea323e62312e519ae65ec58cca66
Signed-off-by: Yunhong Jiang <yunhong.jiang@intel.com>
Cache compute node info in Hypervisor api

When extend hypervisor api, it would be much better to cache the
compute node info like is done for instance and flavor requests.

Partially implement blueprint pci-api-support

Change-Id: I73a96db7beb4cc0f017008f81e9f671382ad9105
Add v3 api samples for availability_zone

This patch adds api sample tests and api samples for availability_zone.
Because the detail's response use host name as the key of dict and samples
tests didn't support replacing key with variable, so startup nova's service
with specific hostname instead of uuid.

Partially implements blueprint v3-api-unittests
Partially implements blueprint v3-api-specification

Change-Id: I9090a73a8293f26d63744a10d7ffbfc887e37e7c
Add v3 api samples for scheduler-hints

Adds api sample tests and api samples for scheduler-hints

Partially implements blueprint v3-api-unittests
Partially implements blueprint v3-api-specification

Change-Id: If7bc47dc92c1e7f33f80ea567688d067c42fa219
Add V3 api samples for access_ips

Adds api sample tests and api samples for access_ips

Change-Id: I9d9249a48e7f7ccdff0146fbe5b25463d66a93a7
Conditionalise automatic enabling of disabled host

The change Ib8d67838ceb73c5b1cdc9498c17b335e9e5bb6f3 introduced
automatic behaviour to the nova-compute hosts, which will now
automatically disable themselves if they lose contact with libvirtd,
and re-endable themselves on a successful reconnection.

However, this tramples over any operational service disablement
that may be in place (on an otherwise working host) for any number
of good reasons.

As suggested in bug #1250049, this is a quick fix which attaches
a prefix to any host disable_reason string, and only re-enables
a host if that string is still present.

Fixes-bug: #1250049
Change-Id: Ia7544a357d1385db5fa92e500cb691f49436cfc9
Rename InstanceType exceptions to Flavor

Goes through and renames the various exceptions with InstanceType in the
name or message string to use Flavor instead.

Part of bp flavor-instance-type-dedup

Change-Id: I59d77086af0747f95ad61a569e784b766c8909a9
Move some libvirt specific flags into a group

This is backwards compatible via deprecated groups. This change
moves the flags defined in nova/virt/libvirt/driver.py into a
"libvirt" configuration group.

There are other flags which need to be moved, but this change
is quite large, so I will handle those in follow up changes.

Now that the flags in nova/virt/libvirt/driver.py are in a libvirt
group, we can drop the libvirt_ from in front of many of the flag
names as well.

Progresses blueprint libvirt-opts-group.

DocImpact: the following flags are moved into a "libvirt" flag
group: rescue_image_id, rescue_kernel_id, rescue_ramdisk_id,
libvirt_type, libvirt_uri, libvirt_inject_password,
libvirt_inject_key, libvirt_inject_partition, use_usb_tablet,
live_migration_uri, live_migration_flag, block_migration_flag,
live_migration_bandwidth, snapshot_image_format,
libvirt_vif_driver, libvirt_volume_drivers

The following flags have had the libvirt_ dropped from the start of
their flag name now that they are in a flag group:
    libvirt_type (now virt_type)
    libvirt_uri (now conn_uri)
    libvirt_inject_password
    libvirt_inject_key
    libvirt_inject_partition
    libvirt_vif_driver
    libvirt_volume_drivers
    libvirt_disk_prefix
    libvirt_wait_soft_reboot_seconds
    libvirt_non_blocking
    libvirt_cpu_mode
    libvirt_cpu_model
    libvirt_snapshots_directory

The name for the virt_type flag was influenced by bug 1195361,
which shows confusion as to what the flag is setting (the
virtualization type libvirt uses, not the driver libvirt uses
to achieve that).

Finally, the flag libvirt.nonblocking has been renamed to
libvirt.api_thread_pool, which is more descriptive.

Change-Id: I11e79eb5fd79c459374b9bf04707a4e6d79e993f