Support EC2 API wildcards for DescribeTags filters

The EC2 API supports glob-style wildcards in filters.

From http://docs.aws.amazon.com/AWSEC2/latest/APIReference/ApiReference-query-DescribeTags.html:

"You can use wildcards in a filter value. An asterisk (*) matches zero or more characters, and
a question mark (?) matches exactly one character. You can escape special characters using a
backslash (\) before the character. For example, a value of \*amazon\?\\ searches for the literal
string *amazon?\."

This patch converts the glob-style regex to a python regex and uses re.match to do the rest. The
approach to the conversion was taken from the python fnmatch library.

Fixes bug #1205099

Change-Id: Ie9cac182faabf50fc0d40480db39a46a7e97fbec
Add task_state filter for nova list

e.g. nova list --status REBOOT

Fixes: bug #1196999

Change-Id: I55f3367952c1de756a1a87c7313a0dbb2da3f613
Set the image_meta for the instance booted from a volume

If the VM is booted from a volume, image_meta should be retrieved from the field
volume_image_metadata in volume.

Fixed Bug 1191069.

Change-Id: I4df185ab5eeb819327606163565501c845411a7e
Switch to Oslo-Incubator's EnvFilter rootwrap

Nova-Network used to have its own class for DnsMasq rootwrap
filtering. This was generalized into a new EnvFilter rootwrap
filter that can specify both commands and environments to pass.

Update to openstack.common.rootwrap to git a117358 of
oslo-incubator's rootwrap. This includes the following commits
merged from oslo:

    3b16d31 Remove DnsmasqFilter and DeprecatedDnsmasqFilter
    f9d1a59 Handle empty arglists in Filters
    233baa2 Handle empty PATH environment variable
    3e74c00 Add IpFilter, IPNetnsExecFilter and EnvFilter
    8b95b1e Handle relative path arguments in Killfilter
    7119e29 Enable hacking H404 test.
    ea78eec Update KillFilter to stop at '\0' for readlink() function.

Related to: blueprint quantum-common-rootwrap

Change-Id: I125e5724259850dc4768fb236cbfa5df199008c0
Support networks without gateway

Launching an instance when using network injection on a network
containing a subnet that has no gateway fails in scheduling with
a KeyError when using a legacy network_info provider and generates
an invalid network file for the others.

The following changes were done:

* The legacy transformer for network_info now always provides
  a gateway/gateway_v6 field, with a value of None if no gateway
  is available (this is similar to the other fields that are empty).
* The interfaces template was modified to include the gateway line
  only if a gateway is provided for the interface.

Fixes: bug #1207878
Change-Id: I887757a6bcba528059ace8831a7d23a31b08c630
Put fault message in the correct field

Currently when adding an instance fault, the message field looks
for kwargs['value'], which many exceptions do not have.  This
results in the message field being set to the name of the
exception class, which in some cases is not useful (see the bug
report for an example).

This change puts the exception message in the message field, and
removes it from the details field since there's no need to put it
in both places.  That leaves the details field for actual details,
like stack traces.

Fixes bug 1203880

Change-Id: Ic92a9bd9cb2e139ce7a9f3f1299b7a6ccae82dd1
Personality files can be injected during server rebuild

Injected files are ignored during a server rebuild, this patch fixes
this issue.

Fixes: bug #1187629

Change-Id: I5e7776de07c8513e36c2926b1fc58c66b382a5ec
Fix EC2 API Fault wrapper

Fault wrapper now returns actual EC2 error code instead of HTTP
status code which was ridiculously incorrect behavior.

Fault:__call__ wasn't covered by any unit tests, so I added one.

Implements: blueprint ec2-error-codes
Change-Id: I41f7f0aa09e2925882a0da941f9ea5d4e694e65c
maint: remove redundant default=None for config options

These 73 instances were inconsistent and redundant.

Change-Id: I298f2406535d0805caebdca4f2729d720dd5222f
Remove old legacy network info model in libvirt driver

Update libvirt driver, virt firewall and hypervisor unit tests to
use nova.network.model instead of legacy "network,mapping" tuple.

Partly implement blueprint nova-network-legacy

Co-author: Amir Sadoughi<amir.sadoughi@gmail.com>
Change-Id: I4d38bb81a4f64efdb78f5da52fbbb382981e0b96
Removed code for modular exponentiation, pow() already does this

Change-Id: If21cbcbeb60618769e1659a90381416dfc43ebc4
Cleanup exception handling on evacuate

Some cleanup around catchall exception handing
on evacuate v2 extension to bring it in line
with v3 exception handling. Tests included.

Fixes bug 1210570

Change-Id: I2f2f958acb80240c0fb25f3d538fa204611a33d1
Improve performance of driver's get_available_nodes

get_available_nodes() is called every time an instance is updated.  That
ends up calling get_host_stats(refresh=True), resulting in a
libvirt/XenAPI/etc call.  On a build of a new instance when the instance
record is updated often (task_state and progress), this results in a lot
of unnecessary driver calls.

This changes get_available_nodes() to take a refresh kwarg to pass
through to get_host_stats(), defaulting to False so that it uses
the cache.  Places that used get_available_nodes() and are sensitive to
stale data are changed to add refresh=True.  Also adds a helper driver
method for 'is_node_available'.  And make sure that even if drivers'
get_host_stats() methods get refresh=False that they refresh when they
have no stats.

Fixes bug 1210264

Change-Id: I22c8d70e3e70f2d2421915da66c2618688e0db2b
fix libguestfs mount order when inspecting

variable `mounts` sorted value is
[
    ('/boot', '/dev/vda1'),
    ('/', '/dev/vda2'),
    ('/var', '/dev/vda5'),
    ('/home', '/dev/vda6'),
    ('/home/q', '/dev/vda7')
]
but `/boot` directory is in `/` partition.

Fixes: Bug #1210371
Change-Id: I3d9cd59237bc4fc277415104c9170d2e372a1c25
Fix H501: Do not use locals() for string formatting

Change-Id: Ide587d3311c22f698a82eeee210c5bdbe24eb484
Remove clear_rabbit_queues script

The clear_rabbit_queues script has been around forever, but it doesn't
appear to be widely used - e.g. there is no user documentation for it
and I don't see any mentions of it on mailing lists or Q&A forums.

Also, RabbitMQ now includes a management plugin and CLI which allows
you do do e.g.

  $> rabbitmqadmin delete queue name=$topic
  $> rabbitmqadmin delete queue name=$topic.$host
  $> rabbitmqadmin delete exchange name=nova

oslo.messaging does not expose an API equivalent of this command since
even the existance of a queue is transport driver specific. We could
consider adding transport-specific admin CLIs, but probably not where
the transport already has suitable admin CLIs elsewhere.

For reference, this script was added by this merge:

  https://code.launchpad.net/~mdragon/nova/lp794635/+merge/71437

blueprint: oslo.messaging
Change-Id: If5e6354e8ace4c0b4425c96de558a5cbab4c98bd
Port "Accept is_public=None .." to v3 tree

The following commit(I2a9b1c24cd744c8819c1d41e0ad103d05398401c) have
not been merged to v3 tree now, because the commit was merged to the
existing api after porting flavors.py to v3 tree.

    commit 645c2d3a06bbf8b0854ebcc539ad33dd706866c5
    Author: Xavier Queralt <xqueralt@redhat.com>
    Date:   Wed Jun 12 14:59:29 2013 +0200

        Accept is_public=None when listing all flavors

This patch ports the commit to v3 tree.

Partially implements blueprint nova-v3-api

Change-Id: Id1ff1366c16f0caac50e279b2add5ec9d9b9bf5f
Moves compute.rpcapi.prep_resize call to conductor

This patch moves the prep_resize logic out of scheduler.manager
and creates it on conductor.manager.ComputeTaskManager.

Partially implements bp cold-migrations-to-conductor

Change-Id: I8458071fb13505f55f8d2155c9ec5ee938b1005f
Add option for QEMU Gluster libgfapi support

If gluster is present in qemu_allowed_storage_drivers, glusterfs's
backend will pass a disk configuration to QEMU. This allows QEMU to
access the volume using libgfapi rather than mounting GlusterFS via
fuse.

Implements blueprint glusterfs-native-support
DocImpact

Co-authored-by: Flavio Percoco <flavio@redhat.com>
Change-Id: I939cac05ee333b77abcaa9bd132d5eb3308e60ff
Merge "maint: remove redundant default=None for config options"