maint: remove unused imports from bin/nova-*

Change-Id: I80b58ac41bbb0517221d5fd5ba9938f2b7836779
Allow multiple dns servers when starting dnsmasq

Previously we only allowed one upstream dns server even though we
can store two in the network object. This changes the dns_server
flag to a MultiStrOpt so it can be specified multiple times.

It also adds a new config 'use_network_dns_servers' which specifies
to uses the dns servers from the network if they are set. This
defaults to false to ensure configs do not need to be changed.

DocImpact

Change-Id: I4c96259bce25036392afe6e027ec47b92679ecbc
Fixes a race condition on updating security group rules

During creating the instance, compute manager makes an rpc call
to "allocate_for_instance" from the network manager, which will
in turn end up making a rpc cast to each compute_manager with
an instance in the group(s) than need to be refreshed calling
refresh_instance_security_rules.

When an instance is being created is in a group that has a
rule referring to itself, that chain will end up trying to
refresh the rules for the instance which is still being created.

That call will find its way down to do_refresh_instance_rules()
in libvirt/firewall - which will in turn try to get network_info
from the global "network_infos" in firewall.

But the entry in network_infos for this instance might not of
been added yet, as that only happens when the create of the
instance calls "prepare_instance_filter which comes later in
the sequence as part of driver.spawn

The fix here adds a synchronisation lock to the incomming
refresh call so that it is blocked until any in-progress
thread on the instance is finished.

Change-Id: I550e286aa4777b011633550109e18e03d63a4bbb
Return dest_check_data as expected by the Scheduler

the scheduler driver expects data in return when
calling compute_rcp.check_can_live_migrate_destination
in method 'schedule_live_migration'.

In turn check_can_live_migrate_destination calls
check_can_live_migrate_source that also expects data
returned from the driver.

Fixes bug #1118491

Change-Id: I3dbfa716d5d9bc849c1f67005e7bb1b8ba1ccd30
preserve order of pre-existing iptables chains

Adds new configuration options:

    iptables_top_regex='' (Default)
        When set treated as a regular expression to match
        iptables rules that should always be placed at the
        top of the table before the nova chains.

    iptables_bottom_regex='' (Default)
        When set treated as a regular expression to match
        iptables rules that should always be placed at the
        bottom of the table right before the COMMIT

Additionally the existing iptables tests were no longer
actually testing the chains we use. In some cases we
were looking for chains that haven't existed in the
code base for 2 years. I took the oppurtunity to update
all tests to actually test for the chains we care about.

Flags: DocImpact

Change-Id: I335ca3712d6dd37051cc8e46e1237aaf66a4a94e
Fixes: bug #1116562
Fix inaccuracies in the development environment doc.

Change-Id: If2b0884de54b75e40896cee9b4fb5eb805de2624
Fix regression in non-admin simple_usage:show.

Commit 6ff32210772a67a1b526d9d784030afc90f3ce99 optimized the db query
for usage but replaced a call that had require_context with one that
had require_admin_context. This causes usage show to throw a 403. We
check for admin_or_owner in policy, so there is no need to check for
admin context at the db layer.

Fixes bug 1122267

Change-Id: Iff362f46d83a710f3883538bcb646e3457e0ba64
Allow fixed to float ping with external gateway.

If you ping an a floating ip from an instance with only a fixed ip, the
traffic will be sent to the default gateway. If the default gateway has
a route to the floating ip, it will then be sent to the host of the
instance with the floating ip. The source address will be on the fixed
network, so if the floating addresses are on a different network, the
receiving host to drop the packet due to the default rp_filter.
Essentially, the route for the fixed range is on a different interface
so it the kernel assumes the packet is spoofed and drops it.

In order to fix this issue this patch adds a new config option called
force_snat_range. Traffic that originates in fixed_range and has a
destination in force_snat_range will always be routed. This forces
the packet to hit the fallback floating roule and be snatted to
routing_source_ip. This means the traffic will originate from the
proper network and the packet will make it through.

DocImpact

Fixes bug 1122335

Change-Id: I7e31c25d37a5c4fd02c8238aec58a28af3fd7f6a
Make scheduler remove dead nodes from its cache.

Scheduler HostManager maintains and updates a cache of all compute
nodes in its host_state_map. However, entries were never deleted from
this mapping, even when the compute nodes were marked as deleted.

This patch removes a state_key from host_state_map when it is not
returned from db.compute_node_get_all, and logs a notice about it.
It also adds unit tests which check for this behaviour.

Change-Id: Ibe6c98dd6c2eb02983db47f263d86cb1b76e1c98
Allow aggregate create to have None as the az

Aggregates do not require a zone any more, but in order to keep the current API,
Allow for availibilty zone to be set to None.

Fixes bug 1123468

Change-Id: I5209311f3417b5d7cdf343e5c06104357ed523d5
Fix passing conductor to get_instance_nw_info()

The recent metadata patch missed this one spot, which wasn't caught
because the network stubs weren't updated for the signature change.

Fixes bug 1123435

Change-Id: Iebeb98bba62189476af2c0b85615ea407a8b2ecd
Update tools/regression_tester

* Clarify wording (don't use 'success' and 'failure' instead use 'found'
  and 'no regression test')
* If don't pass gerrit review number run on current branch.  In
  preparation for running via jenkins.
* If no tests were changed, don't run any tests

Change-Id: I5e876e4e5071fc27461d2411fc33db1bf1724405
Add "is not" test to hacking.py

HACKING.rst was updated in I7534ef73e6fd525fd8f4bee594a4b37524699c08.
This adds hacking.py enforcement.

Change-Id: I5d75d536762449d0a7ae38213addd02aeceede0b
Use a more specific error reporting invalid disk hardware

Instead of using NovaException for reporting invalid user
specified disk hardware, use UnsupportedHardware and
UnsupportedVirtType exceptions.

Change-Id: I68ed1c0569bd0c0679e9fff3ffd436197f78a9f0
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Added a service heartbeat driver using Memcached.

Today the heartbeat information of Nova services/nodes
is maintained in the DB, while each service updates the
corresponding record in the Service table periodically
(by default -- every 10 seconds), specifying the timestamp
of the last update. This mechanism is highly inefficient
and does not scale. E.g., maintaining the heartbeat
information for 1,000 nodes/services would require 100 DB
updates per second (just for the heartbeat).

This patch adds nova.servicegroup.drivers.memcached, a
service heartbeat driver using Memcached. You can reduce
DB updates with it.

blueprint memcached-service-heartbeat

Change-Id: I60bdb1cfbce1fea051f276ebfd6ccc4ad8fe6d2b
Resize/Migrate functions for PowerVM driver

Added resize and migrate functionality to PowerVM driver.
Allows resizing of instance and cold migration of
an instance to different host.

bp powervm-compute-resize-migration

Change-Id: I723456266d3ceb4bd4a7ce4273008201b450f533
Stub additional FloatingIP methods in FlatManager

Fixes bug #989746.

Added some stub methods for floating ip allocation in FlatManager in
order to prevent exceptions from being raised when they are called.

Changed get_floating_ip_by_address() in FlatManager to return a dict
with expected keys for a floating ip, rather than None.

Updated patch from Andrew Laski and added unit tests.

Change-Id: I4ee1f5cf986b6f3411605aae5c1bc4e8cc2377b1
Module import style checking changes

* Implementing the * import detection (it is disabled for now)
* New style relative import testing based on syntax rules
* Old style relative import testing based on module search
* Inspection based solution replaced by PYTHONPATH search
  in order to avoid module compile and initialization steps
  (code execution) in a syntax checking phase.

This solution is faster and safer, but does not able to recognize
modules added dynamically to the module scope.

Change-Id: Ifc871f4fdbcd4a9a736170ceb4475f4f2cbe66bc
Add GlusterFS libvirt volume connector

This is to support GlusterFS as a storage backend for Cinder,
which works similarly to the NFS backend.

Change-Id: I047d58ba74d27273ba8f2639bcc77999f8be0f30
Correct misspelling in PowerVM comment

Correct a misspelling in a PowerVM comment

Change-Id: I12ab59826e8448a120ff3ef2b26034ea4aa2f1a0