Fixes a race condition on updating security group rules

During creating the instance, compute manager makes an rpc call
to "allocate_for_instance" from the network manager, which will
in turn end up making a rpc cast to each compute_manager with
an instance in the group(s) than need to be refreshed calling
refresh_instance_security_rules.

When an instance is being created is in a group that has a
rule referring to itself, that chain will end up trying to
refresh the rules for the instance which is still being created.

That call will find its way down to do_refresh_instance_rules()
in libvirt/firewall - which will in turn try to get network_info
from the global "network_infos" in firewall.

But the entry in network_infos for this instance might not of
been added yet, as that only happens when the create of the
instance calls "prepare_instance_filter which comes later in
the sequence as part of driver.spawn

The fix here adds a synchronisation lock to the incomming
refresh call so that it is blocked until any in-progress
thread on the instance is finished.

Change-Id: I550e286aa4777b011633550109e18e03d63a4bbb
Fix inaccuracies in the development environment doc.

Change-Id: If2b0884de54b75e40896cee9b4fb5eb805de2624
Allow fixed to float ping with external gateway.

If you ping an a floating ip from an instance with only a fixed ip, the
traffic will be sent to the default gateway. If the default gateway has
a route to the floating ip, it will then be sent to the host of the
instance with the floating ip. The source address will be on the fixed
network, so if the floating addresses are on a different network, the
receiving host to drop the packet due to the default rp_filter.
Essentially, the route for the fixed range is on a different interface
so it the kernel assumes the packet is spoofed and drops it.

In order to fix this issue this patch adds a new config option called
force_snat_range. Traffic that originates in fixed_range and has a
destination in force_snat_range will always be routed. This forces
the packet to hit the fallback floating roule and be snatted to
routing_source_ip. This means the traffic will originate from the
proper network and the packet will make it through.

DocImpact

Fixes bug 1122335

Change-Id: I7e31c25d37a5c4fd02c8238aec58a28af3fd7f6a
Added a service heartbeat driver using Memcached.

Today the heartbeat information of Nova services/nodes
is maintained in the DB, while each service updates the
corresponding record in the Service table periodically
(by default -- every 10 seconds), specifying the timestamp
of the last update. This mechanism is highly inefficient
and does not scale. E.g., maintaining the heartbeat
information for 1,000 nodes/services would require 100 DB
updates per second (just for the heartbeat).

This patch adds nova.servicegroup.drivers.memcached, a
service heartbeat driver using Memcached. You can reduce
DB updates with it.

blueprint memcached-service-heartbeat

Change-Id: I60bdb1cfbce1fea051f276ebfd6ccc4ad8fe6d2b
Stub additional FloatingIP methods in FlatManager

Fixes bug #989746.

Added some stub methods for floating ip allocation in FlatManager in
order to prevent exceptions from being raised when they are called.

Changed get_floating_ip_by_address() in FlatManager to return a dict
with expected keys for a floating ip, rather than None.

Updated patch from Andrew Laski and added unit tests.

Change-Id: I4ee1f5cf986b6f3411605aae5c1bc4e8cc2377b1
Fix script argument parsing.

The vm_vdi_cleaner.py script's argument parsing was broken and unloved.

Change-Id: I1662534d65fd0e8472e092ed9f10d6cc059f1cab
Delete baremetal interfaces when their parent node is deleted

Change-Id: I504cbed0adea86cab840c40ea9b9f96faf299f73
Assign unique names with os-multiple-create.

This patch introduces a new configuration option,
multi_instance_display_name_template, which makes
it so that all instances are given a unique
display_name and hostname when requesting that
multiple instances get created at the same time.

Implements bp multi-boot-instance-naming

DocImpact

Change-Id: Id8e7dac1adadbb101dd1027807867ab13d3b7e80
Add image to request_spec during resize

Image is needed by ImagePropertiesFilter during scheduling.

Resolves bug 1117882.

Change-Id: Icfbba85e2fdeb609d5fdabb6a7c3167505fe23b0
clean up missing whitespace after ':'

Change-Id: I85a0348b978d4045ebb12a99f1bd9d5888a65b3b
libvirt: Fix LXC container creation

Recent changes to the blockdev creation introduced in
7be531fe9462f2b07d4a1abf6687f649d1dfbb89 caused a regression
when LXC containers were being created.

When the image is being fetched from glance, the disk mappings
for the disk bus, disk information, etc is checked before being
copied. However since LXC does not understand this concept it was
not passing any information, so the image was never being copied.

Send some basic information about the disk when the instance is being
created in order for the image to begin. Fixes LP: #1126348

Change-Id: I90daf853903737013be0cd23f78c7f61e4e15a77
Signed-off-by: Chuck Short <chuck.short@canonical.com>
Fix instance can not be deleted after soft reboot

The reason is that:
1. soft reboot will wait for instance become to 'shutdown', and then start it
2. delete operation also wait for this, and then clean up the instance
3. if soft reboot found the instance become to 'shutdown' firstly, it will
start it immediately
4. then the delete operation will go to the _wait_for_destroy loop, and the
loop may be endless
5. when we delete the instance again, because the lock was hold by the delete
operation before, this one will wait the lock and don't implement actually.
So the domain id is checked during _wait_for_destroy loop, if it changed and
the instance is still running, we need to destroy it again.
If the domain is booted after _wait_for_destroy, it may result in
unfilter_instance failed because the nwfilter is in use, so doing the same
thing as above.

Fixes Bug #1111213

Change-Id: I98dc902dd86fa828f5821465c611953e08f9f637
Add libvirt XML schema support for resource tuning parameters

Extend the libvirt XML config classes to support for CPU, disk IO,
network traffic tuning parameters.

Blueprint: quota-instance-resource
DocImpact
original reviews https://review.openstack.org/#/c/14151/

Change-Id: I2827ea0e731fefe5bbaec6c385268a3f5b89f7f6
Add support for instance vif traffic control.

Part 2 for blueprint quota-instance-resource, by adding exposed params
"vif_inbound_average, vif_inbound_burst, vif_inbound_peak,
vif_outbound_average, vif_outbound_burst, vif_outbound_peak" through
nova flavor-set, user can implement instance net traffic control policy.
for example:
nova flavor-set m1.medium set vif_inbound_average=10240000,
then the max inbound speed is 10MB/s. The usecase details of other params
will be noted on openstack-manuals.

Partly implement blueprint quota-instance-resource
DocImpact
original reviews https://review.openstack.org/#/c/14151/

Change-Id: I9934daadbf0e136e88530f5c23b556d070395bd0
remove intermediate libvirt downloaded images

* nova/virt/images.py (fetch_to_raw): When converting a downloaded
image to raw, ensure we delete the original download image.
I.E. the .part file.  The regression was introduced in 7c265343.
* nova/tests/test_libvirt.py (test_fetch_raw_image): A new test
for this function which wasn't tested previously.

Fixes bug: 1125068
Change-Id: Ie6f2cff1a554b8500fd66f0832566ae1b317be40
Fix typo/bug in generic UC dropper

Change *fields -> *columns.

There should be tests for other backends mysql and postgres
but at this moment it is not so easy to implement it.

Fixes bug 1123848

Change-Id: I65a618760d499501589d5536c9f7dc64483b19ed
Imported Translations from Transifex

Change-Id: I1016d574022230b3fdfd6ae580a657aeb45bb1a0
Add generic dropper for duplicate rows

Add generic function that help us to drop duplicated rows from table,
before creating unique constarints. It is very usefull in migrations,
because it will allow DBA to make migrations without IntegrityError
exceptions.

It deletes all duplicates of rows except row with max value in id column.

blueprint db-unique-keys

Change-Id: I7e48d3eb78cf9a5d30752541b143b0c9fa1f838a
Remove race condition (in TaskLog)

Delete all duplicate rows with same (task_name, host, period_beginning,
period_ending) expect one with the biggest value in `id` column.

Create UC on fields (task_name, host, period_beginning, period_ending)

Replace Select then Insert -> Try to Insert.

blueprint db-unique-keys

Change-Id: I99f45a6de6630482e9c56738f894cc55a37fffaf
Merge "clean up missing whitespace after ':'"