Allow fixed to float ping with external gateway.

If you ping an a floating ip from an instance with only a fixed ip, the
traffic will be sent to the default gateway. If the default gateway has
a route to the floating ip, it will then be sent to the host of the
instance with the floating ip. The source address will be on the fixed
network, so if the floating addresses are on a different network, the
receiving host to drop the packet due to the default rp_filter.
Essentially, the route for the fixed range is on a different interface
so it the kernel assumes the packet is spoofed and drops it.

In order to fix this issue this patch adds a new config option called
force_snat_range. Traffic that originates in fixed_range and has a
destination in force_snat_range will always be routed. This forces
the packet to hit the fallback floating roule and be snatted to
routing_source_ip. This means the traffic will originate from the
proper network and the packet will make it through.

DocImpact

Fixes bug 1122335

Change-Id: I7e31c25d37a5c4fd02c8238aec58a28af3fd7f6a
Added a service heartbeat driver using Memcached.

Today the heartbeat information of Nova services/nodes
is maintained in the DB, while each service updates the
corresponding record in the Service table periodically
(by default -- every 10 seconds), specifying the timestamp
of the last update. This mechanism is highly inefficient
and does not scale. E.g., maintaining the heartbeat
information for 1,000 nodes/services would require 100 DB
updates per second (just for the heartbeat).

This patch adds nova.servicegroup.drivers.memcached, a
service heartbeat driver using Memcached. You can reduce
DB updates with it.

blueprint memcached-service-heartbeat

Change-Id: I60bdb1cfbce1fea051f276ebfd6ccc4ad8fe6d2b
Stub additional FloatingIP methods in FlatManager

Fixes bug #989746.

Added some stub methods for floating ip allocation in FlatManager in
order to prevent exceptions from being raised when they are called.

Changed get_floating_ip_by_address() in FlatManager to return a dict
with expected keys for a floating ip, rather than None.

Updated patch from Andrew Laski and added unit tests.

Change-Id: I4ee1f5cf986b6f3411605aae5c1bc4e8cc2377b1
Assign unique names with os-multiple-create.

This patch introduces a new configuration option,
multi_instance_display_name_template, which makes
it so that all instances are given a unique
display_name and hostname when requesting that
multiple instances get created at the same time.

Implements bp multi-boot-instance-naming

DocImpact

Change-Id: Id8e7dac1adadbb101dd1027807867ab13d3b7e80
libvirt: Fix LXC container creation

Recent changes to the blockdev creation introduced in
7be531fe9462f2b07d4a1abf6687f649d1dfbb89 caused a regression
when LXC containers were being created.

When the image is being fetched from glance, the disk mappings
for the disk bus, disk information, etc is checked before being
copied. However since LXC does not understand this concept it was
not passing any information, so the image was never being copied.

Send some basic information about the disk when the instance is being
created in order for the image to begin. Fixes LP: #1126348

Change-Id: I90daf853903737013be0cd23f78c7f61e4e15a77
Signed-off-by: Chuck Short <chuck.short@canonical.com>
fix incorrectly defined ints as strs

Change-Id: Ie2fd2d9e9eaecba2b7713862211c3060c24e51a6
Fixes: bug #1128911
Remove unused nova.db.api:network_get_by_bridge

Change-Id: Id75a46aec71cf08196f8444bae9957a001aa4fcc
Change all instances of the non-word "inteface" to "interface"

Change-Id: I64f92a6346461670330a0c6e066b6685d4fc1b06
Ensure rpc result is primitive types.

This new rpc method was not ensuring that the result was converted to
primitive types.  This would cause it to break when using an rpc backend
other than impl_kombu.

Change-Id: I0a21b559d12ff7d0c96341fcec887b2436753eda
Default SG rules for the Security Group "Default"

Added in the API os-security-group-default-rules

This allows create, delete, list, and get (of individual rules) for
rules that will be pre-populated into the Security Group "default"
that is populated in all projects on creation.

These rules will not be applied retroactively, as it is designed
to allow the creation of a "reasonable" base-line set of sg rules.

The new rules live in a separate table that mirrors the relevant
structures of the security_group_rules table.

Added unit tests/API samples for the new API calls

Related to bp default-rules-for-default-security-group

DocImpact

Change-Id: I7ab51e68aff562bb869538197a0eca158fc3220c
Fibre channel block storage support (nova changes)

implements blueprint libvirt-fibre-channel

These changes constitute the required libvirt changes to
support attaching Fibre Channel volumes to a VM.

This requires a few new packages to be available on the system
sysfsutils -- This is needed to discover the FC HBAs
sg3-utils -- this package is needed for scsi device discovery
multipath -- This package is needed for multipath support.

Typical Fibre Channel arrays support exporting volumes via
multiple ports, so multipath support is highly desirable
for fault tolerance.  If multipath is not installed,
the new FibreChannel libvirt volume driver will still work.

If multipath is enabled, the new Fibre Channel volume driver
detects each of the attached devices for the volume, and
properly removes every one of them on detach.

In order to use this, the cinder volume driver's
initialize_connection will simply return a dictionary
with a new driver_volume_type called 'fibrechan'.

The target_wwn can be a single entry or a list of wwns
that correspond to the list of remote wwn(s) that will
export the volume.

return {'driver_volume_type': 'fibre_channel',
        'data': {'target_lun', 1,
                 'target_wwn': '1234567890123'}}

or

return {'driver_volume_type': 'fibre_channel',
        'data': {'target_lun', 1,
                 'target_wwn': ['1234567890123',
                                '0987654321321']}}

Change-Id: Ifccc56f960ef434f7cb56a9367e4cad288358440
Provide way to pass rxtx factor to quantum

This patch adds a config option quantum_extension_sync_interval which by
default is set to 600 seconds. After 600 seconds the next time nova needs to
create a resource in quantum it will requery quantum for the extensions
that it has loaded.

Hopefully in Havana something smarter will be implemented rather than using
time sync. I believe a better way to implement this would be to have each
compute host keep a pool of persistant sockets open to quantum for it to
communicate over. Resync'ing then would only happen if the connection to
quantum broke because quantum needs to be restarted in order to change the
extensions it is using

Implemenets blueprint pass-rxtx-factor-to-quantum

Change-Id: I001cfaf348a3f0c8144c0226559752555c486db0
Change CIDR column size on migration version 149

As discussed in review https://review.openstack.org/#/c/22238 cidr column needs
up to 43 characters in some cases, and this should be upgrade to avoid data get
lost.

Related to bug-1127696

Change-Id: I35d4565389d03fce86103289cf6447f824efeba0
Add a safe_minidom_parse_string function.

Adds a new utils.safe_minidom_parse_string function and
updates external API facing Nova modules to use it.
This ensures we have safe defaults on our incoming API XML parsing.

Internally safe_minidom_parse_string uses a ProtectedExpatParser
class to disable DTDs and entities from being parsed when using
minidom.

Fixes LP Bug #1100282.

Change-Id: Ib90d6379320ff1d007f8a661f7ddaa286ba6918e
Fix key check in instance actions formatter.

The 'key in object' check doesn't work for sqlalchemy models so nothing
was getting copied over during the format.  This patch switches to using
get() to retrieve a value.

Bug 1126593

Change-Id: I726c2a624928247de41a077c23fe80742cbf9044
Fix XML config tests for disk/net/cpu tuning

Due to a bug in the XML matching code, some XML formatting
problems related to disk/net/cpu tuning were not flaged as
errors. The problems were all simply ordering of the elements.
This patch also fixes some indentation problems at the same time

Fixes bug #1130146
Change-Id: Iee392ed823002b8231b33c27b487c2538b2dc8cd
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Fix XMLMatcher error reporting

The nova.tests.matchers.XMLMismatch class implements the
get_details() method returning a dictionary with the expect
and actual XML output. It is returning a string values, but
the testtools code requires that the dictionary values are
instances of the testtools.content.Content object.

To quote the 'get_details' docs on testtools.matchers.Mismatch

     :return: a dict mapping names to Content objects. name is a string to
         name the detail, and the Content object is the detail to add
         to the result. For more information see the API to which items from
         this dict are passed testtools.TestCase.addDetail.

Related bug #1130146
Change-Id: I75ec07d8b34b04c7b8009229fda49b344ed13bec
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Merge "Assign unique names with os-multiple-create."
Merge "Stub additional FloatingIP methods in FlatManager"
Support running periodic tasks immediately at startup

This change adds a parameter to allow periodic tasks that would
otherwise have to wait for a lengthy initial interval to run immediately
when the periodic tasks are started. This patch also applies the new
facility in the the _sync_power_states task in the compute manager to
get more timely access to Openstack server state when the compute node
starts up.

Change-Id: I461a5dbaa7d18919941ec31402910eef188d8e8c