Don't truncate osapi_glance_link or osapi_compute_link prefixes

When osapi_glance_link_prefix or osapi_compute_link_prefix is
defined and used to update the links return in API responses, do
not drop the path component of the overriding link prefix.

Change-Id: I290e3f85b94e28d25a54621c1699081cbbf48456
Closes-Bug: 1304099
Adds verbosity to child cell update log messages

Child Cells frequently update their parents with their capabilities and
capacities. The debug log message contains the capabilities or capacities
that are being updated. This patch adds the names of the parent cells to
which the information is being sent to the debug log message.

Change-Id: Ic34191da191369e64a016c8300a8da294aaa9e07
Closes-Bug: 1351301
Lock attach_volume

There are some issues with instance and volume cleanup when the volume
is not in a fully attached state so it will be safer to not attempt a
terminate_instance while there are attachments in progress.

Change-Id: I4347794e51004a881bf4ef5ee30f65ac28773e51
Closes-Bug: #1355348
Add a policy for handling retrospective vetos

For some time, we've been informally implementing a "revert early"
policy around handling -2 worthy complaints after a patch has been
merged. These reverts can be contentious, so it's appropriate we make
it very clear what the policy and procedure here is in order to lessen
the contentiousness of these reverts.

Co-authored-by: Michael Still <mikal@stillhq.com>

Change-Id: I90e4153a80c859a928630809aef31e7313f08906
Pass certificate, key and cacert to glanceclient

When glance is configured with ssl nova does not pass it's certificate,
key, or cacert.  Updated the code to pass the certificate, key, and cacert used in the
nova.conf file, if they are set.  Removed nested if statement.

Change-Id: I23fcd070994e435cb42aa09c824d13553503a7c5
Closes-Bug: #1279381
Remove unused oslo.config import

Commit 46e88320e6e6231550f3e2b40312c51f55e059f5 added
the oslo.config import in one of it's patch series but
in the revision that was ultimately merged, the import
was no longer used - but we forgot to remove the import.

This cleans that up.

Change-Id: I3fcfdab625be68a19162b5053ad4f6f180a72a4d
Add sqlite dev packages to devref env setup doc

While setting up a new Ubuntu Trusty development
system without devstack I ran into an undocumented
native package dependency for sqlite, so add those
here.

Change-Id: Ib82ea58e85f6ccab6008f2d19a31d0e890136ca7
libvirt: deprecated volume_drivers config parameter

In Icehouse the 'vif_driver' parameter was deprecated

  commit 9f6070e194504cc2ca2b7f2a2aabbf91c6b81897
  Author: Lars Kellogg-Stedman <lars@redhat.com>
  Date:   Fri Apr 4 14:58:12 2014 -0400

    mark vif_driver as deprecated and log warning

and then in Juno removed

  commit 7561c8ded211d53e8745d1420a73b82bd0fc35cf
  Author: Daniel P. Berrange <berrange@redhat.com>
  Date:   Thu Jul 24 11:53:47 2014 +0100

    libvirt: remove 'vif_driver' config parameter

The 'volumes_drivers' parameter does the same job as the
'vif_driver' parameter, but for volumes instead of vifs.
The reasons for deprecating it, in preparation for its
removal in Lxxxx are thus the same.

The API for the LibvirtBaseVolumeDriver class is not
considered to be a public extension point. As such it
is not required or expected to remain compatible across
releases.

The main reason for its existance is for people who are
developing new cinder drivers, to be able to write out
of tree nova volume drivers. The recommended practice
is to instead work in-tree to nova on a branch. This is
an approach that would be required in order to submit
the work for review and eventual merge in any case.

DocImpact: libvirt.volume_drivers config param for
           nova.conf is deprecated, to be removed in
           the Lxxxx release
Related-bug: #1362191
Change-Id: I1f884f36e24fe478922671ca3eaf6b7c21fd8a5a
Sync oslo-incubator module log:

Pull in new default log level for stevedore.

Generated with: python update.py --base nova ../nova --modules log

Changes:
d9060f6 Check for namedtuple_as_object support before using it
ad248f6 Specify namedtuple_as_object=False when using simplejson
037dee0 Set stevedore log level to WARN by default

Change-Id: Ibe9d2e6e8d97e3199b18ef59c2346a7157574273
Check requirements.txt files for missing (used) requirements

Invoke the pip-missing-reqs tool to find packages used by nova but
not explicitly included in its requirements.txt file.

Add simplejson to the requirements to fix the failing check.

Change-Id: Ia90416c613c0acea04f3d4ab4898f550cb702a49
libvirt: fix bug when releasing port(s)

During the process of deleting an instance, if
serial_console is enabled it has to check domain for
allocated port(s) to be released - Currently this is doing
after the domain has been undefined from libvirt which creates
a bug. This patch fixes the problem by moving this part of code before
undefining the domain.

Also 5fa74bc0b2 moves that part of code inside the method
_detach_encrypted_volumes which has nothing to do here.

Change-Id: Ie187748deff7b0143d1ca26e767c6665746094c5
Closes-Bug: #1366832
Drop verbose and useless nova-api log information

Instead of printing two consecutive debug logs if there is a body, print
one. If no body, only display the method being called and not the
content-type as that isn't used for debugging. Shrink log output to make
log files smaller and easier to read.

Change-Id: I0e07876eee0064bd429ce9ea3254f7b95b991511
Fixes Hyper-V volume mapping issue on reboot

Hyper-V does not manage iSCSI mounted passthrough disks
correctly on reboot in some circumstances where disk
number assignment can change, resulting in instances potentially
booting with the wrong volume attached.

Co-Authored-By: Adelina Tuvenie <atuvenie@cloudbasesolutions.com>
Co-Authored-By: Adrian Bora <abora@cloudbasesolutions.com>

Change-Id: I99fa75aaa950de2ada8a03bd23c910c3ff810d69
Closes-Bug: #1322926
Bump FakeDriver's resource numbers

Bump the amount of memory and disk that FakeDriver reports so one can
schedule larger instances. One should be able to schedule an instance
larger with more then 8GB of ram onto the FakeDriver.

Since the FakeVirt driver is now used for much more then just unit
tests, subclass FakeVirt to decouple the amount of resources FakeVirt reports
from the amounts in the api_samples. This way the api_samples don't need changing every
time the resource amounts reported by FakeVirt are tweaked.

Change-Id: Ifd3b8bd1abf985d279de334776ed5feb4fa72a27
Related-Bug: #1268943
disk/vfs: ensure guestfs capabilities

Ensures that guestfs is available and well configured. Also
when checking for capabilities this commit introduces an in-memory
flag to avoid repeating the process if succeeded.
If we are able to load guestfs but something wrong happens a fatal
error is raised.

DocImpact: We should add a not for ubuntu like systems that use
libguestfs to execute the command 'update-guestfs-appliance' for
configuring it.

Closes-Bug: #1275267
Closes-Bug: #1157922
Change-Id: I97b3a23829ea1f3aadfe08ca6448b35010d2f312
fix up assertEqual(None...) check to catch more cases

To finally close out bug #1280522 it turned out that our hacking check
to eliminate assertEqual(None, ...) was pretty sub optimal. The
perfect fix would actually use the tokenizer, however this is a better
regex which catches a few more missing bits in our codebase. It's
probably close enough for most of what we need.

Fix the relevant tests that would fail the updating hacking check.

Change-Id: Ic8cb1192e001409d827c8da55fe536681895944b
Closes-Bug: #1280522
virt: move assertPublicAPISignatures into base test class

The nova/tests/virt/test_driver.py has a useful method for
validating subclass APIs vs the base class contract. This
can be used in many places in Nova, so move it into the
base TestCase class for broader reuse.

Related-bug: #1369151
Change-Id: Ib49e888aae31fb0d47c1d11f35648e9cb2595c75
Split up libvirt volume's connect_volume method

The connect_volume method in libvirt/volume.py does a couple of
things. It does the host OS integration/setup tasks, and it returns
the libvirt xml. In some cases, only the libvirt xml is required,
and the setup tasks are not necessary.

The following patch separates the setup tasks and libvirt xml return
in connect_volume. A new method, called get_config, is added to just
return the xml config, separate from the connect_volume method.
The connect_volume method will continue to return the libvirt xml,
so that existing functionality is not broken. This way, get_config
can be used to get the libvirt xml without any side-effects, e.g.
re-running shell commands to setup the volume.

There will be a follow up patch, where libvirt's driver.py will
switch over to use get_config and connect_volume will no longer
need to return the libvirt xml.

Change-Id: I5d300f8fe99f714d63d51197dc89f556c3a5d43b
Related-Bug: #1359617
Remove stale code from ObjectListBase

This patch removes some old attribute handlers, leftover from the time before
we had fields.

Change-Id: Ief8a37e1fd3a3079babc35595f6345984432cb4b
Update devref

* Devref mentioned 10.10 instead of 14.04 as the tested ubuntu
  environment
* Point to our official git mirror instead of github

Change-Id: I4514faeaebe200a2f616bc37fca02c2927edf599