Optimize 'fixed_ip_bulk_create' function

1) All single 'insert' operations were changed by
single 'multiline insert' operation

2) Explicit flush() was removed

3) Using code from oslo.db provides
getting error value from DBDuplicateEntry exception

Change-Id: Ib3d99b950bcf5e97ac5d6099d5ccfbe98906b339
Clearer default implmentation for dhcp_options...

'pass' is less clear than 'return None' for the base implementation.

Change-Id: Iff53d5eb7e88e18849696d858b3d910b6ea71e63
Remove flush_operations in the volume usage output

flush_operations is not used by upper layer code and also
according to
ftp://libvirt.org/libvirt/java/javadoc/org/libvirt/DomainBlockStats.html
http://libvirt.org/ruby/api/Libvirt/Domain/BlockStats.html

seems the last value should be errs instead of flush_operations
so remove it from libvirt driver layer.

Change-Id: I438b6fa40e9d97e3ffddda03919c5d577e91c155
image cache clean-up to clean swap disk

Currently swap backing files are never removed from the system.
The image clean-up code should recognise at least obsolete unused files
need to be removed.

Change-Id: I09519a73c7233f2dc6f46f6cd094a3e1cc3dc395
Partial-Bug: 1253991a
Add virtual interface before add fixed IP on nova-network

nova allow user to add fixed IP to an instance when the instance
is running. This action will fail due to no virtual interface
will be created before create fixed ip.
TypeError: 'NoneType' object is unsubscriptable will be reported.

Change-Id: I5885806d1965022816633d0105606e7aaf763b3a
Closes-Bug: #1294939
Use real disk size to consider a resize down

When resize an instance, we don't allow a resize down operation.
Currently, a flavor ephemeral size will be used to do the check,
but usually we might create ephemeral size less than flavor
ephemeral size. This patch use the real disk size to do the resize
instead of the flavor ephemeral size.

Also, the test cases are updated to change from dict to objects,
if we change the flavor from dict to objects will benefit from it.

Change-Id: I03006a2ee8edf310a3ad374ebb7e4cd4034f8250
Closes-Bug: #1324005
Don't touch info_cache after refreshing it in Instance.refresh()

We were touching info_cache to force it to be marked dirty in the
instance object after refreshing it. However, it isn't dirty, because
it has just been refreshed and the relationship to instance hasn't
been altered. Also, the dirty state on the instance object is
unconditionally reset immediately after this anyway.

Change-Id: Ibc8e9d95210d14c946f6f820492da2331c0d7fab
VMware: fix regression for 'TaskInProgress'

Commit 73d4a7c1b4681accd993c1109a9eff837ff612f4 caused a regression
with the snapshot deletion. That is, the snapshot deletion will retry
if the resource is busy. The fact that the exception was not supported
in olso caused the problem.

Version 0.6.0 supports this exception.

Change-Id: I12fcc7d039a123e681c51978b9cf632af8f9644b
Closes-bug: #1369463
VMware: pass vm_ref to _get_and_set_vnc_config

In the spawn refactor we get the vm_ref when the instance is created.
Passing it to the method _get_and_set_vnc_config saves a function
call. We are able to do this as we already have the vm_ref for the
instance in question.

Change-Id: I7b8d056af7f49ebd31beff2876edb9e6115d0a2c
VMware: pass vm_ref to _set_machine_id

Save a method call to get the vm_ref. In spawn this value exists so
we can pass it to the method if necessary.

Change-Id: Ied43fb09fb2eec2d2c22fce17b4f98ae3f58860e
VMware: Fix problem transferring files with ipv6 host

Need to protect the host name with '[' and ']' before
we create a http/https connection

Change-Id: I9320c47f446d1e6b6b088802c3916a88b118d9cf
Closes-bug: #1390906
Fixed quotas double decreasing problem

When we delete vm with status "soft-delete" we shouldn't change
quota values because they were changed during soft-deleting.

Closes-bug: #1333145
Change-Id: I3ed7b3245b8908faa02903c74503918f061016bd
Remove unused oslo.config import

Change-Id: I7cc52bfcda72faf9fbdf67553d21b35823505dd6
Specify storage IP for iscsi connector

The virt drivers supply a connector hash to cinder volumes when
attaching. This used to basically default to CONF.my_ip. This patch
allows the host to have a separate config option in the case the block
traffic is on a separate network.

DocImpact: Adds my_block_storage_ip config option for block traffic going
over a separate network.
Closes-Bug: #1347025

Change-Id: I2bab7226376acd7da78e038a230bf65613547ba6
Hard reboot doesn't re-create instance folder

When use share storage like ceph, instance's disk data is in ceph,
instance's disk doesn't rely on local instance folder anymore. So
when rebooting a instance, we should create the instance folder if
it is missed.

Closes-Bug: #1349978

Change-Id: I5948c7979a7dbe7764fba6cad1d9867893824024
Limit InstanceList join to system_metadata in os-simple-tenant-usage

The simple-tenant-usage API is calling get_active_by_window_joined with
all of the default instance attribute fields when really only
system_metadata is needed (to get the related flavor). That adds
unnecessary load times on the DB query and data processing when
constructing the instance list, especially given this is a query over a
time range of instances, so as you add more instances to scale it's
going to slow down.

This simply limits the expected_attrs in the get_active_by_window_joined
call to system_metadata so we don't load the other unnecessary
attributes.

Closes-Bug: #1383469

Change-Id: Ic77d51cd45730ca664e68cfe0cb1671b5442d4a9
Revert pause/unpause state when host restart

Give following scenario for instance task state
1) start to pause an active instance
2) the nova-compute died because of unexpected situation
3) when nova-compute start again , it's task state is pausing
it should be changed to None because host restarted

Also ,for the paused instance and its state is unpausing,
we need to unpase the instance.

Change-Id: If72a36dafd894ac47c0b255d701a3f9965ff2590
Closes-Bug: #1290639
libvirt: use qemu (qdisk) disk driver for Xen >= 4.2.0

According to the Xen ML [1] "qdisk is better supported and blktap is
unmaintained, we choose qdisk over blktap whenever possible" (change was
introduced in Xen's commit 0166217 [2]). The Xen documentation states
that the disk driver "should not be specified, in which case libxl will
automatically determine the most suitable backend" [3]. However, even if
libvirt does it OK [4] not setting the driver name so that libxl chooses
the best backend, it sets the file format to 'raw' so that a wrong
backend will be chosen by libxl.  Since 'qdisk' seems to be the
preferred option, it seems safe to set it as the backend.

[1] http://lists.xen.org/archives/html/xen-devel/2013-08/msg02633.html
[2] http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=0166217103e18368424fbd5ffff01c1ea50d0b17
[3] http://xenbits.xen.org/docs/unstable/misc/xl-disk-configuration.txt
[4] http://libvirt.org/git/?p=libvirt.git;a=blob;f=src/libxl/libxl_conf.c;h=1210500858b00b1d10b0bc921ab8b5efc87e630f;hb=HEAD#l847

Closes-Bug: #1326238
Change-Id: Ia52d52a01916fac92dc8e648c462e237c0bfc915
Get EC2 metadata localip return controller node ip

When Get EC2 metadata localip, it will return
<instanceIP, Controller_Local_IP>, but for ec2
metadata localip, it expects to return instanceIP.
This patch fixes the issue.

Change-Id: I6c42e713265ebb3138f7a94df5ee996e6f68d02f
Closes-Bug: #1334857
Type conflict in trusted_filter.py using attestation_port default value

When trusted filter (nova/nova/scheduler/filters/trusted_filter.py)
in nova scheduler is running with default value of attestation_port:

default='8443'

method _do_request() in AttestationService class has this line:

action_url = "https://%s:%d%s/%s" % (self.host,
self.port, self.api_url, action_url)

It is easy to see that default type of attestation_port is string.
But in action_url self.port is required as integer (%d).
It leads to conflict.

This change provides more tests than is required only to cover this bug
fix. This cases are testing AttestationService _do_request() method
using different status_codes and different texts returned by mocked
request method.

Change-Id: I1ad57b5bd1986360416948fd00dec22456dc29a7
Closes-Bug: #1381468