improve efficiency of image transfer during migration

This reduces time to transfer a qcow2 image
with a virtual size of 10G, over GigE,
from about 7 minutes to about 30 seconds.

There are multiple inefficiencies in the existing process.
Taking an example of a qcow2 image with 10G virtual size,
the process was:

  qcow2 -> raw -> read -> send -> write -> qcow2

qcow2 to raw takes 20s,
transfer of the resultant 10G is another 4m9s, and
conversion back to qcow takes 2m33s.
I.E. a total of about 7 minutes.

So instead we try to avoid the initial qcow2 to raw
conversion completely, which results in the whole
process completing in about 30s, in the common
case where no conversion to raw is done on the destination.

We also optimize the case where the source qcow2
image doesn't have a backing file, and then directly
copy the source image without merging a backing file.

Note this will also improve the situation when
resizing/migrating within the same host as
needles conversions are avoided in that case too.

We also optimize the case where raw images are being used
by trying to use `rsync -Sz` rather than `scp`.
That compresses runs of zeros and create sparse destination files.
Testing a 10G raw image showed a saving of 30s in transfer time.
Also the network was greatly reduced (corresponding to holes
in the source), as was space usage at the destination.
This gain is limited though by rsync inefficiently reading
all the holes at the source:
https://bugzilla.samba.org/show_bug.cgi?id=8918

Thanks to David Naori <dnaori@redhat.com> for testing and ideas.

Change-Id: I9e87f912ef2717221c244241cda2f1027a4ca66a
Get rid of pointless 'suffix' parameter in libvirt imagebackend

The libvirt 'imagebackend' module has various methods which take
a 'name' and 'suffix' argument. They are never used in isolation,
however, so passing them separately has no benefit. Indeed all
but one caller passes a name which already has a suffix appended
The suffix argument can thus trivially be removed completely.

Change-Id: I2ee0f3d104a2eb5139fc85c2db68e36255b82dad
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Move setup of libvirt disk cachemode into imagebackend module

There is alot of repeated code setting the disk cache mode.
Move this into the imagebackend module so that it can be
done in one place

Change-Id: I7bff6b0b2e7e40a5aef3126634ca814082f8cf4f
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Don't set device='cdrom' for all disks in libvirt driver

When the root disk image was tagged as being an ISO image,
the libvirt driver would mistakenly set device='cdrom' for
all disks, instead of just the first image

Change-Id: Idf29df5c460f8ebefd2e462a53d72280d5af4c5e
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Ensure to use 'hdN' for IDE disk device in libvirt driver

When configuring the root disk for an image which is tagged as
an ISO, the libvirt driver sets the bus=ide, but still uses
the normal root device name, eg dev=vda. It must instead
use dev=hda for the IDE bus

Change-Id: If0a4c50043d6e4d9e54a5dcbdc05effbb72eced6
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Don't hardcode use of 'virtio' for root disk in libvirt driver

The libvirt driver is mistakenly using 'virtio' for the root
disk, instead of the hypervisor specific disk bus.

Change-Id: I742184b08a3ecd428af21ae5beb059bdbf38a9cd
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Move more libvirt disk setup into the imagebackend module

Move the setup of target bus and target dev out of the libvirt
driver and into the imagebackend module to reduce code duplication

Change-Id: I6796c125af769236f93341361ee07c8319817f7f
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Refactor libvirt imagebackend module to reduce code duplication

Currently each subclass of the Image class in imagebackend,
repeats the impl of the 'libvirt_info' method. These methods
are almost identical, only varying on two per-class settings.
By providing the driver_format and source_type to the Image
class constructor, it is possible to get to a single impl
of the 'libvirt_info' method.

Change-Id: I0807eed7f03aedf4980eedeb70195076ed5f4710
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Move libvirt disk config setup out of main get_guest_config method

The get_guest_config method in the libvirt driver is getting
quite large and complicated, particularly wrt disk config.
Moving the disk config setup out into a separate method,
get_guest_storage_method, eases understanding of the code

Change-Id: Ibbfbb88cafa45c98792e6732da726ab4afe4ca00
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Fixes nova-manage fixed list with deleted networks

The fix addresses the bug reported in bug 1025827
currently command 'nova-manage fixed list' will return
'Command failed.' message when there is no network defined or
all networks have been deleted or even when a network gets
deleted, then a new network gets created. In all these cases,
the command produces the 'Command failed.' error message which
does not really tell what went wrong. This fix will produce
'No fixed IP found' in these conditions.
changes made for globalization.
changes made for based on the comments for patchset 7.

Change-Id: I3c2e9bf3fa3c748e680c1df27d243a648ed47cf5
Merge "improve efficiency of image transfer during migration"
Merge changes Ibbfbb88c,I0807eed7,I6796c125,I742184b0,If0a4c500,Idf29df5c,I7bff6b0b,I2ee0f3d1

* changes:
  Move libvirt disk config setup out of main get_guest_config method
  Refactor libvirt imagebackend module to reduce code duplication
  Move more libvirt disk setup into the imagebackend module
  Don't hardcode use of 'virtio' for root disk in libvirt driver
  Ensure to use 'hdN' for IDE disk device in libvirt driver
  Don't set device='cdrom' for all disks in libvirt driver
  Move setup of libvirt disk cachemode into imagebackend module
  Get rid of pointless 'suffix' parameter in libvirt imagebackend
Merge "Fixes nova-manage fixed list with deleted networks"