sp: changing un-shared treatment in the createVolume function

Up until now, the only way to create a 'template' disk was by using the
copyImage command.  This has 2 limitations:
1. It forces the user to copy all the data instead of doing an in-place
convert of an image to a template image
2. Resulting image cannot be qcow which is wasteful in some cases

This patch addresses that, and allows using an existing disk as a
template disk, by allowing to "share" a disk that isn't marked as
"shared", but can be shared as it doesn't have snapshots. This disk will
be marked as "shared" upon first use.

Change-Id: Id9a4b84795b176b9696684045e9774d324a42eca
Signed-off-by: Oved Ourfali <oourfali@redhat.com>
Reviewed-on: http://gerrit.ovirt.org/23260
Reviewed-by: Ayal Baron <abaron@redhat.com>
securable: refactor the securable implementation

In order to remove the storage pool metadata we need more flexibility
in defining the isSafe method.
In fact the different backends (with/without metadata) will rely on the
StoragePool SPMness to protect their methods.

In this patch:
- transform the Securable base class and MetaSecurable in a decorator

Change-Id: Id5a8be7536748481746795e27701fbecc7c3318c
Signed-off-by: Federico Simoncelli <fsimonce@redhat.com>
Reviewed-on: http://gerrit.ovirt.org/22115
Reviewed-by: Nir Soffer <nsoffer@redhat.com>
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
Reviewed-by: Ayal Baron <abaron@redhat.com>
storageTests: Configure via environment variables

These are used as permutations for the test. We want to automate the
test and call it with whatever storage type(s) and metadata
version(s).

Change-Id: I84f1ee57e43d921ae38afaabddecca9f38dd2687
Signed-off-by: Vered Volansky <vvolansk@redhat.com>
Reviewed-on: http://gerrit.ovirt.org/23259
Reviewed-by: Allon Mureinik <amureini@redhat.com>
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
Reviewed-by: Nir Soffer <nsoffer@redhat.com>
vdsm.spec: specify sysV commands for EL6 only

To provide EL7 builds we must update the spec to not
include sysV commands since it uses systemd.

Change-Id: Ie52ae97d48d795d53529152b68380fbd13d7927d
Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1051860
Signed-off-by: Douglas Schilling Landgraf <dougsland@redhat.com>
Reviewed-on: http://gerrit.ovirt.org/23235
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
netinfo: ifcfg: ignore 0 suffix

'IPADDR0', 'GATEWAY0', 'PREFIX0' and 'NETMASK0' means to initscripts
exactly as their unsuffixed siblings. With this patch, Vdsm reports them
as 'IPADDR', so the do not confuse Engine.

A proper change would be to make Engine/Vdsm API independent of
ifcfg-specific details. This patch is required anway, for the sake of
older Engines.

Change-Id: I5c9a206a5b96d0b38aab9c3d62d6074424b419be
Bug-Url: https://bugzilla.redhat.com/987832
Signed-off-by: Dan Kenigsberg <danken@redhat.com>
Reviewed-on: http://gerrit.ovirt.org/23329
Reviewed-by: Assaf Muller <amuller@redhat.com>
Reviewed-by: Antoni Segura Puimedon <asegurap@redhat.com>
Reviewed-by: Sandro Bonazzola <sbonazzo@redhat.com>
remove redundant libvirt call

Checking UUIDString() adresses the bug #603494, which
is closed ages ago. While it is nice to support really
ancient libvirt versions, it costs us one extra
libvirt call. Given global libvirt driver lock, it
does matter in mass VM creation.

Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=603494
Signed-off-by: Peter V. Saveliev <peet@redhat.com>
Change-Id: I06a71c2e44332f50b3203a0a8f1bfa799914b684
Reviewed-on: http://gerrit.ovirt.org/11054
Reviewed-by: Francesco Romani <fromani@redhat.com>
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
Tested-by: Adam Litke <alitke@redhat.com>
caps: check availability abort on EIO in libvirt

commit bbeb165e42673cddc87495c3d12c4a7f7572013c
added support for VIR_MIGRATE_ABORT_ON_ERROR,
but libvirt 1.0.5.8 shipped in Fedora 19
do not support that, even if it should be supported
starting from upstream version 1.0.1.

This patch provides a workaround for this; if libvirt
misses this flag, it forces the the cluster level at
no more than version 3.3.

Change-Id: I7dd857ac3c00c48d4de3a50931cb259f45ff4f5d
Signed-off-by: Francesco Romani <fromani@redhat.com>
Reviewed-on: http://gerrit.ovirt.org/23273
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
Change MANAGEMENT_NETWORK to MANAGEMENT_NETWORK*S*

Before: MANAGEMENT_NETWORK was either 'ovirtmgmt' or 'rhevm',
        and the value was gotten from automake
After: MANAGEMENT_NETWORKS = ('ovirtmgmt', 'rhevm')

This change allows the following patch to solve the bug.

Bug-Url: https://bugzilla.redhat.com/1015009
Change-Id: I69a39b7887267d54d4dce27d4818d32a7cc0cd68
Signed-off-by: Assaf Muller <amuller@redhat.com>
Reviewed-on: http://gerrit.ovirt.org/23182
Reviewed-by: Antoni Segura Puimedon <asegurap@redhat.com>
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
Add netinfo.getDefaultGateway()

Used in patch:
https://bugzilla.redhat.com/1015009

On a new VDSM installation, or an old one upgrading to
3.4, we detect the current networking configuration and
persist it to files of our own format. More on this at:
http://www.ovirt.org/Feature/NetworkReloaded#Unified_persistence

It is necessary to detect the host default gateway so
the information could be persisted to files.

Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1015009
Change-Id: I15be5d69b5ef8f9e4cb9a30a3739a0b060225e0e
Signed-off-by: Assaf Muller <amuller@redhat.com>
Reviewed-on: http://gerrit.ovirt.org/23346
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
Reviewed-by: Antoni Segura Puimedon <asegurap@redhat.com>
vdsm: allow hooks to pass down dictionaries in json format

This will allow to pass, and recover network settings in the
*_network_setup hook as json dictionaries using the same
method that is used for XML. In this case the environment
variable _hook_json is used.

before_network_setup hook exposes the requested network
configuration that can be patched by the hook.
after_network_setup exposes the final system network
configuration.

Change-Id: Ie07c511e9740fd19a1c27baf87e91f9a427d0dcd
Signed-off-by: Miguel Angel Ajo <miguelangel@ajo.es>
Reviewed-on: http://gerrit.ovirt.org/20330
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
Tested-by: Dan Kenigsberg <danken@redhat.com>
Tested-by: Miguel Angel Ajo Pelayo <majopela@redhat.com>
Extend setupNetworks API to accept defaultRoute

Previously defaultRoute was True for the management network,
now it is received from the client. For backwards compatability,
if defaultRoute is not received from the client, it is set
if the network being configured is the management network.

Change-Id: I9b2ff711155eb0bd79ab84e979eb82c846362374
Bug-Url: https://bugzilla.redhat.com/1015009
Signed-off-by: Assaf Muller <amuller@redhat.com>
Reviewed-on: http://gerrit.ovirt.org/22720
Reviewed-by: Antoni Segura Puimedon <asegurap@redhat.com>
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
Tested-by: Dan Kenigsberg <danken@redhat.com>
mkimage: do not create world-readable image

mkimage generates world-readable images by default.
mkimage spawns a genisoimage process through the cpopen package
to do the actual work.
It is of course possible to fix the permissions once the ISO image
is created with a trivial os.chmod() call, but this will leave
open a window opportunity to still exploit the bug.
A more correct and secure way to fix the permissions of the newly
generated image is to setup the umask just before to exec.

The current python-cpopen package lacks a way to set the umask
of the child before to run it, so this patch depends on an enhanced
python-cpopen, temporarily located here:

https://github.com/mojaves/python-cpopen

With this enhanced cpopen, fixing the permissions is trivially made
by passing the correct umask at the ISO image creation.

PatchSet v2:
the cpopen patch has been merged upstream:

https://github.com/ficoos/cpopen/commits/master

PatchSet v3:
* updated vdsm.spec.in and debian.control to require an updated cpopen.

PatchSet v4:
* the minimum required cpopen version is 1.2.3-5

PatchSet v5:
* rebased against master
* make iso image not group-writable
* added explicit permission tests (umask tests alread committed
  in the cpopen package)

Change-Id: I893a1310d9988c52cec9f48dfd17dfa1647da4dc
Bug-Url: https://bugzilla.redhat.com/1034172
Signed-off-by: Francesco Romani <fromani@redhat.com>
Reviewed-on: http://gerrit.ovirt.org/21946
Reviewed-by: Yaniv Bronhaim <ybronhei@redhat.com>
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
sd: add inquireClusterLock method to StorageDomain

The leader version and the spm id may be maintained by the cluster
lock in some implementations (e.g. SANLock).
This patch introduces the capability of inquiring the cluster lock
for these values.

Change-Id: I10e64d74319ea6591a7edf8e17809d367a758386
Signed-off-by: Federico Simoncelli <fsimonce@redhat.com>
Reviewed-on: http://gerrit.ovirt.org/21426
Reviewed-by: Nir Soffer <nsoffer@redhat.com>
Reviewed-by: Ayal Baron <abaron@redhat.com>
sp: use setDomainsMap in reconstructMaster

The metadata access executed in reconstructMaster should be replaced
by the existing setDomainsMap. In order to do so we also need to set
the master domain (used by setDomainsMap) and temporarily set the
storage pool as secure (in the same way we do in create).

Change-Id: I0ce8ad258be09b40ee9ad06399cc841f374bf816
Signed-off-by: Federico Simoncelli <fsimonce@redhat.com>
Reviewed-on: http://gerrit.ovirt.org/23385
Reviewed-by: Ayal Baron <abaron@redhat.com>
sp: refactor out the metadata access from StoragePool

All the storage pool metadata operations have been relocated to a
specific backend: StoragePoolDiskBackend. It is now possible,
implementing the required API, to use different backends for the
pool metadata.

A backend may require other attributes and methods in addition to
the regular API used by StoragePool, these specific features should
be handled outside of the StoragePool implementation, typically in
the HSM class where the backward compatibility layer is maintained.

Change-Id: I75493d1db60e51cccd5231b516f963c970d24c99
Signed-off-by: Federico Simoncelli <fsimonce@redhat.com>
Reviewed-on: http://gerrit.ovirt.org/22132
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
sp: receive domains map in connectStoragePool

This patch adds the support for the StoragePoolMemoryBackend and
the required support in the HSM class to instantiate it and to
live upgrade from older backends.

The connectStroagePool API has been updated to receive the extra
domainsMap dictionary that triggers the use of the new backend.

Change-Id: I393677f643a62e3711af2a3cfb8b4b9a5ce11c2d
Signed-off-by: Federico Simoncelli <fsimonce@redhat.com>
Reviewed-on: http://gerrit.ovirt.org/21427
Reviewed-by: Ayal Baron <abaron@redhat.com>
Changing vdsm-tool and vdsmd pre-tasks outputs to be more meaningful to user

Previously to this change the output was misleading. Users
could not understand how to handle configuration issues properly.

Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1029812
Change-Id: If29f41081a70c0734cd78dab36f86c794a44b2d3
Signed-off-by: Yaniv Bronhaim <ybronhei@redhat.com>
Reviewed-on: http://gerrit.ovirt.org/23253
Reviewed-by: Vered Volansky <vvolansk@redhat.com>
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
gluster: additional parsing of host UUID in verb glusterVolumesList

This patch adds parsing of host UUID for all the bricks in the
verb glusterVolumesList.
This information is stored in bricksInfo which is list of dictionary
where each dictionary has name and hostUUID.
sample:
---------------------------------------------------------------
     'bricksInfo': [{
                     'name': '192.168.122.2:/tmp/t_b1',
                     'hostUuid': '04eb591b-2fd3-489e-a22c-5d342a3c713d'
                     },
                    {
                     'name': '192.168.122.2:/tmp/t_b2',
                     'hostUuid': '04eb591b-2fd3-489e-a22c-5d342a3c713d'
                     }]
---------------------------------------------------------------

Change-Id: I9057f3aea0c0ed8fb4d4ec26eaf66fe80ec581e2
Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1038988
Signed-off-by: ndarshan <dnarayan@redhat.com>
Reviewed-on: http://gerrit.ovirt.org/22693
Reviewed-by: Aravinda VK <avishwan@redhat.com>
Reviewed-by: Sahina Bose <sabose@redhat.com>
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
To allow downgrade of vdsm we need to support vdsm-tool old API

During upgrading or downgrading vdsm version we perform reconfigure
process. This process has been changed between 3.3 to 3.4. This patch
tries the current version implementation of reconfigure, and if it fails
it will fallback to vdsmd reconfigure verb, which exists in all
previous versions of vdsm.

This workaround need to be removed in vdsm 3.5.

Change-Id: I8bd7f9048acc4f53f3da66e21d5ad2c5caf2a5ad
Signed-off-by: Yaniv Bronhaim <ybronhei@redhat.com>
Reviewed-on: http://gerrit.ovirt.org/23333
Reviewed-by: Alon Bar-Lev <alonbl@redhat.com>
Reviewed-by: Dan Kenigsberg <danken@redhat.com>
Reviewed-by: Douglas Schilling Landgraf <dougsland@redhat.com>
sp: fix pool membership check in attachSD

A race between attachStorageDomain and connectStoragePool using
StoragePoolMemoryBackend could lead to the new domain not being
attached (on storage).

The race happens in StoragePool.attachSD:

        domains = self.getDomains()
        if sdUUID in domains:
            return True

if the domain map has been already updated by connectStoragePool
the domain is deemed attached and the remaining operations are
skipped.

In this patch:
- use only validateAttachedDomain in detachSD (validatePoolSD is
  redundant)
- make attachSD symmetric to detachSD using validateAttachedDomain
  to check if the domain is already attached

Change-Id: I4169ddcb4da29c8b806b5788080c16349b642197
Signed-off-by: Federico Simoncelli <fsimonce@redhat.com>
Reviewed-on: http://gerrit.ovirt.org/23446
Reviewed-by: Dan Kenigsberg <danken@redhat.com>