CA-113392: Fix PBD password-secret lifecycle

When creating an SR, if there are any secrets in the device-config then they
are duplicated and a unique one is used for each PBD creation.

However, XenCenter's "Repair SR" feature creates new PBDs directly, copying the
device-config field from existing ones. This bypasses the secret-duplication
which is in the SR-create code path.

This patch pushes the secret-duplication logic down to the PBD creation which
solves the XenCenter issue pending a revised design of the secret datamodel.
Also the master is no-longer special cased for the same reason. This has the
added benefit of making secret creation/destruction entirely symmetrical: the
secrets created with PBDs are destroyed when the PBD is destroyed and the
secrets created manually must be destroyed manually.

XenCenter should now be updated to tidy up after itself: if it chooses to
create secrets around passwords used in SR creation, then they should be
destroyed in a finally block after the SR creation is complete. This is safe
since all PBDs will have duplicated the secret.

Note this patch applies (as the previous semantics did) to any key in device
config ending in '_secret'. The ones used by the SM backends are
'password_secret' and 'chappassword_secret'.

Signed-off-by: Si Beaumont <simon.beaumont@citrix.com>
CA-111637: Change PBD.plug to be a localhost operation.

During the return in a pool of a slave host that rebooted,
the PBD.plug will fail if the slave is not set as alive.
It is anyway an operation that is performed on the master.

Signed-off-by: Jerome Maloberti <jerome.maloberti@citrix.com>
CA-111637: Remove set_live in pool.hello in HA.

In HA the host live value can be different from the liveset determined
by xha, allowing a user to start a VM on host that just came back in
a pool, but before its recovery process finished.
This commit fix this problem by forbidding the live value to be changed
outside of HA recovery process, the HA liveset becomes the only liveness
state of a host in HA.

Signed-off-by: Jerome Maloberti <jerome.maloberti@citrix.com>
CA-111637: Use the HA live_set when computing a restart plan for VMs.

Previously, after a host failure in HA, the function
Xapi_ha_vm_failover.compute_restart_plan which choose which hosts
should restart the failed VMs would pickup host that are live and
enabled. In some cases, a host may be live but not in the HA
live_set, for example if it returned in the pool before the HA
recovery process finished.
This situation is bad since the live host would have new VMs
started on it, while later it may be marked as dead, once the
HA recovery process finished.
This commit add the live_set parameter to compute_restart_plan
and all functions that need it by transitivity.
Some functions are called during the HA recovery, where the
live_set is available, or at startup, in this case the live_set
is created from all live and enabled hosts.

Signed-off-by: Jerome Maloberti <jerome.maloberti@citrix.com>
Dodger/HFX-955:CA-111637: Reorder Hosts and VMs recovery after a host crash in HA.

When some hosts are considered dead in HA, restart_auto_run_vms were
processing them in this way:
 - for each dead host
   - List all resident VMs
   - Host.set_live=false
   - call Xapi_hooks.host_post_declare_dead which can be very long
   - set all resident VM to `Halted (including Control Domain)
This process was conflicting with db_sync if a host had the bad taste
of coming back to life while restart_auto_run_vms was stuck in
host_post_declare_dead.
This commit reorder the actions to put the shortest first:
 - for each dead host
   - set all resident VMs excluding Control Domain to `Halted
   - Host.set_live=false
 - for each dead host
   - call Xapi_hooks.host_post_declare_dead

Signed-off-by: Jerome Maloberti <jerome.maloberti@citrix.com>
CA-112880: Fix race condition writing memory-actual

There was a race condition between the sender and receiver during migration.
This is due to the interplay with the monitor threads. Monitor_rrds.update_rrds
is run every 5 seconds and checks for changes. This module also has a cache of
values and only writes to the database if the cache is dirtied to avoid
excessive database writes. There is such a monitor thread running on each host.

The monitor thread on the sender may wake and write a value to the database
after the domain has been resumed on the receiver but before it is destroyed on
the sender. This could be a spuriously low value in the case where Xen is
reclaiming the memory on the sender side during the domain destroy.

The receiver however may have already updated its cache and written its value
to the database. It never writes on subsequent runs of the monitor thread since
it thinks the value that is in its cache (the correct value) is already in the
database (which it is not).

To remedy this we need to ensure that the monitor for a given host only updates
the VM_metrics in the database for VMs that are resident on that host. We also
provide a means to mark the cache as dirty when the migration has finished
since the receiver may have its cache updated before the resident_on field is
set.

Signed-off-by: Si Beaumont <simon.beaumont@citrix.com>
Handle invalid edition on xapi startup

We changed the editions in a later version of XenServer. If an old xapi
restarts during an upgrade, we may have an invalid edition in the database. If
the edition found in the database isn't in the list of allowed editions, we
instead use the "default edition" (the "lowest" edition returned by
V6client.get_editions).

CA-121350

Signed-off-by: Mike McClurg <mike.mcclurg@citrix.com>

fix
Dodger/HFX-999:HFX-990: Don't package sparse_dd in xapi-core RPM

Signed-off-by: Si Beaumont <simon.beaumont@citrix.com>
Adding Fun to resolve (|>)
Signed-off-by: Ravi Pandey <ravi.pandey@citrix.com>
import_raw_vdi: use vhd-tool helper program

The vhd-tool helper is capable of O_DIRECT I/O and minimises
data copies.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
import_raw_vdi: log output from vhd-tool

This can be useful in diagnosing errors.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
HFX-990: Revert to using Uuid instead of Uuidm

Signed-off-by: Si Beaumont <simon.beaumont@citrix.com>
HFX 990: Correct vhd-tool path for 6.0.2 backport

Signed-off-by: Si Beaumont <simon.beaumont@citrix.com>
Merge pull request #1611 from ravippandey/Dodger/HFX-1059

[HFX-1059] CA-121350: Handle invalid edition on xapi startup
Don't package sparse_dd.conf in xapi-core RPM
Merge pull request #1614 from ravippandey/Dodger/HFX-999

Dodger/HFX-999: Multiple commits
Merge pull request #1608 from ravippandey/Dodger/HFX-955

Dodger/HFX-955 : Multiple commits
Merge pull request #1605 from ravippandey/Dodger/HFX-1037

[HFX-1037] CA-113392: Fix PBD password-secret lifecycle
Merge pull request #1606 from ravippandey/Dodger/HFX-1031

[HFX-1031] CA-112880: Fix race condition writing memory-actual