CA-83837: lookup a domid from a VM uuid via xenstore rather than the domain info list

Unfortunately some libraries (such as libxenguest) will destroy a domain
beneath us, causing us to lose track of the domid corresponding to a uuid.
We are in control of xenstore, so we use the /vm/uuid/domains map to find the
lookup the domid. Note this also means that 'add' has to add such a key for
already-running VMs (such as dom 0)

Without this change a VM.suspend will leave VDIs attached.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
CA-83837: if a cancellable xenstore watch simultaneously succeeds/fails/is cancelled, apply a prioritisation policy.

We consider that an operation which has succeeds (but simultaneously/historically has failed or been cancelled) has actually succeed.
We consider that an operation which has not succeeded but failed (but simultaneously/historically been cancelled) has actually failed.

So the event priority is:
  success > failed > cancelled

Without this, a cancel that arrives late may be said to have actually cancelled an operation which has already terminated with success/faulure (because cancel is always asynchronous).
Without this, an operation which failed in the past but then later succeeds (eg VBD.unplug) might be said to be still a failure even though it has now completed successfully.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
CA-83837: watch the "hotplug-status" node as well as the "shutdown-done" node

The old XenServer/XCP-only protocol used the "shutdown-done" node, while
upstream in other kernels (such as PVops) they use the "hotplug-status"
key instead.

We need to watch the "hotplug-status" key and if the device is removed,
generate the correct event and clean up any resources allocated.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
CA-83837: VBD_unplug is now much more robust

VBD_unplug may be called when the domain has been destroyed (in suspend
by libxenguest) and when the xenstore device has already been shutdown.

All the cleanup steps are now Opt.iters over option types, to do as
much cleaning as possible. Note this depends on the new 'domid_of_uuid'
function which uses xenstore rather than the domain info list.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
[xapi] avoid fetching unnecessary events from xenopsd when monitoring a task

If we call UPDATES.get None then we get an event on every object in the system.
Instead we only need to:

  last_id <- UPDATES.last_id
  TASK.stat
  UPDATES.from last_id

which will fetch only new events, and only in the case where the task hasn't
already finished.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
[xapi] Only update the storage.db if something has actually changed

We implement DP.destroy by iterating over every active VDI. If there
are 1000 active VDIs then we were pointlessly writing the database
1000 times. This saves 15s per DP.detach.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
CA-84673: Metadata import should create a network if it can't find one

This already happens for "normal" VM imports. There is no good reason
why a metadata import should fail if it cannot find a network, rather than
creating an internal network.

This will prevent issues during, e.g., cross pool migration, where metadata
export+import is used to migrate VM metadata to the remote pool.

Signed-off-by: Rob Hoes <rob.hoes@citrix.com>
CA-83711: Stop generating spurious backslashes
CA-83076: avoid xapi and xenopsd both "owning" the Vbd.backend xenopsd field

The xapi/xenopsd separatation needs to be strictly as follows:

1. all the static VM metadata (Vm.t, Vbd.t, Vif.t, Pci.t) is owned and
   updated by xapi, whenever the user requests it via the XenAPI

2. all the dynamic VM state (Vm.state, Vbd.state, Vif.state, Pci.state)
   is owned and updated by xenopsd. Xapi will receive events on this
   and update some data in the pool database.

In particular a xenopsd API call (Vbd.insert, Vbd.eject) must not write
any static metadata directly, since this will conflict with parallel
updates from xapi.

Furthermore all dynamic VM state updates from xenopsd need to be "complete"
and not require correlating with static VM state. For example the disk
"backend" which is currently associated with a CDROM must be completely
described in the Vbd.state.

The XenAPI VBD.insert/VBD.eject control flow is now:

1. xapi calls xenopsd with VBD.insert/eject
2. xenopsd performs the xenstore-level operation (including stashing away
   the VDI in xenstore)
3. xenopsd generates an event on the VBD
4. xapi receives the VBD event and calls VBD.stat
5. xapi sets VBD.{empty, VDI} in the pool database based on the VBD.stat
   results
6. xapi inserts a barrier event to ensure all events are processed

After step (5), a XenAPI event will be generated which will be received
by the event.from in xapi_xenopsd, which will cause xapi to push the
new Vbd.t configuration to xenopsd, ensuring that the insert/eject will
take effect on next reboot.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Merge commit 'f8135d9d148b4b36e8b621904c0e2d1ea5988182'
Merge commit 'ca911fd9de6b9f69aeeee6a4dbe66524ac2240d2'
Merge commit '99f82b7cecb961f9bf0da966a38e67bae618653b'
CA-67536: Remove unplug_force from vif_operations

The unplug_force operation was recently added to the vif_operations enum.
However, this causes problems during rolling pool upgrade, while the
master has been upgraded, but some of the slaves have not. Operations
such as Db.VIF.get_record on running VMs were be broken by this, which broke
live migrations between un-upgraded hosts.

After this patch, the "unplug" operation is used for both "unplug" as well
as "unplug_force" in VIF.allowed_operations, which is fine, since they are
always allowed at the same time.

Signed-off-by: Rob Hoes <rob.hoes@citrix.com>
CA-83040: choose_suspend_sr ignores SRs not shared with host

When a VM was migrated within the pool from local storage to local storage, the
suspend SR was not reset, which caused suspend to fail because the VM's
suspend_SR was not visible to the host on which it was being suspended. We fix
the problem by modifying choose_suspend_sr to ignore a VM's suspend_SR field if
that SR has no PBD linking it with the VM's resident host.

Another commit will follow which blanks out the suspend_SR field on a VM when
it's suspend SR will no longer be visible after a migrate.

Signed-off-by: Mike McClurg <mike.mcclurg@citrix.com>
whitespace: delete trailing space

Signed-off-by: Mike McClurg <mike.mcclurg@citrix.com>
CA-83040: fix suspend_SR on intra-pool storage migrate

Signed-off-by: Mike McClurg <mike.mcclurg@citrix.com>
CA-83040: fix suspend_SR on VM metadata import

Signed-off-by: Mike McClurg <mike.mcclurg@citrix.com>
CA-78558: Be robust against absent PIF_metrics

Signed-off-by: Rob Hoes <rob.hoes@citrix.com>
CA-83076: quicktest now checks that VBDs have currently_attached = true after VM.start

It's useful to explicitly check this -- it's surprising how far you can get with a broken VBD.stat

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
CA-83076: add the same xenstore key in VBD.plug as we do in VBD.insert

Without this VBD.stat is unable to determine the VDI from the VBD, breaking
VBD hotplug.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>