CP-8888: Add diagnostic stats reporting

Signed-off-by: John Else <john.else@citrix.com>
CP-8888: Stop stats reporter before exiting

Signed-off-by: John Else <john.else@citrix.com>
CA-108189: Prevent timeouts when collecting bugtools

Prevent timeouts when collecting bugtools by increasing the timeout for
the xenopsd bugtool plugin from 60 seconds to 600 seconds.  Since it
took about 200 seconds for 460 VMs, 600 seconds should be enough for
very large hosts.

Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
CA-143838: let network configure all physical and bond interfaces on startup

If a VLAN is used for storage and is configured to use jumbo frames (MTU =
9000), then XS does not immediately enable jumbo frames on the interface at
boot time. Specifically, if HA is enabled, the HA daemon is unable to access
the storage device, and blocks the host's startup process for a long time.

Networkd can be told to bring up interfaces and bridges at boot time, which is
before xapi is started, and before anything else may need the network. This is
done by marking the interface/bridge as "persistent". Currently, all
interfaces/bridges with an IP address in dom0 are made persistent. This
includes VLAN bridges, but not the interface that the VLAN is based on, which
is what caused the bug.

To simplify, this patch makes all physical and bond interfaces/bridge
persistent, besides all other interface/bridges that have an IP address.

Signed-off-by: Rob Hoes <rob.hoes@citrix.com>
CP-9366: Use HVM feature-flags instead of checking PV drivers.

Changes:
* xapi_vm_lifecycle check_operation_error uses the flags to check
  whether we can perform a given operation.
* In xapi_host.ml the pre-evacuation check no longer checks for PV
  drivers to learn whether a VM can be migrated.  It was already
  calling Xapi_vm_lifecycle.check_operation_error.
* Appropriate errors are defined: one for each of three features.
* Datamodel refers to a new error instead of vm_missing_pv_drivers.

Related work not done:
* Datamodel has some operations that should mention the new errors.
  They should have mentioned vm_missing_pv_drivers in the past, so
  even now we are no worse off than before.

Signed-off-by: Thomas Sanders <thomas.sanders@citrix.com>

Conflicts:
	ocaml/xapi/xapi_vm_lifecycle.ml
CP-9366: Update VM-metrics "other"

Now when qemu writes a feature-flag into the control node, xapi will
notice and so will trigger the existing code that can copy those entries
into the "other" vm guest metrics parameter.

To make that work, when checking whether the guest metrics should really
be updated, now we go ahead even if only the "other" map has changed.

Signed-off-by: Thomas Sanders <thomas.sanders@citrix.com>
Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

Conflicts:
	ocaml/xenops/xenops_server_xen.ml
CP-9366: Allow vCPU hotplug for HVM (if feature-vcpu-hotplug is set).

Removed code that raises an Unimplemented exception unconditionally
for hvm guests when attempting vCPU hotplug.
This allows the existing code that decides based on the feature-flag
to work properly.

Signed-off-by: Thomas Sanders <thomas.sanders@citrix.com>
Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

Conflicts:
	ocaml/xenops/xenops_server_xen.ml
Merge pull request #1894 from robhoes/hvmlinux

CP-9366: Use HVM feature-flags instead of checking PV drivers
CA-143944: Allow suspend when PV drivers are present (as before)

A few commits ago, we made a change to allow suspending based on the presence of
the "feature-suspend" key. Unfortunately, the Windows guest agent only writes
this key on startup and not on resume, so that xapi refuses any suspend request
after a resume (and probably after migration). As a workaround, we re-add the
old-style check, where the presence of the PV-drivers implies that all features
are enabled.

Signed-off-by: Rob Hoes <rob.hoes@citrix.com>
Merge pull request #1896 from robhoes/pvdrivers-master

CA-143944: Allow suspend when PV drivers are present (as before)
Merge pull request #1875 from rosslagerwall/bugtool-timeout

CA-108189: Prevent timeouts when collecting bugtools
CA-141502: Missing 'type' argument in uuid_invalid handled
Signed-off-by: Ravi Pandey <ravi.pandey@citrix.com>
CA-141502: Log warning when messages get GC'ed
Signed-off-by: Ravi Pandey <ravi.pandey@citrix.com>
CA-141502: Making message_destroy more robust

While destroying a message with a given ref, if
the ref does not exit under refs dir, a handle_invalid
error is thrown and the message is left undestroyed.

This patch reads the refs from all the messages under
/var/xapi/blobs/messages and matches it with the given
ref. Once the match is found, it retrives that message
file and is then destroyed.
Signed-off-by: Ravi Pandey <ravi.pandey@citrix.com>
CA-140684: Handle cancellation of VM migration properly

Suggested-by: Yunlei Ding <yunlei.ding@citrix.com>
Signed-off-by: Euan Harris <euan.harris@citrix.com>
Signed-off-by: Rob Hoes <rob.hoes@citrix.com>

Conflicts:
	ocaml/xapi/xapi_vm_migrate.ml
Merge pull request #1897 from ravippandey/CA-141502_tr3

CA-141502: Message_destroy fixes
Merge pull request #1898 from robhoes/cancel

CA-140684: Handle cancellation of VM migration properly
Add clearwater-whetstone release to datamodel

Signed-off-by: John Else <john.else@citrix.com>
Update published version of PCI subsystem fields

These fields have been backported to Clearwater for hotfix Whetstone.

Signed-off-by: John Else <john.else@citrix.com>
Merge pull request #1890 from robhoes/persistent-trunk

CA-143838: let network configure all physical and bond interfaces on sta...