Don't default to master host-uuid if sr-create shared=false

sr-create had been defaulting the host-uuid to master if it wasn't
specified. This would cause problems when creating a local SR without
specifying the host-uuid. We now raise a CLI exception if we create
non-shared SR without specifying the host-uuid.

CA-98185

Signed-off-by: Mike McClurg <mike.mcclurg@citrix.com>
CA-129192: restore metadata for all SR

CA-129192/SCTX-1728: revise restore-sr-metadata.py so that it could restore
metadata for all SR with command line:
   ./restore-sr-metadata.py -f <input file> -u all
or just omit the sr-uuid:
   ./restore-sr-metadata.py -f <input file>

xe-restore-metadata is modified accordingly.

Signed-off-by: Kaifeng Zhu <kaifeng.zhu@citrix.com>
Add a configurable idle timeout to stunnel

Signed-off-by: Jon Ludlam <jonathan.ludlam@eu.citrix.com>

Signed-off-by: Akshay <akshay.ramani@citrix.com>
CA-135934: use the same estimate of dom0 'static_max' everywhere

We sample the amount of free pages on a host and add an estimate
of dom0 'static_max' to it to create the boot_free_mem.

Later we substract a different estimate of dom0 'static_max' and
erroneously conclude there is free memory when there isn't.

Note there is no good estimate of dom0 'static_max' because dom0
doesn't really know how much memory it has: the xen command-line
says one thing; the domain_getinfo hypercall says another thing;
the "free" command-line says a 3rd thing; and the balloon driver
says a 4th thing. However the main property we need is that we use
the same estimate everywhere.

Signed-off-by: David Scott <dave.scott@citrix.com>
Remove the hardcoded disabling of ballooning for dom0

To express "don't balloon me", set the dynamic-min and dynamic-max
to the same values.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Merge pull request #1768 from djs55/CA-135934

CA-135934: use the same estimate of dom0 'static_max' everywhere
Our cookie are even more broken than I thought.

Ours have spaces in the values. The newer Stringext cookie extraction
doesn't allow this, unlike the previous cohttp version. So rather than
use stringext, use a hand-rolled parser.

Signed-off-by: Jon Ludlam <jonathan.ludlam@citrix.com>
Merge pull request #1767 from jonludlam/fix_migration

Our cookie are even more broken than I thought.
Add 'xe vdi-export' for symmetry with 'xe vdi-import'

  xe vdi-export uuid=<VDI> filename=<filename>

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
xe vdi-{export,import}: display a progress bar if "--progress" is given

Note we now send 'PrintStderr' commands to the CLI while it is busy
downloading or uploading disk images. A multithreaded CLI is necessary
for this to work.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
{import,export}_raw_vdi use vhd-tool via vhd_tool_wrapper

This allows both functions to use O_DIRECT.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
{import,export}_raw_vdi: update Task.progress

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
VDI.export: allow the user to specify a format; currently either 'raw' or 'vhd'

The HTTP handler accepts an additional argument:

  GET /export_raw_vdi?vdi=<vdi>&format=<raw|vhd>...

The content-disposition HTTP header is used to suggest a useful filename, and
the content-type HTTP header is set to either application/octet-stream or
application/vhd.

The CLI accepts an optional argument:

  xe vdi-export uuid=<uuid> filename=<filename> [format=(raw|vhd)]

.vhd format export will require a newer vhd-tool (>= 0.6.5)

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
import_raw_vdi: normalise whitespace

$ camlp4o -printer o -no_comments ocaml/xapi/import_raw_vdi.ml | md5sum
e74faa7367f4c06d818a04fd87dfeeb5
$ git checkout HEAD^
$ camlp4o -printer o -no_comments ocaml/xapi/import_raw_vdi.ml | md5sum
e74faa7367f4c06d818a04fd87dfeeb5

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
import_raw_vdi: support importing vhd-format data

This requires vhd-tool >= 0.6.6

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
xe vdi-import: add support for [type=(raw|vhd)]

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
VDI.export: add /dev/mapper to the vhd parent search path

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
VDI.export: add a 'base' argument to export only VDI differences

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Revert "CA-118466: Prevent snapshot revert from deleting VM VDIs that were attached"
Fixes CA-121278

This reverts commit 6989a08e8244a8c10318940ea65109e2169ff243.

The logic to skip disks attached after the snapshot isn't quite right,
and is causing problems in our test infrastructure by filling the storage
with disks which really should have been removed.

Signed-off-by: Euan Harris <euan.harris@citrix.com>
Merge pull request #1775 from euanh/backout-ca118466

Revert "CA-118466: Prevent snapshot revert from deleting VM VDIs that were attached"