CA-129192: restore metadata for all SR

CA-129192/SCTX-1728: revise restore-sr-metadata.py so that it could restore
metadata for all SR with command line:
   ./restore-sr-metadata.py -f <input file> -u all
or just omit the sr-uuid:
   ./restore-sr-metadata.py -f <input file>

xe-restore-metadata is modified accordingly.

Signed-off-by: Kaifeng Zhu <kaifeng.zhu@citrix.com>
Start SMAPIv1 proxies listening for requests via the message switch
Fix SMAPIv1 backwards compatibility on pool slaves

The two facts:
  Only the master should list the SM plugins directly.
  When the proxies are being started up on the master, the SM database
    records haven't been created.

Imply that we should query Sm.* when running on the master, but use
the SM database records on the slave.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
All SMAPI access is now routed through the xcp-idl client code

If we have message switch enabled then it will be used, otherwise
we still use unix domain sockets.

Note the special-case handling of SR.probe.

Signed-off-by: David Scott <dave.scott@citrix.com>
storage: create per-SR unix domain sockets in storage.d/<name>

This avoids clashing with the global unix domain socket 'storage'

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
message-switch: explicitly specify which xenopsd instance we want

Normally we use the 'default' one but this policy could be refined later.

Signed-off-by: David Scott <dave.scott@citrix.com>
lspci: look in both /sbin/lspci and /usr/bin/lspci

Ideally we should use $PATH-- forkexecd should do this if asked
to execute a relative filename. In the short-term, cope with both
CentOS and Ubuntu paths.

Signed-off-by: David Scott <dave.scott@citrix.com>
arm: robustify the parsing of /proc/cpuinfo

We should not expect there to be x86-only keys. We should ignore any lines
which don't have the 'key: value' format.

Signed-off-by: David Scott <dave.scott@citrix.com>
arm: don't fail (float_of_string) if some /proc/cpuinfo keys are missing

Signed-off-by: David Scott <dave.scott@citrix.com>
Add a configurable idle timeout to stunnel

Signed-off-by: Jon Ludlam <jonathan.ludlam@eu.citrix.com>

Signed-off-by: Akshay <akshay.ramani@citrix.com>
Remove last use of Re_str from xapi

Signed-off-by: Jon Ludlam <jonathan.ludlam@citrix.com>
Merge pull request #1766 from jonludlam/remove_re_str

Remove last use of Re_str from xapi
Merge pull request #1741 from djs55/smapiv2

Use xcp-idl for SMAPIv2 services
CA-135934: use the same estimate of dom0 'static_max' everywhere

We sample the amount of free pages on a host and add an estimate
of dom0 'static_max' to it to create the boot_free_mem.

Later we substract a different estimate of dom0 'static_max' and
erroneously conclude there is free memory when there isn't.

Note there is no good estimate of dom0 'static_max' because dom0
doesn't really know how much memory it has: the xen command-line
says one thing; the domain_getinfo hypercall says another thing;
the "free" command-line says a 3rd thing; and the balloon driver
says a 4th thing. However the main property we need is that we use
the same estimate everywhere.

Signed-off-by: David Scott <dave.scott@citrix.com>
Remove the hardcoded disabling of ballooning for dom0

To express "don't balloon me", set the dynamic-min and dynamic-max
to the same values.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Merge pull request #1768 from djs55/CA-135934

CA-135934: use the same estimate of dom0 'static_max' everywhere
Our cookie are even more broken than I thought.

Ours have spaces in the values. The newer Stringext cookie extraction
doesn't allow this, unlike the previous cohttp version. So rather than
use stringext, use a hand-rolled parser.

Signed-off-by: Jon Ludlam <jonathan.ludlam@citrix.com>
Merge pull request #1767 from jonludlam/fix_migration

Our cookie are even more broken than I thought.
Add 'xe vdi-export' for symmetry with 'xe vdi-import'

  xe vdi-export uuid=<VDI> filename=<filename>

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
xe vdi-{export,import}: display a progress bar if "--progress" is given

Note we now send 'PrintStderr' commands to the CLI while it is busy
downloading or uploading disk images. A multithreaded CLI is necessary
for this to work.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>