Small tidy up: move 'type size' outside the functor

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
streaming: add a function to expand 'Empty' elements into explicit writes

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
Import sparse_dd.ml from xen-api/ocaml/xapi/sparse_dd

We will remove all the legacy streaming code and leave only the
commandline parser for backwards compatibility.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
streaming: add a function to expand Copy elements into full writes

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
cli: when writing NBD, expand the Copy and Empty elements in the stream

The expansion of Copy and Empty elements can be shared amongst
multiple outputs, including NBD and 'XenServer chunked' (next)

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
cli: streaming: connect to the server using a raw file descriptor

We currently don't support SSL because the NBD library uses
Lwt_unix.file_descr directly, and doesn't support Lwt_ssl.socket.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
cli: streaming: add support for XenServer-style chunked encoding

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
cli: stream: separate notions of destination and transport (encoding)

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
cli: stream: implement 'put' transport

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
sparse_dd: frontend now compiles

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
cli: streaming: remove spurious debug logging

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
cli: streaming: fix XenServer chunked encoding

We accidentally sent the length of the header rather than the
length of the whole block.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
sparse_dd: hook up machine-readable log printing

This is used for signalling progress to a calling program.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
sparse_dd: add debugging around the backing vhd discovery

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
cli: streaming: cap the progress bar to 100%

The size of the stream is based on an estimate and might be slightly
wrong.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
cli: streaming: add an ETA to the progress bar

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
streaming: size should be in bytes; we should always give a total

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
cli: streaming: fix progress bar, add summary output at the end

Signed-off-by: David Scott <dave.scott@eu.citrix.com>
cli: streaming: negotiate a transport encoding, if one isn't provided

In the case of XenServer HTTP PUT, we can tell from the HTTP response
header that 'chunked' is a valid encoding. Otherwise we PUT as a raw
image.

Signed-off-by: David Scott <dave.scott@eu.citrix.com>