ploop_receive(): pass errno to ploop_err()

Not sure why it was omitted in this very case. nread() always sets
errno so it makes sense to show it.

Signed-off-by: Kir Kolyshkin <kir@openvz.org>
ploop copy -s: add -o to specify output fd

While ploop_send() can write to any file descriptor, there was no way
in ploop copy tool to specify one. Add -o output_fd option. For one
thing, it lets us have stdout free for log messages.

Also, document -o in ploop(8) man page.

Signed-off-by: Kir Kolyshkin <kir@openvz.org>
ploop_send(): ability to profile timings

This patch makes it possible to profile critical ploop_send() timings.

Example output:

	# exec 3>/dev/null
	# ploop -vvvvv copy -s /dev/ploop47147 -o 3

[ 1.783753] psend: SEND 0 4 (sync)
[ 6.784464] psend: FLUSH
[ 6.784476] psend: SYNCFS
[ 6.784717] psend: FIFREEZE
[ 6.785863] psend: IOC_SYNC
[ 6.790003] psend: READ 0 1048576
[ 6.799990] psend: SEND 0 1048576
[ 6.800000] psend: READ 1048576 1048576
[ 6.809942] psend: SEND 1048576 1048576
[ 6.809952] psend: READ 0 4096
[ 6.810571] psend: SEND 0 512 (1st sector)
[ 6.810579] psend: IOCTL TRACK_STOP
[ 6.810593] psend: SEND 0 0 (close)
[ 6.811377] psend: DONE

Signed-off-by: Kir Kolyshkin <kir@openvz.org>
ploop_send(): use fdatasync() if writing locally

If we are writing to the local file (rather than to a pipe with ploop
copy on the other end), there is no need to send magic SYNC_MARK and
wait for 5 seconds for the other side to complete sync. Instead,
just do fdatasync().

Signed-off-by: Kir Kolyshkin <kir@openvz.org>
Introduce ploop_copy_send(), ploop_copy_receive()

This patch introduces two new functions -- ploop_copy_send()
and ploop_copy_receive() -- to replace old ploop_send() and
ploop_receive(), which are to be marked as obsoleted.

The only difference for now is new ploop_copy* functions receive
their argument via a structure, making it possible to add more
arguments.

Signed-off-by: Kir Kolyshkin <kir@openvz.org>
ploop copy: switch to new ploop_copy_* functions

Also, mark ploop_send/ploop_receive as deprecated for linker.

Signed-off-by: Kir Kolyshkin <kir@openvz.org>
ploop_copy_receive(): add ifd parameter

Adds an ability to read data from the specified file descriptor,
instead of stdin.

Signed-off-by: Kir Kolyshkin <kir@openvz.org>
ploop copy receive: add -i input_fd

Add an ability for ploop copy receiving side to specify FD to read from.
Document it in ploop(8).

Signed-off-by: Kir Kolyshkin <kir@openvz.org>
ploop_copy_send(): ignore is_pipe

When checking if we need to disable logging to stdout (fd 1), because
we are writing data to it, it doesn't matter if it is a pipe or maybe
someone clever closed stdout/stderr so open(file) returned fd of 1.

So, remove is_pipe check.

Signed-off-by: Kir Kolyshkin <kir@openvz.org>
ploop_copy_send(): check ofd == stderr

In case we are going to write data to stderr, we need to disable
logging to stderr. Unfortunately currently there's no way to disable
only stderr, leaving stdout intact, so we disable both. Note that
the last error message is still available via ploop_get_last_error().

Hopefully this is a rare situation when someone is reusing stderr.

Signed-off-by: Kir Kolyshkin <kir@openvz.org>
ploop_copy_*: add feedback_fd

Before now, ploop copy protocol used to be one way, from a sender to a
receiver. While it is perfectly fine for most scenarios, there is one
particular operation that can benefit from the explicit completion
notification -- this is remote fsync() introduced in commit d56654a.

Now, if there is a way to establish either a two-way pipe (say using
socketpair() or ssh port forwarding) or just a second pipe back from
the receiver to the sender (not sure how to do this but there might
be a way), we can finally eliminate that ugly 5s wait for the sending
side sync to happen, instead proceeding as soon as it is done.

Overall effect is to optimize the frozen time during container
live migration. Appropriate support is to be added to ploop tools
and then vzmigrate (unfortunately it requires this newer ploop
to be available on both sides, otherwise we fall back to that 5s wait).

Signed-off-by: Kir Kolyshkin <kir@openvz.org>
ploop copy: add -f feedback_fd option

Add support for feedback_fd for ploop copy tool. See previous commit
for explanation.

Document it in ploop(8). TODO: add examples.

Signed-off-by: Kir Kolyshkin <kir@openvz.org>
ploop_copy_send(): introduce/use do_pread macro

We do the same thing a few times -- read, check if the result is
negative. Let's wrap it in a macro.

While at it, fix some error checks and appropriate messages:
* negative number is an error (and is taken care of inside a macro
* 0 is an EOF and it depends whether it's an error or OK
* any number less than required can be either a short read error or OK

Signed-off-by: Kir Kolyshkin <kir@openvz.org>
ploop_copy_send(): implement async send

As pointed out by a comment (removed by this patch), reads from ploop
delta files are O_DIRECT, as ploop is avoiding using page cache
(to prevent double caching, as caching is done on an inner fs level).

So, O_DIRECT reads are slow, and therefore it is good if we can do those
in parallel with writes. This is what this patch implements.

We use two buffers and a separate thread to send data. First, a buffer
is read into, then we signal a sending thread that there is data to
send, specifying the buffer to send it, and then we go on to the next
read operation, using the other buffer this time. When we signal a
sending thread again.

A pthread mutex and a condidional variable is used for synchronization,
so we don't step onto our own toes.

Some performance data below.

This is how sync read/send looks like without this patch, i.e. when read
and send are done in a syncronous fashion (numbers are relative microsecond
timestamps):

0.0097 PREAD
0.0136 SEND
0.0096 PREAD
0.0156 SEND
0.0097 PREAD
0.0153 SEND
0.0097 PREAD
0.0149 SEND
0.0099 PREAD
0.0139 SEND
0.0096 PREAD
0.0166 SEND
0.0097 PREAD
0.0182 SEND

it takes 0.01 second to read a block and 0.015 seconds to send it, and
since it is done syncronously it takes 0.025 seconds to read and send a
block.

After this patch, send and read are parallel (well, after reading the
initial block), so it saves us MIN(read_time, send_time) multiplied
by the number of iteration. Given the data from the example above,
savings should be about 0.01 seconds per block, or 0.07 seconds total.

Now, 0.07 seconds is imprortant because the last part of ploop copy work
during live migration happens after the container is frozen, so it helps
to improve (minimize) container frozen time.

Signed-off-by: Kir Kolyshkin <kir@openvz.org>
ploop_copy*: check arg for non-NULL

Rather than segfault on NULL argument, return SYSEXIT_PARAM.

Signed-off-by: Kir Kolyshkin <kir@openvz.org>