protect the ramlist with a separate mutex

Add the new mutex that protects shared state between ram_save_live
and the iothread.  Writes need to take both locks (with the iothread
lock outside).  Reads can take either lock.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Umesh Deshpande <udeshpan@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Orit Wasserman <owasserm@redhat.com>
buffered_file: Move from using a timer to use a thread

We still protect everything except the wait with the iothread lock.
But we moved from a timer to a thread.  Steps one by one.

We also need to detect when we have finished with a variable "complete".

Signed-off-by: Juan Quintela <quintela@redhat.com>
migration: make qemu_fopen_ops_buffered() return void

We want the file assignment to happen before the thread is created to
avoid locking, so we just do it before creating the thread.

Signed-off-by: Juan Quintela <quintela@redhat.com>

Reviewed-by: Orit Wasserman <owasserm@redhat.com>
migration: stop all cpus correctly

FIXME: wrong commit message

You can only stop all cpus from the iothread or an vcpu.  As we want
to do it from the migration_thread, we need to do this dance with the
botton handlers.

This patch is a request for ideas.  I can move this function to cpus.c, but
wondered if there is an easy way of doing this?

Signed-off-by: Juan Quintela <quintela@redhat.com>
migration: make writes blocking

Move all the writes to the migration_thread, and make writings
blocking.  Notice that are still using the iothread for everything
that we do.

Signed-off-by: Juan Quintela <quintela@redhat.com>
migration: remove unfreeze logic

Now that we have a thread, and blocking writes, we don't need it.

Signed-off-by: Juan Quintela <quintela@redhat.com>
migration: take finer locking

Instead of locking the whole migration_thread inside loop, just lock
migration_fd_put_notify, that is what interacts with the rest of the
world.

Signed-off-by: Juan Quintela <quintela@redhat.com>
buffered_file: Unfold the trick to restart generating migration data

This was needed before due to the way that the callbacks worked.

Signed-off-by: Juan Quintela <quintela@redhat.com>
buffered_file: don't flush on put buffer

We call buffered_put_buffer with iothread held, and buffered_flush() does
synchronous writes.  We only want to do the synchronous writes outside.

Signed-off-by: Juan Quintela <quintela@redhat.com>
buffered_file: unfold buffered_append in buffered_put_buffer

It was the only user, and now buffered_put_buffer just do the append

Signed-off-by: Juan Quintela <quintela@redhat.com>
savevm: New save live migration method: pending

Code just now does (simplified for clarity)

    if (qemu_savevm_state_iterate(s->file) == 1) {
       vm_stop_force_state(RUN_STATE_FINISH_MIGRATE);
       qemu_savevm_state_complete(s->file);
    }

Problem here is that qemu_savevm_state_iterate() returns 1 when it
knows that remaining memory to sent takes less than max downtime.

But this means that we could end spending 2x max_downtime, one
downtime in qemu_savevm_iterate, and the other in
qemu_savevm_state_complete.

Changed code to:

    pending_size = qemu_savevm_state_pending(s->file, max_size);
    DPRINTF("pending size %lu max %lu\n", pending_size, max_size);
    if (pending_size >= max_size) {
        ret = qemu_savevm_state_iterate(s->file);
     } else {
        vm_stop_force_state(RUN_STATE_FINISH_MIGRATE);
        qemu_savevm_state_complete(s->file);
     }

So what we do is: at current network speed, we calculate the maximum
number of bytes we can sent: max_size.

Then we ask every save_live section how much they have pending.  If
they are less than max_size, we move to complete phase, otherwise we
do an iterate one.

This makes things much simpler, because now individual sections don't
have to caluclate the bandwidth (it was implossible to do right from
there).

Signed-off-by: Juan Quintela <quintela@redhat.com>
migration: include qemu-file.h

They don't use/know anything about buffered-file.

Signed-off-by: Juan Quintela <quintela@redhat.com>
migration-fd: remove duplicate include

Signed-off-by: Juan Quintela <quintela@redhat.com>
memory: introduce memory_region_test_and_clear_dirty

This function avoids having to do two calls, one to test the dirty bit, and
other to reset it.

Signed-off-by: Juan Quintela <quintela@redhat.com>
ram: Use memory_region_test_and_clear_dirty

This avoids having to do two walks over the dirty bitmap, one reading
the dirty bits, and another cleaning them.

Signed-off-by: Juan Quintela <quintela@redhat.com>
ram: optimize migration bitmap walking

Instead of testing each page individually, we search what is the next
dirty page with a bitmap operation.  We have to reorganize the code to
move from a "for" loop, to a while(dirty) loop.

Signed-off-by: Juan Quintela <quintela@redhat.com>
buffered_file: move to migration.c

We want to make QEMUBufferedFile a thinner wrapper, using MigrationState
directly.  Move its contents to migration.c.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
migration: make functions static

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
migration: inline qemu_fopen_ops_buffered into migrate_fd_connect

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
migration: add XFER_LIMIT_RATIO

The "magic" divisions by 10 are there because of the value of BUFFER_DELAY.
Introduce a constant to explain them better.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>