cpu: move target-independent CPU_INTERRUPT_* flags to qom/cpu.h

The next patch will use one of them in cpu_common_reset.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
exec: introduce CPU_INTERRUPT_TLBFLUSH

Each change to the memory mapping invalidates the TLB, because new entries
in the TLB are based on a different AddressSpaceDispatch than before.
Synchronize the flush to prepare for atomically switching to a new
AddressSpaceDispatch.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
exec: make iotlb RCU-friendly

After the previous patch, TLBs will synchronize on every change to
the memory mapping.  This patch augments that with synchronization
of the MemoryRegionSections cached in the iotlb array.

Every change to RCU-protected data can now force an exit of the
current translation block, and the new data structure is updated
in the CPUArchState, so that each CPU_INTERRUPT_TLBFLUSH interrupt
constitutes the boundary of an RCU read-side critical section.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
exec: add a reference to the region returned by address_space_translate

Once address_space_translate will only be protected by RCU, the returned
MemoryRegion might disappear as soon as the RCU read-side critical section
ends.  Avoid this by adding a reference to the region, and dropping it
in the caller of address_space_translate.

This generalizes what was done for address_space_map to other callers of
address_space_translate.  It is necessary to call address_space_translate
out of the BQL.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
exec: put address space dispatch under RCU critical section

With this change, address space dispatch can be moved outside the
BQL.  The actual I/O would still have to happen within the lock.

The next step would be to introduce a function that can only
be called from outside the BQL, address_space_rw_unlocked.  The
function would do something like

    mr = address_space_translate(...)
    if (!mr->unlocked) {
        locked = true;
        qemu_mutex_lock_iothread();
    }
    ...dispatch...
    if (locked) {
        qemu_mutex_unlock_iothread();
    }

(Note that subpages are already ready to be changed to unlocked
access).

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
xen: do not use __-named variables in mapcache

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
xen: add a lock for the mapcache

Extend the existing dummy mapcache_lock/unlock macros to cover all of
xen-mapcache.c.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
rcu: introduce RCU-enabled QLIST

Add RCU-enabled variants on the existing bsd DQ facility. Each
operation has the same interface as the existing (non-RCU)
version. Also, each operation is implemented as macro.

Using the RCU-enabled QLIST, existing QLIST users will be able to
convert to RCU without using a different list interface.

Signed-off-by: Mike Day <ncmike@ncultra.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
exec: remove qemu_safe_ram_ptr

This is not needed since the RAM list is not modified anymore by
qemu_get_ram_ptr.  Replace it with qemu_get_ram_block.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
exec: protect mru_block with RCU

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
cosmetic changes preparing for the following patch

Signed-off-by: Mike Day <ncmike@ncultra.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Convert ram_list to RCU

Allow "unlocked" reads of the ram_list by using an RCU-enabled
DQ. Most readers of the list no longer require holding the list mutex.

The ram_list now uses a QLIST instead of a QTAILQ. The difference is
minimal.

Signed-off-by: Mike Day <ncmike@ncultra.org>
[More cleanup of comments on which functions need "extra" protection
 in the caller.  These are qemu_get_ram_block (static), qemu_get_ram_ptr,
 qemu_ram_ptr_length, qemu_ram_addr_from_host. - Paolo]
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
compatfd: switch to QemuThread

qemu_thread_create already does signal blocking and detaching for us.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
portio: Allow to mark portio lists as coalesced MMIO flushing

This will enable us to remove all remaining explicit calls of
qemu_flush_coalesced_mmio_buffer in IO handlers.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
cirrus: Mark vga io region as coalesced MMIO flushing

This allows to remove the explicit qemu_flush_coalesced_mmio_buffer
calls - the memory core will invoke them now.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
vga: Mark relevant portio lists regions as coalesced MMIO flushing

This allows to remove the explicit qemu_flush_coalesced_mmio_buffer
calls.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
memory: Add global-locking property to memory regions

This introduces the memory region property "global_locking". It is true
by default. By setting it to false, a device model can request BQL-free
dispatching of region accesses to its r/w handlers. The actual BQL
break-up will be provided in a separate patch.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
memory: Provide address_space_rw_unlocked

Break up address_space_rw into two version: The standard one continues
to require the caller to hold BQL on invocation, the unlocked one takes
or avoids BQL depending on the locking strategy of the target memory
region and its coalesced MMIO handling.

As memory_region_read/write_accessor may now also run without BQL held,
we need to move coalesced MMIO flushing earlier in the dispatch process.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
kvm: First step to push iothread lock out of inner run loop

This opens the path to get rid of the iothread lock on vmexits in KVM
mode. On x86, the in-kernel irqchips has to be used because we otherwise
need to synchronize APIC and other per-cpu state accesses that could be
changed concurrently.

s390x and ARM should be fine without specific locking as their
pre/post-run callbacks are empty. Power requires locking for the pre-run
callback.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
kvm: Switch to unlocked PIO

Do not take the BQL before dispatching PIO requests of KVM VCPUs.
Instead, call the unlocked version of address_space_rw. This enables
completely BQL-free PIO handling in KVM mode for upcoming devices with
fine-grained locking.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>