docs: add dirty bit to qcow2 specification

The dirty bit will make it possible to perform lazy refcount updates,
where the image file is not kept consistent all the time.  Upon opening
a dirty image file, it is necessary to perform a consistency check and
repair any incorrect refcounts.

Therefore the dirty bit must be an incompatible feature bit.  We don't
want old programs accessing a file with stale refcounts.

Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
qcow2: introduce dirty bit

This patch adds an incompatible feature bit to mark images that have not
been closed cleanly.  When a dirty image file is opened a consistency
check and repair is performed.

Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
docs: add lazy refcounts bit to qcow2 specification

The lazy refcounts bit indicates that this image can take advantage of
the dirty bit and that refcount updates can be postponed.

Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
qemu-iotests: ignore qemu-img create lazy_refcounts output

Hide the default lazy_refcounts=off output from qemu-img like we do with
other image creation options.  This ensures that existing golden outputs
continue to pass despite the new option that has been added.

Note that this patch applies before the one that actually introduces the
lazy_refcounts=on|off option.  This ensures git-bisect(1) continues to
work.

Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
qcow2: implement lazy refcounts

Lazy refcounts is a performance optimization for qcow2 that postpones
refcount metadata updates and instead marks the image dirty.  In the
case of crash or power failure the image will be left in a dirty state
and repaired next time it is opened.

Reducing metadata I/O is important for cache=writethrough and
cache=directsync because these modes guarantee that data is on disk
after each write (hence we cannot take advantage of caching updates in
RAM).  Refcount metadata is not needed for guest->file block address
translation and therefore does not need to be on-disk at the time of
write completion - this is the motivation behind the lazy refcount
optimization.

The lazy refcount optimization must be enabled at image creation time:

  qemu-img create -f qcow2 -o compat=1.1,lazy_refcounts=on a.qcow2 10G
  qemu-system-x86_64 -drive if=virtio,file=a.qcow2,cache=writethrough

Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
qemu-io: add "abort" command to simulate program crash

Avoiding data loss and corruption is the top requirement for image file
formats.  The qemu-io "abort" command makes it possible to simulate
program crashes and does not give the image format a chance to cleanly
shut down.  This command is useful for data integrity test cases.

Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
qemu-iotests: add 039 qcow2 lazy refcounts test

This tests establishes the basic post-conditions of the qcow2 lazy
refcounts features:

  1. If the image was closed normally, it is marked clean.

  2. If an allocating write was performed and the image was not close
     normally, then it is marked dirty.

     a. Written data can be read back successfully.
     b. The image file can be repaired and will be marked clean again.

Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>