CA-137830: enable memory caching for the VHD cache in IntelliCache

This patch enables memory caching for the VHD file acting as the cache
in IntelliCache, despite the fact that it is opened in R/W mode. Without
this, read caching interacts badly with IntelliCahce. This is safe to do
so as the cache file is always thrown away.

Signed-off-by: Thanos Makatos <thanos.makatos@citrix.com>
Reviewed-by: Ross Lagerwall <ross.lagerwall@citrix.com>
CA-137830: change behaviour of vhd_cache_enabled

vhd_cache_enabled must return true not only if the VHD_CACHED_ENABLED
bit is set, but also if the VHD file in opened in read only mode
(VHD_OPEN_RDWR is not set). This ensures that if the leaf is opened R/W
and for buffered I/O (which is the case for IntelliCache), the parent
chain is is not considered 'cached'.

Signed-off-by: Thanos Makatos <thanos.makatos@citrix.com>
Reviewed-by: Ross Lagerwall <ross.lagerwall@citrix.com>
CA-137830: don't pass TD_OPEN_NO_O_DIRECT to parent in IntelliCache

If IntelliCache is used, TD_OPEN_NO_O_DIRECT must not be passed to the
parent otherwise tapdisk will perform buffered I/O to the parent, and
this is meaningless as each block read from the parent is only read
once.

Signed-off-by: Thanos Makatos <thanos.makatos@citrix.com>
Reviewed-by: Ross Lagerwall <ross.lagerwall@citrix.com>
Merge branch 'CA-137830' into xs64bit

Signed-off-by: Thanos Makatos <thanos.makatos@citrix.com>

GitHub: closes #94 on xapi-project/blktap
CAR-1442: Allow 'tap-ctl create' to disable O_DIRECT

Signed-off-by: Germano Percossi <germano.percossi@citrix.com>
Reviewed-by: Ross Lagerwall <ross.lagerwall@citrix.com>

GitHub: closes #86 on xapi-project/blktap
Revert "Revert "Merge branch 'CA-135969' into xs64bit""

This reverts commit 0ef84c6b7c3a6f43e9e7315bbae5aec029f04e0d.

Signed-off-by: Thanos Makatos <thanos.makatos@citrix.com>
Reviewed-by: Chandrika Srinivasan <chandrika.srinivasan@citrix.com>
CA-135969: block all other signals

And use sigaction for handling signals.

Signed-off-by: Thanos Makatos <thanos.makatos@citrix.com>
Reviewed-by: Chandrika Srinivasan <chandrika.srinivasan@citrix.com>
Merge branch 'CA-135969' into xs64bit

Signed-off-by: Thanos Makatos <thanos.makatos@citrix.com>

GitHub: closes #95 on xapi-project/blktap
CA-138289: produce ring statistics in /dev/shm

blkback maintains shared ring statistics in a file in sysfs
(vbd-domid-devid/io_ring). This patch implements the same functionality
in tapdisk: tapdisk exports the same information under a new file in
/dev/shm (e.g /dev/shm/vbd3-domid-devid/io_ring), once every thirty
seconds for performance improvements. NB we don't deal with other
processes touching these files.

Signed-off-by: Thanos Makatos <thanos.makatos@citrix.com>
Reviewed-by: Siddharth Vinothkumar <siddharth.vinothkumar@citrix.com>

GitHub: closes #93 on xapi-project/blktap
CA-138331: properly fail write I/O barrier requests

Signed-off-by: Thanos Makatos <thanos.makatos@citrix.com>
Reviewed-by: Germano Percossi <germano.percossi@citrix.com>

GitHub: closes #96 on xapi-project/blktap
CA-139357: ignore SIGCHLD in master tapback

Signed-off-by: Thanos Makatos <thanos.makatos@citrix.com>
Reviewed-by: Frediano Ziglio <frediano.ziglio@citrix.com>
CA-135774: don't preallocate memory, use malloc and cache it instead

Signed-off-by: Thanos Makatos <thanos.makatos@citrix.com>
Reviewed-by: Felipe Franciosi <felipe.franciosi@citrix.com>
CA-135774: Wait before releasing request-associated buffers

Signed-off-by: Felipe Franciosi <felipe.franciosi@citrix.com>
Reviewed-by: Thanos Makatos <thanos.makatos@citrix.com>
CA-135774: Replace libcrypto with minimal md5 implementation

libcrypto uses about 150KB or private dirty memory per tapdisk process.
Replace libcrypto's md5 implementation with a public domain OpenSSL
compatible md5 implementation to save memory.

Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
Reviewed-by: Thanos Makatos <thanos.makatos@citrix.com>
CA-135774: Use mmap rather than posix_memalign for allocating memory

Use mmap rather than posix_memalign so that memory can be properly
returned to the OS rather than simply returned to a free pool.

Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
Reviewed-by: Thanos Makatos <thanos.makatos@citrix.com>
CA-135774: Return the correct number of requests copied

Previously, it was possible that the caller asked for 1 request to be
copied but the valued returned was > 1.

Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
Reviewed-by: Thanos Makatos <thanos.makatos@citrix.com>
CA-135774: Limit memory used when under memory pressure

Force tapdisk to process requests 1 at a time when under memory pressure
to limit the amount of memory used.  Always keep at least 1 request in
the bufcache for this purpose.

Use memory cgroups to listen to critical memory pressure notifications.
The algorithm used is as follows:
Enter low memory mode when a memory pressure notification is received.
Use truncated exponential backoff to decide when to exit low memory
mode.  Add a random factor to prevent all tapdisks entering and exiting
at the same time.
If no notification is received in a set period of time when in normal
mode, reset the backoff factor.

Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
Reviewed-by: Thanos Makatos <thanos.makatos@citrix.com>
Merge branch 'CA-135774' into xs64bit

Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>

GitHub: closes #104 on xapi-project/blktap
CA-140403: Unregister bufcache expire event on disconnect

Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
Reviewed-by: Thanos Makatos <thanos.makatos@citrix.com>

GitHub: closes #105 on xapi-project/blktap
CA-140391: check for overflow

Signed-off-by: Thanos Makatos <thanos.makatos@citrix.com>
Reviewed-by: Mate Lakat <mate.lakat@citrix.com>

GitHub: closes #108 on xapi-project/blktap