CA-135774: don't preallocate memory, use malloc and cache it instead

Signed-off-by: Thanos Makatos <thanos.makatos@citrix.com>
Reviewed-by: Felipe Franciosi <felipe.franciosi@citrix.com>
CA-135774: Wait before releasing request-associated buffers

Signed-off-by: Felipe Franciosi <felipe.franciosi@citrix.com>
Reviewed-by: Thanos Makatos <thanos.makatos@citrix.com>
CA-135774: Replace libcrypto with minimal md5 implementation

libcrypto uses about 150KB or private dirty memory per tapdisk process.
Replace libcrypto's md5 implementation with a public domain OpenSSL
compatible md5 implementation to save memory.

Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
Reviewed-by: Thanos Makatos <thanos.makatos@citrix.com>
CA-135774: Use mmap rather than posix_memalign for allocating memory

Use mmap rather than posix_memalign so that memory can be properly
returned to the OS rather than simply returned to a free pool.

Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
Reviewed-by: Thanos Makatos <thanos.makatos@citrix.com>
CA-135774: Return the correct number of requests copied

Previously, it was possible that the caller asked for 1 request to be
copied but the valued returned was > 1.

Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
Reviewed-by: Thanos Makatos <thanos.makatos@citrix.com>
CA-135774: Limit memory used when under memory pressure

Force tapdisk to process requests 1 at a time when under memory pressure
to limit the amount of memory used.  Always keep at least 1 request in
the bufcache for this purpose.

Use memory cgroups to listen to critical memory pressure notifications.
The algorithm used is as follows:
Enter low memory mode when a memory pressure notification is received.
Use truncated exponential backoff to decide when to exit low memory
mode.  Add a random factor to prevent all tapdisks entering and exiting
at the same time.
If no notification is received in a set period of time when in normal
mode, reset the backoff factor.

Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
Reviewed-by: Thanos Makatos <thanos.makatos@citrix.com>
Merge branch 'CA-135774' into xs64bit

Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>

GitHub: closes #104 on xapi-project/blktap
CA-140403: Unregister bufcache expire event on disconnect

Signed-off-by: Ross Lagerwall <ross.lagerwall@citrix.com>
Reviewed-by: Thanos Makatos <thanos.makatos@citrix.com>

GitHub: closes #105 on xapi-project/blktap
CA-140391: check for overflow

Signed-off-by: Thanos Makatos <thanos.makatos@citrix.com>
Reviewed-by: Mate Lakat <mate.lakat@citrix.com>

GitHub: closes #108 on xapi-project/blktap
CA-140998: don't create the io_ring stats file uninitialised

Signed-off-by: Thanos Makatos <thanos.makatos@citrix.com>
Reviewed-by: Siddharth Vinothkumar <siddharth.vinothkumar@citrix.com>

GitHub: closes #106 on xapi-project/blktap
CA-140516: use /dev/zero as a source of a zeroed piece of memory

Signed-off-by: Thanos Makatos <thanos.makatos@citrix.com>
Reviewed-by: Chandrika Srinivasan <chandrika.srinivasan@citrix.com>

GitHub: closes #107 on xapi-project/blktap
CA-136792: produce blkback-style stats

Produce I/O statistics in /dev/shm/vbd3-domid-devid. We use one file for
all seven counters as otherwise we're wasting too much memory. Stats are
updated once every five seconds as this is the same frequency the RRD
script reads this file.

Read/write latency stats are produced in
/dev/shm/vbd3-<domid>-<devid>/statistics along with the other stats, in
order to avoid using a separate file as this consumes additional memory
and we're already under heavy pressure to minimise tapdisk's memory
footprint.

In blkback, rd_usecs/wr_usecs stats are cleared each time the sysfs file
is read. In tapdisk it's impossible to tell when the file got read,
therefore we reset these stats right after updating /dev/shm.

Signed-off-by: Thanos Makatos <thanos.makatos@citrix.com>
Reviewed-by: Siddharth Vinothkumar <siddharth.vinothkumar@citrix.com>

GitHub: closes #97 on xapi-project/blktap
CA-136792: produce raw stats

Signed-off-by: Thanos Makatos <thanos.makatos@citrix.com>