mksquashfs: make -mem take a normal byte value, optionally with a K, M or G

Rather than the hack I used for the original -xxx-queue options which
took a integer value which was considered to be in megabytes.

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
process_fragments: remove commented out debugging SQUASHFS_TRACE

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
gzip_wrapper: LZO_WRAPPER_H should be GZIP_WRAPPER_H

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
action: protect against multiple inclusion

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
caches-queues-lists: protect against multiple inclusion

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
compressor: protect against multiple inclusion

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
error: protect against multiple inclusion

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
info: protect against multiple inclusion

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
process_fragments: protect against multiple inclusion

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
progressbar: protect against multiple inclusion

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
pseudo: protect against multiple inclusion

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
restore: protect against multiple inclusion

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
unsquashfs: protect against multiple inclusion

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
unsquashfs_info: protect against multiple inclusion

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
xattr: protect against multiple inclusion

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
unsquashfs: update version for release

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
mksquashfs: update version for release

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
README: update for release

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
mksquashfs: optimise fragment duplicate checking

Oops, one last commit before release, which I noticed I'd forgotten
to do when writing up the release notes ...

Make the uncompressed fragment cache and the reserve cache grow
to maximum size before re-using freed items on the free list.

Up until now (unless we're appending) it has been policy to re-use
items on the free list if available, rather than grow the cache.
This was done for a very good reason, up until recently Mksquashfs
lacked the ability to tune its memory usage to the amount
of memory in the machine being run on.  Lacking the ability to
determine if the maximum size of the caches was a good fit
for the machine, it has always been wiser to avoid aggressively
growing the caches, and to only allow the caches to grow as a
result of readahead.

The advantages of doing this was it actively avoided growing the
caches so they were too large for the machine being run on, if it
had low memory.  The flip side of this is the caches often times
never grow to maximum size.

This is important because the size of the fragment cache
directly affects fragment lookup performance in fragment
duplicate checking.  The larger the cache, the more likelihood
the fragment being looked up will still be in the cache, and
this avoids a costly re-read from the filesystem and
decompress.

So on the one hand the failure to aggressively grow the caches has
been good, on the other hand it has always impacted the performance
of fragment duplicate checking.

Now that Mksquashfs accurately tunes its memory usage to
the amount of memory in the machine being run on, we don't have to
worry about aggressively growing the caches to maximum size - we
know the computed maximum size will not be larger than the machine
can handle.  So change to growing the caches in preference to
taking off the free list.

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>
gzip_wrapper: fix nasty speed/compression regression

Final pre-release tests threw up an odd regression in compression
and speed with the gzip_wrapper.c code.  Sample filesystem compression
was ~0.025% (or 1Mbyte extra per 4Gbytes of compressed data) worse
than the previous Mksquashfs 4.2.

A git bisect rather unexpectedly narrowed the culprit down to this
line in gzip_wrapper.c

res = deflateInit2(stream, compression_level, Z_DEFLATED,
		window_size, 9, Z_DEFAULT_STRATEGY);

This replaced the equivalent

res = deflateInit(stream, compression_level);

When the gzip_wrapper code was extended to allow the window_size
used in compression to be changed.

The culprit in the line is the 9, which is the amount of memory
deflate uses to compress the data.  I took advantage of the fact
that deflateInit2() allows one to increase the memory usage from the
default of 8 (used by deflateInit()) to the maximum of 9.  As explained
by the comments in zlib.h, this is supposed to increase speed.

From zlib.h

"    The memLevel parameter specifies how much memory should be allocated
   for the internal compression state.  memLevel=1 uses minimum memory but is
   slow and reduces compression ratio; memLevel=9 uses maximum memory for
   optimal speed.  The default value is 8.  See zconf.h for total memory usage
   as a function of windowBits and memLevel. "

It in fact appears from my tests that increasing the value from 8 to 9
both reduces compression and reduces speed too.  This is probably
a bug in zlib ...

Using the default value of 8, rather than 9, fixes the small compression
and speed regression observed.

Signed-off-by: Phillip Lougher <phillip@squashfs.org.uk>