Use rbtree bitmaps for dumpe2fs, debugfs, and tune2fs

For large file systems, using the rbtree bitmap can save a lot of
memory.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
e2fsck: add support for field widths in messages using %-expansion

This will come in handy when printing checksums, some of which are
32-bit and some of which are 16-bit.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
e2fsck: print the current and expected block group checksums

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
e2fsck: print a notice when we've started suppressing a problem code

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
e2fsck: add the max_count_problems setting in e2fsck.conf

Also add appropriate documentation for options/max_count_problems and
problems/0xXXXXXX/max_count settings in /etc/e2fsck.conf

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
e2fsck: add logging capability

Add the ability to log messages about a file system to a specified
directory, using a file name templace that can be specified in
/etc/e2fsck.conf.  This allows us to suppress the output of overly
verbose e2fsck outputs while still allowing the full logging output to
go to an appropriate file.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
tests: have "make testnew" target use local mke2fs

The "make testnew" target in the tests/ subdirectory (used to
help create new test cases) should use the locally-built mke2fs
binary instead of the installed system binary, to avoid cases
where the system binary does not support some new functionality
being tested.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
tests: add field sizes to inode/super struct tests

In addition to validating the ordering of fields within the inode
and superblock structures, also validate the field sizes.  Otherwise
it is possible to incorrectly change the size of one of these fields
without getting any kind of error from these tests.  Failures would
only show up later in the test image checks if the field that is
changed is before another in-use field.

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
e2fsck: abort if ext2fs_check_desc() returns a memory failure

We were checking for ENOMEM, but in fact if the malloc() fails,
ext2fs_check_desc() will return EXT2_ET_NO_MEMORY.

Addresses-Google-Bug: #6208183

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
e2fsck: avoid unnecessary close/reopen for non-MMP filesystems

For file systems that do not use MMP, there's no reason to close the
file system and then re-open the file system a second time, since
EXT2_FLAG_SKIP_MMP has no meaning for non-MMP file systems anyway.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Cc: Andreas Dilger <adilger@whamcloud.com>
e2fsck: report ext2fs_open2() and ext2fs_check_desc() errors

Print the actual errors returned by ext2fs_open2() and
ext2fs_check_desc() before we fall back to the backup block group
descriptors so that it's easier to see if there is some obscure
failure that is causing e2fsck to think that it should use the backup
block group descriptors.

Addresses-Google-Bug: #6208183

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
mke2fs: add option to use direct I/O (mke2fs -D)

This can be useful when using mke2fs on loaded servers, since
otherwise mke2fs can dirty a huge amount of memory very quickly,
leading to other applications not being happy at all.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
e2fsck: remove EXT4_EOFBLOCKS_FL flag handling

We've decided to remove EOFBLOCKS_FL from the ext4 file system entirely,
because it is not actually very useful and it is causing more problems
than it solves. We're going to remove it from e2fsprogs first and then
after the new e2fsprogs version is common enough we can remove the
kernel part as well.

This commit changes e2fsck to not check for EOFBLOCKS_FL. Instead we
simply search for initialized extents past the i_size as this should not
happen. Uninitialized extents can be past the i_size as we can do
fallocate with KEEP_SIZE flag.

Also remove the EXT4_EOFBLOCKS_FL from lib/ext2fs/ext2_fs.h since it is
no longer needed.

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
libext2fs: move a modulo operation out of a hot loop.

Filesystem shrinking in particular is a heavy user of this loop in
ext2fs_new_inode(). This change makes resize2fs use 24% less CPU time
for shrinking a 100G filesystem.

Signed-off-by: Sami Liedes <sami.liedes@iki.fi>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
resize2fs: use EXT2_FLAG_64BITS

By passing EXT2_FLAG_64BITS to ext2fs_open2() we can avoid some
unnecessary redirection in critical paths. While resize2fs does not
currently otherwise support so big filesystems that this would matter,
passing this flag is entirely harmless and only tells libext2fs that
the caller has been recompiled against current headers.

With this change the CPU time needed to shrink a 100G filesystem drops
by 20%.

Signed-off-by: Sami Liedes <sami.liedes@iki.fi>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
libext2fs: document EXT2_FLAG_64BITS in ext2fs_open2()

From 5c7302de65d689ad84427649aab4d361b21270df Mon Sep 17 00:00:00 2001
From: Sami Liedes <sami.liedes@iki.fi>
Date: Sat, 10 Mar 2012 22:25:55 +0200
Subject: [PATCH] libext2fs: Document EXT2_FLAG_64BITS in ext2fs_open2().

Signed-off-by: Sami Liedes <sami.liedes@iki.fi>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
libext2fs: Implement ext2fs_find_first_zero_generic_bmap().

This function searches a bitmap for the first zero bit within a range.
It checks if there is a bitmap backend specific implementation
available (if the relevant field in bitmap_ops is non-NULL). If not,
it uses a generic and slow method by repeatedly calling test_bmap() in
a loop. Also change ext2fs_new_inode() to use this new function.

This change in itself does not result in a large speedup, rather it
refactors the code in preparation for the introduction of a faster
find_first_zero() for bitarray based bitmaps.

Signed-off-by: Sami Liedes <sami.liedes@iki.fi>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
libext2fs: update dirty flags if the uninit flags are cleared

Update the block group descriptor checksum and mark the superblock and
allocation bitmaps as dirty in check_inode_uninit() and
check_block_uninit().

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
debugfs: teach seti and freei to free a block of inodes

In order to the inode allocation functions, it's useful to set and
clear a range of inodes.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
libext2fs: implement fast find_first_zero() for bitarray bitmaps.

With this change the CPU time needed to shrink a 100G filesystem drops
to 0.8% of the original (17 CPU seconds instead of 2057).

Signed-off-by: Sami Liedes <sami.liedes@iki.fi>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>