libext2fs, tests: allow /etc/mtab file to be missing

The environment variable EXT2FS_NO_MTAB_OK will suppress the error
code EXT2_NO_MTAB_FILE when the /etc/mtab file can not be found.  This
allows the e2fsprogs regression test suite to be run in chroots which
might not have an /etc/mtab file.

By default will still want to complain if the /etc/mtab file is
missing, since we really don't want to discourage distributions and
purveyors of embedded systems from running without an /etc/mtab file.
But if it's missing it only results in a missing sanity check that
might cause file system corruption if the file system is mounted when
programs such as e2fsck, tune2fs, or resize2fs is running, so there is
no potential security problems that might result if this environment
variable is set inappropriately.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
e2fsck: check extent-mapped directories with really large logical blocks

E2fsck was missing a check for directories with logical blocks so
large that i_size > 2GB.  Without this check the test image found in
the new test f_toobig_extent_dir will cause e2fsck to die with a
memory allocation failure:

Error storing directory block information (inode=12, block=0, num=475218819): Memory allocation failed
e2fsck: aborted

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reported-by: Andrey Melnikov <temnota.am@gmail.com>
e2fsck: correctly deallocate invalid extent-mapped symlinks

The function deallocate_inode() in e2fsck/pass2.c was buggy in that it
would clear out the inode's mode and flags fields before trying to
deallocate any blocks which might belong to the inode.

The good news is that deallocate_inode() is mostly used to free inodes
which do not have blocks: device inodes, FIFO's, Unix-domain sockets.

The bad news is that if deallocate_inode() tried to free an invalid
extent-mapped inode, it would try to interpret the root of the extent
node as block numbers, and would therefore mark various file system
metadata blocks (the superblock, block group descriptors, the root
directory, etc.) as free and available for allocation.  This was
unfortunate.

(Try running an older e2fsck against the test file system image in the
new test f_invalid_extent_symlink, and then run e2fsck a second time
on the fs image, and weep.)

Fortunately, this kind of file system image corruption appears to be
fairly rare in actual practice, since it would require a very unlucky
set of bits to be flipped, or a buggy file system implementation.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
debugfs: properly set up extent header in do_write

do_write doesn't fully set up the first extent header on a new
inode, so if we write a 0-length file, and don't write any data
to the new file, we end up creating something that looks corrupt
to kernelspace:

EXT4-fs error (device loop0): ext4_ext_check_inode:464: inode #12: comm ls: bad header/extent: invalid magic - magic 0, entries 0, max 0(0), depth 0(0)

Do something similar to ext4_ext_tree_init() here, and
fill out the first extent header upon creation to avoid this.

Reported-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Tested-by: Robert Yang <liezhi.yang@windriver.com>
debian: remove old symlinks to /usr/share/doc

(From NMU'ed change)

Addresses-Debian-Bug: #698879

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
debian: remove old symlinks to /usr/share/doc

(Merged in NMU'ed change for Debian-Bug: #698879)

Merge commit '7dc67c0ad5daa7cbdf6d79a73bb34e04d55f7406' into maint

Conflicts:
	debian/changelog
e2fsck: don't try to stop mmp if there is no superblock set up

Under some failure cases, we can get to fatal_error()
without even having a superblock set up.  In that case,
ext2fs_mmp_stop() will segfault when it tries to dereference
fs->super.

Check for the existence of a superblock before we go
down the ext2fs_mmp_stop() path to avoid this problem.

Reported-by: Hubert Kario <hkario@redhat.com>
Addresses-Red-Hat-Bugzilla: #997972
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
libext2fs: ext2fs_dup_handle should not alias MMP buffers

It turns out that resize2fs uses ext2fs_dup_handle to duplicate fs handles.  If
MMP is enabled, this causes both handles to share MMP buffers, which is bad
news when it comes time to free both handles.  Change the code to (we hope) fix
this.  This prevents resize2fs from failing with a double-free error when
handed a MMP filesystem.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
resize2fs: use blk64_t and location getters for free_gdp_blocks()

free_gdp_blocks needs to be taught to use 64-bit fields and the appropriate
getters, otherwise it'll truncate high block numbers (when, say, resizing a
>16T fs) and mark the low numbered group descriptor blocks as free.  Yikes.

Reported-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
resize2fs: fix interior extent node corruption

If we have an extent tree like this (from debuge2fs's "ex" command):

Level Entries       Logical            Physical Length Flags
...
 2/ 2  60/ 63 13096 - 13117 650024 - 650045     22
 2/ 2  61/ 63 13134 - 13142 650062 - 650070      9
 2/ 2  62/ 63 13193 - 13194 650121 - 650122      2
 2/ 2  63/ 63 13227 - 13227 650155 - 650155      1 A)
 1/ 2   4/ 14 13228 - 17108 655367            3881 B)
 2/ 2   1/117 13228 - 13251 650156 - 650179     24 C)
 2/ 2   2/117 13275 - 13287 650203 - 650215     13
 2/ 2   3/117 13348 - 13353 650276 - 650281      6
...

and we resize the fs in such a way that all of those blocks must
be moved down, we do them one at a time.  Eventually we move 1-block
extent A) to a lower block, and then follow it with the other
blocks in the next logical offsets from extent C) in the next
interior node B).

The userspace extent code tries to merge, so when it finds that
logical 13228 can be merged with logical 13227 into a single extent,
it does.  And so on, all through extent C), up to block 13250 (why
not 13251?  [1]), and eventually move the node block as well.
So we end up with this when all the blocks are moved post-resize:

Level Entries       Logical            Physical Length Flags
...
 2/ 2 120/122 13193 - 13193  33220 -  33220      1
 2/ 2 121/122 13194 - 13194  33221 -  33221      1
 2/ 2 122/122 13227 - 13250  33222 -  33245     24 D)
 1/ 2   5/ 19 13228 - 17108  34676            3881 E) ***
 2/ 2   1/222 13251 - 13251  33246 -  33246      1 F)
 2/ 2   2/222 13275 - 13286  33247 -  33258     12
...

All those adjacent blocks got moved into extent D), which is nice -
but the next interior node E) was never updated to reflect its new
starting point - it says the leaf extents beneath it start at 13228,
when in fact they start at 13251.

So as we move blocks one by one out of original extent C) above, we
need to keep updating C)'s parent node B) for a proper starting point.
fix_parents() does this.

Once the tree is corrupted like this, more corruption can
ensue post-resize, because we traverse the tree by interior nodes,
relying on their start block to know where we are in the tree.
If it gets off, we'll end up inserting blocks into the wrong part
of the tree, etc.

I have a testcase using fsx to create a complex extent tree which
is then moved during resize; it hit this corruption quite easily,
and with this fix, it succeeds.

Note the first hunk in the commit is for going the other way,
moving the last block of an extent to the extent after it; this
needs the same sort of fix-up, although I haven't seen it in
practice.

[1] We leave the last block because a single-block extent is its
own case, and there is no merging code in that case.  \o/

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
libext2fs: fix a coding style for EXT2_NO_MTAB_FILE

When we define an error in lib/ext2fs/ext2_err.et.in, we will always use
EXT2_ET_* prefix for a new error.  But EXT2_NO_MTAB_FILE doesn't obey
this rule.  So fix it.

Signed-off-by: Zheng Liu <wenqing.lz@taobao.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
e2fsck: don't report uninit extents past EOF invalid

Commit d3f32c2db8 introduced a regression that caused e2fsck failures
in xfstests generic 013, 070, 083, 091, and 263.  Uninitialized
extents created by fallocate() at the end of file with the
FALLOC_FL_KEEP_SIZE flag were identified as invalid.  However,
because the file size is not increased when FALLOC_FL_KEEP_SIZE is
used, uninitialized extents can correctly contain blocks located past
the end of file.

Fix this by filtering out possible invalid extents if they are
uninitialized and extend past the block containing the end of file.

Signed-off-by: Eric Whitney <enwlinux@gmail.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
tests: add another test for uninit extents past eof

Commit d3f32c2db8 was intended to detect extents found outside their
proper location in the extent tree, including invalid extents at the
end of an extent block.  However, it incorrectly reported legal
uninitialized extents created by fallocate() at the end of file with
the FALLOC_FL_KEEP_SIZE flag as false positives.  xfstests
generic/263 (among others) caught this problem, while the e2fsprogs
test f_uninit_ext_past_eof did not.  The latter test failed to
detect the problem in part because it uses a test file whose i_size
is 0.

Add a test derived from the fsx-based test case in xfstests
generic/263 consisting of a file with non-zero length, more than
four extents total, and two uninitialized extents past EOF to
reliably reproduce commit d3f32c2db8's false positive behavior.

Signed-off-by: Eric Whitney <enwlinux@gmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
resize2fs: add debugging support for resize2fs -M calcuations

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
resize2fs: fix -M size calculations to avoid cutting off the inode table

If the file system's inode table blocks in the last block group are
located in the middle or the end of the block group, it's possible for
resize2fs -M to use a size which will require relocating the inode
table blocks in the last block group.  This can lead to all sorts of
problems, so solve it by simply guaranteeing that we will never do
that.

Reported-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
resize2fs: relocate inode table blocks if necessary when shrinking

If the file system is being shrunk, and a block group's inode table
falls beyond the end of the inode table, we need to try to relocate
the inode table blocks.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
tests: add test for resize2fs -M with inode table in middle of block group

Eric Sandeen reported that Fedora's mke2fs when compiled for ppc was
creating a file system which caused problems with resize2fs -M.
Closer examination showed that the problem was file system which
looked like this:

Filesystem features:      ext_attr dir_index filetype sparse_super
Inode count:              512
Block count:              1247
   ...

Group 0: (Blocks 1-1024)
  Primary superblock at 1, Group descriptors at 2-2
  Block bitmap at 66 (+65), Inode bitmap at 67 (+66)
  Inode table at 68-99 (+67)

Group 1: (Blocks 1025-1246)
  Backup superblock at 1025, Group descriptors at 1026-1026
  Block bitmap at 1090 (+65), Inode bitmap at 1091 (+66)
  Inode table at 1092-1123 (+67)

It's not obvious to me why Fedora's ppc mke2fs is creating file
systems like this (I can't reproduce this on debian ppc systems), but
resize2fs -M should be able to deal with such file systems, which is
what this test is designed to check.

Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
libext2fs: only link an inode into a directory once

The ext2fs_link helper function link_proc does not check the value of
ls->done, which means that if the function finds multiple empty spaces
that will fit the new directory entry, it will create a directory
entry in each of the spaces.  Instead of doing that, check the done
value and don't do anything more if we've already added the directory
entry.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
libext2fs: set the large_file feature flag when setting i_size > 2GB

If someone tries to write a file that is larger than 2GB, we need to
set the large_file feature flag to affirm that i_size_hi can hold
meaningful contents.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
libext2fs: fix a minor grammatical error in the error catalog

'an block' should be 'a block'.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>