btrfs-progs: Add minimum device size check

Btrfs has global block reservation, so even mkfs.btrfs can execute
without problem, there is still a possibility that the filesystem can't
be mounted.
For example when mkfs.btrfs on a 8M file on x86_64 platform, kernel will
refuse to mount due to ENOSPC, since system block group takes 4M and
mixed block group takes 4M, and global block reservation will takes all
the 4M from mixed block group, which makes btrfs unable to create uuid
tree.

This patch will add minimum device size check before actually mkfs.
The minimum size calculation uses a simplified one:
minimum_size_for_each_dev = 2 * (system block group + global block rsv)
and global block rsv = leafsize << 10

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
btrfs-progs: add supported attr flags to btrfs(5)

The chattr(1) manpage suffers from the same problems mount(1)
had: many options listed, not kept up to date for various
filesystems.

I've submitted a manpage update for chattr(1) which says to
refer to filesystem-specific manpages for supported attributes;
this patch updates btrfs(5) to list the attributes supported
by btrfs.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
[added some asciidoc markups, adjusted formatting]
Signed-off-by: David Sterba <dsterba@suse.cz>
btrfs-progs: replace BTRFS_NUM_MIRRORS with BTRFS_MAX_MIRRORS

The chunk-recover.c/BTRFS_NUM_MIRRORS in the userspace means
the same thing as ctree.h/BTRFS_MAX_MIRRORS in the kernelspace,
so to stay consistent with the kernelspace, just make this movement
in the userspace:
	chunk-recover.c/BTRFS_NUM_MIRRORS
		===>
	ctree.h/BTRFS_MAX_MIRRORS

This provides convenience for future use.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
btrfs-progs: deal with malloc failure in btrfs-image

Handle the malloc failure for dump_worker in the same way as
the restore worker.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
btrfs-progs: cleanup possible silent failure in btrfs-image

If the malloc above fails, the btrfs-image will exit directly
without any error messages.
Now just return the ENOMEM errno and let the caller prompt the
error message.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
btrfs-progs: use BTRFS_SUPER_INFO_SIZE to replace raw 4096 in btrfs-image

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
btrfs-progs: Avoid double-free of fs_devices->list

I found the following patch is insufficient.

===============================================================================
commit 6e6b32ddf58db54f714d0f263c2589f4859e8b5e
Author: Adam Buchbinder <abuchbinder@google.com>
Date:   Fri Jun 13 16:43:56 2014 -0700

    btrfs-progs: Fix a use-after-free in the volumes code.
===============================================================================

"btrfs filesystem show <dev>" with this patch causes segmentation fault
if "<dev>" is a not-mounted Btrfs filesystem.

===============================================================================
Label: none  uuid: <cut here>
        Total devices 1 FS bytes used 112.00KiB
        devid    1 size 59.12GiB used 2.04GiB path /dev/sdd1

Segmentation fault (core dumped)
===============================================================================

It's due to double-free of fs_devices->list as follows.

===============================================================================
cmd_show
  -> list_del(&fs_devices->list)     # 1st one.
  -> btrfs_close_devices(fs_devices)
    -> list_del(&fs_devices->list)   # <- 2nd one introduced at 6e6b32dd.
                                          Double-free happens here.
===============================================================================

First list_del() can safely be removed because fs_devices->list will be
deleted by second one, soon.

Signed-off-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com>
Cc: Adam Buchbinder <abuchbinder@google.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
btrfs-progs: fix btrfs-image old_restore fsck failure

Steps to reproduce:
	# mkfs.btrfs -f <dev1>
	# btrfs-image <dev1> <image_file>
	# btrfs-image -r -o <image_file> <dev2>
	# btrfs check <dev2>

btrfs check output:
	: read block failed check_tree_block
	: Couldn't read tree root
	: Couldn't open file system

The btrfs-image should not mess with the chunk tree under the old_restore way.

The new restore way was introduced by:
	commit d6f7e3da0dae7b60cb7565f8a47c3b9045c52d1d
	Btrfs-progs: make btrfs-image restore with a valid chunk tree V2
	...
And the following commit enhanced the new restore on the valid chunk tree
building stuff:
	commit ef2a8889ef813ba77061f6a92f4954d047a78932
	Btrfs-progs: make image restore with the original device offsets
	...

But the second commit should not effect the old_restore way since the
old_restore way doesn't try to build a valid chunk tree at all.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Btrfs-progs: fsck: reduce memory usage for extents check

Steps to reproduce:

 # mkfs.btrfs -f /dev/sda9 -b 2g
 # mount /dev/sda9 /mnt
 # dd if=/dev/zero of=/mnt/data bs=4k oflag=direct
 # btrfs file df /mnt

    Data, single: total=1.66GiB, used=1.66GiB
    System, single: total=4.00MiB, used=16.00KiB
    Metadata, single: total=200.00MiB, used=67.88MiB

For a filesystem without snapshots, 70M metadata, extent
checking eats max memory about 110M, this is a nightmare
for some system with low memory.

It is very likely that extent record can be freed quickly
for a filesystem without snapshots, improve this by trying
if it can free memory after adding data/tree backrefs.

This patch reduces max memory cost from 110M to 40M for
extents checking for the above case.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
btrfs-progs: cleanup unnecessary free if malloc fails in btrfs-image

Don't bother free the buffer if the malloc failed.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
btrfs-progs: add X inode flag to btrfs(5)

After the discussion in
http://thread.gmane.org/gmane.comp.file-systems.btrfs/36334

the 'X' will be mentioned in the manpage because new e2fsprogs/lsattr
will display it and represents the NOCOMPRESS bit of an inode.

Signed-off-by: David Sterba <dsterba@suse.cz>
Btrfs-progs: fsck: deal with snapshot one by one when rebuilding extent tree

Previously, we deal with node block firstly and then leaf block which can
maximize readahead. However, to rebuild extent tree, we need deal with snapshot
one by one.

This patch makes us deal with snapshot one by one if we need rebuild extent
tree otherwise we drop into previous way.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Btrfs-progs: fsck: add ability to rebuild extent tree with snapshots

This patch makes us to rebuild a really corrupt extent tree with snapshots.
To implement this, we have to verify whether a block is FULL BACKREF.

This idea come from Josef Bacik:

1) We walk down the original tree, every eb we encounter has
btrfs_header_owner(eb) == root->objectid.  We add normal references
for this root (BTRFS_TREE_BLOCK_REF_KEY) for this root.  World peace
is achieved.

2) We walk down the snapshotted tree.  Say we didn't change anything
at all, it was just a clean snapshot and then boom.  So the
btrfs_header_owner(root->node) == root->objectid, so normal backref.
We walk down to the next level, where btrfs_header_owner(eb) !=
root->objectid, but the level above did, so we add normal refs for all
of these blocks.  We go down the next level, now our
btrfs_header_owner(parent) != root->objectid and
btrfs_header_owner(eb) != root->objectid.  This is where we need to
now go back and see if btrfs_header_owner(eb) currently has a ref on
eb.  If it does we are done, move on to the next block in this same
level, we don't have to go further down.

3) Harder case, we snapshotted and then changed things in the original
root.  Do the same thing as in step 2, but now we get down to
btrfs_header_owner(eb) != root->objectid && btrfs_header_owner(parent)
!= root->objectid.  We lookup the references we have for eb and notice
that btrfs_header_owner(eb) no longer refers to eb.  So now we must
set FULL_BACKREF on this extent reference and add a
SHARED_BLOCK_REF_KEY for this eb using the parent->start as the
offset.  And we need to keep walking down and doing the same thing
until we either hit level 0 or btrfs_header_owner(eb) has a ref on the
block.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Btrfs-progs: fsck: clear out log tree in repair mode

Repair mode will commit transaction which will make us
fail to load log tree anymore.

Give a warning to common users, if they really want to
coninue, we will clear out log tree.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Btrfs-progs: fsck: avoid pinning same block several times

This can not only give some speedups but also avoid forever loop
with a really broken filesystem.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Btrfs-progs: fsck: add ability to check reloc roots

When encountering system crash or balance enospc errors,
there maybe still some reloc roots left.

The way we store reloc root is different from fs root:

reloc root's root key(BTRFS_RELOC_TREE_OBJECTID, ROOT_ITEM, objectid)
fs root's root key(objectid, ROOT_ITEM, -1)
reloc data's root key(BTRFS_DATA_RELOC_TREE_OBJECTID, ROOT_ITEM, 0)

So this patch use right key to search corresponding root node, and
avoid using normal fs root cache for reloc roots.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Btrfs-progs: check, fix csum check in the presence of non-inlined refs

When we have non-inlined extent references, we were failing to find the
corresponding extent item for an existing csum item in the csum tree.

Reproducer:

   mkfs.btrfs -f /dev/sdd
   mount /dev/sdd /mnt

   xfs_io -f -c "falloc 780366 135302" /mnt/foo
   xfs_io -c "falloc 327680 151552" /mnt/foo
   xfs_io -c "pwrite -S 0xff -b 131072 0 131072" /mnt/foo
   sync

   for i in `seq 1 40`; do btrfs subvolume snapshot /mnt /mnt/snap$i ; done
   umount /mnt

   btrfs check /dev/sdd

The check command exited with status 1 and the following output:

   Checking filesystem on /dev/sdd
   UUID: 2416ab5f-9d71-457e-bb13-a27d4f6b399a
   checking extents
   checking free space cache
   checking fs roots
   checking csums
   There are no extents for csum range 12980224-12984320
   Csum exists for 12980224-12984320 but there is no extent record
   found 1388544 bytes used err is 1
   total csum bytes: 132
   total tree bytes: 704512
   total fs tree bytes: 573440
   total extent tree bytes: 16384
   btree space waste bytes: 564479
   file data blocks allocated: 19341312
    referenced 14606336
   Btrfs v3.14.1-94-g80597e7

After this change it no longer erroneously reports a missing extent for the
csum item and exits with a status of 0.

Also added missing btrfs_prev_leaf() return value checks, as we were ignoring
errors and non-existence of left siblings completely.

Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
btrfs check: Pre-sort keys in a block while searching

When we think we might have a messed-up block with keys out of order
(e.g. during fsck), we still need to be able to find a key in the block.
To deal with this, we copy the keys, keeping track of where they came from
in the original node/leaf, sort them, and then do the binary search.

Signed-off-by: Hugo Mills <hugo@carfax.org.uk>
Signed-off-by: David Sterba <dsterba@suse.cz>
btrfs check: Attempt to fix misordered keys with bitflips in them

If someone has had bad RAM which has been used to store a metadata block,
there's a chance that one or more of the keys has had a bit changed. The
block checksum doesn't help us here, because it's made on the bad data.

To fix this, if there's a block with a bad key order in it, we find out-of-
order keys by bracketing them between the good keys either side of them,
and then attempting to flip each bit of the key in turn.

If precisely one of those bitflips puts the broken key back into order
relative to its two neighbours, we probably have a fix for the bitflip,
and so we write it back to the FS.

This doesn't repair bitflipped keys at the start or end of a metadata
block, nor bitflips in any other data structure.

Signed-off-by: Hugo Mills <hugo@carfax.org.uk>
Signed-off-by: David Sterba <dsterba@suse.cz>
Btrfs-progs: fsck: disallow partial opening if critical roots corrupted

If btrfs tree root is corrupted, fsck will hit the following segmentation.

enabling repair mode
Check tree block failed, want=29376512, have=0
Check tree block failed, want=29376512, have=0
Check tree block failed, want=29376512, have=0
Check tree block failed, want=29376512, have=0
Check tree block failed, want=29376512, have=0
read block failed check_tree_block
Couldn't read tree root
Checking filesystem on /dev/sda9
UUID: 0e1a754d-04a5-4256-ae79-0f769751803e
Critical roots corrupted, unable to fsck the FS
Segmentation fault (core dumped)

In btrfs_setup_all_roots(), we could tolerate some trees(extent tree, csum tree)
corrupted, and we have did careful check inside that function, it will
return NULL if critial roots corrupt(for example tree root).

The problem is that we check @OPEN_CTREE_PARTIAL flag again after
calling btrfs_setup_all_roots() which will successfully return
@fs_info though critial roots corrupted.

Fix this problem by removing @OPEN_CTREE_PARTIAL flag check outsize
btrfs_setup_all_roots().

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>