Rudimentary support for --one-file-system (-x) option

Handle this by recording st_dev and making sure we don't cross into any
files on a different device.

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Add option to turn off extent lookup during csum phase.

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Merge branch 'one-file-system'

Conflicts:
	duperemove.c

Fixed minor merge conflict in parse_options().

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Pretty print fiemap flags in filerec-test.

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Rename filerec-test target to show-shared-extents.

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
The extent search status bar never showed 100% completion - we needed to
send one last update from find_all_dups.

The bar updates are choppy. This is probably because we update on each
bucket, and the hash buckets can be various sizes. Update instead on the
total number of blocks hashed. This doesn't fix the choppiness completely,
but does smooth it out a bit more.

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Don't leak directory fd in walk_dir()

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Abort if we try to open a file more than once via filerec_open().

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Abort if we try to leave dedupe_extent_list() with a non-empty open files
list.

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Fix too many file descriptors error during dedupe

new_dedupe_ctxt() wasn't properly limiting the number of files we would try
to dedupe at a time. As a result, large lists of duplicate extents would
queue too many files per pass and get EMFILE from filerec_open().

Fix this by setting the correct value of ctxt->max_queable.

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Print a stack trace when we abort()

Should make it a little easier to figure out where a crash came from.

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Move memstat tracking code memstats.c

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Make the Makefile less sad

This isn't perfect, but much better than before:

- separated out each program cfiles

- make objects variable via substition

- use implicit rule (.c.o) to build objects

- put headers into their own variable

- depend the whole thing on the headers define.
  - this is inefficient but i will provide a better way soon
  - in the meantime, changing a header doesn't require a manual
    'make clean'. This is far more 'modern' behavior.

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
dedupe_extent_list() cleanups

- move code to open file + put on list into filerec_open_once()

- use list_is_last to detect end of list rather than count

- assert on some end-cases that shouldn't happen

Code flow is still tricky but hopefully more readable at least now.

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
dedupe_extent_list() fixes

We could exit the loop with a live context if there was exactly
MAX_DEDUPES_PER_IOCTL + 1 extents. The context gets cleaned up but that
means we might not dedupe that extent. Fix this by adding a previous extent
to the context and forcing a dedupe in that situation.

We also weren't bubbling up ENOMEM in case of error from
add_extent_to_dedupe().

Also, don't try dedupe if we ever only got one extent to succesfully be
added.

This patch also adds some asserts to help catch loop errors in the future.

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Print the correct extent filename in case of error adding it to dedupe
context.

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Import unlikely() from linux kernel source and use it for abort_on().

This is the only place where unlikely makes sense in the current code.

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Merge branch 'dedupe_extent_list_fixes'
Obey --lookup-extents when loading from a hashfile.

Normally the block flags aren't set if --lookup-extents=no is specified, so
the block flags check always comes out negative. However, we'll get the
block flags set if the user loads from a hashfile. To fix this, explicitely
ignore them during the extent search.

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Added xxhash algorithm

Signed-off-by: Timofey Titovets <nefelim4ag@gmail.com>