Cleanup for_each_dupe

No actual functionality is changed. for_each_dupe() is moved into
duperemove.c with the rest of the extent finding code. The function callback
pointer is removed in favor of just calling walk_dupe_block directly. As a
result, we no longer need the dupe_walk_ctxt so that is unraveled. Naming of
filerec and block variables in these functions are changed to be more
consistent.

This makes reading through the extent finding code far easier.

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Keep file blocks on per blocklist/filerec list

We have a performance issue when visiting very large buckets - walking
hundreds of thousands of hashes can take a lot of time. Previously
this was solved by pushing large buckets to an alternative algorithm. This had
a couple problems, the most obvious being how to define a 'very large
bucket'. It also didn't attempt solve the underlying problem.

A better solution is to have each dupe_blocks_list maintain a tree of
per-filerec hash blocks. The extent finding code then can walk the
per-filerec list of blocks, thus avoiding visiting any nodes that it
won't care about.

This is also much faster than the previous solution. The time for an
extent search from hashfile of my home directory (~750 gigs) dropped
from 55 minutes to 22 minutes.

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Print a status bar during our extent search. Width of the bar is somewhat
arbitrary based on what I thought looked decent. I also have duperemove
print more information on the total number of hash buckets that will need to
be visited.

Signed-off-by: Mark Fasheh <mfasheh@suse.de>