tejr/checkem — pull request #1 by pruby (2012-02-10 00:35:48+00:00)
Title: Two fixes - changing directories making tests fail, missing identical files when there are other files of the same size

Directory structure used to test these:

$ ls -l test/
total 24
-rw-r--r-- 1 tim tim 4 Feb 10 12:56 a
-rw-r--r-- 1 tim tim 4 Feb 10 12:57 b
-rw-r--r-- 1 tim tim 4 Feb 10 12:57 c
-rw-r--r-- 1 tim tim 4 Feb 10 12:57 d
-rw-r--r-- 1 tim tim 7 Feb 10 12:57 e
-rw-r--r-- 1 tim tim 7 Feb 10 12:57 f

$ md5sum test/*
d3b07384d113edec49eaa6238ad5ff00 test/a
c157a79031e1c40f85931829bc5fc552 test/b
258622b1688250cb619f3c9ccaefb7eb test/c
d3b07384d113edec49eaa6238ad5ff00 test/d
ca1aedec008657bee2e6bbcc63e21053 test/e
ca1aedec008657bee2e6bbcc63e21053 test/f

Initial run with:

./checkem test/

failed due to File::Find changing directory in to test/ . The test within the "wanted" function was then equivalent to:

if (-f "test/a")
...

Which no longer works since we're inside test/

The first commit passes an option to File::Find to prevent it from changing the current directory.

The second issue is that after that fix it found that test/e and test/f are identical, but missed test/a and test/d being identical. These were being split up by test/b and test/c, which are the same size but with different content. Since test/a and test/d were not adjacent entries in the list, it didn't find them.

The second commit keeps a record of the MD5s of each file to avoid re-calculating them. It then stores an hash of files keyed by their MD5 digests and uses that to find if a file with that digest has already been seen.