filter for only files which are files and readable, don't throw exception when file is returned which turns out not to be a file anyway (could have changed in the meantime), use MD5 instead of SHA1 for increased hashing speed
strip trailing spaces and organize imports
handle errors calculating hashes and unexpected special files by ignoring them
don't include non-file files or files causing errors in file listing