add file filtering + duplicate file handling