added file validator (now it only removes duplicates)
changed interface
implemented file validator & file splitter