works for case when duplicates are in the same file