add dunplicate filter function