add duplicate share filter
improve peformance of duplicate share filter