modified duplicate remover to run optimizer