downsampling and dupchecking added