duplicate rows are handled by first deleting the duplicate record