additional fixes when allowDuplicates is true