adjusting initial database and removing duplicate characters on normalization