added support for duplicate entries within a single dataset and url generation