added Id shingling
adjusted tokenization