doc2term (pypi): A fast NLP tokenizer that detects tokens and remove duplications and punctuations
Keywords: tokenizer|NLP|punctuation|standarization|duplicate-detection|text-processing|text-tokenizing|doc2term|nlp|nlp-library