changed vectorization behaviour of tokenize