Question: why tokenization is important in nlp?
Answer: In order to get our computer to understand any text, we need to break that word down in a way that our machine can understand. That's where the concept of tokenization in Natural Language Processing (NLP) comes in. Simply put, we can't work with text data if we don't perform tokenization.