YAML Metadata
Warning:
empty or missing yaml metadata in repo card
(https://huggingface.co/docs/hub/model-cards#model-card-metadata)
Text preprocessing
This tokenizer has been trained with tweets that have been preprocessed as follows:
- User mentions (@user_name) have been replaced with the word user.
- URLs have been replace with the word url.
- WIP. If you are going to use this tokenizer, we recommend you to preprocess your own dataset in the same manner.
- Downloads last month
- 0
Unable to determine this model's library. Check the
docs
.