hjian42 commited on
Commit
1616a20
1 Parent(s): 6cda070

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -0
README.md CHANGED
@@ -8,6 +8,7 @@ license: cc-by-nc-4.0
8
  - In the paper, it is referred as `HuggingFace-BERTweet (TB2+EWT)` in the POS table.
9
 
10
  ## How to use the model
 
11
 
12
  ```python
13
  from transformers import AutoTokenizer, AutoModelForTokenClassification
 
8
  - In the paper, it is referred as `HuggingFace-BERTweet (TB2+EWT)` in the POS table.
9
 
10
  ## How to use the model
11
+ - **PRE-PROCESSING**: when you apply the model on tweets, please make sure that tweets are preprocessed by the [TweetTokenizer](https://github.com/VinAIResearch/BERTweet/blob/master/TweetNormalizer.py) to get the best performance.
12
 
13
  ```python
14
  from transformers import AutoTokenizer, AutoModelForTokenClassification