dqnguyen commited on
Commit
f9365bd
1 Parent(s): 0eedbed

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -59
README.md CHANGED
@@ -26,62 +26,3 @@ For further information or requests, please go to [BERTweet's homepage](https://
26
  <img width="275" alt="irony" src="https://user-images.githubusercontent.com/2412555/135724595-15f4f2c8-bbb6-4ee6-82a0-034769dec183.png" />
27
  </p>
28
 
29
- ### <a name="models2"></a> Pre-trained models
30
-
31
-
32
- Model | #params | Arch. | Pre-training data
33
- ---|---|---|---
34
- `vinai/bertweet-base` | 135M | base | 850M English Tweets (cased)
35
- `vinai/bertweet-covid19-base-cased` | 135M | base | 23M COVID-19 English Tweets (cased)
36
- `vinai/bertweet-covid19-base-uncased` | 135M | base | 23M COVID-19 English Tweets (uncased)
37
- `vinai/bertweet-large` | 355M | large | 873M English Tweets (cased)
38
-
39
- ### <a name="usage2"></a> Example usage
40
-
41
-
42
- ```python
43
- import torch
44
- from transformers import AutoModel, AutoTokenizer
45
-
46
- bertweet = AutoModel.from_pretrained("vinai/bertweet-base")
47
-
48
- # For transformers v4.x+:
49
- tokenizer = AutoTokenizer.from_pretrained("vinai/bertweet-base", use_fast=False)
50
-
51
- # For transformers v3.x:
52
- # tokenizer = AutoTokenizer.from_pretrained("vinai/bertweet-base")
53
-
54
- # INPUT TWEET IS ALREADY NORMALIZED!
55
- line = "SC has first two presumptive cases of coronavirus , DHEC confirms HTTPURL via @USER :crying_face:"
56
-
57
- input_ids = torch.tensor([tokenizer.encode(line)])
58
-
59
- with torch.no_grad():
60
- features = bertweet(input_ids) # Models outputs are now tuples
61
-
62
- ## With TensorFlow 2.0+:
63
- # from transformers import TFAutoModel
64
- # bertweet = TFAutoModel.from_pretrained("vinai/bertweet-base")
65
- ```
66
-
67
- ### <a name="preprocess"></a> Normalize raw input Tweets
68
-
69
- Before applying `fastBPE` to the pre-training corpus of 850M English Tweets, we tokenized these Tweets using `TweetTokenizer` from the NLTK toolkit and used the `emoji` package to translate emotion icons into text strings (here, each icon is referred to as a word token). We also normalized the Tweets by converting user mentions and web/url links into special tokens `@USER` and `HTTPURL`, respectively. Thus it is recommended to also apply the same pre-processing step for BERTweet-based downstream applications w.r.t. the raw input Tweets. BERTweet provides this pre-processing step by enabling the `normalization` argument. This argument currently only supports models "`vinai/bertweet-base`", "`vinai/bertweet-covid19-base-cased`" and "`vinai/bertweet-covid19-base-uncased`".
70
-
71
- - Install `emoji`: `pip3 install emoji==0.6.0`
72
- - The `emoji` version must be either 0.5.4 or 0.6.0. Newer `emoji` versions have been updated to newer versions of the Emoji Charts, thus not consistent with the one used for pre-processing our pre-training Tweet corpus.
73
-
74
- ```python
75
- import torch
76
- from transformers import AutoTokenizer
77
-
78
- # Load the AutoTokenizer with a normalization mode if the input Tweet is raw
79
- tokenizer = AutoTokenizer.from_pretrained("vinai/bertweet-base", normalization=True)
80
-
81
- # from transformers import BertweetTokenizer
82
- # tokenizer = BertweetTokenizer.from_pretrained("vinai/bertweet-base", normalization=True)
83
-
84
- line = "SC has first two presumptive cases of coronavirus, DHEC confirms https://postandcourier.com/health/covid19/sc-has-first-two-presumptive-cases-of-coronavirus-dhec-confirms/article_bddfe4ae-5fd3-11ea-9ce4-5f495366cee6.html?utm_medium=social&utm_source=twitter&utm_campaign=user-share… via @postandcourier"
85
-
86
- input_ids = torch.tensor([tokenizer.encode(line)])
87
- ```
 
26
  <img width="275" alt="irony" src="https://user-images.githubusercontent.com/2412555/135724595-15f4f2c8-bbb6-4ee6-82a0-034769dec183.png" />
27
  </p>
28