wyu1 commited on
Commit
5af0b57
1 Parent(s): 968c177

update readme

Browse files
Files changed (1) hide show
  1. README.md +2 -5
README.md CHANGED
@@ -5,20 +5,17 @@ license: cc-by-4.0
5
  # DictBERT model (uncased)
6
 
7
  -- This is the model checkpoint of our [ACL 2022](https://www.2022.aclweb.org/) paper "*Dict-BERT: Enhancing Language Model Pre-training with Dictionary*" [\[PDF\]](https://aclanthology.org/2022.findings-acl.150/).
8
- In this paper, we propose DictBERT, a novel pre-trained language model by leveraging rare word definitions in English dictionaries (e.g., Wiktionary). DictBERT is based on the BERT architecture, trained under the same setting as BERT. Please refer more details in our paper.
9
 
10
  ## Evaluation results
11
 
12
- When fine-tuned DictBERT on downstream tasks, this model achieves the following results:
13
-
14
- CoLA is evaluated by matthews; STS-B is evaluated by pearson; others are evaluated by accuracy.
15
 
16
  | | MNLI | QNLI | QQP | SST-2 | CoLA | MRPC | RTE | STS-B | Average |
17
  |:----:|:-----------:|:----:|:----:|:-----:|:----:|:-----:|:----:|:----:|:-------:|
18
  | BERT(HF) | 84.12 | 90.69 | 90.75 | 92.52 | 58.89 | 86.17 | 68.67 | 89.39 | 82.65 |
19
  | DictBERT | 84.36 | 91.02 | 90.78 | 92.43 | 61.81 | 87.25 | 72.90 | 89.40 | 83.74 |
20
 
21
-
22
  HF: huggingface checkpoint for BERT-base uncased
23
 
24
  ### BibTeX entry and citation info
 
5
  # DictBERT model (uncased)
6
 
7
  -- This is the model checkpoint of our [ACL 2022](https://www.2022.aclweb.org/) paper "*Dict-BERT: Enhancing Language Model Pre-training with Dictionary*" [\[PDF\]](https://aclanthology.org/2022.findings-acl.150/).
8
+ In this paper, we propose DictBERT, which is a novel pre-trained language model by leveraging rare word definitions in English dictionaries (e.g., Wiktionary). DictBERT is based on the BERT architecture, trained under the same setting as BERT. Please refer more details in our paper.
9
 
10
  ## Evaluation results
11
 
12
+ When fine-tuned BERT and our DictBERT on GLEU benchmarks tasks. CoLA is evaluated by matthews, STS-B is evaluated by pearson, and others are evaluated by accuracy. DictBERT model achieves the following results:
 
 
13
 
14
  | | MNLI | QNLI | QQP | SST-2 | CoLA | MRPC | RTE | STS-B | Average |
15
  |:----:|:-----------:|:----:|:----:|:-----:|:----:|:-----:|:----:|:----:|:-------:|
16
  | BERT(HF) | 84.12 | 90.69 | 90.75 | 92.52 | 58.89 | 86.17 | 68.67 | 89.39 | 82.65 |
17
  | DictBERT | 84.36 | 91.02 | 90.78 | 92.43 | 61.81 | 87.25 | 72.90 | 89.40 | 83.74 |
18
 
 
19
  HF: huggingface checkpoint for BERT-base uncased
20
 
21
  ### BibTeX entry and citation info