racai-andrei commited on
Commit
568ad9d
1 Parent(s): 875f10a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +14 -0
README.md CHANGED
@@ -10,6 +10,9 @@ datasets:
10
 
11
  This repository contains the uncased Romanian DistilBERT (named Distil-BERT-base-ro in the paper). The teacher model used for distillation is: [dumitrescustefan/bert-base-romanian-cased-v1](https://huggingface.co/dumitrescustefan/bert-base-romanian-cased-v1).
12
 
 
 
 
13
  ## Usage
14
 
15
  ```python
@@ -53,3 +56,14 @@ We evaluated the model in comparison with its teacher on 5 Romanian tasks:
53
  |--------------------------------|:----:|:----:|:---:|:----:|:---:|:--:|:---:|
54
  | bert-base-romanian-cased-v1 | 98.00 | 96.46 | 85.88 | 98.07 | 79.61 | 95.58 | 79.11 |
55
  | distilbert-base-romanian-cased | 97.97 | 97.08 | 83.35 | 98.40 | 83.01 | 96.31 | 80.57 |
 
 
 
 
 
 
 
 
 
 
 
 
10
 
11
  This repository contains the uncased Romanian DistilBERT (named Distil-BERT-base-ro in the paper). The teacher model used for distillation is: [dumitrescustefan/bert-base-romanian-cased-v1](https://huggingface.co/dumitrescustefan/bert-base-romanian-cased-v1).
12
 
13
+ The model was introduced in [this paper](https://arxiv.org/abs/2112.12650). The adjacent code can be found
14
+ [here](https://github.com/racai-ai/Romanian-DistilBERT).
15
+
16
  ## Usage
17
 
18
  ```python
 
56
  |--------------------------------|:----:|:----:|:---:|:----:|:---:|:--:|:---:|
57
  | bert-base-romanian-cased-v1 | 98.00 | 96.46 | 85.88 | 98.07 | 79.61 | 95.58 | 79.11 |
58
  | distilbert-base-romanian-cased | 97.97 | 97.08 | 83.35 | 98.40 | 83.01 | 96.31 | 80.57 |
59
+
60
+ ### BibTeX entry and citation info
61
+ ```bibtex
62
+ @article{avram2021distilling,
63
+ title={Distilling the Knowledge of Romanian BERTs Using Multiple Teachers},
64
+ author={Andrei-Marius Avram and Darius Catrina and Dumitru-Clementin Cercel and Mihai Dascălu and Traian Rebedea and Vasile Păiş and Dan Tufiş},
65
+ journal={ArXiv},
66
+ year={2021},
67
+ volume={abs/2112.12650}
68
+ }
69
+ ```