racai-andrei commited on
Commit
bc5520e
1 Parent(s): 8e3369a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +14 -0
README.md CHANGED
@@ -11,6 +11,9 @@ datasets:
11
 
12
  This repository contains the uncased Romanian DistilBERT (named Distil-RoBERT-base in the paper). The teacher model used for distillation is: [readerbench/RoBERT-base](https://huggingface.co/readerbench/RoBERT-base).
13
 
 
 
 
14
  ## Usage
15
 
16
  ```python
@@ -54,3 +57,14 @@ We evaluated the model in comparison with the RoBERT-base on 5 Romanian tasks:
54
  |--------------------------------|:----:|:----:|:---:|:----:|:---:|:--:|:---:|
55
  | RoBERT-base | 98.02 | 97.15 | 85.14 | 98.30 | 79.40 | 96.07 | 81.18 |
56
  | distilbert-base-romanian-uncased | 97.12 | 95.79 | 83.11 | 98.01 | 79.58 | 96.11 | 79.80 |
 
 
 
 
 
 
 
 
 
 
 
 
11
 
12
  This repository contains the uncased Romanian DistilBERT (named Distil-RoBERT-base in the paper). The teacher model used for distillation is: [readerbench/RoBERT-base](https://huggingface.co/readerbench/RoBERT-base).
13
 
14
+ The model was introduced in [this paper](https://arxiv.org/abs/2112.12650). The adjacent code can be found
15
+ [here](https://github.com/racai-ai/Romanian-DistilBERT).
16
+
17
  ## Usage
18
 
19
  ```python
 
57
  |--------------------------------|:----:|:----:|:---:|:----:|:---:|:--:|:---:|
58
  | RoBERT-base | 98.02 | 97.15 | 85.14 | 98.30 | 79.40 | 96.07 | 81.18 |
59
  | distilbert-base-romanian-uncased | 97.12 | 95.79 | 83.11 | 98.01 | 79.58 | 96.11 | 79.80 |
60
+
61
+ ### BibTeX entry and citation info
62
+ ```bibtex
63
+ @article{avram2021distilling,
64
+ title={Distilling the Knowledge of Romanian BERTs Using Multiple Teachers},
65
+ author={Andrei-Marius Avram and Darius Catrina and Dumitru-Clementin Cercel and Mihai Dascălu and Traian Rebedea and Vasile Păiş and Dan Tufiş},
66
+ journal={ArXiv},
67
+ year={2021},
68
+ volume={abs/2112.12650}
69
+ }
70
+ ```