prajjwal1 commited on
Commit
f0dc457
1 Parent(s): 008ccf2

new readme

Browse files
Files changed (1) hide show
  1. README.md +48 -9
README.md CHANGED
@@ -1,5 +1,24 @@
1
- The following model is a Pytorch pre-trained model obtained from converting Tensorflow checkpoint found in the [official Google BERT repository](https://github.com/google-research/bert). These BERT variants were introduced in the paper [Well-Read Students Learn Better: On the Importance of Pre-training Compact Models](https://arxiv.org/abs/1908.08962). These models are supposed to be trained on a downstream task.
2
- If you use the model, please consider citing the paper
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ```
4
  @misc{bhargava2021generalization,
5
  title={Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics},
@@ -9,13 +28,33 @@ If you use the model, please consider citing the paper
9
  archivePrefix={arXiv},
10
  primaryClass={cs.CL}
11
  }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
  ```
13
- Original Implementation and more info can be found in [this Github repository](https://github.com/prajjwal1/generalize_lm_nli).
14
 
15
- You can check out:
16
- - `prajjwal1/bert-tiny` (L=2, H=128)
17
- - `prajjwal1/bert-mini` (L=4, H=256)
18
- - `prajjwal1/bert-small` (L=4, H=512)
19
- - `prajjwal1/bert-medium` (L=8, H=512)
 
 
20
 
21
- [@prajjwal_1](https://twitter.com/prajjwal_1)
 
1
+ ---
2
+ language:
3
+ - en
4
+
5
+ license:
6
+ - mit
7
+
8
+ tags:
9
+ - BERT
10
+ - MNLI
11
+ - NLI
12
+ - transformer
13
+ - pre-training
14
+
15
+ ---
16
+
17
+ The following model is a Pytorch pre-trained model obtained from converting Tensorflow checkpoint found in the [official Google BERT repository](https://github.com/google-research/bert).
18
+
19
+ This is one of the smaller pre-trained BERT variants, together with [bert-mini](https://huggingface.co/prajjwal1/bert-mini), [bert-tiny](https://huggingface.co/prajjwal1/bert-tiny), [bert-small](https://huggingface.co/prajjwal1/bert-small) and [bert-medium](https://huggingface.co/prajjwal1/bert-medium). They were introduced in the study [Well-Read Students Learn Better: On the Importance of Pre-training Compact Models](https://arxiv.org/abs/1908.08962), and ported to HF for the study [Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics](https://arxiv.org/abs/2110.01518). These models are supposed to be trained on a downstream task.
20
+
21
+ If you use the model, please consider citing both the papers:
22
  ```
23
  @misc{bhargava2021generalization,
24
  title={Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics},
 
28
  archivePrefix={arXiv},
29
  primaryClass={cs.CL}
30
  }
31
+
32
+ @article{DBLP:journals/corr/abs-1908-08962,
33
+ author = {Iulia Turc and
34
+ Ming{-}Wei Chang and
35
+ Kenton Lee and
36
+ Kristina Toutanova},
37
+ title = {Well-Read Students Learn Better: The Impact of Student Initialization
38
+ on Knowledge Distillation},
39
+ journal = {CoRR},
40
+ volume = {abs/1908.08962},
41
+ year = {2019},
42
+ url = {http://arxiv.org/abs/1908.08962},
43
+ eprinttype = {arXiv},
44
+ eprint = {1908.08962},
45
+ timestamp = {Thu, 29 Aug 2019 16:32:34 +0200},
46
+ biburl = {https://dblp.org/rec/journals/corr/abs-1908-08962.bib},
47
+ bibsource = {dblp computer science bibliography, https://dblp.org}
48
+ }
49
+
50
  ```
 
51
 
52
+ Other models to check out:
53
+ - `prajjwal1/bert-tiny` (L=2, H=128) [Model Link](https://huggingface.co/prajjwal1/bert-tiny)
54
+ - `prajjwal1/bert-mini` (L=4, H=256) [Model Link](https://huggingface.co/prajjwal1/bert-mini)
55
+ - `prajjwal1/bert-small` (L=4, H=512) [Model Link](https://huggingface.co/prajjwal1/bert-small)
56
+ - `prajjwal1/bert-medium` (L=8, H=512) [Model Link](https://huggingface.co/prajjwal1/bert-medium)
57
+
58
+ Original Implementation and more info can be found in [this Github repository](https://github.com/prajjwal1/generalize_lm_nli).
59
 
60
+ Twitter: [@prajjwal_1](https://twitter.com/prajjwal_1)