add performance info
Browse files
README.md
CHANGED
@@ -1,8 +1,23 @@
|
|
1 |
## BabyBERTA
|
2 |
|
3 |
-
|
|
|
|
|
4 |
It is intended for language acquisition research, on a single desktop with a single GPU - no high-performance computing infrastructure needed.
|
5 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
6 |
This model was trained by [Philip Huebner](https://philhuebner.com), currently at the [UIUC Language and Learning Lab](http://www.learninglanguagelab.org).
|
7 |
|
8 |
More info can be found [here](https://github.com/phueb/BabyBERTa).
|
|
|
1 |
## BabyBERTA
|
2 |
|
3 |
+
### Overview
|
4 |
+
|
5 |
+
BabyBERTA is a light-weight version of RoBERTa trained on 5M words of American-English child-directed input.
|
6 |
It is intended for language acquisition research, on a single desktop with a single GPU - no high-performance computing infrastructure needed.
|
7 |
|
8 |
+
### Performance
|
9 |
+
|
10 |
+
The provided model is the best-performing out of 10 that were evaluated on the [Zorro](https://github.com/phueb/Zorro) test suite.
|
11 |
+
This model was trained for 400K steps, and achieves an overall accuracy of 80.3,
|
12 |
+
comparable to RoBERTa-base, which achieves an overall accuracy of 82.6 on the latest version of Zorro (as of October, 2021).
|
13 |
+
The latter value is slightly larger than that reported in the paper (Huebner et al., 2020) because the authors previously lower-cased all words in Zorro before evaluation.
|
14 |
+
Lower-casing of proper nouns is detrimental to RoBERTa-base because RoBERTa-base has likely been exposed to proper nouns that are title-cased.
|
15 |
+
Because BabyBERTa is not case-sensitive, performance is not influenced by this change.
|
16 |
+
|
17 |
+
|
18 |
+
|
19 |
+
### Additional Information
|
20 |
+
|
21 |
This model was trained by [Philip Huebner](https://philhuebner.com), currently at the [UIUC Language and Learning Lab](http://www.learninglanguagelab.org).
|
22 |
|
23 |
More info can be found [here](https://github.com/phueb/BabyBERTa).
|