phueb commited on
Commit
3a4027c
1 Parent(s): f5e1df5

add performance info

Browse files
Files changed (1) hide show
  1. README.md +16 -1
README.md CHANGED
@@ -1,8 +1,23 @@
1
  ## BabyBERTA
2
 
3
- BabyBERTA is a slightly-modified and much smaller RoBERTa model trained on 5M words of American-English child-directed input.
 
 
4
  It is intended for language acquisition research, on a single desktop with a single GPU - no high-performance computing infrastructure needed.
5
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6
  This model was trained by [Philip Huebner](https://philhuebner.com), currently at the [UIUC Language and Learning Lab](http://www.learninglanguagelab.org).
7
 
8
  More info can be found [here](https://github.com/phueb/BabyBERTa).
 
1
  ## BabyBERTA
2
 
3
+ ### Overview
4
+
5
+ BabyBERTA is a light-weight version of RoBERTa trained on 5M words of American-English child-directed input.
6
  It is intended for language acquisition research, on a single desktop with a single GPU - no high-performance computing infrastructure needed.
7
 
8
+ ### Performance
9
+
10
+ The provided model is the best-performing out of 10 that were evaluated on the [Zorro](https://github.com/phueb/Zorro) test suite.
11
+ This model was trained for 400K steps, and achieves an overall accuracy of 80.3,
12
+ comparable to RoBERTa-base, which achieves an overall accuracy of 82.6 on the latest version of Zorro (as of October, 2021).
13
+ The latter value is slightly larger than that reported in the paper (Huebner et al., 2020) because the authors previously lower-cased all words in Zorro before evaluation.
14
+ Lower-casing of proper nouns is detrimental to RoBERTa-base because RoBERTa-base has likely been exposed to proper nouns that are title-cased.
15
+ Because BabyBERTa is not case-sensitive, performance is not influenced by this change.
16
+
17
+
18
+
19
+ ### Additional Information
20
+
21
  This model was trained by [Philip Huebner](https://philhuebner.com), currently at the [UIUC Language and Learning Lab](http://www.learninglanguagelab.org).
22
 
23
  More info can be found [here](https://github.com/phueb/BabyBERTa).