pszemraj commited on
Commit
96dac12
1 Parent(s): f5e092e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +20 -0
README.md CHANGED
@@ -26,6 +26,26 @@ It achieves the following results on the evaluation set:
26
  - Accuracy: 0.1881
27
  - Num Input Tokens Seen: 766509056
28
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
29
  ## Training procedure
30
 
31
  ### Training hyperparameters
 
26
  - Accuracy: 0.1881
27
  - Num Input Tokens Seen: 766509056
28
 
29
+ ## evals
30
+
31
+ tl;dr its bad/would need more training:
32
+
33
+
34
+ hf (pretrained=pszemraj/griffin-llama3t-8L-v0.02-fineweb,trust_remote_code=True,dtype=float), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: 4
35
+
36
+ | Tasks |Version|Filter|n-shot| Metric | Value | | Stderr |
37
+ |--------------|------:|------|-----:|----------|----------:|---|---------:|
38
+ |winogrande | 1|none | 0|acc | 0.4964|± | 0.0141|
39
+ |piqa | 1|none | 0|acc | 0.5332|± | 0.0116|
40
+ | | |none | 0|acc_norm | 0.5299|± | 0.0116|
41
+ |openbookqa | 1|none | 0|acc | 0.1280|± | 0.0150|
42
+ | | |none | 0|acc_norm | 0.2320|± | 0.0189|
43
+ |lambada_openai| 1|none | 0|perplexity|638060.0702|± |43608.0044|
44
+ | | |none | 0|acc | 0.0000|± | 0.0000|
45
+ |boolq | 2|none | 0|acc | 0.3783|± | 0.0085|
46
+ |arc_easy | 1|none | 0|acc | 0.2614|± | 0.0090|
47
+ | | |none | 0|acc_norm | 0.2744|± | 0.0092|
48
+
49
  ## Training procedure
50
 
51
  ### Training hyperparameters