jtatman
/

llama-3.2-1b-trismegistus

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

jtatman commited on 24 days ago

Commit

b246f55

·

verified ·

1 Parent(s): e8f20ca

Update README.md

Files changed (1) hide show

README.md +14 -7

README.md CHANGED Viewed

@@ -56,13 +56,20 @@ Run it.
 ## Evaluation and Metrics
-|    Tasks     |Version|Filter|n-shot|  Metric  |   | Value |   |Stderr|
-|--------------|------:|------|-----:|----------|---|------:|---|-----:|
-|hellaswag     |      1|none  |     0|acc       |↑  | 0.4400|±  |0.0050|
-|              |       |none  |     0|acc_norm  |↑  | 0.5728|±  |0.0049|
-|lambada_openai|      1|none  |     0|acc       |↑  | 0.4450|±  |0.0069|
-|              |       |none  |     0|perplexity|↓  |18.4993|±  |0.8544|
 ## Environmental Impact

 ## Evaluation and Metrics
+|    Tasks    |Version|Filter|n-shot| Metric |   |Value |   |Stderr|
+|-------------|------:|------|-----:|--------|---|-----:|---|-----:|
+|arc_challenge|      1|none  |     0|acc     |↑  |0.3345|±  |0.0138|
+|             |       |none  |     0|acc_norm|↑  |0.3695|±  |0.0141|
+|arc_easy     |      1|none  |     0|acc     |↑  |0.6044|±  |0.0100|
+|             |       |none  |     0|acc_norm|↑  |0.5694|±  |0.0102|
+|boolq        |      2|none  |     0|acc     |↑  |0.6410|±  |0.0084|
+|hellaswag    |      1|none  |     0|acc     |↑  |0.4400|±  |0.0050|
+|             |       |none  |     0|acc_norm|↑  |0.5728|±  |0.0049|
+|openbookqa   |      1|none  |     0|acc     |↑  |0.2260|±  |0.0187|
+|             |       |none  |     0|acc_norm|↑  |0.3540|±  |0.0214|
+|piqa         |      1|none  |     0|acc     |↑  |0.7002|±  |0.0107|
+|             |       |none  |     0|acc_norm|↑  |0.7024|±  |0.0107|
+|winogrande   |      1|none  |     0|acc     |↑  |0.5785|±  |0.0139|
 ## Environmental Impact