phatvo
/

Meta-Llama3.1-8B-Instruct-RAFT

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

phatvo commited on Aug 2

Commit

a2b682a

•

1 Parent(s): 94d2300

Update README.md

Files changed (1) hide show

README.md +6 -3

README.md CHANGED Viewed

@@ -22,10 +22,13 @@ pipeline_tag: text-generation
 LORA adapters of `meta-llama/Meta-Llama-3.1-8B-Instruct`, trained on 100 context samples from the HotpotQA dataset using the RAFT method, enable the model to better reason through the context and return more accurate outcomes.
 ### Evaluation
-Evaluated on 1k samples of validation set of HotpotQA.
 |      type    | exatch_match| f1         | precision     | recall  |
 |--------------|-------------|------------|---------------|---------|
-| pretrained   | 0.33        | 0.4195     | 0.4391        | 0.4931  |
-| finetuned    | 0.359       | **0.4821** | 0.4897        | 0.5281  |

 LORA adapters of `meta-llama/Meta-Llama-3.1-8B-Instruct`, trained on 100 context samples from the HotpotQA dataset using the RAFT method, enable the model to better reason through the context and return more accurate outcomes.
 ### Evaluation
+Evaluated on FULL validation set of HotpotQA.
 |      type    | exatch_match| f1         | precision     | recall  |
 |--------------|-------------|------------|---------------|---------|
+| pretrained   | 0.2980        | 0.3979     | 0.4116        | 0.5263  |
+| finetuned    | 0.3606       | **0.4857** | 0.4989        | 0.5318  |
+Finetuned version increases **22% on F1 and 15% on average**