Hugging Face's logo

Ousso1117
/
SFT-base-Llama-3-8B-Instruct

PEFT
Safetensors
trl
sft
unsloth
Generated from Trainer
Model card Files Community
Ousso1117 commited on
Commit
bf63c00
•
1 Parent(s): 165089c

Model save

Browse files
Files changed (2) hide show
  1. README.md +9 -14
  2. adapter_model.safetensors +1 -1
README.md CHANGED
@@ -1,25 +1,25 @@
1
  ---
2
  library_name: peft
3
- license: llama3.1
4
- base_model: unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit
5
  tags:
6
  - trl
7
  - sft
8
  - unsloth
9
  - generated_from_trainer
10
  model-index:
11
- - name: SFT-unsloth-Llama-3-8B-Instruct
12
  results: []
13
  ---
14
 
15
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
16
  should probably proofread and complete it, then remove this comment. -->
17
 
18
- # SFT-unsloth-Llama-3-8B-Instruct
19
 
20
- This model is a fine-tuned version of [unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit](https://huggingface.co/unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit) on an unknown dataset.
21
  It achieves the following results on the evaluation set:
22
- - Loss: 0.2950
23
 
24
  ## Model description
25
 
@@ -47,20 +47,15 @@ The following hyperparameters were used during training:
47
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
48
  - lr_scheduler_type: linear
49
  - lr_scheduler_warmup_steps: 5
50
- - num_epochs: 3
51
  - mixed_precision_training: Native AMP
52
 
53
  ### Training results
54
 
55
  | Training Loss | Epoch | Step | Validation Loss |
56
  |:-------------:|:------:|:----:|:---------------:|
57
- | 1.053 | 0.4129 | 20 | 0.8088 |
58
- | 0.757 | 0.8258 | 40 | 0.7216 |
59
- | 0.6764 | 1.2387 | 60 | 0.6206 |
60
- | 0.5476 | 1.6516 | 80 | 0.4872 |
61
- | 0.4231 | 2.0645 | 100 | 0.3801 |
62
- | 0.3276 | 2.4774 | 120 | 0.3222 |
63
- | 0.2932 | 2.8903 | 140 | 0.2950 |
64
 
65
 
66
  ### Framework versions
 
1
  ---
2
  library_name: peft
3
+ license: llama3
4
+ base_model: meta-llama/Meta-Llama-3-8B-Instruct
5
  tags:
6
  - trl
7
  - sft
8
  - unsloth
9
  - generated_from_trainer
10
  model-index:
11
+ - name: SFT-base-Llama-3-8B-Instruct
12
  results: []
13
  ---
14
 
15
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
16
  should probably proofread and complete it, then remove this comment. -->
17
 
18
+ # SFT-base-Llama-3-8B-Instruct
19
 
20
+ This model is a fine-tuned version of [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) on an unknown dataset.
21
  It achieves the following results on the evaluation set:
22
+ - Loss: 0.7187
23
 
24
  ## Model description
25
 
 
47
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
48
  - lr_scheduler_type: linear
49
  - lr_scheduler_warmup_steps: 5
50
+ - num_epochs: 1
51
  - mixed_precision_training: Native AMP
52
 
53
  ### Training results
54
 
55
  | Training Loss | Epoch | Step | Validation Loss |
56
  |:-------------:|:------:|:----:|:---------------:|
57
+ | 1.0531 | 0.4129 | 20 | 0.8121 |
58
+ | 0.7559 | 0.8258 | 40 | 0.7187 |
 
 
 
 
 
59
 
60
 
61
  ### Framework versions
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:49d11e586b2fbf0a9c58d30d390814d3ec64966bf5e90c63846452f16ecc2e55
3
  size 167832240
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f0e69ff931a13045dad59bd43a613f951b03b8863fa28eeb21a55597c45412aa
3
  size 167832240