Hugging Face's logo

kevinautomation
/
llama2_instruct_generation

PEFT
TensorBoard
Safetensors
trl
sft
Generated from Trainer
Model card Files Metrics Community
kevinautomation commited on
Commit
976fcf6
•
1 Parent(s): b5929fb

kevinautomation/llama2-test1

Browse files
Files changed (6) hide show
  1. README.md +6 -30
  2. adapter_config.json +2 -1
  3. adapter_model.safetensors +2 -2
  4. runs/Feb04_14-35-35_753840eac725/events.out.tfevents.1707057408.753840eac725.188.0 +3 -0
  5. tokenizer.model +3 -0
  6. training_args.bin +1 -1
README.md CHANGED
@@ -19,7 +19,7 @@ should probably proofread and complete it, then remove this comment. -->
19
 
20
  This model is a fine-tuned version of [NousResearch/Llama-2-7b-hf](https://huggingface.co/NousResearch/Llama-2-7b-hf) on the generator dataset.
21
  It achieves the following results on the evaluation set:
22
- - Loss: 1.6730
23
 
24
  ## Model description
25
 
@@ -45,43 +45,19 @@ The following hyperparameters were used during training:
45
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
46
  - lr_scheduler_type: constant
47
  - lr_scheduler_warmup_steps: 0.03
48
- - training_steps: 500
49
 
50
  ### Training results
51
 
52
  | Training Loss | Epoch | Step | Validation Loss |
53
  |:-------------:|:-----:|:----:|:---------------:|
54
- | 1.8748 | 0.0 | 20 | 1.8101 |
55
- | 1.8661 | 0.01 | 40 | 1.7812 |
56
- | 2.0194 | 0.01 | 60 | 1.7690 |
57
- | 1.8829 | 0.01 | 80 | 1.7597 |
58
- | 1.8803 | 0.01 | 100 | 1.7469 |
59
- | 1.8252 | 0.02 | 120 | 1.7190 |
60
- | 1.8251 | 0.02 | 140 | 1.7057 |
61
- | 1.8709 | 0.02 | 160 | 1.7001 |
62
- | 1.7803 | 0.02 | 180 | 1.6953 |
63
- | 1.7745 | 0.03 | 200 | 1.6934 |
64
- | 1.8591 | 0.03 | 220 | 1.6893 |
65
- | 1.8532 | 0.03 | 240 | 1.6873 |
66
- | 1.7216 | 0.04 | 260 | 1.6862 |
67
- | 1.7967 | 0.04 | 280 | 1.6876 |
68
- | 1.718 | 0.04 | 300 | 1.6829 |
69
- | 1.7781 | 0.04 | 320 | 1.6800 |
70
- | 1.8361 | 0.05 | 340 | 1.6801 |
71
- | 1.7576 | 0.05 | 360 | 1.6793 |
72
- | 1.8326 | 0.05 | 380 | 1.6784 |
73
- | 1.7663 | 0.05 | 400 | 1.6773 |
74
- | 1.7169 | 0.06 | 420 | 1.6766 |
75
- | 1.7736 | 0.06 | 440 | 1.6760 |
76
- | 1.8094 | 0.06 | 460 | 1.6755 |
77
- | 1.7607 | 0.07 | 480 | 1.6736 |
78
- | 1.8041 | 0.07 | 500 | 1.6730 |
79
 
80
 
81
  ### Framework versions
82
 
83
- - PEFT 0.7.1
84
- - Transformers 4.37.0
85
  - Pytorch 2.1.0+cu121
86
  - Datasets 2.16.1
87
- - Tokenizers 0.15.0
 
19
 
20
  This model is a fine-tuned version of [NousResearch/Llama-2-7b-hf](https://huggingface.co/NousResearch/Llama-2-7b-hf) on the generator dataset.
21
  It achieves the following results on the evaluation set:
22
+ - Loss: 1.8095
23
 
24
  ## Model description
25
 
 
45
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
46
  - lr_scheduler_type: constant
47
  - lr_scheduler_warmup_steps: 0.03
48
+ - training_steps: 20
49
 
50
  ### Training results
51
 
52
  | Training Loss | Epoch | Step | Validation Loss |
53
  |:-------------:|:-----:|:----:|:---------------:|
54
+ | 1.9342 | 0.0 | 20 | 1.8095 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
55
 
56
 
57
  ### Framework versions
58
 
59
+ - PEFT 0.8.2
60
+ - Transformers 4.37.2
61
  - Pytorch 2.1.0+cu121
62
  - Datasets 2.16.1
63
+ - Tokenizers 0.15.1
adapter_config.json CHANGED
@@ -22,5 +22,6 @@
22
  "q_proj",
23
  "v_proj"
24
  ],
25
- "task_type": "CAUSAL_LM"
 
26
  }
 
22
  "q_proj",
23
  "v_proj"
24
  ],
25
+ "task_type": "CAUSAL_LM",
26
+ "use_rslora": false
27
  }
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:56521249d37b4e0fc40bb66b9b66349db2d9d997d0d03ff2614aa756f2678bbc
3
- size 134235048
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e44ce263e6fd885f50d82ca515b9325375b43ee36ededb75acf161ce88bc2e41
3
+ size 48
runs/Feb04_14-35-35_753840eac725/events.out.tfevents.1707057408.753840eac725.188.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cfc8108bb3b37309a61b1a860bcc5bb7dcd94d5518c69759466f091094a78e52
3
+ size 5757
tokenizer.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347
3
+ size 499723
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e4d0f271a52c5d365a163c34f8c64133308142c6b8f5e7474b541d7139597c12
3
  size 4728
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:805d14aed7489201f00e7fa3436d78850f5490def30e7a97d622f928a992ba8b
3
  size 4728