Hugging Face's logo

rajtest
/
trained_weigths_updated

PEFT
TensorBoard
Safetensors
trl
sft
unsloth
Generated from Trainer
Model card Files Metrics Community
rajtest commited on
Commit
fbd40f1
•
1 Parent(s): bb9045d

rajtest/Llama_v4_update_senti

Browse files
Files changed (6) hide show
  1. README.md +3 -5
  2. adapter_config.json +3 -3
  3. adapter_model.safetensors +1 -1
  4. runs/Aug20_12-20-06_a92550c9e7b0/events.out.tfevents.1724156411.a92550c9e7b0.696.0 +3 -0
  5. runs/Aug20_12-20-23_a92550c9e7b0/events.out.tfevents.1724156429.a92550c9e7b0.696.1 +3 -0
  6. training_args.bin +1 -1
README.md CHANGED
@@ -19,7 +19,7 @@ should probably proofread and complete it, then remove this comment. -->
19
 
20
  This model is a fine-tuned version of [unsloth/llama-2-7b-bnb-4bit](https://huggingface.co/unsloth/llama-2-7b-bnb-4bit) on the None dataset.
21
  It achieves the following results on the evaluation set:
22
- - Loss: 0.5419
23
 
24
  ## Model description
25
 
@@ -47,16 +47,14 @@ The following hyperparameters were used during training:
47
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
48
  - lr_scheduler_type: cosine
49
  - lr_scheduler_warmup_ratio: 0.03
50
- - num_epochs: 3
51
  - mixed_precision_training: Native AMP
52
 
53
  ### Training results
54
 
55
  | Training Loss | Epoch | Step | Validation Loss |
56
  |:-------------:|:------:|:----:|:---------------:|
57
- | 0.5118 | 0.9981 | 262 | 0.5076 |
58
- | 0.4495 | 2.0 | 525 | 0.5086 |
59
- | 0.3469 | 2.9943 | 786 | 0.5419 |
60
 
61
 
62
  ### Framework versions
 
19
 
20
  This model is a fine-tuned version of [unsloth/llama-2-7b-bnb-4bit](https://huggingface.co/unsloth/llama-2-7b-bnb-4bit) on the None dataset.
21
  It achieves the following results on the evaluation set:
22
+ - Loss: 0.4471
23
 
24
  ## Model description
25
 
 
47
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
48
  - lr_scheduler_type: cosine
49
  - lr_scheduler_warmup_ratio: 0.03
50
+ - num_epochs: 1
51
  - mixed_precision_training: Native AMP
52
 
53
  ### Training results
54
 
55
  | Training Loss | Epoch | Step | Validation Loss |
56
  |:-------------:|:------:|:----:|:---------------:|
57
+ | 0.4447 | 0.9981 | 262 | 0.4471 |
 
 
58
 
59
 
60
  ### Framework versions
adapter_config.json CHANGED
@@ -21,12 +21,12 @@
21
  "revision": null,
22
  "target_modules": [
23
  "down_proj",
 
24
  "q_proj",
25
- "up_proj",
26
  "gate_proj",
 
27
  "o_proj",
28
- "v_proj",
29
- "k_proj"
30
  ],
31
  "task_type": "CAUSAL_LM",
32
  "use_dora": false,
 
21
  "revision": null,
22
  "target_modules": [
23
  "down_proj",
24
+ "k_proj",
25
  "q_proj",
 
26
  "gate_proj",
27
+ "up_proj",
28
  "o_proj",
29
+ "v_proj"
 
30
  ],
31
  "task_type": "CAUSAL_LM",
32
  "use_dora": false,
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:90dc9f8485566a8b998758a8ff5899ca4c6862754155826089f8d23bf7fdac09
3
  size 159967880
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:603c3f78985c7c11313954e433e72b13c97d6192d0167ef7e066ec81e19d3d3e
3
  size 159967880
runs/Aug20_12-20-06_a92550c9e7b0/events.out.tfevents.1724156411.a92550c9e7b0.696.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dfc0108f373eb44e16cd53ca05a44d92d98522b083612b37ada27a8496e44625
3
+ size 5431
runs/Aug20_12-20-23_a92550c9e7b0/events.out.tfevents.1724156429.a92550c9e7b0.696.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c95a17ff96a5109b81fd99faae85e397aee54f45c3d8693c926d1ba35d3fbcd4
3
+ size 11817
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a65a998c63d86528e0413a88be1ecd9b126b582fe8ee1b42a4c17e72970c75cb
3
  size 5176
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9dbef1d548b2b8c3354d1dbb9557a6402c521aac8def9e8ac29ed79140274fec
3
  size 5176