genne commited on
Commit
ca1d975
1 Parent(s): c4b5f6e

End of training

Browse files
Files changed (2) hide show
  1. README.md +18 -6
  2. adapter_model.safetensors +1 -1
README.md CHANGED
@@ -9,11 +9,6 @@ base_model: jingyeom/freeze_KoSoLAR-10.7B-v0.2_1.4_dedup
9
  model-index:
10
  - name: lora_freeze_KoSoLAR-10.7B-v0.2_1.4_dedup_SFT-DPO
11
  results: []
12
- datasets:
13
- - jondurbin/truthy-dpo-v0.1
14
- - Intel/orca_dpo_pairs
15
- - HuggingFaceH4/ultrafeedback_binarized
16
- - argilla/distilabel-math-preference-dpo
17
  ---
18
 
19
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -37,9 +32,26 @@ More information needed
37
 
38
  ## Training procedure
39
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
40
  ### Training results
41
 
42
- :)
43
 
44
  ### Framework versions
45
 
 
9
  model-index:
10
  - name: lora_freeze_KoSoLAR-10.7B-v0.2_1.4_dedup_SFT-DPO
11
  results: []
 
 
 
 
 
12
  ---
13
 
14
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
32
 
33
  ## Training procedure
34
 
35
+ ### Training hyperparameters
36
+
37
+ The following hyperparameters were used during training:
38
+ - learning_rate: 1e-05
39
+ - train_batch_size: 1
40
+ - eval_batch_size: 8
41
+ - seed: 42
42
+ - distributed_type: multi-GPU
43
+ - num_devices: 6
44
+ - gradient_accumulation_steps: 8
45
+ - total_train_batch_size: 48
46
+ - total_eval_batch_size: 48
47
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
48
+ - lr_scheduler_type: cosine
49
+ - lr_scheduler_warmup_ratio: 0.03
50
+ - num_epochs: 3
51
+
52
  ### Training results
53
 
54
+
55
 
56
  ### Framework versions
57
 
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:47708157ea8d4c0b5f6cb4c541d5ea5ea9f0aade2dcb40813a32b033084505f3
3
  size 10249120
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f443f560587757126ccacc9aaa78713e63f5acc9d61d64e5edc5c029e7054420
3
  size 10249120