Zangs3011 commited on
Commit
9a7edbf
1 Parent(s): d572c8f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -3
README.md CHANGED
@@ -30,10 +30,10 @@ Breaking it down further, each epoch took only 5.8 hours and cost a mere `$19.25
30
  - Total finetuning Cost: $57.75
31
  - Model Path: meta-llama/Llama-2-70b-hf
32
  - Dataset: databricks/databricks-dolly-15k
33
- - Learning rate: (not provided in the original data)
34
  - Number of epochs: 3
35
- - Data split: (not provided in the original data, assuming Training: 90% / Validation: 10%)
36
- - Gradient accumulation steps: (not provided in the original data)
37
 
38
  license: apache-2.0
39
  ---
@@ -52,3 +52,9 @@ Prompt Used:
52
  [response]
53
  ```
54
 
 
 
 
 
 
 
 
30
  - Total finetuning Cost: $57.75
31
  - Model Path: meta-llama/Llama-2-70b-hf
32
  - Dataset: databricks/databricks-dolly-15k
33
+ - Learning rate: 0.0002
34
  - Number of epochs: 3
35
+ - Data split: Training 90% / Validation 10%
36
+ - Gradient accumulation steps: 4
37
 
38
  license: apache-2.0
39
  ---
 
52
  [response]
53
  ```
54
 
55
+ Loss metrics
56
+
57
+ Training loss (Blue) Validation Loss (orange):
58
+ ![training loss](train-loss.png "Training loss")
59
+
60
+