phanerozoic commited on
Commit
6bbf3e4
1 Parent(s): b74a303

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +15 -17
README.md CHANGED
@@ -49,23 +49,21 @@ To enhance output quality and thematic consistency, custom stopping strings incl
49
  - "\n"
50
 
51
  ## Training Hyperparameters and Fine-Tuning Details:
52
- \`\`\`yaml
53
- micro_batch_size: 1
54
- batch_size: 0
55
- epochs: 1
56
- learning_rate: "2e-5"
57
- lr_scheduler_type: "linear"
58
- lora_rank: 8
59
- lora_alpha: 16
60
- lora_dropout: 0.05
61
- cutoff_len: 256
62
- warmup_steps: 8
63
- optimizer: "adamw_torch"
64
- grad_accumulation: 1
65
- train_runtime: 1697.081 seconds
66
- total_flos: 1.3663655883177984e+16
67
- train_loss: 1.7511341453808817
68
- \`\`\`
69
 
70
  ## Testing and Evaluation:
71
  During the testing phase, we conducted a series of evaluations to compare Llama3-Pirate-Talk-8b-v0.1 against the base Llama3 model. These tests involved complex navigational and general knowledge questions designed to assess the model's ability to maintain its thematic integrity while responding accurately to technically demanding prompts. The model demonstrated a strong thematic presence with consistent use of pirate vernacular. However, it showed limitations in handling high-precision technical content, which is an expected trade-off given its thematic specialization. These insights have been instrumental in identifying areas for further model refinement.
 
49
  - "\n"
50
 
51
  ## Training Hyperparameters and Fine-Tuning Details:
52
+ - micro_batch_size: 1
53
+ - batch_size: 0
54
+ - epochs: 1
55
+ - learning_rate: "2e-5"
56
+ - lr_scheduler_type: "linear"
57
+ - lora_rank: 8
58
+ - lora_alpha: 16
59
+ - lora_dropout: 0.05
60
+ - cutoff_len: 256
61
+ - warmup_steps: 8
62
+ - optimizer: "adamw_torch"
63
+ - grad_accumulation: 1
64
+ - train_runtime: 1697.081 seconds
65
+ - total_flos: 1.3663655883177984e+16
66
+ - train_loss: 1.7511341453808817
 
 
67
 
68
  ## Testing and Evaluation:
69
  During the testing phase, we conducted a series of evaluations to compare Llama3-Pirate-Talk-8b-v0.1 against the base Llama3 model. These tests involved complex navigational and general knowledge questions designed to assess the model's ability to maintain its thematic integrity while responding accurately to technically demanding prompts. The model demonstrated a strong thematic presence with consistent use of pirate vernacular. However, it showed limitations in handling high-precision technical content, which is an expected trade-off given its thematic specialization. These insights have been instrumental in identifying areas for further model refinement.