PEFT
Safetensors
llama
Generated from Trainer
muellerzr HF staff commited on
Commit
96f90aa
1 Parent(s): 08e2fe1

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +9 -5
README.md CHANGED
@@ -2,18 +2,23 @@
2
  library_name: peft
3
  tags:
4
  - generated_from_trainer
5
- base_model: llama3-8B
6
  model-index:
7
  - name: qlora_decrease_lr_promptfix
8
  results: []
9
- license: llama3
10
- datasets:
11
- - muellerzr/llama-3-8b-self-align-data-generation-results
12
  ---
13
 
14
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
15
  should probably proofread and complete it, then remove this comment. -->
16
 
 
 
 
 
 
 
 
 
17
  [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
18
  <details><summary>See axolotl config</summary>
19
 
@@ -74,7 +79,6 @@ group_by_length: false
74
  bf16: auto
75
  fp16:
76
  tf32: false
77
- chat_template: chatml
78
 
79
  gradient_checkpointing: true
80
  gradient_checkpointing_kwargs:
 
2
  library_name: peft
3
  tags:
4
  - generated_from_trainer
5
+ base_model: meta-llama/Llama-3-8B
6
  model-index:
7
  - name: qlora_decrease_lr_promptfix
8
  results: []
 
 
 
9
  ---
10
 
11
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
12
  should probably proofread and complete it, then remove this comment. -->
13
 
14
+ ## Llama-3 8B Self-Instruct: PEFT Edition
15
+
16
+ This model is the result of recreating the [StarCoder2 Self-Instruct](https://huggingface.co/blog/sc2-instruct) pipeline, but applied to Llama-3-8B.
17
+
18
+ It could not have been done without the blood, sweat, and tears of my dear friends who have helped me along the way with training my first LLM.
19
+
20
+ A blog will come shortly detailing the many training runs and failures during this.
21
+
22
  [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
23
  <details><summary>See axolotl config</summary>
24
 
 
79
  bf16: auto
80
  fp16:
81
  tf32: false
 
82
 
83
  gradient_checkpointing: true
84
  gradient_checkpointing_kwargs: