winddude commited on
Commit
3323d85
1 Parent(s): 032a647

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -0
README.md CHANGED
@@ -19,6 +19,8 @@ This lora was trained on 250k post and response pairs from 43 different fincial,
19
 
20
  ## Training Details
21
 
 
 
22
  * Training took ~30hrs on 5x3090s and used almost 23gb of vram on each. DDP was used for pytorch parallelism.
23
 
24
  * 1 note worthy change I will mention now, is this was trained with casualLM rather than seq2seq like a number of the other instruct models have been. I can't explain why they used seq2seq for data collators, other than that's what alpaca lora originally used. Llama as a generative model was trained for casualLM so to me it makes sense to use that when fine tuning.
 
19
 
20
  ## Training Details
21
 
22
+ Base Model: llama 7b
23
+
24
  * Training took ~30hrs on 5x3090s and used almost 23gb of vram on each. DDP was used for pytorch parallelism.
25
 
26
  * 1 note worthy change I will mention now, is this was trained with casualLM rather than seq2seq like a number of the other instruct models have been. I can't explain why they used seq2seq for data collators, other than that's what alpaca lora originally used. Llama as a generative model was trained for casualLM so to me it makes sense to use that when fine tuning.