eelxpeng commited on
Commit
9eb8bc4
1 Parent(s): ebc9bd5

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +18 -0
README.md ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ datasets:
4
+ - HuggingFaceH4/ultrafeedback_binarized
5
+ language:
6
+ - en
7
+ ---
8
+
9
+ # Introduction
10
+ This model vistagi/Mixtral-8x7b-v0.1-sft is trained with Ultrachat-200K dataset through supervised finetuning using Mixtral-8x7b-v0.1 as the baseline model. The training is done with bfloat16 precision using LoRA.
11
+
12
+ ## Details
13
+ Used Librarys
14
+ - torch
15
+ - deepspeed
16
+ - pytorch lightning
17
+ - transformers
18
+ - peft