Nexesenex commited on
Commit
9c60a70
1 Parent(s): 414c8d0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +20 -0
README.md CHANGED
@@ -1,3 +1,23 @@
1
  ---
2
  license: llama2
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: llama2
3
  ---
4
+ Airoboros c34 2.2.1 Mistral GGUF
5
+
6
+ CodeLlama 34b With Airoboros 2.2.1 dataset by Jon Durbin
7
+
8
+ Then
9
+
10
+ With Mistral AI 7b 0.1 delta bits compared to Llama2 (extracted by Undi95), merged by myself.
11
+
12
+ ---
13
+
14
+ Base model (CodeLlama) training context : 16k (max context up to 96k with the base ROPE)
15
+
16
+ Mistral injection training context : 8k (Sliding Windows Attention is likely inoperant on such a merge/injection)
17
+
18
+ ---
19
+
20
+ For test and amusement only.
21
+
22
+
23
+ Prompt : Airoboros