jisukim8873 commited on
Commit
c27fc77
1 Parent(s): 1d16fc9

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +30 -0
README.md ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ datasets:
4
+ - Open-Orca/SlimOrca
5
+ language:
6
+ - en
7
+ ---
8
+
9
+ # Model Details
10
+ * Model Description: This model is test for data ordering.
11
+ * Developed by: Jisu Kim
12
+ * Model Type: Large Language Model
13
+
14
+ # Model Architecture
15
+ This model is based on falcon-7B. We fine-tuning this model for data ordering task.
16
+
17
+ falcon-7B is a transformer model, with the following architecture choices:
18
+
19
+ * Grouped-Query Attention
20
+ * Sliding-Window Attention
21
+ * Byte-fallback BPE tokenizer
22
+
23
+ # Dataset
24
+ We random sample Open-Orca dataset. (We finetune the 100,000 dataset)
25
+
26
+ # Guthub
27
+ https://github.com/trailerAI
28
+
29
+ # License
30
+ Apache License 2.0