kyujinpy commited on
Commit
1598341
1 Parent(s): ddd954e

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +64 -0
README.md CHANGED
@@ -1,3 +1,67 @@
1
  ---
 
 
 
 
 
 
2
  license: cc-by-nc-sa-4.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ language:
3
+ - en
4
+ datasets:
5
+ - garage-bAInd/Open-Platypus
6
+ library_name: transformers
7
+ pipeline_tag: text-generation
8
  license: cc-by-nc-sa-4.0
9
  ---
10
+
11
+ # **PlatYi-34B-200K-Q**
12
+ <img src='./PlatYi.png' width=256>
13
+
14
+ ## Model Details
15
+
16
+ **Model Developers** Kyujin Han (kyujinpy)
17
+
18
+ **Input** Models input text only.
19
+
20
+ **Output** Models generate text only.
21
+
22
+ **Model Architecture**
23
+ PlatYi-34B-200K-Q is an auto-regressive language model based on the Yi-34B transformer architecture.
24
+
25
+ **Blog Link**
26
+ Blog: [Coming soon...]
27
+ Github: [Coming soon...]
28
+
29
+ **Base Model**
30
+ [01-ai/Yi-34B](https://huggingface.co/01-ai/Yi-34B)
31
+
32
+ **Training Dataset**
33
+ [garage-bAInd/Open-Platypus](https://huggingface.co/datasets/garage-bAInd/Open-Platypus).
34
+
35
+ **Notice**
36
+ While training, I used QLoRA.
37
+ But, `lora_r` values is 64.
38
+
39
+ # **Model Benchmark**
40
+
41
+ ## Open leaderboard
42
+ - Follow up as [link](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).
43
+
44
+ | Model | Average | ARC | HellaSwag | MMLU | TruthfulQA | Winogrande | GSM8K |
45
+ | --- | --- | --- | --- | --- | --- | --- | --- |
46
+ | **PlatYi-34B-200K-Q** | NaN | NaN | NaN | NaN | NaN | NaN | NaN |
47
+ | PlatYi-34B-Q | 69.86 | 66.89 | 85.14 | 77.66 | 53.03 | 82.48 | 53.98 |
48
+ | [01-ai/Yi-34B](https://huggingface.co/01-ai/Yi-34B) | 69.42 | 64.59 | 85.69 | 76.35 | 56.23 | 83.03 | 50.64 |
49
+ | [01-ai/Yi-34B-200K](https://huggingface.co/01-ai/Yi-34B-200K) | 70.81 | 65.36 | 85.58 | 76.06 | 53.64 | 82.56 | 61.64 |
50
+
51
+ # Implementation Code
52
+ ```python
53
+ ### KO-Platypus
54
+ from transformers import AutoModelForCausalLM, AutoTokenizer
55
+ import torch
56
+
57
+ repo = "kyujinpy/PlatYi-34B-200K-Q"
58
+ OpenOrca = AutoModelForCausalLM.from_pretrained(
59
+ repo,
60
+ return_dict=True,
61
+ torch_dtype=torch.float16,
62
+ device_map='auto'
63
+ )
64
+ OpenOrca_tokenizer = AutoTokenizer.from_pretrained(repo)
65
+ ```
66
+
67
+ ---