DopeorNope commited on
Commit
b50693b
1 Parent(s): cef9023

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +65 -0
README.md ADDED
@@ -0,0 +1,65 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - ko
4
+ library_name: transformers
5
+ pipeline_tag: text-generation
6
+ license: cc-by-nc-sa-4.0
7
+ ---
8
+ **The license is `cc-by-nc-sa-4.0`.**
9
+
10
+ # **🐻‍❄️You_can_cry_Snowman-13B🐻‍❄️**
11
+ ![img](https://drive.google.com/uc?export=view&id=11c1FV1hKPXriGJRVhNDN-9up0wMF9QZk)
12
+
13
+
14
+
15
+ ## Model Details
16
+
17
+ **Model Developers** Seungyoo Lee(DopeorNope)
18
+
19
+ I am in charge of Large Language Models (LLMs) at Markr AI team in South Korea.
20
+
21
+ **Input** Models input text only.
22
+
23
+ **Output** Models generate text only.
24
+
25
+ **Model Architecture**
26
+ You_can_cry_Snowman-13B is an auto-regressive language model based on the SOLAR architecture.
27
+
28
+ ---
29
+
30
+ ## **Base Model**
31
+
32
+ [kyujinpy/Sakura-SOLAR-Instruct](https://huggingface.co/kyujinpy/Sakura-SOLAR-Instruct)
33
+
34
+ [Weyaxi/SauerkrautLM-UNA-SOLAR-Instruct](https://huggingface.co/Weyaxi/SauerkrautLM-UNA-SOLAR-Instruct)
35
+
36
+
37
+ ## **Implemented Method**
38
+
39
+ I have merged two models by increasing the parameter size to create a larger model.
40
+
41
+ I wanted to check how much the performance of the SOLAR base model changes when the scale of the parameters is increased.
42
+
43
+ ---
44
+
45
+ # Implementation Code
46
+
47
+
48
+ ## Load model
49
+ ```python
50
+
51
+ from transformers import AutoModelForCausalLM, AutoTokenizer
52
+ import torch
53
+
54
+ repo = "DopeorNope/You_can_cry_Snowman-13B"
55
+ OpenOrca = AutoModelForCausalLM.from_pretrained(
56
+ repo,
57
+ return_dict=True,
58
+ torch_dtype=torch.float16,
59
+ device_map='auto'
60
+ )
61
+ OpenOrca_tokenizer = AutoTokenizer.from_pretrained(repo)
62
+ ```
63
+
64
+
65
+ ---