asiansoul commited on
Commit
06a47d6
β€’
1 Parent(s): abed5fd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +58 -1
README.md CHANGED
@@ -4,4 +4,61 @@ license_name: other
4
  license_link: LICENSE
5
  ---
6
 
7
- Another 2 Korean Model Mix by [Reborn Merge Method](https://medium.com/@puffanddmx82/reborn-elevating-model-adaptation-with-merging-for-superior-nlp-performance-f604e8e307b2)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
  license_link: LICENSE
5
  ---
6
 
7
+ Another 2 Korean Model Mix by [Reborn Merge Method](https://medium.com/@puffanddmx82/reborn-elevating-model-adaptation-with-merging-for-superior-nlp-performance-f604e8e307b2)
8
+
9
+ Keep in mind that the accuracy of your desired questions may vary.
10
+
11
+ ```
12
+ merge history : mistmatch interpolation
13
+
14
+ reference_model_name = "MLP-KTLim/llama-3-Korean-Bllossom-8B"
15
+ base_model_name = "NousResearch/Meta-Llama-3-8B-Instruct"
16
+ target_model_name = "maum-ai/Llama-3-MAAL-8B-Instruct-v0.1"
17
+
18
+ Interpolating tensor 'model.embed_tokens.weight' to match the shape: torch.Size([145088, 4096]) vs torch.Size([128256, 4096])
19
+ Interpolating tensor 'lm_head.weight' to match the shape: torch.Size([145088, 4096]) vs torch.Size([128256, 4096])
20
+ Interpolating tensor 'model.embed_tokens.weight' to match the shape: torch.Size([128256, 4096]) vs torch.Size([128257, 4096])
21
+ Interpolating tensor 'lm_head.weight' to match the shape: torch.Size([128256, 4096]) vs torch.Size([128257, 4096])
22
+ ```
23
+
24
+ Ollama Create
25
+ ```
26
+ jaylee@lees-MacBook-Pro-2 % ./ollama create Joah -f ./gguf/Joah-Llama-3-MAAL-MLP-KoEn-8B-Reborn/Modelfile_Q5_K_M
27
+ transferring model data
28
+ creating model layer
29
+ creating template layer
30
+ creating system layer
31
+ creating parameters layer
32
+ creating config layer
33
+ using already created layer sha256:4eadb53f0c70683aeab133c60d76b8ffc9f41ca5d49524d4b803c19e5ce7e3a5
34
+ using already created layer sha256:8ab4849b038cf0abc5b1c9b8ee1443dca6b93a045c2272180d985126eb40bf6f
35
+ writing layer sha256:ae2974c64ea5d6f488eeb1b10717a270f48fb3452432589db6f5e60472ae96ac
36
+ writing layer sha256:74ef6315972b317734fe01e7e1ad5b49fce1fa8ed3978cb66501ecb8c3a2e984
37
+ writing layer sha256:83882a5e957b8ce0d454f26bcedb2819413b49d6b967b28d60edb8ac61edfa58
38
+ writing manifest
39
+ success
40
+ ```
41
+
42
+ MODELFILE
43
+ ```
44
+ FROM joah-llama-3-maal-mlp-koen-8b-reborn-Q5_K_M.gguf
45
+ TEMPLATE """{{ if .System }}<|start_header_id|>system<|end_header_id|>
46
+
47
+ {{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>
48
+
49
+ {{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>
50
+
51
+ {{ .Response }}<|eot_id|>"""
52
+
53
+
54
+ SYSTEM """
55
+ μΉœμ ˆν•œ μ±—λ΄‡μœΌλ‘œμ„œ μƒλŒ€λ°©μ˜ μš”μ²­μ— μ΅œλŒ€ν•œ μžμ„Έν•˜κ³  μΉœμ ˆν•˜κ²Œ λ‹΅ν•˜μž. λͺ¨λ“  λŒ€λ‹΅μ€ ν•œκ΅­μ–΄(Korean)으둜 λŒ€λ‹΅ν•΄μ€˜.
56
+ """
57
+
58
+ PARAMETER num_keep 24
59
+ PARAMETER temperature 0.7
60
+ PARAMETER num_predict 3000
61
+ PARAMETER stop "<|start_header_id|>"
62
+ PARAMETER stop "<|end_header_id|>"
63
+ PARAMETER stop "<|eot_id|>"
64
+ ```