mmnga commited on
Commit
9a0e30c
1 Parent(s): 9aee719

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +15 -0
README.md CHANGED
@@ -11,8 +11,23 @@ language:
11
  # Model Card for Model ID
12
 
13
  <!-- Provide a quick summary of what the model is/does. -->
 
 
 
 
14
 
 
 
 
 
 
 
15
 
 
 
 
 
 
16
 
17
  ## Model Details
18
 
 
11
  # Model Card for Model ID
12
 
13
  <!-- Provide a quick summary of what the model is/does. -->
14
+ *実験モデルです / This is an experimental model.*
15
+ [lightblue/suzume-llama-3-8B-japanese](https://huggingface.co/lightblue/suzume-llama-3-8B-japanese)と、
16
+ [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct)の差分をchat-vectorアプローチで抽出し、
17
+ [meta-llama/Meta-Llama-3-70B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct)に適用しました
18
 
19
+ ## 手順/procedure
20
+ - ja
21
+ 1. `meta-llama/Meta-Llama-3-8B-Instruct`と`lightblue/suzume-llama-3-8B-japanese`の差分を作成
22
+ 2. shapeが異なるので、差分をmeta-llama/Meta-Llama-3-70B-Instruct用にアップサンプリング
23
+ 3. 前から 8-layer、最後から8-layerはそのまま適用
24
+ 4. 中間layerを引き延ばして適用
25
 
26
+ - en
27
+ 1. Create the difference between `meta-llama/Meta-Llama-3-8B-Instruct` and `lightblue/zume-llama-3-8B-japanese`
28
+ 2. Since the shapes are different, the difference is upsampled for meta-llama/Meta-Llama-3-70B-Instruct
29
+ 3. Apply the 8 layers from the front and 8 layers from the end as they are.
30
+ 4. Continue applying the middle layer
31
 
32
  ## Model Details
33