Hugging Face's logo

xverse
/
XVERSE-13B

Text Generation
Transformers
PyTorch
xverse
custom_code
Model card Files Community
9
pom commited on
Commit
60441ff
•
1 Parent(s): 18ab44d

Update xverse model

Browse files
Files changed (13) hide show
  1. MODEL_LICENSE.pdf +0 -0
  2. README.md +41 -75
  3. pytorch_model-00003-of-00003.bin → pytorch_model-00001-of-00010.bin +2 -2
  4. pytorch_model-00001-of-00003.bin → pytorch_model-00002-of-00010.bin +2 -2
  5. pytorch_model-00002-of-00003.bin → pytorch_model-00003-of-00010.bin +2 -2
  6. pytorch_model-00004-of-00010.bin +3 -0
  7. pytorch_model-00005-of-00010.bin +3 -0
  8. pytorch_model-00006-of-00010.bin +3 -0
  9. pytorch_model-00007-of-00010.bin +3 -0
  10. pytorch_model-00008-of-00010.bin +3 -0
  11. pytorch_model-00009-of-00010.bin +3 -0
  12. pytorch_model-00010-of-00010.bin +3 -0
  13. pytorch_model.bin.index.json +404 -404
MODEL_LICENSE.pdf CHANGED
Binary files a/MODEL_LICENSE.pdf and b/MODEL_LICENSE.pdf differ
 
README.md CHANGED
@@ -12,8 +12,8 @@ inference: false
12
  **XVERSE-13B** 是由深圳元象科技自主研发的支持多语言的大语言模型（Large Language Model），主要特点如下：
13
 
14
  - **模型结构**：XVERSE-13B 使用主流 Decoder-only 的标准 Transformer 网络结构，支持 8K 的上下文长度（Context Length），为同尺寸模型中最长，能满足更长的多轮对话、知识问答与摘要等需求，模型应用场景更广泛。
15
- - **训练数据**：构建了 1.4 万亿 token 的高质量、多样化的数据对模型进行充分训练，包含中、英、俄、西等 40 多种语言，通过精细化设置不同类型数据的采样比例，使得中英两种语言表现优异，也能兼顾其他语言效果。
16
- - **分词**：基于 BPE（Byte-Pair Encoding）算法，使用上百 GB 语料训练了一个词表大小为 100,278 的分词器，能够同时支持多语言，而无需额外扩展词表。
17
  - **训练框架**：自主研发多项关键技术，包括高效算子、显存优化、并行调度策略、数据-计算-通信重叠、平台和框架协同等，让训练效率更高，模型稳定性强，在千卡集群上的峰值算力利用率可达到 58.5%，位居业界前列。
18
 
19
  ## Model Introduction
@@ -21,88 +21,55 @@ inference: false
21
  **XVERSE-13B** is a multilingual large language model, independently developed by Shenzhen Yuanxiang Technology. Its key features are as follows:
22
 
23
  - **Model Structure**: XVERSE-13B uses the mainstream Decoder-only Transformer network structure, supports 8k context length, the longest one among models of the same size, which can meet the need of longer multi-round dialogues, knowledge question-answering, and summarization. This makes the model more versatile in application scenarios.
24
- - **Training Data**: The model has been thoroughly trained on a diversified and high-quality dataset consisting of 1.4 trillion of tokens, including more than 40 languages such as Chinese, English, Russian, and Spanish. The sampling ratio of different types of data is finely set, which makes the performance of Chinese and English excellent, and also takes into account the effect of other languages.
25
- - **Tokenization**: Based on the BPE (Byte-Pair Encoding) algorithm, a tokenizer with a vocabulary size of 100,278 has been trained using hundreds of gigabytes of language data. This tokenizer is capable of supporting multilingual without the need for additional vocabulary expansion.
26
  - **Training Framework**: Several key technologies have also been independently developed, including efficient operators, memory optimization, parallel scheduling strategies, overlap of data-computation-communication, and synergy between platforms and frameworks. These advancements enhance training efficiency and model stability. With these technologies, the peak computational power utilization rate on a thousand-card cluster can reach 58.5%, ranking at the forefront of the industry.
27
 
28
  ## 评测结果
29
 
30
- 为验证模型的各项能力，我们选取了多个学科综合能力评测集，包括 [MMLU](https://arxiv.org/abs/2009.03300)（英文）、 [C-Eval](https://cevalbenchmark.com/)（中文）、[AGIEval](https://arxiv.org/abs/2304.06364)（中英） 、[GAOKAO-Bench](https://github.com/OpenLMLab/GAOKAO-Bench)（中英）、[GAOKAO-English](https://github.com/ExpressAI/AI-Gaokao)（英文），评测结果如下：
31
-
32
- | 模型\数据集 | MMLU | C-Eval | AGIEval<sup>1</sup> | GAOKAO-Bench<sup>1</sup> | GAOKAO-English<sup>1</sup> |
33
- | :------------------------: | :--------------: | :--------------: | :-----------------: | :----------------------: | :------------------------: |
34
- | Baichuan-13B | 51.6<sup>2</sup> | 53.6<sup>3</sup> | 40.5 | 45.9 | 56.9 |
35
- | Llama-1-13B | 46.9<sup>4</sup> | 28.8 | 27.3 | 26.4 | 38.1 |
36
- | Llama-2-13B | 54.8<sup>4</sup> | 35.6 | 33.4 | 35.4 | 60.6 |
37
- | moss-moon-003-base (16B) | 24.7 | 33.1<sup>3</sup> | 26.8 | 28.5 | 34.7 |
38
- | OpenLLaMA-13B | 42.4 | 24.7 | 24.0 | 25.6 | 33.3 |
39
- | OPT-13B | 25.2 | 25.0 | 24.2 | 24.4 | 31.1 |
40
- | Pythia-12B | 25.1 | 26.2 | 25.3 | 25.3 | 26.8 |
41
- | Ziya-LLaMA-13B-Pretrain-v1 | 43.9 | 30.2 | 27.2 | 26.4 | 37.6 |
42
- | **XVERSE-13B** | **55.1** | **54.7** | **41.4** | **53.9** | **66.5** |
 
 
43
 
44
  > <sup>1：只针对其中的单项选择题进行测试，即排除了填空题、开放性问题和多项选择题</sup>
45
- > <sup>2：来源于 [Baichuan-13B](https://github.com/baichuan-inc/Baichuan-13B) 的汇报结果</sup>
46
- > <sup>3：来源于 [C-Eval](https://cevalbenchmark.com/) 的汇报结果</sup>
47
- > <sup>4：来源于[Llama 2 论文](https://arxiv.org/abs/2307.09288)的汇报结果</sup>
48
- >
49
- > 对于 MMLU ，我们采用作者提供的[评测工具](https://github.com/hendrycks/test)，C-Eval、AGIEval、GAOKAO-Bench、GAOKAO-English 与 MMLU 的评测方式相同，且统一采用 **5-shot** 构造测试样本。
50
 
 
 
51
 
52
  ## Model Evaluation
53
 
54
- In order to validate the various abilities of the model, we have chosen several comprehensive capability benchmarks across multiple disciplines, including [MMLU](https://arxiv.org/abs/2009.03300) (English), [C-Eval](https://cevalbenchmark.com/) (Chinese), [AGIEval](https://arxiv.org/abs/2304.06364) (Chinese and English), [GAOKAO-Bench](https://github.com/OpenLMLab/GAOKAO-Bench) (Chinese and English), [GAOKAO-English](https://github.com/ExpressAI/AI-Gaokao) (English), the evaluation results are as follows:
55
-
56
- | Models\Datasets | MMLU | C-Eval | AGIEval<sup>1</sup> | GAOKAO-Bench<sup>1</sup> | GAOKAO-English<sup>1</sup> |
57
- | :------------------------: | :--------------: | :--------------: | :-----------------: | :----------------------: | :------------------------: |
58
- | Baichuan-13B | 51.6<sup>2</sup> | 53.6<sup>3</sup> | 40.5 | 45.9 | 56.9 |
59
- | Llama-1-13B | 46.9<sup>4</sup> | 28.8 | 27.3 | 26.4 | 38.1 |
60
- | Llama-2-13B | 54.8<sup>4</sup> | 35.6 | 33.4 | 35.4 | 60.6 |
61
- | moss-moon-003-base (16B) | 24.7 | 33.1<sup>3</sup> | 26.8 | 28.5 | 34.7 |
62
- | OpenLLaMA-13B | 42.4 | 24.7 | 24.0 | 25.6 | 33.3 |
63
- | OPT-13B | 25.2 | 25.0 | 24.2 | 24.4 | 31.1 |
64
- | Pythia-12B | 25.1 | 26.2 | 25.3 | 25.3 | 26.8 |
65
- | Ziya-LLaMA-13B-Pretrain-v1 | 43.9 | 30.2 | 27.2 | 26.4 | 37.6 |
66
- | **XVERSE-13B** | **55.1** | **54.7** | **41.4** | **53.9** | **66.5** |
 
 
67
 
68
  > <sup>1: Tests are conducted only on single-answer multiple-choice questions, thus excluding fill-in-the-blanks, open-ended questions, and multiple-answer multiple-choice questions.</sup>
69
- > <sup>2: Reporting results from [Baichuan-13B](https://github.com/baichuan-inc/Baichuan-13B).</sup>
70
- > <sup>3: Reporting results from [C-Eval](https://cevalbenchmark.com/).</sup>
71
- > <sup>4: Reporting results from [Llama 2](https://arxiv.org/abs/2307.09288).</sup>
72
- >
73
- > For MMLU, we adopt the [evaluation tools](https://github.com/hendrycks/test) provided by the authors, C-Eval, AGIEval, GAOKAO-Bench, GAOKAO-English are the same as MMLU, and uniformly use **5-shot** to construct the test samples.
74
-
75
- ### MMLU 各类别指标
76
-
77
- MMLU Category Results
78
-
79
- | 模型\类别 | Average | STEM | Social Science | Humanities | Others |
80
- | :------------------------: | :------: | :------: | :------------: | :--------: | :------: |
81
- | Baichuan-13B | 51.6 | 41.6 | 60.9 | 47.4 | 58.5 |
82
- | Llama-1-13B | 46.9 | 35.8 | 53.8 | 45.0 | 53.3 |
83
- | Llama-2-13B | 54.8 | 44.1 | 62.6 | 52.8 | 61.1 |
84
- | moss-moon-003-base (16B) | 24.7 | 23.0 | 24.0 | 25.2 | 26.3 |
85
- | OpenLLaMA-13B | 42.4 | 34.7 | 48.6 | 40.0 | 47.1 |
86
- | OPT-13B | 25.2 | 23.9 | 24.1 | 25.9 | 26.3 |
87
- | Pythia-12B | 25.1 | 24.8 | 23.0 | 26.1 | 26.0 |
88
- | Ziya-LLaMA-13B-Pretrain-v1 | 43.9 | 36.3 | 48.8 | 41.1 | 50.3 |
89
- | **XVERSE-13B** | **55.1** | **44.5** | **64.4** | **50.5** | **62.9** |
90
-
91
- ### C-Eval 各类别指标
92
-
93
- C-Eval Category Results
94
-
95
- | 模型\类别 | Average | STEM | Social Science | Humanities | Others |
96
- | :------------------------: | :------: | :------: | :------------: | :--------: | :------: |
97
- | Baichuan-13B | 53.6 | 47.0 | 66.8 | 57.3 | 49.8 |
98
- | Llama-1-13B | 28.8 | 27.5 | 33.9 | 27.7 | 27.7 |
99
- | Llama-2-13B | 35.6 | 34.5 | 39.8 | 36.2 | 33.2 |
100
- | moss-moon-003-base (16B) | 33.1 | 31.6 | 37.0 | 33.4 | 32.1 |
101
- | OpenLLaMA-13B | 24.7 | 25.5 | 23.5 | 24.2 | 24.7 |
102
- | OPT-13B | 25.0 | 24.4 | 24.6 | 25.9 | 25.4 |
103
- | Pythia-12B | 26.2 | 26.8 | 25.1 | 26.7 | 25.4 |
104
- | Ziya-LLaMA-13B-Pretrain-v1 | 30.2 | 27.8 | 34.3 | 32.0 | 29.0 |
105
- | **XVERSE-13B** | **54.7** | **45.6** | **66.2** | **58.3** | **56.9** |
106
 
107
  ### Loading with Transformers
108
 
@@ -122,10 +89,9 @@ The XVERSE-13B model can be loaded for inference using the following code:
122
  >>> print(tokenizer.batch_decode(generated_ids, skip_special_tokens=True))
123
  ```
124
 
125
- 更多有关相关细节，包括文本生成demo和环境依赖，请参考我们的[Github](https://github.com/xverse-ai/XVERSE-13B)。
126
-
127
- For more details, including the demo of text generation and environmental dependencies, please refer to our [Github](https://github.com/xverse-ai/XVERSE-13B).
128
 
 
129
 
130
  ## 局限性与免责申明
131
 
 
12
  **XVERSE-13B** 是由深圳元象科技自主研发的支持多语言的大语言模型（Large Language Model），主要特点如下：
13
 
14
  - **模型结构**：XVERSE-13B 使用主流 Decoder-only 的标准 Transformer 网络结构，支持 8K 的上下文长度（Context Length），为同尺寸模型中最长，能满足更长的多轮对话、知识问答与摘要等需求，模型应用场景更广泛。
15
+ - **训练数据**：构建了 3.2 万亿 token 的高质量、多样化的数据对模型进行充分训练，包含中、英、俄、西等 40 多种语言，通过精细化设置不同类型数据的采样比例，使得中英两种语言表现优异，也能兼顾其他语言效果。
16
+ - **分词**：基于 BPE（Byte-Pair Encoding）算法，使用上百 GB 语料训练了一个词表大小为 100,534 的分词器，能够同时支持多语言，而无需额外扩展词表。
17
  - **训练框架**：自主研发多项关键技术，包括高效算子、显存优化、并行调度策略、数据-计算-通信重叠、平台和框架协同等，让训练效率更高，模型稳定性强，在千卡集群上的峰值算力利用率可达到 58.5%，位居业界前列。
18
 
19
  ## Model Introduction
 
21
  **XVERSE-13B** is a multilingual large language model, independently developed by Shenzhen Yuanxiang Technology. Its key features are as follows:
22
 
23
  - **Model Structure**: XVERSE-13B uses the mainstream Decoder-only Transformer network structure, supports 8k context length, the longest one among models of the same size, which can meet the need of longer multi-round dialogues, knowledge question-answering, and summarization. This makes the model more versatile in application scenarios.
24
+ - **Training Data**: The model has been thoroughly trained on a diversified and high-quality dataset consisting of 3.2 trillion of tokens, including more than 40 languages such as Chinese, English, Russian, and Spanish. The sampling ratio of different types of data is finely set, which makes the performance of Chinese and English excellent, and also takes into account the effect of other languages.
25
+ - **Tokenization**: Based on the BPE (Byte-Pair Encoding) algorithm, a tokenizer with a vocabulary size of 100,534 has been trained using hundreds of gigabytes of language data. This tokenizer is capable of supporting multilingual without the need for additional vocabulary expansion.
26
  - **Training Framework**: Several key technologies have also been independently developed, including efficient operators, memory optimization, parallel scheduling strategies, overlap of data-computation-communication, and synergy between platforms and frameworks. These advancements enhance training efficiency and model stability. With these technologies, the peak computational power utilization rate on a thousand-card cluster can reach 58.5%, ranking at the forefront of the industry.
27
 
28
  ## 评测结果
29
 
30
+ 为了综合评估模型的性能，我们在一系列标准数据集上进行了全面测试，包括C-Eval、CMMLU、Gaokao-Bench、MMLU、GAOKAO-English、AGIEval、RACE-M、CommonSenseQA、PIQA、GSM8K和HumanEval。这些评估覆盖了模型在多个领域的能力，具体包括中文问答、英文问答、语言理解、常识问答、逻辑推理、数学问题解答以及编程能力。评估结果如下：
31
+
32
+ | 能力维度 | 数据集 | | XVERSE-13B-2 | XVERSE-13B | Baichuan2-13B | Llama1-13B | Llama2-13B |
33
+ | :--------: | :------------------------: | :----: | :----------: | :--------: | :-----------: | :--------: | :--------: |
34
+ | 中文问答 | C-Eval | 5-shot | 63.5 | 54.7 | 58.1 | 28.8 | 35.6 |
35
+ | | CMMLU | 5-shot | 66.2 | 59.1 | 62.0 | 31.5 | 38.4 |
36
+ | | Gaokao-Bench<sup>1</sup> | 5-shot | 67.5 | 53.9 | 54.3 | 26.4 | 35.4 |
37
+ | 英文问答 | MMLU | 5-shot | 61.2 | 55.1 | 59.2 | 46.9 | 54.8 |
38
+ | | GAOKAO-English<sup>1</sup> | 5-shot | 73.7 | 66.5 | 67.7 | 38.1 | 60.6 |
39
+ | 中英文问答 | AGIEval<sup>1</sup> | 5-shot | 54.5 | 41.4 | 48.2 | 27.3 | 33.4 |
40
+ | 语言理解 | RACE-M | 0-shot | 84.6 | 74.2 | 68.9 | 61.6 | 63.0 |
41
+ | 常识问答 | CommonSenseQA | 7-shot | 74.0 | 69.5 | 65.6 | 62.0 | 67.3 |
42
+ | 推理 | PIQA | 0-shot | 80.8 | 79.0 | 78.5 | 80.1 | 80.5 |
43
+ | 数学 | GSM8K | 4-shot | 54.9 | 18.4 | 52.7 | 17.8 | 28.7 |
44
+ | 代码 | HumanEval | 0-shot | 39.6 | 15.9 | 17.1 | 15.8 | 18.3 |
45
 
46
  > <sup>1：只针对其中的单项选择题进行测试，即排除了填空题、开放性问题和多项选择题</sup>
 
 
 
 
 
47
 
48
+ 对于上述所有比较模型，我们优先汇报其官方公布的结果。在缺少官方结果的情况下，我们采用了 [OpenCompass 榜单](https://opencompass.org.cn/leaderboard-llm)的报告结果。其他结果则来自于我们自行执行的评估流程所获得的数据。
49
+ 对于 MMLU ，我们采用作者提供的[评测工具](https://github.com/hendrycks/test)，C-Eval、AGIEval、GAOKAO-Bench、GAOKAO-English 与 MMLU 的评测方式相同，其余评测数据集使用 [OpenCompass 评估框架](https://github.com/open-compass/OpenCompass/)进行评估。
50
 
51
  ## Model Evaluation
52
 
53
+ To comprehensively assess the performance of the model, we conducted extensive testing across a range of standard datasets, including C-Eval, CMMLU, Gaokao-Bench, MMLU, GAOKAO-English, AGIEval, RACE-M, CommonSenseQA, PIQA, GSM8K and HumanEval. These evaluations spanned multiple capabilities of the model, specifically including Chinese question answering, English question answering, language comprehension, common sense questioning, logical reasoning, mathematical problem-solving, and coding ability. The results of the evaluations are as follows:
54
+
55
+ | Capability Dimension | Dataset | | XVERSE-13B-2 | XVERSE-13B | Baichuan2-13B | Llama1-13B | Llama2-13B |
56
+ | :--------------------: | :------------------------: | :----: | :----------: | :--------: | :-----------: | :--------: | :--------: |
57
+ | Chinese QA | C-Eval | 5-shot | 63.5 | 54.7 | 58.1 | 28.8 | 35.6 |
58
+ | | CMMLU | 5-shot | 66.2 | 59.1 | 62.0 | 31.5 | 38.4 |
59
+ | | Gaokao-Bench<sup>1</sup> | 5-shot | 67.5 | 53.9 | 54.3 | 26.4 | 35.4 |
60
+ | English QA | MMLU | 5-shot | 61.2 | 55.1 | 59.2 | 46.9 | 54.8 |
61
+ | | GAOKAO-English<sup>1</sup> | 5-shot | 73.7 | 66.5 | 67.7 | 38.1 | 60.6 |
62
+ | Chinese & English QA | AGIEval<sup>1</sup> | 5-shot | 54.5 | 41.4 | 48.2 | 27.3 | 33.4 |
63
+ | Language Understanding | RACE-M | 0-shot | 84.6 | 74.2 | 68.9 | 61.6 | 63.0 |
64
+ | Common Sense QA | CommonSenseQA | 7-shot | 74.0 | 69.5 | 65.6 | 62.0 | 67.3 |
65
+ | Reasoning | PIQA | 0-shot | 80.8 | 79.0 | 78.5 | 80.1 | 80.5 |
66
+ | Math | GSM8K | 4-shot | 54.9 | 18.4 | 52.7 | 17.8 | 28.7 |
67
+ | Coding | HumanEval | 0-shot | 39.6 | 15.9 | 17.1 | 15.8 | 18.3 |
68
 
69
  > <sup>1: Tests are conducted only on single-answer multiple-choice questions, thus excluding fill-in-the-blanks, open-ended questions, and multiple-answer multiple-choice questions.</sup>
70
+
71
+ For all the comparison models mentioned above, we prioritize the disclosure of their officially published results. In the absence of official data, we refer to the reported outcomes from [OpenCompass Leaderboard](https://opencompass.org.cn/leaderboard-llm). Results not covered by the aforementioned sources are derived from our own evaluation pipline.
72
+ For MMLU, we adopt the [evaluation tools](https://github.com/hendrycks/test) provided by the authors, C-Eval, AGIEval, GAOKAO-Bench, GAOKAO-English are the same as MMLU. For the remaining evaluation datasets, the [OpenCompass](https://github.com/open-compass/OpenCompass/) is employed for evaluation.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
73
 
74
  ### Loading with Transformers
75
 
 
89
  >>> print(tokenizer.batch_decode(generated_ids, skip_special_tokens=True))
90
  ```
91
 
92
+ 更多细节，包括对话demo、模型微调及量化等，请参考我们的[Github](https://github.com/xverse-ai/XVERSE-13B)。
 
 
93
 
94
+ For more details, including chat demo, model fine-tuning and quantization, please refer to our [Github](https://github.com/xverse-ai/XVERSE-13B).
95
 
96
  ## 局限性与免责申明
97
 
pytorch_model-00003-of-00003.bin → pytorch_model-00001-of-00010.bin RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:defb5f082a0b8924456672401a1d691cae47ce0e01d20e07255c8718d0d7b955
3
- size 7512554533
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:340c72360c3fbb2bdf6f98ab9ab8a2678285e5344d1b0a43ac8a05c4723f6f7f
3
+ size 2508131049
pytorch_model-00001-of-00003.bin → pytorch_model-00002-of-00010.bin RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:18696855f373f92e774ac4ae12d80fe66b1c1e9acc7b98cd1029fef9e9d228d3
3
- size 10065941445
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e1f5ac861b0b2bb760f64d526573a0b36d3eaf2f86cd7c65d7147d2a483ad880
3
+ size 3172057468
pytorch_model-00002-of-00003.bin → pytorch_model-00003-of-00010.bin RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:10bb394d79391a86c887edce8e7d2f1b921c60fcd9aefe89ee5fe6de9b370a0f
3
- size 10009010414
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6e5b048d9f9afca9a97660c8e9afd59d77be3ebfaa8b5993cc83de24b5c5c106
3
+ size 3172057468
pytorch_model-00004-of-00010.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b3867d8f4a57d9eacc7f6c74777d743b02bf1bd23336474385a6a5bdc37a670f
3
+ size 3172057532
pytorch_model-00005-of-00010.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5d5da23d80349815ef029e6ebfe434b62ccc406419d079af06023c3d571ae1ed
3
+ size 3172057532
pytorch_model-00006-of-00010.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:98c6d1632f78291d34a2c7ceeb95a6917fa8569d840a79a53ee8f362375c7c2a
3
+ size 3172057532
pytorch_model-00007-of-00010.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:16abf185e57889e0564ac3bfb61970f5e382c916c78a5409e4b47011c5403a21
3
+ size 3172057532
pytorch_model-00008-of-00010.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a4c25f7d9baa7e94e9c62520ab4a735dd16a92596c9bdef64d4a3ff6af019244
3
+ size 3172057532
pytorch_model-00009-of-00010.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b85c0810178333d726825e52a28659bf27b9a3a32909a125665b00ff2b9f9b4e
3
+ size 1693507250
pytorch_model-00010-of-00010.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4f71e8ca1c28e9689538e5573e25ff30d4fc142ab520bcd5b2d55ede6ea62590
3
+ size 1029571307
pytorch_model.bin.index.json CHANGED
@@ -1,410 +1,410 @@
1
  {
2
  "metadata": {
3
- "total_size": 27430067200
4
  },
5
  "weight_map": {
6
- "lm_head.weight": "pytorch_model-00003-of-00003.bin",
7
- "model.embed_tokens.weight": "pytorch_model-00001-of-00003.bin",
8
- "model.layers.0.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
9
- "model.layers.0.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
10
- "model.layers.0.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
11
- "model.layers.0.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
12
- "model.layers.0.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
13
- "model.layers.0.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
14
- "model.layers.0.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
15
- "model.layers.0.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
16
- "model.layers.0.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
17
- "model.layers.0.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
18
- "model.layers.1.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
19
- "model.layers.1.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
20
- "model.layers.1.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
21
- "model.layers.1.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
22
- "model.layers.1.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
23
- "model.layers.1.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
24
- "model.layers.1.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
25
- "model.layers.1.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
26
- "model.layers.1.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
27
- "model.layers.1.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
28
- "model.layers.10.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
29
- "model.layers.10.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
30
- "model.layers.10.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
31
- "model.layers.10.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
32
- "model.layers.10.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
33
- "model.layers.10.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
34
- "model.layers.10.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
35
- "model.layers.10.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
36
- "model.layers.10.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
37
- "model.layers.10.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
38
- "model.layers.11.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
39
- "model.layers.11.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
40
- "model.layers.11.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
41
- "model.layers.11.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
42
- "model.layers.11.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
43
- "model.layers.11.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
44
- "model.layers.11.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
45
- "model.layers.11.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
46
- "model.layers.11.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
47
- "model.layers.11.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
48
- "model.layers.12.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
49
- "model.layers.12.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
50
- "model.layers.12.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
51
- "model.layers.12.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
52
- "model.layers.12.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
53
- "model.layers.12.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
54
- "model.layers.12.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
55
- "model.layers.12.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
56
- "model.layers.12.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
57
- "model.layers.12.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
58
- "model.layers.13.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
59
- "model.layers.13.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
60
- "model.layers.13.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
61
- "model.layers.13.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
62
- "model.layers.13.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
63
- "model.layers.13.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
64
- "model.layers.13.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
65
- "model.layers.13.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
66
- "model.layers.13.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
67
- "model.layers.13.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
68
- "model.layers.14.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
69
- "model.layers.14.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
70
- "model.layers.14.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
71
- "model.layers.14.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
72
- "model.layers.14.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
73
- "model.layers.14.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
74
- "model.layers.14.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
75
- "model.layers.14.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
76
- "model.layers.14.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
77
- "model.layers.14.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
78
- "model.layers.15.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
79
- "model.layers.15.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
80
- "model.layers.15.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
81
- "model.layers.15.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
82
- "model.layers.15.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
83
- "model.layers.15.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
84
- "model.layers.15.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
85
- "model.layers.15.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
86
- "model.layers.15.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
87
- "model.layers.15.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
88
- "model.layers.16.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
89
- "model.layers.16.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
90
- "model.layers.16.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
91
- "model.layers.16.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
92
- "model.layers.16.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
93
- "model.layers.16.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
94
- "model.layers.16.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
95
- "model.layers.16.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
96
- "model.layers.16.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
97
- "model.layers.16.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
98
- "model.layers.17.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
99
- "model.layers.17.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
100
- "model.layers.17.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
101
- "model.layers.17.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
102
- "model.layers.17.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
103
- "model.layers.17.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
104
- "model.layers.17.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
105
- "model.layers.17.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
106
- "model.layers.17.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
107
- "model.layers.17.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
108
- "model.layers.18.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
109
- "model.layers.18.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
110
- "model.layers.18.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
111
- "model.layers.18.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
112
- "model.layers.18.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
113
- "model.layers.18.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
114
- "model.layers.18.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
115
- "model.layers.18.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
116
- "model.layers.18.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
117
- "model.layers.18.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
118
- "model.layers.19.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
119
- "model.layers.19.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
120
- "model.layers.19.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
121
- "model.layers.19.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
122
- "model.layers.19.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
123
- "model.layers.19.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
124
- "model.layers.19.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
125
- "model.layers.19.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
126
- "model.layers.19.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
127
- "model.layers.19.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
128
- "model.layers.2.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
129
- "model.layers.2.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
130
- "model.layers.2.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
131
- "model.layers.2.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
132
- "model.layers.2.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
133
- "model.layers.2.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
134
- "model.layers.2.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
135
- "model.layers.2.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
136
- "model.layers.2.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
137
- "model.layers.2.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
138
- "model.layers.20.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
139
- "model.layers.20.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
140
- "model.layers.20.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
141
- "model.layers.20.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
142
- "model.layers.20.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
143
- "model.layers.20.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
144
- "model.layers.20.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
145
- "model.layers.20.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
146
- "model.layers.20.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
147
- "model.layers.20.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
148
- "model.layers.21.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
149
- "model.layers.21.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
150
- "model.layers.21.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
151
- "model.layers.21.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
152
- "model.layers.21.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
153
- "model.layers.21.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
154
- "model.layers.21.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
155
- "model.layers.21.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
156
- "model.layers.21.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
157
- "model.layers.21.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
158
- "model.layers.22.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
159
- "model.layers.22.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
160
- "model.layers.22.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
161
- "model.layers.22.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
162
- "model.layers.22.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
163
- "model.layers.22.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
164
- "model.layers.22.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
165
- "model.layers.22.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
166
- "model.layers.22.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
167
- "model.layers.22.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
168
- "model.layers.23.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
169
- "model.layers.23.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
170
- "model.layers.23.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
171
- "model.layers.23.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
172
- "model.layers.23.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
173
- "model.layers.23.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
174
- "model.layers.23.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
175
- "model.layers.23.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
176
- "model.layers.23.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
177
- "model.layers.23.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
178
- "model.layers.24.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
179
- "model.layers.24.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
180
- "model.layers.24.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
181
- "model.layers.24.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
182
- "model.layers.24.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
183
- "model.layers.24.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
184
- "model.layers.24.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
185
- "model.layers.24.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
186
- "model.layers.24.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
187
- "model.layers.24.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
188
- "model.layers.25.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
189
- "model.layers.25.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
190
- "model.layers.25.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
191
- "model.layers.25.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
192
- "model.layers.25.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
193
- "model.layers.25.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
194
- "model.layers.25.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
195
- "model.layers.25.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
196
- "model.layers.25.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
197
- "model.layers.25.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
198
- "model.layers.26.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
199
- "model.layers.26.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
200
- "model.layers.26.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
201
- "model.layers.26.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
202
- "model.layers.26.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
203
- "model.layers.26.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
204
- "model.layers.26.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
205
- "model.layers.26.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
206
- "model.layers.26.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
207
- "model.layers.26.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
208
- "model.layers.27.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
209
- "model.layers.27.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
210
- "model.layers.27.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
211
- "model.layers.27.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
212
- "model.layers.27.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
213
- "model.layers.27.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
214
- "model.layers.27.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
215
- "model.layers.27.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
216
- "model.layers.27.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
217
- "model.layers.27.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
218
- "model.layers.28.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
219
- "model.layers.28.mlp.down_proj.weight": "pytorch_model-00002-of-00003.bin",
220
- "model.layers.28.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
221
- "model.layers.28.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
222
- "model.layers.28.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
223
- "model.layers.28.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
224
- "model.layers.28.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
225
- "model.layers.28.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
226
- "model.layers.28.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
227
- "model.layers.28.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
228
- "model.layers.29.input_layernorm.weight": "pytorch_model-00002-of-00003.bin",
229
- "model.layers.29.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
230
- "model.layers.29.mlp.gate_proj.weight": "pytorch_model-00002-of-00003.bin",
231
- "model.layers.29.mlp.up_proj.weight": "pytorch_model-00002-of-00003.bin",
232
- "model.layers.29.post_attention_layernorm.weight": "pytorch_model-00002-of-00003.bin",
233
- "model.layers.29.self_attn.k_proj.weight": "pytorch_model-00002-of-00003.bin",
234
- "model.layers.29.self_attn.o_proj.weight": "pytorch_model-00002-of-00003.bin",
235
- "model.layers.29.self_attn.q_proj.weight": "pytorch_model-00002-of-00003.bin",
236
- "model.layers.29.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00003.bin",
237
- "model.layers.29.self_attn.v_proj.weight": "pytorch_model-00002-of-00003.bin",
238
- "model.layers.3.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
239
- "model.layers.3.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
240
- "model.layers.3.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
241
- "model.layers.3.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
242
- "model.layers.3.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
243
- "model.layers.3.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
244
- "model.layers.3.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
245
- "model.layers.3.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
246
- "model.layers.3.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
247
- "model.layers.3.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
248
- "model.layers.30.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
249
- "model.layers.30.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
250
- "model.layers.30.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
251
- "model.layers.30.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
252
- "model.layers.30.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
253
- "model.layers.30.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
254
- "model.layers.30.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
255
- "model.layers.30.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
256
- "model.layers.30.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
257
- "model.layers.30.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
258
- "model.layers.31.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
259
- "model.layers.31.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
260
- "model.layers.31.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
261
- "model.layers.31.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
262
- "model.layers.31.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
263
- "model.layers.31.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
264
- "model.layers.31.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
265
- "model.layers.31.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
266
- "model.layers.31.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
267
- "model.layers.31.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
268
- "model.layers.32.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
269
- "model.layers.32.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
270
- "model.layers.32.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
271
- "model.layers.32.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
272
- "model.layers.32.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
273
- "model.layers.32.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
274
- "model.layers.32.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
275
- "model.layers.32.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
276
- "model.layers.32.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
277
- "model.layers.32.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
278
- "model.layers.33.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
279
- "model.layers.33.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
280
- "model.layers.33.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
281
- "model.layers.33.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
282
- "model.layers.33.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
283
- "model.layers.33.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
284
- "model.layers.33.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
285
- "model.layers.33.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
286
- "model.layers.33.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
287
- "model.layers.33.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
288
- "model.layers.34.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
289
- "model.layers.34.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
290
- "model.layers.34.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
291
- "model.layers.34.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
292
- "model.layers.34.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
293
- "model.layers.34.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
294
- "model.layers.34.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
295
- "model.layers.34.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
296
- "model.layers.34.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
297
- "model.layers.34.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
298
- "model.layers.35.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
299
- "model.layers.35.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
300
- "model.layers.35.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
301
- "model.layers.35.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
302
- "model.layers.35.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
303
- "model.layers.35.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
304
- "model.layers.35.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
305
- "model.layers.35.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
306
- "model.layers.35.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
307
- "model.layers.35.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
308
- "model.layers.36.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
309
- "model.layers.36.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
310
- "model.layers.36.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
311
- "model.layers.36.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
312
- "model.layers.36.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
313
- "model.layers.36.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
314
- "model.layers.36.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
315
- "model.layers.36.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
316
- "model.layers.36.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
317
- "model.layers.36.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
318
- "model.layers.37.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
319
- "model.layers.37.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
320
- "model.layers.37.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
321
- "model.layers.37.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
322
- "model.layers.37.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
323
- "model.layers.37.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
324
- "model.layers.37.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
325
- "model.layers.37.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
326
- "model.layers.37.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
327
- "model.layers.37.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
328
- "model.layers.38.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
329
- "model.layers.38.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
330
- "model.layers.38.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
331
- "model.layers.38.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
332
- "model.layers.38.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
333
- "model.layers.38.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
334
- "model.layers.38.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
335
- "model.layers.38.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
336
- "model.layers.38.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
337
- "model.layers.38.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
338
- "model.layers.39.input_layernorm.weight": "pytorch_model-00003-of-00003.bin",
339
- "model.layers.39.mlp.down_proj.weight": "pytorch_model-00003-of-00003.bin",
340
- "model.layers.39.mlp.gate_proj.weight": "pytorch_model-00003-of-00003.bin",
341
- "model.layers.39.mlp.up_proj.weight": "pytorch_model-00003-of-00003.bin",
342
- "model.layers.39.post_attention_layernorm.weight": "pytorch_model-00003-of-00003.bin",
343
- "model.layers.39.self_attn.k_proj.weight": "pytorch_model-00003-of-00003.bin",
344
- "model.layers.39.self_attn.o_proj.weight": "pytorch_model-00003-of-00003.bin",
345
- "model.layers.39.self_attn.q_proj.weight": "pytorch_model-00003-of-00003.bin",
346
- "model.layers.39.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00003.bin",
347
- "model.layers.39.self_attn.v_proj.weight": "pytorch_model-00003-of-00003.bin",
348
- "model.layers.4.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
349
- "model.layers.4.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
350
- "model.layers.4.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
351
- "model.layers.4.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
352
- "model.layers.4.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
353
- "model.layers.4.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
354
- "model.layers.4.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
355
- "model.layers.4.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
356
- "model.layers.4.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
357
- "model.layers.4.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
358
- "model.layers.5.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
359
- "model.layers.5.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
360
- "model.layers.5.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
361
- "model.layers.5.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
362
- "model.layers.5.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
363
- "model.layers.5.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
364
- "model.layers.5.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
365
- "model.layers.5.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
366
- "model.layers.5.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
367
- "model.layers.5.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
368
- "model.layers.6.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
369
- "model.layers.6.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
370
- "model.layers.6.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
371
- "model.layers.6.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
372
- "model.layers.6.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
373
- "model.layers.6.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
374
- "model.layers.6.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
375
- "model.layers.6.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
376
- "model.layers.6.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
377
- "model.layers.6.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
378
- "model.layers.7.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
379
- "model.layers.7.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
380
- "model.layers.7.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
381
- "model.layers.7.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
382
- "model.layers.7.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
383
- "model.layers.7.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
384
- "model.layers.7.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
385
- "model.layers.7.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
386
- "model.layers.7.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
387
- "model.layers.7.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
388
- "model.layers.8.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
389
- "model.layers.8.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
390
- "model.layers.8.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
391
- "model.layers.8.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
392
- "model.layers.8.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
393
- "model.layers.8.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
394
- "model.layers.8.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
395
- "model.layers.8.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
396
- "model.layers.8.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
397
- "model.layers.8.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
398
- "model.layers.9.input_layernorm.weight": "pytorch_model-00001-of-00003.bin",
399
- "model.layers.9.mlp.down_proj.weight": "pytorch_model-00001-of-00003.bin",
400
- "model.layers.9.mlp.gate_proj.weight": "pytorch_model-00001-of-00003.bin",
401
- "model.layers.9.mlp.up_proj.weight": "pytorch_model-00001-of-00003.bin",
402
- "model.layers.9.post_attention_layernorm.weight": "pytorch_model-00001-of-00003.bin",
403
- "model.layers.9.self_attn.k_proj.weight": "pytorch_model-00001-of-00003.bin",
404
- "model.layers.9.self_attn.o_proj.weight": "pytorch_model-00001-of-00003.bin",
405
- "model.layers.9.self_attn.q_proj.weight": "pytorch_model-00001-of-00003.bin",
406
- "model.layers.9.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00003.bin",
407
- "model.layers.9.self_attn.v_proj.weight": "pytorch_model-00001-of-00003.bin",
408
- "model.norm.weight": "pytorch_model-00003-of-00003.bin"
409
  }
410
  }
 
1
  {
2
  "metadata": {
3
+ "total_size": 17578695680
4
  },
5
  "weight_map": {
6
+ "lm_head.weight": "pytorch_model-00010-of-00010.bin",
7
+ "model.embed_tokens.weight": "pytorch_model-00001-of-00010.bin",
8
+ "model.layers.0.input_layernorm.weight": "pytorch_model-00001-of-00010.bin",
9
+ "model.layers.0.mlp.down_proj.weight": "pytorch_model-00001-of-00010.bin",
10
+ "model.layers.0.mlp.gate_proj.weight": "pytorch_model-00001-of-00010.bin",
11
+ "model.layers.0.mlp.up_proj.weight": "pytorch_model-00001-of-00010.bin",
12
+ "model.layers.0.post_attention_layernorm.weight": "pytorch_model-00001-of-00010.bin",
13
+ "model.layers.0.self_attn.k_proj.weight": "pytorch_model-00001-of-00010.bin",
14
+ "model.layers.0.self_attn.o_proj.weight": "pytorch_model-00001-of-00010.bin",
15
+ "model.layers.0.self_attn.q_proj.weight": "pytorch_model-00001-of-00010.bin",
16
+ "model.layers.0.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00010.bin",
17
+ "model.layers.0.self_attn.v_proj.weight": "pytorch_model-00001-of-00010.bin",
18
+ "model.layers.1.input_layernorm.weight": "pytorch_model-00001-of-00010.bin",
19
+ "model.layers.1.mlp.down_proj.weight": "pytorch_model-00001-of-00010.bin",
20
+ "model.layers.1.mlp.gate_proj.weight": "pytorch_model-00001-of-00010.bin",
21
+ "model.layers.1.mlp.up_proj.weight": "pytorch_model-00001-of-00010.bin",
22
+ "model.layers.1.post_attention_layernorm.weight": "pytorch_model-00001-of-00010.bin",
23
+ "model.layers.1.self_attn.k_proj.weight": "pytorch_model-00001-of-00010.bin",
24
+ "model.layers.1.self_attn.o_proj.weight": "pytorch_model-00001-of-00010.bin",
25
+ "model.layers.1.self_attn.q_proj.weight": "pytorch_model-00001-of-00010.bin",
26
+ "model.layers.1.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00010.bin",
27
+ "model.layers.1.self_attn.v_proj.weight": "pytorch_model-00001-of-00010.bin",
28
+ "model.layers.10.input_layernorm.weight": "pytorch_model-00003-of-00010.bin",
29
+ "model.layers.10.mlp.down_proj.weight": "pytorch_model-00003-of-00010.bin",
30
+ "model.layers.10.mlp.gate_proj.weight": "pytorch_model-00003-of-00010.bin",
31
+ "model.layers.10.mlp.up_proj.weight": "pytorch_model-00003-of-00010.bin",
32
+ "model.layers.10.post_attention_layernorm.weight": "pytorch_model-00003-of-00010.bin",
33
+ "model.layers.10.self_attn.k_proj.weight": "pytorch_model-00003-of-00010.bin",
34
+ "model.layers.10.self_attn.o_proj.weight": "pytorch_model-00003-of-00010.bin",
35
+ "model.layers.10.self_attn.q_proj.weight": "pytorch_model-00003-of-00010.bin",
36
+ "model.layers.10.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00010.bin",
37
+ "model.layers.10.self_attn.v_proj.weight": "pytorch_model-00003-of-00010.bin",
38
+ "model.layers.11.input_layernorm.weight": "pytorch_model-00003-of-00010.bin",
39
+ "model.layers.11.mlp.down_proj.weight": "pytorch_model-00003-of-00010.bin",
40
+ "model.layers.11.mlp.gate_proj.weight": "pytorch_model-00003-of-00010.bin",
41
+ "model.layers.11.mlp.up_proj.weight": "pytorch_model-00003-of-00010.bin",
42
+ "model.layers.11.post_attention_layernorm.weight": "pytorch_model-00003-of-00010.bin",
43
+ "model.layers.11.self_attn.k_proj.weight": "pytorch_model-00003-of-00010.bin",
44
+ "model.layers.11.self_attn.o_proj.weight": "pytorch_model-00003-of-00010.bin",
45
+ "model.layers.11.self_attn.q_proj.weight": "pytorch_model-00003-of-00010.bin",
46
+ "model.layers.11.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00010.bin",
47
+ "model.layers.11.self_attn.v_proj.weight": "pytorch_model-00003-of-00010.bin",
48
+ "model.layers.12.input_layernorm.weight": "pytorch_model-00003-of-00010.bin",
49
+ "model.layers.12.mlp.down_proj.weight": "pytorch_model-00004-of-00010.bin",
50
+ "model.layers.12.mlp.gate_proj.weight": "pytorch_model-00004-of-00010.bin",
51
+ "model.layers.12.mlp.up_proj.weight": "pytorch_model-00004-of-00010.bin",
52
+ "model.layers.12.post_attention_layernorm.weight": "pytorch_model-00003-of-00010.bin",
53
+ "model.layers.12.self_attn.k_proj.weight": "pytorch_model-00003-of-00010.bin",
54
+ "model.layers.12.self_attn.o_proj.weight": "pytorch_model-00003-of-00010.bin",
55
+ "model.layers.12.self_attn.q_proj.weight": "pytorch_model-00003-of-00010.bin",
56
+ "model.layers.12.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00010.bin",
57
+ "model.layers.12.self_attn.v_proj.weight": "pytorch_model-00003-of-00010.bin",
58
+ "model.layers.13.input_layernorm.weight": "pytorch_model-00004-of-00010.bin",
59
+ "model.layers.13.mlp.down_proj.weight": "pytorch_model-00004-of-00010.bin",
60
+ "model.layers.13.mlp.gate_proj.weight": "pytorch_model-00004-of-00010.bin",
61
+ "model.layers.13.mlp.up_proj.weight": "pytorch_model-00004-of-00010.bin",
62
+ "model.layers.13.post_attention_layernorm.weight": "pytorch_model-00004-of-00010.bin",
63
+ "model.layers.13.self_attn.k_proj.weight": "pytorch_model-00004-of-00010.bin",
64
+ "model.layers.13.self_attn.o_proj.weight": "pytorch_model-00004-of-00010.bin",
65
+ "model.layers.13.self_attn.q_proj.weight": "pytorch_model-00004-of-00010.bin",
66
+ "model.layers.13.self_attn.rotary_emb.inv_freq": "pytorch_model-00004-of-00010.bin",
67
+ "model.layers.13.self_attn.v_proj.weight": "pytorch_model-00004-of-00010.bin",
68
+ "model.layers.14.input_layernorm.weight": "pytorch_model-00004-of-00010.bin",
69
+ "model.layers.14.mlp.down_proj.weight": "pytorch_model-00004-of-00010.bin",
70
+ "model.layers.14.mlp.gate_proj.weight": "pytorch_model-00004-of-00010.bin",
71
+ "model.layers.14.mlp.up_proj.weight": "pytorch_model-00004-of-00010.bin",
72
+ "model.layers.14.post_attention_layernorm.weight": "pytorch_model-00004-of-00010.bin",
73
+ "model.layers.14.self_attn.k_proj.weight": "pytorch_model-00004-of-00010.bin",
74
+ "model.layers.14.self_attn.o_proj.weight": "pytorch_model-00004-of-00010.bin",
75
+ "model.layers.14.self_attn.q_proj.weight": "pytorch_model-00004-of-00010.bin",
76
+ "model.layers.14.self_attn.rotary_emb.inv_freq": "pytorch_model-00004-of-00010.bin",
77
+ "model.layers.14.self_attn.v_proj.weight": "pytorch_model-00004-of-00010.bin",
78
+ "model.layers.15.input_layernorm.weight": "pytorch_model-00004-of-00010.bin",
79
+ "model.layers.15.mlp.down_proj.weight": "pytorch_model-00004-of-00010.bin",
80
+ "model.layers.15.mlp.gate_proj.weight": "pytorch_model-00004-of-00010.bin",
81
+ "model.layers.15.mlp.up_proj.weight": "pytorch_model-00004-of-00010.bin",
82
+ "model.layers.15.post_attention_layernorm.weight": "pytorch_model-00004-of-00010.bin",
83
+ "model.layers.15.self_attn.k_proj.weight": "pytorch_model-00004-of-00010.bin",
84
+ "model.layers.15.self_attn.o_proj.weight": "pytorch_model-00004-of-00010.bin",
85
+ "model.layers.15.self_attn.q_proj.weight": "pytorch_model-00004-of-00010.bin",
86
+ "model.layers.15.self_attn.rotary_emb.inv_freq": "pytorch_model-00004-of-00010.bin",
87
+ "model.layers.15.self_attn.v_proj.weight": "pytorch_model-00004-of-00010.bin",
88
+ "model.layers.16.input_layernorm.weight": "pytorch_model-00004-of-00010.bin",
89
+ "model.layers.16.mlp.down_proj.weight": "pytorch_model-00004-of-00010.bin",
90
+ "model.layers.16.mlp.gate_proj.weight": "pytorch_model-00004-of-00010.bin",
91
+ "model.layers.16.mlp.up_proj.weight": "pytorch_model-00004-of-00010.bin",
92
+ "model.layers.16.post_attention_layernorm.weight": "pytorch_model-00004-of-00010.bin",
93
+ "model.layers.16.self_attn.k_proj.weight": "pytorch_model-00004-of-00010.bin",
94
+ "model.layers.16.self_attn.o_proj.weight": "pytorch_model-00004-of-00010.bin",
95
+ "model.layers.16.self_attn.q_proj.weight": "pytorch_model-00004-of-00010.bin",
96
+ "model.layers.16.self_attn.rotary_emb.inv_freq": "pytorch_model-00004-of-00010.bin",
97
+ "model.layers.16.self_attn.v_proj.weight": "pytorch_model-00004-of-00010.bin",
98
+ "model.layers.17.input_layernorm.weight": "pytorch_model-00004-of-00010.bin",
99
+ "model.layers.17.mlp.down_proj.weight": "pytorch_model-00005-of-00010.bin",
100
+ "model.layers.17.mlp.gate_proj.weight": "pytorch_model-00005-of-00010.bin",
101
+ "model.layers.17.mlp.up_proj.weight": "pytorch_model-00005-of-00010.bin",
102
+ "model.layers.17.post_attention_layernorm.weight": "pytorch_model-00004-of-00010.bin",
103
+ "model.layers.17.self_attn.k_proj.weight": "pytorch_model-00004-of-00010.bin",
104
+ "model.layers.17.self_attn.o_proj.weight": "pytorch_model-00004-of-00010.bin",
105
+ "model.layers.17.self_attn.q_proj.weight": "pytorch_model-00004-of-00010.bin",
106
+ "model.layers.17.self_attn.rotary_emb.inv_freq": "pytorch_model-00004-of-00010.bin",
107
+ "model.layers.17.self_attn.v_proj.weight": "pytorch_model-00004-of-00010.bin",
108
+ "model.layers.18.input_layernorm.weight": "pytorch_model-00005-of-00010.bin",
109
+ "model.layers.18.mlp.down_proj.weight": "pytorch_model-00005-of-00010.bin",
110
+ "model.layers.18.mlp.gate_proj.weight": "pytorch_model-00005-of-00010.bin",
111
+ "model.layers.18.mlp.up_proj.weight": "pytorch_model-00005-of-00010.bin",
112
+ "model.layers.18.post_attention_layernorm.weight": "pytorch_model-00005-of-00010.bin",
113
+ "model.layers.18.self_attn.k_proj.weight": "pytorch_model-00005-of-00010.bin",
114
+ "model.layers.18.self_attn.o_proj.weight": "pytorch_model-00005-of-00010.bin",
115
+ "model.layers.18.self_attn.q_proj.weight": "pytorch_model-00005-of-00010.bin",
116
+ "model.layers.18.self_attn.rotary_emb.inv_freq": "pytorch_model-00005-of-00010.bin",
117
+ "model.layers.18.self_attn.v_proj.weight": "pytorch_model-00005-of-00010.bin",
118
+ "model.layers.19.input_layernorm.weight": "pytorch_model-00005-of-00010.bin",
119
+ "model.layers.19.mlp.down_proj.weight": "pytorch_model-00005-of-00010.bin",
120
+ "model.layers.19.mlp.gate_proj.weight": "pytorch_model-00005-of-00010.bin",
121
+ "model.layers.19.mlp.up_proj.weight": "pytorch_model-00005-of-00010.bin",
122
+ "model.layers.19.post_attention_layernorm.weight": "pytorch_model-00005-of-00010.bin",
123
+ "model.layers.19.self_attn.k_proj.weight": "pytorch_model-00005-of-00010.bin",
124
+ "model.layers.19.self_attn.o_proj.weight": "pytorch_model-00005-of-00010.bin",
125
+ "model.layers.19.self_attn.q_proj.weight": "pytorch_model-00005-of-00010.bin",
126
+ "model.layers.19.self_attn.rotary_emb.inv_freq": "pytorch_model-00005-of-00010.bin",
127
+ "model.layers.19.self_attn.v_proj.weight": "pytorch_model-00005-of-00010.bin",
128
+ "model.layers.2.input_layernorm.weight": "pytorch_model-00001-of-00010.bin",
129
+ "model.layers.2.mlp.down_proj.weight": "pytorch_model-00002-of-00010.bin",
130
+ "model.layers.2.mlp.gate_proj.weight": "pytorch_model-00002-of-00010.bin",
131
+ "model.layers.2.mlp.up_proj.weight": "pytorch_model-00002-of-00010.bin",
132
+ "model.layers.2.post_attention_layernorm.weight": "pytorch_model-00001-of-00010.bin",
133
+ "model.layers.2.self_attn.k_proj.weight": "pytorch_model-00001-of-00010.bin",
134
+ "model.layers.2.self_attn.o_proj.weight": "pytorch_model-00001-of-00010.bin",
135
+ "model.layers.2.self_attn.q_proj.weight": "pytorch_model-00001-of-00010.bin",
136
+ "model.layers.2.self_attn.rotary_emb.inv_freq": "pytorch_model-00001-of-00010.bin",
137
+ "model.layers.2.self_attn.v_proj.weight": "pytorch_model-00001-of-00010.bin",
138
+ "model.layers.20.input_layernorm.weight": "pytorch_model-00005-of-00010.bin",
139
+ "model.layers.20.mlp.down_proj.weight": "pytorch_model-00005-of-00010.bin",
140
+ "model.layers.20.mlp.gate_proj.weight": "pytorch_model-00005-of-00010.bin",
141
+ "model.layers.20.mlp.up_proj.weight": "pytorch_model-00005-of-00010.bin",
142
+ "model.layers.20.post_attention_layernorm.weight": "pytorch_model-00005-of-00010.bin",
143
+ "model.layers.20.self_attn.k_proj.weight": "pytorch_model-00005-of-00010.bin",
144
+ "model.layers.20.self_attn.o_proj.weight": "pytorch_model-00005-of-00010.bin",
145
+ "model.layers.20.self_attn.q_proj.weight": "pytorch_model-00005-of-00010.bin",
146
+ "model.layers.20.self_attn.rotary_emb.inv_freq": "pytorch_model-00005-of-00010.bin",
147
+ "model.layers.20.self_attn.v_proj.weight": "pytorch_model-00005-of-00010.bin",
148
+ "model.layers.21.input_layernorm.weight": "pytorch_model-00005-of-00010.bin",
149
+ "model.layers.21.mlp.down_proj.weight": "pytorch_model-00005-of-00010.bin",
150
+ "model.layers.21.mlp.gate_proj.weight": "pytorch_model-00005-of-00010.bin",
151
+ "model.layers.21.mlp.up_proj.weight": "pytorch_model-00005-of-00010.bin",
152
+ "model.layers.21.post_attention_layernorm.weight": "pytorch_model-00005-of-00010.bin",
153
+ "model.layers.21.self_attn.k_proj.weight": "pytorch_model-00005-of-00010.bin",
154
+ "model.layers.21.self_attn.o_proj.weight": "pytorch_model-00005-of-00010.bin",
155
+ "model.layers.21.self_attn.q_proj.weight": "pytorch_model-00005-of-00010.bin",
156
+ "model.layers.21.self_attn.rotary_emb.inv_freq": "pytorch_model-00005-of-00010.bin",
157
+ "model.layers.21.self_attn.v_proj.weight": "pytorch_model-00005-of-00010.bin",
158
+ "model.layers.22.input_layernorm.weight": "pytorch_model-00005-of-00010.bin",
159
+ "model.layers.22.mlp.down_proj.weight": "pytorch_model-00006-of-00010.bin",
160
+ "model.layers.22.mlp.gate_proj.weight": "pytorch_model-00006-of-00010.bin",
161
+ "model.layers.22.mlp.up_proj.weight": "pytorch_model-00006-of-00010.bin",
162
+ "model.layers.22.post_attention_layernorm.weight": "pytorch_model-00005-of-00010.bin",
163
+ "model.layers.22.self_attn.k_proj.weight": "pytorch_model-00005-of-00010.bin",
164
+ "model.layers.22.self_attn.o_proj.weight": "pytorch_model-00005-of-00010.bin",
165
+ "model.layers.22.self_attn.q_proj.weight": "pytorch_model-00005-of-00010.bin",
166
+ "model.layers.22.self_attn.rotary_emb.inv_freq": "pytorch_model-00005-of-00010.bin",
167
+ "model.layers.22.self_attn.v_proj.weight": "pytorch_model-00005-of-00010.bin",
168
+ "model.layers.23.input_layernorm.weight": "pytorch_model-00006-of-00010.bin",
169
+ "model.layers.23.mlp.down_proj.weight": "pytorch_model-00006-of-00010.bin",
170
+ "model.layers.23.mlp.gate_proj.weight": "pytorch_model-00006-of-00010.bin",
171
+ "model.layers.23.mlp.up_proj.weight": "pytorch_model-00006-of-00010.bin",
172
+ "model.layers.23.post_attention_layernorm.weight": "pytorch_model-00006-of-00010.bin",
173
+ "model.layers.23.self_attn.k_proj.weight": "pytorch_model-00006-of-00010.bin",
174
+ "model.layers.23.self_attn.o_proj.weight": "pytorch_model-00006-of-00010.bin",
175
+ "model.layers.23.self_attn.q_proj.weight": "pytorch_model-00006-of-00010.bin",
176
+ "model.layers.23.self_attn.rotary_emb.inv_freq": "pytorch_model-00006-of-00010.bin",
177
+ "model.layers.23.self_attn.v_proj.weight": "pytorch_model-00006-of-00010.bin",
178
+ "model.layers.24.input_layernorm.weight": "pytorch_model-00006-of-00010.bin",
179
+ "model.layers.24.mlp.down_proj.weight": "pytorch_model-00006-of-00010.bin",
180
+ "model.layers.24.mlp.gate_proj.weight": "pytorch_model-00006-of-00010.bin",
181
+ "model.layers.24.mlp.up_proj.weight": "pytorch_model-00006-of-00010.bin",
182
+ "model.layers.24.post_attention_layernorm.weight": "pytorch_model-00006-of-00010.bin",
183
+ "model.layers.24.self_attn.k_proj.weight": "pytorch_model-00006-of-00010.bin",
184
+ "model.layers.24.self_attn.o_proj.weight": "pytorch_model-00006-of-00010.bin",
185
+ "model.layers.24.self_attn.q_proj.weight": "pytorch_model-00006-of-00010.bin",
186
+ "model.layers.24.self_attn.rotary_emb.inv_freq": "pytorch_model-00006-of-00010.bin",
187
+ "model.layers.24.self_attn.v_proj.weight": "pytorch_model-00006-of-00010.bin",
188
+ "model.layers.25.input_layernorm.weight": "pytorch_model-00006-of-00010.bin",
189
+ "model.layers.25.mlp.down_proj.weight": "pytorch_model-00006-of-00010.bin",
190
+ "model.layers.25.mlp.gate_proj.weight": "pytorch_model-00006-of-00010.bin",
191
+ "model.layers.25.mlp.up_proj.weight": "pytorch_model-00006-of-00010.bin",
192
+ "model.layers.25.post_attention_layernorm.weight": "pytorch_model-00006-of-00010.bin",
193
+ "model.layers.25.self_attn.k_proj.weight": "pytorch_model-00006-of-00010.bin",
194
+ "model.layers.25.self_attn.o_proj.weight": "pytorch_model-00006-of-00010.bin",
195
+ "model.layers.25.self_attn.q_proj.weight": "pytorch_model-00006-of-00010.bin",
196
+ "model.layers.25.self_attn.rotary_emb.inv_freq": "pytorch_model-00006-of-00010.bin",
197
+ "model.layers.25.self_attn.v_proj.weight": "pytorch_model-00006-of-00010.bin",
198
+ "model.layers.26.input_layernorm.weight": "pytorch_model-00006-of-00010.bin",
199
+ "model.layers.26.mlp.down_proj.weight": "pytorch_model-00006-of-00010.bin",
200
+ "model.layers.26.mlp.gate_proj.weight": "pytorch_model-00006-of-00010.bin",
201
+ "model.layers.26.mlp.up_proj.weight": "pytorch_model-00006-of-00010.bin",
202
+ "model.layers.26.post_attention_layernorm.weight": "pytorch_model-00006-of-00010.bin",
203
+ "model.layers.26.self_attn.k_proj.weight": "pytorch_model-00006-of-00010.bin",
204
+ "model.layers.26.self_attn.o_proj.weight": "pytorch_model-00006-of-00010.bin",
205
+ "model.layers.26.self_attn.q_proj.weight": "pytorch_model-00006-of-00010.bin",
206
+ "model.layers.26.self_attn.rotary_emb.inv_freq": "pytorch_model-00006-of-00010.bin",
207
+ "model.layers.26.self_attn.v_proj.weight": "pytorch_model-00006-of-00010.bin",
208
+ "model.layers.27.input_layernorm.weight": "pytorch_model-00006-of-00010.bin",
209
+ "model.layers.27.mlp.down_proj.weight": "pytorch_model-00007-of-00010.bin",
210
+ "model.layers.27.mlp.gate_proj.weight": "pytorch_model-00007-of-00010.bin",
211
+ "model.layers.27.mlp.up_proj.weight": "pytorch_model-00007-of-00010.bin",
212
+ "model.layers.27.post_attention_layernorm.weight": "pytorch_model-00006-of-00010.bin",
213
+ "model.layers.27.self_attn.k_proj.weight": "pytorch_model-00006-of-00010.bin",
214
+ "model.layers.27.self_attn.o_proj.weight": "pytorch_model-00006-of-00010.bin",
215
+ "model.layers.27.self_attn.q_proj.weight": "pytorch_model-00006-of-00010.bin",
216
+ "model.layers.27.self_attn.rotary_emb.inv_freq": "pytorch_model-00006-of-00010.bin",
217
+ "model.layers.27.self_attn.v_proj.weight": "pytorch_model-00006-of-00010.bin",
218
+ "model.layers.28.input_layernorm.weight": "pytorch_model-00007-of-00010.bin",
219
+ "model.layers.28.mlp.down_proj.weight": "pytorch_model-00007-of-00010.bin",
220
+ "model.layers.28.mlp.gate_proj.weight": "pytorch_model-00007-of-00010.bin",
221
+ "model.layers.28.mlp.up_proj.weight": "pytorch_model-00007-of-00010.bin",
222
+ "model.layers.28.post_attention_layernorm.weight": "pytorch_model-00007-of-00010.bin",
223
+ "model.layers.28.self_attn.k_proj.weight": "pytorch_model-00007-of-00010.bin",
224
+ "model.layers.28.self_attn.o_proj.weight": "pytorch_model-00007-of-00010.bin",
225
+ "model.layers.28.self_attn.q_proj.weight": "pytorch_model-00007-of-00010.bin",
226
+ "model.layers.28.self_attn.rotary_emb.inv_freq": "pytorch_model-00007-of-00010.bin",
227
+ "model.layers.28.self_attn.v_proj.weight": "pytorch_model-00007-of-00010.bin",
228
+ "model.layers.29.input_layernorm.weight": "pytorch_model-00007-of-00010.bin",
229
+ "model.layers.29.mlp.down_proj.weight": "pytorch_model-00007-of-00010.bin",
230
+ "model.layers.29.mlp.gate_proj.weight": "pytorch_model-00007-of-00010.bin",
231
+ "model.layers.29.mlp.up_proj.weight": "pytorch_model-00007-of-00010.bin",
232
+ "model.layers.29.post_attention_layernorm.weight": "pytorch_model-00007-of-00010.bin",
233
+ "model.layers.29.self_attn.k_proj.weight": "pytorch_model-00007-of-00010.bin",
234
+ "model.layers.29.self_attn.o_proj.weight": "pytorch_model-00007-of-00010.bin",
235
+ "model.layers.29.self_attn.q_proj.weight": "pytorch_model-00007-of-00010.bin",
236
+ "model.layers.29.self_attn.rotary_emb.inv_freq": "pytorch_model-00007-of-00010.bin",
237
+ "model.layers.29.self_attn.v_proj.weight": "pytorch_model-00007-of-00010.bin",
238
+ "model.layers.3.input_layernorm.weight": "pytorch_model-00002-of-00010.bin",
239
+ "model.layers.3.mlp.down_proj.weight": "pytorch_model-00002-of-00010.bin",
240
+ "model.layers.3.mlp.gate_proj.weight": "pytorch_model-00002-of-00010.bin",
241
+ "model.layers.3.mlp.up_proj.weight": "pytorch_model-00002-of-00010.bin",
242
+ "model.layers.3.post_attention_layernorm.weight": "pytorch_model-00002-of-00010.bin",
243
+ "model.layers.3.self_attn.k_proj.weight": "pytorch_model-00002-of-00010.bin",
244
+ "model.layers.3.self_attn.o_proj.weight": "pytorch_model-00002-of-00010.bin",
245
+ "model.layers.3.self_attn.q_proj.weight": "pytorch_model-00002-of-00010.bin",
246
+ "model.layers.3.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00010.bin",
247
+ "model.layers.3.self_attn.v_proj.weight": "pytorch_model-00002-of-00010.bin",
248
+ "model.layers.30.input_layernorm.weight": "pytorch_model-00007-of-00010.bin",
249
+ "model.layers.30.mlp.down_proj.weight": "pytorch_model-00007-of-00010.bin",
250
+ "model.layers.30.mlp.gate_proj.weight": "pytorch_model-00007-of-00010.bin",
251
+ "model.layers.30.mlp.up_proj.weight": "pytorch_model-00007-of-00010.bin",
252
+ "model.layers.30.post_attention_layernorm.weight": "pytorch_model-00007-of-00010.bin",
253
+ "model.layers.30.self_attn.k_proj.weight": "pytorch_model-00007-of-00010.bin",
254
+ "model.layers.30.self_attn.o_proj.weight": "pytorch_model-00007-of-00010.bin",
255
+ "model.layers.30.self_attn.q_proj.weight": "pytorch_model-00007-of-00010.bin",
256
+ "model.layers.30.self_attn.rotary_emb.inv_freq": "pytorch_model-00007-of-00010.bin",
257
+ "model.layers.30.self_attn.v_proj.weight": "pytorch_model-00007-of-00010.bin",
258
+ "model.layers.31.input_layernorm.weight": "pytorch_model-00007-of-00010.bin",
259
+ "model.layers.31.mlp.down_proj.weight": "pytorch_model-00007-of-00010.bin",
260
+ "model.layers.31.mlp.gate_proj.weight": "pytorch_model-00007-of-00010.bin",
261
+ "model.layers.31.mlp.up_proj.weight": "pytorch_model-00007-of-00010.bin",
262
+ "model.layers.31.post_attention_layernorm.weight": "pytorch_model-00007-of-00010.bin",
263
+ "model.layers.31.self_attn.k_proj.weight": "pytorch_model-00007-of-00010.bin",
264
+ "model.layers.31.self_attn.o_proj.weight": "pytorch_model-00007-of-00010.bin",
265
+ "model.layers.31.self_attn.q_proj.weight": "pytorch_model-00007-of-00010.bin",
266
+ "model.layers.31.self_attn.rotary_emb.inv_freq": "pytorch_model-00007-of-00010.bin",
267
+ "model.layers.31.self_attn.v_proj.weight": "pytorch_model-00007-of-00010.bin",
268
+ "model.layers.32.input_layernorm.weight": "pytorch_model-00007-of-00010.bin",
269
+ "model.layers.32.mlp.down_proj.weight": "pytorch_model-00008-of-00010.bin",
270
+ "model.layers.32.mlp.gate_proj.weight": "pytorch_model-00008-of-00010.bin",
271
+ "model.layers.32.mlp.up_proj.weight": "pytorch_model-00008-of-00010.bin",
272
+ "model.layers.32.post_attention_layernorm.weight": "pytorch_model-00007-of-00010.bin",
273
+ "model.layers.32.self_attn.k_proj.weight": "pytorch_model-00007-of-00010.bin",
274
+ "model.layers.32.self_attn.o_proj.weight": "pytorch_model-00007-of-00010.bin",
275
+ "model.layers.32.self_attn.q_proj.weight": "pytorch_model-00007-of-00010.bin",
276
+ "model.layers.32.self_attn.rotary_emb.inv_freq": "pytorch_model-00007-of-00010.bin",
277
+ "model.layers.32.self_attn.v_proj.weight": "pytorch_model-00007-of-00010.bin",
278
+ "model.layers.33.input_layernorm.weight": "pytorch_model-00008-of-00010.bin",
279
+ "model.layers.33.mlp.down_proj.weight": "pytorch_model-00008-of-00010.bin",
280
+ "model.layers.33.mlp.gate_proj.weight": "pytorch_model-00008-of-00010.bin",
281
+ "model.layers.33.mlp.up_proj.weight": "pytorch_model-00008-of-00010.bin",
282
+ "model.layers.33.post_attention_layernorm.weight": "pytorch_model-00008-of-00010.bin",
283
+ "model.layers.33.self_attn.k_proj.weight": "pytorch_model-00008-of-00010.bin",
284
+ "model.layers.33.self_attn.o_proj.weight": "pytorch_model-00008-of-00010.bin",
285
+ "model.layers.33.self_attn.q_proj.weight": "pytorch_model-00008-of-00010.bin",
286
+ "model.layers.33.self_attn.rotary_emb.inv_freq": "pytorch_model-00008-of-00010.bin",
287
+ "model.layers.33.self_attn.v_proj.weight": "pytorch_model-00008-of-00010.bin",
288
+ "model.layers.34.input_layernorm.weight": "pytorch_model-00008-of-00010.bin",
289
+ "model.layers.34.mlp.down_proj.weight": "pytorch_model-00008-of-00010.bin",
290
+ "model.layers.34.mlp.gate_proj.weight": "pytorch_model-00008-of-00010.bin",
291
+ "model.layers.34.mlp.up_proj.weight": "pytorch_model-00008-of-00010.bin",
292
+ "model.layers.34.post_attention_layernorm.weight": "pytorch_model-00008-of-00010.bin",
293
+ "model.layers.34.self_attn.k_proj.weight": "pytorch_model-00008-of-00010.bin",
294
+ "model.layers.34.self_attn.o_proj.weight": "pytorch_model-00008-of-00010.bin",
295
+ "model.layers.34.self_attn.q_proj.weight": "pytorch_model-00008-of-00010.bin",
296
+ "model.layers.34.self_attn.rotary_emb.inv_freq": "pytorch_model-00008-of-00010.bin",
297
+ "model.layers.34.self_attn.v_proj.weight": "pytorch_model-00008-of-00010.bin",
298
+ "model.layers.35.input_layernorm.weight": "pytorch_model-00008-of-00010.bin",
299
+ "model.layers.35.mlp.down_proj.weight": "pytorch_model-00008-of-00010.bin",
300
+ "model.layers.35.mlp.gate_proj.weight": "pytorch_model-00008-of-00010.bin",
301
+ "model.layers.35.mlp.up_proj.weight": "pytorch_model-00008-of-00010.bin",
302
+ "model.layers.35.post_attention_layernorm.weight": "pytorch_model-00008-of-00010.bin",
303
+ "model.layers.35.self_attn.k_proj.weight": "pytorch_model-00008-of-00010.bin",
304
+ "model.layers.35.self_attn.o_proj.weight": "pytorch_model-00008-of-00010.bin",
305
+ "model.layers.35.self_attn.q_proj.weight": "pytorch_model-00008-of-00010.bin",
306
+ "model.layers.35.self_attn.rotary_emb.inv_freq": "pytorch_model-00008-of-00010.bin",
307
+ "model.layers.35.self_attn.v_proj.weight": "pytorch_model-00008-of-00010.bin",
308
+ "model.layers.36.input_layernorm.weight": "pytorch_model-00008-of-00010.bin",
309
+ "model.layers.36.mlp.down_proj.weight": "pytorch_model-00008-of-00010.bin",
310
+ "model.layers.36.mlp.gate_proj.weight": "pytorch_model-00008-of-00010.bin",
311
+ "model.layers.36.mlp.up_proj.weight": "pytorch_model-00008-of-00010.bin",
312
+ "model.layers.36.post_attention_layernorm.weight": "pytorch_model-00008-of-00010.bin",
313
+ "model.layers.36.self_attn.k_proj.weight": "pytorch_model-00008-of-00010.bin",
314
+ "model.layers.36.self_attn.o_proj.weight": "pytorch_model-00008-of-00010.bin",
315
+ "model.layers.36.self_attn.q_proj.weight": "pytorch_model-00008-of-00010.bin",
316
+ "model.layers.36.self_attn.rotary_emb.inv_freq": "pytorch_model-00008-of-00010.bin",
317
+ "model.layers.36.self_attn.v_proj.weight": "pytorch_model-00008-of-00010.bin",
318
+ "model.layers.37.input_layernorm.weight": "pytorch_model-00008-of-00010.bin",
319
+ "model.layers.37.mlp.down_proj.weight": "pytorch_model-00009-of-00010.bin",
320
+ "model.layers.37.mlp.gate_proj.weight": "pytorch_model-00009-of-00010.bin",
321
+ "model.layers.37.mlp.up_proj.weight": "pytorch_model-00009-of-00010.bin",
322
+ "model.layers.37.post_attention_layernorm.weight": "pytorch_model-00008-of-00010.bin",
323
+ "model.layers.37.self_attn.k_proj.weight": "pytorch_model-00008-of-00010.bin",
324
+ "model.layers.37.self_attn.o_proj.weight": "pytorch_model-00008-of-00010.bin",
325
+ "model.layers.37.self_attn.q_proj.weight": "pytorch_model-00008-of-00010.bin",
326
+ "model.layers.37.self_attn.rotary_emb.inv_freq": "pytorch_model-00008-of-00010.bin",
327
+ "model.layers.37.self_attn.v_proj.weight": "pytorch_model-00008-of-00010.bin",
328
+ "model.layers.38.input_layernorm.weight": "pytorch_model-00009-of-00010.bin",
329
+ "model.layers.38.mlp.down_proj.weight": "pytorch_model-00009-of-00010.bin",
330
+ "model.layers.38.mlp.gate_proj.weight": "pytorch_model-00009-of-00010.bin",
331
+ "model.layers.38.mlp.up_proj.weight": "pytorch_model-00009-of-00010.bin",
332
+ "model.layers.38.post_attention_layernorm.weight": "pytorch_model-00009-of-00010.bin",
333
+ "model.layers.38.self_attn.k_proj.weight": "pytorch_model-00009-of-00010.bin",
334
+ "model.layers.38.self_attn.o_proj.weight": "pytorch_model-00009-of-00010.bin",
335
+ "model.layers.38.self_attn.q_proj.weight": "pytorch_model-00009-of-00010.bin",
336
+ "model.layers.38.self_attn.rotary_emb.inv_freq": "pytorch_model-00009-of-00010.bin",
337
+ "model.layers.38.self_attn.v_proj.weight": "pytorch_model-00009-of-00010.bin",
338
+ "model.layers.39.input_layernorm.weight": "pytorch_model-00009-of-00010.bin",
339
+ "model.layers.39.mlp.down_proj.weight": "pytorch_model-00009-of-00010.bin",
340
+ "model.layers.39.mlp.gate_proj.weight": "pytorch_model-00009-of-00010.bin",
341
+ "model.layers.39.mlp.up_proj.weight": "pytorch_model-00009-of-00010.bin",
342
+ "model.layers.39.post_attention_layernorm.weight": "pytorch_model-00009-of-00010.bin",
343
+ "model.layers.39.self_attn.k_proj.weight": "pytorch_model-00009-of-00010.bin",
344
+ "model.layers.39.self_attn.o_proj.weight": "pytorch_model-00009-of-00010.bin",
345
+ "model.layers.39.self_attn.q_proj.weight": "pytorch_model-00009-of-00010.bin",
346
+ "model.layers.39.self_attn.rotary_emb.inv_freq": "pytorch_model-00009-of-00010.bin",
347
+ "model.layers.39.self_attn.v_proj.weight": "pytorch_model-00009-of-00010.bin",
348
+ "model.layers.4.input_layernorm.weight": "pytorch_model-00002-of-00010.bin",
349
+ "model.layers.4.mlp.down_proj.weight": "pytorch_model-00002-of-00010.bin",
350
+ "model.layers.4.mlp.gate_proj.weight": "pytorch_model-00002-of-00010.bin",
351
+ "model.layers.4.mlp.up_proj.weight": "pytorch_model-00002-of-00010.bin",
352
+ "model.layers.4.post_attention_layernorm.weight": "pytorch_model-00002-of-00010.bin",
353
+ "model.layers.4.self_attn.k_proj.weight": "pytorch_model-00002-of-00010.bin",
354
+ "model.layers.4.self_attn.o_proj.weight": "pytorch_model-00002-of-00010.bin",
355
+ "model.layers.4.self_attn.q_proj.weight": "pytorch_model-00002-of-00010.bin",
356
+ "model.layers.4.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00010.bin",
357
+ "model.layers.4.self_attn.v_proj.weight": "pytorch_model-00002-of-00010.bin",
358
+ "model.layers.5.input_layernorm.weight": "pytorch_model-00002-of-00010.bin",
359
+ "model.layers.5.mlp.down_proj.weight": "pytorch_model-00002-of-00010.bin",
360
+ "model.layers.5.mlp.gate_proj.weight": "pytorch_model-00002-of-00010.bin",
361
+ "model.layers.5.mlp.up_proj.weight": "pytorch_model-00002-of-00010.bin",
362
+ "model.layers.5.post_attention_layernorm.weight": "pytorch_model-00002-of-00010.bin",
363
+ "model.layers.5.self_attn.k_proj.weight": "pytorch_model-00002-of-00010.bin",
364
+ "model.layers.5.self_attn.o_proj.weight": "pytorch_model-00002-of-00010.bin",
365
+ "model.layers.5.self_attn.q_proj.weight": "pytorch_model-00002-of-00010.bin",
366
+ "model.layers.5.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00010.bin",
367
+ "model.layers.5.self_attn.v_proj.weight": "pytorch_model-00002-of-00010.bin",
368
+ "model.layers.6.input_layernorm.weight": "pytorch_model-00002-of-00010.bin",
369
+ "model.layers.6.mlp.down_proj.weight": "pytorch_model-00002-of-00010.bin",
370
+ "model.layers.6.mlp.gate_proj.weight": "pytorch_model-00002-of-00010.bin",
371
+ "model.layers.6.mlp.up_proj.weight": "pytorch_model-00002-of-00010.bin",
372
+ "model.layers.6.post_attention_layernorm.weight": "pytorch_model-00002-of-00010.bin",
373
+ "model.layers.6.self_attn.k_proj.weight": "pytorch_model-00002-of-00010.bin",
374
+ "model.layers.6.self_attn.o_proj.weight": "pytorch_model-00002-of-00010.bin",
375
+ "model.layers.6.self_attn.q_proj.weight": "pytorch_model-00002-of-00010.bin",
376
+ "model.layers.6.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00010.bin",
377
+ "model.layers.6.self_attn.v_proj.weight": "pytorch_model-00002-of-00010.bin",
378
+ "model.layers.7.input_layernorm.weight": "pytorch_model-00002-of-00010.bin",
379
+ "model.layers.7.mlp.down_proj.weight": "pytorch_model-00003-of-00010.bin",
380
+ "model.layers.7.mlp.gate_proj.weight": "pytorch_model-00003-of-00010.bin",
381
+ "model.layers.7.mlp.up_proj.weight": "pytorch_model-00003-of-00010.bin",
382
+ "model.layers.7.post_attention_layernorm.weight": "pytorch_model-00002-of-00010.bin",
383
+ "model.layers.7.self_attn.k_proj.weight": "pytorch_model-00002-of-00010.bin",
384
+ "model.layers.7.self_attn.o_proj.weight": "pytorch_model-00002-of-00010.bin",
385
+ "model.layers.7.self_attn.q_proj.weight": "pytorch_model-00002-of-00010.bin",
386
+ "model.layers.7.self_attn.rotary_emb.inv_freq": "pytorch_model-00002-of-00010.bin",
387
+ "model.layers.7.self_attn.v_proj.weight": "pytorch_model-00002-of-00010.bin",
388
+ "model.layers.8.input_layernorm.weight": "pytorch_model-00003-of-00010.bin",
389
+ "model.layers.8.mlp.down_proj.weight": "pytorch_model-00003-of-00010.bin",
390
+ "model.layers.8.mlp.gate_proj.weight": "pytorch_model-00003-of-00010.bin",
391
+ "model.layers.8.mlp.up_proj.weight": "pytorch_model-00003-of-00010.bin",
392
+ "model.layers.8.post_attention_layernorm.weight": "pytorch_model-00003-of-00010.bin",
393
+ "model.layers.8.self_attn.k_proj.weight": "pytorch_model-00003-of-00010.bin",
394
+ "model.layers.8.self_attn.o_proj.weight": "pytorch_model-00003-of-00010.bin",
395
+ "model.layers.8.self_attn.q_proj.weight": "pytorch_model-00003-of-00010.bin",
396
+ "model.layers.8.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00010.bin",
397
+ "model.layers.8.self_attn.v_proj.weight": "pytorch_model-00003-of-00010.bin",
398
+ "model.layers.9.input_layernorm.weight": "pytorch_model-00003-of-00010.bin",
399
+ "model.layers.9.mlp.down_proj.weight": "pytorch_model-00003-of-00010.bin",
400
+ "model.layers.9.mlp.gate_proj.weight": "pytorch_model-00003-of-00010.bin",
401
+ "model.layers.9.mlp.up_proj.weight": "pytorch_model-00003-of-00010.bin",
402
+ "model.layers.9.post_attention_layernorm.weight": "pytorch_model-00003-of-00010.bin",
403
+ "model.layers.9.self_attn.k_proj.weight": "pytorch_model-00003-of-00010.bin",
404
+ "model.layers.9.self_attn.o_proj.weight": "pytorch_model-00003-of-00010.bin",
405
+ "model.layers.9.self_attn.q_proj.weight": "pytorch_model-00003-of-00010.bin",
406
+ "model.layers.9.self_attn.rotary_emb.inv_freq": "pytorch_model-00003-of-00010.bin",
407
+ "model.layers.9.self_attn.v_proj.weight": "pytorch_model-00003-of-00010.bin",
408
+ "model.norm.weight": "pytorch_model-00009-of-00010.bin"
409
  }
410
  }