File size: 2,239 Bytes
562d547 92cfa71 562d547 92cfa71 562d547 b026eab 562d547 92cfa71 562d547 92cfa71 b026eab 562d547 92cfa71 562d547 92cfa71 b026eab 562d547 92cfa71 562d547 92cfa71 b026eab 562d547 92cfa71 562d547 92cfa71 b026eab 562d547 92cfa71 562d547 92cfa71 b026eab 562d547 92cfa71 562d547 92cfa71 b026eab 562d547 92cfa71 562d547 92cfa71 b026eab 92cfa71 562d547 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 |
---
license: apache-2.0
tags:
- merge
- mergekit
- lazymergekit
- deepseek-ai/deepseek-llm-7b-base
---
# Breeze-13B-32k-Base-v1_0
Breeze-13B-32k-Base-v1_0 is a merge of the following models using [mergekit](https://github.com/cg123/mergekit):
* [deepseek-ai/deepseek-llm-7b-base](https://huggingface.co/deepseek-ai/deepseek-llm-7b-base)
* [deepseek-ai/deepseek-llm-7b-base](https://huggingface.co/deepseek-ai/deepseek-llm-7b-base)
* [deepseek-ai/deepseek-llm-7b-base](https://huggingface.co/deepseek-ai/deepseek-llm-7b-base)
* [deepseek-ai/deepseek-llm-7b-base](https://huggingface.co/deepseek-ai/deepseek-llm-7b-base)
* [deepseek-ai/deepseek-llm-7b-base](https://huggingface.co/deepseek-ai/deepseek-llm-7b-base)
* [deepseek-ai/deepseek-llm-7b-base](https://huggingface.co/deepseek-ai/deepseek-llm-7b-base)
* [deepseek-ai/deepseek-llm-7b-base](https://huggingface.co/deepseek-ai/deepseek-llm-7b-base)
## 🧩 Configuration
```yaml
dtype: bfloat16
merge_method: linear
slices:
- sources:
- layer_range: [0, 8]
model: deepseek-ai/deepseek-llm-7b-base
- layer_range: [0, 8]
model: meta-llama/Meta-Llama-3-8B
parameters:
weight: 0
- sources:
- layer_range: [4, 12]
model: deepseek-ai/deepseek-llm-7b-base
- layer_range: [4, 12]
model: meta-llama/Meta-Llama-3-8B
parameters:
weight: 0
- sources:
- layer_range: [8, 16]
model: deepseek-ai/deepseek-llm-7b-base
- layer_range: [8, 16]
model: meta-llama/Meta-Llama-3-8B
parameters:
weight: 0
- sources:
- layer_range: [12, 20]
model: deepseek-ai/deepseek-llm-7b-base
- layer_range: [12, 20]
model: meta-llama/Meta-Llama-3-8B
parameters:
weight: 0
- sources:
- layer_range: [16, 24]
model: deepseek-ai/deepseek-llm-7b-base
- layer_range: [16, 24]
model: meta-llama/Meta-Llama-3-8B
parameters:
weight: 0
- sources:
- layer_range: [20, 28]
model: deepseek-ai/deepseek-llm-7b-base
- layer_range: [20, 28]
model: meta-llama/Meta-Llama-3-8B
parameters:
weight: 0
- sources:
- layer_range: [24, 32]
model: deepseek-ai/deepseek-llm-7b-base
- layer_range: [24, 32]
model: meta-llama/Meta-Llama-3-8B
parameters:
weight: 0
tokenizer_source: union
``` |