metadata
base_model:
- codellama/CodeLlama-70b-hf
tags:
- mergekit
- merge
- code
license: mit
pipeline_tag: conversational
BigCodeLLama LFG 🚀
Experimental CodeLlaMA frankenstein to see how it benchmarks
Models Merged with base codellama/CodeLlama-70b-hf
The following models were included in the merge:
- ../CodeLlama-70b-hf
- ../CodeLlama-70b-Instruct-hf
- ../CodeLlama-70b-Python-hf
Configuration
The following YAML configuration was used to produce this model:
dtype: bfloat16
merge_method: passthrough
slices:
- sources:
- layer_range: [0, 69]
model:
model:
path: ../CodeLlama-70b-hf
- sources:
- layer_range: [66, 76]
model:
model:
path: ../CodeLlama-70b-Instruct-hf
- sources:
- layer_range: [42, 66]
model:
model:
path: ../CodeLlama-70b-hf
- sources:
- layer_range: [13, 37]
model:
model:
path: ../CodeLlama-70b-Python-hf
- sources:
- layer_range: [10, 80]
model:
model:
path: ../CodeLlama-70b-Instruct-hf