---
tags:
- merge
- mergekit
- lazymergekit
- btherien/JOB-3150994_410M_it-132366_tr-pile-train_scratch
License: apache-2.0
---

# 405M_TIES-merge_pile_300B_into_slimp_300B_from_pile_replay5_density-0.95

405M_TIES-merge_pile_300B_into_slimp_300B_from_pile_replay5_density-0.95 is a merge of the following models using [mergekit](https://github.com/cg123/mergekit):
* [btherien/JOB-3150994_410M_it-132366_tr-pile-train_scratch](https://huggingface.co/btherien/JOB-3150994_410M_it-132366_tr-pile-train_scratch)

## 🧩 Configuration

\```yamlmodels:
  - model: btherien/Model_-410M_It_-132366_Tr_-slim-pajama-300B-replay5_finetune
    # no parameters necessary for base model
  - model: btherien/JOB-3150994_410M_it-132366_tr-pile-train_scratch
    parameters:
      density: 0.95
      weight: 1.0
merge_method: ties
base_model: btherien/Model_-410M_It_-132366_Tr_-slim-pajama-300B-replay5_finetune
parameters:
  normalize: true
dtype: float16\```