README.md

metadata

license: apache-2.0
tags:
  - merge
  - mergekit

tinyllama_frankenmerge

This model is a merge of the following models made with mergekit:

• TinyLlama/TinyLlama-1.1B-intermediate-step-1195k-token-2.5T
• TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T

🧩 Configuration

slices:
  - sources:
    - model: TinyLlama/TinyLlama-1.1B-intermediate-step-1195k-token-2.5T
      layer_range: [0, 16]
  - sources:
    - model: TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T
      layer_range: [6, 22]
merge_method: passthrough
dtype: float16```