README.md

metadata

base_model: []
library_name: transformers
tags:
  - mergekit
  - merge

lobotollama369

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the passthrough merge method.

Models Merged

The following models were included in the merge:

• /scratch-4

Configuration

The following YAML configuration was used to produce this model:

dtype: bfloat16
merge_method: passthrough
slices:
- sources:
  - layer_range: [0, 29]
    model: /scratch-4
- sources:
  - layer_range: [30, 35]
    model: /scratch-4
- sources:
  - layer_range: [36, 40]
    model: /scratch-4
- sources:
  - layer_range: [41, 45]
    model: /scratch-4
- sources:
  - layer_range: [46, 49]
    model: /scratch-4
- sources:
  - layer_range: [50, 54]
    model: /scratch-4
- sources:
  - layer_range: [55, 59]
    model: /scratch-4
- sources:
  - layer_range: [60, 64]
    model: /scratch-4
- sources:
  - layer_range: [65, 69]
    model: /scratch-4
- sources:
  - layer_range: [70, 74]
    model: /scratch-4
- sources:
  - layer_range: [75, 79]
    model: /scratch-4
- sources:
  - layer_range: [80, 84]
    model: /scratch-4
- sources:
  - layer_range: [85, 126]
    model: /scratch-4