Novaciano/Qwen2.5-0.5B-Cliff

merge

This is a merge of pre-trained language models created using mergekit.

Merge Details

The Brothers

Abyss & Cliff

Both is the combination of the best models QWEN2.5-0.5B of the Open LLM Leaderscore.

Merge Method

This model was merged using the Arcee Fusion merge method using FlofloB/100k_fineweb_continued_pretraining_Qwen2.5-0.5B-Instruct_Unsloth_merged_16bit as a base.

Models Merged

The following models were included in the merge:

• CoolSpring/Qwen2-0.5B-Abyme-merge3

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: FlofloB/100k_fineweb_continued_pretraining_Qwen2.5-0.5B-Instruct_Unsloth_merged_16bit
  - model: CoolSpring/Qwen2-0.5B-Abyme-merge3
    parameters:
      density: 0.53
      weight: 0.6
merge_method: arcee_fusion
base_model: FlofloB/100k_fineweb_continued_pretraining_Qwen2.5-0.5B-Instruct_Unsloth_merged_16bit
tokenizer_source: union
parameters:
  int8_mask: true
dtype: bfloat16
random_seed: 0