SvdH/Vengeance-22B

Vengeance

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the DARE TIES merge method using SvdH/RPLament-22B as a base.

Models Merged

The following models were included in the merge:

• TheDrummer/Cydonia-22B-v1.3
• concedo/Beepo-22B

Configuration

The following YAML configuration was used to produce this model:

merge_method: dare_ties
base_model: SvdH/RPLament-22B
parameters:
  int8_mask: true
dtype: bfloat16
models:
  - model: SvdH/RPLament-22B
    use_cache: true
    parameters:
      weight: 0.60
      density: 0.78
  - model: TheDrummer/Cydonia-22B-v1.3
    use_cache: true
    parameters:
      weight: 0.20
      density: 0.66
  - model: concedo/Beepo-22B
    use_cache: true
    parameters:
      weight: 0.20
      density: 0.54