frankenllama-16b

This is a merge of pre-trained language models created using mergekit.

This is my personal frankenmerge of llama-3-8B after a light and unintentional tuning for 3 epochs on the noro dataset in Alpaca format. I have yet to test it or further tune it. So please don't take it too seriously.

Merge Details

Merge Method

This model was merged using the passthrough merge method.

Models Merged

The following models were included in the merge:

reallad/llama-3-8b-alpaca-noro

Configuration

The following YAML configuration was used to produce this model:

slices:
  - sources:
    - model: reallad/llama-3-8b-alpaca-noro
      layer_range: [0,32]
  - sources:
    - model: reallad/llama-3-8b-alpaca-noro
      layer_range: [0,32]
merge_method: passthrough
dtype: bfloat16
tokenizer_source: base