Llama 3 Experiments
Collection
My merging experiments for the Llama-3 series of models.
•
6 items
•
Updated
This is a self-merge of NousResearch/Hermes-2-Pro-Llama-3-8B mergekit. For the purposes of experimentation.
Another fail, need to read more about merging and look at more examples, and maybe try more with base models then SFT?
This model was merged using the passthrough merge method.
The following models were included in the merge:
The following YAML configuration was used to produce this model:
slices:
- sources:
- model: NousResearch/Hermes-2-Pro-Llama-3-8B
layer_range: [0, 32]
- sources:
- model: NousResearch/Hermes-2-Pro-Llama-3-8B
layer_range: [0, 32]
merge_method: passthrough
dtype: bfloat16
tokenizer_source: union