---
base_model:
- unsloth/Llama-3.2-3B-Instruct
- unsloth/Llama-3.2-3B
library_name: transformers
tags:
- mergekit
- merge

---

# Details

This is an experimental merge I plan to use for future projects, it shows promising results from my limited testing. Further testing should probably be done! I just don't have the time, nor compute right now.


### Configuration

The following YAML configuration was used to produce this model:

```yaml
models:
  - model: unsloth/Llama-3.2-3B
    parameters:
      weight: 0.5
      density: 0.7

  - model: unsloth/Llama-3.2-3B-Instruct 
    parameters:
      weight: 0.5
      density: 0.6

merge_method: ties
base_model: unsloth/Llama-3.2-3B
parameters:
  normalize: true
  int8_mask: true
dtype: bfloat16
tokenizer_source: unsloth/Llama-3.2-3B
```