metadata

license: llama2
language:
  - en
pipeline_tag: conversational
tags:
  - Xwin
  - WinterGoddess
  - frankenmerge
  - 120b

Xwinter 120B

A Goliath-120b style frankenmerge of Xwin-LM-70b-v0.1 and WinterGoddess-1.4x-70b. Meant as a slight update to Goliath by using the successor model to Euryale.

There's a similar merge called WinterGoliath-123b by @ChuckMcSneed.

Prompting Format

Vicuna and Alpaca.

Merge process

The models used in the merge are Xwin-LM-70b-v0.1 and WinterGoddess-1.4x-70b.

The layer mix:

- range 0, 16
  Xwin
- range 8, 24
  WinterGoddess
- range 17, 32
  Xwin
- range 25, 40
  WinterGoddess
- range 33, 48
  Xwin
- range 41, 56
  WinterGoddess
- range 49, 64
  Xwin
- range 57, 72
  WinterGoddess
- range 65, 80
  Xwin

Acknowledgements

@Xwin-LM For creating Xwin

@Sao10K For creating WinterGoddess

@alpindale For creating the original Goliath

@chargoddard For developing mergekit.