WesPro

/

State-of-the-MoE_RP-2x7B

like 1

here that's the mergekits's config.yml (pretty much the MoE's recipe:

base_model: H:\merge\TheSpice-7b-FT-v0.3.1
gate_mode: random
dtype: bfloat16
experts:

• source_model: H:\merge\daybreak-kunoichi-2dpo-7b
  positive_prompts:
  • "play"
• source_model: H:\merge\Noromaid-7B-0.4-DPO
  positive_prompts:
  • "role"