TinyEnsemble-3x1.1B-TinyMoE / mergekit_moe_config.yml
JoPmt's picture
Upload folder using huggingface_hub
c2e8be6 verified
raw
history blame
525 Bytes
base_model: cognitivecomputations/TinyDolphin-2.8-1.1b
gate_mode: cheap_embed
dtype: bfloat16
experts:
- source_model: cognitivecomputations/TinyDolphin-2.8-1.1b
positive_prompts: ["write", "explain", "summarize", "how", "what", "acting"]
- source_model: 78health/TinyLlama_1.1B-function-calling
positive_prompts: ["code", "python", "javascript", "programming", "script", "run", "create"]
- source_model: DaertML/TinyGauss-1.1B
positive_prompts: ["count", "math", "algorithm", "crypto", "logic", "reason"]