MoE-Merging / .ipynb_checkpoints /mergekit_moe_config-checkpoint.yml
Cartinoe5930's picture
Upload folder using huggingface_hub
fcd0be7 verified
base_model: mistralai/Mistral-7B-Instruct-v0.2
dtype: float16
experts:
- source_model: mistralai/Mistral-7B-Instruct-v0.2
positive_prompts:
- "Provide clear and concise answers"
- "Understand the context of the question"
- "Give detailed explanations"
- "Understand and respond to questions"
- "Answer a wide range of questions"
- source_model: Open-Orca/Mistral-7B-OpenOrca
positive_prompts:
- "Can you answer questions"
- "Provide a detailed explanation"
- "Summarize the main points"
- "How does this information relate"
- "Tell me more about"
- source_model: openchat/openchat-3.5-0106
positive_prompts:
- "Assist with general conversation"
- "Provide information on"
- "Generate creative and engaging content"
- "Help with problem-solving and critical thinking"
- "Provide context and background information on various subjects"
- source_model: WizardLM/WizardMath-7B-V1.1
positive_prompts:
- "Solve this math problem"
- "Explain the solution process"
- "Provide a step-by-step breakdown"
- "Apply this mathematical theorem"
- "Simplify this complex math expression"