Edit model card

GGUF

Experimental RP-oriented MoE, the idea was to get a model that would be equal to or better than Mixtral 8x7B and it's finetunes in RP/ERP tasks.

There's:

Llama 3 SnowStorm v1.15B 4x8B

base_model: Sao10K_L3-8B-Stheno-v3.1
gate_mode: random
dtype: bfloat16
experts_per_token: 2
experts:
  - source_model: Nitral-AI_Poppy_Porpoise-1.0-L3-8B
  - source_model: NeverSleep_Llama-3-Lumimaid-8B-v0.1-OAS
  - source_model: openlynn_Llama-3-Soliloquy-8B-v2
  - source_model: Sao10K_L3-8B-Stheno-v3.1

Models used

Difference(from SnowStorm v1.0)

Vision

llama3_mmproj

image/png

Prompt format: Llama 3

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric Value
Avg. 68.01
AI2 Reasoning Challenge (25-Shot) 60.67
HellaSwag (10-Shot) 81.60
MMLU (5-Shot) 68.12
TruthfulQA (0-shot) 51.69
Winogrande (5-shot) 76.56
GSM8k (5-shot) 69.45
Downloads last month
17
Safetensors
Model size
24.9B params
Tensor type
BF16
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for xxx777xxxASD/L3-SnowStorm-v1.15-4x8B-B

Quantizations
4 models

Collection including xxx777xxxASD/L3-SnowStorm-v1.15-4x8B-B

Evaluation results