Hugging Face
Models
Datasets
Spaces
Posts
Docs
Solutions
Pricing
Log In
Sign Up
solidrust
/
MixtureofMerges-MoE-2x7b-SLERPv0.9-AWQ
like
0
Text Generation
Transformers
Safetensors
mixtral
4-bit precision
AWQ
Inference Endpoints
Merge
mergekit
lazymergekit
jsfs11/MixtureofMerges-MoE-2x7b-v7
jsfs11/MixtureofMerges-MoE-2x7bRP-v8
Eval Results
text-generation-inference
awq
License:
apache-2.0
Model card
Files
Files and versions
Community
Train
Deploy
Use this model
Edit model card
jsfs11/MixtureofMerges-MoE-2x7b-SLERPv0.9 AWQ
Model Summary
jsfs11/MixtureofMerges-MoE-2x7b-SLERPv0.9 AWQ
Model creator:
jsfs11
Original model:
MixtureofMerges-MoE-2x7b-SLERPv0.9
Model Summary
MixtureofMerges-MoE-2x7b-SLERPv0.9 is a merge of the following models using
LazyMergekit
:
jsfs11/MixtureofMerges-MoE-2x7b-v7
jsfs11/MixtureofMerges-MoE-2x7bRP-v8
Downloads last month
3
Safetensors
Model size
1.95B params
Tensor type
I32
·
FP16
·
Text Generation
Inference API (serverless) has been turned off for this model.
Merge of
jsfs11/MixtureofMerges-MoE-2x7b-v7
jsfs11/MixtureofMerges-MoE-2x7bRP-v8
Collection including
solidrust/MixtureofMerges-MoE-2x7b-SLERPv0.9-AWQ
2x7B AWQ
Collection
Mixture of experts 2 x 7B.
•
20 items
•
Updated
Apr 30
Evaluation results
normalized accuracy
on AI2 Reasoning Challenge (25-Shot)
test set
Open LLM Leaderboard
73.120
normalized accuracy
on HellaSwag (10-Shot)
validation set
Open LLM Leaderboard
88.760
accuracy
on MMLU (5-Shot)
test set
Open LLM Leaderboard
65.000
mc2
on TruthfulQA (0-shot)
validation set
Open LLM Leaderboard
74.830
accuracy
on Winogrande (5-shot)
validation set
Open LLM Leaderboard
83.580
accuracy
on GSM8k (5-shot)
test set
Open LLM Leaderboard
69.220
View on Papers With Code