Model Card for Model Mistral-7B-v0.1-5-over-16
I'm experimenting with the weight matrices in neural networks.
This is a clone of Mistral-7B-v0.1 with some weight matrices replaced.
I'm interested in seeing how the adjustmenets affect performance on existing metrics.
Model Details
Research in progress! Demons could come out of your nose if you use this.
Model Description
A modification of mistralai/Mistral-7B-v0.1.
Thanks to their team for sharing their model.
- Modified by: Dr. Alex W. Neal Riasanovsky
- Model type: pre-trained
- Language(s) (NLP): English
- License: Apache-2.0
Bias, Risks, and Limitations
Use your own risk.
I have no idea what this model's biases and limitations are.
I just want to see if the benchmark values are similar to those from Mistral-7B-v0.1.
I am setting up a long computational experiment to test some ideas.
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 12.16 |
| IFEval (0-Shot) | 21.18 |
| BBH (3-Shot) | 17.54 |
| MATH Lvl 5 (4-Shot) | 2.19 |
| GPQA (0-shot) | 4.14 |
| MuSR (0-shot) | 6.14 |
| MMLU-PRO (5-shot) | 21.75 |
- Downloads last month
- 8
Inference Providers
NEW
This model is not currently available via any of the supported third-party Inference Providers, and
the model is not deployed on the HF Inference API.
Evaluation results
- strict accuracy on IFEval (0-Shot)Open LLM Leaderboard21.180
- normalized accuracy on BBH (3-Shot)Open LLM Leaderboard17.540
- exact match on MATH Lvl 5 (4-Shot)Open LLM Leaderboard2.190
- acc_norm on GPQA (0-shot)Open LLM Leaderboard4.140
- acc_norm on MuSR (0-shot)Open LLM Leaderboard6.140
- accuracy on MMLU-PRO (5-shot)test set Open LLM Leaderboard21.750