Model Details
- Model Description: This model is test for data ordering.
- Developed by: Juhwan Lee
- Model Type: Large Language Model
Model Architecture
This model is based on Mistral-7B-v0.1. We fine-tuning this model for data ordering task.
Mistral-7B-v0.1 is a transformer model, with the following architecture choices:
- Grouped-Query Attention
- Sliding-Window Attention
- Byte-fallback BPE tokenizer
Dataset
We random sample Open-Orca dataset. (We finetune the 100,000 dataset)
Guthub
License
Apache License 2.0
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 62.16 |
| AI2 Reasoning Challenge (25-Shot) | 62.03 |
| HellaSwag (10-Shot) | 83.55 |
| MMLU (5-Shot) | 62.69 |
| TruthfulQA (0-shot) | 45.82 |
| Winogrande (5-shot) | 79.08 |
| GSM8k (5-shot) | 39.80 |
- Downloads last month
- 73
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.
Model tree for juhwanlee/llmdo-Mistral-7B-case-c-v1
Dataset used to train juhwanlee/llmdo-Mistral-7B-case-c-v1
Evaluation results
- normalized accuracy on AI2 Reasoning Challenge (25-Shot)test set Open LLM Leaderboard62.030
- normalized accuracy on HellaSwag (10-Shot)validation set Open LLM Leaderboard83.550
- accuracy on MMLU (5-Shot)test set Open LLM Leaderboard62.690
- mc2 on TruthfulQA (0-shot)validation set Open LLM Leaderboard45.820
- accuracy on Winogrande (5-shot)validation set Open LLM Leaderboard79.080
- accuracy on GSM8k (5-shot)test set Open LLM Leaderboard39.800