BigWeave Viable
Collection
All viable models from the BigWeave series, excluding quantized versions
•
7 items
•
Updated
YAML Metadata
Warning:
The pipeline tag "conversational" is not in the official list: text-classification, token-classification, table-question-answering, question-answering, zero-shot-classification, translation, summarization, feature-extraction, text-generation, text2text-generation, fill-mask, sentence-similarity, text-to-speech, text-to-audio, automatic-speech-recognition, audio-to-audio, audio-classification, voice-activity-detection, depth-estimation, image-classification, object-detection, image-segmentation, text-to-image, image-to-text, image-to-image, image-to-video, unconditional-image-generation, video-classification, reinforcement-learning, robotics, tabular-classification, tabular-regression, tabular-to-text, table-to-text, multiple-choice, text-retrieval, time-series-forecasting, text-to-video, image-text-to-text, visual-question-answering, document-question-answering, zero-shot-image-classification, graph-ml, mask-generation, zero-shot-object-detection, text-to-3d, image-to-3d, image-feature-extraction, other
BigWeave v15 103b
The BigWeave models aim to experimentally identify merge settings for increasing model performance. The version number merely tracks various attempts and is not a quality indicator. Only results demonstrating good performance are retained and shared.
Prompting Format
Mistral, Vicuna and Alpaca.
Merge process
This is a self-merge of 152334H/miqu-1-70b-sf. By conducting exl2 measurements, we identify the most relevant layers. These layers are then duplicated in pairs to ensure overlaps.
Merge configuration:
slices:
- sources:
- model: 152334H/miqu-1-70b-sf
layer_range: [0,3]
- sources:
- model: 152334H/miqu-1-70b-sf
layer_range: [1,5]
- sources:
- model: 152334H/miqu-1-70b-sf
layer_range: [3,7]
- sources:
- model: 152334H/miqu-1-70b-sf
layer_range: [5,9]
- sources:
- model: 152334H/miqu-1-70b-sf
layer_range: [7,18]
- sources:
- model: 152334H/miqu-1-70b-sf
layer_range: [16,21]
- sources:
- model: 152334H/miqu-1-70b-sf
layer_range: [19,27]
- sources:
- model: 152334H/miqu-1-70b-sf
layer_range: [25,30]
- sources:
- model: 152334H/miqu-1-70b-sf
layer_range: [28,32]
- sources:
- model: 152334H/miqu-1-70b-sf
layer_range: [30,34]
- sources:
- model: 152334H/miqu-1-70b-sf
layer_range: [32,36]
- sources:
- model: 152334H/miqu-1-70b-sf
layer_range: [34,38]
- sources:
- model: 152334H/miqu-1-70b-sf
layer_range: [36,40]
- sources:
- model: 152334H/miqu-1-70b-sf
layer_range: [38,42]
- sources:
- model: 152334H/miqu-1-70b-sf
layer_range: [40,44]
- sources:
- model: 152334H/miqu-1-70b-sf
layer_range: [42,46]
- sources:
- model: 152334H/miqu-1-70b-sf
layer_range: [44,48]
- sources:
- model: 152334H/miqu-1-70b-sf
layer_range: [46,51]
- sources:
- model: 152334H/miqu-1-70b-sf
layer_range: [49,77]
- sources:
- model: 152334H/miqu-1-70b-sf
layer_range: [75,79]
- sources:
- model: 152334H/miqu-1-70b-sf
layer_range: [77,80]
merge_method: passthrough
dtype: float16
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 71.67 |
| AI2 Reasoning Challenge (25-Shot) | 69.71 |
| HellaSwag (10-Shot) | 86.41 |
| MMLU (5-Shot) | 71.25 |
| TruthfulQA (0-shot) | 66.10 |
| Winogrande (5-shot) | 80.35 |
| GSM8k (5-shot) | 56.18 |
- Downloads last month
- 2,289
Collection including llmixer/BigWeave-v15-103b
Evaluation results
- normalized accuracy on AI2 Reasoning Challenge (25-Shot)test set Open LLM Leaderboard69.710
- normalized accuracy on HellaSwag (10-Shot)validation set Open LLM Leaderboard86.410
- accuracy on MMLU (5-Shot)test set Open LLM Leaderboard71.250
- mc2 on TruthfulQA (0-shot)validation set Open LLM Leaderboard66.100
- accuracy on Winogrande (5-shot)validation set Open LLM Leaderboard80.350
- accuracy on GSM8k (5-shot)test set Open LLM Leaderboard56.180