File size: 5,851 Bytes
b0da50d afd6d45 b0da50d afd6d45 5d88647 4984a3c 5d88647 ce27591 4984a3c ce27591 4984a3c ce27591 4984a3c ce27591 afd6d45 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 |
---
language:
- en
license: apache-2.0
library_name: transformers
base_model:
- HuggingFaceH4/mistral-7b-anthropic
- ajibawa-2023/Code-Mistral-7B
- Undi95/BigL-7B
model-index:
- name: autocodit
results:
- task:
type: text-generation
name: Text Generation
dataset:
name: AI2 Reasoning Challenge (25-Shot)
type: ai2_arc
config: ARC-Challenge
split: test
args:
num_few_shot: 25
metrics:
- type: acc_norm
value: 66.38
name: normalized accuracy
source:
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=adowu/autocodit
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: HellaSwag (10-Shot)
type: hellaswag
split: validation
args:
num_few_shot: 10
metrics:
- type: acc_norm
value: 84.82
name: normalized accuracy
source:
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=adowu/autocodit
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: MMLU (5-Shot)
type: cais/mmlu
config: all
split: test
args:
num_few_shot: 5
metrics:
- type: acc
value: 65.09
name: accuracy
source:
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=adowu/autocodit
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: TruthfulQA (0-shot)
type: truthful_qa
config: multiple_choice
split: validation
args:
num_few_shot: 0
metrics:
- type: mc2
value: 59.95
source:
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=adowu/autocodit
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: Winogrande (5-shot)
type: winogrande
config: winogrande_xl
split: validation
args:
num_few_shot: 5
metrics:
- type: acc
value: 80.51
name: accuracy
source:
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=adowu/autocodit
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: GSM8k (5-shot)
type: gsm8k
config: main
split: test
args:
num_few_shot: 5
metrics:
- type: acc
value: 60.65
name: accuracy
source:
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=adowu/autocodit
name: Open LLM Leaderboard
---
# AUTOCODIT
# Description
This model represents an innovative fusion of three cutting-edge language models: **BigL-7B**, **Code-Mistral-7B**, and **mistral-7b-anthropic**, leveraging the strengths of each to create a more powerful and versatile tool. The integration process employs the TIES merge method, meticulously combining these models to enhance performance and adaptability across a broad spectrum of natural language processing tasks.
**Creation Process**
The model was crafted through a strategic merging process, utilizing the TIES merge method. This approach was chosen for its effectiveness in preserving the unique capabilities of each constituent model while ensuring seamless interoperability. The base model for this fusion was **HuggingFaceH4/mistral-7b-anthropic**, selected for its robust architecture and performance.
The merge parameters were carefully calibrated to achieve the optimal balance between the models, with the following configuration:
- **BigL-7B** was integrated with a density of 0.9 and a weight of 0.8, contributing its extensive language understanding and generation capabilities.
- **Code-Mistral-7B** was incorporated with a density of 0.7 and a weight of 0.7, enhancing the model's proficiency in code-related tasks and technical language comprehension.
- **mistral-7b-anthropic** served as the foundation, with its parameters set to a density of 0.9 and a weight of 0.8, ensuring the model's general language processing abilities remained at the forefront.
**Features**
- **Model Type:** `MistralForCausalLM`
- **Vocabulary Size:** 32,000 tokens, encompassing a wide array of linguistic elements for comprehensive language coverage.
- **Maximum Position Embeddings:** 32,768, facilitating the processing of extended passages of text.
- **Hidden Size:** 4,096, enabling the model to capture complex patterns and nuances in the data.
- **Num Attention Heads:** 32, allowing for detailed attention to various aspects of the input.
- **Num Hidden Layers:** 32, providing depth to the model's understanding and generation capabilities.
**Applications**
This model is adept at a wide range of natural language processing tasks, including but not limited to text generation, language translation, code synthesis, and more. Its unique blend of features from **BigL-7B**, **Code-Mistral-7B**, and **mistral-7b-anthropic** makes it particularly effective in scenarios requiring a deep understanding of both human and programming languages.
---
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_adowu__autocodit)
| Metric |Value|
|---------------------------------|----:|
|Avg. |69.57|
|AI2 Reasoning Challenge (25-Shot)|66.38|
|HellaSwag (10-Shot) |84.82|
|MMLU (5-Shot) |65.09|
|TruthfulQA (0-shot) |59.95|
|Winogrande (5-shot) |80.51|
|GSM8k (5-shot) |60.65|
|