File size: 1,914 Bytes
25d4f53 c97317d 25d4f53 c97317d 25d4f53 c97317d |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 |
---
language:
- en
library_name: transformers
pipeline_tag: text-generation
datasets:
- teknium/OpenHermes-2.5
- TokenBender/python_eval_instruct_51k
- codefuse-ai/Evol-instruction-66k
tags:
- code
license: apache-2.0
model-index:
- name: SpeechlessCoder
results:
- task:
type: text-generation
dataset:
type: openai_humaneval
name: HumanEval
metrics:
- name: pass@1
type: pass@1
value: 0.0
verified: false
---
<p><h1> speechless-starcoder2-7b </h1></p>
Code: https://github.com/uukuguy/speechless
Use the following dataset to fine-tune bigcode/starcoder2-7b in order to improve the model's reasoning and planning abilities.
Total 986k samples.
- teknium/OpenHermes-2.5
- TokenBender/python_eval_instruct_51k
- Spider
- codefuse-ai/Evol-instruction-66k
## How to Prompt the Model
This model accepts the Alpaca instruction format.
For example:
```
You are an intelligent programming assistant.
### Instruction:
Implement a linked list in C++
### Response:
```
## HumanEval
| Metric | Value |
| --- | --- |
| humaneval-python | |
## lm-evaluation-harness
```json
{'ARC (acc_norm)': ,
'HellaSwag (acc_norm)': ,
'MMLU (acc)': ,
'TruthfulQA (mc2)': ,
'Winoground (acc)': ,
'GSM8K (acc)': ,
'DROP (f1)': ,
'Open LLM Score': }
```
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_uukuguy__speechless-starcoder2-7b)
| Metric | Value |
|-----------------------|---------------------------|
| Avg. | |
| ARC (25-shot) | |
| HellaSwag (10-shot) | |
| MMLU (5-shot) | |
| TruthfulQA (0-shot) | |
| Winogrande (5-shot) | |
| GSM8K (5-shot) | |
| DROP (3-shot) | |
|