Sample Input on Postman API:
Number of epochs: 10 Number of Data points: 10000
Creative Writing: Write a question or instruction that requires a creative medical response from a doctor.
The instruction should be reasonable to ask of a person with general medical knowledge and should not require searching. In this task, your prompt should give very specific instructions to follow. Constraints, instructions, guidelines, or requirements all work, and the more of them the better.
Reference dataset: https://github.com/Kent0n-Li/ChatDoctor
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 50.70 |
| AI2 Reasoning Challenge (25-Shot) | 54.95 |
| HellaSwag (10-Shot) | 79.94 |
| MMLU (5-Shot) | 44.40 |
| TruthfulQA (0-shot) | 44.76 |
| Winogrande (5-shot) | 70.01 |
| GSM8k (5-shot) | 10.16 |
- Downloads last month
- 1,613
Dataset used to train vikash06/doctorLLM10k
Evaluation results
- normalized accuracy on AI2 Reasoning Challenge (25-Shot)test set Open LLM Leaderboard54.950
- normalized accuracy on HellaSwag (10-Shot)validation set Open LLM Leaderboard79.940
- accuracy on MMLU (5-Shot)test set Open LLM Leaderboard44.400
- mc2 on TruthfulQA (0-shot)validation set Open LLM Leaderboard44.760
- accuracy on Winogrande (5-shot)validation set Open LLM Leaderboard70.010
- accuracy on GSM8k (5-shot)test set Open LLM Leaderboard10.160