Files changed (1) hide show
  1. README.md +117 -0
README.md CHANGED
@@ -24,6 +24,109 @@ prompt_template: 'Below is an instruction that describes a task. Write a respons
24
 
25
  '
26
  quantized_by: TheBloke
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
27
  ---
28
 
29
  <!-- header start -->
@@ -330,3 +433,17 @@ The training data primarily targets the knowledge level of medical students,
330
  which may result in limitations when addressing the needs of board-certified physicians.
331
  The model has not been tested in real-world applications, so its efficacy and accuracy are currently unknown.
332
  It should never be used as a substitute for a doctor's opinion and must be treated as a research tool only.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
24
 
25
  '
26
  quantized_by: TheBloke
27
+ model-index:
28
+ - name: medalpaca-13B-GPTQ-4bit
29
+ results:
30
+ - task:
31
+ type: text-generation
32
+ name: Text Generation
33
+ dataset:
34
+ name: AI2 Reasoning Challenge (25-Shot)
35
+ type: ai2_arc
36
+ config: ARC-Challenge
37
+ split: test
38
+ args:
39
+ num_few_shot: 25
40
+ metrics:
41
+ - type: acc_norm
42
+ value: 29.35
43
+ name: normalized accuracy
44
+ source:
45
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TheBloke/medalpaca-13B-GPTQ-4bit
46
+ name: Open LLM Leaderboard
47
+ - task:
48
+ type: text-generation
49
+ name: Text Generation
50
+ dataset:
51
+ name: HellaSwag (10-Shot)
52
+ type: hellaswag
53
+ split: validation
54
+ args:
55
+ num_few_shot: 10
56
+ metrics:
57
+ - type: acc_norm
58
+ value: 26.32
59
+ name: normalized accuracy
60
+ source:
61
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TheBloke/medalpaca-13B-GPTQ-4bit
62
+ name: Open LLM Leaderboard
63
+ - task:
64
+ type: text-generation
65
+ name: Text Generation
66
+ dataset:
67
+ name: MMLU (5-Shot)
68
+ type: cais/mmlu
69
+ config: all
70
+ split: test
71
+ args:
72
+ num_few_shot: 5
73
+ metrics:
74
+ - type: acc
75
+ value: 25.44
76
+ name: accuracy
77
+ source:
78
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TheBloke/medalpaca-13B-GPTQ-4bit
79
+ name: Open LLM Leaderboard
80
+ - task:
81
+ type: text-generation
82
+ name: Text Generation
83
+ dataset:
84
+ name: TruthfulQA (0-shot)
85
+ type: truthful_qa
86
+ config: multiple_choice
87
+ split: validation
88
+ args:
89
+ num_few_shot: 0
90
+ metrics:
91
+ - type: mc2
92
+ value: 49.51
93
+ source:
94
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TheBloke/medalpaca-13B-GPTQ-4bit
95
+ name: Open LLM Leaderboard
96
+ - task:
97
+ type: text-generation
98
+ name: Text Generation
99
+ dataset:
100
+ name: Winogrande (5-shot)
101
+ type: winogrande
102
+ config: winogrande_xl
103
+ split: validation
104
+ args:
105
+ num_few_shot: 5
106
+ metrics:
107
+ - type: acc
108
+ value: 53.12
109
+ name: accuracy
110
+ source:
111
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TheBloke/medalpaca-13B-GPTQ-4bit
112
+ name: Open LLM Leaderboard
113
+ - task:
114
+ type: text-generation
115
+ name: Text Generation
116
+ dataset:
117
+ name: GSM8k (5-shot)
118
+ type: gsm8k
119
+ config: main
120
+ split: test
121
+ args:
122
+ num_few_shot: 5
123
+ metrics:
124
+ - type: acc
125
+ value: 0.0
126
+ name: accuracy
127
+ source:
128
+ url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=TheBloke/medalpaca-13B-GPTQ-4bit
129
+ name: Open LLM Leaderboard
130
  ---
131
 
132
  <!-- header start -->
 
433
  which may result in limitations when addressing the needs of board-certified physicians.
434
  The model has not been tested in real-world applications, so its efficacy and accuracy are currently unknown.
435
  It should never be used as a substitute for a doctor's opinion and must be treated as a research tool only.
436
+
437
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
438
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_TheBloke__medalpaca-13B-GPTQ-4bit)
439
+
440
+ | Metric |Value|
441
+ |---------------------------------|----:|
442
+ |Avg. |30.62|
443
+ |AI2 Reasoning Challenge (25-Shot)|29.35|
444
+ |HellaSwag (10-Shot) |26.32|
445
+ |MMLU (5-Shot) |25.44|
446
+ |TruthfulQA (0-shot) |49.51|
447
+ |Winogrande (5-shot) |53.12|
448
+ |GSM8k (5-shot) | 0.00|
449
+