ExplainCPE: A Free-text Explanation Benchmark of Chinese Pharmacist Examination
Authors: Dongfang Li
Jindi Yu
Baotian Hu
Zhenran Xu
Min Zhang
Venue: EMNLP 2023 Findings (2023)
Keywords: datasets
interpretability
large language model

Abstract: In the field of Large Language Models (LLMs), researchers are increasingly exploring their effectiveness across a wide range of tasks. 
However, a critical area that requires further investigation is the interpretability of these models, particularly the ability to generate rational explanations for their decisions. Most existing explanation datasets are limited to the English language and the general domain, which leads to a scarcity of linguistic diversity and a lack of resources in specialized domains, such as medical. To mitigate this, we propose ExplainCPE, a challenging medical dataset consisting of over 7K problems from Chinese Pharmacist Examination, specifically tailored to assess the model-generated explanations. From the overall results, only GPT-4 passes the pharmacist examination with a 75.7\% accuracy, while other models like ChatGPT fail. Further detailed analysis of LLM-generated explanations reveals the limitations of LLMs in understanding medical text and executing computational reasoning. With the increasing importance of AI safety and trustworthiness, ExplainCPE takes a step towards improving and evaluating the interpretability of LLMs in the medical domain.

ExplainCPE: A Free-text Explanation Benchmark of Chinese Pharmacist Examination
Dongfang Li", Jindi Yu,, Baotian Hu', Zhenran Xu and Min Zhang
Harbin Institute of Technology (Shenzhen), Shenzhen, China
crazyofapple@gmail. com (hubaotian, zhangmin2021 J@hit. edu. cn
{225051013, suzhenran)@stu.hit.. wet.stuu.it.ed.u.. mit.edu..cm cn
Abstract
A critical aspect that remains unexplored is the interpretability of these models, specifically the ability to provide accurate and faithful rationale for their decisions (Wei et al., 2022b; Turpin et al., 2023). The degree to which these models plain their reasoning is of crucial significance, es- pecially in high-stakes domains such healtheare, where the clarity of generated responses directly impacts decision-mand makiinn and outcomes (Rudin, 2019).
In the field of Large Language Models (LLMs), researchers are increasingly exploring their ef- feetiveness across wide range of tasks. How- ever, a critical area that requires further inves- tigation the interpretability of these models, particularly the ability to generate rational ex- planations for their decisions. Most existing explanation datasets limited the English language and the general domain, which leads to scarcity linguistic diversity and a lack of resources in specialized domains, such med- ical. To mitigate this, we propose ExplainCPE. challenging medical dataset consisting of over 7K problems from Chinese Pharmacist Examination, specifically tailored to assess the generate explanations. From the overall results, only GPT-4 passes the pharma- cist examination with a 75.7% accuracy, while other models like ChatGPT fail. Further de- tailed analysis of LLM- generated explanations reveals the limitations of L.LMs understand- ing medical text and executing computational reasoning. With the increasing importance of safety and trustworthiness, ExplainCPE takes step towards improving and evaluat- ing the interpretability of LLMs in the medi- cal domain. The dataset available at https: //github.cc cot.ub..cm/HHTTZ/// TMG/ExpLainCEE.
Current benchmarks for LLMs mostly focus on the exam performance, such as MMLU (Hendrycks et al., 2021) and AGIEval (Zhong et al., 2023). These datasets do not allow for detailed assess- ment of LLMs' justifications of their decisions, because of the unavailability of high-quality and professional explanations. Moreover, accurately measuring the explainability of these LLMs is difficult task due to the lack of comprehensive and standardized datasets that come from unbi- ased and trustworthy sources (Li et al., 2023). Existing benchmarks predominantly are from on- line forums and consumer feedback and only con- sist of English-language general knowledge ques- tions (Wiegreffe and Marasovic, 2021), which results in insufficient thematic and linguistic di- versity. Overall, the lack of appropriate evalua- tion datasets has prevented full understanding of L.MMs' strengths and weaknesses in the field of interprctability.
Introduction
Advancements in the field of Large Language Mod- els (LLMs) (Radford et al., 2019; Brown et al.. 2020; Ouyang et al., 2022), exemplified by models such as GPT-4 (OpenAI, 2023), have opened up new possibilities and challenges across a myriad of natural language processing (NLP) tasks (Wei et al., 2022a). These models have shown re- markable success in understanding and generating human- like text, promoting rescarch that spans wide array of applications (Bubeck et al., 2023).
To address this gap, we introduce ExplainCPE, challenging medical benchmark dataset in Chi- nese, encompassing over 7K instances. This datasct, specifically tailored to evaluate the capac- ity of model explainability, diversifics the linguistic scope of interpretability research and allows for a rigorous assessment of model performance in specialized, high-stakes domain. An example from our dataset is presented in Table 1. The in-depth analysis of LLMs performance ExplainCPE brings to light several critical observations. First,
Both authors contributed equally this work. Conresponding author.

---

Question
The patient, female, 65 years old, received the second cycle of chemotherapy for breast cancer. Which antiemetic drugs are prohibited?
Options
A.Metoclopramidd// B.Ondansetron.
C.Dexamethasone
D.Diphenhydramine.
E.Aprepitant.
Gold Explanation
This question examines antiemetic drugs. Optional antiemetic drugs are mainly dopamine receptor blockers (such as metoclopramide), 5-HT3 receptor blockers (such as ondansetron, tropisetron, and palonosetron, etc.). corticosteroids (such as dexamethasone), anticholinergies and antihistamines (such as diphenhydramine), and NK-1 receptor blockers (such as aprepitant), etc. Special emphasis should be placed on the prohibition of metoclopramide as antiemetic in patients with breast cancer. Therefore, choose A for this question.
GPT-4 Response
This question examines the contraindications of antiemetic drugs, Metoclopramide is dopamine D2 receptor antagonist, and its anti-emetic effect is achieved by blocking central and peripheral 5-H13 receptors and dopamine D2 receptors. Elderly patients over 65 years of age are contraindicated for metoclopramide, because the drug may increase central nervous system adverse reactions in the elderly population, such as drowsiness, ataxia, and akathisia. Other antiemetics, such as ondansetron, dexamethasone. Siphenhydramine, and aprepitant, are safe in older patients. Therefore, the correct answer is A.
Answer: A. Metoclopramide. Analysis:
ChatGPT Response
Answer: B. Ondansetron. Analysis: Ondansetron is 5-HT3 receptor antagonist, mainly used prevent nausca and vomiting caused by chemotherapy for malignant tumors. However, ondansetron is contraindicated in patients with hepatic insufficiency or pregnancy, because these patients have reduced ability to metabolize and clear drugs, which may casily cause drug accumulation and cause side effects. For the specific situation of the patient, should he selected according to the clinical judgment of the doctor. But in this case, ondansetron is contraindicated, so the correct answer is B.
Table 1:
translated example from our ExplainCPE dataset with response of GPT-4 and ChatGPT (V: correct answer option). The blue text represents the given answer in the response. The red text represents the error in the GPT-4 response, the reason for choosing the option is breast cancer rather than old age.
we find substantial limitations in understanding of these LLMs over medical text and their abil- ity to execute computational reasoning effectively. For example, only GPT-4 passed Chinese Pharma- cist Examination with 75.7% accuracy, while other models like ChatGPT failed. Through the case analysis of GPT-4 and ChatGPT, we found that the explanations generated by LLMs still have flaws such as contradictory, insufficient analysis, con- fused logic, and how to improve its interpretability is the part that LLMs should pay attention to in the future. Furthermore, report heterogeneous preferences for in-context learning among different LLMs, suggesting varying strategies for explana- tion generation. For example, models with little chatting ability such as BELLE (Ji et al., 2023b,a) are more sensitive to the number of few-shot ex- amples than with ChatGPT with strong chatting ability. To the best of our knowledge, we are the first to propose a free-text explanation benchmark in Chinese medical examination and further ex- plore the interpretability of LLMs in the medical field. We provide a baseline for future research on explanation generation research, and this dataset can also be used to improve the interpretability of these large language models. As the broader issues of AI safety and trustworthiness gain attrac- tion, our work represents a pioneering step towards enhancing the medical interpretability of LLMs, underscoring the urgent need to develop Al that
is not only intelligent, but also transparent, robust, unbiased and reliable. Our main contributions can be summarized as follows:
We introduce ExplainCPE, challenging benchmark for generating free-text explana- tions in Chinese medical QA, which provides a baseline for future research on explanation generated by LLMs, and can be used to study how to improve the ability of the model to generate explanation.
We analyze the basic attributes of the dataset, such as the average length of questions, op- tions, and explanations. Additionally, we ex- amine the high-level categories of questions, which can assist researchers in understanding the distribution of categories in ExplainCPE and the interpretability performance of the models.
We conduct experiments on the ExplainCPE dataset to demonstrate its effectiveness and feasibility. Our findings reveal that differ- ent LLMS exhibit varying preferences for in- context learning. We analyze error cases and identify some limitations of current LLMs, which can serve as directions for future devel- opment.

---

0.030
0.006
0.025
0.004
Donsity
Denalty
0.015
0.003
0.010
0.002
0.005
0.001
0.000
0.000
0
50
100
160
200
200
400
600
800
question length
explanation __lnnthhhhhhh
Figure 1: Distribution of questions and explanations length in ExplainCPE.
Related Work
the SNLI dataset by introducing e-SNLI, which en- compasses an additional layer of human-annotated natural language explanations for entailment re- lations. Rajani et al. (2019) gather human expla- nations for commonsense reasoning in the form of natural language sequences and highlighted an- notations within novel dataset known as Com- mon Sense Explanations (CoS-E). Aggarwal et al. (2021) develop a first-of-its-kind dataset named ECQA, comprising negative properties, as well as free-flow expla- nations for 1 11,000 question answer pairs derived from the CQA dataset. Ye and Durrett (2022) as- the performance of four LLMs across three textual reasoning datasets utilizing prompts con- taining explanations multiple styles. Their find- ings indicate that human-evaluated high-quality explanations are more likely to coincide with accu- rate predictions.
2.1 Medical Question Answering
In the medical domain, addressing questions can be particularly challenging due to their special- ized and complex nature. Consequently, commu- nity efforts have been directed towards advancing biomedical question question-answering systems, such as BioASQ (Tsatsaronis et al., 2012, 2015). Another system, ScaReader (Zhang et al., 2018), was pro- posed to answer clinical medical questions by lever- aging documents extracted from medical publica- tions. In study by Yue al. (2020), the authors performed comprehensive analysis of the em- rQA (Pampari et al., 2018) dataset to evaluate the capacity of QA systems to utilize clinical domain knowledge and generalize to novel questions. Fur- thermore. Jin et al. (2019) introduced PubMedQA. system that generates questions based arti- cle titles and can be answered using their respec- tive abstracts. Li al. (2020) developed large- scale medical multiple-choice question dataset and proposed novel reading comprehension model, KMQA, capable of incorporating both structural medical knowledge and plain text.
annotated positive and
2.3 LLMs Benchmarks
New NLP benchmarks are urgently needed to align with the rapid development of LLMs.. MMLU(Hendrycks et al., 2021) is a collection of English-language materials that encompasses knowledge from 57 different disciplines includ- ing elementary mathematics, US history, puter science, law, and more. To attain high ac- curacy on this test, models must possess exten- sive world knowledge and problem solving abil- ity. Another significant contribution to this field is the C-EVAL(Huang et al., 2023), which rep- resents the first comprehensive effort to evaluate foundational models' knowledge and reasoning capabilities within Chinese context. C-EVAL consists of multiple-choice questions designed to assess performance across four difficulty levels:
2.2 Free-text Explanation
Since deep learning became the dominant paradi- agm in NLP research, how to interpret the predic- tions of neural models has become an essential part of model transparency. In explainable NLP, various forms of explanations exist, including ex- tractive rationales, semi-structured structured ex- planations, and free-text explanations. Saha et al. (2022) examine the impact of sample hardness the capacity of both LLMs and humans to eluci- date data labels. Camburu al. (2018) augment

---

middle school, high school, college, and profes- sional. These questions cover 52 diverse disci- plines, spanning from humanities to science and engineering, thereby providing holistic evalua- tion of the model's s capabilities. Zhang et al. (2023) introduces the GAOKAO AArKAAcOBchhart (GAOKAO- Bench), an intuitive benchmark that employs ques- tions from the Chinese Gaokao examination as test samples for evaluating LLMs. Most benchmarks focus on evaluating the performance of LLMs in answering or answering questions, but few focus on the ability of LLMs to explain the answers given.
Train
Dev
Test
#Questions
6867
500
189
Avg. words of Q
28.31
28.44
37.79
Avg. words of O
8.12
8.55
9.76
Avg. words of E 120.52
116.32
171.94
Max words of Q
338
259
164
Max words of O 146
95
57
Max words of E 1011
604
685
Options per problem
5
Table 2: ExplainCPE dataset statistics, where Q. A. represents the Question, Answer, and Explanation. respectively.
3.2 Data Statistic
3 ExplainCPE Dataset
The training, development, and test sets comprise 6,867, 500, and 189 questions, respectively, with average lengths of 28.31, 28.44, and 37.79 words. A summary of the dataset statistics can be found in Table 2. Figure illustrates the distribution of question and explanation lengths across the train- ing, development, and test sets.
3.1 Dataset Collection
The National Licensed Pharmacist Examination in China, collaboratively administered by the Min- istry of Personnel and the State Food and Drug Administration, serves as the basis for our question set.
In order to evaluate the performance and gener- alizability of our models, we have compiled a test set using examples from the previous two years (2020-2021) of the official examination. Each offi- cial question's 's explanation is sourced from official examination solution. Additionally, have col- lected over 7,000 instances from various sources, including the internet and exereise books. The in- stance in ExplainCPE dataset is multiple choice question with five options.
3.3 Data Analysis
In order to investigate the properties of the Ex- plainCPE dataset, we primarily focus on the diver- sity of questions in this subsection. Our aim is to determine the categories of problems that LLMs excel at handling. To achieve this, we performed a multi-level classification of the dataset, comprising three levels.
At the first level, questions are classified into positive and negative categories. Positive ques- tions which is also called direct question prompt the respondent to select the correct option, while negative questions require identifying the incorrect option among the options provided.
In addition to the official questions, we also col- laborated with three doctoral students from Peking Union Medical College (all of whom have under- gone standardized residency training). They manu- ally reviewed 320 samples from the collected data to evaluate the completeness and accuracy of the label and explanations. The evaluation resulted in the 99, 99.4%//9.00 00 accuracy rate, with 318/317 out of the 320 samples being deemed correct.
At the second level, questions are categorized into groups: logical reasoning, drug knowledge, scenario analysis, mathematical calculation, dis- ease knowledge, general knowledge. and others.
Finally, the third level, questions classi- fied into 14 categories based their content: anti- inflammatory, infection, tumor, anesthesia, cardio- vaseular, weight loss, orthopedics, nervous system, respiratory system, digestive system, urinary sys- tem, endocrine, immune system, and others.
Following the removal of duplicate and incom- plete questions (c.g.. those lacking answers or options), we randomly divided the remaining in- stances into training and development sets based on predetermined ratio. To further enhance the quality of our dataset, we inspected instances with an edit distance of less than 0.1 and manually re- moved questions containing different words that conveyed the same meaning.
We randomly selected 1200 instances from the training and development sets and manually as- signed three-level classification each ques- tion. The proportional distribution of each category

---

few-shot examples were the same for all models in each respective prompt type.
Logical reasaring 42%
4.2 Model Comparison
To compare the performance of different models, we evaluated several LLMs on our test dataset. We recognize that LLMs can be classified as chat or non-chat models, depending on their ability engage in human-like conversation. Chat models, which are pre-trained with vast amounts of data and fine-tuned through reinforcement learning from human feedback (RLHF), include GPT-4 (Ope- nAI, 2023), ChatGPT (OpenAI, 2022), ChatGLM- 6B (Du et al., 2022: Zeng et al., 2023), and Yuat (ClueAI, 2023), Non-chat models, on the other hand, are typically pre-trained on unsu- pervised plain text and fine-tuned on code or in- structional data but do have sufficient RLHF enable human-like conversation. Examples of non- chat models include GPT-3 (Ouyang et al., 2022), BELLE (Ji et al., 2023b,a), and GPT-3 (Brown et al., 2020). Consequently, non-chat models are more inclined to predict the next word complete given task rather than engage in conversation. In this section, provide brief introduction the LLMs used in our experiments.
99%
brug Anowledge 25%
Hegativel 9%
24%
34%
General knowledge
Pho
23%
Figure 2: The distributions of proportions for each cate- gory at two levels ExplainCPE. the first layer of categories, positive questions account for the majority. In the second category. logic questions and knowledge questions account for the majority.
within the dataset is presented in Figure 2. A more detailed proportional distribution of each category within the dataset is presented in Appendix B.
Experiments
4.1 Prompting
Prompting has a significant impact on the output of generative language models, so we standard- ized the structure of our prompts. In order to bet- ter analyze the performance and interpretability of language models, we designed prompts to re- quest the model to provide answer option along with explanation in the test set. An example of the template and a fully instantiated prompt can be found in Appendix A. Two types of prompt templates were utilized: with and without instruc- tions. The purpose of this design was to explore the influence of instructions on different models. In the zero-shot setting, the Lacholleeappleeaeeeeeee slot was left blank. Additionally, it should be noted that prompts without instructions are the same as prompts with instructions in the zero-shot setting.
ChatGPT (OpenAI, 2022) is a large language model with hundreds of billions of parame- ters, specifically designed for human-like con- versation across wide range of topics. Chat- GPT's text understanding ability is derived from language model pre- pre-training, its reason- ing ability is derived from code pre-training, its logical reasoning ability is derived from su- pervised instruction training, and its dialogue ability derived from RLHE.
GPT-4 (OpenAI, 2023) represents the latest milestone in OpenAl's deep learning scaling efforts, and is a large multimodal model that exhibits human-level performance on various professional and academic benchmarks. GPT- 4 outperforms ChatGPT most tasks.
To investigate the impact of in-context on model performance, we designed prompts with different numbers of few-shot examples, including zero- shot, one-shot, four-shot, and eight-shot prompts. For one-shot prompts, we randomly selected sin- gle instance from the training set. For four-shot and eight-shot prompts, we manually selected instances with varying question types to ensure model pre- dictions were balanced. It should be noted that the
GPT-3 (Ouyang et al., 2022) is a series of models. In this paper, we simply call text- davinci-003 with GPT-3. Texi-davinci-003 is capable of performing any language task with better quality, longer output, and more consistent instruction-following than GPT-3.
ChatGLM-6B (Du et al., 2022; Zeng al.,

---

80
40
Acef (6)
35
Acc
30
20
0
4
6
X-shot
0
4
GPT- w/ instruction
GPT-4 w/o instruction
X-shot
ChatGPT w/ instruction
GPT-3 w/ instruction
ChatGPT instruction
GPT-3 instruction
ChatGLM-68 w instruction
BELLE-7B-2N w instruction
ChatGLM-6B w/o instruction
BELLE-7B-2h w/o instruction
Figure 3: Performance comparison under different few-shot numbers. Left: Models with chatting ability such GPT-4, ChatGPT and ChatGLM-6B. Right: Models without enough chatting ability such as GPT-3 and BELLE. Each model has 2 settings, with instruction and without instruction.
2023) is an open-source dialogue language model that supports both Chinese and En- glish bilinguals. Utilizing technology simi- lar to ChatGPT, it is optimized for Chinese question-answeriigg and dialogue. After about IT identifiers of Chinese and English bilin- gual training, supplemented by supervision, fine-tuning, feedback self-help, human feed- back reinforcement learning, and other tech- nologies, ChatGLM-6B with 6.2 billion pa- rameters can generate answers that closely align with human preferences.
Model
Ace( %
Rouge-1
Rouge-2
Rouge-L
GPT-4
75.7
0.384
0.140
0.247
ChaiGPT
54.5
0.341
0.114
0,216
GPT-3
40.2
ChatGLM-6B
29.1
0.315
0.099
0.184
BELLE-7B-2M
33.3
Yuat
Table 3: Performance comparison on ExplainCPE dataset.
Results
One of the main objectives of our dataset is to eval- uate the interpretability of models by assessing the quality the generated text. Therefore, we not only measured the accuracy of the models but also required a useful and efficient evaluation metric. Evaluating interpretability is long-standing prob- lem due to the diversity of interpretation forms and content.
The BELLE-7B-24 (Ji et al., 2023b,a) model is based on Bloomz-7bl-mt and trained on 2M pieces of Chinese data, combined with 50,000 pieces of English data open-sourced by Stanford-Alpace
has demonstrated good Chinese instruction understanding and response generation capabilities.
As suggested by Wiegreffe et al. (2022), the quality of explanations generated by the joint method needs to be further verified. Therefore, we chose two methods to evaluate the interpretabil- ity of the models: automatic metrics and human evaluation. For automatic metrics, we used Rouge to measure the quality of the explanations provided by the models and accuracy measure the mod- els' performance. Due the model's 's input length limitation, we could not conduct eight-shot experi-
ChatYuan (ClueAI, 2023) further trained based on PromptCLUE-large (Zhang and Xu, 2022), combined with hundreds of millions of functional dialogues and multiple rounds of dialogue data. Yuat is capable of an- swering questions in fields such as law, and can be used for question-anse answering. dialogue context, creative writing.

---

Well-formed
Support
Correciness
Validity
Novelty
GPT4
4,99
4.96
4.03
4.60
4.57
Human Evaluation
CharGPT
4.99
4.96
3.16
4.24
4.39
ChaiGLM-66
4.48
3.16
3.37
GPT4
3.72
4.54
4.42
4,30
4.10
GPT-4 Evaluation
ChatGPT
3.42
4.00
3.84
3.94
3.84
ChaiGLM-6B
2.90
3.32
3.06
3.24
3.34
Table Human evaluation results(top) and GPT-4 evaluation results(bottom)) of different models five perspectives.
ments for some models, such as GPT-3. Moreover, some models did not respond with answers and explanations when requested them, which why some models lack Rouge score.
less of the size of the model parameters or whether instructions are given, going from zero-shot to one- shot often results in significant performance im- provement, which better than any subsequent increase of few-shot examples.
5.1 Automatic Metries
Secondly, when comparing chat models and non- chat models, GPT-3 a model with a large num- ber of parameters but weak dialogue ability, while GPT-4 and ChatGPT are models with strong dia- logue ability, Regardless of whether instructions provided, the performance of GPT-3 increases with an inerease in few-shot examples, but GPT-4 and ChatGPT tend achieve their maximum per- formance one-shot setting. This suggests that for a model with a large number of parameters and strong dialogue ability, one-shot setting a good choice. Conversely, for models with weak dialogue ability, their performance is somewhat proportional to the number of few-shot examples.
The performance of each model in each setting can be found in Appendix Table 3 presents the best performance of each model on the test set, regardless of whether the prompt of each model is consistent. Not surprisingly, GPT-4 the best- performing model, achieving 75.7% accuracy with the most suitable set of 1-shot without instruction. Therefore, GPT-4 has demonstrated the ability to pass the National Licensed Pharmacist Examina- tion in China and has outperformed more than 80% of the people who take the examination. In contrast, ChatGPT achieved an accuracy rate of 54.5% on the test set, which would not be sufficient to pass the examination. GPT-3, ChatGLM-6B: BELLE-7B-2M, and Yuat achieved accura- cies of 40.2%, 29. 1%, 33.3%, and 27.0% the test set, respectively. Models with fewer param- eters generally perform worse than models with more parameters. This can be attributed to smaller parameter sizes, which means that models may not have the capacity to remember enough knowledge or understand the meaning of context.
Thirdly, when comparing the figures, the models in the left picture have strong dialogue ability. Therefore, in the case of the same number of few-shot examples, providing instructions better than not providing instructions. However, in the right picture, the models have weak dialogue ability. Therefore, in the case of the same number of few-shot examples, not providing instructions is better.
Regarding interpretable automatic evaluation in- dicators, GPT-4 achieved the best results in expla- nation generation with Rouge-L score of 0.247, followed by ChatGPT, ChatGL.M-6B, and BELLE- 7B-2M. ChatGLM-6B yielded unexpected results in metrics, despite its relatively small parameter size, with high accuracy and Rouge scores.
5.2 Human Evaluation
From the perspective of interpretability. there are certain limitations in using the rouge evaluation metric to evaluate the interpretability of the model. So also used human evaluation to assess the qualitative properties of the generated explanations. We follow Monsen and Rennes (2022), Wiegreffe et al. (2022) and Kunz al. (2022) asking annota- tors to rate from to according to the following questions for each e.
We plotted line charts of model performance as a function of the number of few-shots. The line chart is divided into two, the chart on the left for chat models and the chart on the right for non-chat models. From the figure, we identified three key findings.
Is well-formed sentence?
Firstly, it is evident from Figure that regard-
Does support the label?

---

Type
All
Positive
Negative
Logical
Drug
Scenario
Math 2
Disease
General
others
Num type
189
128
61
78
38
17
GPT-4
75.7
74.2
78.71
76.9
73.3
76.31
100.01
100.01
70.6
66.7
ChatGPT
54.5
52.3
59.01
51.3
60.0f
47.4
00.0
100.01
58.81
83.31
GPT-3
40.2
40.61
39.3
38.5
46.71
34.2
00.0
0.33
58.81
16.7
ChaiGLM-6B
29.1
31.3
24.6
26.9
31.6
50.01
66.71
17.6
33.3
BELLE-7B-2M
33.3
40.61
18.0
39.71
31.1
21.1
50.01
33.3
41.21
16.7
ChatYuan
27.0
28.91
23.0
25.6
26.3
50.01
33.31
41.21
16.7
Table 5: Performance of models on different types of samples in ExplainCPE.
Is the content of e factually correct?
fails on all such questions. These findings sug- gest that GPT-4 has stronger logical reasoning, sce- nario analysis, and mathematical calculation abili- ties than other models. The superior performance of GPT-4 and ChatGPT on negative questions indi- cates their better understanding of text and ability to answer questions.
Does provide valid reasoning path for the label?
Does add new information, rather than re- combining information from the input?
Due the low accuracy of some models and the poor quality of the generated explanations, we only manually evaluated a subset of the models, and the results are presented in Table 4. As expected, GPT-4 remains the best performer. It is notewor- thy that the performance of GPT-4 and ChatGPT in terms of well-formed and support is the same. This indicates that both GPT-4 and ChatGPT can comprehend the question's requirements, provide the label, and generate a complete and coherent ex- planation that supports the label. However, GPT-4 outperforms ChatGPT in terms of the correctness of the explanation, effectiveness of the explana- tion process, and novelty. ChatGLM lags behind ChatGPT and GPT-4 on all five indicators. And We also ask GPT-4 to evaluate responses, the rela- tive scores of different metries consistent with human evaluation results.
We analyze specific error cases of ChatGPT and GPT-4 identify the limitations of current LLMss. A Appendix D outlines the reasons for explanation errors. Although the results of LLMs are impres- sive, they are not yet perfect. In example 1, GPT-4 provides the correct answer but the wrong expla- nation, which is difficult to detect. Thus, models should pay close attention to such errors before widespread use. In Example 2, although the model has certain calculation capability, the reliability of its calculation still not guaranteed. In Example 3, neither GPT-4 nor ChatGPT fully comprehends the detailed requirements of the question, leading to errors. Therefore, LLMs still have scope for im- provement in text comprehension and generating explanations.
6 Conclusion
5.3 Error Analyses
In this subsection, we present an analysis of the performance of the models from an overall perspec- tive and specific examples. Table 5 displays the performance of the models different types of questions. Notably, GPT-4 and ChatGPT perform better on negative questions, while other models perform better on positive questions. Moreover, GPT-4 demonstrates improvement in logical rea- soning. whereas other models do not. While GPT- 4 improves in scenario analysis questions, other models exhibit decline. Conversely, GPT-4 de- clines in general knowledge questions while other models improve. GPT-4 correctly solves two math- ematical calculation questions, whereas ChatGPT
In this work, we propose ExplainCPE, a challeng- ing medical dataset for natural language explana- tion evaluation. Our study ExplainCPE dataset demonstrates the potential of LLMs in medical question answering with explanations. Our anal- ysis of model performance on different types of questions reveals the strengths and limitations of different LLMs in terms of in-context learning. The error cases point out the need for further im- provement in LLMs in explanation generation and text comprehension. Further work dataset improve and evaluate the model inter- pretability.

---

Limitations
References
Association for Shourya Aggarwal, Divyanshu Mandowara, Vishwajeet Agrawal, Dinesh Khandelwal. Parag Singla, and Di- nesh Garg. 2021. Explanations for commonsensega: New dataset and models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer- ence Naturul Langwage Processing, ACL/IJCNPP 2021, (Volume 1: Long Papers). Virtual Event, Au- gust 1-6, 2021, pages Computational Linguistics.
Due to the lack of interpretable benchmarks in the medical professional field, we present ExplainCPE: in this puper. While there are many explainable methods, we only contribute to the Explanation Generation. Moreover, most of the current inter- pretable methods are aimed at classification tasks. For LLMs which are used to generate response, new interpretable methods are necessary. We ex- plore the ability of LLMs in medical diagnosis and interpretability. While model performance can be well assessed by accuracy, automatic assessment of interpretability is still lacking.
Tom Brown, Benjamin Mann. Nick Ryder. Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in Neural Information Processing Systems (NeurIPS), 33:1877-1901.
However, our analysis of ExplainCPE dataset is just a preliminary exploration, and there is still much room for further research and development. For example, future work can focus on improving the quality and diversity of the explanations in the dataset, expanding the coverage of medical knowledge, and exploring new evaluation metrics for interpretability. In addition, more advanced LLMs can be developed to further improve the performance of medical question answering with explanations by utilizing the data in the training set. We believe that the ExplainCPE dataset can serve as a valuable resource for the research community to advance the field of medical question answering and LLMs.
Sébastien Bubeck, Varun Chandrasckaran, Ronen EI- dan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Pe- ter Lee, Yin Tat Lee, Yuanzhi Li, Scott M. Lundberg, Harsha Nori, Hamid Palangi, Marco Túlio Ribeiro, and Yi Zhang. 2023. Sparks of artificial general in- telligence: Early experiments with GPT-4. CoRR, abs/2303.12712.
Oana-Mane Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. 2018. e-snli: Natu- ral language inference with natural language expla- nations. In Neural Information Processing Systems.
ClueAL 2023. Chatyuan: Large language model for dialogue in chinese and english.
Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. 2022. Glm: General language model pretraining with autoregres- sive blank infilling. In Proceedings of the 60th An- nual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 320- 335.
Ethical Statement
This paper is concerned about proposing dataset on explanations of medical question answers. The data in this dataset are all from Chinese Pharmacist Examination related exercises. Moreover, the cases in the exercises are all fictitious cases, and there is no personal privacy, discrimination or attack content. Judging from its impact, this data set can be used to improve the interpretation ability in human medical diagnosis, reduce misdiagnosis, and contribute to human intelligent medical care.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Stein- hardt. 2021. Measuring massive multitask language understanding. In 9th International Conference on Learning Representations. ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net.
Yuzhen Huang, Yuzhuo Bai, Zhihao Zhu, Junlei Zhang, Jinghan Zhang, Tangjun Su, Junteng Liu, Chuancheng Lv. Yikai Zhang, Jiayi Lei, Yao Fu, Maosong Sun, and Junxian He. 2023. C-eval: A multi-level multi-discipline chinese evaluation suite for foundation models. CoRR, abs/2305.0833
Acknowledgments
We thank Yuxiang Wu for valuable feedback and support on running experiments. We thank Xinshuo Hu for helpful discussions and sug- gestions. This work is jointly supported by grants: Natural Science Foundation of China (No. 62006061, 82171475), Strategic Emerging Industry Development Special Funds of Shenzhen (N0.CYY2002000011133333333
Yunjie Ji, Yong Deng, Yan Gong, Yiping Peng, Qiang Niu, Baochang Ma, and Xiangang Li. 2023a. Belle: Be everyone's 's large language model engine. https: //gi thub. wwt..///mjaaaaeccaii.oaa BELLE
Yunjie Ji, Yong Deng. Yan Gong, Yiping Peng. Qiang Niu, Baochang Ma, and Xiangang Li. 2023b. Explor- ing the impact of instruction data scaling on large

---

language models: An empirical study on real-world use cases. arXiv preprint arXiv:2303. 147.44
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenA! blog, 1(8):9.
Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William W. Cohen, and Xinghua Lu. 2019. Pubmedqa: A dataset for biomedical research question answering. In Proceedings of the 2019 Conference on Empiri- cal Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNIPP 2019, Hong Kong, China, November 3-7, 2019, pages 2567-2577. Association for Computational Linguistics.
Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong. and Richard Socher. 2019. Explain your- self! leveraging language models for commonsense reasoning. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019. Florence, Italy, July 28- August 2. 2019, Vol- ume 1: Long Papers, pages 4932-4942. Association for Computational Linguistics.
Jenny Kunz, Martin Jirenius, Oskar Holmström, and
Marco Kuhlmann. 2022. Human ratings do not re- flect downstream utility: A study of free-text ex- planations for model predictions. In Proceedings of the Fifth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP. Black- boxNLP@ EMNLP 2022, Abu Dhabi, United Arab Emirates (Hybrid), December 8, 2022, pages 164- 177. Association for Computational Linguistics.
Cynthia Rudin. 2019. Stop explaining black box ma- chine learning models for high stakes decisions and use interpretable models instead. Nat. Mach. Intell., 1(5):206-215.
Swarnadeep Saha, Peter Hase, Nazneen Rajani, and Mo- hit Bansal. 2022. Are hard examples also harder to explain? A study with human and model-generated explanations. In Proceedings of the 2022 Confer- ence on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 2121-2131. Association for Computational Linguisties.
Dongfang Li, Baotian Hu, Qingeai Chen, Weihua Peng, and Anqi Wang. 2020. Towards medical machine reading comprehension with structural knowledge and plain text. In Proceedings of the 2020 Confer- ence on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 1427-1438. Association for Computa- tional Linguistics.
George Tsatsaronis, Georgios Balikas, Prodromos Malakasiotis, Ioannis Partalas, Matthias Zschunke, Michael R. Alvers, Dirk Weissenborn, Anastasia Krithara, Sergios Petridis, Dimitris Polychronopou- los, Yannis Almirantis, John Pavlopoulos, Nico- las Baskiotis, Patrick Gallinari, Thierry Artières, Axel-Cyrille Ngonga Ngomo, Norman Heino, Éric Gaussier, Liliana Barrio-Alvers, Michael Schroeder, Ion Androutsopoulos. and Georgios Paliouras. 2015. An overview of the BIOASQ large-scale biomedical semantic indexing and question answering competi- tion. BMC Bioinform.. 16:138:1-1388
Jianquan Li. Xidong Wang, Xiangbo Wu. Zhiyi Zhang, Xiaolong Xu, Jie Fu. Prayag Tiwari, Xiang Wan, and Benyou Wang. 2023. Huatuo-26m, a large-scale chinese medical QA dataset. CoRR, abs/2305.01555
Julius Monsen and Evelina Rennes. 2022. Perceived text quality and readability in extractive and abstrac- tive summaries. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, LREC 2022, Marseille, France, 20-25 June 2022, pages 305-312. European Language Resources As- sociation.
George Tsatsaronis, Michael Schroeder, Georgios Paliouras, Yannis Almirantis, Ion Androutsopoulos, frii Gaussier, Patrick Gallinari, Thierry Artières, Michael R. Alvers, Matthias 7schunke, and Axel- Cyrille Ngonga Ngomo. 2012. Bioasq: A challenge on large-scale biomedical semantic indexing and question answering. In Information Retrieval and Knowledge Discovery in Biomedical Text, Papers from the 2012 AAAI Fall Symposium, Arlington, Vir- ginia, USA, November 2-4, 2012, volume FS-12-05 of AAAI Technical Report. AAAI.
OpenAI. 2022. Chatgpt: Optimizing language models for dialogue.
OpenAI. 2023. abs/2303.08774.
GPT-4 technical report. CoRR,
Long Ouyang. Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroli Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Gray, et al. 2022. Training language models to follow instruc- tions with human feedhack. In Advances in Neural Information Processing Systems (NeurIPS).
Miles Turpin, Julian Michael, Ethan Perez, and Samuel R. Bowman. 2023. Language models don't 't always say what they think: Unfaithful ex- planations in mhain-of-thougee prompting. CoRR, abs/2305.04
Anusri Pampari, Preethi Raghavan, Jennifer J. Liang, and Jian Peng. 2018 emrqa: A large corpus for question answering on electronic medical records. In Proceedings of the 2018 Conference Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31- November 4, 2018, pages 2357-2366 Association for Computational Linguis- tics.
Haochun Wang. Chi Liu, Nuwa Xi, Zewen Qiang. Sendong Zhao, Bing Qin, and Ting Liu. 2023a. Hu- atuo: Tuning llama model with chinese medical knowledge.

---

Haochun Wang. Chi Liu, Sendong Zhao, Bing Qin, and Ting Liu. 2023b. Chatglm-med: H fp X E + XI it Wishtiia Rāj. https://githubs com/ SCIR-HI/Med- ChatGLM.
Intelligence, (AAAI-18). the 30th innovative Applica- tions of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artifi- cial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pages 5713-5713. AAAI Press.
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. 2022a. Emer- gent abilities of large language models. Trans. Mach. Learn. Res., 2022.
Xiaotian Zhang, Chunyang Li, Yi Zong. Zhengyu Ying, Liang He, and Xipeng Qiu. 2023. Evaluating the performance of large language models on GAOKAO benchmark. CoRR, abs/2305.12474
Xuanwei Zhang and Liang Xu. 2022. Promptclue: zero-shot learning model that supports full chinese tasks.
Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoe V Le, and Denny Zhou. 2022b. Chain-of-thought prompt- ing elicits reasoning in large language models. In Advances in Neural Information Processing Systems, volume 35, pages 24824-24837. Curran Associates, Inc.
Wanjun Zhong, Ruixiang Cui, Yiduo Guo, Yaobo Liang, Shuai Lu, Yanlin Wang, Amin Saied, Weizhu Chen, and Nan Duan. 2023. Agieval: human-centric benchmark for evaluating foundation models. CoRR, abs/2304.06366
Sarah Wiegreffe, Jack Hessel, Swabha Swayamdipta, Mark O. Riedl, and Yejin Choi. 2022. Reframing human-ai collaboration for generating free-text ex- planations. In Proceedings of the 2022 Conference of the North American Chapter the Association for Computational Linguistics: Human Language Tech- nologies, NAACL 2022, Seattle, WA, United States, July 10-15, 2022, pages 632-658. Association for Computational Linguistics.
Sarah Wiegreffe and Ana Marasovic. 2021. Teach me to explain: A review of datasets for explainable natural language processing. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, NeurlPS Datasets and Bench- marks 2021, December 2021, virtual.
Xi and Greg Durrett. 2022. The unreliability of explanations in few-shot prompting for textual rea- soning. In Advances in Neural Information Process- ing Systems, volume 35, pages 30378-30392. Curran Associates, Inc.
Xiang Yue, Bernal Jimenez Gutierrez, and Huan Sun. 2020. Clinical reading comprehension: thorough analysis the emrqa dataset. In Proceedings of the 58th Annual Meeting of the Association for Compu- tational Linguistics, ACL 2020, Online, July 5-10, tional Linguistics.
2020, pages 4486. Association for Computa-
Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang. Hanyu Lai, Ming Ding, Zhuoyi Yang. Yifan Xu, Wendi Zheng. Xiao Xia, Weng Lam Tam, Zixuan Ma, Yufei Xue, Jidong Zhai, Wenguang Chen, Zhiyuan Liu. Peng Zhang, Yuxiao Dong, and Jie Tang. 2023, GLM-130b: An open bilingual pre-trane praitrained model. In The Eleventh International Conference on Learning Representations (ICLR).
Xiao Zhang, Ji Wu, Zhiyang He, Xien Liu, and Ying Su. 2018. Medical exam question answering with large-scale reading comprehension. In Proceedings of the Shirty-Secondl AAAI Conference on Artificial

---

A Prompting Template
as metoclopramide), 5-HT3 receptor block- ers (such as ondansetron, tropisetron, and palonosetron, etc.). corticosteroids (such as dexamethasone), anticholinergics and antihis- tamines (such diphenhydramine), and NK-1 receptor blockers (such aprepitant), etc. Spe- cial emphasis should be placed on the prohi- bition of metoclopramide as an antiemetic in patients with breast cancer. Therefore, choose A for this question.
There are two types of prompt templates. prompt- ing with instruction and prompting without instruc- tion. And you can check the template in Table 6. You can also see the template instantiation in Ta- ble 7.
B Distribution of Categories
In Figure 4, we show the proportion distribution of each type in the dataset in more detail.
ChatGPT Response: Answer: B. On- dansetron. Analysis: Ondansetron is 5-HT3 receptor antagonist, mainly used to prevent nausea and vomiting caused by chemotherapy for malignant tumors. However, ondansetron is contraindicated in patients with hepatic insuf- ficiency or pregnancy, because these patients have reduced ability to metabolize and clear drugs, which may easily cause drug accumu- lation and cause side effects. For the specific situation of the patient, it should be selected ac- cording the clinical judgment of the doctor. But in this case, ondansetron is contraindicated, so the correct answer is
C Performance Comparison
Perhaps due to the training data or model size, these models do not respond well to a given multiple-choice question. We have already evalu- ated two popular medical and Luatuo-Llama--ed-- Maaeeee--hhneee-aan and found that they struggled with our multi-choice questions.
Wang et (2023b) constructed Chinese med- ical instruction data set through the medical knowl- edge graph and GPT3.5 API, and based on this, fine-tuned the instructions of ChatGLM-6B to im- prove the gueseinn-answering effect of ChatGLM in the medical field called Med-ChatGLMM (Wang et al., 2023b). Based the same data, we also trained medical version of the LLaMA model called Huatuo (Wang al., 2023a).
GPT-4 Response: Answer: Metoclo- pramide. Analysis:
This question examines the contraindications of antiemetic drugs. Meto- clopramide is a dopamine D2 receptor antag- onist, and its anti-emetic effect is achieved by blocking central and peripheral 5-HT3 recep- tors and dopamine D2 receptors. Elderly pa- tients over 65 years of age are contraindicated for metoclopramide, because the drug may in- crease central nervous system adverse reac- tions in the elderly population, such as drowsi- ness, ataxia, and akathisia. Other antiemetics, such as ondansetron, dexamethasone, diphen- hydramine, and aprepitant, are safe in older patients. Therefore, the correct answer is A. Error Analyse: GPT-4 is right. ChatGPT is wrong. Although GPT-4 makes right pre- diction, its explanation is flawed because the reason given by GPT-4 is old age but the real reason is cancer.
In Table 8, we show the results of each model under all settings, including the number of few- shots and with or without instructions.
D Error Example
In this section, we mainly show some classic errors made by ChatGPT and GPT-4. We analyzed these errors in order to study the current shortcomings of LLMs.
Example 1
Question: The patient, female, 65 years old, re- ceived the second cycle of chemotherapy for breast cancer. Which antiemetic drugs are pro- hibited?
Option A:Metoclopramidee
Example 2
Option B:Ondansetron.
Question: 100 mg of certain drug is injected intravenously, blood samples are collected im- mediately, and the drug concentration is mea- sured be 5 g/ml, what is the apparent vol- ume of distribution? Option A: 5L.
Option C:Dexamethasone.
Option D:Diphenhydramine.
Option A:Aprepitant.
Gold Explanation: This question examines
antiemetic drugs. Optional antiemetic drugs are mainly dopamine receptor blockers (such

---

Respiratory system
Digestive system
topse A Nervous 9%
Endocrine 6%
Tumor 5%
7%
Infection
Uniaary System
7%
11%
Cardiovascular
4%
13%
Logical reasoning 42%
Others 32%
Others 64%
Drug knowledge 26%
Positive
%
Negative 9%
Logical rearoning
Cardiovascular 8%
Drug knowledge 33%
25%
Scenario analysisi 7%
iinfaction
Disease knowledge 8%
Endecrine
Othera
5%
General knowledge 13%
2%
Others
62%
Others
96%
Others 83%
Car diovassuu cocaa
Oeter 19%
ZO1t.
Otferc
19%
Figure 4: Proportional Distribution of each category of ExplainCPE dataset.

---

Type
Prompt Template
w/ instruction
[[eew shot examples))
[= [|question]]
[options]
w/ instruction
This question is multiple-choice question. please answer the following questions and give an explanation.
fite_sshot examples))
Question: [qquestion] [[options]
w/o instruction
[ffe__shot_exxee )
[[: [gquestion)}
[[options]]
w/o instruction
[[few_shrr shot wttssssooteaaeee examples))
Question: [question)]
[[options)
Table 6: Prompt template. The solid line separates whether it is with instruction. And the dotted line separates the Chinese and English versions.
ADCC: BB
Hiji:: 11
This question is multiple-choice question, please answer the following questions and give an explanation. Question: The photosensitivity of a drug refers to the sensitivity of the drug to photodegradation. Which of the following drugs is the most photosensitizing? Option A: Chlorpromazine.
Option B: Sodium nitroprusside.
Option C: Vitamin B2.
Option D: Folic acid.
Option E: Hydrocortisone.
Table 7: Instantiated example of prompt template. The dotted line separates Chinese and English versions.
Option B: 2[.
total drug dose/drug concentration. In this question, the total dose of the drug 100 mg, and the drug concentration is μ g/ml (note that the unit should be the same). The total dose of the drug is converted to μ to be 100,000 g. which is substituted into the cal- culation formula: Vd = 100,000 g/5 g/ml 20,000ml 20L.. Because the distribution of drugs not only occurs the blood, but may also be distributed cells, tissues and organs, etc., the apparent distribution volume com- mon drugs is relatively large, generally rang- ing from tens of liters to hundreds of liters (ie: 0.031/kg to 0.8L//k).. The answer to this ques- tion is option D, and 50L is within the apparent volume of distribution of common drugs.
Option 2
Option D: 50L.
Option E: 29ml.
Gold Explanation: This question examines the calculation of the apparent volume of dis- tribution. The formula for calculating the apparent volume of distribution is "V=X/C". where V is the apparent volume of distri- bution, X is the amount of drug in the body, and is the blood drug concentration. V=X/C=100mg/5 и g/ml=20L. Others are in- terference options, so choose for this ques- tion.
ChatGPT Response: Answer: D.50L. Analy- sis: Apparent volume of distribution (Vd) is parameter representing the distribution range of drug, and its calculation formula is: Vd
GPT-4 Response: Answer: C.20L. Explana- tion: The apparent volume of distribution (Vd)

---

is a theoretical value used to describe the range of drug distribution in the body. The formula for calculating Vd is: Vd = drug dose initial drug concentration. In this question, the drug dose is 100 mg. and the measured drug concen- tration is 5 μ g/ml. First convert the dose to mi- crograms: 100mg 100,000 ja g. Then apply the Vd formula: Vd 100,000 gg/3 5 g/ml = 20,000ml, which is 20L. Therefore, the appar- ent volume of distribution the drug is 20I., and the correct answer is C.
quine an antimalarial drug used for the pre- vention and treatment of malaria, especially against infection by susceptible strains of Plas- modium. It can not only kill the malaria par- asite, but also reduce the transmission prob- ability of the malaria parasite, it is used prevent the recurrence and transmission of malaria. Other options Quinine, primaquine, amodiaquine, and artemisinin are mainly used in the treatment of malaria. So the correct an- swer is B.
Error Analyse: GPT-4 is right. ChatGPT is wrong. ChatGPT gets an answer in the middle process, but it do not take that answer.
Error Analyse: Both GPT-4 and ChatGPT are wrong. The question asks for drug to pre- vent malaria, but ChatGPT gives drug to treat malaria. GPT-4 says that Hydroxychloroquine can prevent malaria, but Hydroxychloroquine is actually used to control malaria, not to pre- vent malaria.
Example 3
Question: What are the medicines used to pre- vent relapse and transmission and prevent malaria?
Option A:Quinine.
Example 4
Option B:Hlydoxyyhloroqqune.
Question: Which medicines require water re- striction while taking?
Option C:Primaquine.//
Option D:Amodiauuine.
Option A:cough syrup./
Option E:Artemisinin.
Option B:Diprophylline.
Gold Explanation: This question examines the action characteristics of primaquine. Pri- maquine can kill Plasmodium vivax, Malaria malaria, and Plasmodium falciparum. It can also kill the gametocytes of various Plasmod- ium parasites, and is clinically used as the drug of choice for controlling recurrence and pre- venting transmission. Quinine, chloroquine, and artemisinin are used as drugs control the symptoms of malaria. So choose for this question.
Option Cursodeoxycholic acid.
Option D:ritonavir.
Option E:Alendronate Sodium.
Gold Explanation: This question exam- ines drugs that should restrict drinking wa- ter. Cough medicines such as cough syrup, licorice mixture, etc. These viscous medicines will adhere to the inflamed throat and play role, After use, drink less water, especially hot water, to avoid washing the local medicines. Taking theophylline or theophylline controlled- release tablets, aminophylline, cholophylling dihydroxyprophylline etc., because can in- crease renal blood flow, has a diuretic effect, increases urine output and easily causes dehy- dration, dry mouth, Polyuria or heart palpita- tions; at the same time, asthmaties are often accompanied by low blood volume. There- fore, it is advisable to pay attention to the right amount of fluids and drink plenty of boiled wa- ter. Iproynpr, hydroxyyeehhylcoumaall dehydrocholic acid and ursodeoxycholic acid in choleretic drugs can cause excessive secre- tion of bile and diarrhea. Therefore, drink plenty of water when taking it to avoid dehydra- tion caused by excessive diarrhea. Among pro- tease inhibitors, ritonavir, indinavir, nelfinavir, amprenavir, lopinavir, etc., most them can
ChatGPT Response: Answer: E. Artemisinin. Analysis: Artemisinin is currently an effective drug for treating malaria, especially for falci- parum malaria and refractory malaria. In addi- tion, artemisinin also has certain antitumor and antiviral effects. Quinine is another drug used treat malaria, but it is mainly used treat the symptoms and complications of malaria, not to treat malaria itself directly. Hydroxy- chloroquine, primaquine, and amodiaquine are also drugs used to treat malaria, but drug re- sistance has emerged and their use is limited. Therefore, artemisinin is considered to be the drug of choice for the treatment of malaria.
GPT-4 Response: Answer: B. Hydroxychloro- quine. Analysis: This question examines malaria prevention drugs. Hydroxychloro-

---

cause urethral stones or kidney stones. There- fore, sufficient hydration should be ensured during the treatment period. In order to avoid the occurrence of stones, it is advisable to in- crease the daily water intake, and the daily drinking water should be more than 2000ml. Bisphosphonates are irritating to the esopha- gus and must be taken with more than 200ml of water; among them, sodium alendronate, sodium patradronate, and disodium clodronate can cause water when they are used to treat hypercalcemia. Electrolyte disorder, so atten- tion should be paid to replenishing fluids that the daily urine output can reach more than 2000ml. Therefore, choose A for this question. ChatGPT Response: Answer: B. Dipro- phylline. Analysis: Diprophylline is metab- olized in the body to produce uric acid, and uric acid is easy to cause stones, so you need to limit the amount of water you drink when tak- ing it. Drugs such as cough syrup, ursodeoxy- cholic acid, ritonavir, and alendronate do not require water restriction.
Option B:If skin and mucous membrane dry- ness occurs during treatment, the dose should be reduced or the drug should be discontin- ued.v
Option C:Adverse reactions are common, but most of them can be recovered after stopping the drug.
Option D:The first months of treatment may lead to abnormal liver enzymes, check liver function if necessary.
Option E:Option E: Some patients experi- enced transient exacerbation of rash after 2-4 weeks of medication.
Gold Explanation: This question examines the precautions for oral retinoic acid drugs. (1) Adverse reactions of isotretinoin are common, but most of them can be recovered after stop- ping the drug, and serious adverse reactions are rare or even rare. (2) The most common adverse reaction is dry skin and mucous mem- branes, and it is recommended to be used with skin barrier repair agents. (3) Rare adverse re- actions include musculoskeletal pain, elevated blood lipids, abnormal liver enzymes, and dry eyes, which usually oceur in the first months of treatment, and should be used with caution in patients with obesity, dyslipidemia, and liver disease. (4) Isotretinoin has a clear teratogenic effect. Female patients of childbearing age and their spouses should strictly avoid pregnancy month before treatment, during treatment and within 3 months after treatment. (5) It should be used with caution in patients who already have obvious depressive symptoms or suffer from depression.
GPT-4 Response: Answer: Alendronate Sodium. Analysis: This question examines the contraindications and precautions of the drug. Alendronate Sodium (Alendronate Sodium) is a drug used to treat fractures, osteoporosis and other symptoms. It improves bone density by inhibiting bone resorption. Drinking wa- ter should be limited when taking alendronate sodium, because it needs be taken on an empty stomach, and you must stand or sit up- right for at least 30 minutes immediately, and avoid lying down after taking the medicine to prevent side effects such as esophagitis, stric- ture and ulcer. Other options, such as cough syrup, dyphylline, ursodeoxycholic acid, and ritonavir, do not require fluid restriction. There- fore, the correct answer is F. alendronate. Error Analyse: Both GPT-4 and ChatGPT are wrong. The reason for the error the lack of mechanism by which cough syrup works. Example
(6) Some patients will ex- perience short-term exacerbation of skin rash after 2 to 4 weeks of use, which is usually tran- sient, and those with severe reactions need to reduce the dose even stop the drug. There- fore, choose B for this question.
ChatGPT Response: Answer: A. Contracep- tive measures should be taken during treatment and within 3 months after drug withdrawal. Analysis: Isotretinoin capsule is a medicine used to treat acne, and its main ingredient is retinoic acid. Contraceptive measures should be taken during drug treatment and within 3 months after stopping the drug, because the drug can affect fetal development. Know the contraindications before treatment, such pregnant women, lactating women, and those
Question:A 28-year-old female patient was given isotretinoin capsules because of acne. Which statement about drug education for this patient is false?
Option A:Contraceptive measures should be taken during treatment and within 3 months after drug withdrawal.

---

with severe liver and kidney impairment. If dry skin and mucous membranes are found during treatment, the dose should be reduced or the drug should be stopped to avoid aggravating the condition. The first 2 months of treatment may lead to abnormal liver enzymes, so the patient's liver function should be checked be- fore treatment, and if necessary, liver function should be checked regularly during the med- ication. Some patients experience transient exacerbation of the rash after 2 to 4 weeks of medication. This is normal phenomenon, and there is no need to stop the medication, and it will be relieved within few days. Therefore, the correct answer this question is A.
Option B:Selegiline.
Option C:Adamantadine.
Option D:Entacapone.
Option E.Benserazidd--levodoa.
Gold Explanation: This question examines the drug treatment of Parkinson's. 1. Early- onset patients without mental decline: (1) Pramipexole, selegiline or compound levodopa + COMT inhibitor (entecapone) should be the first choice if the treatment guidelines in the United States and Europe are followed; (2) If the patient cannot afford high-priced drugs due to economic reasons, amantadine is the first choice; (3) If due to the needs of special work, the patient strives significantly improve mo- tor symptoms or suffers from cognitive decline, the first choice the first choice Compound levodopa or compound levodopa + COMT in- hibitor (entecapone); (4) obvious tremor but poor efficacy of other drugs + trihexyphenidyl. 2. Late onset or accompanied by mental de- cline (1) Compound levodopa is generally the first choice for treatment. (2) Try not to use ticholinergic drugs, especially for elderly male patients. Therefore, choose E for this question.
GPT-4 Response: Answer: B. If skin and mu- cous membrane dryness occurs during treat- ment, the dose should be reduced or the drug should be discontinued. Analysis: Isotretinoin capsules may cause some adverse reactions during the treatment of acne. Contraceptive measures should be taken during treatment and within 3 months after drug withdrawal, because isotretinoin has strong teratogenic effect (choice A is correct). Dry skin or dry mucous membranes may occur during treat- ment, but dose reduction or drug withdrawal is not required; usually it can be relieved by using mild skin care products, lip balm, artifi- cial tears, etc. (option B is wrong). Adverse reactions are common, but most of them be recovered after stopping the drug (choice Cis correct). The first 2 months of treatment may lead to abnormal liver enzymes, check liver function if necessary (option D is correct). Some patients experienced transient exacerba- tion of rash during 2 to 4 weeks of medication (choice E is correct). Therefore, the wrong statement is option B.
ChatGPT Response: Answer: B. Selegiline Analysis: The main manifestation of Parkin- son's 's disease is movement disorder, because Parkinson's is disease is caused by the loss of dopaminergic neurons, so dopamine receptor agonists are one of the treatments for Parkin- son's disease drugs Pramipexole is also an effective treatment for Parkinson's 's disease in young people, but is less suitable for this patient because of its more side effects in older patients. Amantadine works similarly to dopamine in treating Parkinson's 's discase, but it often causes side effects in elderly patients, so it is not suitable for this patient. Entacapone is a combination drug for the treatment of Parkin- son's disease, which consists of levodopa and entapazide, but this drug prone to side ef- fects such as lowering of blood pressure, and is not suitable for elderly patients with mild cognitive impairment. Therefore, it is a better choice to choose B. selegiline, which is a se- lective dopamine D2 and D3 receptor agonist, which will not cause aggravation of dyskinesia symptoms such muscle stiffness and tremor, and can improve cognitive cognitive barriers. GPT-4 Response: Answer: D. Entacapone
Error Analyse: GPT-4 is right. ChatGPT is wrong. The explanation given by ChatGPT is correct and conforms to the gold explanation, but the label it gives is wrong.
Example 6
Question:The patient, male, 62 years old, de- veloped bradykinesia 4 years ago and was diag- nosed with Parkinson's 's disease. Symptoms are currently progressively worsening with mild cognitive impairment. Which drug should be used?
Option A:pramipexoe.

---

Analysis: Parkinson's 's disease treatment drugs include dopamine precursors (levodopa and carbidopa), dopamine receptor agonists (se- legiline, pramipexole), COMT inhibitors Enta- capone, MAO-B inhibitors (Utahsi, Ruizuosi, etc.), anticholinergic amantadine. This patient is 62 years old, with medical history of 4 years. The treatment effect of dopamine pre- cursor combined with dopamine receptor ago- nist is more accurate. Considering the patient has mild cognitive impairment, the anticholin- ergic drug amantadine should be avoided. It is planned to use entacapone the COMT in- hibitor, which is beneficial to prolong the effect of dopamine prodrugs and improve the symp- toms of patients. Therefore, Entacapone was chosen. The correct answer D.
Error Analyse: Both are wrong. The model is more inclined to prescribe medicine to the disease, and it is difficuul to make diagnosis based on the specific situation of the patient.

---

Model
Type
Prompt
Ace(%)
Rouge-1
Rouge-2
Rouge-L
GPT4
O-shot
w/ instruction
70.4
0.343
0.107
0.209
GPT-
1 I-shot
w/ instruction
75.1
0.383
0.135
0.247
GPT-
4 4-stot
w/ instruction
75.7
0.382
0.136
0.245
GPT4
8-shot
w/ instruction
74.6
0.368
0.127
0.228
GPT4
O-shot
w/o instruction
70.4
0.343
0.107
0.209
GPT4
1 I-shot
w/o instruction
75.7
0.384
0.140
0.247
GPT4
4-shot
w/o instruction
74.1
0.381
0.131
0.243
GPT4
8-shot
w/o instruction
69.8
0.360
0.123
0.228
ChatGPT
O-shot
w/ instruction
49.7
0.336
0.105
0.206
ChatGPT
-hot
w/ instruction
53.4
0.342
0.113
0.214
ChatGPT
4-shot
w/ instruction
50.8
0.356
0.123
0.228
ChatGPT
8-shot
w/ instruction
50.3
0.373
0.129
0.230
ChaiGPT
O-shot
w/o instruction
49.7
0.336
0.105
0.206
ChatGTT
1-shot
w/o instruction
54.5
0.341
0.114
0.216
ChatGPT
4-shot
w/o instruction
52.9
0.355
0.119
0.219
ChatGPT
8-shot
w/o instruction
49.7
0.356
0.120
0.221
GPT-3
O-shot
w/ instruction
28.0
GPT-3
1-shot
w/ instruction
30.7
GPT-3
4-shot
w/ instruction
33.8
GPT-3
O-shot
w/o instruction
28.0
GPT-3
1 1-shot
w/o instruction
34.4
GPT-3
4-shot
w/o instruction
40.2
CharGLM-66
O-shot
w/ instruction
24.3
0.281
0.082
0.156
CharGLM-6B
1 1-shot
w/ instruction
28.0
0.294
0.089
0.178
ChatGLMM-B
4-shot
w/ instruction
23.3
0.310
0.094
0.184
ChatGLM-6B
8-shot
w/ instruction
29.1
0.315
0.099
0.184
CharGLM-6B
O-shot
w/o instruction
24.3
0.281
0.082
0.156
ChatGLM-6B
1 shoo
w/o instruction
24.3
0.306
0.094
0.180
ChatGLM-6B
4-shot
w/o instruction
23.8
0.309
0.097
0.185
ChatGLM-6B
8-shot
w/o instruction
26.5
0.309
0.097
0.181
BELLE-7B-21 2M
O-shot
w/ instruction
28.0
0.267
0.067
0.169
BELLE-7B-2M
1 1-shot
w/ instruction
25.9
0.208
0.054
0.132
BELLE-7B-2M
4-shot
w/ instruction
29.1
BELLE-7B-2M
O-shot
w/o instruction
28.0
0.267
0.067
0.169
BELLE-7B-2M
1 I-shot
w/o instruction
30.7
0.208
0.054
0.132
BELLE-7B-2M
4-shot
w/o instruction
33.3
Yuat
O-shot
w/ instruction
27.0
ChatYuan
1-shot
w/ instruction
26.5
CharGLM-Med
0-shot
w/ instruction
17.5
ChatGLM-Med
1-shot
w/ instruction
10.1
Huatuo-Llama -aaino---aa--Mee-----
Med-Chinese
0-shot
w/ instruction
18.5
Huatuo- Lalooo-llaa-MM-aa Med-Chinese
1-shot
w/ instruction
19.6
Table 8: Performance comparison ExplainCPE.


Reviews:
[{"review_id": "J1fQhbSmCE", "reviewer": "Reviewer_1fgk", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for taking the time to review our paper and for providing feedback. We'd like to address your concerns point by point.\n\n**Quality of the Proposed Dataset & Human Evaluation on GPT Models vs. Expert Evaluation on Golden Labels:**\nThe essence of the paper centers on introducing the new Chinese pharmacist examination dataset. We ensured during its formation that the data was gathered from trustworthy and authoritative sources. The dataset collection process is elaborated upon in Line 233-258. **Each official question's explanation is sourced from official examination solution books**, such as the one from [2017](https://www.amazon.com/%E8%8D%AF%E5%AD%A6%E7%BB%BC%E5%90%88%E7%9F%A5%E8%AF%86%E4%B8%8E%E6%8A%80%E8%83%BD-%E5%9B%BD%E5%AE%B6%E6%89%A7%E4%B8%9A%E8%8D%AF%E5%B8%88%E8%B5%84%E6%A0%BC%E8%80%83%E8%AF%95%E5%91%BD%E9%A2%98%E7%A0%94%E7%A9%B6%E5%A7%94%E5%91%98%E4%BC%9A/dp/B06ZZFK25X/), which is authored by the National Pharmacist Qualification Examination Questionnaire Research Committee. This ensures the accuracy and rigor of the explanations.**In addition to the official questions, we also collaborated with three doctoral students from Peking Union Medical College (all of whom have undergone standardized residency training). They manually reviewed 320 samples from the collected data to evaluate the completeness and accuracy of the label and explanations. The evaluation resulted in the 99.4%/99.0% accuracy rate, with 318/317 out of the 320 samples being deemed correct.** We will enhance the paper by incorporating this detailed evaluation of dataset quality. Such improvements can encompass both a statistical assessment of label correctness and a qualitative review of the coherence and accuracy of the provided explanations. \n\n\n**Sources for Data Collection:**\nThe confidentiality and integrity of the data sources are crucial for the credibility of our dataset. We would like to extend the paper in the appendix to include a detailed section on the sources of our data collection, the methodologies used, and the precautions taken to ensure data fidelity.\n\n**Appropriateness for the NLP Application Track:**\nWhile the paper might seem resource-centric due to its focus on a new dataset, the evaluation of the dataset on popular GPT models makes it relevant to the NLP application track. We believe that introducing and evaluating new datasets is pivotal to pushing the boundaries of current models and driving research in specialized areas, like Chinese pharmacist examinations in this case. However, we respect the thematic guidelines of the track and are willing to make necessary amendments to better suit the scope.\n\nWe believe that with the suggested improvements, our paper will stand as a valuable contribution to the community. We appreciate your insightful feedback and are eager to refine our work based on it.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper proposes a new data set of Chinese Pharmacist examination. This data set includes not only the correct answer but also the explanations.\n\n2. The authors evaluate on three popular GPT models, ChatGLM, ChatGPT and GPT-4.", "reasons_to_reject": "1. The major contribution of this paper is the proposed dataset. However, the quality of this data set is not evaluated. For example, whether the label is correct, whether the explanation is accurate and coherent.\n\n2. The human evaluation is conducted on the results of GPT models. An expert evaluation on the golden labels should be included.\n\n3. The sources where the data set is collected should be described in detail.\n\n4. It's more like a resource paper and may not be proper for the NLP application track.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposes a Chinese pharmacist examination data set, which are mostly multiple choice problem. For each question, both the correct answer and the textual explanation are given. The authors use this data set to evaluate the performance of some GPT models, like ChatGLM, ChatGPT and GPT-4.", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "xGoFE7yY0g", "reviewer": "Reviewer_xaZa", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful feedback on our paper. We appreciate both the commendations and the critiques, and would like to address your concerns directly.\n\n**Nature of the Paper:**\nWe understand the concern about the paper primarily being dataset-centric without proposing a novel method. However, we'd like to emphasize that the introduction of specialized datasets, especially in domains as critical as medicine, is pivotal for the advancement of natural language processing (NLP) techniques. ExplainCPE fills a gap in the NLP community by providing a resource that can help in understanding the intricacies of medical data.\n\n**Comparison of LLMs:**\nWhile the paper doesn't introduce a new model, the comparative study on how different LLMs, including GPT-4, perform on ExplainCPE offers invaluable insights. This analysis not only benchmarks the models but also directs future research towards the areas where they may be lacking, especially in specialized fields like medicine.\n\n**EMNLP Acceptance Rate:**\nWhile we acknowledge EMNLP's competitive acceptance rate, the unique nature and the value of ExplainCPE, combined with the thorough evaluation on prominent models, make a strong case for its consideration. Our work can act as a foundation for many downstream tasks and inspire novel methodologies tailored for medical NLP tasks.\n\nIn conclusion, while the primary contribution of our paper is the dataset, its significance and potential implications for future research are immense. We hope the committee will recognize the potential impact of ExplainCPE on the community and the advancements it could drive in medical NLP tasks.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The creation of ExplainCPE, a medical dataset consisting of over 7K problems from Chinese Pharmacist Examination, is a significant contribution. \n2. The comparison of different LLMs' performance, with GPT-4 outperforming others, provides valuable information about the strengths and weaknesses of current models in the medical domain.\n", "reasons_to_reject": "1. A dataset paper that didn't propose a novel method.\n2. EMNLP has an acceptance rate. ", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper introduces an important and novel dataset, ExplainCPE, in the specialized domain of medicine.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "mHCSakhpA6", "reviewer": "Reviewer_CyCb", "summary": "", "questions": "- GPT-3.5 v.s. ChatGPT: In most of the literature, the `text-davinci-003` model is usually called \"GPT-3\" while the `ChatGPT` model is usually called `GPT-3.5`.", "limitations": "", "rating": 3, "confidence": 5, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to express our sincere gratitude for your meticulous review and valuable feedback on our paper. Your detailed observations offer us an opportunity to clarify certain aspects and provide a clearer perspective. We'd like to address each of your concerns.\n\n**Choice of LLM Baselines:**\n\nWe appreciate your suggestion to assess domain-specific models like BenTsao and DoctorGLM using the resources provided. In our revised paper, we plan to include test results from additional Chinese medical Large Language Models (LLMs) to make our benchmarking more comprehensive. It's worth noting that many checkpoints were not publicly disclosed at the time our paper was originally submitted.\n\nWe have already evaluated two popular medical LLMs—ChatGLM-Med and Huatuo-Llama-Med-Chinese—and found that they struggled with our multi-choice questions. The accuracy scores were as follows:\n\n- ChatGLM-Med in a zero-shot setting: 17.5\n- ChatGLM-Med in a one-shot setting: 10.1\n- Huatuo-Llama-Med-Chinese in a zero-shot setting: 18.5\n- Huatuo-Llama-Med-Chinese in a one-shot setting: 19.6\n\nThese findings indicate room for improvement in these domain-specific models.\n\n**GPT-3.5 vs. ChatGPT Nomenclature:**\nWe apologize for the oversight regarding the nomenclature of the GPT models. You are right; the naming convention we used can lead to confusion. We will correct this in the revised paper, adhering to the widely accepted nomenclature in the literature.\n\n**Incremental Work and Further Revision:**\nWe understand your sentiments regarding the paper's merits and the areas it could benefit from further refinement. Our primary goal was to introduce a novel dataset and provide preliminary insights into its challenges. We take your feedback seriously and believe that another round of revision can significantly enhance the paper, addressing the mentioned weaknesses.\n\nIn conclusion, we believe that with the suggested improvements and additional evaluations, our work will present a more solid contribution to the research community. We appreciate the time and expertise you invested in reviewing our paper and are committed to refining our work based on your valuable feedback.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- This paper releases a new Chinese medical Q&A dataset that builds a new benchmark for testing the performance of medical LLMs.\n\n- Associated with the Q&A data, the medical explanations are also offered as a way to verify the correctness of knowledge stored in LLMs when making generations.", "reasons_to_reject": "- The compared baselines are all not specifically in the medical domain. For instance, although ChatGLM-6B, BELLE, and ChatYuan were trained on Chinese corpus, they were not designed to answer medical questions. Surprisingly, GPT-4, without specific fine-tuning, yields over 75\\% on the benchmark. It would be more convincing about the challenge of this benchmark if we could test more Chinese medical LLMs over it, e.g., a bunch of open Chinese medical LLMs is available at https://github.com/HqWu-HITCS/Awesome-Chinese-LLM#%E5%8C%BB%E7%96%97.\n", "questions_to_authors": "- GPT-3.5 v.s. ChatGPT: In most of the literature, the `text-davinci-003` model is usually called \"GPT-3\" while the `ChatGPT` model is usually called `GPT-3.5`.", "paper_topic_and_main_contributions": "This paper releases a medical Q&A dataset with 7K problems from the Chinese Pharmacist Examination. The authors then did a comprehensive analysis of the state-of-the-art large language models over this new benchmark. The results show most LLMs trained on general text do not answer well.\n", "excitement": 3, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes ExplainCPE, a new dataset focusing on the Chinese Pharmacist Examination. This dataset is unique in that it includes not only multiple-choice questions but also their corresponding textual explanations. Three prevalent GPT models (ChatGLM, ChatGPT, and GPT-4) are evaluated on this dataset. While reviewers appreciate the contribution of a new dataset and some preliminary evaluations, they also raise concerns regarding the quality of the dataset, the scope of the evaluations, and the paper's fit within the NLP application track. The quality of the dataset is not sufficiently validated. All reviewers agree that the creation of a new medical dataset specific to the Chinese Pharmacist Examination is a valuable contribution to the community. Questions have been raised regarding the correctness of the labels and explanations. The evaluation lacks a comparison with domain-specific LLMs, which would provide a better context for understanding the utility of the dataset. The paper should also provide more information on how the dataset was collected, which would help assess its reliability and usefulness. Considering both positive and important issues highlighted by the reviewers, this paper falls into the "Borderline Sound" category.
Decision: Accept-Findings