E xplain CPE : A Free-text Explanation Benchmark of C hinese Pharmacist Examination (2023)

Abstract: In the field of Large Language Models (LLMs), researchers are increasingly exploring their effectiveness across a wide range of tasks. However, a critical area that requires further investigation is the interpretability of these models, particularly the ability to generate rational explanations for their decisions. Most existing explanation datasets are limited to the English language and the general domain, which leads to a scarcity of linguistic diversity and a lack of resources in specialized domains, such as medical. To mitigate this, we propose ExplainCPE, a challenging medical dataset consisting of over 7K problems from Chinese Pharmacist Examination, specifically tailored to assess the model-generated explanations. From the overall results, only GPT-4 passes the pharmacist examination with a 75.7% accuracy, while other models like ChatGPT fail. Further detailed analysis of LLM-generated explanations reveals the limitations of LLMs in understanding medical text and executing computational reasoning. With the increasing importance of AI safety and trustworthiness, ExplainCPE takes a step towards improving and evaluating the interpretability of LLMs in the medical domain.

## introduction advancements in the field of large language models (llms) @xcite @xcite , exemplified by models such as @xcite , have opened up new possibilities and challenges across a myriad of natural language processing (nlp) tasks @xcite . these models have shown remarkable success in understanding and generating human-like text, promoting research that spans a wide array of applications @xcite . a critical aspect that remains unexplored is the interpretability of these models, specifically the ability to provide accurate and faithful rationale for their decisions @xcite . the degree to which these models can explain their reasoning is of crucial significance, especially in high-stakes domains such as healthcare, where the clarity of generated responses directly impacts decision-making and outcomes @xcite . current benchmarks for llms mostly focus on the exam performance, such as mmlu @xcite and agieval @xcite . these datasets do not allow for a detailed assessment of llms' justifications of their decisions, because of the unavailability of high-quality and professional explanations. moreover, accurately measuring the explainability of these llms is a difficult task due to the lack of comprehensive and standardized datasets that come from unbiased and trustworthy sources @xcite . existing benchmarks predominantly are from online forums and consumer feedback and only consist of english-language general knowledge questions @xcite , which results in insufficient thematic and linguistic diversity. overall, the lack of appropriate evaluation datasets has prevented a full understanding of llms' strengths and weaknesses in the field of interpretability. to address this gap, we introduce explaincpe, a challenging medical benchmark dataset in chinese, encompassing over 7k instances. this dataset, specifically tailored to evaluate the capacity of model explainability, diversifies the linguistic scope of interpretability research and allows for a rigorous assessment of model performance in a specialized, high-stakes domain. an example from our dataset is presented in ## results one of the main objectives of our dataset is to evaluate the interpretability of models by assessing the quality of the generated text. therefore, we not only measured the accuracy of the models but also required a useful and efficient evaluation metric. evaluating interpretability is a long-standing problem due to the diversity of interpretation forms and content. as suggested by @xcite , the quality of explanations generated by the joint method needs to be further verified. therefore, we chose two methods to evaluate the interpretability of the models: automatic metrics and human evaluation. for automatic metrics, we used rouge to measure the quality of the explanations provided by the models and accuracy to measure the models' performance. due to the model's input length limitation, we could not conduct eight-shot experiments for some models, such as gpt-3. moreover, some models did not respond with answers and explanations even when we requested them, which is why some models lack a rouge score. ## conclusion in this work, we propose explaincpe, a challenging medical dataset for natural language explanation evaluation. our study on explaincpe dataset demonstrates the potential of llms in medical question answering with explanations. our analysis of model performance on different types of questions reveals the strengths and limitations of different llms in terms of in-context learning. the error cases point out the need for further improvement in llms in explanation generation and text comprehension. further work can use our dataset to improve and evaluate the model interpretability.