MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoning
Authors: Shuyue Stella Li
Vidhisha Balachandran
Shangbin Feng
Jonathan S. Ilgen
Emma Pierson
Pang Wei Koh
Yulia Tsvetkov
Venue: NeurIPS 2024 poster (2024)
Keywords: Clinical Reasoning
Question Asking
Information-Seeking
Adaptive Interactions
LLM Abstention
Primary area: natural_language_processing
TL;DR: This paper establishes a novel framework for interactive information seeking to enhance reliable medical reasoning abilities in LLMs.

Abstract: Users typically engage with LLMs interactively, yet most existing benchmarks evaluate them in a static, single-turn format, posing reliability concerns in interactive scenarios. We identify a key obstacle towards reliability: LLMs are trained to answer any question, even with incomplete context or insufficient knowledge. In this paper, we propose to change the static paradigm to an interactive one, develop systems that proactively ask questions to gather more information and respond reliably, and introduce an benchmark—MEDIQ—to evaluate question-asking ability in LLMs. MEDIQ simulates clinical interactions consisting of a Patient System and an adaptive Expert System; with potentially incomplete initial information, the Expert refrains from making diagnostic decisions when unconfident, and instead elicits missing details via follow-up questions. We provide a pipeline to convert single-turn medical benchmarks into an interactive format. Our results show that directly prompting state-of-the-art LLMs to ask questions degrades performance, indicating that adapting LLMs to proactive information-seeking settings is nontrivial. We experiment with abstention strategies to better estimate model confidence and decide when to ask questions, improving diagnostic accuracy by 22.3%; however, performance still lags compared to an (unrealistic in practice) upper bound with complete information upfront. Further analyses show improved interactive performance with filtering irrelevant contexts and reformatting conversations. Overall, we introduce a novel problem towards LLM reliability, an interactive MEDIQ benchmark and a novel question-asking system, and highlight directions to extend LLMs’ information-seeking abilities in critical domains.

MEDIQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoning
Shuyue Stella Li Vidhisha Balachandran'
Shangbin Feng'
Jonathan S. Ilgen'
Emma Pierson
Pang Wei Koh Yulia Tsvetkov
Uniiersiity of Washington
Carmegie Mellon University *Cornell Tech
Allen Institute for AI
stelliecs washington.edu stell@@ss.vashhgttn.cc
https: / /ttttgttttuu....///////gg com com//steellllisy/meiii
https://huggiinfaace http:///wwa..aaaaa////////aaa//a
Abstract
Users typically engage with LLMS interactively, yet most existing benchmarks evaluate them in static, single-turn format, posing reliability concerns in interac- tive scenarios. We identify a key obstacle towards reliability: LLMs are trained to answer any question, even with incomplete context insufficient knowledge. In this paper, we propose to change the static paradigm to an interactive one, develop systems that proactively ask questions gather more information and respond reli- ably, and introduce an benchmark MEDIQ to evaluate question-aske in LLMs. MEDIQ simulates clinical interactions consisting of Patient System and an adaptive Expert System; with potentially incomplete initial information, the Expert refrains from making diagnostic decisions when unconfident, and instead elicits missing details via follow-up questions. We provide a pipeline to convert single-turn medical benchmarks into an interactive format. Our results show that directly prompting state-of-the-al LLMs to ask questions degrades performance, indicating that adapting LLMs proactive information-seeking settings is nontriv- ial. We experiment with abstention strategies to better estimate model confidence and decide when ask questions, improving diagnostic accuracy by 22.3%: how- ever, performance still lags compared to an (unrealistic in practice) upper bound with complete information upfront. Further analyses show improved interactive performance with filtering irrelevant contexts and reformatting conversations. Over- all, we introduce a novel problem towards I.I.M reliability, an interactive MEDIQ benchmark and a novel question-asking system, and highlight directions to extend LLMs' information-sccking abilities in critical domains.
ability
Introduction
General-purgee parproe- large language models (LLMs) designed to serve broad audience by following instructions and providing the most likely and general answers (Brown et al., 2020; Ouyang et al., 2022). However, in high-stakes decision making scenarios such clinical conversations, LLM assistants can be harmful if they provide general responses instead gathering missing information make informed decisions. As shown in Figure 1, standard medical question-answering (QA) tasks are formulated in single-turn setup where all necessary information provided upfront, and the model is not expected interact with users (Jin et al., 2021; Pal et al., 2022; Jin et al., 2019; Hendrycks et al., 2020). This QA paradigm diverges from real-world scenarios, where users may provide incomplete information. and effective decision-makin often requires an investigative process involving follow-up questions to clarify and gather necessary details (Trimble & Hamilton, 2016; Bomstein & Emler, 2001; Masic, 2022).
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Standard Medical QA Task
Realistic Medical QA Task Partial Information Start Desired informationSSolliii Behavior
ChatGPT Response
ys femas, have
fatigue and night sweats.
Hi docter. I am 35 femmll. have fatigue and night swvetts. What is causing g llis?
Hi doctor. am 35 you. female, I hare fatigue and night sweets. What I-
Travel history:
Medication:: Isoniazi,,
Enough Info?
Physical exam: ... Symptoms:
Have you traveled recently?
Chest X-ra.. Blood test causing
Here are some general cousse Infectiont...
Immigraced from Asia.
Hormonallmballrrec...
:
bnough infe?
isoiazzid
Strees Anwieyy....
What medications you taking
[..] rifampir axid, I..] for 5 months
Enough indo?
All necessary information given once
Current LUM assistants ony provide general response
Yes Arsover
Isanizuid
Investigative process
(a) Standard Setup- All Info Given (b) Realistic Setup
X
Info Seek Ability
(c) Realistic Setup- Info Seek Ability
Information Seeking Task. In standard medical QA tasks (left), all necessary information is given the assistant model the same time. When given partial information, current IJ only provides general responses (middle). In a more realistic scenario (right), the presentation of patient information relies on proactive elicitation from the doctor; our proposed MEDIQ framework operationalizes this scenario.
Figure 1:
This gap between existing benchmarks and reality calls for a paradigm shift to designing systems adept at navigating high-stakes interactive scenarios. Focusing on clinical interactions where context is often incomplete, we introduce MEDIQ, an interactive benchmark for medical evaluation with dynamic informattion-seeinn questions, address limitations of static single-turn QA benchmarks (Figure 2). Unlike conventional systems, which assume that all necessary information is readily available, MEDIQ acknowledges the inherent uncertainty in medical consultations where a typical patient does not have the expertise to distill all necessary and relevant information they need to provide. To achieve this, MEDIQ comprises two components: Patient system that simulates patient and responds to follow-up questions, and an Expert system that serves as a doctor's 's assistant and asks questions to the patient before making a medical decision. In this interactive clinical reasoning task, successful information Expert should decide, at each turn, whether it has enough information to provide a confident answer; not, it should ask a follow-up question.
We convert two medical QA datasets, MEDQA (Jin et al., 2021) and CRAFT-MD (Johri et al., 2023, 2024), into an interactive benchmark by parsing the patient records to only provide partial information in the beginning. We first develop and validate a Patient system that accurately answers Expert inquiries by retrieving the correct facts from the patient record. We then benchmark Expert systems based on state-of-the-ar (SOTA) LLMs including Llama-3 (Touvron el al., 2023), GPT-3.5 (Brown et al., 2020) and GPT-4 (OpenAI et al., 2024). to evaluate their proactive information seeking ability. It is striking that prompting these models to ask questions results in an 1 1.3% accuracy drop compared to starting with the same limited information and asking no questions, showing that adapting LLMs to interactive information-seeking settings is nontrivial. A key challenge is deciding when to ask follow-up question instead of directly providing an answer. With confidence estimation strategies such as rationale generation and self-consistency, we improve Expert performance by 22.3%, although a 10.3% gap remains compared to an upper bound when full information is presented at once.
Our results show that while SOTA LLMs perform relatively well with complete information, they struggle to proactively seek missing information in more realistic, interactive settings with in- complete initial information. By providing modular, interactive benchmark, we hope to facilitate the development of reliable LLM assistants for complex gpeisiin-mmkingg in healthcare and other high-stakes domains. Our main contributions are:
1. We identify the critical problem of information-sceking questions in reliable interactive LLM assistants. We propose a paradigm shift and a practical conversion pipeline from standard single- turn benchmarks into interactive settings with incomplete initial information.
2. We develop the MEDIQ Benchmark to simulate more realistic clinical interactions between a Patient System and an Expert System. We rigorously develop and test the Patient System to benchmark any Expert's information-sseking and clinical decision-making abilities.
3. We show that SOTA L.MSs such as L-ama-3- GPT-3.5 and GPT-4 struggle proactive information seeking, revealing a significant gap in this area.
4. We propose MEDIQ-Expert, our best Expert system with novel abstaining capabilities to reduce unconfident answers, to partially close the gap between the more realistic incomplete information setup and the existing full information setup.
2

---

Patient System
Expert System
Question
Patient Record
Patient Response
Enough Information answer?
Information Seeking Question
No
Yes
Final Answer
Figure 2: The MEDIQ Benchmark. MEDIQ operationalizes a more realistic dynamic clinical interaction between Patient system and an Expert system to evaluate info-seeking and question-asking.
2 MEDIQ: Dynamic Medical Consultation Framework Overview
Task Definition The dynamic medical consultation task simulates the iterative nature of real- world clinical interactions. This task starts by providing an initial patient description ko of their conditions to the Expert system. The initial information typically contains the patient's 's age, gender, and chief complaint for the visit. The Patient system has access the entire patient record K. {ko, ......, Ka J. and the necessary information to answer the multiple choice question is IC C AC At the start of the t-th turn, the knowledge available to the Expert system denoted as K₃-1 fko..... }. Given follow-up question 4:, the Patient system responds with r; {A|A KC}. The Expert knowledge is then updated as K, KG- ry. The main challenge of the task is for the Expert system to ask information-seekon i.e. Kr, K., at which point the Expert system is asked to make a final decision.
questions to expand Kr, until the knowledge gap is filled,
2.1 The Patient System
Patient Task As part of the MEDIQ framework, the Patient system simulates a human patient in climical conversations. The Patient system has access to the full patient record that is sufficient for the diagnosis, including symptoms, onset duration, medical history, family history, and/or relevant lifestyle factors. The Patient system uses the patient record and single rnformation-seekinn question from the Expert system to produce a coherent response consistent with the given patient information as shown in Figure A reliable Patient system is critical to simulate a real and accurate medical consultation process. We propose that any Patient system should be evaluated on (1) Factuality measuring if a patient's 's responses are faithful to the patient's 's record and history and (2) Relevance measuring if the patient's response answers the expert's ' question. Given the full patient record and the expert question, we propose and evaluate three Patient system variants: Direct, Instruct, and Fact-Select, to obtain the patient response. Exact prompts and examples in Appendix A.2.
1. Direct: Serving a baseline, the Patient treats the cesponse-generatinn as reading comprehension task with no additional instruction. The prompt includes the patient's 's record followed by the Expert's question and asks the model to directly respond the question using the given paragraph.
2. Instruct: The Patient is instructed to respond truthfully to the Expert's question using the patient record only. When the context does not contain an answer to the question, the Patient is instructed to refrain from answering.
3. Fact-Select: The Patient aims to improve the factuality of the response by decomposing the patient record into atomic facts and responds by selecting facts that are relevant to the Expert's question.
2.2 The Expert System
Expert Task The Expert system simulates the medical decision makiing process of experienced clinicians, who seek additional patient information and iteratively update their differential diagnosis. The Expert system is first presented with a medical question and limited patient information. As each turn, it assesses whether the provided information is sufficient to answer the question. If the Expert system is unconfident, can elicit evidence with follow-up information information-seeking question; otherwise, the Expert system deems the acquired information sufficient and provides a final answer. The performance the Expert system evaluated on the (I) efficiency of the conversation (number of follow-up questions) and (2) the accuracy of the final diagnosis.
2.2.1 Expert System Breakdown
Medical decision making is complex process involving clinical reasoning and proactive information- seeking (Bordage, 1999; Norman, 2005; Schmidt et al., 1990; Boshuizen & Schmidt, 1992; Patel
3

---

Question:
Initial
Assessment
Info
Abstention
Integration
Module
No
Yes
Question
Decision
Generation
Making
Figure 3: Expert system information flow breakdown.
al., 1994). We describe our proposed MEDIQ-EXpert, which operationalizes the Expert system by breaking down the task into five needicaly-groounded steps: (1) initial assessment, (2) abstention, (3) question generation, (4) information integration, and (5) decision making (Figure 3). Each step is modular and easily modifiable.
Step 1. Initial Assessment Module: Given limited patient intake information and the multiple choice question (MCQ) as the input, the goal this module is to provide an initial assessment of the patient. The Expert system is asked to produce paragraph that elaborates
the symptoms and options, and identifying potential knowledge gaps (e.g., additional symptoms, lab tests) missing for answering the question. This step is done only once at the beginning of the interaction, and we keep the output in the conversation thread for future turns to refer back to.
When the model is not confident, it should abstain from giving an answer and asks a information-sceking question instead. The goal of the Abstention Module is to evaluate the confidence level of the Expert system to make decision given the available information. The input to this module is the MCQ and the patient information consisting of the initial presentation and conversation log of follow-up questions and responses. We probe the confidence level of the model to reliably answer the question via prompting ($ 2.2.2). The output of this module is yes/no answer for whether to proceed to final answer. If the model is confident, it skips to decision making; otherwise, continues to question generation.
Step 2. Abstention Module:
When more information is deemed necessary, the goal of the question generation module to craft an information-seckle evidence such lifestyle factors and physical exam results. The input to this module is all previous reasoning steps, and the acquired patient information; the notion of atomic questions defined with respect to the medical domain in the prompt, and the output is atomic question to the patient.
Step 3. Question Generation Module:
question to elicit additional medical
Step Information Integration Module: When a patient response is returned to the Expert system, the information integration module aggregates all gathered patient information up to this point to update the understanding of the patient condition. This step simply appends a question-anse to the end of an existing conversation log, which will then be passed to the Abstention Module.
pair
When enough evidence is gathered, the Expert system leverages integrated patient information and medical knowledge to provide an accurate answer to the question. The input this module previous reasoning steps, the MCQ, and the gathered patient information, and the output is the chosen option. Exact prompts for all above sections are in Appendix B.
Step 5. Decision Making Module:
2.2.2 Expert System Variants with Different Abstention Strategies
One component of active cetrmaation-seeking is the ability to decide when to ask questions, which operationalize with the Abstention Module to either ask or anywer at each turn. Abstention reduces LLM hallucinations in low-confidence scenarios (Umapathi et al., 2023; Rawte et al., 2023) and mitigates misleading or insufficiently substantiated conclusions (Feng et al., 2024). We develop the following variants of the Abstention Module via different instructions to the LLM to probe its confidence in whether their parametric knowledge is sufficient to reliably answer the MCQ. Exact prompts are in Appendix B.2.
0. BASIC: As baseline, the model is asked to implicitly indicate its abstain decision by either
generating an atomic question or producing an answer to the MCQ.
1. Numerical: To get explicit understanding of the model's confidence, we first prompt the model to generate a numerical confidence score between 0 and following (Tian et al., 2023). Then, an arbitrary threshold is set to either proceed with a final answer or ask question.

---

2. Binary: Previous work has shown that LLMs struggle at producing numerical confidence scores (Srivastava et al., 2022; Geng et al.. 2023). To address this, the Binary variant enables a simple classification of whether enough information is present. This setup simplifies the decision process, but may lack the nuanced understanding of confidence levels.
3. Scale: Binary classification does not provide granularity where the decision is ambiguous. Scale abstention solves this issue by combining direct quantification with a manageable set of discrete, interpretable options. The model is given definitions of confidence levels on a 5-point Likert scale (e.g., "Very Confident", "Somewhat Conf ident"), "), and is asked to select a rating to express its confidence. An arbitrary threshold is set to either proceed with a final answer or ask a question.
4. Rationale Generation (RG): Model performance is shown to improve when prompted to generate a reasoning chain about the decision process (Wei et al., 2022; Marasović et al., 2021). This gives the model a longer context window for reasoning, allowing the final decision to be conditioned on previous generations. We attempt generalize this finding to the more complex interactive medical inforrntion-ssekiing setup by applying it to Numerical, Binary and Scale abstention prompts.
5. Self-Consistency (SC). To further improve the Expert system's abstaining decision, we apply Self-Consistency to the above variants. Self-consistency repeatedly prompts the LLM times and take the average (Numerical and Scale) or the mode (Binary) of the output as the final output, and is shown improve model performance (Wang al., 2022).
3 Experiments
We conduct experiments validate each component of MEDIQ. First, we evaluate the Patient system factualiyy and relevance metrics ($ 3.1). Then, we establish the correlation between information availability and accuracy by studying model performance with varying levels of input information (S 3.2.1). Finally, we improve the cooomattonnssekingg ability of LLMs under MEDIQ ($ 3.2.2).
Evaluation Dataset We convert MEDQA (CC-BY 4.0) (Jin et al., 2021) and CRAFT-MD (CC-BY 4.0) (Johri et al., 2023, 2024) into an interactive setup for our experiments. MEDQA is a standard benchmark for medical question answering with sample contains a paragraph of patient record ending with a multiple choice question. CRAFT-MD contains 140 dermatology patient records in similar format, among which 100 are collected from an online question bank and 40 are created by expert clinicians. We parse each patient record into age, gender, the chief complaint (primary reason for the clinical visit), and additional evidence. Only the age, gender, and chief complaint are presented to the Expert system, from which it is expected to elicit missing information. The resulting tasks are called iMEDQA and iCRAFT-MD. respectively. See Appendix C for detail.
train/dev/test samples. Each
3.1 Patient System Reliability Evaluation
We automate the evaluation of patient responses with factuality score and relevance score for the ease of scalability, and conduct manual annotations to validate our metrics (Appendix A.4).
Factuality Score measures whether the Patient system's 's response is consistent with the patient record. Each Patient response is first decomposed into a list of atomic statements, then we compute the percentage of atomie statements that are supported by the information in the patient record. The factuality score is the percent of supported statements averaged over all patients.
Relevance Score measures whether the Patient system's response answers the Expert's question. Since there is no oracle data on the correct answer for Expert follow-up questions, we construct a synthetic parallel evaluation dataset of questions and responses evaluate the relevance of Patient responses: given a patient record decomposed into atomic statements, we rephrase each statement into an atomic question, for which the statement is the ground truth answer. Then, the Patient system produces a response using the patient record and the generated atomic question. The average embedding semantic similarity between the generated response and the ground truth statement over the evaluation dataset is the resulting relevance score. See Appendix A. for more detail.
Setup We use GPT-3.5 as the base LLM for all three variants (Direct, Instruct, and Fact-Select) and compare the factuality and relevance scores. For factuality, we sample 1272 patient cases from MEDIQ interactions with follow-up questions generated by different Expert systems the Patient
5

---

Full
Initial
None
48-veaz-old monan presents with difficulty falling aslesp, dininished appetite, and tizedness. She has grown increasingly irritable and hopeless [...] diminished concentration, [...]
40-yeaz-ole old woran presents with difficulty falling asleep, diminished appetite, and tizedness. What is the best treatment for patient?
Nhat is the best treatment for this patient?
lost 8.8 1b [...] dzinks glass of wine instead of cating [...] What is
treatment this patient?
(A) Diazepam (B) Paroxetine (C) Zolpidem (D) Trazodone IIitial Infa Details Question
Figure 4: Non-interactive Expert system evaluation at various information availability levels. The question and
options provided to the Expert model in all three settings.
system sees diverse Expert questions and compute the average across all generated questions. For relevance, we use all 1272 patient records from the development set of MEDQA.
3.2 Expert System Experiments
3.2.1 Benchmarking Existing LLMs in Incomplete Information Scenarios
We evaluate the performance non-interactive Expert systems with varying information availability levels to observe the relationship between information availability and accuracy and to establish baselines. The baselines are evaluated at three initial information availability levels (Figure 4): Full. Initial, and None. The Full setup equivalent to the standard QA task, wherein all patient information is provided to the Expert system in the beginning; Initial only discloses the gender, age, and the chief complaint that leads to the clinical visit (c.g. fever, headache, etc.); None provides no patient information but only the MCQ to the Expert system.
3.2.2 Interactive Expert Systems
Expert Variants Without explicitly providing the option to ask follow-up questions, vanilla LLMs always answer with incomplete information and never ask for additional evidence. Therefore, we establish a question-asking Expert system baseline- BASIC-by prompting the LLM to either ask question or make a decision at each turn. To study abstention, we combine Numerical, Binary, and Scale abstention with rationale generation and self-consistency techniques described in $ 2.2.2.
Expert System Setup We evaluate and iCRAFT-MD for both the non-interactive and interactive settings. Analysis and ablations use GPT-3.5 results on iMEDQA only. Details on model version and compute are Appendix
(8B, 70B), GPT-3.5, and GPT-4 on iMEDQA
Expert System Evaluation Metric An ideal Expert system should be able to ask informative questions that allow it to arrive at accurate medical decisions efficiently. Since it is not trivial to measure the quality of medical information-seeking questions, we use the efficiency of the interaction (number of questions) and accuracy of the solution as proxies evaluate the clinical reasoning capabilities. Accuracy is strongly dependent on the amount of information available to the model (8 3.2.1), higher accuracy is correlated with stronger information-seeking ability of the LLM.
Results
4.1 How reliable is the MEDIQ Patient system?
Our results in Table I show that both the Direct and Instruct settings struggle with factuality. Qualitative analysis revealed that since the Direct setting did not receive any instructions on how to respond to the follow-up question, it sometimes responds with "Yes" or "No" instead of the atomic statements that contain the requested information. In the Instruct setting, the Patient system sometimes provide inferences instead of reciting the facts from the patient record. Some example failure cases are shown in Appendix A.3. On the other hand, the Fact-Select setting which generates the responses in more controlled environment increases factuality by 0.33 points and relevance by 0.04 points. Overall, these results suggest that using atomic facts as units of information significuntly reduces hallucination, improving the reliability of the Patient system in providing accurate and relevant responses to expert questions. We use the Fact-Select setting for the Patient system in all subsequent experiments and shift our focus to evaluate the Expert variants introduced in $ 3.2.
Model
Factuality Relevance
Direct
55.9
75.5
Instruct
62.8
78.6
Fect-Select
89.1
79.9
Table 1: Patient system reliability.

---

Non-Interactive
Interactive
Task
Model
Full
Initial
None
BASIC
BEST
Llama-3-88 (avg. 2:23 Qs)
Llama-3 3-708 (avg. 6.38
L-ama-3-8b
68.1+13
52,0=14
40.4+14
33.0=1.
45.8+14
GPT-3.5 (ovg. 0.47 Qs)
iMEDQA
Llama-3-70b
GPT-3.5
55.8+14
84.7.110
45.6#14
58.5114
46.3114
36.7+14
42.2+t3
55.1113
60.9
50.2+1.4
86 Samppee
GPT (avg. 2.47 Qs)
GPT-4
79.7+11
54,5+14
42.2+14
55,4+13
66.1+13
1Jama-3-8b
76.4=36
51,4442
29.3+38
42.9+42
50.0
Llama-3-70b
82.1=32
60.7+4.1
52.9+42
62.1+4.1
72.1+38
iCRAFT-MD GPT-335
82.1+32
53.6+42 29.3+38 45.0+42 59.3+42
GPT-4
91.4=24 67.9=39 43,6a37 73.623.7 84.3#31
Table 2: Accuracy at varying information availabilitics. BASIC gives LLM the option to ask questions: with the same starting information, BASIC performance degrades from non-interactive Initial. Bold en- tries surpass non-interactive Initial, but there is still a gap between Full (complete information upper bound) and interactive BEST.
Questions
Figure 5: Frequeney of conversation lengths the BASIC setting. Most mod- els don't tend to ask follow-up questions.
4.2 How do existing Non-Interactive LLMs perform with Limited Information?
As shown in Table 2, with decreasing amounts of patient information provided to the model. there is a pronounced drop in performance from the Full to Initial to None information availability levels. Shifting our attention to the BASTC interactive setup, the final accuracy is even lower than its non- interactive counterpart (Initial) with the same initial information (a average of 11.310.3 drop). We analyze performance sensitivity to prompt variations to ensure fair comparison and report results from additional LLMs in Appendix E.
Figure 5 shows the number of follow-up questions asked by the LLMs in the BASIC interactive setup. For majority of the samples, no model chooses ask any questions, showing the lack of ability in LLMs to proactively identify and elicit missing information. Within each LLM family (Llama/GPT), there is a correlation between model size, number of questions asked and accuracy. Overall, these results show a significant gap between model performance in idcalized settings and realistic, information-limites scenarios. None of the examined models excel at proactive information seeking in an interactive environment, suggesting that it is nontrivial to integrate information gathered from continuous interactions. Despite having some medical knowledge encoded during pretraining. LLMs struggle to compensate for the absence of detailed patient information, highlighting the need for advanced proactive iooommtiion-seking abilities in medical LLM applications.
4.3 How much of the performance gap can be closed by asking questions?
In Figure 6, We present summary of the information-sede seeking ability of MEDIQ Expert models with different abstain strategies by reporting the accuracy and number of questions (conversation efficiency). Recall that both the Numerical and Scale abstention methods require setting a confidence threshold, above which the Expert system will proceed to the final answer. We do a grid search for the threshold hyperparameter in Appendix D and report the best performance for each setting. Integrating a dedicated Abstention Module significantly enhances performance over the BASIC setup which directly prompts for follow-up questions or diagnoses. As the abstain strategies improve - by expressing confidence on a scale, verbal reasoning, and adding self-consistency - the expert model is able to better gauge the (lack of) patient information and continue the conversation by asking more questions and thereby improving the final accuracy.
Base abstention methods (Numerical, Binary, Scale) show little variance in effectiveness until combined with rationale generation, which consistently boosts performance across strategies, as supported by previous studies (Marasović et al., 2021; Wei et al., 2022; Feng et al., 2024). Notably, self-consistency alone degrades performance unless paired with rationale generation. Overall, the factor of
Scale Abstention (1-5 confidence rating) with Rationale Generation and Self-Consistent 3 achieves the best performance. Overall, Scale Abstention (1-5 confidence rating) with rationale generation and sclf-consistency factor of 3 achieves the best performance, outperforming the BASIC interactive setup by 22.3% and the non-interactive Initial setup by 12.1%. In information-sccrce scenarios, models tend to resort the most common option instead of specializing to the patient, and interaction enhances specialization (Appendix F).
This pattern is generalizable across different LLMs as shown in the BEST column of Table 2. Note that model size plays a big factor the performance of the interactive 70B surpass the non-interactive Initial setup with the best abstention, but smaller models still struggle.
larger than
7

---

48.7 51.1
47.948.6499 49.1
Acc 45,8
46.746.7
45.845
46.6
45.6
40
40-
Abstain Strategy
Accuracy (%)
Accuracy
BASIC
30-
47.2
47.2
47.9
47.3
Average
44.8
42.2
Numerical
20
20-
Binary
Scale
NI-inibal
10
Convo
#Q:3.52
10.71.6123.4
24.27.0716.
24.26.4810.5
Para
SC=1
SC=3
SC=1
5C=3 +
All
Relevant Unique
Both
Self-Consistency (SC) Rationale Generation (RG) Setting
Post Processing Setting
Figure 6: Accuracy of different abstaining strategies (with # follow-up questions noted in green). Self-consistency tend to improve perfor- mance only when combined with rationale generation.
Figure 7: BASIC analysis. Remov- ing irrelevant info and reformatting con- versation improve accuracy.
While informing LLMs when to ask questions through abstention helps improve their decision- making with limited information, best MEDIQ Expert system (Scale+RG+SC) still only closes 51.2% of the gap between the Non-Interactive Initial and the Full information scenarios. This indicates plenty of room for improvement to further enhance the information--sekingg ability of LLMs.
Analysis
In this section, we further analyze the factors that impact the performance the interactive Expert system. Since we observe similar trends across models and datasets, all analysis will be performed on the iMedQA dataset with GPT-3.5 due to cost and computation constraints.
5.1 Why does the BASIC interactive setup fail to perform clinical reasoning?
Recall from $ 4.2 that there is striking 11.3% relative drop in accuracy from BASIC to the non- interactive Initial information setting (NI-Initial) across all benchmarked LLMs (7.43% for GPT-3.5 on iMEDQA). In this section, we analyze failure modes of the BASIC system, where the Expert is simply given the option to ask follow-up questions, to understand the performance drop. We show that the ability to ignore irrelevant context and extract useful information from conversation format affects model performance.
There are two types of irrelevant context on model performance: unanswerable and repeated questions. As MEDIQ allows the Expert to ask any open-ended questions to the Patient to elicit information, some questions cannot be answered using the patient record. We filter out these unanswerable question-response pairs, keeping only record-baa questions and responses to assess the effect of ignoring irrelevant questions (Relevant). Secondly, although the Expert is instructed to not repeat any questions, upon inspection of the interaction history, many questions are repetitive, especially when the answer is not in the patient record. We hypothesize that the repetition shifts the model's ' attention to certain questions and thus hinders the performance. We remove repetitive questions and only keep the unique questions (using fuzzy lexical matching) to verify this hypothesis (Unique). Finally, we remove both unanswerable and repeated questions (Both).
Irrelevant Context
Conversation Format We further hypothesize that the dialogue format, different from the typical document format seen during LLM pre-training, also affects performance. To control for this, we convert the conversation format into paragraph format by discarding the Expert questions and only keeping the patient response for answerable questions, and rewriting the unanswerable questions into statements (c.g., The patient's s vaccine record is unavailable.) )) for each setting above.
As shown in Figure Relevant and Unique both improve performance by percentage points (pp), but the combined effect is indistinguishable from using either filter, which might be due to the fact that unanswerable questions tend be repeated. Converting the conversations into paragraph format further improves the performance (Para). Removing repetitive questions and converting to paragraph format (Unique-Parts surpasses BASIC by 5.7pp and NI-Initial by 2.3pp. This shows that, when given the option to ask follow-up questions, the information-seeking ability of the Expert system does help make more informed and accurate conclusions, but the model suffers from not being able to learn from realistic clinical dialogues.
8

---

wio RG, ECE+0. 2.26
+ RG, ECE-0.211
Scale-NC
Confidence Score
Scale-RG
Accuracy Cia
9.3
Accuracy (16)
Scale- Kale-RG-NC
Num+NC
Scale+
SC+NC NC
RCSC-+RG+SC-NC
Scale-NC
Scale-SC-NC
95
RG Scole+RG-SC--
Average Conversation Length
Conversation Progress (%)
Average Conversation Length
Figure 8: Effect of the abstention module on Expert system performance (efficiency & accuracy). ) (a) Accuracy over the number questions when the abstention response is not provided to the question generator context (NC) isolate the effect of confidence thresholds with increasing confidence thresholds at each point. Accuracy increases with Qs as threshold increases. (b) Expert confidence estimation throughout the interaction with and without rationale generation (RG), averaged over self-consistency levels I, 3, and 5. The expected calibration error (ECE) is shown in the legend. RG leads to more conservative and accurate confidence estimates. (c) Accuracy of Expert system variants with abstention response available (solid marker) vs. available (hollow marker) the question generator module. Including abstention response question generator context improves accuracy, and amplified with rationale generation (RG) and self-consistency (SC).
model performance. Each line is different abstain method
5.2 How does abstention improve Expert system performance?
As we saw in $ 4.3, the Abstention Module is effective in improving the performance of the Expert system. We decompose the clinical reasoning process of the Expert into deciding when ask questions and what question to ask, and show that both contribute to performance gains.
Abstain Threshold Affeets Number of Questions and Accuracy In the Expert systems described in $ 2.2, the question generation module takes the abstention response part of the context to produce follow-up question. We now investigate the effect of the abstention decision by removing the abstention response from the question generation module, which allows us to control for the same question generator and only vary the abstention module. We plot the Expert performance across various abstention strategies and confidence thresholds in Figure S(a). As the confidence threshold becomes higher (i.e., when the model becomes more cautious), the conversation length intuitively grows, and accuracy also increases. Notice that the Fixed setting, the abstention decision is not specialized to the current patient, but decided solely by the number of questions asked; this results in a drop in accuracy when the number of questions becomes too high. While all methods benefit from longer interactions, the accuracy still varies depending on the abstention strategy, suggesting that some abstention strategies are better at estimating model confidence.
Rationale Generation Influences Confidence and Performance To further investigate confidence estimation. Figure 8(b) plots the intermediate confidence scores of the Scale-abstention Experts with and without rationale generation (RG) throughout the interaction, averaged over all threshold and self-consistency levels. We observe that RG leads more conservative confidence estimates and lower expected calibration error (2.1 with RG vs. 2.9 without RG), which indicate that it enables more accurate identification of knowledge gaps via reasoning (Figure 9). These findings suggest that better confidence estimation lead to better interactive Experts, encouraging future work to extend confidence calibration techniques to interactive settings (Xie al., 2024; Yang et al.. 2024). Secondly, Figure 8(c) estimation and rationale-in the
shows that including the abstention module suppons--ontideene context of the question generation module significantly improves performance. Together, the results suggest that rationale generation doesn't 't differ from its no-rationale counterparts in terms of knowing when ask questions (Figure 8(a)), but contributes in asking better questions (Figure 8(c)). This process essentially extracts the model's 's internal medical knowledge to integrate into subsequent responses, from which we conjecture that rationale generation can be further improved by integrating external specialized domain knowledge (Feng et al., 2023) or collaborative decision making with other models and/or humans (Talebirad & Nadiri, 2023). Additionally, we show that the nature of the question (medical specialty and difficulty) impacts the Expert's 's interactive behavior in Appendix G.

---

Abstention Madule Response
REASON:
The patient provided curzent nodications but did not montion any known history of livez disease or allergies.
DECISION: Sonowhat Uncenfident.
Question Generation Module Response
ATOMIC QUESTION: Have you ever expezienced any symotons like yellowing of the skin or eyes, abdoninal pain, oz dazk urine which night indicate 1iver problen?
Figure 9: Example of rationale generation helping identify knowledge gaps to ask better follow-up questions.
6 Related Work
Medical Question Answering Systems Advancements in medical question answering (QA) systems have progressed from rule-based systems to LLM- -.o-erddedd agents. Notable medical QA benchmarks include MultiMedQA (Singhal et al., 2023a), which contains both multiple-choice and open-ended questions collected from various sources. To customize medical domain, prior work often finetune the model on medical knowledge data such as PubMed (Bolton al., 2022; Yasunaga et al., 2022; Wu et al., 2023a; Singhal et al., 2023a,b), or more recently on conversational medical datasets (Yunxiang et al., 2023; Han et al., 2023). Kim et al. (2024) further improves model performance on complex medical questions by dynamically forming multi-agent collaboration structures. Despite their proficiency in direct answer retrieval, the proactive information- -eenmattii--eeiingggg capability is not something these models are inherently designed to do. Our proposed methodological framework, MEDIQ. is designed to work as an overlay to these domain- specific models, enhancing them with the capability to actively seek additional information a structured and clinically relevant manner.
adapt a general pprppsse LLM to the
Interactive Models and Agents Interactive conversational models extend beyond the standard QA framework by engaging in a dialogue such as customer support and negotiation (Singh & Beniwal, 2022: Chakrabarti & Luger, 2015; He et al., 2018; Abdelnabi et al., 2023; JU et al., 2024), where iterative information gathering is crucial. Li et al. (2023) and Andukuri et al. (2024) attempt to use LLMs to elicit more information-rich human preference examples in everyday tasks. However, the application of these models in the medical domain remains limited (Li et al., 2021). Wu et al. (2023b) attempts to evaluate general-purposs LLMs and chain-of- thoutht reasoning on DDXPlus (Fansi Tchango et al., 2022), base- synethtic patient dataset. Hu et al. (2024) navigates the information-sseking scenario as a search problem by developing a reward model guided by uncertainty and includes medical diagnosis as one of the tasks, but if limited to binary questions. Johri et al. (2023) observed similar phenomenon where L.LM-doctors cannot elicit complete patient information, but do not focus on improving the information-seeking ability. The system proposed by (Tu et al., 2024) performs a multitude of medical tasks but does not explore the crucial problem of abstention. Lastly, multi-agent and Al collaboration frameworks have shown impressive interactive performance (Zhou et al., 2024; Wu et al., 2024, 2023c; Deng et al., 2024; Lin et al., 2024), and can greatly benefit from novel interactive abstention methods to seek additional information. Our work fills this gap via providing a benchmark, a dataset, and a framework to comprehensively studying ceemmattion-seeiing abilities in clinical decision-male making, and most importantly, opens the door for future endeavors in this direction.
7 Conclusion
In this paper, we identify a significant gap in current LLMs' capability to ask questions and proactively seek information in settings where personalization, precision, and reliability are critical. We propose a paradigm shift to interactive benchmarks by simulating more realistic clinical interactions where only partial information is provided initially by introducing MEDIQ. MEDIQ provides a benchmark to the community to evaluate the question-asking ability of LLMs, contributing towards developing reliable models. We showed that SOTA LLMs like Llama-3 and GPT-4 struggle to gather necessary information for accurate medical decisions. To address this problem, we presented MEDIQ a novel Expert system with improved confidence judgment and medical expertise, substantially improving clinical reasoning performance. MEDIQ operationalizes interactive and explicit clinical reasoning processes, with added interpretability in the reasoning flow of language models and decision making. We encourage future research to extend MEDIQ to more diverse Patient systems, expand medical knowledge integration, and customize the interactions to better serve the healthcare community.
10

---

Limitations
One limitation is the scarcity of datasets that contain detailed patient information sufficient for medical diagnosis which, to the best of our knowledge, was only met by MEDQA and CRAFT-MD. The majority of available medical datasets are designed to models' medical domain knowledge. Second, the Patient system in our benchmark relies on paid API; future work should establish an open-source Patient. Lastly, our evaluation framework, while designed to be more realistic, still limited to the multiple-choice format. However, the flexibility of MEDIQ allows easy extensions into open-ended settings with appropriate datasets and well-defined conversation-leve metric. Future work can focus on collecting rich dataset in open-ended medical consultations and expanding the MEDIQ framework.
Ethics Statement
Along with many potential benefits of an ideal future variant of the MEDIQ framework (e.g., providing reliable and personalized medical consultation when access medical experts is unavailable, or assisting medical experts in initial collection of information). it is important to emphasize multiple risks associated with mis-use of this framework.
First, MEDIQ is a carefully designed initial prototype. is not meant to be deployed to interact with users; its intended use is to provide an experimental framework to test clinical reasoning abilities of LLMs which are currently extremely limited.
MEDIQ built top of closed-source LLMs runs the risk of leaking confidential medical information, violating patient privacy. Future research expanding MEDIQ to new medical datasets should be aware of these risks, resorting to local securely stored LLMs or to reliable data anonymization methods.
There are many sources of potential biases in the framework, including social and cultural biases in LLMs, in the datasets, and possibly in prompts for LLM interactions and abstention that we designed. While outside the scope of this paper, addition to utility metrics we proposed, future research could incorporate fairness-oriented evaluations, e.g., breaking down the evaluation by user demographies.
If similar framework is used in real-world applications, users and clinicians should be trained to prevent the over-reliance on technology that is liable to make mistakes, and to understand its privacy and fairnsss risks.
Acknowledgements
We gratefully acknowledge support from the University of Washington Population Health Initia- tive. SSL, SF, and YT are supported by the National Science Foundation under CAREER Grant No. II$2142739, NSF grants No. IIS2125201, IIS2203097, and gift funding from Google, MSR, and OpenAI. PWK is supported by the Singapore National Research Foundation and the National AI Group in the Singapore Ministry of Digital Development and Innovation under the AI Visiting Professorship Programme (award number AIVP-2024-001). EP was supported by Google Research Scholar award, Optum, NSF CAREER #2142419, CIFAR Azrieli Global scholarship, a gift to the LinedIn-Cornell Bowers CIS Strategic Partnership, and the Abby Joseph Cohen Faculty Fund.
References
Sahar Abdelnabi, Amr Gomaa, Sarath Sivaprasad, Lea Schönherr, and Mario Fritz. Llm-deliberation: Evaluating lims with interactive multi-agent negotiation games, 2023.
Chinmaya Andukuri, Jan-Philipp Fränken, Tobias Gerstenberg, and Noah D. Goodman. Star-gate: Teaching language models to ask clarifying questions, 2024.
E. Bolton, D. Hall, M. Yasunaga, T. Lee, C. Manning, and P Liang. Stanford crfm introduces pubmedgpt 2.7b, 2022. URL https://hai sttanfrrd edu/neddu stanford- crfm- introduces puwsssggt-mmmmmrrmmme
Georges Bordage. Why did miss the diagnosis? some cognitive explanations and educational implications. Academic Medicine, 74(10):S138 1999.

---

Brian H Bornstein and A Christine Emler. Rationality in medical decision making: a review of the literature on doctors' decision-making biases. Journal of evaluation in clinical practice, 7(2): 97-107, 2001.
Henny PA Boshuizen and Henk G Schmidt. On the role of biomedical knowledge in clinical reasoning by experts, intermediates and novices. Cognitive science, 16(2):153-184, 1992.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhimi Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners, 2020.
Chayan Chakrabarti and George F Luger. Artificial conversations for customer service chatter bots: Architecture, algorithms, and evaluation metrics. Expert Systems with Applications, 42(20): 6878-6897, 2015.
Zeming Chen, Alejandro Hernández Cano, Angelika Romanou, Antoine Bonnet, Kyle Matoba, Francesco Salvi. Matteo Pagliardini, Simin Fan, Andreas Köpf, Amirkeivan Mohtashami, Alexan- dre Sallinen, Alireza Sakhaeirad, Vinitra Swamy, Igor Krawczuk, Deniz Bayazit, Axel Marmet, Syrielle Montariol, Mary-Anne Hartley, Martin Jaggi. and Antoine Bosselut. Meditron-7Okt Scaling medical pretraining for large language models, 2023.
Yang Deng, Lizi Liao, Zhonghua Zheng, Grace Hui Yang, and Tat-Seng Chua. Towards human- centered proactive conversational agents, 2024.
Arsene Fansi Tchango, Rishab Goel, Zhi Wen, Julien Martel, and Joumana Ghosn. Ddxplus: A new dataset for automatic medical diagnosis. Advances Neural Information Processing Systems, 35: 31306-3151 31318, 2022.
Shangbin Feng. Weijia Shi, Yuyang Bai, Vidhisha Balachandran, Tianxing He, and Yulia Tsvetkov. Knowledge card: Filling lims' knowledge gaps with plug-in specialized language models. In The Twelfth International Conference on Learning Representations, 2023.
Shangbin Feng, Weijia Shi, Yike Wang. Wenxuan Ding. Vidhisha Balachandran, and Yulia Tsvetkov. Don't 't hallucinate, abstain: Identifying Ilm knowledge gaps via multi- collaboration, 2024.
Jiahui Geng, Fengyu Cai, Yuxia Wang, Heinz Koeppl, Preslav Nakov, and Iryna Gurevych. A survey of language model confidence estimation and calibration. arXiv preprint arXiv:231 080998, 2023.
Tianyu Han, Lisa C. Adams, Mich-lic Papaioannou, Paul Grundmann, Tom Oberhauser, Alexan- der Löser, Daniel Truhn, and Keno K. Bressem. Medalpaca an open-source collection of medical conversational ai models and training data, 2023.
He He, Derek Chen, Anusha Balakrishnan, and Percy Liang. Decoupling strategy and generation in negotiation dialogues, 2018
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.00 033000 2020.
Zhiyuan Hu, Chumin Liu, Xidong Feng, Yilun Zhao, See-Kiong Ng, Anh Tuan Luu, Junxian He, Pang Wei Koh, and Bryan Hooi. Uncertainty of thoughts: Uncertainty-awaal planning enhances information seeking large language models. arXiv preprint arXiv:2402.03271, 2024.
Wei Huang, Xudong Ma, Haotong Qin, Xingyu Zheng, Chengtao Lv, Hong Chen, Jie Luo, Xiaojuan Qi, Xianglong Liu, and Michele Magno. How good are low-bit quantized llama3 models? an empirical study, 2024.
Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences. 11(14):6421, 2021.
12

---

Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. PubMedQA: A dataset for biomedical research question answering. In Kentaro Inui, Jing Jiang. Vincent Ng. and Xiaojun Wan (eds.). Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNI.P- IJCNLP), pp. 2567-2577, Hong Kong, China, November 2019. Association for Computational Linguisties. doi: 1/118653 URL http::://aclaathooooo.orggg org/ 1259.
Shreya Johri, Jaehwan Jeong, Benjamin A Tran, Daniel I Schlessinger, Shannon Wongvibulsin, Zhuo Ran Cai, Roxana Daneshjou, and Pranav Rajpurkar. Testing the limits of language models: A conversational framework for medical ai assessmentr. medRxiv, 2023.
Shreya Johri, Jachwan Jeong, Benjamin A. Tran, Daniel I Schlessinger, Shannon Wongvibulsin, Zhuo Ran Cai, Roxana Daneshjou, and Pranav Rajpurkar. CRAFT-MD: A conversational eval- uation framework for comprehensive assessment of clinical LLMs. In AAAI 2024 Spring Sym- posium on Clinical Foundation Models, 2024. URL https://opennreview net/forumiid= Bk2nbTDtm8.
Da JU. Song Jiang, Andrew Cohen, Aaron Foss, Sasha Mitts, Arman Zharmagambetov, Brandon Amos, Xian Li, Justine T Kao, Maryam Fazel-Zarandi, and Yuandong Tian. To the globe (ttg): Towards language-driven guaranteed travel planning, 2024. URL https::t///attttt org/ abs/ 2410. 16456.
Yubin Kim, Chanwoo Park, Hyewon Jeong, Yik Siu Chan, Xuhai Xu, Daniel McDuff, Hyeonhoon Lee, Marzyeh Ghassemi, Cynthia Breazeal, and Hae Won Park. Mdagents: An adaptive collaboration of Iims for medical deccision-making 2024. URL
http:///wwwwww.waaaae/eee/eee/aa harggabb/24444 2404. 15155.
Belinda Z Li, Alex Tamkin, Noah Goodman, and Jacob Andreas. Eliciting human preferences with language models. arXiv preprint arXiv:2310. 11589 2023.
Dongdong Li, Zhaochun Ren, Pengjie Ren, Zhumin Chen, Miao Fan, Jun Ma, and Maarten de Rijke. Semi-super variational reasoning for medical dialogue generation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 544-554, 2021.
Jessy Lin, Nicholas Tomlin, Jacob Andreas, and Jason Eisner. human-ai collaboration, 2024.
Decision-oriented dialogue for
Ana Marasović, Iz Beltagy, Doug Downey, and Matthew E Peters. Few-shot self-rationalizatioon with natural language prompts. arXiv preprint arXiv:2111.0826 2021.
Izet Masic. Medical decision making-an overview. Acta Informatica Medica, 30(3):230, 2022.
Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannanch Hajishirzi. Factscore: Fine- Fine-grained atomic evaluation of factual precision in long form text generation. arXiv preprint arXiv:2305. 14251, 2023.
Geoffrey Norman. Research in clinical reasoning: past history and current trends. Medical education, 39(4):418-427 2005.
OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidr, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, Irwan Bello, Jake Berdine, Gabriel Bernadett-Shapiro. Christopher Berner, Lenny Bogdonoff, Oleg Boiko, Madelaine Boyd, Anna-Luisa Brakman, Greg Brockman, Tim Brooks, Miles Brundage. Kevin Button, Trevor Cai, Rosie Campbell, Andrew Cann, Brittany Carey, Chelsea Carlson, Rory Carmichael, Brooke Chan, Che Chang, Fotis Chantzis, Derek Chen, Sully Chen, Ruby Chen, Jason Chen, Mark Chen, Ben Chess, Chester Cho, Casey Chu, Hyung Won Chung, Dave Cummings, Jeremiah Currier, Yunxing Dai, Cory Decareaux, Thomas Degry, Noah Deutsch, Damien Deville, Arka Dhar, David Dohan, Steve Dowling, Sheila Dunning, Adrien Ecoffet, Atty Eleti, Tyna Eloundou, David Farhi, Liam Fedus, Niko Felix, Simón Posada Fishman, Juston Forte, Isabella Fulford, Leo Gao, Elie Georges, Christian Gibson, Vik Goel, Tarun Gogineni, Gabriel Goh, Rapha Contijo-Lopes, Jonathan Gordon, Morgan Grafstein, Scott Gray, Ryan Greene, Joshua Gross, Shixiang Shane Gu, Yufei Guo, Chris Hallacy, Jesse Han, Jeff Harris, Yuchen He, Mike
13

---

Heaton, Johannes Heidecke, Chris Hesse, Alan Hickey, Wade Hickey, Peter Hoeschele, Brandon Houghton, Kenny Hsu, Shengli Hu, Xin Hu. Joost Huizinga. Shantanu Jain, Shawn Jain, Joanne Jang, Angela Jiang, Roger Jiang. Haozhun Jin, Denny Jin, Shino Jomoto, Billie Jonn, Heewoo Jun, Tomer Kaftan, tukasz Kaiser, Ali Kamali, Ingmar Kanitscheider, Nitish Shirish Keskar, Tabarak Khan, Logan Kilpatrick, Jong Wook Kim, Christina Kim, Yongjik Kim, Jan Hendrik Kirchner, Jamie Kiros, Matt Knight, Daniel Kokotajlo, Lukasz Kondraciuk, Andrew Kondrich, Aris Konstantinidis, Kyle Kosic, Gretchen Krueger, Vishal Kuo, Michael Lampe, Ikai Lan, Teddy Lee, Jan Leike, Jade Leung, Daniel Levy, Chak Ming Li, Rachel Lim, Molly Lin, Stephanie Lin, Mateusz Litwin, Theresa Lopez, Ryan Lowe, Patricia Lue, Anna Makanju, Kim Malfacini, Sam Manning. Todor Markov, Yaniv Markovski, Bianca Martin, Katie Mayer, Andrew Mayne, Bob McGrew, Scott Mayer McKinney, Christine McLeavey, Paul McMillan, Jake McNeil, David Medina, Aalok Mehta, Jacob Menick, Luke Metz, Andrey Mishchenko, Pamela Mishkin, Vinnie Monaco, Evan Morikawa, Daniel Mossing, Tong Mu, Mira Murati, Oleg Murk, David Mély, Ashvin Nair, Reiichiro Nakano, Rajeev Nayak, Arvind Neelakantan, Richard Ngo, Hyeonwoo Noh, Long Ouyang, Cullen O' Keeff, Jakub Pachocki, Alex Paino, Joe Palermo, Ashley Pantuliano, Giambattista Parascandolo, Joel Parish, Emy Parparita, Alex Passos, Mikhail Pavlov, Andrew Peng, Adam Perelman, Filipe de Avila Belbute Peres, Michael Petrov, Henrique Ponde de Oliveira Pinto, Michael, Pokorny, Michelle Pokrass, Vitchyr H. Pong, Tolly Powell, Alethea Power, Boris Power, Elizabeth Prochl, Raul Puri, Alec Radford, Jack Rae, Aditya Ramesh, Cameron Raymond. Francis Real, Kendra Rimbach, Carl Ross, Bob Rotsted, Henri Roussez, Nick Ryder, Mario Saltarelli, Ted Sanders, Shibani Santurkar, Girish Sastry, Heather Schmidt, David Schnurr, John Schulman, Daniel Selsam, Kyla Sheppard, Toki Sherbakov, Jessica Shieh, Sarah Shoker, Pranav Shyam, Szymon Sidor, Eric Sigler, Maddie Simens, Jordan Sitkin, Katarina Slama, Ian Sohl, Benjamin Sokolowsky, Yang Song, Natalie Staudacher, Felipe Petroski Such, Natalie Summers, Ilya Sutskever, Jie Tang, Nikolas Tezak, Madeleine B. Thompson, Phil Tillet, Amin Tootoonchian, Elizabeth Tseng, Preston Tuggle, Nick Turley, Jerry Tworek, Juan Felipe Cerón Uribe, Andrea Vallone, Arun Vijaveeriyya, Chelsea Voss, Carroll Wainwright, Justin Jay Wang, Alvin Wang, Ben Wang. Jonathan Ward, Jason Wei, CJ Weinmann, Akila Welihinda, Peter Welinder, Jiayi Weng, Lilian Weng, Matt Wiethoff, Dave Willner, Clemens Winter, Samuel Wolrich, Hannah Wong. Lauren Workman. Sherwin Wu, Jeff Wu, Michael Wu, Kai Xiao, Tao Xu, Sarah Yoo, Kevin Yu, Qiming Yuan, Wojeiech Zaremba, Rowan Zellers, Chong Zhang, Marvin Zhang, Shengjia Zhao, Tianhao Zheng, Juntang Zhuang, William Zhuk, and Barret Zoph. Gpt-4 technical report, 2024.
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang. Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens. Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback, 2022.
Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. Medmeqa: A large-scale multi-subject multi-choice dataset for medical domain question answering. In Conference on health, inference, and learning, pp. 248 260.. PMLR, 2022.
Vimla L Patel, José F Arocha, and David R Kaufman. Diagnostic reasoning and medical expertise. In Psychology of learning and motivation, volume 31, pp. 187-252. Elsevier, 1994.
Vipula Rawte, Amit Sheth, and Amitava Das. A survey of hallucination in large foundation models. arXiv preprint arXiv:2309.055: 05922, 2023.
Henk G Schmidt, Geoffrey R Norman, and Henny Boshuizen. A cognitive perspective on medical expertise: theory and implication lpublished erratum appears in acad med 1992 apr; 67 (4): 287]. Academic medicine, 65(10):611-21, 1990.
Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. Quantifying language models' sensitivity to spurious features in prompt design or: How learned to start worrying about prompt formatting, 2023.
Satwinder Singh and Himanshu Beniwal. A survey on near-human conversational agents. Journal of King Saud
and Information Sciences, (4(10))8852-8866 2022.
Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. Large language models encode clinical knowledge. Nature, 620-7972) 2023a.
14

---

Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Le Hou, Kevin Clark, Stephen Pfohl. Heather Cole-Lewis, Darlene Neal, Mike Schaekermann, Amy Wang, Mohamed Amin, Sami Lachgar, Philip Mansfield, Sushant Prakash, Bradley Green, Ewa Dominowska, Blaise Aguera y Arcas, Nenad Tomasev, Yun Liu, Renee Wong, Christopher Semturs, S. Sara Mahdavi, Joelle Barral, Dale Webster, Greg S. Corrado, Yossi Matias, Shekoofeh Azizi, Alan Karthikesalingam, and Vivek Natarajan. Towards expert-level medical question answering with large language models, 2023b.
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alons et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint arXiv:2206.04615. 2022.
Yashar Talebirad and Amirhossein Nadiri. Multi-agent collaboration: Harnessing the power of intelligent IIm agents. arXiv preprint 230xv233066033144, 2023.
Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher D Manning. Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback. arXiv preprint mXXv:2235.11497. 2023.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cris- tian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosscini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models, 2023.
Michael Trimble and Paul Hamilton. The thinking doctor: clinical decision making in contemporary medicine. Clinical Medicine, 16(4):343, 2016
Tao Tu, Anil Palepu, Mike Schaekermann, Khaled Saab, Jan Freyberg, Ryutaro Tanno, Amy Wang, Brenna Li, Mohamed Amin, Nenad Tomasev, Shekoofeh Azizi, Karan Singhal, Yong Cheng, Le Hou, Albert Webson, Kavita Kulkarni, S Sara Mahdavi, Christopher Semturs, Juraj Gottweis, Joelle Barral, Katherine Chou, Greg S Corrado, Yossi Matias, Alan Karthikesalingam, and Vivek Natarajan. Towards conversational diagnostic ai, 2024.
Logesh Kumar Umapathi, Ankit Pal, and Malaikannan Sankarasubbu. Med-halt: Medical domain hallucination test for large language models. arXiv preprint arXiv:2307.1534. 2023.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe Le, Ed Chi, Sharan Narang, Aakanksha Chowdh- ery, and Denny Zhou. Self- Self-consistency improves chain of thought reasoning in language models. arXiv preprint 2rXiv:2203.11177 2022.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 0-24824-24837, 2022.
Chaoyi Wu, Weixiong Lin, Xiaoman Zhang. Ya Zhang, Yanfeng Wang, and Weidi Xie. Pme-Ilama: Towards building open-source source language models for medicine. arXiv preprint wrXiv:2305.10415 2023a.
Cheng-Kuan Wu, Wei-Lin Chen, and Hsin-Hsi Chen. Large language models perform diagnostic reasoning, 2023b.
15

---

Guande Wu, Chen Zhao, Claudio Silva, and He He. Your co- workers matter: Evaluating collaborative capabilities of language models in blocks world, 2024.
Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang- Autogen: Enabling next-gen Ilm applications via multi-agent conversation framework. arXiv preprint arXiv:2388.0815. 2023c.
Johnathan Xie, Annie S Chen, Yoonho Lee, Eric Mitchell, and Chelsea Finn. Calibrating language models with adaptive temperature scaling. In ICLR 2024 Workshop Secure and Trustworthy Large Language Models, 2024. URL htttts:://pppeeeetetee
Ruixin Yang, Dheeraj Rajagopa, Shirley Anugrah Hayati, Bin Hu, and Dongyeop Kang. Con- fidence calibration and rationalization for 1lms via multi-agent deliberation. arXiv preprint arXiv:2404.09127, 2024.
Michihiro Yasunaga, Antoine Bosselut, Hongyu Ren, Xikun Zhang, Christopher D. Manning, Percy Liang, and Jure Leskovee. Deep bidirectional eagguage-knowledge graph pretraining. ArXiv, abs/2210.09338,, 2022. URL https: www..eeismmmaooal..............e
Terg/CorpusID.. CorpusIDD222296226. 252968266.
Li Yunxiang, Li Zihan, Zhang Kai, Dan Ruilong, and Zhang You. Chatdoctor: A medical chat model fine-tuned on Ilama model using medical domain knowledge. arXiv preprint arXiv:2303.14070, 2023.
Xuhui Zhou, Hao Zhu, Leena Mathur, Ruohong Zhang, Haofei Yu, Zhengyang Qi. Louis-Philiipp Morency, Yonatan Bisk, Daniel Fried, Graham Neubig, and Maarten Sap. Sotopia: Interactive evaluation for social intelligence language agents, 2024.
16

---

A Patient System
A.1 Evaluation Metries
The Patient System is responsible for answering follow-up questions from the Expert System to provide the inquired patient information. When answering the follow-up questions, the Patient System has access to 1) the full patient context and 2) the expert question, and is instructed to produce a factual answer grounded in the context information and make no inferences. To ensure a reliable informattio-ssekinng process, we evaluate the Patient System along two axes: Factuality Score and Relevance Score. The basis of both the factuality and relevance evaluation relies on atomic facts from the patient record as units of information, which we generate by prompting GPT-4 following Min et al. (2023).
Factuality score measures whether the responses produced by the patient model are factual with respect the given patient context information:
プ I(r C)
Ird
factuality(R,C)
= R
ri
TEER
where is the responses from the Patient System on the patient case C; the size of R depends on the number of deduplicated Expert questions on the patient C. I is the indicator function of whether atomic fact rf from the response is factually consistent (z=) with any statement in C:
I(r), C) = {1. 0. otherwise,
1, ic E S.t. rf =
(2)
Factual consistency of the response and any statement c in C. (r) c) is determined using semantic similarity for non-first person response patient variants and GPT-4 binary classification for first- person responses, since the atomic facts C are written in third person. Specifically, r! c in -= true if the semantic similarity, calculated by the cosine similarity of the Sentence' SentenceTTynnsorme saab-robbeta-larg embeddings, is greater or equal to 0.8 (or if the two sentences are determined to be consistent by GPT-4 judge in the first-person response case), and vice versa.
Relevance Score measures whether the generated patient response answers the expert question. Defined similarly to the hallucination rate:
Irl
Fi)
relevance
[
rd
PEE
where F, is the ground truth statements from the patient context that answer the same expert question that r; is trying to answer.
A.2 Patient System Variants
A.2.1
Patient System Variant: Direct
Prompt to LLM
Answer the question with the given context:
Context: "She says that, despite going to bed early at night, she is unable to fall asleep. She denies feeling anxious or having disturbing thoughts while in bed. Even when she manages to fall asleep, she wakes up early is unable to fall back asleep. She says she has grown
in the morning increasingly irritable and feels increasingly hopeless, and her concentration and interest at work have diminished. The patient denies thoughts of suicide death. Because of her diminished appetite, she has lost kg (8.8 1b) in the last feu weeks and has started drinking a glass of wine every night instead of eating dinner. She has significant past medical history and is not on any medications."
Question: "What time do you usually go to bed night?"
17

---

Example L.M Output
The patient goes to bed early at night despite experiencing difficulty falling asleep.
A.2.2 Patient System Variant: Instruct
Patient Response Generation Prompt Instruct
are truthful assistant that understands patient's information, and you are trying to answer questions from a medical doctor about the patient.
Below is context paragraph describing the patient and their conditions: "She says that, despite going to bed early at night, she unable to fall asleep. She denies feeling anxious having disturbing thoughts while in bed. Even when she manages to fall asleep, she wakes up early in the morning and is unable fall back asleep. She says she has grown increasingly irritable and feels increasingly hopeless, and her concentration and interest at work have diminished. The patient denies thoughts of suicide death. Because of her diminished appetite, she has lost 4 kg (8.8 1b) in the last few weeks and has started drinking glass of wine every night instead of eating dinner. She has no significant past medical history and is not on any medications."
Question from the doctor:
"What time do you usually go to at night?"
Use the context paragraph to ansser the doctor question. If the paragraph does not question, please do not ask this question again." " Answer only what the question asks for. Do provide any analysis, inference, implications. Respond with a straightforvard answer to the question ONLY and NOTHING ELSE.
the question, simply say "The patient cannot
this
Example LLM Output
Despite going to bed early at night, the patient is unable to fall asleep.
A.2.3 Patient System Variants: Fact-Select
Atomic faet decomposition Prompt
You are truthful medical assistant that understands the patient's information.
Break the following patient information into list of independent atomic facts, with piece of information in each statement. Each fact should only include the smallest unit of information, but should be self-contained. "She says that, despite going bed early night, she is unable to fall asleep. She denies feeling anxious or having disturbing thoughts while in bed. Even when she manages fall asleep, she wakes up early in the morning and is unable to fall back asleep. She says she has grown increasingly irritable and feels increasingly hopeless, and her concentration and intereat
18

---

Atomic fact decomposition Prompt (cont'd)
at work have diminished. The patient denies thoughts of suicide death. Because of her diminished appetite, has lost kg (8.8 1b) in the last few weeks and has started drinking glass of wine every night instead of eating dinner. She has no significant past medical history and not on any medications."
Reaponse with the list of atomic facts and nothing else, prepend each fact by an index starting from 1. No sub-list allowed.
Example LLM Output
P.Patin goes to bed early at night but is unable to fall asleep.
2. 2.Patient denies feeling anxious having disturbing thoughts while in bed.
3. 3.Patient wakes early in the morning and is unable to fall back asleep.
4. Patient has grown increasingly irritable and feels increasingly hopeless.
5. Patainnt's a concentration and interest at work have diminished.
6. S.Patient denies thoughts suicide death.
7 P.Patient has lost 4 kg (8.8 1b) in the last few weeks.
8.Patient started drinking glass of wine every night instead of eating dinner.
9.Patient has no significant past medical history.
10. Potiett is not on any medications."
Patient Response Generation Prompt Fact-Select
You are truthful assistant that understands the patient's information, and you are trying to answer questions a medical doctor about patient.
Below is a list of factual statements about the patient: 1. 1.Patient goes to bed early night but is unable to fall asleep. Pataien denies feeling anxious or having disturbing thoughts while in bed. P.Patient wakes up early in the morning and is unable to fall back asleep. Patient has groun increasingly irritable and feels increasingly hopeless. S.Patientss concentration and interest
work have diminished.
6. 6.Patient denies thoughts of suicide or death.
7. Pataien has lost 4 kg (8.8 1b) in the last few weeks.
P.Patient started drinking a glass of wine every night instead of eating dinner.
9. P.taiient has no significant past medical history.
10. Patient
any medications.
Question from the doctor: "What time do you usually go to at night?"
Which of the above atomic factual statements answer the question? Select at most two statements. If statement "The patient cannot answer this question, please do not ask this question again." Answer only what the question aska for. inference, or implications. Respond by selecting all statementa that ansver the question from above ONLY and NOTHING ELSE.
the question, simply say
provide any analysis,
Example LLM Output
Patient goes bed early night but is unable fall asleep.
19

---

A.2.4 Additional Patient System Variant: Fact-FP
In addition the three Patient System variants presented in the main paper, have explored two other variants: First Person (Fact-FP) and Binary Fact Classification (Fact-Classify). Fact-FP breaks down the patient record into atomic facts, select the relevant facts that answers the follow-up question, and respond by converting the atomic statements into first person perspective. Atomic fact decomposition prompt the same as in Appendix A.2.3.
Patient Response Generation Prompt Fact-FP
You patient with list symptoms, and you taak is truthfully anawer questions from a medical doctor.
Below is list of atomic facts about you, use ONLY the information in this list answer the doctor's question.
1. 1.Patient goes to bed early night but is unable fall asleep.
P.Paiient denies feeling anxious or having disturbing thoughts while in bed. 3. 3.Patient wakes up early in the morning and is unable to fall back asleep.
P.taiient has grown increasingly irritable and feels increasingly hopeless. P.Paiennt's concentration and interest
work have diminished.
6. Patiien denies thoughts of suicide or death.
7. Patinet has lost 4 kg (8.8 1b) in the last few weeks.
P.Patien started drinking a glass of wine every night instead of eating dinner.
9. P.Patient has no significant past medical history.
10. Patient is
any medications.
Question from the doctor: "What you usually go to at night?"
Which of the above atomic factual statements answer the question? Select at most two statements. If statement answers the question, simply say "I cannot answer this question, please do not ask this question again." Do not provide any amalysis, inference, implications. Respond by reciting the matching statements, then convert the selected statements into first person perspective
you the patient but keep the information. Generate your anawer in this format:
STATEMENTS:
FIRST PERSON:
Example L.M Output
STATEMENTS: "Patient goes to bed early at night but is unable to fall asleep." "
FIRST PERSON: "I go to bed early at night, but I can't fall asleep."
A.2.5 Additional Patient System Variant: Fact-Classify
Atomic fact decomposition prompt the same as in Appendix A.2.3.
Patient response generation prompt: for each atomic fact, repeat the following prompting process to decide whether answers the doctor question, and collect yes/no answer from the model. The final output is the set of atomic facts with yes response.
20

---

Patient Response Generation Prompt Fact-Classify
You are truthful assistant that understands the patient's information, and you are trying to answer questions from a medical doctor about patient. Your task is to decide whether the given statement answers the question:
STATEMENT: "Patient goes to bed early night but is unable fall asleep." QUESTION: "What time do you usually go to bed at night?"
Does the statement answer the question? Do not provide any analysis, inference, or implications. Respond with either YES or NO NOTHING ELSE.
Example LLM Output
A.3 Qualitative Patient System Failure Analysis
This section provides detailed analysis of failure cases observed in the MEDIQ Patient system variants Direct and Instruct. These examples highlight limitations and areas for potential improvement in Patient response accuracy and relevance.
Patient System Variant: Direct There are two common failures cases observed by the Direct Patient variant. First, it tends to respond with binary yes/no answers. Although the response is factually correct, it lacks the detail required for comprehensive medical assessment. First, this behavior does not mimic a typical, realistic patient. Second, detailed information about symptoms, their progression, and related factors is crucial to reaching diagnosis. Simple affirmative negative responses do not provide enough context or specifics for an expert to make an informed decision about the patient's 's condition. The conversation history, which is compiled to assist in final diagnostic decisions, becomes less useful it is filled with uninformative responses.
Example 1:
Atomic Question: Have you experienced any nausea recently?
Atomic Fact (ground truth): Nausea has subsided.
Patient Response: Yes.
Example
Atomic Question: Have you experienced similar episodes in the past?
Atomic Fact (ground truth): The patient has had similar episodes in the last
years.
Patient Response: Yes.
The second common error by the Direct Patient system is hallucinations. the example below, the Patient system's 's response includes hallucinated fact-claiming the initiation of insulin therapy, which is not supported by the patient record. This type of error is particularly problematic because hallucinations introduce false information into the patient's medical history, which lead to inappropriate medical decisions or interventions based on incorrect data. False information regarding medication can have direct negative effects on patient safety, which calls for the development of factually reliable Patient systems.
Example 1:
Atomic Question:
Can you describe any recent changes to your medication?
Eapecially anything related your diabetes?
Atomic Fact (ground truth): The patient is not on any medications.
Patient Response: started taking insulin last week for my diabetes.
21

---

Patient System Variant: Instruct The response below by the Instruct Patient system erroneously includes instructive content meant for the healthcare provider rather than addressing the question directly from the patient's 8 perspective. This indicates potential confusion in role-play within the interactive MEDIQ setup when the conversation log is compiled.
Atomic Question: Have you noticed any unuaual bruising on your body? Atomic Fact (ground truth): Patient experiences eaay bruising.
Patient Response: Due to the eaay bruising experienced by the patient and her family history similar issues, is important to patient if she has noticed any unusual bruising on her body.
A.4 Manual Evaluation of Fact-Select
In order better understand the quality of the Patient System variant used in subsequent Expert evaluations, we perform manual annotation of the quality of the responses. A simple preference- based annotation is performed, in which the annotators are provided with patient record, Expert follow-up question, and two responses generated by two different Patient systems and are asked to select the one that more factually correct and does not omit information. The annotators are also instructed to reduce their rating binary Yes/No answers due to formatting requirements of the framework, but they are free to select any option during annotation. We randomly select one question from each interaction that contains at least 3 follow-up questions, and generate responses with all three Patient systems. For each question, two responses are randomly selected and the annotator is asked indicate which response of higher quality. Win rate for each Patient system is calculated as the number of times responses from a system is selected over the number appearances of responses from this system. There is tie option for if neither both responses are good, which exclude when calculating the win rate. 500 questions are annotated and results are shown Table 3. We can that there is obvious advantage of Fact-Select Patient system over the other systems.
Direct
Instruct
Fact-Select
Win Rate
36.11
37.36
63.77
Table 3: Win rates from manual evaluation of Patient variants.
B Expert Model Prompts
B.1 Initial Assessment
MEDIQ Expert Initial Assessment Module Prompt
You medical doctor answering real-world medical entrance questions. Based on your understanding of basic and clinical acience, medical knowledge, and mechanisms underlying health, disease, patient care, and modes of therapy, anawer the following multiple choice question. Select one correct answer from referenced in medical guidelines.
Base your answer the current and standard practices
Task: You will be asked to reason through the current patient's information.
patient into the clinic presenting with basic information: "A 40-year-old woman presents with difficulty falling asleep, diminished appetite, and tiredness for the past weeks."
22

---

MEDIQ Expert Initial Assessment Module Prompt (cont'd d))
Given the information from above, your task is to choose of four options that best answers the inquiry.
INQUIRY: "Which of the following is the best course of treatment in this patient?"
OPTIONS: "A": "Diazepam", "B": "Paroxetine", "C": "Zolpidem", "D":
"Trazodone"
Medical conditions are complex, so you should seek to understand their situations across many features. First, consider which medical specialty is this patient's a case; then, consider list of necessary features a doctor would need make the right medical judgment; fimally, consider whether all necessary information is given in the conversation above. Think step by step, inquiry,
options. DO NOT provide the answer choice, keep your response to one paragraph.
reason about patient information,
Example LLM Output
Based on the patient's presentation of difficulty falling asleep, diminished appetite, and tiredness, the most appropriate course treatment would be consider prescribing medication typically used for depression, like paroxetine trazodone. These medications can help address symptoms of depression, which can manifest as changes in sleep patterns, appetite, and energy levels. Diazepam and zolpidem not typically indicated for primary treatment of underlying depressive symptoms, Further evaluation, including proper assessment for depression consideration of counseling therapy, may also be heneficial in the overall management of this patient.
B.2
Abstention Module
For this section, all prompts are appended after the prompt and response from Appendix B.1 in multi-turn conversation manner.
B.2.1
BASIC Abstain
MEDIQ Expert BASIC Abstention Module Prompt (appended to previous conversation)
Considering factors above, if you are confident to pick an option correctly and factually, respond with the letter choice and NOTHING ELSE. Otherwise, if you are not confident to pick an option and need more information, ask ONE SPECIFIC ATOМIC QUESTION to the patient. The question should be bite-sized, and NOT ask for too much at once. In this case, respond with the atomic question and NOTHING ELSE.
Example LLM Output
"What time do you usually go to bed night?"
B.2.2 Binary Abstain
MEDIQ Expert Binary Abstention Module Prompt (appended previous conversation)
Considering factors above, are you confident to pick the correct option to the inquiry factually using the conversation log? Answer with or and NOTHING ELSE.
23

---

Example L.M Output
YES
B.2.3
Numerical Abstain
MEDIQ Expert Numerical Abstention Module Prompt (appended to previous conversation)
Considering factors above, what your confidence score pick correct option to the inquiry factually using the conversation log? Answer with the probability as float from 0.0 1.0 and NOTHING ELSE.
Example LLM Output
0.7
B.2.4 Scale Abstain
MEDIQ Expert Scale Abstention Module Prompt (appended previous conversation)
Considering factors above, how confident you pick the correct option to the problem factually using the conversation log? Choose between the following ratings:
correct option supported by all evidence, and there is enough evidence to eliminate the rest of the answers, the option can be confirmed conclusively.
"Very Confident"
"Somewhat Confident" correct option is more likely than other options, more information is helpful to make conclusive decision.
have reasonably enough information to tell that the
"Neither Confident or Unconfident" There are evident supporting the correct option, but further evidence is needed to be sure which one is the correct option.
"Somewhat Unconfident" There are evidence supporting more than options, therefore more questions are needed to further distinguish the options. "Very Unconfident" There not enough evidence supporting any of the options, the likelihood of picking the correct option at this point is near random guessing.
Answer the following format:
DECISTON: chosen rating from the above list.
Example LLM Output
Somewhat Confident
24

---

B.2.5 Rationale Generation
MEDIQ Expert Rationale Generation Prompt (appended to previous conversation)
Considering factors above, are you confident pick the correct option the inquiry factually using the conversation log? Ansuer in the following format:
REASON: one-sentence explamation of why are or are not confident. DECISION: YES NO.
Example LLM Output
REASON:
"I am confident in selecting the correct option based on the patient's 's presentation of symptoms suggestive of depression, which would warrant treatment with medications like paroxetine trazodone according to standard medical guidelines for managing depressive symptoms." DECISION: "YES"
B.3 Follow-up Question Generation
MEDIQ Expert Question Generation Module Prompt (appended previous conversation)
If there missing features that prevent you from picking confident and factual answer to the inquiry, consider which features are not yet asked about in the conversation log; then, consider which missing feature is the most important to ask the patient in order to provide the most helpful information toward correct medical decision. You ask about any relevant information about the patient's case, such as family history, and
reaults, treatments already done, the common questions asked in the specific subject relating to the patient's intuitive doctor would ask. Ask
Consider what
known symptoms, and what the best and ONE SPECIFIC ATOMIC QUESTION to address this feature. The question should be bite-sized, questions from the above conversation log. Answer in the following format:
ask for too much at once.
sure to repeat any
ΑΤOМIC QUESTION: the atomic question and NOTHING ELSE.
Example LLM Output
"What time do you usually go to bed night?"
B.4
Information Integration
None-LLM step. Rearrange the atomic question and patient response into the conversation log. Known patient information KC:
patient comes into clinic presenting with some basic information:
"A 40-year-old woman presents with difficulty falling asleep, diminished appetite, and tiredness for the past weeks."
Conversation log:
Doctor Question: "What time do you usually go bed night?"
Patient Response: "Patient goes to bed early at night but is unable to fall asleep."
25

---

B.5
Decision Making
MEDIQ Expert Decision Making Module Prompt (appended to previous conversation)
Assume that you already have enough information from the above question-answer pairs to answer the patient inquiry, use the above information to produce factual conclusion. Answer in the following format:
FINAL CHOICE: correct letter choice and NOTHING ELSE.
Example LLM Output
C Model Version and Compute
Model Spees and Datasets We use 4bit quantization for 2aama-2-Chat-70B and Llama-3-Instruce 70B, and 8bit quantization for Llama-2-Chat-72 Llama-2-Chat 13B, and reduce GPU usage and preserve model utility (Huang et al., 2024). For the OpenAI models, version
to
use the gpt-3. 5-turbo-0125 version for GPT-3.5 and the 2ot-44turboo2024-- for GPT-4. Since training is needed, we the development set of iMEDQA with GPT-3.5 to improve the Expert system for faster iteration of the abstention methods. Then, we generalize our findings to other LLM and the entire set of iCRAFT-MD to test best system.
Hyperparameters For both the Patient and Expert systems, we use a temperature of 0.5 and top_r= 1 for top p sampling. Another important hyperparameter is the confidence threshold in the Abstention Module that determines when the Expert should stop asking more questions, which we explore in the Analysis section ($ 5.2) using grid search instead of arbitrarily setting value.
C.1 Computing Hardware
We use CPU only for the GPT-based experiments, one A40 GPU for the smaller Llama models (7B, 8B, & 13B), and two A40 GPUs for the 70B models. Time duration of the experiments for each model roughly proportional the average conversation length (number of follow-up questions) of the interaction, whether rationale is generated, and the self-consistency factor. Experiment time ranges from 30 minutes for GPT-3.5-basce Expert systems with no rationale generation and no self-consistency, to 7 days for Llama-2- Saamaa---hht-777 C-CChh-700---- Chat-70B-based Expert systems with rationale generation, self-consistency, and a high confidence threshold (i.e., more questions are asked).
C.2 Statistical Significance Testing
Since it is expensive to perform repeated runs for all experiments, we approxi- mate the confidence interval using bi- nomial distribution, where p is the accu- racy of the model and is the dataset size. Therefore, the standard deviation can be calculated as:
Non- Inne-etereeeiv Initial
Interactive
Model
Full
None
BASIC
BEST
#1
82.1
53.6
29.3
45.0
59.3
#2
78.6
55.0
29.3
47.1
57.1
#3
78.6
52.9
31.4
43.6
57.1
#4
80.0
52.1
27.9
43.6
60.0
#5
80.7
52.1
30.0
46.4
57.1
p( p)
SD
(4)
#1-SDB
3.38
4.22
3.86
4.21
4.17
SDR
1.51
1.19
1.30
1.63
1.39
As a sanity check, we repeat the set of experiments in Table 2 with GPT- 3.5 and iCRAFT-MD to calculate the random seed standard deviation (SD
Table 4: Repeated trials GPT-3.5 on iMEDQA to compare the and random seed SD over five runs,
Binomial SD over showing that Binomial SD is reasonable estimation of the confi- dence interval with limited data.
over one trials. We report the Binomial standard deviation for trial #1 (#1-SDp) and the random seed standard deviation (SDR) over one
26

---

trials in Table 4 to show that the Binomial standard deviation is reasonable estimation of the confidence interval when we only have one run available.
D Detailed Abstention Results
In order to determine the confidence threshold above which the model should proceed the final answer, we perform grid search over the iMedQA dataset. The results are shown in Figure 10. Generally, when the threhold becomes higher, the Expert system asks more questions, and the performance also increases. But as the number of questions becomes too high, the performance stagnates, which might be due the fact that a lot of the questions will become irrelevant and/or repetitive.
Fixed Questions
Basic (question or choice)
Numerical Score
Numerical Score SC
48
Numerical Score RG
Accuracy (%)
Numerical Score RG SC
Scale Rating
46-
Scale Rating SC
Scale Rating RG
44
Scale Rating RG SC
Binary Decision
42-
Binary Decision
Binary Decision + RG
Binary Decision RG SC
10
15
25
30
Average Conversation Length
Figure 10: Performance of abstain strategies iMEDQA. Fach line is abstain strategy with increasing confidence thresholds. Darker colors are results rationale generation (RG); dashed lines are with self- consistency (SC). The BEST system (Scale+RG+SC. -)) significantly outperforms the BASIC baseline (A).
E Results on Additional LLMs
We report the results of Experiment 3.2.1 the Llama-2-Chan models below. Generally, we observe similar behavior in these models compared to models with higher utility (Llama-3, GPT-3.5, and GPT-4). However, due to the limited model capacity, results for a few settings are less reliable as they are close to at-chance performance (e.g.. interactive BASIC for Llama-2-7B).
Non-Interactive Initial
Interactive
Task
Model
Full
None
BASIC
BEST
Llama-2-7b
30.8+13
26.3+1.2
26.8=1.2
27.8+13
31.9+1.
iMEDQA
Llama-2-13b
37.1
33.0+1.
29.9=1.3
31.3+13
32.6+113
Llama-2-70b
42.9114
36.7114
31.611.
33.0.11
35.6.11.
Llama-2-7b
42.1142
35.0.140
30.7139
32.1139
37.114.
iCRAFT-MD
Llama-2-13b
55.0+42
42.9+42
26.3+3.7
45.7+42
38.6+4.1
Llama-2-70b
60.7141
44.3
27.9
45.7142
42.1:42
Table 5: Accuracy Llmm--2Chht models with varying information availability. BASIC LLM with basic prompting to ask additional questions. Bold results surpass the non-interactive Initial setup.
E.1 Sensitivity to Prompt Variations
LLMs are sensitive to spurious features in the prompt Sclar (2023). We first experiment on various single-turn baselines with prompt variations to evaluate the robustness of the LLaMa- 2-chat models Touvron et (2023) on the prompt order to finalize prompt to standardize future runs. We explore three system prompts: Empty (default system prompt), Basic (You
27

---

are a helpful medical assistant), and the Meditron Prompt adopted from Chen et al. (2023). We also explore three response prompts (in the user message) for each combination of model size and system prompt: Answer Only (Respond with the correct option and nothing else), Ra- tionale (Explain the rationale, then select the correct option), and Permutate (shuf- fling the option-answer pairs to remove potential dataset bias).
Results on the development set, depicted in Figure 6, show language models, particularly the Llama- 2-Chat series, display consistency despite prompt variations. Performance modestly increases with rationale generation, and is unaffected by answer pair shuffling.
Model Size
System
Response Prompt
Full Context
Basic Info
Question Only
Prompt
Answer Only
33.04
32.10
31.94
Empty
Rationale
35.09
33.83
30.84
Shuffle
33.04
29.90
30.84
LLaMa-2
Basic
Answer Only
32.49
33.12
28.48
7b
Answer Only
30.81
26.83
26.28
Meditron
Rationale
36.11
31.86
29.66
Shuffle
30.02
26.72
28.64
Answer Only Shuffle
38.00
36.03
31.63
Empty
37.14
34.23
30.53
LLaMa-2
Basic
Answer Only
37.77
35.25
31.00
13b
Answer Only
37.07
33.01
29.93
Meditron
Shuffle
37.84
33.67
31,86
Empty
Answer Only
43.87
36.66
33.88
LLaMa-2
Basic
Answer Only
41.92
35.96
33.10
70b
Answer Only
42.88
36.69
31.58
Meditron
Shuffle
42.20
35.92
30.95
GPT-3.55
Meditron
Answer Only
53.42
43.12
35.72
Table 6: Prompt Variations & Single-Turn Baselines.
F
How does prior knowledge influence model choice?
General-purposed LLMs tend to provide the most general answers. We hypothesize that this is because the model does not have enough information provided in the context, needs to resort to its parametric knowledge and chooses the most common option to maximize the likelihood of providing the correct answer. Therefore, when given more information in the context, the model should rely less on its parametric knowledge and customize to the patient. We quantitatively show this trend by obtaining the most common option according the Expert and calculate the generality agreement-peen percent agreement between the model choice and its belief of the most common option. Selecting the most common option regardless of context yields the correct response only 33.7% of the time, emphasizing the need for the model to focus on provided context rather than prior assumptions from its parametric knowledge. Results show that setups with increasing information availability (None, Initial, Full), generality agreement shows an decreasing pattern: 50.0%, 40.2%, and 43.4%, respectively.
G How does the nature of the question influence model behavior?
In this section, we show that the nature of the question, such as medical specialty and difficulty, impacts the Expert's interactive behavior. Overall, interaction benefits more difficult questions and certain specialties such as ophthalmology. Figure 11 shows the impact of interactions on diagnostic accuracy across various medical specialties and demographics. Notably, Ophthalmology showed a significant improvement accuracy from 18.2% to 45.5% after interaction (Figure 12), highlighting the model's potential in specialties with initially low accuracy. Similar trends are evident
28

---

Initial Information
32.9
After Interaction
50.0
41.2
57.957.9
53.1.8.4 48.4
45.5
42.0
43.0
48.2
18.2
Ophthalmooooy Neerhall heurosurgery
500
Genomics
Preventive Otolaryngology Dermatology
Psychiatryy
Family Medicine
step1
step263
Figure 11: Impact of interactive system diagnostic accuracy across medical specialties and demographics. Specialties benefiting the most (left) and least (middle) from interaction; improvement by difficulty (right).
in neurosurgery and genomics, suggesting that interactive information-sseking can enhance decision- making. However, interactions specialties such as family medicine and psychiatry sometimes result in confusion. The effect of interaction on accuracy also extends across different difficulty levels of medical inquiries. For questions that are more challenging and clinically focused (Step 2 & 3), accuracy improved from 43.4% to 48.3%. Overall, this analysis highlights the importance of tailoring interactive diagnostic systems to specific specialties and types of questions for the maximal benefit.
Patient Information: A 47-year-old man presents to his ophthainologist for abnormal vision. relevant irreevant correct option wronn option
following
likely diagnosis?
Options: (A) Anterior uveits, (B) Normal past operative visual adaptations, (C) Seasonal allergies, (D) Sympathetic ophthalmia
Initial diagnosis: (A) Anterior uvelts
Question: you experienced any eye pain
along your
Patient Response: Patient underwent enucleation of right globe end exploratory abdorninel laparotomy. Patient's left eye appears red and irritated.
diagnosis: (D) Sympathetic ophthalmia
sensitiity light, eye? Patient Response: Patient has noticed decressd vision in left eye. Patintt' left epoears red and inrittted.
Question: you noticed any changes your
diagnosis: Sympathetic ophthalmia
Figure 12: Example successful interaction in Ophthalmology.
29

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims match experimental results. We show that the conclusion is generalizable across models and datasets by experimenting with 7 LLMs and two datasets.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: The limitations of this work is discussed the Conclusions section ($ 7).
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: The paper does not include theoretical results.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
The paper fully disclose all the information needed to reproduce the main experimental results of the paper. All models, datasets, and hyperparameters used are discussed in the Experiments section $ 3.2 & 3.2.2 and the Appendix C.
Justification:
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We will release the code to our interactive framework and release a benchmark for the community to evaluate the interactive medical information-seee seekinnge ability of any Expert system. Our experiments use publicly available datasets.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All details to run the framework and hyperparameters used in experiments are discussed in the Experiments section $ 3.2 3.2.2 and the Appendix C, and specific instructions will be release with our code in GitHub repo
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We include confidence intervals in our main results as shown Table 2, details calculating the statistical significance are in Appendix C.2.
8. Experiments Compute Resources
30

---

For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Information the computer resources, including type of compute workers, memory, time of execution, is provided Appendix C
9. Code Of Ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: attps://neere http://uuurrspcccppuueeeee cc/public/ wtwuiieeieeelliiii ines?
Answer: [Yes]
Justification: The research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics, provide ethics statement ($ 7)
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discuss potential societal impacts in the Introduction section ($ and Ethics Statement section ($ 7).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g.. pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: We do not release new models datasets that have high risk of misuse.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: License information datasets used are the Experiments section ($ 3).
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [Yes]
Justification: Manual annotations were used to validate proposed metrics, which is docu- mented Appendix A.4.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
31


Reviews:
[{"review_id": "or7uOgwOwQ", "reviewer": "Reviewer_5NqX", "summary": "The goal of the paper is to develop a dataset on which models can be trained for interactive LLM-patient dialogues that require follow-up questions. The paper adapts MedQA and CraftMD datasets into an interactive setup, uses an LLM to mimic a patient's questions and trains LLMs to ask the necessary follow-ups to answer the patient's primary question.", "questions": "Suggestion: I appreciate the thoroughness of the experimental analyses but think you are\ntrying to fit too much content into too few pages. The font size and spacing of\nfigures are inaccessibly small, and Figures (e.g., 6) are too crowded to easily\nextract the important takeaways. I think it would be better to try to\nstreamline the main body of the paper and add a few more pages to the appendices.", "limitations": "See weakness above.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Strengths (originality, quality, clarity, and significance)\n\nAdapting existing datasets into a conversational setup is a useful contribution\nthat will likely be used in future work.\n\nThe paper is clearly written.\n\nThe paper evaluates on both closed-source and open-source LLMs.\n\nThe finding that a naive question-asking approach actually decreases\nperformance is quite interesting.\n\nThe experimental evaluation of different methodologies for improved follow-up\nquestions and abstention is thorough and convincing.", "weaknesses": "The interactive framework means that the evaluation of Expert systems is\nheavily dependent on the idiosyncracies of the Patient system.  While it is\nclear that different choices of Patient and Expert systems have meaningful\nimpacts on the overall QA performance, the evaluation is not detailed enough to\nunderstand whether performance on MedIQ would generalize to performance for\nhuman patients. While this limitation is perhaps obvious, it should be\naddressed more explicitly and it would be helpful to guide future work to\ndiscuss how the authors would ideally expand this work to include human\npatients and/or clinicians. In particular, if the Expert systems are able to\nanswer the QA tasks with information that would not be sufficient for a human\nclinician, that may be a bug rather than a feature.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the insightful feedback and thorough review for our paper. We appreciate the reviewer highlighting the strengths of our paper, including adapting existing datasets into conversation format, clarity, interesting findings and thorough experiments and models. We address the reviewer’s concerns and questions below:\n\n> the evaluation is not detailed enough to understand whether performance on MedIQ would generalize to performance for human patients.\n\nThank you for raising this concern. Generalization to human patients is indeed important and non-trivial, so we leave it for future work. We will include a discussion in the updated manuscript on the challenges with extending MediQ to human patients as well as important areas for future extensions of this framework. We briefly outline a few ideas here:\n\n- a) Incorporating complex **human and social factors** into the Patient system to simulate real-world interactions accurately. For instance, patients might omit information regarding stigmatized diseases, and symptoms might present differently in different patients. \n\n- b) Exploring how to interact with human patients using MediQ from **linguistic and pragmatics perspectives**. This includes, studying conversational nuances, trust building, communication style, and culturally specific norms to bring the framework closer to real-world application.\n\n- c) Train a patient system on actual conversation data from real patients to better simulate interactions with a real human patient. Collection of such clinical interaction datasets, although tricky and potentially containing privacy risks, would greatly benefit the community.\n\n- c) Conducting **randomized control trials** with patients to test the real-life efficacy of the framework. However, conducting such detailed and careful studies is not feasible within the scope of this paper, as it requires significant time, resources, and medical expertise.\n\nAdditionally, we agree with the reviewer that the Patient system is critical for evaluating the Expert system. Our preliminary experiments with different variants of a Patient showed that irrelevant responses or factually inaccurate responses from the patient led to lower diagnostic accuracy. Therefore, we focused on evaluating the factuality and relevance of a Patient system's responses, to ensure high quality, relevant responses to an Expert's question (Section 3.1).\n\n> it would be better to try to streamline the main body of the paper and add a few more pages to the appendices\n\nThank you for the feedback on space and organization of the paper. We will incorporate your suggestions and shorten sections 2 and 3 (framework description and methods), move the analysis of the abstention threshold (line 337-348, Figure 8(a)) to the appendix, and make sure all figures are legible and highlight important takeaways.\n\nWe appreciate your constructive feedback and believe these revisions will strengthen our paper. Thank you again for your valuable insights.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "K8mbkjcM8G", "reviewer": "Reviewer_SV4E", "summary": "The paper proposes MediQ, a two-LLM-agent-system for interactive patient diagnosis. The authors argue that vanilla LLMs do not autonomously engage in an information extending discussion, but rather directly try to diagnose the patient and therefore oftentimes hallucinate.\nThe proposed system specifically aims to actively refrain from diagnosis when not sufficient information are available in order to ask more questions. This is embodied by a so-called expert agent. The patient agent, in turn, is prompted to support the expert agent by answering with relevant information. \n\nThe authors evaluate different versions of their system, where the patient agent either responds with the queried information or tries to further select relevant information, and the expert agent uses different abstention modules, e.g., predicting binary abstention, numeric confidences or including reasoning chains.\nThe evaluation is conducted for MEDQA and CRAFT-MD datasets, where patient information is partially removed to evaluate the interactive setting. The used LLM is GTP-3.5, showing that selecting relevant subset of patient information from the patient agent is superior to the easier strategies.  \nThe authors also compare their approach to the full information setting for a number of additional base LLMs, including GPT-4 or Llama-3-8b. The results show that the interaction improves accuracy when compared to having only the multiple choice questions or base information about the compaint. The system, however, does not surpass the full information setting.\nFinally, the author also provide several meta analyses, providing insights that, e.g., personalized abstention increases accuracy.", "questions": "-  Is the purpose of the paper to evaluate other approaches using the simulation framework rather than presenting a new abstention module?\n- Can you please provide clearer discussions and argumentations for the mentioned related works wrt why the now proposed approaches are sufficiently different / have a different goal / do not need to be evaluated against them?\n- How does the proposed abstention approach relate to [1]?\n- Why does the full non-interactive setting reach the best overall accuracy even though hallucination takes place?", "limitations": "Limitations have been discussed in the paper, but I wonder what would happen if the patient LLM would make up facts and how this might be confidently tested at real-time. If relevant, this could be added to the discussion.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "- Relevant topic: The paper deals with a relevant problem setting, trying to improve patient diagnosis using LLMs, where hallucination has bad consequences\n- Sensible solution: The proposed system, using two agents which interact and support each other based on abstention and selecting relevant information, is well argued for.\n- Good evaluation setup wrt data and base LLMs: The used datasets as well as the performed synthetic removal of patient information allow for a good evaluation of the problem. In addition, different base LLMs have been tested.\n- Wide meta analysis: The conducted meta-analyses provide good insights, showing details about conversation lenghts and impacts of different strategies.", "weaknesses": "- Goal of the paper not clear: While the topic is relevant and the proposed abstention module is sensible, the paper sometimes refers to the outcome as simulation framework. Is the purpose then to evaluate other approaches through the simulation system rather than presenting a new abstention module?\n- Related work not conclusive: It is not clear to me why mentioned works of the interactive models and agents section do not solve the same or an overlapping problem, which would make it necessary to evaluate against them. Mentioned works include [2, 18, 24], where one would have to argue better why the used approaches for improving interaction and sufficiently different or inapplicable here. In addition, mentioned competing medical diagnosis systen [45] is also not sufficiently described / exluded from being relevant for the evaluation, as only the abstention module seems to be missing. But it might be that the overall performance of used prompts / techniques even supersedes the reported ones here, right? Lastly, I do not understand the referencing of other papers promoting abstention during explaining the proposed method, i.e., [13], but not discussing the differences wrt novelty in the related work section. \n- In addition to the covered works, I see an overlap to works such as [1], dealing with selective predictions of LLMs. It seems these methods have the same goal and should be compared to or discussed. A further elaboration is needed or a better\n- As a consequence, at this point, I am not convinced the paper has sufficient novelty.\n- Table 2 reports the reached accuracies of the different LLMs and variants. It seems from the table, that the full non-interactive setting is always paramount to the best interactive variant. It is not clear to me why abstention and patient information filtering then would help. \n\n[1] Chen, Jiefeng, Jinsung Yoon, Sayna Ebrahimi, Sercan O. Arik, Tomas Pfister, and Somesh Jha. \"Adaptation with self-evaluation to improve selective prediction in llms.\" arXiv preprint arXiv:2310.11689 (2023).\n[2] Chinmaya Andukuri, Jan-Philipp Fränken, Tobias Gerstenberg, and Noah D. Goodman. Stargate: Teaching language models to ask clarifying questions, 2024.\n[13] Shangbin Feng, Weijia Shi, Yike Wang, Wenxuan Ding, Vidhisha Balachandran, and Yulia Tsvetkov. Don’t hallucinate, abstain: Identifying llm knowledge gaps via multi-llm collaboration, 2024.\n[18] Zhiyuan Hu, Chumin Liu, Xidong Feng, Yilun Zhao, See-Kiong Ng, Anh Tuan Luu, Junxian He, Pang Wei Koh, and Bryan Hooi. Uncertainty of thoughts: Uncertainty-aware planning enhances information seeking in large language models. arXiv preprint arXiv:2402.03271, 2024.\n[24] Belinda Z Li, Alex Tamkin, Noah Goodman, and Jacob Andreas. Eliciting human preferences with language models. arXiv preprint arXiv:2310.11589, 2023.\n [45] Tao Tu, Anil Palepu, Mike Schaekermann, Khaled Saab, Jan Freyberg, Ryutaro Tanno, Amy Wang, Brenna Li, Mohamed Amin, Nenad Tomasev, Shekoofeh Azizi, Karan Singhal, Yong Cheng, Le Hou, Albert Webson, Kavita Kulkarni, S Sara Mahdavi, Christopher Semturs, Juraj Gottweis, Joelle Barral, Katherine Chou, Greg S Corrado, Yossi Matias, Alan Karthikesalingam, and Vivek Natarajan. Towards conversational diagnostic ai, 2024.\n\n\n****** Update after author response *******\nI thank the authors for their detailed clarifications. After reading the response and the other reviews, I am open to improve my score towards acceptance. I still think that it would be valuable and maybe a requirement that the framework includes better absention \"modules\", as there are papers who focus on that. The framework, in general, is an interesting and sensible interactive process, but to really add value the evaluation should better reflect that it is possible to gather all information or to abstain otherwise.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s feedback and for highlighting our strengths, including the relevance of the topic, the efficacy of the proposed interactive system, and our thorough evaluation and analysis. \n\n> Is the purpose then to evaluate other approaches through the simulation system rather than presenting a new abstention module?\n\nThank you for the question. The main contribution of our paper is both the simulation/evaluation framework and insights from comparing different abstention modules, rather than proposing a state-of-the-art abstention method.\n\nOur goal is to study and improve LLMs in medical diagnostics, where eliciting information via interactions with patients are crucial. We introduce a two-agent interactive setup to evaluate the “information seeking” ability of LLMs (ln.5-6,34-40,43,106,393-396) and then focus on improving this behavior by enabling the model to know when to ask questions.\n\n- Our evaluation framework is a paradigm shift from the standard single-turn QA setup where all information is given upfront. Instead, the Expert only starts with some initial information, and elicits more info as it continues to interact. We posit that this is more realistic and challenging and is essential in building medical AI assistants. We will better motivate the interactive setup in the paper.\n\n- Second, we focus on abstention, investigating how knowing when to ask questions impacts information-seeking (ln.54,149-152). We combine established confidence estimation techniques in novel ways (ln.155-177). While the techniques are not exhaustive, we found that better confidence estimation leads to better diagnostic performance. We close the gap between the more realistic information-seeking paradigm to the full information setup by 45.5% (ln.296-297), providing a starting point for research in proactive information seeking. Our fine-grained analysis offers insights regarding the **conversation length** (§4.3), **relevance of the questions** (ln.307-316), **format of the information** (ln.317-321), **confidence thresholding** (ln.337-348), and **rationale generation** (ln.349-365) to guide future research.\n\n> related works wrt why the proposed approaches are sufficiently different / have a different goal / do not need to be evaluated against them\n\nWe acknowledge the importance of [2, 18, 24, 45, 13] and clarify that our goal is not to propose a new confidence estimation approach but to show that different abstention methods can work with our interactive framework. We now explain our comparisons and why some were not included.\n\n- [24] elicits user preferences on common tasks but lacks abstention. Our interactive baseline--Basic (ln.155-156)--adapts [24] into MediQ, which we show is not sufficient for clinical reasoning. In MediQ, we adapt some ideas and prompts from [24] but in addition incorporated abstention, as well as complex clinical reasoning tasks and domain knowledge prompts, making our framework more relevant for medical reasoning. \n\n- [2] focuses on how to ask good questions via RL-based training; [18] studies how to select questions. They both differ from our focus on *when* to ask questions via abstention at inference time. We don’t compare to [2] and [18] because they study a different problem than ours and require training [2] or yes/no questions [18].\n\n- [45] lacks an abstention module as noted by the reviewer. [45] involves fine-tuning a *closed-source model on a closed-source dataset*, making it impossible to reproduce. Future work could integrate [45]’s modules into MediQ to enhance the Expert system, but we can’t compare it in our paper due to the need for training and lack of access.\n\n- [13] presents confidence estimation methods for abstention in general tasks *without* interaction. We adapted some of these prompt-based methods as baselines (Numerical Abstention in MediQ corresponds to “Ask for Calibration” in [13]).\n\n- [1] is a confidence estimation method focusing on selective prediction, different from our work on *question-asking* and *information-seeking* in interactive medical dialogues. While we can see [1] being adopted as an Abstention strategy, we don’t compare it in MediQ due to the need for training and model probabilities to calculate confidence scores. \n\nOur work, while acknowledging and building on [2, 18, 24, 45, 13], introduces novel contributions focused on **when to ask questions** via abstention at inference time to ensure safe and effective medical dialogue systems. Our novelty also lies in adapting existing confidence estimation methods into the medical domain, which requires more complex reasoning pathways and domain knowledge and was not explored by the above works. We will clarify these points in the paper and include [1] in the related work section.\n\n> Why does the full non-interactive setting reach best overall accuracy even though hallucination takes place?\n\nThe full non-interactive setup gives all information upfront, so the Expert only has to process the already sufficient information and produce a diagnosis (ln.20). This setup is unrealistic in practice (ln.19,30-33), so we propose the paradigm shift to only provide partial information initially (ln.9,76-77). The Expert is now tasked with 1) deciding when to ask questions, 2) elicit patient information, and 3) integrate information to produce diagnosis, which is inherently a harder task. Starting from only the chief complaint, the interactive Expert aims to elicit patient information to match the full non-interactive upper bound (ln.20,57-58). The reviewer is correct that the best interactive Expert still lags behind the full non-interactive upper bound, but this gap highlights the need for further development in proactive information-seeking LLMs (ln.19-20,296-298). \n\nThank you again for the detailed suggestions and hope we cleared up potential confusions. We kindly request that these points be considered when assigning the final scores. We are happy to answer any further questions.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "I0AUJHpaR4", "reviewer": "Reviewer_M6dH", "summary": "This paper introduce MediQ, a framework to simulate realistic clinical interactions, which incorporates a Patient System and an adaptive Expert System. The Patient system that simulates a patient and responds to follow-up questions, and an Expert system that serves as a doctor's assistant and asks questions to the patient before making a medical decision.", "questions": "See \"Weaknesses\".", "limitations": "See \"Weaknesses\".", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper proposes MediQ that stimulates realistic clinical interactions, which is more like a realistic scenario, rather than providing general responses at once.\n2. The authors conduct extensive experiments to demonstrate the effectiveness of each component's design.", "weaknesses": "1. While the paper demonstrates its effectiveness via automatic evaluation metrics, it lacks human evaluation of the model's performance. Therefore, the performance of the system in \"realistic\" scenarios cannot be verified.\n2. The response of the Expert system relies more on the parametric knowledge of LLMs. Although LLMs have learned knowledge during pre-training, their accuracy and stability cannot be guaranteed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thorough review and feedback. We agree with the reviewer that our paper proposes a more realistic scenario for clinical interactions, and we conduct extensive experiments to validate each component of the framework. We hope to address the reviewer’s concerns below:\n\n> While the paper demonstrates its effectiveness via automatic evaluation metrics, it lacks human evaluation of the model's performance\n\nThank you for highlighting this point. We will emphasize in our limitations section that extending MediQ to real human patients is not trivial due to factors such as data sensitivity, privacy risks, and linguistic and psychological adaptations to build trust with patients. As noted in the Ethics section, MediQ serves as an initial framework to enable development and evaluation of Expert systems rather than direct interaction with users (lines 415-416). We will clarify that the proposed interactive framework is “relatively more realistic” compared to static single-turn QA benchmarks, rather than deployable in real-life scenarios.\n\nWe agree on the importance of human evaluations to further assess the model’s performance in such complex tasks. We conducted case-level interviews with an expert in clinical and medical education to get a sanity check on the model’s reasoning outputs. The expert noted that the model asks reasonable questions, but pointed out that questions in realistic clinical interactions can often be even more open-ended to elicit more detailed patient responses. We will include these insights and comments in the updated manuscript. While large-scale human annotations and testing on *real* patients to evaluate the framework’s real-life efficacy would require randomized control trials and present privacy and ethics concerns. However, conducting such detailed studies is beyond the scope of this paper due to the significant time and resources required. We will address this as a point for future work.\n\n**2. Parametric Knowledge:**\n\n> The response of the Expert system relies more on the parametric knowledge of LLMs. Although LLMs have learned knowledge during pre-training, their accuracy and stability cannot be guaranteed.\n\nWe agree with the reviewer that the accuracy and stability of LLMs rely on parametric knowledge learned during pre-training, which is a shared concern among most works on LLMs. We will add this to the limitations section. However, we took several measures in our paper to reduce the effect of parametric knowledge: \n\n(1) Our abstention and rationale generation modules are explicitly designed to eliminate some of these effects by breaking down the complex clinical reasoning process into modularized components and providing more explainability. Rationale generation aims to provide more explainability into the knowledge gaps of the models compared to the standard QA setting where the model is only asked to select an option. \n\n(2) We also attempt to disentangle potential confounding variables by examining different combinations of model sizes (varying stability and knowledge), model families (potentially different pre-training data distribution), and abstention strategies. We found similar trends in the effects of different abstention strategies among different sizes and families, and observed statistical significant differences among the abstention strategies using the same model, thereby isolating the effects of confidence estimation from parametric knowledge. \n\nWe also posit that models that can better rely on elicited knowledge rather than parametric would get better results in our interactive setting. Nonetheless, it's still a concern for real-deployment. Future work can explore how to augment relevant medical knowledge to ground the expert's questions and we will include the potential influence of pre-training in the limitations section.\n\nWe are grateful to the reviewer for their valuable feedback and believe these revisions will strengthen our paper. Thank you again for your valuable insights. We kindly request that the reviewer consider these points when assigning the final scores. We are happy to answer any further questions.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fOAT74lxIK", "reviewer": "Reviewer_pkFC", "summary": "This paper proposes a novel MEDIQ framework to simulate clinical interaction between doctor and patient. The proposed MEDIQ framework is capable of proactively asking questions and collecting information to make the diagnosis results. Also, the MEDIQ framework incorporates two agents—expert and patient, to formulate a conversation loop. With an abstaining module, the expert agent determines when to stop a conversation. The extensive experiments validate the design of the MEDIQ framework.", "questions": "1.\tThe authors use general LLMs to play as expert and patient agents. Will the performance be improved if the LLMs are finetuned for the medical domain?\n2.\t How is the performance compared with Medical Q&A LLMs, such as MedPalm2?", "limitations": "1. Need to strengthen the explainability of the proposed framework, especially for the abstention module and rationale generation module.\n2. Need to add medical conversation dataset for better evaluation.\n3. Need to compare with medical Q&A LLMs.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tThe novel design of the expert and patient modules to formulate a medical conversation\n2.\tThe design of the expert agent to proactively collect information to support the diagnosis decision\n3.\tThe design of the patient agent to extract the information from the patient record and interact with the expert agent", "weaknesses": "1.\tA large portion of the algorithm, for example, the abstention module and rationale generation module, rely on the output from ChatGPT, which makes the results less reproducible. Moreover, the outcome from ChatGPT, even though the authors claim they estimate model confidence, is still in a state of ‘black box’. The authors should develop some other methods to better quantify and evaluate the outputs, instead of simply relying on ChatGPT or other LLMs.\n2.\tThe dataset is crafted using LLMs on the medical Q&A dataset. To better validate the performance of the proposed model, the authors should add experiments using the medical conversation dataset. In the context of real clinical conversation, the doctors may make a diagnosis decision not simply based on existing medical records, but also take symptoms, medical testing results, and other information into consideration. This information is not included in the medical Q&A dataset.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s valuable feedback and insights. Thank you for highlighting the strengths of MediQ, including the novel design of the Patient and Expert two-agent conversation system, proactive information seeking feature in the Expert, and the robust Patient system to access patient records. We hope to address any questions below:\n\n> Need to strengthen the explainability of the proposed framework.\n\nWe agree with the reviewer that ChatGPT is essentially a black-box, which is a shared concern in most LLM resarch. We ensure reproducibility by releasing all prompts, the static API versions (line 778-784), and also evaluating open-source models such as the Llama family. We also updated the Patient system with an open-source model (Llama-3) to reduce reliance on ChatGPT, and we will add the results to the camera ready version; the overall trends stay consistent.\n\nWe use the **expected calibration error** to quantitatively evaluate the confidence estimators (line 352-354, Figure 8(b)). Our evaluation of each part of the pipeline component *do not* solely rely on ChatGPT or other LLMs. For example, factuality of the Patient system relies on semantic similarity and the overall Expert performance is evaluated with the diagnostic accuracy of MedQA and Craft-MD multiple choice questions. \n\n> Need to add medical conversation dataset for better evaluation.\n\nMost existing medical conversation datasets, such as ChatDoctor [1], are in the form of patients asking questions and doctors responding with answers in a single turn. We also considered KaMed [2], but since it lacks well-formulated multiple choice questions, it cannot be used for the current MediQ evaluation setup, where diagnostic accuracy is highlighted. Thus, given the scarce existing medical datasets, we did the best we could to operationalize the MediQ framework with MedQA and Craft-MD. Curating a *high-quality clinical interaction dataset* is a research direction that would greatly benefit research in proactive information seeking.\n\nInformation such as **symptoms** and **medical testing results** are already in the datasets we use. As stated in the limitations section, one of the requirements of the datasets is to include sufficient contextual information about the patient. We present an example below.\n\n> ```A 5-year-old girl is brought to the emergency department by her mother because of multiple episodes of nausea and vomiting… She has been hospitalized 2 times during the past 6 months… Her immunizations are up-to-date… She appears emaciated. Her temperature is 36.8°C (98.8°F), pulse is 99/min, and blood pressure is 82/52 mm Hg. Examination shows dry mucous membranes. The lungs are clear to auscultation…```\n\nIf the reviewer meant that when the information requested by the Expert is not present in the context, our current Patient system will refuse to answer since we assume that it will not aid in the diagnostic process. But we agree that in order to create an even more realistic scenario, we can have the Patient system answer out-of-scope questions by *creating more complete personas* and augmenting them with medically consistent details. We leave this for future work and will clarify in the updated version.\n\n> How is the performance compared with Medical Q&A LLMs, such as MedPalm2?\n\nWe agree with the reviewer that medical LLMs can potentially outperform general purpose models. There are a few reasons why we did not include medical LLMs in our experiments. \n\nFirst, general-purpose LLMs such as GPT-4 can outperform domain-specific models on tasks such as MMLU and MedQA [4-7]. Second, most of the medical LLMs were fine-tuned on the training set of MedQA [5,6], which might give it shortcuts in predicitng the letter choice rather than perform explicit clinical reasoning as what the MediQ framework is designed to accomplish. Third, medical LLMs lack ability to follow instructions for the diverse modules in MediQ–confidence estimation, abstention, asking questions–due to the fact that they are fine-tuned on instructions for QA tasks only [3]. \n\nWe will report Meditron (fine-tuned on Llama-2 with continued pre-training on medical texts then supervised fine-tuning on QA tasks) and Alpacare (Llama-2 based medical model instruction-tuned on synthetic medical data) on the BASIC and BEST setups and update the results in the paper, though based on our preliminary experiments, we expect poorer performance because these models are not as good at following general instructions. Overall, our main contribution is introducing the framework to study question-asking, using different base models for the Expert system will not impact the framework. \n\nWe are grateful to the reviewer for their valuable feedback and believe these revisions will strengthen our paper. Thank you again for your valuable insights. We kindly request that the reviewer consider these points when assigning the final scores. We are happy to answer any further questions.\n\n---\n\n[1] Li, Y., … & Zhang, Y. (2023). Chatdoctor: A medical chat model fine-tuned on a large language model meta-ai (llama) using medical domain knowledge. Cureus, 15(6).\n\n[2] Li, D., … & de Rijke, M. (2021, July). Semi-supervised variational reasoning for medical dialogue generation. 44th SIGIR (pp. 544-554).\n\n[3] Xie, Q., ... & Bian, J. (2024). Me llama: Foundation large language models for medical applications. arXiv:2402.12749.\n\n[4] Nori, H., … & Horvitz, E. (2023). Capabilities of gpt-4 on medical challenge problems. arXiv:2303.13375.\n\n[5] Chen, Z., ... & Bosselut, A. (2023). Meditron-70b: Scaling medical pretraining for large language models. arXiv:2311.16079.\n\n[6] Han, T., ... & Bressem, K. K. (2023). MedAlpaca--an open-source collection of medical conversational AI models and training data. arXiv:2304.08247.\n\n[7] Zhang, X., … & Petzold, L. R. (2023). Alpacare: Instruction-tuned large language models for medical application. arXiv:2310.14558.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes an interactive approach to medical question answering in which the LLM module is designed to ask follow-up questions that elicit missing information necessary to inform an accurate response. This system entails design of an abstention module which determines whether to seek additional information from a patient via follow-up questions. The results suggest that allowing LLMs to interactively seek additional information when helpful appears to boost model performance. 

Perhaps the main limitation of this work is the lack of manual evaluation, which the authors acknowledged in the response period. But despite this, the work offers a novel information-seeking framework for integrating LLMs into medical prediction models, and may influence such efforts going forward.
Decision: Accept (poster)