Table 16: MedQA-USMLE Example of Failure Case (first category). We compare the generated output from the Flan-T5 XL model with KARD and reranker against the gold correct rationale from ChatGPT. The answer is in blue. We mark the hallucination in the generated rationale as yellow and highlight the factually correct statements in the generated rationale and its supporting fact in the passage as red.
Question: A 35-year-old man comes to the physician because of itchy, watery eyes for the past week. He has also been sneezing multiple times a day during this period. He had a similar episode 1 year ago around springtime. He has iron deficiency anemia and ankylosing spondylitis. Current medications include ferrous sulfate, artificial tear drops, and indomethacin. He works as an elementary school teacher. His vital signs are within normal limits. Visual acuity is 20/20 without correction. Physical examination shows bilateral conjunctival injection with watery discharge. The pupils are  $3\mathrm{mm}$  equal, and reactive to light. Examination of the anterior chamber of the eye is unremarkable. Which of the following is the most appropriate treatment? (A) Erythromycin ointment (B) Ketotifen eye drops (C) Warm compresses (D) Fluorometholone eye drops
Retrieved Passage	Outputs
KARD(Ours, reranker)	Mansonella perstans. A 36-year-old man was admitted to the outpatient clinic at the Goundi Missionary Hospital in the south of Chad in May 2001. He complained of visual impairment in the left eye, ocular and abdominal pruritus, and abdominal pain. He had previously been treated with DEC for "M. perstans" infection five months prior to his visit. A blood sample was taken at 11:00 am, and examined microscopically as a thick blood film stained with Giemsa's solution. The thick blood film revealed the presence of "M. perstans", and no other parasites were found. He had 3% eosinophilia. A visual acuity test showed a reduction of visual acuity to 4/10 for the left eye, while the right eye was 9/10. (...)	We refer to Wikipedia articles on medicine for help. The symptoms suggest allergic conjunctivitis, and the most appropriate treatment is warm compresses for comfort and relief. Erythromycin ointment is used for bacterial conjunctivitis, keto-tifen eye drops are used for glaucoma, and fluorometholone eye drops are used for dry eyes. An-swer: C
Gold(Oracle)	Emedastine. Emedastine (trade name Emadine) is a second generation antihistamine used in eye drops to alleviate the symptoms of allergic conjunctivitis. It acts as a H receptor antagonist. It works by blocking the action of histamine that causes allergic symptoms. It is used in form of the difumarate. The emedastine difumarate is a white, crystalline, water-soluble fine powder. Emedastine eye drops is usually applied twice a day to the affected eye. When the patients with allergic conjunctivitis were treated with 0.05% emedastine difumarate ophthalmic solution for six weeks, the signs and symptoms such as redness, itching and swelling of the eyes were relieved. Emedastine appears to be devoid of effects on adrenergic, dopaminergic and serotonin receptors. This drug was developed by Alcon, which is global medical company specializing in eye care products.	We refer to Wikipedia articles on medicine for help. The patient's symptoms suggest allergic conjunctivitis, which can be treated with antihistamine eye drops like Ketotifen. Erythromycin ointment is used to treat bacterial infections, warm compresses may help with symptoms but do not treat the underlying issue, and Fluorometholone eye drops are used to treat inflammation and do not address the allergic response. Answer: B