## Appendix D Failure Patterns of IR  
In Table 13, we showcase examples for each of the two failure patterns mentioned in the main text. In this table, the first example illustrates the first failure pattern, where the model fails to perform the correct one-step reasoning for disease diagnosis since the symptoms and patient history mentioned in the case description are very common symptoms (e.g., cough with sputum, increased urinary frequency) and are not disease- or condition-specific. Although the better option for this case is to examine the patient's abdomen due to their age and the lacking information of their abdominal condition, the IR system can only return some miscellaneous evidences about (1) a similar case, (2) lung cancer and smoking, (3) urinary tract infection, and (4) cholesterol issue, which can not actually answer the given question. The second example for explaining the second failure pattern is a case about a mother with gestational diabetes/hypertension and her baby. In this case, the IR system can identify the medical condition of the mother and retrieve relevant information to this mother. However, the question is actually asking about the baby, which is only described in the very end of this question. Unfortunately, the IR system only returns the evidence for the first-step reasoning (mother's diagnosis) but not for the second-step (diagnose the baby's disease based on the mother's diagnosis). \begin{table}
\begin{tabular}{p{14.2 pt} p{284.5 pt}} \hline \hline
**Question** & A 67-year-old man presents to a primary care clinic to establish care after moving from another state. According to his prior medical records, he last saw a physician 4 years ago and had no significant medical problems at that time. Records also show a normal EKG and normal colonoscopy results at that time. The patient reports feeling well overall, but review of systems is positive for 1 year of mild cough productive of clear sputum and 2 years of increased urinary frequency. He denies fever, chills, dyspnea, dysuria or hematuria. He denies illicit drug use but has been drinking approximately 1-2 beers per night and smoking 1 pack of cigarettes per day since age 20. Physical exam is unremarkable. Which of the following tests is indicated at this time? \\ \hline
**Options** & **A: Abdominal ultrasound, B: Bladder ultrasound, C: Colonoscopy, D: Serum prostate specific antigen (PSA) testing, E: Sputum culture** \\ \hline
**Evidence** & 1.... This patient is a 67-year-old man with weight loss of 10 pounds in 4 weeks and a 35 pack-year history of cigarette smoking. He quit smoking 10 years ago. He had left shoulder pain for 4 months with no dyspnea, cough, hemoptysis, or other symptoms. Massage and other musculoskeletal manipulation did not improve his symptoms.... 2. Lung cancer, which was rare prior to 1900 with fewer than 400 cases described in the medical literature, is considered a disease of modern man.... Tobacco consumption is the primary cause of lung cancer,... Given the magnitude of the problem, it is incumbent that every internist has a general knowledge of lung cancer and its management. 3. The symptoms and signs of UTI vary markedly with age.... 4. Cigarette smoking is a well-established risk factor in men and probably accounts for the increasing incidence and severity of atherosclerosis in women.... 5. Smoking within 30 min of waking, smoking daily, smoking more cigarettes per day, and waking at night to smoke are associated with tobacco use disorder.... Serious medical conditions, such as lung and other cancers, cardiac and pulmonary disease, perinatal problems, cough, shortness of breath, and accelerated skin aging, often occur. 6. Alcohol and cigarette smoking are well-known modifiers of cholesterol.... \\ \hline
**Question** & A 37-year-old G1P1001 delivers a male infant at 9 pounds 6 ounces after a C-section for preeclampsia with severe features. The mother has a history of type II diabetes with a hemoglobin A1c of 12.8% at her first obstetric visit.