SlowGuess/NeurIPS_2023_OCR huggingface dataset:
There are two potential sources for the candidate passages. One approach is to retrieve candidate passages using the rationale  $\boldsymbol{r}_{ij}$  as a query, while another approach is to use the question (input data)  $\boldsymbol{x}_i$  as the query. It is intuitive to consider combining candidate passages from both approaches, as this can provide additional information to the reranker. Specifically, the reranker can learn the differences between passages that are highly similar to the question and passages that are highly similar to the
<sup>3</sup>https://huggingface.co/datasets/ywchoi/pubmed_ABSTRACT_0
<sup>4</sup>https://pubmed.ncbi.nlm.nih.gov/
rationale. This training setup enables the reranker to prioritize passages that are more relevant to the rationale among the passages retrieved from the question as a query during inference time.
In the implementation, we can control the composition of the candidate set by adjusting hyperparameters  $\kappa_{1}$  and  $\kappa_{2}$  while fixing the candidate size as 8. In Table 9, we provide an analysis of the impact of both values on the reranker performance, by measuring the task accuracy with the Flan-T5 base model. We observe that relying solely on passages related to the question for composing the candidate set is not a viable approach, as it hinders the reranker's ability to learn which passage is relevant to the rationale effectively.
C.3 Knowledge used in KARD training  $(\hat{D}_{ij})$
In order to enhance the reasoning ability of a small LM, it is crucial to retrieve passages that contain the proper knowledge to assist the small LM in generating rationales obtained from the large language model. It is intuitive to utilize the rationale itself as a query for retrieval, focusing to retrieve passages that are relevant to the given rationale. To validate this intuition empirically, we instead use the question as a query for the retrieval during training, retrieving passages denoted as  $\mathsf{topk}(\rho (\pmb {d}|\pmb {x}_i;\mathcal{D}),k)$ . In Table 10, we present the empirical evidence that using the passages relevant to the question during training actually leads to performance degradation, as these passages are highly unlikely to contain the necessary knowledge for generating rationales.
C.4 Examples of Failure Cases
In Section 6.1 of the main paper, we provide an analysis of the failure cases. In this section, we showcase an example for each category of failure cases.
The first category involves the case where the reranker fails to retrieve pertinent knowledge required for generating a rationale. In Table 16, we present an example of the failure case that corresponds to the first category. In this example, the reranker fails to retrieve the relevant passage necessary for generating the correct rationale. Therefore, the small Language Model (LM) generates rationales that include a hallucination: suggesting warm compresses as the most suitable treatment for allergic conjunctivitis, which is factually incorrect.
The second category is the case where the reranker successfully retrieves the relevant passage which is the silver passage obtained from the gold rationale from ChatGPT. In the example illustrated in Table 17, the retrieved passage contains information about pulmonary contusion, which is one of the answer options. If the small LM comprehends the given passage accurately, it should recognize that pulmonary contusion is not the correct answer, as elevated pulmonary capillary wedge pressure and troponins are unrelated to pulmonary contusion but rather associated with a cardiac contusion. However, despite being trained with KARD, the small LM fails to reason based on this knowledge, resulting in the generation of the incorrect rationale.
Furthermore, we also provide examples of failure cases from StrategyQA. Unlike MedQA-USMLE, most of the failure cases in StrategyQA fall into the first category, where the reranker fails. This is due to the fact that StrategyQA questions often require complex and multiple sources of knowledge to answer.
In Table 18, we present an example that corresponds to the first category. The question requires knowledge about the background color of the Azerbaijani flag as well as the color composition of the Powerpuff Girls. However, the reranker retrieves a passage related to the flag of Japan, which is not helpful for answering the given question.
In Table 19, we show the case that belongs to the second category, which is a rare case in StrategyQA. In this example, the reranker successfully retrieves a passage containing information about the number of fused vertebrae in human sacrums. However, the small LM fails to answer the question due to its inability to comprehend the retrieved passage. Moreover, since the question requires additional knowledge about the number of fused vertebrae in the sacrums of an Alaskan Malamute, it is challenging to answer the question solely based on the available knowledge, particularly if the small LM lacks intrinsic knowledge regarding this specific domain.
Our proposed approach aims to enhance the performance of small language models in tasks that involve knowledge-intensive reasoning. As demonstrated in Section 5 of the main paper, our method is beneficial in domains that require professional knowledge, such as the medical field.
However, it is crucial to exercise caution when employing small language models with our method in real-world clinical applications. These models have the potential to generate statements that are factually incorrect, as explicitly mentioned in Section 6 of the main paper and Section C.4. Therefore, thorough attention and careful consideration are required when utilizing small language models in such contexts, even with our proposed method.
It is worth noting that even large language models, which have not been extensively validated in real-world clinical settings, should also be used with caution. The potential for generating inaccurate information exists across various language models, and their deployment in clinical sites should be approached with careful consideration and validation [48, 41, 40].
The following are multiple-choice questions about medical knowledge. Generate a detailed step-by-step explanation for each question and answer.
Question: A 22-year-old male marathon runner presents to the office with the complaint of right-sided rib pain when he runs long distances. Physical examination reveals normal heart and lung findings and an exhalation dysfunction at ribs 4-5 on the right. Which of the following muscles or muscle groups will be most useful in correcting this dysfunction utilizing a direct method?
(A) anterior scalene (B) latissimus dorsi (C) pectoralis minor (D) quadratus lumborum
Answer: (C)
Explanation: We refer to Wikipedia articles on medicine for help. Among the options, only pectoralis minor muscle origins from the outer surfaces of the 3rd to 5th ribs.
Question: A 36-year-old male presents to the office with a 3-week history of low back pain. He denies any recent trauma but says that he climbs in and out of his truck numerous times a day for his job. Examination of the patient in the prone position reveals a deep sacral sulcus on the left, a posterior inferior lateral angle on the right, and a lumbosacral junction that springs freely on compression. The most likely diagnosis is
(A) left-on-left sacral torsion (B) left-on-right sacral torsion (C) right unilateral sacral flexion (D) right-on-right sacral torsion
Answer: (D)
Explanation: We refer to Wikipedia articles on medicine for help. The deep sulcus on the left, a posterior ILA on the right, with a negative spring test suggests a right-on-right sacral torsion. All other options have a deep sulcus on the right.
Question: A 44-year-old man comes to the office because of a 3-day history of sore throat, nonproductive cough, runny nose, and frontal headache. He says the headache is worse in the morning and ibuprofen does provide some relief. He has not had shortness of breath. Medical history is unremarkable. He takes no medications other than the ibuprofen for pain. Vital signs are temperature  $37.4^{\circ}\mathrm{C}$ $(99.4^{\circ}\mathrm{F})$ , pulse  $88 / \mathrm{min}$ , respirations  $18 / \mathrm{min}$ , and blood pressure  $120 / 84 \mathrm{mmHg}$ . Examination of the nares shows erythematous mucous membranes. Examination of the throat shows erythema and follicular lymphoid hyperplasia on the posterior oropharynx. There is no palpable cervical adenopathy. Lungs are clear to auscultation. Which of the following is the most likely cause of this patient's symptoms?
(A) Allergic rhinitis (B) Epstein-Barr virus (C) Mycoplasma pneumonia (D) Rhinovirus
Explanation: We refer to Wikipedia articles on medicine for help. The symptoms, especially the headache, suggest that the most likely cause is Rhinovirus. Epstein-Barr virus will cause swollen lymph nodes but there is no palpable cervical adenopathy. Lungs are clear to auscultation suggests it's not Mycoplasma pneumonia.
Question: A previously healthy 32-year-old woman comes to the physician 8 months after her husband was killed in a car crash. Since that time, she has had a decreased appetite and difficulty falling asleep. She states that she is often sad and cries frequently. She has been rechecking the door lock five times before leaving her house and has to count exactly five pieces of toilet paper before she uses it. She says that she has always been a perfectionist but these urges and rituals are new. Pharmacotherapy should be targeted to which of the following neurotransmitters?
(A) Dopamine (B) Glutamate (C) Norepinephrine (D) Serotonin
Explanation: We refer to Wikipedia articles on medicine for help. The patient feels sad and among the options, only Dopamine and Serotonin can help increase positive emotions. Serotonin also affects digestion and metabolism, which can help the patient's decreased appetite and sleep difficulty.
Question: A 42-year-old man comes to the office for preoperative evaluation prior to undergoing adrenalectomy scheduled in 2 weeks. One month ago, he received care in the emergency department for pain over his right flank following a motor vehicle collision. At that time, blood pressure was  $160/100 \mathrm{mmHg}$  and CT scan of the abdomen showed an incidental  $10\mathrm{-cm}$  left adrenal mass. Results of laboratory studies, including complete blood count, serum electrolyte concentrations, and liver function tests, were within the reference ranges. The patient otherwise had been healthy and had never been told that he had elevated blood pressure. He takes no medications. A follow-up visit in the office 2 weeks ago disclosed elevated urinary normetanephrine and metanephrine and plasma aldosterone concentrations. The patient was referred to a surgeon, who recommended the adrenalectomy. Today, vital signs are temperature  $36.6^{\circ}\mathrm{C}$ $(97.9^{\circ}\mathrm{F})$ , pulse  $100 / \mathrm{min}$ , respirations  $14 / \mathrm{min}$ , and blood pressure  $170 / 95 \mathrm{mmHg}$ . Physical examination discloses no significant findings. Initial preoperative preparation should include treatment with which of the following?
(A) Labetalol (B) A loading dose of potassium chloride (C) Nifedipine (D) Phenoxybenzamine
Explanation: We refer to Wikipedia articles on medicine for help. The symptoms and the adrenal mass suggested pheochromocytoma, and the blood pressure indicates hypertension. Phenoxybenzamine is used to treat hypertension caused by pheochromocytoma.
Question: [question] Answer: [answer] Explanation:
Table 11: MedQA [23]. 5-shot Chain-of-Thought prompt [48] for rationale generation with ChatGPT [42].
Table 12: StrategyQA [14]. 0-shot Chain-of-Thought prompt [28] for rationale generation with ChatGPT [42].
The following are multiple choice questions (with answers). Generate a detailed step-by-step explanation for each question and answer.
Question: [question]
Answer: [answer]
Explanation: Let's think step by step.
Table 13: OpenbookQA [39]. 3-shot Chain-of-Thought prompt for rationale generation with ChatGPT [42].
The following are multiple choice questions (with answers). Generate a detailed step-by-step explanations for each question and answer.
Question: The sun is responsible for
(A) puppies learning new tricks (B) children growing up and getting old (C) flowers wilting in a vase (D)
plants sprouting, blooming and wilting
Answer: (D)
Explanation: We refer to basic knowledge about the effects of sun on living organisms. The sun provides the energy required for photosynthesis in plants, which allows them to sprout, bloom, and eventually wilt. The other options are not related to the effects of the sun.
Question: When standing miles away from Mount Rushmore
(A) the mountains seem very close (B) the mountains are boring (C) the mountains look the same as from up close (D) the mountains seem smaller than in photographs
Answer: (D)
Explanation: This question requires some basic knowledge about perspective and the way our eyes perceive distance. When we stand miles away from an object like Mount Rushmore, it appears smaller than it does in photographs because our eyes are capturing a smaller visual angle. Option (A) is incorrect because if the mountains seemed very close, we would not be standing miles away. Option (B) is subjective and not related to perceptual phenomena. Option (C) is untrue because when we are up close to Mount Rushmore, we can see details that we cannot see from far away.
Question: When food is reduced in the stomach
(A) the mind needs time to digest (B) take a second to digest what I said (C) nutrients are being deconstructed
(D) reader's digest is a body of works
Answer: (C)
Explanation: This question requires basic knowledge about the digestive system. When food is in the stomach, it is broken down and deconstructed into nutrients that can be absorbed by the body. Option (A) is incorrect because the mind is not directly involved in the digestive process. Option (B) is a play on words and not related to digestion. Option (D) is a reference to a literary magazine and not related to the digestive process.
Question: [question] Answer: [answer] Explanation:
Table 14: Input and output example for KARD (training) in MedQA-USMLE [23].
The following are multiple-choice questions about medical knowledge. Generate a step-by-step explanation for each question:
Question: A 23-year-old pregnant woman at 22 weeks gestation presents with burning upon urination. She states it started 1 day ago and has been worsening despite drinking more water and taking cranberry extract. She otherwise feels well and is followed by a doctor for her pregnancy. Her temperature is  $97.7^{\circ}\mathrm{F}$ $(36.5^{\circ}\mathrm{C})$ , blood pressure is  $122/77\mathrm{mmHg}$ , pulse is  $80/\mathrm{min}$ , respirations are  $19/\mathrm{min}$ , and oxygen saturation is  $98\%$  on room air. Physical exam is notable for an absence of costovertebral angle tenderness and a gravid uterus. Which of the following is the best treatment for this patient?
A. Ampicillin B. Ceftriaxone C. Doxycycline D. Nitrofurantoin
Knowledge: Urinary tract infection. Urinary tract infections are more concerning in pregnancy due to the increased risk of kidney infections. During pregnancy, high progesterone levels elevate the risk of decreased muscle tone of the ureters and bladder, which leads to a greater likelihood of reflux, where urine flows back up the ureters and towards the kidneys. While pregnant women do not have an increased risk of asymptomatic bacteriuria, if bacteriuria is present they do have a  $25 - 40\%$  risk of a kidney infection. Thus if urine testing shows signs of an infection—even in the absence of symptoms—treatment is recommended. Cephalexin or nitrofurantoin are typically used because they are generally considered safe in pregnancy. A kidney infection during pregnancy may result in premature birth or pre-eclampsia (a state of high blood pressure and kidney dysfunction during pregnancy that can lead to seizures). Some women have UTIs that keep coming back in pregnancy and currently there is not enough research on how to best treat these infections.
We refer to Wikipedia articles on medicine for help. The patient is pregnant, which limits the options for treatment due to the risk of harm to the fetus. Nitrofurantoin is considered safe for use in pregnancy and is an effective treatment for uncomplicated urinary tract infections. Ampicillin and Ceftriaxone are not typically used to treat UTIs, and Doxycycline is contraindicated in pregnancy.
Table 15: Input and output example for KARD (training) in StrategyQA [14].
The following are multiple-choice questions. Generate a step-by-step explanation for each question:
Question: Is the cuisine of Hawaii suitable for a vegan?
A. yes B. no
Knowledge: Cuisine of the Mariana Islands. The cuisine of the Mariana Islands is largely made up of meat dishes including pork, poultry, horse meat, and meat from other land mammals. Some popular land animals consumed include bats, mice, rats and lizards. Most of the dishes on the islands are cooked in coconut oil. Naturally, the islands' surrounding waters make seafood another popular option. Some seafoods include sea cucumbers, whale's meat, and various fish. It is said that the Mariana Island's cuisine is heavily influenced by its neighbors Papuan, Hawaiian, and American cuisines. The Mariana's cuisine is very international as well. The people enjoy many Asian dishes such as kimchi.
Hawaiian cuisine typically includes a lot of seafood, pork, and other types of meat. These ingredients are not suitable for a vegan as they come from animals. While there may be some plant-based options in Hawaiian cuisine like fruits, vegetables, and grains, the majority of dishes are not vegan-friendly. Therefore, the correct answer is (B) no.
Table 17: MedQA-USMLE Example of Failure Case (second category). We compare the generated output from the Flan-T5 XL model with KARD and reranker against the gold correct rationale from ChatGPT. The answer is in blue. We mark the hallucination in the generated rationale as yellow and highlight the factually correct statements in the generated rationale and its supporting fact in the passage as red.
Question: A 27-year-old man presents to the emergency department after a motor vehicle collision. The patient was the front seat unrestrained driver in a head on collision. The patient's echocardiogram (ECG) is notable only for sinus tachycardia. His temperature is  $99.5^{\circ}\mathrm{F}$ $(37.5^{\circ}\mathrm{C})$ , blood pressure is  $107/58\mathrm{mmHg}$ , pulse is  $120/\mathrm{min}$ , respirations are  $17/\mathrm{min}$ , and oxygen saturation is  $98\%$  on room air. The patient is given 2 liters of Ringer lactate solution and morphine. Initial workup demonstrates that the patient's pulmonary capillary wedge pressure and troponins are elevated. The patient is currently complaining of chest pain. Physical exam is notable for an uncomfortable young man with bruising over his chest wall. Which of the following is the most likely diagnosis? (A) Cardiac contusion (B) Hemorrhage (C) Pulmonary contusion (D) Takotsubo cardiomyopathy
Retrieved Passage
KARD
(Ours, reranker)
Pulmonary contusion. A pulmonary contusion, also known as lung contusion, is a bruise of the lung, caused by chest trauma. As a result of damage to capillaries, blood and other fluids accumulate in the lung tissue. The excess fluid interferes with gas exchange, potentially leading to inadequate oxygen levels (hypoxia). Unlike pulmonary laceration, another type of lung injury, pulmonary contusion does not involve a cut or tear of the lung tissue. A pulmonary contusion is usually caused directly by blunt trauma but can also result from explosion injuries or a shock wave associated with penetrating trauma. With the use of explosives during World Wars I and II, pulmonary contusion resulting from blasts gained recognition. In the 1960s its occurrence in civilians began to receive wider recognition, in which cases it is usually caused by traffic accidents. The use of seat belts and airbags reduces the risk to vehicle occupants. Diagnosis is made by studying the cause of the injury, physical examination and chest radiography. Typical signs and symptoms include direct effects of the physical trauma, such as chest pain and coughing up blood, as well as signs that the body is not receiving enough oxygen, such as cyanosis. The contusion frequently heals on its own with supportive care.
We refer to Wikipedia articles on medicine for help. The patient's history of front seat unrestrained driver in a motor vehicle collision and elevated pulmonary capillary wedge pressure and troponins suggest a pulmonary contusion as the most likely diagnosis. Cardiac contusion or hemorrhage would have presented with additional symptoms such as ECG changes and ECG changes in a different pattern, while Takotsubo cardiomyopathy is not typically associated with chest trauma or ECG changes. Answer: C
(Oracle)
We refer to Wikipedia articles on medicine for help. The patient's elevated troponins and pulmonary capillary wedge pressure, along with chest pain and bruising over the chest wall, suggests cardiac contusion, which can occur from blunt chest trauma like in a motor vehicle collision. Hemorrhage would not cause elevated troponins and pulmonary capillary wedge pressure. Pulmonary contusion would cause respiratory symptoms. Takotsubo cardiomyopathy is a possibility, but not the most likely diagnosis given the patient's history of trauma. Answer: A
Question: Do human sacrums have more fused vertebrae than an Alaskan Malamute? (A) yes (B) no