SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 19: StrategyQA Example of Failure Case (second category). We compare the generated output from the Flan-T5 XL model with KARD and reranker against the gold rationale from ChatGPT. The answer is in blue. We mark the hallucination in the generated rationale as yellow and highlight the factually correct statements in the generated rationale and its supporting fact in the passage as red.
Retrieved Passage	Outputs
KARD(Ours, reranker)	Sacrum. The sacrum ( or ; plural: "sacra" or "sacrums"); in human anatomy is a large, triangular bone at the base of the spine, that forms by the fusing of sacral vertebrae S1-S5 between 18 and 30 years of age.	First, we need to understand what the sacrum is and how it is related to fused vertebrae. The sacrum is a triangular bone located at the base of the spine in humans. Next, we need to understand how vertebrae are fused. In the human spine, there are six fused vertebrae (S1, S2, S3, S4), while in the Alaskan Malamute, there are only two fused vertebrae (S3-S4). Now, we can compare the fused vertebrae in the human sacrum with those of the Alaskan Malamute. The human sacrum has six fused vertebrae, while the Alaskan Malamute has five fused vertebrae. Therefore, the answer to the question is (B) no, human sacrums do not have more fused vertebrae than an Alaskan Malamute. Answer: B
Gold(Oracle)	Axial skeleton. At birth the majority of humans have 33 separate vertebrae. However, during normal development several vertebrae fuse together, leaving a total of 24, in most cases. The confusion about whether or not there are 32-34 vertebrae stems from the fact that the two lowest vertebrae, the sacrum and the coccyx, are single bones made up of several smaller bones which have fused together. This is how the vertebrae are counted: 24 separate vertebrae and the sacrum, formed from 5 fused vertebrae and the coccyx, formed from 3-5 fused vertebrae. If you count the coccyx and sacrum each as one vertebra, then there are 26 vertebrae. If the fused vertebrae are all counted separately, then the total number of vertebrae comes to between 32 and 34.	Firstly, what is a sacrum? A sacrum is a triangular bone at the base of the spine that is composed of five fused vertebrae in humans and many other mammals. Secondly, what is an Alaskan Mala- mute? It is a breed of dog that is known for its strength and endurance. Now, the question is asking whether the human sacrums have more fused vertebrae than an Alaskan Malamute. In general, dog breeds tend to have more vertebrae than humans. However, when it comes to the specific case of the sacrum, the answer is that humans have more fused vertebrae. As mentioned earlier, human sacrums consist of five fused vertebrae, whereas Alaskan Malamutes have four. Therefore, the correct answer is (A) Yes. Answer: A