SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Example 1
Question: Can a computer be programmed entirely in Boolean algebra?
Gold Answer: yes
Unpersonalized Teacher Explanation: Boolean algebra is a branch of mathematics that deals with the values true and false. A computer program is a set of instructions that a computer can execute.
Student Prediction (with Unpersonalized Explanation): no
Personalized Teacher Explanation: Boolean algebra is a branch of mathematics. It is used in computer programming.
Student Prediction (with Personalized Explanation): yes
Example 2
Question: Would Amy Winehouse's death have been prevented with Narcan?
Gold Answer: no
Unpersonalized Teacher Explanation: Amy Winehouse was a British singer who died of alcohol poisoning in 2011. Narcan is a brand name for naloxone, a drug used to treat opioid overdoses.
Student Prediction (with Unpersonalized Explanation): yes
Personalized Teacher Explanation: Narcan is a drug used to reverse opioid overdoses. Amy Winehouse died of alcohol poisoning.
Student Prediction (with Personalized Explanation): no
Example 3
Question: Could Brooke Shields succeed at University of Pennsylvania?
Gold Answer: yes
Unpersonalized Teacher Explanation: Brooke Shields is an American actress. The University of Pennsylvania is a private Ivy League university in Philadelphia.
Student Prediction (with Unpersonalized Explanation): no
Personalized Teacher Explanation: Brooke Shields attended Princeton University.
Student Prediction (with Personalized Explanation): yes
Example 4
Question: Can Planned Parenthood tell your University that you have Herpes simplex virus?
Gold Answer: no
Unpersonalized Teacher Explanation: Planned Parenthood is a non-profit organization that provides reproductive health care. Herpes simplex virus is a virus that causes herpes. Herpes is a sexually transmitted disease.
Student Prediction (with Unpersonalized Explanation): yes
Personalized Teacher Explanation: Planned Parenthood is a non-profit organization that provides reproductive health care. They do not have access to your medical records.
Student Prediction (with Personalized Explanation): no
Example 5
Question: Did US President during Spanish-American War suffer similar demise to Abraham Lincoln?
Gold Answer: yes
Unpersonalized Teacher Explanation: The Spanish-American War was fought in 1898. William McKinley was the US President during the Spanish-American War. McKinley was assassinated in 1901. Abraham Lincoln was the US President during the American Civil War. Lincoln was assassinated in 1865.
Student Prediction (with Unpersonalized Explanation): no
Personalized Teacher Explanation: William McKinley was assassinated in 1901. He was the 25th President of the United States.
Student Prediction (with Personalized Explanation): yes
Figure 12: Qualitative comparison between unpersonalized and personalized explanations generated by a LLaMA-65B teacher model for a LLaMA-7B student model for StrategyQA questions. For all these questions, the personalized explanation leads to the correct student answer but the unpersonalized one does not. A common pattern is that the personalized explanations are shorter, simpler, and more directed toward answering the question.