Democratizing Reasoning Ability: Tailored Learning from Large Language
  Model
Authors: name: Zhaoyang Wang, name: Shaohan Huang, name: Yuxuan Liu, name: Jiahai Wang, name: Minghui Song, name: Zihan Zhang, name: Haizhen Huang, name: Furu Wei, name: Weiwei Deng, name: Feng Sun, name: Qi Zhang

Large language models (LLMs) exhibit impressive emergent abilities in natural
language processing, but their democratization is hindered due to huge
computation requirements and closed-source nature. Recent research on advancing
open-source smaller LMs by distilling knowledge from black-box LLMs has
obtained promising results in the instruction-following ability. However, the
reasoning ability which is more challenging to foster, is relatively rarely
explored. In this paper, we propose a tailored learning approach to distill
such reasoning ability to smaller LMs to facilitate the democratization of the
exclusive reasoning ability. In contrast to merely employing LLM as a data
annotator, we exploit the potential of LLM as a reasoning teacher by building
an interactive multi-round learning paradigm. This paradigm enables the student
to expose its deficiencies to the black-box teacher who then can provide
customized training data in return. Further, to exploit the reasoning potential
of the smaller LM, we propose self-reflection learning to motivate the student
to learn from self-made mistakes. The learning from self-reflection and LLM are
all tailored to the student's learning status, thanks to the seamless
integration with the multi-round learning paradigm. Comprehensive experiments
and analysis on mathematical and commonsense reasoning tasks demonstrate the
effectiveness of our method. The code will be available at
https://github.com/Raibows/Learn-to-Reason.