Orca 2: Teaching Small Language Models How to Reason
Authors: name: Arindam Mitra; name: Luciano Del Corro; name: Shweti Mahajan; name: Andres Codas; name: Clarisse Simoes; name: Sahaj Agrawal; name: Xuxi Chen; name: Anastasia Razdaibiedina; name: Erik Jones; name: Kriti Aggarwal; name: Hamid Palangi; name: Guoqing Zheng; name: Corby Rosset; name: Hamed Khanpour; name: Ahmed Awadallah
Published: 2023-11-18 11:44:52

Abstract: Orca 1 learns from rich signals, such as explanation traces, allowing it to
outperform conventional instruction-tuned models on benchmarks like BigBench
Hard and AGIEval. In Orca 2, we continue exploring how improved training
signals can enhance smaller LMs' reasoning abilities. Research on training
small LMs has often relied on imitation learning to replicate the output of
more capable models. We contend that excessive emphasis on imitation may
restrict the potential of smaller models. We seek to teach small LMs to employ
different solution strategies for different tasks, potentially different from
the one used by the larger model. For example, while larger models might
provide a direct answer to a complex task, smaller models may not have the same
capacity. In Orca 2, we teach the model various reasoning techniques
(step-by-step, recall then generate, recall-reason-generate, direct answer,
etc.). More crucially, we aim to help the model learn to determine the most
effective solution strategy for each task. We evaluate Orca 2 using a
comprehensive set of 15 diverse benchmarks (corresponding to approximately 100
tasks and over 36,000 unique prompts). Orca 2 significantly surpasses models of
similar size and attains performance levels similar or better to those of
models 5-10x larger, as assessed on complex tasks that test advanced reasoning
abilities in zero-shot settings. We open-source Orca 2 to encourage further
research on the development, evaluation, and alignment of smaller LMs.