InfiR : Crafting Effective Small Language Models and Multimodal Small
  Language Models in Reasoning
Authors: name: Congkai Xie; name: Shuo Cai; name: Wenjun Wang; name: Pengxiang Li; name: Zhijie Sang; name: Kejing Yang; name: Yiming Zhang; name: Zhen Li; name: Guanghao Zhu; name: Zeyu Liu; name: Yang Yu; name: Yuhang Liu; name: Su Lu; name: Baoyi He; name: Qi Zhou; name: Xiaotian Han; name: Jianbo Yuan; name: Shengyu Zhang; name: Fei Wu; name: Hongxia Yang
Published: 2025-02-17 09:07:32

Abstract: Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs)
have made significant advancements in reasoning capabilities. However, they
still face challenges such as high computational demands and privacy concerns.
This paper focuses on developing efficient Small Language Models (SLMs) and
Multimodal Small Language Models (MSLMs) that retain competitive reasoning
abilities. We introduce a novel training pipeline that enhances reasoning
capabilities and facilitates deployment on edge devices, achieving
state-of-the-art performance while minimizing development costs. \InfR~ aims to
advance AI systems by improving reasoning, reducing adoption barriers, and
addressing privacy concerns through smaller model sizes. Resources are
available at https://github. com/Reallm-Labs/InfiR.