From the Least to the Most: Building a Plug-and-Play Visual Reasoner via Data Synthesis (2024)

Abstract: We explore multi-step reasoning in vision-language models (VLMs). The problem is challenging, as reasoning data consisting of multiple steps of visual and language processing are barely available. To overcome the challenge, we first introduce a least-to-most visual reasoning paradigm, which interleaves steps of decomposing a question into sub-questions and invoking external tools for resolving sub-questions. Based on the paradigm, we further propose a novel data synthesis approach that can automatically create questions and multi-step reasoning paths for an image in a bottom-up manner. Our approach divides the complex synthesis task into a few simple sub-tasks, and (almost entirely) relies on open-sourced models to accomplish the sub-tasks. Therefore, the entire synthesis process is reproducible and cost-efficient, and the synthesized data is quality guaranteed. With the approach, we construct 50k visual reasoning examples. Then, we develop a visual reasoner through supervised fine-tuning, which is capable of generally enhancing the reasoning abilities of a wide range of existing VLMs in a plug-and-play fashion. Extensive experiments indicate that the visual reasoner can consistently and significantly improve four VLMs on four VQA benchmarks.

## introduction large language models (llms) @xcite have demonstrated remarkable performance across various tasks in natural language processing (nlp). encouraged by the success, the artificial intelligence community is now enthusiastically exploring ways to enable llms to process information from modalities beyond language, leading to a recent surge in the development of large multimodal models (lmms) @xcite @xcite @xcite . by adopting a pre-training and instruction tuning paradigm, representations of multiple modalities are effectively fused in deep architectures, bringing substantial advancements in tasks, such as image captioning @xcite , visual question answering (vqa) @xcite , and optical character recognition (ocr) @xcite , among others. in this work, we explore vision language models (vlms) as a typical example of lmms. despite the rapid progress, state-of-the-art vlms still face challenges in reasoning over visual content. as exemplified in figure 1 , intuitively, the question can be correctly solved following a "least-to-most" paradigm @xcite , in which the ques-tion is decomposed into a series of sub-questions, and an answer is deduced by resolving the subquestions step by step. however, existing vlms are inept at performing such multi-step reasoning because (1) multi-step reasoning paths, like the one shown in figure 1 (right), are rarely included in the training data @xcite foot_0 . the vlms have few opportunities to develop the reasoning capability from the subsequent post-training. and (2) different from reasoning over text, solving questions in a vision-language context may require manipulating the input image (e.g., marking a specific area) and deducting the next steps from both textual and visual intermediate results. the requirement, however, is difficult to accomplish for most vlms, whether open-sourced or proprietary. we present least-to-most visual reasoning, a general paradigm to guide vlms to decompose a given question into sub-questions and invoke tools to resolve each sub-question for handling diverse visual reasoning tasks. while there have been extensive studies for llms regarding chain-of-thought reasoning @xcite @xcite and tool-invoking @xcite , the techniques are less explored in the context of vlms. since data scarcity is a major obstacle, we propose a novel data synthesis approach, dubbed least-to-most synthesis, to automatically generate a "(question, reasoning path)" tuple for a given image in a bottom-up manner. specifically, the pipeline of least-to-most synthesis consists of four steps: (1) entity recognition: recognizing all entities in an image; (2) node construction: constructing three types of nodes, each aggregating an image with a few entities and some textual features; (3) reasoning process synthesis: synthesizing a reasoning path from a sampled chain of nodes. based on the nodes, the reasoning path is formed by connecting a sequence of sub-questions and tool arguments generated by an llm; and (4) question synthesis: generating the main question by recursively combining the sub-questions in the reasoning path through an llm. our approach (almost entirely) relies on open-sourced models. therefore, it offers several advantages, including cost-efficiency, reproducibility, and ensured data quality, over the common practice where data are obtained by querying powerful proprietary lmms like gpt-4v @xcite . based on least-to-most synthesis, we build a large scale visual reasoning dataset (vireo) with 50k examples, and tailor llava-1.5-7b @xcite as a visual reasoner through supervised fine-tuning on vireo. the reasoner can be generally applied to off-the-shelf vlms in a plug-and-play fashion to enhance their reasoning capabilities. we conduct experiments with four representative vlms as showcases. evaluation results across four vqa benchmarks indicate that the reasoner can consistently improve all vlms over all tasks, with absolute performance gains ranging from 0.71% to 39%. our contributions are three-fold: i. we introduce the least-to-most visual reasoning paradigm to synergize question-decomposition and tool-invoking in vlms for solving complex visionlanguage tasks. ii. we propose least-to-most synthesis, a reproducible, cost-efficient, and data quality-assured algorithm for automatically creating multi-step visual reasoning data (almost) using open-source models. iii. we use least-to-most synthesis to construct the vireo dataset of 50k examples for fine-tuning a reasoner model foot_2 . extensive experiments illustrate that the reasoner can consistently and significantly enhance existing vlms in a plug-and-play fashion across five vqa benchmarks. ## method we elaborate on our method for multi-step reasoning in vlms. first, we formalize least-to-most visual reasoning that delineates how a visual reasoner solves a complex problem according to an image ( §3.1). then, we present details of least-tomost synthesis by which a vlm can be tuned as the visual reasoner to perform reasoning ( §3.2). ## experiments to assess the efficacy of the proposed method, we fine-tune an llava-1.5-7b 7 as the reasoner, and plug the reasoner into four representative vlms with various sizes and conduct experiments on four standard vqa benchmarks. in the implementation, we obtain 10k training examples for question and combiner, respectively, by querying gpt-4. subsequently, we synthesize a visual reasoning dataset (vireo) with 50k examples following the least-to-6 https://huggingface.co/meta-llama/ meta-llama-3-8b-instruct 7 https://huggingface.co/llava-hf/llava-1. 5-7b-hf most synthesis approach, and perform instruction tuning on vireo to obtain the reasoner. more implementation details are presented in appendix b. ## discussions although the main results affirmed the high quality of vireo and the general benefits of the reasoner to enhance vlms on multiple benchmarks, we are still curious about the following research questions: (1) rq1: how does the capability of the reasoner vary w.r. ## conclusions we propose a data synthesis approach to enhancing multi-step reasoning capabilities of visionlanguage models. the approach decomposes the synthesis task into several simple sub-tasks, and finish the sub-tasks (almost) with open-sourced models. based on the approach, we build a large-scale visual reasoning dataset, and develop a visual reasoner by tuning on the dataset. evaluation results across four vqa benchmarks indicate that the visual reasoner can generally improve the reasoning capabilities of a number of existing vlms.