Wings: Learning Multimodal LLMs without Text-only Forgetting

Authors: yi-kai  zhang
shiyin  lu
yang  li
yanqing  ma
qing-guo  chen
zhao  xu
weihua  luo
kaifu  zhang
de-chuan  zhan
han-jia  ye
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 14:09:24

Abstract: Multimodal large language models (MLLMs), initiated with a trained LLM, first align images with text and then fine-tune on multimodal mixed inputs. However, during the continued training, the MLLM catastrophically forgets the text-only instructions that the initial LLM masters. In this paper, we present Wings, a novel MLLM that excels in both text-only and multimodal instructions. By examining attention across layers of MLLM, we find that *text-only forgetting* is related to the attention shifts from pre-image to post-image text. From that, we construct an additional Low-Rank Residual Attention (LoRRA) block that acts as the "modality learner" to expand the learnable space and compensate for the attention shift. The complementary learners, like "wings" on either side, are connected in parallel to each layer's attention block. The LoRRA mirrors the structure of attention but utilizes low-rank connections to ensure efficiency. Initially, image and text inputs are aligned with visual learners operating alongside the main attention, balancing focus on visual elements. Later, textual learners are integrated with token-wise routing, blending the outputs of both modality learners collaboratively. Our experimental results demonstrate that Wings outperforms equally-scaled MLLMs in both text-only and visual question-answering tasks. Wings with *compensation of learners* addresses text-only forgetting during visual modality expansion in general MLLMs.

Main paper: 
 introduction: 
 Large Language Models (LLMs) [34, 53, 92, 112] are making significant strides toward Artificial General Intelligence (AGI) systems. Multimodal Large Language Models (MLLMs), as a visual expansion of LLMs, have demonstrated astonishing performance in vision-related captioning [14, 16, 67] , understanding [7, 33, 119] , and reasoning [114, 124, 130] . Common MLLMs build upon powerful pre-trained LLMs that take mixed textual and visual tokens as inputs. The visual ones are acquired using an image encoder and a projector. We describe instructions processed by the LLM without images as text-only instructions. In comparison, multimodal instructions incorporate visual feature tokens into text-only sequences. Modality fusing at the token level provides a flexible and effective pipeline for training MLLMs to comprehend visual information [76, 79, 80] . However, training on multimodal instructions seems to impair the pre-existing profound knowledge, especially making MLLM forget how to respond to text-only instructions like the initial LLM [85, 89] . MLLM experiences a drastic performance decline on text-only evaluation. We term it as the text-only forgetting of MLLM. [80] , and DeepSeek-VL [85] in text-only and multimodal QA tasks, with dark green indicating WINGS with the comprehensive performance; Interacting with multimodal instructions.In practical applications, MLLMs also require engaging in text-only or interleaved conversations. As demonstrated in Figure 1 , users often start with text-only inquiries and then, if not fully satisfied with the response, proceed to supplement questions with visual content. For multimodal instructions, MLLMs still rely on text to capture critical elements, as images may offer redundant information [15, 17, 84] . The first existing approaches replay extensive text-only or interleaved [60, 145] training data to mitigate catastrophic forgetting in MLLMs [71, 85, 89] . However, increasing training data incurs additional computational overhead and data collection challenges. Secondly, some applications [40] switch between LLM and MLLM based on whether images are included. This intuitive solution inevitably demands more deployment memory [1, 2] and is less cache-friendly in long vision-andlanguage interleaved conversations [42, 75, 100] . Therefore, it is crucial to train MLLM while preserving the text-only performance efficiently.Given that the visual input tokens can be inserted at any position within the text sequence, we begin by examining the text before and after the inserted position to mark the impact of the visual part. Considering that MLLM's attention weights reflect the focus on tokens and influence the decisionmaking process, we first analyze the attention weights across each layer of the MLLM. Specifically, for each layer, we compute the attention weight proportion on all text tokens before and after the inserted image, termed as Layer-level Attention Weights (LAWS) of the before and after image text. From this, we examine the dynamic of attention across all layers as MLLM-Laws. Through training and sampling over 100 diverse MLLMs, we find that a well-trained model with superior text-only performance shows a positive correlation of MLLM-LAWS between the text segments before and after the image. This suggests that in a well-structured feature space, the main branch attention on text exhibits similar trends across layers, which is statistically linked to the semantic similarity of the text around the visual part. A closer similarity indicates minor disruption to MLLM's core attention, while a negative correlation shows that excessive focus on visual tokens shifts attention away from the text, significantly impacting MLLM-Laws.Based on this observation, we propose WINGS, which introduces an extra module that acts as the boosted learner to compensate for the attention shift. We integrate complementary visual and textual learners in parallel at each layer's attention block, with visual learners enhancing focus on visual tokens and textual learners on text, respectively. In the first stage, visual features align with textual feature tokens, with all visual learners operating parallel to the main branch attention. The visual learners allocate some attention to visual tokens, mitigating the attention shift in the main branch. Subsequently, textual learners are integrated in parallel. We implement token-wise soft-routing based on shifted attention weights to harmonize the learning on visual and textual tokens. We design the Low-Rank Residual Attention (LoRRA) as the architecture for learners to ensure high efficiency. Figure 3 shows that the visual and textual learners on either side, like light feathers woven into "wings". Experiments show that our WINGS comprehensively achieves superior performance in text-only under the same training condition and exceeds other equal-level MLLMs on multimodal benchmarks. In addition, we construct the Interleaved Image-Text (IIT) benchmark with multi-turn evaluations towards a general mixed-modality scenario. The samples are from text-only questions to strongly image-related conversations. WINGS achieve leading performance across various visionrelevance partitions. Overall, our contributions are as follows: (1) We claim and verify the text-only forgetting phenomenon of MLLM is related to the attention shift of cross-layer MLLM-LAWS before and after the image. (2) WINGS construct the visual and textual learners and introduce a router based on shifted attention weights for collaborative learning to compensate for attention shifts. (3) Experiments on text-only, visual-question-answering, and newly constructed Interleaved Image-Text (IIT) benchmarks demonstrate the comprehensive and versatile performance of WINGS. 
a closer look at attention shift in multimodal llms: 
 In this section, we introduce the development from initialized LLM to MLLM. Next, we devise the MLLM-LAWS metric for representing attention shift and discuss the insights in building WINGS.Large Language Models (LLMs). Even though existing Transformer-based [117] models [20, 82, 99, 131] like BERT [57] and OPT [139] have demonstrated profound language understanding capabilities, there has been a recent surge in powerful Generative Pre-trained Transformers (GPT) [10] under the auto-regressive language modeling paradigm. Both public [53, 54, 112, 113] and private [3, 92, 94, 110] solutions show remarkable progress in language comprehension and generation [90, 123] . These LLMs generally exceed a billion parameters, including pre-training [22, 32, 49, 55] , supervised fine-tuning with instructions [26, 103, 108, 122] , and reinforcement learning from human feedback [23, 95, 106, 146] on massive training data.MLLMs is becoming increasingly popular [18, 19, 62, 71, 133] . Unlike vision-centric multimodal frameworks [68, 134] such as CLIP series [98] , MLLMs aim to align new modality features as the input of LLM with an additional encoder and perform multimodal questionanswering [74, 79, 80, 125, 138, 144] . As illustrated in Figure 2 (a), it enables the combined training of mixed multimodal tokens, facilitating rapid deployment across various applications [24, 25, 44, 81, 119] . One example of this pipeline is the LLaVA [79] series, which integrates a CLIP vision encoder with a linear projection to LLM Vicuna [21] and innovatively introduces instruction-following training data. Following this, some methods consider the richness of the vision-related training context [14, 45, 61] , the scaled visual backbone [51, 72, 78] , or the enhanced connectors [11, 121] to boost the visual effectiveness of MLLMs. Additionally, some works introduce monolithic multimodal solutions [30, 87, 109, 120] . Recently, some work has focused on the general capabilities of MLLM, specifically their performance on new modalities without suffering catastrophic forgetting of the text-only question-answering skills initially mastered by LLM [38, 73, 89] . For example, DeepSeek-VL [85] suggests that supplementing additional text-only training data can mitigate this forgetting. Others [77, 89] try to incorporate interleaved visual-textual data into training to retain language knowledge. However, these methods are limited by training resources and data collection costs. We aim to preserve or even boost performance with text-related training data as little as possible. Some studies [65, 76, 105, 111, 132, 140] also consider expanding the scalability of LLM, such as using Mixture-of-Expert (MoE) with numerous parallel FFNs in the Transformer block alongside a sparse gating network for efficient selection. There are some methods to configure effective information and feedback examples to enhance in-context learning abilities [128, 129] . These methods, however, require a massive increase in training parameters or inference costs. In WINGS, the newly designed parallel learners of Low-Rank Residual Attention (LoRRA) are similar to MoE on attention block, but with at least three orders of magnitude less in resource consumption. Compared to some LoRA-related methods [46] , WINGS focuses on parallel processing within the attention block rather than in certain in-block linear mappings [88, 118] , particularly addressing the issue of capability forgetting in existing architectures.The significant decline in text-only performance is closely linked to the observed related shift during the training process. Research on cross-modal learning [35, 66, 73] shows that transferring to new modalities affects feature distribution, output values, and activation levels. Considering attention weights highlight where MLLM's focus depends on visual or textual tokens for decision-making [97] , we investigate how attention shifts among different parts of the sequences, mainly where divided by the visual feature tokens. Specifically, we study over 100 diverse MLLMs to uncover how attention is allocated to each part for a text-only better MLLM. We take a closer look at the cross-layer dynamic curve of attention proportion on all text tokens before and after the inserted image.For a instruction x and its hidden states in MLLM as h = [h 1 , h 2 , • • • , h s ] consisting of s mixed visual and textual tokens. Let a l ij represent the attention weight between the i th and j th tokens in the l th of the L-layers MLLM. We have, for ∀i, s j=0 a l ij h l-1 = 1. As shown in Figure 2 (a), since the sequence of flattened visual tokens is continuously interleaved with the textual sequence, we denote the index set of the visual tokens asV itself = {v start , v start + 1, • • • , v end }.We refer to the textual sequence before the visual tokens as V before , and similarly, after the visual part as V after . For an MLLM with L layers, we define the Layer-level Attention Weights (MLLM-LAWS) as:EQUATIONwhere token index set V * can be V itself , V before , or V after as mentioned above, and for simplicity, we omit h l-1 in a l V * (•) of LAWS V * . In practice, LAWS V * characterizes the MLLM's attention on the current sequence V itself , V before , or V after regarding the dynamic curve over all MLLM-layers. As shown in Figure 2 (b), the attention to the textual part initially increases and then decreases as the layers progress, while the trend for the visual one is often the opposite. We find that when the MLLM forgets the text-only instructions, the LAWS of the textual sequence after the visual ones show a deviation from the initial trend of rising and then declining. This implies a shift of layer-level attention in the text following the image V after compared to that preceding the image V before . The dynamics labeled as 3 ⃝ in Figure 2 (b) show the red curve for better text-only performance towards the worse blue one. To quantify this, we compute the Pearson Correlation Coefficient [91] between LAWS before and after the visual sequence. Formally,Attention Shift = E x [-ρ (LAWS Vbefore , LAWS Vafter )] + 1 .Studying the attention shift of over 100 diverse MLLMs, we find a positive correlation between the shift and the text-only performance degradation. In Figure 2 (c), each point represents a trained MLLM. We find that one reason for the poor text-only performance of MLLMs is the misalignment of textual LAWS before and after the visual sequence, which largely stems from the main branch's attention block lacking sufficient capacity for continued fine-tuning [127, 143] . Next, We focus on how to mitigate the shifted attention weights. Starting with LAWS we give the MLLM "wings". 
wings: flying to generality with low-rank residual attention learners: 
 In this section, we explore a sufficiently reliable and convenient mechanism to alleviate attention shifts. Specifically, we introduce an additional attention extension structure to assist in learning the main branch's attention. The WINGS architecture operates intuitively by incorporating visual and textual learners designed to mitigate shifts in attention. A dynamic attention-weighted router, guided by negative feedback from biased attention weights, adjusts the outputs of these visual and textual learners. WINGS aims to excel in text-only and visual question-answering tasks with high generality. We start with the typical training pipeline for MLLM like LLaVA [79] (subsection 3.1). Following this, we explore the motivation behind employing parallel modality learners and explain its implementation (subsection 3.2). Finally, we describe the training process for WINGS (subsection 3.3).Following the mainstream architecture of visual-encoder-based MLLM, we take mixed visual and textual features as inputs. For a one-turn conversation, the sequence of the visual feature tokens may appear at any position in the input x. We represent the feature tokens as:EQUATIONwhere we omit the superscript of layer-index l for the 0 th layer. The v start and v end represent the starting and ending indices of the visual feature tokens, usually obtained through the vision encoder ψ and projector W proj , as x V = W proj • ψ (x image ). Correspondingly, x T = the remaining 0 to v start and v end to length s denote features of the textual system prompt or user instructions. We consider the posterior of the ground-truth answer as:EQUATIONHere, φ represents the main branch LLM, which consists of Transformer decoder layers [116] .Considering the interleaved image tokens, we omit the loss calculation for the "next visual token."Motivation: Learning to mitigate the attention shift with modality-specific auxiliary structures.As mentioned in subsection 2.2, MLLM-Laws demonstrates the attention shift in the sequence following the visual features. The shift results from excessive dependency on visual features. This issue may stem from the insufficient alignment within mixed inputs [7, 15, 127] , or the main branch struggles to accommodate capacity expansion during continual learning, where new modalities can obscure existing knowledge. It suggests adding a small, adjustable factor to the shifted mixed modality features and regulating unnecessary fluctuations in MLLM-LAWS. Consequently, we aim to adopt an efficient, learnable module as the visual "wing". Compared to the image-text mixed feature inputs of the main branch, it should focus specifically on extracting visual information to share the burden of overly shifted attention. The interaction between the current hidden state and visual features is conducted within this module. Similarly, to balance the auxiliary function of the visual learner, we also construct a symmetrical textual learner. Moreover, it is crucial to appropriately distribute the two learners across both modalities to ensure they function collaboratively.Structure: parallel learner of attention & token-wise router of attention outputs. To capture key information in shifted modalities while ensuring efficiency, we design a multihead Low-Rank Residual Attention (LoRRA) learner at every layer. It takes input from the hidden state and interacts with the initial visual or text-only feature. The learner facilitates cross-cascading with the initialNorm & FFNNorm & FFNNorm & FFN We introduce extra modules parallel to the main attention, serving as boosted learners to compensate for the attention shift. We train the visual learners on one side, alleviating some shifted attention. Then, we collaboratively learn visual and textual learners based on routing shifted attention weights. They are like light feathers woven "wings".projected information. Specifically, for the l th layer, the visual/text-only learner is formulated as:EQUATIONwhere the matrix W Q , W K , W V , and W O is low-rank and is obtained by the dot product of W a ∈ R d×d and W b ∈ R d×d , and d is relatively small enough. The symbol 1 is represented as the identity matrix. The structure of multihead LoRRA preserves the effectiveness of the cross-attention structure and employs efficient low-rank mapping to reduce computational demands. Following LoRA [46] , LoRRA learners also employ random Gaussian initialization for W a and sets W b to zero. Since W O lacks a residual, the output of LoRRA is zero at the beginning of training. As shown in Figure 3 , the visual and textual features are fed into their respective side learners, like two "wings" woven together. The outputs of two learners from each layer are then weighted sum to the attention of the main branch. As illustrated in the left part of Figure 4 , a router receives attention weights to generate the balance weights of visual and textual learners for each token. In summary, we formulate the WINGS block as:EQUATIONwhere a ∈ R s×s represents the attention weights of the current main branch. The router is formalized as Router (a) = W[:, : s] • a ⊤ , which is implemented by a single-layer dynamic MLP, W ∈ R 2×smax . It receives the attention weights a and processes them using Softmax on two modality learners.The architecture of WINGS comprises four elements: vision encoder, projector, initialized LLM, and the learners with a router. During the training process, the vision encoder is consistently fixed. Firstly, we only fine-tune the projector and visual learners. We primarily employ image-text pairs for visual alignment, while the outputs of visual learners are directly added to the main branch. For this part, the visual learners primarily handle the visual focus, minimizing disturbances to the main branch during continued learning. Subsequently, textual learners are paralleled with visual learners on the attention block of LLMs. The router begins by learning to allocate visual and textual learners from the attention weights of the main branch. At this stage, both types of learners work more effectively together to focus attention on key tokens. To summarize, WINGS prioritizes enhancing visual learners first. Subsequently, it "spreads its wings" by concurrently learning and routing visual and textual learners based on shifted attention weights. During inference, the routed weights of the visual wings branch are deactivated for text-only instructions, while multimodal instructions activate both wings. 
experiments: 
 In this section, we first introduce the benchmarks for evaluating WINGS, including Table 1 : text-only forgetting on the same multimodal training data, Table 2 : comparison with general MLLMs, and Figure 5 : analysis on the Interleaved Image-Text (IIT) benchmark with varying levels of vision-related conversation. Following that, we outline the training details and configurations of the WINGS, and delve into experimental analysis across each benchmark. Following that, we perform an ablation study on various learning rates with different training parts. Finally, we provide supplementary descriptions regarding WINGS' overhead compared to general MLLMs and how its innovative compensatory learners help effectively mitigate attention issues.Evaluation Setups. We aim to assess through MLLM how much visual information is required for evaluation. For example, generic multimodal instructions require MLLMs to strongly capture image aspects, whereas text-only instructions focus on the text. We introduce three types of benchmarks:• Standard text-only benchmarks. We are particularly interested in the text-only performance improvement of WINGS under the same training data and resource conditions. Different datasets including interdisciplinary exams like MMLU [43] , CMMLU [64] , ARC-Easy, ARC-Challenge [27] , language understanding and knowledge such as WinoGrande [102] , Open-bookQA [8] , Race-Middle, Race-High [59] , WSC [126] , CHID [141] , reasoning such as Hel-laSwag [136] , SIQA [104] , PIQA [9] , OCNLI [47] , and math and code-related tasks such as GSM8K [28] and MBPP [4] are comprehensively evaluated. • General multimodal benchmarks. We evaluate on MMMU [135] , MME [37] , MMBench [83] (MMB) in English (EN) and Chinese (CN), ScienceQA [86] for test (SciQA), SEED-Bench [63] for image part (SEED), AI2D [56] for test, and HallusionBench [41] (HallB). Model Summaries & Implementation Details. We release the WINGS base and WINGS pro , with Qwen1.5-7B LLM [6] and SigLIP [137] visual encoder as the foundations. We also introduce the WINGS 1.8B version, adapted to Qwen1.5-1.8B LLM for edge device compatibility. As illustrated in Figure 4 , we only optimize the projector and the image learners of WINGS for the first alignment stage. The LLM branch adaptation is incorporated during the second instruction tuning stage. We train for 1 epoch with the AdamW optimizer and the Cosine learning schedule. Typically, the learning rates for the first and second stages are set at 1e -3 and 2e -6 (with the projector part as 1e -5 ), respectively. For WINGS base , approximately 1m training data to align image learners and about 0.6m supervised fine-tuning instructions for the next stage (the same as LLaVA v1.5 [79] ). In the WINGS pro , we use the same aligned data and approximately 2m training data for learning image-text learners. These two types of MLLM require about 1.5 and 6 days of training on 8× A100 GPUs, respectively.The training datasets for WINGS mini are consistent with the WINGS pro . It takes approximately 5 days to run on 4× A100 GPUs.Details in Figure 2 . We adopt various multimodal to text-only sample ratios (25:1, 20:1, 10:1, 5:1, 2:1, 1:1, 1:2, . . ., 1:25) plus an all:0 setup (12 combinations total) to ensure sufficient scenarios. The learning rate is kept consistent with the setup described above. We sample 5 models per epoch, excluding 12 failed ones due to issues like gradient explosion, resulting in 108 for analysis.Text-only Comparison in Fair Data and Resource Environments. As shown in Table 1 , "Vicuna-v1.5 + CLIP" corresponds to LLaVA v1.5 , and "Qwen1.5 + SigLIP" serves as the foundation for WINGS. When comparing LLM itself and the rest of MLLMs, we observe that fine-tuning with multimodal instructions, compared to the "Qwen LLM", there is text-only forgetting in the forgetting effects of CLIP and SigLIP are similar. In contrast, parameter-efficient fine-tuning methods like LoRA result in less text-only forgetting but underperform on multimodal questions. Overall, WINGS' visual and textual learners are credibly demonstrated to retain performance on text-only tasks while also performing well on visual-related questions. In datasets like CHID, OCNLI, and SIQA, MLLMs show improved text-only performance due to increased language diversity (e.g., Chinese context) or semantic similarity in their fine-tuning data.General Evaluation in Text-Only and Multimodal Tasks. We present the performance of 9, roughly 8B open-source MLLMs, 2 roughly 2B, and 2 private API ones evaluated in the general text-only and multimodal tasks. Table 2 shows that WINGS series can perform better on text-only and multimodal question-answering datasets. It achieves state-of-the-art performance on 13 out of 18 datasets, significantly surpassing LLaVA v1.5 with the same architecture. We find that WINGS is equally effective for more efficient foundations, as shown in the "Efficient Multimodal LLMs" parts. WINGS can still capture key elements and demonstrate good scalability as the parameter increases. Although WINGS base does not receive additional training for the text-only component, it is still able to achieve comparable performance.To finely evaluate MLLMs, we construct a series of text-only and multimodal mixed multi-turn conversations. We extract instructions from MMLU, CMMLU, OpenbookQA, HellaSwag, MMMU, MMBench, SEED-Bench, and AI2D datasets with similar semantics by chroma [39] . We then polish the connection between some instructions using GPT-3.5 Turbo to make them closer to real-world conversations. We set up 6 vision-content configurations, categorized by the multi-turn content as: (T), (T, T), (T, T, T), (T, T, V), (T, V), and (V). For instance, (T, T, V) indicates two consecutive text-only queries followed by a visual question requiring a response. Referencing Figure 5 , we address three questions to comprehensively analyse WINGS:• Can WINGS sustain performance with interleaved evaluation? We find that part (a) highlights WINGS surpassing LLaVA v1.5 and the same-backbone as LLaVA v1.5 (Qwen-SigLIP) for each multi-turn setting, especially in text-centric dialogues. • How do WINGS fare with different learning rate settings? Part (b) demonstrates that using a lower learning rate maintains proficiency in text-only tasks but falls short in multimodal questions, while a higher rate boosts multimodal abilities but not text-only. Applying a higher learning rate to the projector and a lower one to the others achieves the optimal. • Are all components of WINGS equally effective? In part (c), we examine that incorporating visual learners alone slightly preserves text-only abilities, likely by minimizing disruption to the LLM, but diminishes performance on multimodal tasks.In the diverse IIT bench, which ranges from text-rich to multimodal contexts, the effectiveness of WINGS is particularly evident. As shown in Figure 1 , within real-world applications, textual content offers insights for following visual tasks. WINGS excels in handling text-only tasks while improving performance on visual-related instructions. 
conclusion: 
 We propose WINGS, which includes visual and textual learners, to alleviate text-only forgetting. The learner is composed of efficient Low-Rank Residual Attention (LoRRA). We start by considering the shifted attention weights in MLLM and, in the first stage, focus on learning the visual learner. Then, we co-train the visual and textual learners with routing based on the shifted attention weights. WINGS demonstrates remarkable performance on text-only, visual-question-answering, and newly constructed Interleaved Image-Text (IIT) benchmarks. WINGS allows for maintaining text-only performance with limited resources and further enhances performance in well-resourced settings. Should we only add additional modules on top of an LLM branch or, like WINGS, create two distinct learners for visual and textual modalities? We delve into the low-rank adaptation (LoRA) [46] and Prefix-tuning [70] for minimally adapt to the LLM component. These techniques introduce optimization parameters beyond the primary branch. These lightweight adjustments align with extensive modifications, effectively minimizing text-only forgetting but concurrently curbing crossmodal positive transfer. 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: We discuss the broader impacts in the Appendix D.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? 
 
 Appendix: WINGS is a universal plugin that can be integrated with any multimodal mixed-input MLLMs. Notably, it introduces a new concept of competitive reuse among multiple expert groups: we may not require the experts to the Transformer block's MLP layer at a scale three orders of magnitude larger; instead, a minor update in the attention for better allocation may suffice. This idea is also found in some variants of LoRA [36, 115] . In the future, we will gradually explore the future of MLLMs.Regarding the extent to which WINGS alleviates attention shifts, we acknowledge that the main branch of WINGS (without visual and textual learners) still exhibits attention shifts. However, since the outputs from the visual and textual learners compensate within the hidden state, we extract portions of the attention weight matrices from both learners and add them to the main branch's weights. Resultsshow that throughout the training process, the attention shift phenomenon is continually mitigated under the influence of these learners. Essentially, the primary branch's attention ideally retains its original text-only capabilities, while the visual solutions are implemented within the auxiliary learner.Despite WINGS' strong adaptability for embedding auxiliary attention learners in various MLLMs, integrating visual learners requires restarting the feature alignment training, incurring extra costs. Additionally, its deployment on edge devices faces limitations, with WINGS 1.8B offering a solution at the expense of performance. Furthermore, WINGS still requires some text-only data to replay and enhance overall performance, aiming for integration into more generic AI systems in the future. This dialogue is represented as (T, V), consisting of a text-only QA from MMLU [43] and a visual QA from MMMU [135] . It can be observed that, due to the sampling, both include questions from the Sociology category.Answer Justification: We have read and strictly followed the Code of Ethics.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).Justification: There is no such risk in this work.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes]Justification: We have listed and cited relevant datasets and models in detail in the paper to ensure that the license and terms are met.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.• If this information is not available online, the authors are encouraged to reach out to the asset's creators. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 


Decision: This paper unanimously receives positive rates thanks to clear representation and convincing experiments. Although overall reviews are positive, the additional description in the rebuttal should be reflected in the final draft.

Reviews: review: paper_summary: paper_summary: The paper addresses how to solve the well-known problem of multimodal large language models (MLLMs), text-only forgetting referring to the phenomenon of MLLMs showing drastic performance drops on text-only instructions. The paper first observes based on the analysis over 100 MLLMs that the performance drop is related to attention shifts in text tokens before and after visual tokens within mixed visual-and-textual inputs. To compensate for the attention shift, the paper adds separate visual and textual learners with a router to each of the LLM blocks. The visual and textual learners are implemented with a newly designed Lor-Rank Residual Attention (LoRRA). The proposed method, Wings, achieves improvements over baselines on text-only benchmarks while achieving competitive or better performance on multimodal benchmarks, including a newly collected interleaved Image-Text (IIT) benchmark.
strength_weakness: strength_weakness: This work is really intriguing. However, the rationale behind how the authors designed the proposed Low-Rank Residual Attention (LoRRA) as well as the ablation studies proving some critical design choices are missing.
For example,
- why did the authors add the residual terms (the identity matrices in Equation 4)?
- please provide the results without the router, if possible.

Also, please provide computation overheads of Wings in FLOPs, and how much does Wings increase training and inference time (comparison between Wings_base and the Qwen + SigLIP model)?
questions: questions: - Please provide the details of 100 MLLMs explored for the attention shift analysis.
- How can we get a, the attention weights of shape sxs from the LLM main branch? And please elaborate more on how the router operates on a, that is, more details about computing weighted sums of visual and textual learner outputs.
- In Figure 5 (b), did the authors ablate learning rates in the second stage of training? in other words, did the authors use the same learning rate of 1e-3 in the first training stage?
- Does the authors plan to release the second-stage training data for Wings_pro, and the new IIT benchmark?

Minor comment: If the indicator variable becomes zero for any i in equation (3), the probability is zero. Is the equation (3) correct, or there are some typos?
limitations: limitations: The paper addressed the limitations of the work.
paper_summary: paper_summary: This paper addresses a significant challenge that when Multimodal Large Language Models (MLLMs) as they expand LLMs’ capabilities to include vision tasks. Specifically, it highlights the issue of "text-only forgetting," which occurs when MLLMs trained with visual inputs struggle to effectively process text-only instructions. The problem is attributed to inconsistent attention shifts between text and visual inputs before and after training. To solve this, this paper proposes Wings, a method adding visual and textual learners to balance attention shifts. Their approach shows improved performance on both text-only and multimodal tasks, including a newly constructed Interleaved Image-Text (IIT) benchmark for mixed-modality interactions.
strength_weakness: strength_weakness: The paper provides a detailed analysis of the impact of image tokens on the distribution of existing tokens and introduces the Wings structure. However, to ensure that the Wings structure effectively minimizes the overhead of text-only training data while maintaining model performance, more time-based quantitative analyses are necessary to demonstrate the module's effectiveness.
It is observed that methods like DeepSeek [1], which have published their performance on text-only datasets, do not show significant performance degradation. This might be due to their extensive use of text-only training data. Thus, there is a trade-off between the overhead of text-only training data (i.e., excessive training costs) and model optimization strategies (or training methods). Please show why Wings is efficient in this trade-off. Specifically, to what extent can text-only data be reduced in Wings training process while still maintaining text-only performance?
In Figure 2, it is shown that image tokens are inserted among text tokens. Is this insertion specific to chat models with system prompts, or does it also occur in base-version models?
Concerning the IIT benchmark, when constructing multimodal samples (e.g., (T,T,V) as shown in Figure 5), how are these samples constructed, and how is dataset balance maintained? Please provide additional implementation details.

[1] DeepSeek-VL: Towards Real-World Vision-Language Understanding.
questions: questions: 1. Wings note that the text-only performance of MLLMs and the performance of the initial LLMs. This makes a compelling argument for a more general model. Could you provide examples illustrating the phenomenon of performance degradation on text-only in existing models?
2. In line 131, how is the activation value for each token computed for each layer? Is the averaging done along the columns or rows of the attention matrix, i.e., which dimension is averaged?
3. There are other methods for adding additional modules to MLLMs, such as CogVLM [1]. Please clarify the similarities and differences, particularly why Wings-architecture can alleviate the forgetting problem in text-only data.
4. Provide more specific implementation details of the model structure mentioned in line 161?
5. Please include more ablation studies on training Wings on the base LLM, such as the effect when the initial LLM is base-version and does not have a system prompt. If the images are always positioned at the beginning, does Wings still work effectively?
6. Please provide more experimental results regarding the efficiency gains with LoRRA. Compared to LoRA, it specifically handles modalities. How does its efficiency compare to LoRA, and what are its advantages?
7. Please provide more inference details, particularly how the data flows through the Wings modules (the learners for each modality) and the routing mechanism.
8. In Figure 4, the second stage requires fine-tuning the LLM parts of the MLLM. How much resource overhead does this require? Is it significantly more than methods like LLaVA?
9. Please provide the detailed parameter counts for the 1.8B and 7B model versions and include detailed descriptions of each dataset in subsequent versions.
10. In the IIT benchmark, why does the (T, T, T) configuration not show performance improvement compared to (T, T)? Does it indicate the presence of noisy text-only few-shot samples?

Some Tips:
Please add a description of $\mathbb{1}$ and its subscript in Eq. 3.
Add details to the weighted sum description in line 189, specifically how the attention weight matrix is applied in the router module.
The ARC-Easy dataset mentioned in line 217 is missing a citation.
What is referred to by "Efficient Multimodal LLMs" in line 260?
Provide a more detailed explanation of Partitioned LR in Figure 5.
Please include the prompts used to generate data with GPT-3.5 Turbo mentioned in line 268 in subsequent versions.

[1] CogVLM: Visual Expert for Pretrained Language Models.
limitations: limitations: The Limitation section in the appendix states that a relatively small amount of text-only training data is still needed to activate the MLLM's capabilities, which is reasonable. Additionally, Wings is a multimodal framework trained from a general LLM. The generalization capabilities and other limitations of Wings can be comprehensively evaluated in the future.
paper_summary: paper_summary: Multimodal large language models (MLLMs) are initiated with a trained vision encoder and LLM, then fine-tuned on multimodal mixed inputs. In this process, the LLM catastrophically forgets the text-only instructions. This paper first reveals that text-only forgetting is related to the attention shifts from pre-image to post-image texts. Based on this analysis, the authors proposed Wings, the complementary visual and textual learners. The experiment results show that Wings outperforms similar-scaled MLLMs in both text-only and visual question-answering tasks.
strength_weakness: strength_weakness: - The discrepancy seems to exist between the motivation/analysis and the method. There is a lack of detailed explanation on how the Wings module improves the correlation between pre-image and post-image attentions.
- The proposed Low-Rank Residual Attention (LoRRA) module appears to be a variant of LoRA, but there is no detailed motivation or comparative analysis provided.
- The ablation study is insufficient. Ablation experiments are only compared on the Interleaved Image and Text (IIT) benchmark. It would be better to include comparisons across various benchmarks used in the main results as well.
- The simple baseline to prevent forgetting is simply utilizing text-only data, yet there are no comparisons.
questions: questions: - The most intuitive and direct approach to increase the attention correlation can be simply adding correlation loss between the attentions. What would the results be if we applied the correlation loss directly?
- The proposed method looks somewhat similar to P-LoRA [1]. How does it compare with this?
- For ablation studies (Figure 5), which model is used between Wings-base and Wings-pro?

[1] Dong, Xiaoyi, et al. "Internlm-xcomposer2: Mastering free-form text-image composition and comprehension in vision-language large model." arXiv preprint arXiv:2401.16420 (2024).
limitations: limitations: The paper addresses limitations in the paper.
paper_summary: paper_summary: This paper introduces the text-only forgetting phenomenon, where multimodal large language models (MLLMs) experience a significant decline in performance on text-only evaluations. The authors claim that this phenomenon is related to the attention shift of cross-layer MLLM-LAWS (Layer-level Attention Weights) before and after processing images. To address this, the authors propose WINGS, which utilizes visual and textual learners to compensate for the attention shift. Experiments across text-only, VQA, and IIT benchmarks demonstrate the effectiveness of WINGS.
strength_weakness: strength_weakness: 1. Although the paper provides a valuable observation about the correlation between text-only forgetting and MLLM-LAWS correlation in Figure 2, it lacks a deep discussion on the underlying reasons for this phenomenon. MLLM-LAWS correlation measures the attention states before and after the image, but in the text-only evaluation, no image involved. Thus, it remains unclear why text-only forgetting is related to MLLM-LAWS correlation.

2. Figure 2 shows that text-only performance is correlated with MLLM-LAWS. However, it would be interesting to know if the multimodal performance is also correlated with MLLM-LAWS. If so, the correlation score could be a valuable metric for MLLM model selection or evaluation.

3. Table 1 shows that LoRa efficiently mitigates the text-only forgetting problem but degrades multimodal performance. The authors do not provide any explanation for this observation.
questions: questions: 1. What are the 100 diverse MLLMs visualized in Figure 2(c)? Are these models sampled from the same type of MLLMs? Does the correlation occur in MLLMs trained on interleaved image-text datasets?

2. Is the relationship between text-only performance and MLLM-LAWS consistent across various scales of LLM backbones?
limitations: limitations: The major limitation of this paper is the lack of a deep understanding of the phenomenon shown in Figure 2. While the paper identifies a correlation between text-only forgetting and MLLM-LAWS, it does not explore the underlying mechanisms behind this relationship.

Comments: title: Response to the Additional Questions - 2
Response to the Additional Questions - 1
Additional questions
comment: Thank you very much for your response and support!

* The study of Attention Shift in the 100 MLLMs does include some overlapping training epochs. However, their training data also contains certain differences (as adjusting the ratio of multimodal to text-only data **requires random sampling**). Recently, we have been researching how to generate better training data for MLLMs. In the final version, **we will train more MLLMs to supplement additional experiments as much as possible**. Furthermore, we will consider different optimization strategies, alignment methods, architectures, and model performances to investigate the generalizability of attention shifts further. Thank you very much for your comprehensive and detailed suggestions.

* Wings is a better structural and strategic assistant for LLMs to enhance learning from visual input. The overall intuition behind Wings is that the existing LLM structure **is prone to attention shift when new image (visual modality) inputs**. In the first stage, the visual wings **concentrate on the main visual attention** (during which the compensation scale may be too large for certain samples, as you mentioned). However, in the second phase, through the weight allocation of the router and the learning of the textual wings, attention **is balanced between the LLM branch and the visual wings**. This balancing relationship in the learning process resembles **a form of regularization**: for example, when attention toward images increases, the textual wings **also drive an increased focus on the text-only parts**. In Wings, not all inputs will pass through both sides of the learner (for instance, text-only data will not pass through the visual learner), so we prefer to view Wings as **a suite of structural regularizations** that help LLMs learn better during multimodal extension.

    Given the differentiation of tokens, we cannot achieve an explicit regularization constraint here because we lack prior information that applies universally to each token; not all tokens should necessarily focus less on images.

    We are attempting to extend Wings to more modalities. Thank you very much for your effective suggestions, and we will continue to focus on **improvements related to regularization constraints**.

**Thank you once again!** We've learned a great deal from our discussion with you. **We will certainly continue to work hard to develop the more generalized MLLM. We truly appreciate your support! Thank you!**
Thank you for the clarification. Then, it seems that Eq (5) is incorrect. It would be beneficial to update the equation correctly.

I appreciate your proactive and sincere engagement during the discussion period. To conclude, I would like to summarize my review:

- As I noted in my initial review, I believe the text-only forgetting problem targeted by Wings is very important, and the performance is impressive.
- During rebuttal and discussion, several concerns are resolved, but major concerns remain:
	- The attention shift analysis is interesting but not fully convincing in terms of generalizability.
		- Although the analysis involves over 100 MLLMs, the models are trained by adjusting the ratio between multimodal and text-only data within the same dataset and sampling models at different training steps in a single run. This raises doubts about whether the findings would apply to MLLMs with entirely different data distributions, optimization strategies, or architectures.
	- It is also unconvincing that Wings effectively resolves attention shifts, as it does not directly address the problem. There is no explicit regularization to ensure the suppression of attention shifts.
		- Even though the Wings module operates independently from the main LLM attention blocks, its attention output is still combined with the main block, which means it can still overly rely on visual features. The authors describe Wings' role as "compensation," but if the compensation scale is too large, it essentially becomes another form of attention shift.

For these reasons, I believe this paper remains borderline. However, given the importance of the problem and the impressive performance, I am inclined to raise my score to borderline accept. If there's anything I may have missed or misunderstood regarding my remaining concerns, please feel free to correct me and share your thoughts.
Sorry, there may have been some misunderstandings. Let's clarify everything comprehensively:

* **A1:** **Both the router's output and 'softmax'** are misunderstood.

    $\mathbf{w}$ **are not** $\in \mathbb{R}^{s \times s}$. The router only activates the top-(sequence_length) columns of the single-layer MLP, meaning it maps **from the sequence dimension to one dimension**. Therefore, the router's output weights $\mathbf{w} \in \mathbb{R}^{1 \times s}$. There are **two** such single-layer MLPs **for both the visual learner and the textual learner.**

    We denote the maximum value of the sequence length as $S$, the weight of the MLP (router) as $\mathbf{W} \in \mathbb{R}^{1 \times S}$, and the attention weights as $\mathbf{a}$. **We have**
    $$
    \mathbf{w} = \mathbf{W}[:, :s] \cdot \mathbf{a}^{\top}.
    $$
    Thus, $\mathbf{w} \in \mathbb{R}^{1 \times s}$. We guess that the misunderstanding comes from **the 'softmax' in line 193** of the main paper.

    Exactly, the softmax is applied **between the weights of visual and textual learners**, **not** on the sequence_length_dim. *E.g.*, if **concat the $\mathbf{w}$ of visual and textual, and get a matrix** $\hat{\mathbf{w}} \in \mathbb{R}^{2 \times s}$,  we let $\sum_j \hat{\mathbf{w}}_{i j}=1$). As the visual/textual ratio in the first stage is 1:0, in the second stage, the 'softmax' ensures that the weights of both **are summarized to 1 for each token (like 1+0=1 in the first stage)**. We also mentioned related content in Reviewer GnKS's **A4**.

* **A2:** After clarifying the misunderstanding from the previous question, this question **aligns with what you mentioned**; the router's output weights **are actually** $\in \mathbb{R}^{1 \times s}$.

* **A3:** After explaining the previous misunderstanding, it can be observed that the balance between visual and textual learners is achieved through the router's 'softmax' (at the token level). This ensures **the sum of weights for each token is related and their gradients are interconnected**. Furthermore, the router weights differ and are correlated across visual and textual learners, varying for each token while **preserving a balance among the visual and textual learners.**

**We sincerely apologize** for the misunderstanding. **The motivations you understand and the router structure you find reasonable are indeed what we designed in Wings.** We apologize again for our insufficient expression. **We will make sure to provide detailed explanations in the final version.** We’re truly sorry for the inconvenience this has caused you.

Thank you very much! If you have any more questions, please feel free to let us know.

**Thank you once again!**

We provide **the code for the Router as follows:**
```
import torch
import torch.nn as nn
import torch.nn.functional as F
from transformers.activations import ACT2FN
from typing import Optional

class Router(nn.Module):
    def __init__(self, max_length: int, act_fn: Optional[str] = 'silu'):
        """Router class.

        Args:
            max_length (int): The maximum value of the sequence length.
            act_fn (Optional[str]): The activate function.
        """
        super().__init__()
        self.linear = nn.Linear(max_length, 2)
        self.act_fn = ACT2FN[act_fn]

    def forward(self, self_attn_weights, **kwargs):
        """forward of Router.

        Args:
            self_attn_weights (torch.Tensor): The attention weight from the LLM (main branch).
                shape: [batch_size x num_heads x sequence_len x sequence_len]

        Returns:
            list of [torch.Tensor, torch.Tensor]: Two Tensors, router weights on visual and textual learner for each token, respectively.
                shape: [[batch_size x sequence_len x 1], [batch_size x sequence_len x 1]]
        """
        cur_weights = torch.sum(self_attn_weights, dim=1) # sum on head_dim

        cur_weights = self.act_fn(F.linear(
            input=cur_weights,
            weight=self.linear.weight[:, :cur_weights.shape[-1]], # split to sequence_length, cur_weights.shape[-1] is sequence_length
            bias=self.linear.bias
        )) # route with MLP
        cur_weights = cur_weights.softmax(dim=-1) # softmax on two learners
        cur_weights = torch.split(cur_weights, split_size_or_sections=1, dim=-1)
        return list(cur_weights)

batch_size, num_heads, sequence_length = 2, 32, 5
print(f'Start testing, batch_size: {batch_size}, num_heads: {num_heads}, sequence_length: {sequence_length}')

router = Router(max_length=2048)
self_attn_weights = F.softmax(torch.rand((batch_size, num_heads, sequence_length, sequence_length)), dim=-1) # attention weights
router_weights = router(self_attn_weights) # forward

print(f'The router\'s output weights of the visual learner, shape: {router_weights[0].shape}') # [2, 5, 1]
print(router_weights[0])
print(f'The router\'s output weights of the textual learner, shape: {router_weights[1].shape}') # [2, 5, 1]
print(router_weights[1])
```
Thank you for your response once again. However, I still have a few confusing points:

- First, for clarity, let the router's output weights be $\mathbf w$, then, $\mathbf w \in \mathbb R^{s \times s}$ and it multiplies with learners' output of $\mathbb R^{s \times d}$. Each row is the output of a softmax, ensuring $\sum_j \mathbf w_{ij}=1$. Is this correct? 
  - Apart from this question, it would be beneficial to provide a clearer description of the router operation in the paper.
- If so, it seems that the router’s role is to aggregate learner outputs on a token-wise basis. If the motivation is token-level scaling, defining $\mathbf{w}$ as a column vector ($\mathbf{w} \in \mathbb{R}^{1 \times s}$) might be more reasonable.
- Additionally, I’m unsure whether the router’s design aligns with its intended motivation. To achieve token-wise compensation scaling, the router should adjust the balance between learners (i.e., assigning different weights to each learner) rather than focusing on intra-learner aggregation.

If I have misunderstood anything, please let me know.
We are very pleased and grateful for your feedback. We will work on refining the final version to make Wings even better. Thank you very much!
Thank you for your additional explanations and the detailed results provided in the rebuttal. I find your clarifications convincing and I am glad to raise my score. I would also recommend including the results and explanations provided during the rebuttal period in the final version of the paper.
We submitted our reply about eight hours ago, but it seems the system did not send a notification. When you have a moment, could you please take a look at our response? Thank you!
Thank you sincerely for your response!

In comparison to visual learners, textual learners focus more on the attention between **the text-only part** and the sequence, compensating for the shifts in attention at each token (as mentioned in line 149 of the main paper). When the text-only part significantly influences the responses, the textual learner may **compensate with more attention**. To verify this, we extract images and instructions from **the MMMU-VAL dataset** in the categories of `business_economics`, `business_marketing`, `health & medicine_basic_medical_science`, `science_biology`, and `science_chemistry`. Additionally, we extract text-only instructions from **the MMLU dataset** corresponding to the categories of `econometrics`, `marketing`, `professional_medicine`, `college_biology`, and `college_chemistry`.

We then **combine the images with** their strongly related instructions (from MMMU-VAL) and their weakly related ones (from MMLU), *e.g.*, image with **`business_economics` of MMMU-VAL** and the same image with **`econometrics` of MMLU**. The inference results on Wings show that for instructions from MMLU (text-only is more important), the routing weight assigned to the textual learner is greater. Furthermore, in Table 5 of PDF, we compare the situation without textual learners in the 'Only Visual Wings', revealing a decline **in both text-only and multimodal performance**. We also included experiments **with more text-only training samples**, which resulted in improvements in the text-only parts.

|  | MMLU | ARC-Challenge | WinoGrande | MMMU-VAL | ScienceQA |
|-------|------|---------------|------------|----------|-----------|
| Wings | 60.5 | 43.4 | 69.3 | 39.9 | 78.8 |
| w/ more text-only training data | 60.7 | 44.8 | 70.1 | 39.9 | 79.2 |

Thank you again for the discussion. We will continue to work hard to improve Wings! Thank you once again!
Please wait. We have retrained the second stage on more text-only data. A complete response will be provided in about 7 hours. Thank you!
* **Part 3.1:** Additional discussion -- Is the overall Wings over-relying on visual feature
   * The overall Wings may not rely too much on visual features because the attention of **the textual learner** is focused on the text-only part. When there is too much reliance on visual features, the router will **assign a greater weight to the textual learner**. Wings essentially processes part of the visual feature attention through an "independent" module, learning how to allocate this along with the LLM attention block.

   * Wings may have some limitations; for example, due to **the scarcity of spatial relationship instructions** in our training set at that time, the overall Wings might show the over-relying on rare spatial information. We are continuously improving this and will present it in the final version.

   * It seems there may be an interest in expressing the concept of modality expansion: To prevent the LLM from over-relying on visual features, we propose Wings of LLM; if addressing **over-relying audio features** could pave the way for visual, textual, and audio Wings of LLM. That is a general and feasible exploration.

* **Part 3.2:** Additional discussion -- Knowledge forgetting

    * Knowledge forgetting **can be a part of** text-only forgetting. Wings also exhibit forgetting in specific areas, such as knowledge forgetting, such as an approximately 3% decline in the `logical_fallacies` category of the MMLU dataset. This may be related to the **easier-to-forget** reasoning knowledge.

    * We also attempt to observe the relationship **between various metrics and forgetting rates** across different domains. For example, in the MathQA dataset, there is a correlation of approximately 0.629 (across five models) between attention shift and forgetting rates.

    * Knowledge forgetting may relate to the **Incremental Learning**. Wings can add appropriate lightweight structures to achieve a better trade-off in the model's **stability-plasticity dilemma [2]**.

    **[2]** DER: Dynamically Expandable Representation for Class Incremental Learning

Thank you very much for your response. We sincerely look forward to your feedback. We are committed to continually improving Wings. Thank you once again!
Thank you for your detailed response!

* **Part 1:** Table 2 of PDF
   * How to calculate:
       * In the first row of Table 2 in PDF, since the router is not introduced in the first stage, we calculate **the ratio of attention weights** between the visual learner and the LLM attention block **to reflect the importance weights.**

           Specifically, we extract the above two sets of **attention weights** that image tokens receive (the shape is [sequence length x image token length]). We first average across the sequence dimension. Then, we calculate the ratio for each image token and average these ratios to obtain the results.

           We sincerely apologize for the confusion. In our response to reviewer LUvX, we referred to "attention weights" instead of "weights."

       * In the second row of Table 2, we show **the ratio between router weights** on the visual learner and the LLM attention block (set to 1) on image tokens.

           As stated in Eq (5) in the main paper, the output of the LLM attention block (**with weights treated as 1**) is added to the output of the visual learner (with weights determined by the router), *e.g.*, `0.5681:1=0.3623:0.6377`.

   * The explanation provided in Table 2 are:

       1. **The first row** indicates that in the first stage, the visual learner **captures most of the attention** on image tokens due to the tuning strategy of Wings.

       2. **The second row** shows that in the second stage, the router **assigns weights to the visual learner's outputs**, constraining its attention (as noted in Eq. (5) of the main paper). The weight ratio indicates the router prioritizes the main branch LLM's attention outputs, but it does not imply a mitigation of attention shift.

       3. **In Table 3 of the PDF**, we calculate the number of samples where 'the correlation between MLLM-Laws Before Image and After Image' is positive, indicating **a smaller attention shift**. The Wings module mitigates attention shifts with more positive samples compared to LLaVA.

      We appreciate your insights and hope those clarify the meaning of Table 2. **We sincerely apologize for any confusion.**

* **Part 2:** Role of the Router
   * **The role of the router:** The router's weight $r_a$ is multiplied with the outputs of the visual/textual learners and then added to the outputs of the LLM's attention block (as the Eq. (5)). This effectively establishes a routing ratio of $r_a$ : $1$ between the two components. The router can **adjust the compensation scale** of the visual/textual learners **for each token**.

   * The router's motivation stems from the observation that **different tokens** demand **different levels of visual attention compensation [1]**. For example, when inquiring about spatial relationships, more visual cues are needed. In contrast, asking about the capital of the United States does not require visual information. Therefore, we need to customize the compensation of visual and textual attention for each token.

      **[1]** Are We on the Right Way for Evaluating Large Vision-Language Models?

   * We test the ratio between **router weights** on the visual learner and the LLM attention block in the perception and reasoning categories of MME data. The results show that in the perception category, which involves more image descriptions, the router weight **assigned to visual learners is higher**, and the image attention compensation is greater.

   | Model | MME perception | MME reasoning |
   |-------|----------------|---------------|
   | Wings | 0.273:1 | 0.240:1 |

   In summary, the router dynamically allocates the weights for visual/textual learners to compensate for attention to each token.

The additional discussion will be presented in the following comments. Thank you!
First of all, thank you for your thorough response. After reading the response and the paper again, some of my questions were resolved, but I have a few more that I would like to ask:

- Table 2 of PDF
  - Could you provide a detailed explanation of how the values in Table 2 were calculated? The caption mentions “distribution of weights between the learner and the main LLM branch,” but what specific weights are being referred to here? According to Eq (5), the router does not seem to assign weights to the LLM and each learner. How does this relate to the explanation?
  - Additionally, how do these weights in Table 2 indicate that the Wings module mitigates attention shifts?
- Role of the Router
  - What is the role of the router? According to Eq (5), the router does not control the weights between the LLM and visual/textual learners but rather adjusts the internal weights of the outputs of visual/textual learners. Why is this approach necessary, and what is the motivation behind it?


- Additional discussion
  - Wings introduce independent visual learners to prevent the LLM from over-relying on visual features. While this is explicit in terms of the model’s structure, it implicitly addresses the problem of over-dependence on visual features. In other words, even with this approach, there could still be an overall model tendency to depend too much on visual features. Could you discuss your thoughts on this?
  - Additionally, while the paper focused on attention shift and over-focusing on visual features, there might also exist a knowledge forgetting problem in the LLM. What are your thoughts on this?
Thank you for your detailed response. I find the comparison of perception and reasoning results on MME across different stages convincing. I'm also curious if the improvement in reasoning after the second stage is due to the textual module branch of Wings, as I notice the textual wings fine-tuning was introduced in the second stage. If possible, could you provide additional information or experimental results on this? Thank you.
Thank you for your updated rating! If you have any further questions or comments, please feel free to reach out.
Thank the authors for the detailed response. The authors have addressed most of my concerns. I'd like to raise my rating to Weak Accept.
Thank you very much!
Thank the authors for the detailed response. The authors addressed most of my concerns as well as those of other reviewers. I will keep my rating.
Thank you for your feedback; it inspires our continuous improvement.

To address your question about the nature of visual attention in the first training stage, we utilize caption data to align visual information. The Wings module enhances visual attention **related to image descriptions** while improving the model's **perceptual capabilities**. Unlike other MLLM baseline methods that only consider the visual part at the input layer, the Wings module **integrates images and captions across all layers**, deepening the LLM branch's understanding of visual content. However, further reasoning abilities still require the LLM main branch **to learn from instruction training data in the second stage**.

For instance, after the first stage of training, Wings can interpret images and manage tasks like identifying "Is the word in the logo 'the beatles story liverpool?'" within the `perception OCR` category in the MME dataset. However, its reasoning abilities are less robust. 

The table below compares the performance of **baseline and Wings in `perception` and `reasoning` on the MME dataset**. It shows that Wings outperforms baseline methods in perception after the first training stage, with reasoning abilities improving further in the second stage.

| Model | MME perception | MME reasoning |
|-------|----------------|---------------|
| LLaVA (after the first stage of training)  | 1197.53 | 216.18 |
| Wings (after the first stage of training)  | 1286.46 | 241.80 |
| Wings (after the second stage of training) | 1411.76 | 342.07 |

Thank you once again for your support and recognition!
Thank you for the detailed response. The authors have successfully addressed nearly all of my initial concerns, and based on the clarifications provided, I am inclined to raise my score.

After considering the feedback from other reviewers, particularly the points raised by reviewers TqCY and LUvX, I am curious about how Wings compensates for attention shift. Your explanations concerning the structure, forward mechanism, optimization, and experiments are convincing.

However, I have one additional question based on your explanation of the optimization process. You mention that "learners are first trained, so allocate attention in the first stage." Given that much of the data in the first stage is caption instruction, could you please elaborate on the nature of the visual attention that compensates for the LLM branch? Understanding this aspect more clearly would help in fully appreciating the robustness of your approach.
**We add the remaining responses here. Thank you!**

* **Q9:** "more inference details"

* **A9:** Starting from Figure 4 in the paper, we observe that the Learners' structure is embedded at each layer. The visual features from the first layer are fed into the key and value, **while the mapping of query, key, and value is implemented using a residual low-rank mapping**. We modified the fully connected layers in the attention module to use residual and low-rank mapping. In the output mapping matrix, we removed the residual connection since we initialized W_a in the low-rank mapping with Random Gaussian and W_b with Zero. **This allows the structure parallel to the attention module to start training with zero additions**, facilitating quicker adaptation to multimodal scenarios.

* **Q10:** "details of training cost"

* **A10:** **In Table 9 of the PDF**, we present network parameters and costs (in TFLOPS), revealing only **an increase of about 0.1B in parameters and 0.2 TFLOPS in cost**. The training time on 8 A100 GPUs increased by less than 1.5 hours.

* **Q11:** "1.8B and 7B parameters count"

* **A11:** Refer to **Table 9 in the PDF** for parameter counts and forward costs. We commit to providing dataset details in the final version.

* **Q12:** "(T, T, T) weak but (T, T) strong in Figure 5"

* **A12:** We appreciate the reviewer’s keen observation. This may be due to the two text-only in-context samples **belonging to two significantly different datasets**. We re-sampled (T, T, T) **to belong to the same dataset**, and the accuracy of Wings on (T, T, T) improved from 68.4% to 69.7%.