Title: Correlated Low-Rank Adaptation for ConvNets
Venue: NeurIPS 2025
Accepted: yes
Keywords: Low-rank adaptation

convnets

correlated weight update
Abstract: Low-Rank Adaptation (LoRA) methods have demonstrated considerable success in achieving parameter-efficient fine-tuning (PEFT) for Transformer-based foundation models. These methods typically fine-tune individual Transformer layers using independent LoRA adaptations. However, directly applying existing LoRA techniques to convolutional networks (ConvNets) yields unsatisfactory results due to the high correlation between the stacked sequential layers of ConvNets. To overcome this challenge, we introduce a novel framework called Correlated Low-Rank Adaptation (CoLoRA), which explicitly utilizes correlated low-rank matrices to model the inter-layer dependencies among convolutional layers. Additionally, to enhance tuning efficiency, we propose a parameter-free filtering method that enlarges the receptive field of LoRA, thus minimizing interference from non-informative local regions. Comprehensive experiments conducted across various mainstream vision tasks, including image classification, semantic segmentation, and object detection, illustrate that CoLoRA significantly advances the state-of-the-art PEFT approaches. Notably, our CoLoRA achieves superior performance with only 5\% of trainable parameters, surpassing full fine-tuning in the image classification task on the VTAB-1k dataset using ConvNeXt-S. Code is available at [https://github.com/VISION-SJTU/CoLoRA](https://github.com/VISION-SJTU/CoLoRA).


Peer reviews:
comments: SUMMARY:
This paper proposes a correlation low-rank adaptation module for convolutional networks, which is an efficient PEFT strategy. The effectiveness of the approach is validated in classification, detection, and segmentation tasks.

STRENGTHS AND WEAKNESSES:
Strengths:

1. This paper finds that fine-tuning pre-trained ConvNets on downstream tasks is a full-rank learning process, and existing LoRA methods are ineffective within the ConvNets framework.
2. This paper proposes a novel weight update strategy to update adjacent convolution layers together.
3. This paper compares its approach with other PEFT methods on tasks such as object detection, classification, and segmentation, and the results show superior performance.

Weaknesses:
The approach only outperforms SFT in classification tasks, while in detection and segmentation tasks, the performance of CoLoRA is not as good as that of SFT. Without considering the resource consumption of the model, the contribution of CoLoRA would be discounted.

QUESTIONS:
I hope the authors will address the issues with the weaknesses, and I will consider revising my score.

LIMITATIONS:
1. The only shortcoming of this paper is that its performance is slightly lower than that of SFT, which might be a common issue.
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-06-23

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes CoLoRA, a novel Low-Rank Adaptation framework tailored for convolutional networks (ConvNets), addressing the limitations of standard LoRA in highly correlated convolutional layers. Unlike existing LoRA methods that treat layers independently, CoLoRA introduces correlated low-rank matrices to model inter-layer dependencies. To further enhance adaptation efficiency, it incorporates a parameter-free filtering mechanism that expands the receptive field and reduces noise from uninformative regions. Extensive experiments across classification, segmentation, and detection show that CoLoRA consistently outperforms prior PEFT methods. Remarkably, it achieves better performance than full fine-tuning on VTAB-1k using only 5% of trainable parameters.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The proposed Correlated Low-Rank Adaptation (CoLoRA) is both novel and well-motivated, supported by the empirical observation of inter-layer correlations in convolutional networks, as illustrated in Figure 2.\n2. The paper provides a clear and insightful analysis of the spatial limitations of LoRA in ConvNets, and the introduced training-free edge-preserving filtering mechanism (Figure 4) offers a compelling and practical solution.\n3. CoLoRA demonstrates notable performance improvements over state-of-the-art PEFT methods on multiple vision benchmarks, including VTAB-1K, COCO, and ADE20K, with a comparable number of trainable parameters.\n4. The ablation studies presented in Table 3 are thorough and well-designed, effectively validating the contribution of each component within the proposed framework.\n\n\nWeaknesses:\n1. Unlike standard LoRA, CoLoRA applies edge-preserving filtering to the compressed features generated by low-rank projections, which appears to preclude re-parameterization into static weights. As a result, inference-time efficiency may be impacted due to the additional computation. It would be important to quantify this overhead by reporting inference time and computational cost comparisons between CoLoRA and standard LoRA across both classification and detection tasks.\n2. While CoLoRA is evaluated on VTAB-1K, COCO, and ADE20K, many recent visual PEFT methods also benchmark on fine-grained visual classification (FGVC) datasets. A comparison with state-of-the-art methods in this domain would strengthen the generality and competitiveness of the proposed approach.\n\nQUESTIONS:\nPlease refer to the previously mentioned weaknesses. \n\nAdditionally, I would like to understand the degree of correlation between adjacent attention layers in Vision Transformers (ViTs). Is this correlation significantly different from that observed in convolutional networks, thereby making CoLoRA specifically tailored for convolutional architectures?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel method to fine-tune convolutional neural network taking into account the correlation between adjacent convolutional layers. Authors decompose low-rank matrix into a sum of two low-rank matrices: one of them is shared across the pair of adjacent layers and the second one is layer-specific. Apart from that, authors introduce edge-preserving convolutional filter consisting of 4 convolutions to expand the receptive field of their fine-tuning method.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n* Authors consecutively apply their theoretical findings one by one in their experiments showing a gradual increase in quality metrics, supporting usability of each upgrade.\n* This approach introduces an idea that we can exploit the correlation between matrices of adjacent layers and use a shared matrix in their adapters. This idea is quite interesting and could inspire future research.\n\nWeaknesses:\n* The authors argue that ConvNext-B requires a full-rank matrix update via checking the matrix rank of the update after full fine-tuning. However, this does not guarantee that full-rank updates are the only viable option. Imposing structural constraints during training may lead to a different set of parameters. A more rigorous approach would be to evaluate the dependence on rank by fine-tuning the model multiple times with varying ranks and comparing performance metrics across different parameter budgets.\n* The choice of similarity metric for weight update matrices seems unjustified. The proximity of mean singular values for adjacent layers, shown in Figure 2, shows only the relative poximity of Frobenius norms of weight updates, and may not imply real correlation between weight updates. Furthermore, the mean singular value of the weight update primarily depends on the matrix entries magnitude, which, in particular, depends on the learning rate schedule.\n* Transition from (3) and (4) to (5) requires more details and explanations.\n\nQUESTIONS:\n1. Did you try any other similarity metrics (for example Frobenius scalar product or distance between Images) to support the idea of the adjacent layers’ correlation?\n2. How much does applying edge-preserving filter on every step affect fine-tuning time?\n3. Do you iteratively merge low-rank adapters for all methods in your experiments? It seems that there is also no comparison to vanilla LoRA or its version with the iterative merging. Therefore, it is difficult to identify the effect solely from weight sharing.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
The CoLoRA framework proposed in the paper solves the high-rank learning bottleneck in ConvNets fine-tuning and significantly improves the performance of the PEFT method. The paper analyzes the gradient correlation between ConvNets layers in detail and verifies the necessity of correlation modeling through mathematical formulas and experiments.

STRENGTHS AND WEAKNESSES:
Strengths：
This work shows that CoLoRA outperforms existing methods in multiple mainstream visual tasks (classification, segmentation, and detection). The experimental settings are reasonable, based on public datasets and models, and highly reproducible. Using only about 5% of the trainable parameters, the performance can reach or even exceed that of full fine-tuning, demonstrating the practical application potential of the method.


Weaknesses：
The paper contains typos that need correction, and although some limitations are discussed in the appendix, the main text does not clearly address potential shortcomings. Considering there are lots of works on efficient finetuning, the proposed method might be somewhat incremental. 

Overall, its strengths has higher weight than its weakness.

QUESTIONS:
Question：
1.	There are typos in the text. It is recommended to check and modify it.
2.	Although some limitations are mentioned in the appendix, the main text does not clearly point out the possible limitations of the method. For example:
3.	Is CoLoRA applicable to deeper networks (such as ConvNeXt-XL) or more complex tasks (such as video processing).
4.	Is the method sensitive to the initialization of low-rank matrices, and its performance under different data distributions.

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-05