E²LoRA: Efficient and Effective Low-Rank Adaptation with Entropy-Guided Adaptive Sharing
Authors: Minglei Li
Peng Ye
Jingqi Ye
haonan he
Tao Chen
Venue: ICLR 2026 Poster (2026)
Keywords: LoRA
PEFT
Primary area: other topics in machine learning (i.e., none of the above)

Abstract: As large pre-trained models rapidly scale, Parameter-Efficient Fine-Tuning (PEFT) through methods like Low-Rank Adaptation (LoRA) becomes increasingly crucial.  While LoRA has emerged as a cornerstone of PEFT, excelling at preserving performance with minimal additional parameters, exploring parameter-sharing mechanisms of LoRA remains critical to pushing efficiency boundaries. However, existing naive LoRA sharing methods often degrade performance due to sacrificed representational diversity and weakened model expressiveness. To overcome this issue, we conduct an in-depth analysis of pre-trained models using gradient-based proxy entropy, and uncover two critical, previously overlooked properties: Local Similarity and Layer-wise Information Heterogeneity. Building on these insights, we propose E²LoRA, a novel dual-adaptive sharing framework. It enables adaptive sharing interval partitioning, guided by inter-layer proxy entropy similarity, and adaptive rank allocation, informed by layer-wise absolute proxy entropy. This unique design leverages inherently informative properties of pre-trained models to significantly reduce parameter redundancy while maintaining or enhancing expressiveness. Comprehensive evaluations across diverse tasks, modalities, and models consistently demonstrate that E²LoRA achieves an excellent balance of efficiency and effectiveness, consistently matching or surpassing baselines with approximately 50% fewer trainable parameters.

Published as a conference paper at ICLR 2026
E'LORA: EFFICIENT AND EFFECTIVE LOW-RANK ADAPTATION WITH ENTROPY-GUDDDD ADAPTIVE SHARING
Minglei Li Peng Ye1.2,3~ - Jingqi Ye
Haonan He
Tao Chen
I Fudan University Shanghaii Artificial Intelligence Laboratory
3 The Chinese University of Hong Kong
University of Science and Technology of China
Shanghai Innovation Institute
ABSTRACT
As large pre-trained models rapidly scale, Carameter-Efficien Fine-Tuning (PEFT) through methods like Low-Rank Adaptation (LoRA) becomes increas- ingly crucial. While LoRA has emerged as a cornerstone of PEFT, excelling at preserving performance with minimal additional parameters. exploring parameter- sharing mechanisms of LoRA remains critical to pushing efficiency boundaries. However, existing naive LoRA sharing methods often degrade performance due sacrificed representational diversity and weakened model expressiveness. To overcome this issue, we conduct an in-depth analysis of pre-trained models us- ing gradient-based proxy entropy, and uncover two critical, previously overlooked properties: Local Similarity and Layer-wise Information Heterogeneity. Building on these insights, we propose E*LORA, novel dual- dual-adaptive sharing framework. It enables adaptive sharing interval partitioning, guided by inter-layer proxy en- tropy similarity, and adaptive rank allocation, informed by layer-wise absolute proxy entropy. This unique design leverages inherently informative properties of pre-trained models significantly reduce parameter redundancy while maintain- ing enhancing expressiveness. Comprehensive evaluations across diverse tasks, modalities, and models consistently demonstrate that E*LORA A achieves an excel- lent balance of efficiency and effectiveness, consistently matching or surpassing baselines with approximately 50% fewer trainable parameters.
INTRODUCTION
As large language models (LLMs) scale, full fine-tuning becomes increasingly resource-intensive. Purameter-Ffftcient Fine-Tuning (PEFT) methods emerge as alternative techniques, efficiently adapting models by updating only a small set of parameters, significantly reducing training costs (Zaken al.) |021 Liu al. 2021: Hu al., 202I). Among PEFT methods, Low-Rank Adaptation (LoRA) (Hu et al. 2021) stands out. By injecting pluggable low-rank adapters, its sim- plicity and effectiveness have driven its broad adoption across industries and research. a result, numerous follow-up studies and variants are proposed (Mao et al. 2025 [Han et al.) (2024).
Currently, LoRA variants can roughly classified into two main categories: (1) Effectiveness- enhancing variants such AdA (Zhang et al. (023c) dynamically adjust ranks, DoRA (Liu et al., 2024a) restructures LoRA through decomposition, and LoRA+ (Hayou et al.] 2024 opti- mizes learning dynamics via asymmetrical learning rates. However, these methods focus per- formance gains without enhancing parameter efficiency; (2) Efficiency-enhancing methods such as LoRA-FA (Zhang et al. 2023b), which freezes the A matrices and keeps the B matrices trainable, VeRA (Kopiczko et al.) 2023) adds learnable vectors at the middle and output ends of LoRA, freezes and shares the low-rank matrices, and only updates the introduced vectors, Tied-LoRA (Renduch- intala et al. 2023) further investigates which parts of LoRA should be shared or frozen. Besides,
"Equal contribution.
Corresponding author.

---

Published as a conference paper at ICLR 2026
ShareLoRA (Song et al. 2024) shares A matrix across all layers while keeping B matrices layer- specific. While these methods reduce parameters through freezing or sharing, they often incur sig- nificant performance degradation. This is primarily because sharing the same LoRA parameters across all layers leads to a loss of representational diversity, disrupting feature discrimination and weakening the model's expressiveness. This raises critical question: How can we enhance the efficiency of LoRA significantly while maintaining or even surpassing original effectiveness?
To answer this question, we draw inspiration from information theory and conduct an in- depth analysis of pre-trained models using gradient-basee proxy entropy. As depicted
Prozy Entrosy Similarity
Lsc
Par-Layer Proxy Entrapy
Layer Number
0.00
Figure D our analysis reveals two cru-
3.00
cial insights often overlooked by prior shar- ing strategies: (1) Local Similarity: The left "Entropy Similarity Matrix" panels of Fig- ure D show the inter-layer information sim- ilarity of gradients on different models and tasks. We observe distinct block-like high- similarity regions, which highlighted by green boxes. This phenomenon indicates that adjacent layers often exhibit higher informa- tion similarity. Moreover, the size and po- sition of these similarity blocks vary signif- icantly across different models and tasks, im- plying that heuristic sharing ranges like shar- ing across all layers fixed blocks are subop- timal. (2) Layer-wise Information Hetero- geneity: The right "Per-Layer Entropy Com- parison" panels of Figure [
2.7
-10
255
Layer Number
Layer Number
(a) Roberta-base on SST-2
Froxy Entropy Similarity
Par-Layer Proxy Entrapy
Layer Number
3.50
15 12
150
10
-10
Layer (b) LLama3.1-88 on HumanEval
Lager Number
Figure Gradient-based proxy entropy analysis for pre-trained models. Left panels show inter-layer information similarity. with high-similarity blocks highlighted in green. Right panels illustrate abso- lute proxy entropy values across layers, indicating significant variations. Subfigures (a), (b) present the results of Roberta-base on SST-2 and Llama3. 1-BB on HumanEval, respectively. Visualization of CLIP- ViT-B/16 on Cars can be found in the Appendix D.I
illlustraee the ab- solute proxy entropy values of different lay- ers. We observe substantial variations in the volume of information across layers. This heterogeneity means that after defining shar- ing intervals, the informational importance differs significantly across intervals. Applying uniform number of learnable parameters to in- tervals with unequal information content may hinder the performance of information-rics layers, as they are not allocated sufficient capacity. Prior methods overlook these architectural heterogeneities, leading to information capacity loss and performance degradation when parameters are shared in- discriminately. Critically, these structural properties not unique the models in Figure [ As shown in Appendix D.2 we confirm the universality of these insights across a broader range of models, including Qwen3-8B, Phi4-mini, Gemma-9B and GPT-OSS-20B, which cover different model families and architectures.
Building on these critical insights, we introduce E*LORA, novel dual-adaptive parameter-sharing framework that effectively addresses the aforementioned challenges. For Local Similarity, E'LORA dynamically partitions layers into sharing intervals by measuring inter-layer similarity with gradient-based entropy proxy and applying an adaptive threshold. This strategy groups consecutive layers only when they exhibit strong informational resemblance, avoiding heuristic schemes and thereby improving parameter efficiency without sacrificing model expressiveness. For Layer-wise Information Heterogeneity, E"LORA adopts an entropy-driver proxy strategy for rank allocation. We utilize the absolute gradient proxy entropy of each layer direct measure of its informational significance, guiding adaptive rank assignment across intervals. Layers with richer information are allocated higher ranks to preserve expressiveness, while less critical layers use lower ranks. Overall, E achieves an optimized trade-off between efficiency and effectiveness.
Our contributions can be summarized as follows:
Novel Insights into Parameter Sharing: Using a gradient- proxy entropy analysis,
reveal two previously overlooked properties of pre-trained models: (1) Local Similar- ity contiguuu regions where adjacent layers exhibit high information overlap, and (2) Layer- wise Information settenggeeeiy--uusttneeee variation in information content across layers.
2

---

Published as a conference paper at ICLR 2026
These insights highlight the limitations of existing parameter-sharing methods that rely uni- form or heuristic patterns while ignoring such structural heterogeneities.
Dual Adaptive Parameter Sharing Framework: Motivated by these insights, we propose E'LORA, a novel framework that jointly optimizes adaptive parameter sharing and LoRA rank allocation. To address Local Similarity, E*LORA partitions layers into adaptive sharing inter- vals based on gradient information similarity. To tackle Layer-wise Information Heterogeneity, it leverages absolute gradient proxy entropy to dynamically assign LoRA ranks per sharing in- terval. This dual-adaptive design enables highly efficient parameter utilization while preserving or even enhancing model performance.
Balance: We conduct extensive evaluations for the proposed E.LORA across diverse tasks, models, and modalities. Experimental results demonstrate that EP L.LA strikes an excellent balance of parameter efficiency and performance, consistently matching or surpassing sharing and non-sharing baselines while significantly reducing the num- ber of trainable parameters by approximately 50%.
Proven Efficiency-Effiectiveeee
RELATED WORK
Parameter-sharing LoRA Variants have emerged as a promising direction for enhancing the ef- ficiency and effectiveness of LoRA. Various approaches have been proposed to optimize different aspects: ShareLoRA (Song et al. 2024) introduces cross-layer sharing of low-rank matrices (A, B, or both). VeRA (Kopiczko et al. 2023) implements more constrained approach by sharing fixed low- rank matrices across layers, updating only layer-specific scaling vectors. Beyond efficiency, some sharing strategies also target performance improvements. Rasa (He et al.) 2025b) shares a certain ratio of ranks from each adapter, concatenating shared and unique low-rank weights to increase over- all capacity, while RandLoRA (Albert et al. 2025) shares full-rank random bases across layers and introduces trainable diagonal scaling matrices to enable high-rank fine-tuning. HydraLoRA (Tian et al. 2024) further adopts an asymmetric shared-A A design within a router- architecture to mit- igate task interference on heterogeneous data, and BSLoRA (Zhou et al.) decomposes parameters into local, intra-layer, and globally shared components to build a fixed hierarchical sharing struc- ture. In contrast to these methods with manually specified sharing topologies, our approach derives sharing intervals directly from task gradients via entropy and similarity statistics.
Rank-Allocatio LoRA Variants enhance the expressiveness of LoRA by adaptively allocating ranks. For instance, AdaLoRA (Zhang et al. (202cc) begins training with an over-parameterized rank budget and progressively prunes less important ranks. Conversely, IncreLoRA (Zhang et al. 2023a) starts with rank-1 1 adapters and gradually allocates higher ranks to adapters with higher im- portance, avoiding the extra cost overhead of wver-parameterization. ALoRA (Liu et al., (024b) and DyLoRA (Valipour et al. (023) also adjust ranks dynamically, either pruning and reallocating capacity or supporting search-free flexible ranks at inference time, while RA-LORA (Kim et al. 2024) focuses on quantized LLMs by assigning higher ranks to layers with larger quantization
rors. LoRA-GA (Wang et al. (224a) and LoRA-One (Zhang et al., (025) further exploit gradient information to guide rank or parameter allocation, bringing LoRA closer to full fine-tuning behav- ior. GoRA (He et al. 2025a) sets layer-wise ranks upfront via gradient importance from pre-trained weights. This avoids dynamic shape adjustment during training. benefiting distributed strategies like ZeRO (Rajbhandari et al. 2020). RaLoRA (Ye et al. (026)) adaptively aligns ranks with the intrinsic dimensionality of layer-wise wise gradients. All of these methods primarily adjust ranks within layers, whereas our E LoLARA combines lnter-layer parameter sharing with entropy- -uided rank allocation to jointly optimize efficiency and effectiveness.
Entropy and its variants have been widely used in deep learning as measures of information con- tent. Prior work (Lin et al.] 2024 Yang et al.] 2025 Lu et al.) 2024) has employed inter-layer entropy to guide model pruning, simplifying architectures without performance degradation. In addition, entropy-based similarity has been leveraged in tasks such as knowledge distillation (Jiao et al. 2019). model comparison (Hoyos-Osorio & Sanchez-Giraldo 2023 Ali et al.) 2025). and attention optimization (Zhai et al. (023). demonstrating strong empirical effectiveness. Inspired by these successes, and building upon the computational forms for entropy established in related literature, our work is the first to explore the use of such entropy- measures and their derived similarities in the context of LoRA. Specifically, we utilize these proxy entropy-based features,
3

---

Published as a conference paper at ICLR 2026
(1) Vanilla LoRA
(TD Shared LoRA
(IT) BLLORA
(1) Forward Rackword
( Pest-Turing
Red Natrix
Trresholal
PNI Computattoon
Layer-wiss Average
W
Di Different
(2) byydeedd Sharlin (LS5)
Adeptive
Porkittaning
W
Shorting Config
:
Rank & LoRA
Levervals
Wo
Gy
(3) Hekorsgeneity based Rank Allocaston (HRA)
Rank Allocation
(2.42%%
Procy EnLingy
12
Proxy Entroopy
Interval-wrse
(1/2).
Figure 2: Overall LoLORA framework. (I) shows Vanilla LoRA. (II) depicts LoRA shared glob-
ally. (III) details ELLORA: (1) Layer gradients are first obtained. (2) Local Similarity -based Shar-
ing utilizes RMI computation and adaptive interval partitioning configure sharing groups. (3)
Heterogeneity-basee Rank Allocation employs entropy computation to assign appropriate ranks. (4) Finally, these adaptively configured LoRA adapters are applied during post-tuning.
tracted from downstream task gradient data, to adaptively guide parameter sharing-4 previously unexplored direction for LoRA.
METHOD
Our proposed framework ELLORA successfully optimizes the trade-off between parameter effi ciency and model performance in LoRA-based approaches. As shown in Figure [ the core of our method includes two components: Local Similarity-baser Sharing (LSS) and Heterogeneity-batt Rank Allocation (HRA). The overall flow is detailed in Algorithm| 1 We will introduce these two parts in detail below, and then introduce how to combine E*LORA with different LoRA variants.
3.1 LOCAL SIMILARITY-BASED SHARING (LSSS)
To facilitate effective parameter sharing while preserving representational diversity, we first identify groups of layers in the pre-trained model that exhibit high similarity in their information content before fine-tuning. This process consists of two key steps: (1) quantifying inter-layer similarities by analyzing information of the gradient, and (2) adaptively partitioning layers into sharing intervals based on these similarity measures.
Quantifying Inter-Layer Gradient Similarity. We hypothesize that gradient information, derived from specific downstream tasks, reflects the similarity of information across different layers and can guide more efficient parameter sharing. Following (Lin et al.) 2024). we define an entropy-like metric to quantify the information spread within the gradients of each layer. For given layer 1, let G denote its gradients obtained on mini-subset of the training dataset. We calculate this measure, which we term 'proxy entropy' H (G.),, as:
1
1
H(G:) log(oo,) + log (yg( +
where OG, is the standard deviation of the elements in the flattened gradient tensor G,. This ap- proximation effectively captures the spread and uncertainty with the layer's gradients. We provide detailed theoretical justification connecting this proxy entropy to the Frobenius norm and Fisher Information in Appendix B
To assess the redundancy or similarity in information between two layers i and j, compute the Mutual Information (MI) between their respective gradient tensors, G, and G,. based on the proposed proxy entropy. Mutual Information measures the amount of information shared between
4

---

Published as a conference paper at ICLR 2026
two variables, and it is defined as:
I(G,; (G,; G,) - H (GG G2).
Here, H G represents the joint entropy of the gradient tensors G; and Gj: To compute this, we flatten G, and Gj, concatenate them, and then approximate H(G,, G2) using equation | A higher mutual information value indicates greater shared information and potential redundancy between the spec-fic functions of two layers.
Since MI is an absolute quantity, it does not reflect the proportion of shared information. To provide normalized and comparable measure of similarity of different layer pairs, we adopt Relative Mutual Information (RMI):
IG G3)
RMI(G,; G2)
(3)
(i.(H((G.), H(G))
RMI provides a normalized measure of similarity, ranging from 0 to 1, making it suitable for com- paring similarities across different layer pairs regardless of their absolute information content.
Adaptive Sharing Interval Partitioning. With the inter-layer RMI values, we proceed to dynam- ically partition pre-trained model's layers into variable-length sharing intervals. This adaptive partitioning respects the "Local Similarity" property identified in the Introduction, where adjacent layers often exhibit higher similarity, but the range of this similarity varies and is not fixed. Our goal is to group layers that are informationally similar into contiguous blocks, allowing parameter sharing within these blocks.
Our partitioning strategy is greedy, sequential process guided by self- self-adaptive, layer-specific simi- larity thresholds. For each layer Gm in a model with N layers, we first compute the Relative Mutual Information (RMI) between Gm and every other layer G2 (where ls o m). A unique threshold Tm for layer Gm is then determined by averaging these computed RMIs:
N
RMI(Gm, G+).
(4)
Here, the layer-specific Tm serves as a dynamic baseline to identify layers sufficiently similar to Gm. guiding the adaptive partitioning of consecutive layers into sharing intervals. The partitioning algorithm then proceeds greedily and sequentially. Starting a new interval from the current layer, expands by including subsequent layers long their RMI with the interval's starting layer meets the determined threshold. Once the similarity condition is no longer met, the current interval is finalized, and new one commences. This iterative process continues until all layers assigned to a non-overlapping sharing interval, as detailed in Algorithm 2
This process results in set of non-overlapping, consecutive sharing intervals, each containing layers that exhibit a high information similarity via their gradient properties. We further validate this design in Appendix G by comparing it against a globally optimal Dynamic Programming (DP) solution, demonstrating that our greedy strategy achieves comparable performance while being significantly faster and eliminating the need for hyperparameter tuning.
3.2 HETEROGENEITY BASSED RANK ALLOCATION (HRA)
The second crucial component of LORA is the adaptive allocation of LoRA ranks, addressing the Layer-wise Information Heterogeneity property identified in the Introduction. While the previous section detailed how to group layers into sharing intervals reduce parameters, this section focuses on distributing the available LoRA rank capacity among these intervals intelligently. Our goal is to assign more expressive power, i.e., higher ranks, to intervals that carry more critical information, thus preserving overall performance while maintaining parameter efficiency.
For each sharing interval [sk: identified by the Adaptive Sharing Interval Partitioning algorithm, we need to determine representative importance score. Since each interval will be associated with a single shared LoRA adapter, this adapter's 's rank should reflect the information content of the layers it represents. We utilize the gradient proxy entropy H (Gi) calculated for each layer defined in equation I as measure of its information content. To ensure that even the most information-rice layer within an interval, define the representative proxy entropy for interval & the maximum
5

---

Published as a conference paper at ICLR 2026
Algorithm 1: Overall Algorithm for E*LORA Framework
Input: Model gradients ,.. G(N).
Number of layers N, Base rank Tnanilla
Output: Sharing intervals S {sE, Ex]), Allocated ranks R {ru}
Step I: Compute Local Similarity- based Sharing (LSS)
for each layer do
Compute proxy entropy (G))
// based equation
Compute pairwise RMI(G, G
// based on equation
Step 2: Adaptive Partitioning
for each layer I do
Compute threshold 7, N Enst RMI Gx)
Form sharing intervals S greedily;
// based on
Step 3: Heterogeneity- for each interval [Sk: ex] E S do
Rank Allocation (HRA)
Hinterval, max (G:):
// based on equation
Normalize [ } to get allocation factors FA:
// based equation
for each interval k do
rk round Fi (N Twanilla))
// based on equation
return S, R
proxy entropy among all layers within that interval:
Hintervaly = max H(G)). 1e(s
(5)
This approach prevents high-information layers from being "dragged down" by less informative layers within the same shared block, thereby safeguarding the model's critical information flow.
With the representative proxy entropy for each of the K sharing intervals ([s1,e]], ei]....... ., [sx exl ]. normalize these values to derive allocation factor for each interval. This normalization ensures that the sum of factors equals one, allowing for proportional rank distribution. The allocation factor Fg for interval & is given by:
Hintervale
FR
ΣK Hinterval,
A higher allocation factor F& signifies that interval & contains more critical information and thus warrants a larger share of the total LoRA rank budget.
To maintain comparable overall model capacity and ensure fair comparison with vanilla LoRA, we fix the total LoRA rank budget across the entire model. If standard LoRA application assigns fixed rank Tvanilla each of the N transformer layers, the total rank budget is N Ivanilla- Our method distributes this budget adaptively. The allocated rank r3 for the shared LoRA adapter corresponding to interval is calculated
FA round (F& (N X Tvanilla))
Here, the round(.) denotes rounding to the nearest integer. The Local Similarity- bassd Sharing (LSS) primarily reduces parameters by enabling adaptive sharing across similar layers while main- taining model expressiveness. Simultaneously. the Heterogeneity- -based Rank Allocation (HRA) im- proves performance by adaptively allocating higher ranks more information-rich intervals. This dual adaptive mechanism collectively allows LaLORA to significantly reduce trainable parameters while preserving or even enhancing overall model performance. Detailed procedures of HRA are provided in Algorithm 3
3.3 INTEGRATION WITH NON-SHARING AND PARAMETER-STIARIE LORA METHODS
The E'LORA framework seamlessly integrates with both non-sharing and parameter-sharme sharing LoRA approaches. In the non-sharing setting, layer-specific adapters replaced by shared adapters over adaptive intervals, with ranks dynamically assigned based on proxy entropy. In the parameter- sharing setting, globally shared components (A, B or both) initialized at the maximum possible

---

Published as a conference paper at ICLR 2026
Table I: Performance of fine-tuning Roberta-Base on 5 sub-tasks of the GLUE benchmark.
Method
#Params
MNLI
SST2
COLA
QNLI
QQP
Average
FFT
125.00M
87.60
94.80
63.60
92.80
91.90
86.14
DoRA (rank4)
0.17M
85.39
94.61
62.35
92.48
88.98
84.76
DoRA
0.31M
86.03
94.04
61.53
92.29
89.94
84.77
AdaLoRA (rank4)
0.15M
86.85
94.15
58.55
91.13
89.01
83,94
AdaLoRA
0,30M
87.09
94.61
58.55
91.87
89.54
84.33
LoRA+
0.30M
87.20
94.61
63.31
92.25
90.75
85.62
LoRA-FA
0.15M
86.80
94.80
63.60
92.50
90.10
85.56
VeRA
0.04M
85.63
92.77
60.31
91.94
88.51
83.83
LoRA
0.30M
87.49
94.57
63.85
92.73
90.70
85.87
Ours LoRL)
0.16M
86.89
94.69
63.37
92,45
90.44
85.57
Soa
0.16M
86.99
94.19
61.17
92.46
90.15
84.99
Ours Sh"reharrlLoo ShareL
0.08M
86.74
94.61
63.19
92,46
90.00
85.40
rank; during computation, only the dimensions corresponding to each layer's dynamically deter- mined rank are accessed. This dynamic slicing mechanism allows E=1.ORA to retain the efficiency of existing global sharing strategies while enabling proxy entropy-guided rank allocation.
EXPERIMENTS
We assess LORA's effectiveness across diverse model architectures and tasks, including natural language understanding, natural language generation, and image classification. To conduct a com- prehensive assessment, integrate E*LORA with both non-sharing and sharing variants. Specif- ically, for the non-sharing configuration, we combine with vanilla LoRA, while for the sharing setting we adopt ShareLoRA, which globally shares the matrix while retaining layer-specific B matrices. To guarantee fair comparisons, we strictly maintain identical training configurations as those used by both LoRA and SoareLoRA.
4.1 NATURAL LANGUAGE UNDERSTANDING (NLU) TASKS
2019) model on five subsets of GLUE benchmarks [Wang et al., 2018), specifically MNLI, SST-2, CoLA, QNLI, and QQP. Detailed train- ing configurations are provided in Appendix C.3
For NLU tasks, we fine-tune the Roberta- Base (Liu et al.
Results. Table summarizes performance on five GLUE tasks. EoLoRA demonstrates superior pa- rameter efficiency. achieving average score of 85.57 with 0.16M trainable parameters, compara- ble to LoRA's 85.87 (0.3M parameters). Furthermore, E ShareLoRA significantly boosts efficiency and effectiveness: it attains 85.40 with just 0.08M parameters, outperforming ShareLoRA's 84.99 (0.16M parameters). This highlights E2LORA's 's capability to enhance both model efficiency and accuracy by effectively managing parameter sharing.
4.2 NATURAL LANGUAGE GENERATION (NLG) TASKS
For NLG tasks, we evaluate mathematical reasoning and code generation capabilities using 100K subset of MetaMathQA (Yu et al. 2023) for mathematics and 100K code-only subset of Code-FeedBack (Zheng et al. (024) for coding. For evaluation, we employ accuracy scoring on GSM8K (obbe et al. 2021) outputs and PASS@ for HumanEval (Chen al., 202T). Detailed training configurations are provided in Appendix C.4
Results. Table 2 summarizes E*LORA's performance and parameter efficiency on GSM8K and HumanEval. LoLRR, integrated with vanilla LoRA, halves trainable parameters from 6.82M to 3.66M, improving GSMSK accuracy from 70.21 to 70.51 and achieving 1.63-point gain on Hu- manEval. BA further reduces the number of parameters from 2.75M to 1.59M, less than 60% of Loa'eLoRA's parameters. While EhaaeLLoR''s performance on GSM8K remains comparable 70.26% versus 70.51%. 1., significantly improves pass @ I HumanEval from 43.69 to 44.71. This demonstrates E'LoRA's ability to maintain expressiveness and significantly re- duce parameter counts through adaptive sharing interval partitioning and rank allocation. To fur-

---

Published as a conference paper at ICLR 2026
Table 2: Performance of fine-tuning -lama-3.1-8b-Bazz on GSM8K and HumanEval.
GSM8K
HUMANEVAL.
Method
#Params
Ace
#Params
Pass@1
FFT (Attn-only)
1.34B
73.69
1.34B
51.63
DoRA (rank4)
3.74M
67.40
3.74M
43.90
DoRA
6.82M
69.17
6.82M
43.70
AdaLoRA (rank4)
3,41M
56.56
3.41M
35.37
AdaLoRA
6.82M
70.63
6.82M
41.46
LoRA+
6.82M
71.29
6,82M
44.51
LoRA-FA
3,41M
68.66
3,41M
42.07
VeRA
0.33M
67.63
0.33M
42.48
LoRA
6.82M
70.21
6,82M
42.68
Ours LE'L)RA)
3.61M
70.51
3.66M
44.31
ShareLoRA
2.75M
70.51
2.75M
43.69
Ours (E" ShareLoRA)
1.59M
70.26
1.59M
44.71
Table 3: Performance of fine-tuning VLT--V11 across multiple image classification datasets.
Method
#Params
Cars
DTD
Eurosat
GTSRB
Resise45
Sun397
SVHN
Average
Zero-shot
63.75
44.39
42.22
35.22
56.46
62.56
15.53
45.73
FFT (ViT-only)
86.38M
84.23
77.44
98.09
94.31
93.95
75.35
93.04
88.06
DoRA (rank=4)
0.75M
81.43
74.79
98.31
95.68
94.06
76.41
96.37
88.15
DoRA
1.39M
82.44
76.86
98.43
97.25
95.10
77.30
96.63
89.14
AdaLoRA (rank=4)
0.66M
72.35
72.18
96.46
44.16
87.00
70.80
93.49
76.63
ARA
1.31M
73.29
74.31
97.06
58.87
88.51
71.76
94.21
79.72
LoRA+
1.31M
86.61
73.33
98.54
98.99
96.06
76.80
96.98
89.62
LoRA-FA
0.66M
79.55
73.78
98.23
92.84
93.85
74.76
95.96
86.98
VeRA
0.10M
82.36
76.35
98.29
97.33
95.21
76.80
96.48
88.97
LoRA
1.31M
82.26
76.88
98.42
97.10
95.02
77. 77.23
96.62
89.08
Ours LoRA)
0.66M
83.47
77.39
98.59
98.67
95.58
78.17
96.83
89.81
ShareLoRA
0.72M
82.00
75.83
98.35
96.91
94.86
76.83
96.47
88.75
Ours (E ShareLoRA)
0.39M
83.27
77.48
98.50
98.40
95.62
78.02
96.79
89.73
ther verify the method's generality on diverse reasoning tasks, we also evaluated EPLoRA on the CommonSenseQA benchmark. As detailed in Appendix KJ LoLoRA maintains strong efficiency- performance balance even on complex commonsense reasoning subtasks.
4.3 IMAGE CLASSIFICATION TASKS
For image classification tasks, we fine-tune the ViTB-V1T-B/6 model on seven widely used image classification benchmarks. Detailed training configurations are provided in Appendix C.5
Results. Table summarizes E2LORA's robust performance and parameter efficiency across im- age classification tasks. EaLORA consistently achieves superior or comparable accuracy vanilla LoRA while utilizing approximately half trainable parameters, 0.66M versus 1.31M. This yields average accuracy improvement of 0.73, reaching 89.81%, 1,, with significant gains on GTSRB and Cars. Integrating with ShareLoRA. EhSharLLoRA further enhances efficiency, reducing the number of parameters from 0.72M to 0.39M. Simultancously, it boosts average accuracy by around 0.98, reaching 89.73%. Substantial improvements are observed across all datasets. This consistent uplift underscores ELLORA's ss ability to mitigate performance decline in globally shared methods, opti- mizing both efficiency and accuracy.
DISCUSSION
Effect of LSS and HRA. To validate the effectiveness of our proposed components, LSS and HRA, conducted ablation study, the results of which are summarized in Table 4 LSS alone signif- icantly reduces trainable parameters while maintaining competitive accuracy, demonstrating its ef- ficacy in parameter reduction through adaptive sharing. HRA notably enhances model performance
8

---

Published as a conference paper at ICLR 2026
by optimally distributing parameters based layer-wise information heterogeneity. Together, these components synergistically enhance both parameter efficiency and performance.
Table 4: Ablation Study of Different Components of LoRO,, evaluated using
Table 5:
Compare E*LORA and LoRA
with similar amounts of parameters by ad- justing rank, evaluated using Llama3.1-8P on GSM8K.
Llama3. 1-8B on GSM8K.
LSS
HRA
#Params
ACC
V
x
3.51M
69,45
Rank
#Params
Acc
X
V
6,82M
71.19
4
3,41M
67.58
cosine
V
3.75M
66.67
LoRA
8
6.82M
70.21
L11
3.75M
68.48
L2
3.75M
67.93
LoRA
8
3.61M
70.51
KL
3.43M
67.57
16
7.22M
71.19
V
3.61M
70.51
Efficiency Comparison at Matching Parameters. To further assess the parameter efficiency of ELLORA compared to LoRA, we conduct an experiment where we control the rank to ensure sim- ilar parameter counts. As shown in Table under comparable parameter budgets, E*LORA con- sistently outperforms LoRA with higher ranks. Specifically, E*LORA with rank 8 surpasses LoRA with rank 4 by 0.3 points. Moreover, E*LORA with rank 16 outperforms LoRA with rank 8 by margin of 0.98 points. These results confirm that with the same parameter scale, L*LORA also achieves better performance, further demonstrating its superior parameter efficiency. We further extend this marcheter-matched comparison to Computer Vision (CV) and Natural Language Under- standing (NLU) tasks. As detailed in Appendix [ consistent improvements are observed across all modalities, verifying the generalizability of our efficiency gains.
Orthogonality with Existing LoRA Variants. Building on exper- iments where E'LORA augments vanilla LoRA and ShareLoRA, further test its compatibility with diverse PEFT variants. ELLORAA acts as plug-and-play enhance- ment guided by intrinsic proper- ties of the pre-trained model, and thus orthogonal to the particular adapter architecture. We apply its adaptive sharing and rank allocation to performance-oriente DoRA and efficiency-focused VeRA and LoRI; shown in Table 6 the result- ing E2DOR,, E2VeRA, and E'LoRI consistently improve parameter effi- ciency, with E2DoRA matching or surpassing DoRA on NLU, CV, and NLG tasks using about 50% fewer parameters, EVVeRA and ELLoRI reaching comparable or better accuracy while further reducing parameter counts. These results support ELLORA as a caurpoo- compression strategy that amplifies the efficiency of existing and future LoRA-style methods.
Table 6: Demonstration of LoLRA's orthogonality. By applying our adaptive framework existing LoRA variants (DoRA, VeRA, LoRI), we consistently achieve significant parameter reduction while maintaining or improving perfor- mance across GLUE (NLU) and a collection of CV datasets. 'F2X* denotes the combination of ELLORA with method X.
NLU (GLUE)
CV
Method
#Params
Avg.
#Params
Avg.
DoRA
0.31M
84.77
1.39M
89.14 14
E*DoRA
0.15M
84.79
0.75M
89.82
VeRA
0.04M
83.83
0.10M
88.97
EVRARA
0.013M
84.31
0.05M
89.13
LORI-S
0.015M
78.26
0.066M
89.47
E'LORI-S
0.007M
78.96
0.033M
89.43
Effect of Different Similarity Metries. An effective parameter sharing strategy hinges critically on accurately quantifying inter-layer similarity to enable adaptive sharing ranges. As demonstrated in Table and Figure 3 our proposed proxy entropy similarity metric proves highly effective for this purpose. Specifically. Figure 3 a) reveals that proxy entropy similarity captures fine-graine grained local dependencies essential for precise adaptive range partitioning. stark contrast, 3 shows that cosine similarity produces nearly uniform and uninformative similarity matrix, rendering it ill-suited for guiding adaptive parameter sharing. This observation is further corroborated quantita- tively: substituting our entropy-based similarity with other similarity metries leads to significant performance drop, highlighting the indispensable role of entropy-based similarity in achieving both effective and performance- prrformanceepreeerving preserving parameter sharing. Additional visual comparisons of various similarity metrics are provided in Appendix D.3
9

---

Published as a conference paper at ICLR 2026
Proxy Entropy Similarity
Cosine Similarity
1.00
3.00
Number
Layer Number
0.04
ta 12
0.98
53
0.97
2
0.95
-7
+9
Layer Number
Layer Number
(e) Proxy Entropy Similarity
(b) Costne Similarity
Visual comparison of different simi- larity metrics with Jlama-3.1-8B trained on the MetaMathQQ dataset. Visualization of other metrics can be found in the Appendix[... D.3
Figure 3:
Figure 4: Gradient proxy entropy dynamics across training for attention matrices (We. WK. Wr. Wo) in layers 0, of Llama-3.1-81 trained on MetaMathQA.
Step-wise Importance Analysis. We conduct a step- importance analysis during fine-tuning and find that module importance remains stable across steps. Wv modules dominate WK and Wo. and middle layers are more important than upper layers, consistent with prior research. This stabil- ity justifies our static rank allocation based initial gradient entropy without extra computation. Figure 4 and Appendix (E report representative dynamics and further analyses.
Effect of rank. As shown in the left panel of Figure [ scaling the rank consistently improves performance on both datasets. Importantly, our method not only maintains competi- tive performance at lower ranks but also yields substantial performance gains with higher ranks.
Effect of Rank
Effect of Step
72
72
Performance
Performance
64
64
GSM-0K
GSM 8K
56-
HumanEval
56
HumanEval
48
48
Effect of step. The sampling step controls the data used for gradi- ent computation with Subset Size Steps Batch Size. The right panel of Figure 5 shows that performance is stable across wide range of step values, indicating our method is robust to the choice of step.
40
32
40
128
256
512
Rank
Step
Figure 5: Ablation study of rank and step size, evaluated using Llama-3.1-88 on GSM8K and HumanEval.
Robustness Analysis. We evaluate the stability of our partitioning and rank allocation under dif- ferent random seeds reflect sampling variability, reported in Appendix [ We also examine the robustness of RMI-based thresholding under noise perturbations, with results presented in Ap- pendix H These results show that adaptive decisions remain consistent and insensitive small estimation noise, demonstrating the structural stability and practical reliability of the framework.
Experiments on larger and diverse models. We further evaluate E'LORA Llama-2-13B, Qwen3-8B, and GPT-OSS-20B and observe that it consistently outperforms standard LoRA with comparable accuracy and much fewer trainable parameters, as summarized in Appendix L
Computation Time and Reusability. The initialization overhead is negligible, requiring approxi- mately 30 seconds compared to 40 minutes for fine-tuning. Crucially, the determined sharing con- figurations can be reused for the same model and task, rendering this a one-time cost when exploring different training setups, such as hyperparameter tuning.
6 CONCLUSION
In this paper, we propose E*LORA, novel parameter- sharing framework for efficient and effec- tive Low-Rank Adaptation. Inspired by a gradient entropy-based analysis of pre-trained models, we introduced local similarity-based shared area division and layer-wise heterogeneity-base rank allocation. L effectively halves the number of trainable parameters while consistently main- taining surpassing the original performance. Extensive validation confirms the effectiveness and robust generalizability of LoLRRA across diverse model architectures and datasets, alongside its seamless integration with existing sharing and non-sharing LoRA variants across every task.
10

---

Published as a conference paper at ICLR 2026
CONTRIBUTION
Minglei Li: Investigation, Methodology, Software, Visualization, Writing - original draft. Peng Ye: Methodology, Formal analysis, Conceptualization, Visualization, Writing review & editing- Jingqi Ye: Methodology, Validation, Writing review & editing. Haonan He: Software, Validation, Writing - review & editing. Tao Chen: Supervision, Resources, Project administration.
ACKNOWLEDGMENTS
This work is supported by National Key Research and Development Program of China (No. 2022ZD0160101), Shanghai Natural Science Foundation (No. 23ZR1402900) Shanghai Science and Technology Commission Explorer Program Project (24TS1401300) Shanghai Municipal Sci- ence and Technology Major Project (N0.20211H7277X0000)) The computations this research performed using the CFFF platform of Fudan University.
REFERENCES
Paul Albert, Frederic Z Zhang. Hemanth Saratchandran, Cristian Rodriguez-Opazo, Anton den Hengel, and Ehsan Abbasnejad. Randlora: Full-rank parameter-efficien fine-tuning of large mod- els. arXiv preprint arXiv:2502.00887, 2025.
Riccardo Ali, Francesco Caso, Christopher Irwin, and Pietro Liò. Entropy-lens: The information
signature of transformer computations. arXiv preprint arXiv:2502. 16570, 2025.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 2021.
Gong Cheng, Junwei Han, and Xiaoqiang Lu. Remote sensing image scene classification: Bench- mark and state of the art. Proceedings of the IEEE, 105(10):1855-1883 2017.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2 10. 10.22//1.41400 14168, 2021.
Tri Dao. Flashattention-2: Faster attention with better parallelism and work partitioning. arXiv preprint 230.iv::3000006991 2023.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, al. The llama 3 herd of models. arXiv preprini arXiv:2407.21783. 2024.
Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. Parameter-efficient fine-tuning for large models: A comprehensive survey. arXiv preprint arXiv:2403. 14608, 2024.
Soufiane Hayou, Nikhil Ghosh, and Bin Yu. Lorat: Efficient low rank adaptation of large models. arXiv preprint arXiv:2402.1 / 12354, 2024.
Haonan He, Peng Ye, Yuchen Ren, Yuan Yuan, Luyang Zhou, Shucun Ju, and Lei Chen. Gora: Gradient-driven adaptive low rank adaptation. arXiv preprint 17X11:2502.12171, 2025a.
Zhiwei He. Zhaopeng Tu, Xing Wang, Xingyu Chen, Zhijie Wang, Jiahao Xu, Tian Liang, Wenxiang Jiao, Zhuosheng Zhang, and Rui Wang. Rasa: Rank-sharing low-rank adaptation. arXiv preprint TrXiv:2503.12576 2025b.
Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7):21172222) 2019.
Sebastian Houben, Johannes Stallkamp, Jan Salmen, Mare Schlipsing, and Christian Igel. Detection of traffic signs in real-world images: The german traffic sign detection benchmark. In The 2013 international joint conference neural networks (IJCNN), pp. 1-8. leee, 2013.

---

Published as a conference paper at ICLR 2026
Jhoan K Hoyos- Osorio and Luis G Sanchez-Giraldo The jhann divergence. arXiv preprint arXiv:2305. 2023.
Edward Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2021.
Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu. Tinybert: Distilling bert for natural language understanding. arXiv preprint arXiv: 1909. 10351, 2019.
Dhiraj Kalamkar, Dheevatsa Mudigere, Naveen Mellempudi, Dipankar Das, Kunal Banerjee, Sasikanth Avancha, Dharma Teja Vooturi, Nataraj Jammalamadaka, Jianyu Huang, Hector Yuen, et al. A study of 6 for deep learning training. arXiv preprint 1xXiv:195.112320 2019.
Minsoo Kim, Sihwa Lee, Wonyong Sung, and Jungwook Choi. Ra-lora: Rank-adaptive parameter- efficient fine-tuning for accurate 2-bit quantized large language models. In Findings of the Asso- ciation for Computational Linguistics ACL 2024, pp. 15773-15786, 2024.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412,694,, 2014.
Dawid J Kopiczko, Tijmen Blankevoort, and Yuki M Asano. Vera: Vector-based random matrix adaptation. arXiv preprint arXiv:2310.1 1154, 2023.
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-graing the-grained categorization. In Proceedings of the IEEE international conference computer vision work- shops, pp. 554-561, 2013.
Sihao Lin, Pumeng Lyu, Dongrui Liu, Tao Tang, Xiaodan Liang, Andy Song, and Xiaojun Chang. Mīp can be good transformer learner. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2024.
Shih- Shih-Yang Liu, Chien-Yi Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang- Ting Cheng, and Min-Hung Chen. Dora: Weight-decomposet low-rank adaptation. In Forty-first International Conference Machine Learning, 2024a.
Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Lam Tam, Zhengxiao Du Zhilin Yang, and Jie Tang. P- tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks. arXiv preprint 10.0760,, 2021.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv: 1169, 2019.
Zequan Liu, Jiawen Lyn, Wei Zhu, Xing Tian, and Yvette Graham. Alora: Allocating low-rank adaptation for fine-tuning large language models. arXiv preprint arXiv:2403. 16187, 2024b.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv:1711.05100 2017.
arXiv preprint
Yiheng Lu, Ziyu Guan, Yaming Yang, Wei Zhao, Maoguo Gong, and Cai Xu. Entropy induced pruning framework for convolutional neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 3918-3926, 2024.
Kai Lv, Hang Yan, Qipeng Guo, Haijun Lv. and Xipeng Qiu. Adalomo: Low-memory optimization with adaptive learning rate. arXiv preprint 20X 2023.
Yuren Mao, Yuhang Ge, Yijiang Fan, Wenyi Xu, Yu Mi, Zhonghao Hu, and Yunjun Gao. A survey on lora of large language models. Frontiers of Computer Science, 19(7):197605, 2025.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning, volume 2011, pp. 4. Granada, 2011.
12

---

Published as a conference paper at ICLR 2026
Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. Zero: Memory optimizations toward training trillion parameter models. SC20: International Conference for High Perfor- mance Computing, Networking. Storage and Analysis, pp. 1-16. IEEE, 2020.
Adithya Renduchintala, Tugrul Konuk, and Oleksii Kuchaiev. Tied-lora: Enhancing parameter effi- ciency of Iora with weight tying. arXiv preprint arXiv:2311.09578. 2023.
Yurun Song, Junchen Zhao, lan G Harris, and Sangeetha Abdu Jyothi. Sharelora: Parameter effi- cient and robust large language model fine-tuning via shared low-rank adaptation. arXiv preprint arXiv:2406. 10785, 2024.
Chunlin Tian, Zhan Shi, Zhijiang Guo, Li Li, and Cheng-Zhong Xu. Hydralora: An asymmetric lora architecture for efficient fine-tuning. Advances in Neural Information Processing Systems, 37:9565-9584, 2024.
Mojtaba Valipour, Mehdi Rezagholizadch, Ivan Kobyzev, and Ali Ghodsi. Dylora: Parameter- efficient tuning of pre-trained models using dynamic search-free low-rank adaptation. In Pro- ceedings of the 17th Conference of the European Chapter of the Association for Computational 2023.
Linguistics, pp.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.074500 2018.
Shaowen Wang, Linxi Yu, and Jian Li. Lora-ga: Low-rank adaptation with gradient approximation. Advances in Neural Information Processing Systems, 37:54905-54933 2024a.
Zhengho Wang, Jian Liang, Ran He, Zilei Wang, and Tieniu Tan. Lora-pro: Are low-rank adapters properly optimized? arXiv preprint arXiv:2407.18222, 2024b.
Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE computer society conference on computer vision and pattern recognition, pp. 3485-3492. IEEE, 2010.
Liangwei Yang. Yuhui Xu, Juntao Tan, Doyen Sahoo, Silvio Savarese, Caiming Xiong, Huan Wang, and Shelby Heinecke. Entropy-based block pruning for efficient large language models. arXiv preprint arXiv:2504.03794 2025.
Jingqi Ye, Haonan He, Minglei Li, Fujun Han, Tao Chen, and Peng Ye. Gradient intrinsic dimen- sionality alignment: Narrowing the gap between low-rank adaptation and full fine-tuning. In International Conference on Learning Representations, 2026.
Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T Kwok, Zhen- guo Li, Adrian Weller, and Weiyang Liu. Metamath: Bootstrap your own mathematical questions for large language models. arXiv preprint arXiv:2309. /2284, 2023.
Elad Ben Zaken, Shauli Ravfogel, and Yoav Goldberg. Bitfit: Simple parameter-effiiien fine-tuning for transformer-based masked language-modes arXiv preprint (0.i::2/06//0//// 2021.
Shuangfei Zhai, Tatiana Likhomanenko, Etai Littwin, Dan Busbridge, Jason Ramapuram, Yizhe Zhang, Jiatao Gu, and Joshua M Susskind. Stabilizing transformer training by preventing attention entropy collapse. In International Conference on Machine Learning, pp. 00770-40803. PMLR, 2023.
Feiyu Zhang, Liangzhi Li, Junhao Chen, Zhouqiang Jiang, Bowen Wang, and Yiming Qian. In- crelora: Incremental parameter allocation method for efffmeter-efftcient fine-tuning. arXiv preprint arXiv:2308. 12043, 2023a.
Longteng Zhang, Lin Zhang, Shaohuai Shi, Xiaowen Chu, and Bo Li. Lora-fa: Memory--fficient low-rank adaptation for large language models fine-tuning. arXiv preprint arXiv:230..0330 2023b.
13

---

Published as a conference paper at ICLR 2026
Qingru Zhang, Minshuo Chen, Alexander Bukharin, Nikos Karampatziakis, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. Adalora: Adaptive budget allocation for parameter- efficient fine-tuning. arXiv preprint arXtv:2303.10512. 2023c.
Yuanhe Zhang, Fanghui Liu, and Yudong Chen. Lora-one: One-step full gradient could suffice for fine-tuning large language models, provably and efficiently. arXiv preprint urXiv:2502.01235 2025.
Tianyu Zheng, Ge Zhang, Tianhao Shen, Xueling Liu, Bill Yuchen Lin, Jie Fu, Wenhu Chen, and Xiang Yue. Opencodeinterpreeer::: Integrating code generation with execution and refinement. arXiv preprint arXiv:2402. I4658, 2024.
Yuhua Zhou, Ruifeng Li, Changhai Zhou, Fei Yang, and Aimin Pan. Bslora: Enhancing the pa- rameter efficiency of lora with intra-layer and inter-layer sharing. In Forty- International Conference on Machine Learning.
14

---

Published as a conference paper at ICLR 2026
A
NOTATION
Symbol
Description
N
The total number of layers in the pre-trained model.
G
The gradient tensor Relative Mutual Information between layer and j. normalized to [0, 1] to measure similarity.
I-th layer.
(MI(G.
8 [[sk,
The of adaptive sharing intervals, where sæ is the start layer index and eμ is the end layer index of the k-th interval.
H inttevalk
The representative entropy for an interval, defined as the max- imum proxy entropy among all layers within that interval (maxig(aa.es H()))).
F&
The normalised allocation factor for interval used distribute the rank budget.
Teanilla
The base rank used in standard LoRA (c.g.. 8), used to calculate the total rank budget.
Table 7: Notation Table
B THEORETICAL JUSTIFICATION OF PROXY ENTROPY
Below we make explicit (i) what the proposed proxy entropy actually measures, and (ii) how it connects, mathematically rigorous way, to standard information-theoret quantities.
In summary, the proposed proxy entropy is not heuristic approximation to Shannon entropy, but a mathematically well-defined quantity that exactly monotone transform of the per-parameter Frobenius norm of the gradient. Since the gradient Frobenius norm is standard Monte Carlo estimator of the per-layer Fisher-information trace, a classical information-thorette measure of parameter sensitivity, our proxy entropy inherits direct and rigorous connection established cormmation-theorelee principles. The following paragraphs provide the detailed derivation support- ing this conclusion.
(1) From proxy entropy to Frobenius norm. and hr its mean. The sample variance is
For layer E, let E Rd be the flattened gradient
of de
(e, - m2)2
Let G3 be the centered gradient tensor. Then we have
de Σ 9e. u2)2 drop.
||
then
log o; = log ||G | - log de.
Our proxy entropy is
H(Gz) = log(or) + log(2x) + 2
(G+)) is exactly a monotone transform of the per-parameter Frobenius of the centered gradient. Thus, in purely mathematical terms, our metric is not an ad-hoc heuristic but normalized, log-compressed gradient Frobenius norm.
(2) Frobenius norm and Fisher information. the score function of loss log- logliikl, the Fisher information matrix per-layer
In standard settings where the gradient corresponds
Fg E [geg]]
and its trace is
tr(F)) E [ll1 12]
15

---

Published as a conference paper at ICLR 2026
Hence, the squared Frobenius norm of the gradient is unbiased empirical estimator of tr(F)).. i.. of the total Fisher information at layer E. Our proxy entropy HG),,, being a monotone transform of [|Ge]lr, is therefore also a monotone transform of an empirical estimate of the per-layer Fisher trace.
Fisher information measures how sensitive the model to small changes of the parameters: layers with larger Fisher information are more "informative" or important for the task. Since our proxy entropy (Ge) is monotone transform of the per-parameter Frobenius norm of the gradient, it can be viewed as scalar summary of how much Fisher information each layer carries, which ties directly to a classical information-thhoree notion in simple and practical way.
C DETAILED EXPERIMENT SETTINGS
C.I RANDOM SEED CONTROL.
It is important to note that all experimental results are averaged over three independent runs, initialized with distinct random seeds, ensure the robustness and reproducibility our findings.
C.2 BASELINE METHODS.
We compare LoLORA with a range of established parameter-efficteee fine-tuning methods, catego- rized by their primary objective and sharing mechanism. This comprehensive comparison highlights the effectiveness and efficiency of our approach.
Non-Sharing Methods: These methods typically adapt models without explicit parameter sharing across layers.
LoRA (Hu al. 2021 The vanilla Low-Rank Adaptation method, which injects trainable low-rank matrices into pre-trained models to preserve performance with min- imal additional parameters.
DoRA (Liu et al. 2024a): An effectiveness-enhancii variant that re- p--rrmettrrizss LoRA by decomposing updates to pre-trained weights into magnitude and direction components.
LoRA+ (Hayou et al. 2024): y An seattiveness-boosiinn approach that optimizes
LoRA's learning dynamics through imbalance learning rates.
AdaLoRA (Zhang et al. 2023c): An seeetivenes--nnhacinn variant that dynamically
adjusts the rank budget for LoRA adapters during training.
method that freezes the matrices of LoRA adapters, training only the B matrices to reduce trainable parame- ters.
LoRA-FA (Zhang et al. 2023b):
Sharing Methods: These methods explicitly incorporate parameter sharing mechanisms to enhance efficiency.
2024): An efficiency-boostiie variant that shares the A matrix across all layers while maintaining layer-specific matrices.
Sha (Song al.
VeRA (Kopiczko et al. 2023): An efficiency-orientee method that implements a con- strained approach by sharing fixed low-rank matrices across layers and updating only introduced layer-specific scaling vectors.
C.3 DETAILED SETTINGS OF NLU.
We adopt the hyperparameters used in LoRA (Hu et al., 2021). These same hyperparameters are consistently applied across LoRA, ShareLoRA, and combinations our proposed method with them. We use the Adam (Kingma & Ba 2014) optimizer (3) 0.9, B3 0.999, weight decay=0.1) with a cosine learning rate scheduler (6% warmup). LoRA is applied only to the query and value projections. Task-specific hyperparameters (peak learning rates, batch sizes) align with LoRA and ShareLoRA. All experiments use FP32 precision and a maximum sequence length of 512 tokens. The sampling step of method is 64. Other details are shown in Table
16

---

Published as a conference paper at ICLR 2026
Table 8: Hyperparameters for fine-tuning RoBERTa-base LoRA on GLUE benchmarks.
Parameter
MNLI
SST-2
CoLA
QNLI
QQP
Optimizer
AdamW
Warmup Ratio
0.06
LR Schedule
Linear
Batch Size
16
16
32
32
16
# Epochs
30
60
80
25
255
Learning Rate
5E-04
5E-04
4E-04
4E-04
5E-04
LoRA Config.
Tq
To
LoRA a Max Seq. Len.
16
512
C.4 DETAILED SETTINGS OF NLG.
For experiments on Llama3. 10ama3.1-88 1-88--8-Baa Base, including both baseline methods and ELLORA, we use one of the most commonly adopted hyperparameter configurations, unless stated otherwise in the main text. We use the AdamW (Loshchilov & Hutter, 2017) optimizer (3) = 0.9, B2 0.999, weight decay = 5e-4, batch size 64) with a cosine learning rate schedule (3% warmup). We use the BF16 mixed precision strategy (Kalamkar al. 2019) and apply LoRA to linear layers in attention modules. The peak learning rates are configured as follows: 5e-5 for our method and standard baselines (LoR MoRRA/ShareL.oRA. 5e-4 for A and LoRA-FA, and 5e-3 for VeRA to achieve meaningful results. Other details are shown Table
C.5 DETAILED SETTINGS OF IMAGE CLASSIFICATION TASKS.
For image classification tasks, following LoRA-Pro Wang al. 2024b). fine-tune CLT--
B/16 on StanfordCars (Krause et al. 2013), DTD (Krause et al. 2013). EuroSAT (Helber et al.
2019), GTSRB (Houben et al. 2013), RESISC45 (Cheng et al. 2017). SUN397 (Xiao al. 2010), and SVHN (Netzer et al. (011]) with accuracy used as the primary metric. During tine-tuning, only the linear layers of the visual backbone updated using FP32 precision. The classification result is derived from prompts such as "a photo (class)." Other details are shown Table
Table 9: Hyper-parameters used in experiments
Hyperparameter
-lama3.1-8B-1
CLIP-
LR
5c-5
1c-4
LR Decay
0.1
0
Warmup
0.3
0.3
Optimizer
Adam
Adam
Betas
0.9, 0.999
0.9, 0,999
Weight Decay
5c-4
0
Batch Size
64
64
C.6 TRAINING ENVIRONMENT.
All codes L and baseline methods are implemented in PyTorch. For natural language understanding and image classification tasks, we conduct our experiments on single A800 GPU per task, using the Huggingface Transformers framework for model and trainer implementation. For natural language generation tasks, conduct experiments 8 A800 GPUs, utilizing the Deep- Speed ZeRO2 (Rajbhandari et al. 2020) data parallel framework and FlashAttention-2 (Dao 2023 mechanism.
17

---

Published as a conference paper at ICLR 2026
Proxy Entropy Similarity
Per-Layer Proxy Entropy
1.00
0
Layer Number
0.99
4
0.98
Value
6
0.97
-4
0.96
-6
P_proj
10
0.95
-8
v_proj
10
Layer Number
Layer Number
Figure 6: Gradient-based proxy entropy analysis for CLIP-VIT-B/16 Cars. Left panels show
inter-layer information similarity, with high-similarity blocks highlighted in green. Right panels illustrate absolute proxy entropy values across layers, indicating significant variations.
D ADDITIONAL FIGURES AND VISUALIZATIONS
D.1 EXTENDED FIGURES OF GRADIENT-BASED PROXY ENTROPY ANALYSIS
Figure presents the gradient-baa based proxy entropy analysis for CLIP-VIT-B/16 on the Cars dataset, which further corroborates our two key findings: Local Similarity and Layer-wise Information Het- erogeneity.
D.2 UNIVERSALITY ACROSS DIVERSE LLM ARCHITECTURES
To rigorously validate the universality of our observations, extended the gradient-based proxy entropy analysis to a broader range of Large Language Models (LLMs) with varying scales and architectures GSM8K, benchmark of math reasoning. As shown in Figure 7 we visualized the layer-wise gradient similarity and entropy for Qwen3-8B, Phi4-mini, GPT-OSS-20R (a Mixture- of of-Experts model), and Gemma-9B-a utilizing gradients derived from the GSM8K training set.
The results align perfectly with our initial findings in Figure and the CLIP analysis above. Specif- ically:
Local Similarity:
The heatmaps (left panels for each model) consistently exhibit distinct diagonal block patterns across all models, regardless of whether they are Dense or MoE- based. This confirms that adjacent layers tend to share similar gradient information during math reasoning tasks.
Information Heterogeneity:
The bar charts (right panels) display significant fluctuations in proxy entropy values across layers, verifying that different layers contribute unevenly to the learning process on GSM8K.
These consistent patterns across diverse model families, scales, and structures strongly support the generalizability of the underlying principles driving ELLORA
D.3 EXTENDED VISUALIZATIONS OF VARIOUS SIMILARITIES
Figure presents comparative visualizations of alternative similarity metries. Panels (a), (c), and (d), depicting Cosine, L1, and L2 similarities respectively, reveal largely featureless or uniformly high similarity matrices. This lack of discernible local similarity patterns visually confirms their ineffectiveness for precise adaptive region partitioning, aligning with the observed performance degradation in Table While LI and L2 similarities exhibit structural resemblances due to their computational definitions, their distinet magnitudes can lead to varying impacts subsequent pro- cesses, particularly in rank allocation, thereby explaining their differing performances observed in Table 4 Panel (b) shows KI similarity, which does indicate local similarity regions. However, these
18

---

Published as a conference paper at ICLR 2026
(a) Qwen3 88
(b) Phi4-mini
(c) GPT-0SS-208
(6) Gemma-98-it
Figure 7: Gradient-based proxy entropy analysis on diverse LLMs on the GSM8K dataset: (a)
Qwen3-8B, (b) Phi4-mini, (c) GPT-OSS-20B (MoE), and (d) Gemma-9B-it. Consistent with our main observations, all models exhibit clear local similarity patterns (diagonal blocks in heatmaps) and significant layer-wise entropy heterogeneity (varying bar heights) when fine-tuned on math rea- soning tasks.
Cosine Similarity
KL Similarity
L1 Similarity
L2 Similarity
1.00
2.00
1.00
Number
Layer Number
0.90
Layer Number
0.80
12
0.50
28
18
10
0.50
20
19
0.00
%
0.60
0.70
6.75
0.96
Layer Number
Layer Number
Layer Number
Layer Number
(a) Cosine Similarity
(b) Similarity
(c) L1 Simi limilarity
(d) LZ Simi Similarity
Figure 8: Visual comparison of various similarity measures. Panels (a), (b), (c), and (d) display
Cosine, KL, L1. and L2 similarities, respectively.
regions are often excessively broad, hindering the identification of genuinely proximate and infor- mationally relevant layers for fine-grained adaptive sharing. This contrasts sharply with our proxy entropy similarity, which provides the necessary granularity for effective parameter sharing.
E DETAILED STEP-WISE IMPORTANCE ANALYSIS
To assess whether the importance of targeted modules varies significantly during fine-tuning, we conduct experiment using full fine-tuning of Llama-3.1-8B- MetaMathQA (Yu et al., (023). We compute the gradient entropy for each weight matrix every optimization steps to quantify module-wise importance dynamics. For optimization, we employ AdaLomo (Lv et al.) (2023), a layer-wise, state-free, SGD-like optimizer with adaptive learning rates, configured with a standard large learning rate (1e-4) and run for 1,024 steps.
(Grattationi al., 2024) on
As illustrated in Figure 12 the relative importance of targeted modules remains remarkably sta- ble throughout fine-tuning. Notably, across nearly all layers, WV modules dominate in importance, while WK and WQ modules consistently exhibit lower influence-2 finding aligned with prior work demonstrating that fine-tuning WV yields superior performance compared to fine-tuning other mod- ules (Hu al. 2021 Zhang al. 2023c
He et al. 2025a).
To further analyze layer-wise trends, we aggregate the entropy of targeted modules within each layer (Figure 13). The results reveal stable inter-layer importance patterns: middle layers (green/blue) maintain significantly higher importance, while upper layers are consistently least influential. This suggests consistent hierarchy in gradient-driven learning across depths.
19

---

Published as a conference paper at ICLR 2026
These observations directly justify the design rationale of our method. The persistent disparity in module importance- couuled with its temporal stability during of static rank allocation based initial gradient entropy. Specifically, assign higher ranks to modules exhibiting greater importance (e.g.. WQ matrices) as measured before the formal training process. Crucially, this allocation:
validates our approach
Preserves parameter efficiency: Rank assignments are fixed after initialization, requiring no dynamic shape adjustment.
Incurs zero computational overhead: Unlike adaptive methods, our strategy avoids runtime reconfiguration costs.
This design is empirically grounded in the observed hierarchy (WV/WO> WK/WQ) and layer- stability (middle > upper/lower layers), ensuring optimal resource distribution aligns with inherent module roles.
1. W Sat
Whyt 1p
Figure 9: Gradient entropy dynamics across training for attention matrices (WQ, WK, wV, WO) in layers 0-7 of Llama-3. 1-BB trained on MetaMathQA. Each subplot shows the evolution of absolute gradient entropy for one layer, revealing distinct patterns across different attention components.
Figure 10: Gradient entropy dynamics for attention matrices (WQ, WK. WV, WO) in layers 8-15, continuing from Figure 9
20

---

Published as a conference paper at ICLR 2026
Leyen
Leye
pاay 11 Man M.1g
Walffe
My M1y Maa
A
Wat's ppy Ahaee Momhanippalll
Yy
1/.
Figure 11: Gradient entropy dynamies for attention matrices in layers 16-23.
-2:25 Whiptsllys
-2.15
1000
Figure 12: Gradient entropy dynamics for attention matrices in the deepest layers 24-31.
F ANALYSIS ON GRADIENT CALCULATION
Our method initiates by performing an initial gradient computation step on subset of data to iden- tify crucial parameters. The efficiency and memory footprint of this process critical, and our implementation is designed for minimal overhead.
Memory-Effiiien Initialization: To mitigate memory consumption during this initial phase, we employ a layer-wise gradient computation strategy. Gradients for each layer are computed sequentially, followed by immediate aggregation via all-reduce operations and subsequent offloading to CPU memory by the local master process. This approach climi- nates the necessity of simultaneously storing full-model gradients and optimizer states on the GPU, ensuring that GPU memory usage remains at levels comparable to inference, primarily occupied by model weights and activations.
Absence of Optimizer States: A significant advantage of our initialization procedure is that it does not require the instantiation or storage of any optimizer states. This contrasts sharply with typical full fine-tuning processes which necessitate considerable memory for
21

---

Published as a conference paper at ICLR 2026
-15
Aupragge Procy perrory MetaMattin
-2.
Layer
Layer
Lnyer
3.5
Slep
Figure 13: Average gradient entropy across training steps for all 32 layers, grouped into four color- coded sets (layers 0-7: blue; 8-15: green; 16-23: red; 24-31: purple). Each line represents the mean entropy of the four attention weight matrices (WQ, WK, WV, WO) within a layer.
optimizer states like momentum and variance, leading substantial memory savings dur- ing this critical initial phase.
Our initialization process introduces no additional GPU peak memory beyond what is required for standard inference. The peak memory during this phase is primarily comprised of 16-bit model parameters, 16-bit activations, and gradients for single layer at time. This configuration results in significantly lower mem- ory consumption compared to full training, even when advanced optimization techniques like ZeRO-2 employed during the training phase. For further GPU memory reduc- tion, the entire model can be optionally offloaded to the CPU during this initialization step. While this involves storing 16-bit gradients of targeted modules the CPU, which incurs additional CPU memory overhead compared to a naive LoRA setup, this a manageable trade-off given that CPU memory is substantially more affordable and abundant than GPU memory.
GPU Peak Memory Footprint:
G VALIDATION OF THE GREEDY STRATEGY VIA GLOBAL OPTIMIZATION
To rigorously validate the effectiveness of our proposed greedy segmentation strategy, we imple- mented global optimization baseline based on Dynamic Programming (DP), This comparison aims determine whether the local decision- making process of the greedy approach compromises global optimality or downstream performance.
G.1 GLOBAL OPTIMIZATION FORMULATION
We designed global optimization algorithm that operates on the same Relative Mutual Information (RMI) matrix used by our greedy method. The formulation consists of three steps:
Segment Cost Calculation: For every possible contiguous interval [ we compute cost C)i,j), defined as the negative mean similarity of layers within that interval based on the RMI matrix.
Global Objective: The total objective function L is defined as the sum of segment costs plus a regularization term to control granularity:
K
C(ix.ie + K,
(8)
22

---

Published as a conference paper at ICLR 2026
where K represents the number of intervals and À is penalty coefficient for over-
segmentation.
DP Search: We employ a Dynamic Programming algorithm to strictly minimize L, back- tracking identify the globally optimal partition S* for each module type.
G.2 EXPERIMENTAL RESULTS AND ANALYSIS
We conducted a direct comparison between the globally optimal solution (DP) and our greedy ap- proach GLUE (RoBERTa) and CV (CLIP-ViT) tasks using identical training setups. The com- parative results are summarized in Table I0
Table 10: Comparison between the Global Optimal (DP) strategy and Greedy approach. Time denotes the search latency during the grouping phase. Results are averaged across datasets.
Task
Metric
Global Optimal (DP)
Greedy (Ours)
Search Time
0.6 ms
0.2 ms
CV (CLIP-VIT)
Parameters
0.66 M
0.66 M
Performance
89.92
89.81
Search Time
0.6 ms
0.2 ms
GLUE (RoBERTa)
Parameters
0.15
0.16 M
Performance
85.43
85.57
Performance Parity. As evidenced in Table 10, the global optimal grouping does not yield sta- tistically significant improvements a marginal gain on CV (+0.11%) but a slight drop on GLUE (-0.14%). This suggests that the lo- cal similarity structure of Large Language Models (LLMs) is distinct enough that a greedy strategy effectively captures the necessary boundaries without requiring global look-ahead.
greedy approach. Specifically, the DP method shows
Efficiency and Robustness. The greedy approach demonstrates superior efficiency, being approx- imately 3x faster in search time (0.2 ms vs. 0.6 ms). More critically, the DP approach introduces an additional hyperparameter, the penalty coefficient A, which requires tuning. In contrast, our greedy method relies on self-adaptive threshold derived directly from RMI statistics.
Given that the greedy approach offers comparable performance with lower computational complex- ity and higher autonomy (free from hyperparameter tuning), it retained as the core mechanism in LoLAR.
H ROBUSTNESS ANALYSIS OF RMI-BASED THRESHOLDING
To empirically validate the robustness our method against potential fluctuations the Relative Mutual Information (RMI) matrix- and by extension, the derived thresholds 7-we conducted noise perturbation analysis on the image classification task.
H.1 PERTURBATION METHODOLOGY
Since the adaptive thresholds computed directly from the RMI statistics, introducing noise the RMI matrix simulates scenarios where the layer-wise wise similarity estimation might contain errors or fluctuations. We injected Gaussian noise into the computed RMI matrix prior to the partitioning phase:
RMI RMI + where (O,
where o controls the magnitude of the perturbation. We evaluated the downstream performance across spectrum of noise levels, ranging from weak (a 0.001) to large (a 0.05).
23

---

Published as a conference paper at ICLR 2026
H.2 RESULTS AND DISCUSSION
The impact of RMI perturbation on model parameters and classification accuracy is summarized in Table [1
Table 11: Impact of Gaussian noise perturbation on the RMI matrix. The experiment is conducted on image classification tasks. a denotes the standard deviation of the added noise.
Noise Level (σ)
Intensity
Params
Accuracy (%)
0.001
Weak
0.67 M
89.81
0.005
Weak
0.65 M
89.87
0.01
Medium
0.63 M
89.85
0.05
Large
0.55 M
89.30
Stability under Minor Perturbations. As shown in Table 11 under small moderate pertur- bations (a E [0.001, 0.01)). the model performance remains highly stable, fluctuating by less than 0.1%. This indicates that our greedy grouping strategy not brittle; does require "perfect" RMI estimation to function effectively and maintains high performance even with noisy structural signals.
Significance of RMI Structure. Conversely, when significantly larger noise (a = 0.05) in- troduced, disrupting the inherent similarity structure of the RMI matrix, we observe noticeable degradation in performance (dropping to 89.30%). This performance drop confirms that the orig- inal RMI matrix captures essential structural information necessary for near-optimal partitioning. Randomizing or heavily distorting this structure prevents the algorithm from identifying the correct layer boundaries, thereby justifying the validity of our RMI-based design.
I ROBUSTNESS TO SAMPLING VARIABILITY
To explicitly address the stability of our method with respect to the specific choice of data sam- ples, we evaluated the consistency of the derived adaptive configurations coturrationnsssccfcccllllyl the interval partitioning and rank allocation-ane across different random seeds. This analysis ensures that our entropy-based metries are robust the randomness inherent in subset sampling.
1.1 CONSISTENCY ANALYSIS
We re-computed all entropy-based quantities, including the Relative Mutual Information (RMI), the resulting interval boundaries, and the final adaptive rank allocation, using three distinct random seeds (0, 18, and 42). Each seed corresponds to independently sampled mini-subset of data. We then trained the model using these configurations and recorded the final downstream performance.
Table 12 illustrates the results the EuroSAT dataset for the k-proj module of the -LI--ViT-B/16 model. We report the number generated intervals, the specific rank assigned to each interval, and the final classification accuracy.
Structural Stability. As demonstrated Table 12 the structural decisions made by our algorithm are remarkably consistent. The interval partitioning structure (represented by the number of layers in each interval: [2. 1, 3, 1, 3, 2]) and the corresponding rank allocations are identical across all three seeds. This indicates that the maddient-based entropy metrics capture intrinsic properties of the model layers that are invariant to the specific random subset of data used for estimation.
Performance Stability. Consequently, the final downstream performance exhibits minimal vari- ance, fluctuating within a narrow margin (approximately +0.2%). We observed similar behavior across other modules (e.g., Attention and FFN blocks) and datasets. These findings confirm that E'LoRA yields highly stable adaptive decisions that are robust not only to the size of the subset but also the specific choice of samples.
24

---

Published as a conference paper at ICLR 2026
Table 12: Robustness of interval partitioning and rank allocation across different random seeds on EuroSAT (CLIP-ViT--H) ViT-B/16, k- k-proj module). The subset choice varies with the seed, yet the structural decisions remain identical.
Seed
Intervals
Rank Allocation (Interval ID: Rank)
Ace ( (%)
[2, 1, 3, 3, [2, I, 3, 1, 3,
{0: 9, I: 9. 2: 7: 8, 4: 7, 8)
98.56
18
[0: 9, 1: 9, 2: 7, 3:
8,
4: 7, 8}
98.28
42
[2, 1, 3,
{0: 9, I: 9, 2: 7, 3: 8, 4: 7, 5: 8)
98.52
J EXTENDED SRRAMETER-MATCHEEE COMPARISONS
In the main text, we demonstrated the superior parameter efficiency of E*LoRA on the GSM-8K task. To rigorously validate the generalizability of these findings, we expanded the parameter- matched comparisons cover Computer Vision (CV) and Natural Language Understanding (NLU) modalities. Specifically, we compared E LoRA against standard LoRA on the CV task (using CLIP- ViT-B/16) and the NLU task (using RoBERTa on GLUE), strictly controlling the trainable parameter counts.
J.1 EXPERIMENTAL. SETUP
We aligned the parameter budgets by adjusting the rank settings, Due to the efficient parameter- sharing mechanism in LoLoRA, our method can utilize higher base rank while maintaining a low parameter footprint. Consequently, we compared:
*
Low Budget: Standard LoRA (Rank 4) vs. E'LORA (Base Rank 8).
Medium Budget: Standard LoRA (Rank 8) vs. E'LoRA (Base Rank 16).
J.2 RESULTS AND ANALYSIS
The comparative results are summarized in Table 13 At identical parameter budgets, EoLoRA consistently outperforms standard LoRA across both domains.
Table 13: Performance comparison between Vanilla LoRA and E=LoRA under matched parameter counts. ELLoRA achieves higher performance using the same or fewer parameters.
CV (CLIP-VIT)
NLU (RoBERTa)
Method
Config
Params
Ace ( (%)
Params
Score
LoRA
Rank 4
0.66 M
88.03
0.15 M
85.20
LoRA
Rank 8
0.66 M
89.81
0.16 M
85.57
LoRA
Rank 8
1.31 M
89.08
0.30 M
85,87
LoOA
Rank 16
1.33 M
90.45
0.32 M
86.22
Consistent Superiority. As shown in Table 13 EoLoRA significantly surpasses the parameter- matched LoRA baselines. For instance, in the CV task with budget of ~0.66M parameters, E'LoRA exceeds LoRA by +1.78%. Similarly, in the NLU task, E'LoRA consistently achieves higher GLUE scores at both budget levels. This confirms that framework's 's stemming from adaptive interval partitioning and rank allocation are robust and meaningful even when strictly controlling for model size.
25

---

Published as a conference paper at ICLR 2026
K ADDITIONAL EXPERIMENTS ON COMMONSENSE REASONING
To further verify the robustness our method on diverse reasoning tasks beyond GLUE and Math benchmarks, extended evaluation to the CommonSenseQA benchmark using LLaMA-3.1- 8B. This suite encompasses multiple challenging subtasks, including ARC-Challenge (ARC-C), ARC-Easy (ARC-E), BoolQ, OpenBookQA (OBQA), PIQA, and SIQA. These tasks evaluate the model's s instruction-style reasoning capabilities without relying on long-context or code-specific adaptations.
The results, summarized in Table 14 demonstrate that E*LORA maintains strong efficiency- performance balance even on complex commonsense reasoning tasks. Specifically, E*LORA achieves a slight overall improvement (+0.09 points in average accuracy) while utilizing only 48% of the trainable parameters required by standard LoRA (3.26M vs. 6.82M).
Notably, our method exhibits distinct performance characteristics across subtasks. It excels in tasks requiring nuanced reasoning, achieving significant gains on BoolQ (+5. 11 points) and SIQA (+3.05 points). While there are minor performance dips in other areas (e.g., ARC-C), these are effectively offset by the gains in high-complexity tasks. This behavior aligns with our entropy-guided allocation strategy, which prioritizes parameter budget for high-information layers. These findings further reinforce the generality of ELLORA across different model families (LLaMA-3.1) 1),, scales, and task types.
Table 14: Performance comparison CommonSenseeQA using LLaMA-3.1-80 E*LORA achieves comparable or better average performance with significantly fewer parameters.
Method
#Paarms
Avg.
ARC-C
ARC-E
BoolQ
OBQA
PIQA
SIQA
LoRA 6.82M
82.18
82.50
93.09
64.95
86.40
88.46
77.67
Ours LoRA)
3.26M
82.27
81.14
91.45
70.06
84.20
86.07
80.72
L EXPERIMENTS ON LARGER AND DIVERSE MODELS.
To further verify the robustness, scalability, and generality of our method, we conducted experi- ments on a diverse set large language models (L.MMs) using math benchmarks. Specifically, we evaluated ELLORA A against the standard LoRA on Llama-2-13B (Dense), Qwen3-8B (Dense), and GPT-OSS-22 20B (a Sfttuureoo--xperttt MoE, model). As shown Table 15 different scales, families, and architectures.
these models represent
The results consistently demonstrate ELLORA's superior efficiency and performance. On Llama-2- 13B, LoLORA achieves higher accuracy (+0.88) with approximately half the parameters of LoRA. On Qwen3-8B, it matches LoRA's accuracy using only 53% of the parameters. Most impressively, on the complex GPT-OSS-200 (MoE) model, LoLA not only reduces parameters by 56% but also surpasses LoRA's accuracy by a significant margin of 1.14 points. These findings strongly validate that our dual-adaptive framework is generalizable approach, effectively delivering better performance-efficinncy trade-off across various LLM designs. Further analysis, including layer-wise gradient similarity and entropy visualizations for these models, is provided in Appendix L which confirms that the principles motivating our method are intrinsic properties of LLMs.
Table 15: Performance of fine-tuning diverse large models on math benchmarks. E con- sistently achieves better or equal performance with significantly fewer parameters across different model scales, families, and architectures (Dense vs. MoE).
Model (Architecture)
Method
#Params
Accuracy (%)
LoRA
13.11M
52.94
Llama-2-13B (Dense)
Ours (E^LORA)
6.55M
53.82
LoRA
7.67M
82.49
Qwen3-8B (Dense)
Ours (ERLORA)
4.06M
82.49
LoRA
3.98M
82.03
GPT-OSS-200 (MoE)
Ours (E LORA)
1.75M
83.17
26

---

Published as a conference paper at ICLR 2026
M DETAILED ALGORITHM
The detailed processes of Local Similarity-based Sharing(LSS) are shown in Algorithm the
detailed processes of Heterogeneity-base Rank Allocation(HRA) are shown in Algorithm
Algorithm 2: Local Similarity-based Sharing
Notations:
Let R[j][] denote RMI between G and G H. be gradient entropies, L = [L1 .... LN be layer indices.
Input: Model gradients {G .... G(N))
Proxy entropies H = {H1, Hx)
Number of layers N
Output: Sharing intervals S = {[5,, 6n k
for i do
for j E C do
if j + then
R(L <- ComputeRMI (()))
+ 0 // Initialize the interval set 1// Pointer to current layer while p & N do
///// Start index of interval T T Exys R[s][k] // Threshold
cts// Initialize end layer while < N and R[e++ 1][s] 2 do
L Lete+1// Expand interval
S ((s, // Update the interval set p // Move to next layer
return S
Algorithm 3: Hetcrogencity-basedd Rank Allocation
Notations:
Let M = J be the model. I {[ e be sharing intervals, Teansila be the base
LoRA rank.
Input: Model M with N layers
Base rank Fvonilla
Sharing intervals
Output: Allocated ranks R {n*)
R + @ // Initialize the rank set
Rooal +- X Tvanille // Total rank budget
H < 0 // Interval entropy container for each interval [sk: ex] I do
H max H(G")) // Max entropy s4 S1See
H HU (H&
H Hum + -H // Total heterogeneity measure
for each interval [sk, ea] E I do
ag - H H Hum // Allocation factor
F round (aq X Rooal // Bank allocation
R RU [
return R
N LIMITATIONS AND FUTURE WORK
Our work has demonstrated that LoRARA consistently achieves an excellent balance efficiency and effectiveness across diverse tasks, modalities, and various sizes of pre-trained models, often
27

---

Published as a conference paper at ICLR 2026
surpassing existing methods. Despite these promising results, several avenues remain for further exploration.
First, our current experimental validation does not include fine-tuning significantly larger models, such as Llama3-70B.
Second, while ELLORA focuses LoRA, the proposed entropy- based analysis is orthogonal the specific adaptation mechanism. The insights regarding local similarity and layer-wise information heterogeneity rely solely pre-trained weights and gradients. Therefore, future work could explore applying the E*LoRA framework to other PEFT methods, such as adaptive budget allocation for Adapters token-wise optimization in Prefix-tuning.
Third, while ELLORA has been successfully validated across both natural language and vision modalities, its generalizability to a broader range of model types and tasks warrants further inves- tigation. Future work will explore the application of L*LORA to emerging domains such as visual language models and visual question answering.
O LLM USAGE DISCLOSURE
In the preparation of this paper, we used the large language model assist with language refinement, including improving and checking potential grammatical issues as well as enhancing clarity and readability. The model was not used to generate scientific content, ideas, experiments, or analyses. The authors take full responsibility for the accuracy and integrity of the paper's content.
28


Reviews:
[{"review_id": "GkNbfYqFEy", "reviewer": "Reviewer_VLBt", "summary": "This paper proposes a novel PEFT method $E^2LoRA$ to decide parameter-sharing groups across layers and allocates LoRA ranks according to a proxy-entropy measure. The approach first estimates layer-wise information and relative mutual information on a small batch of data, then proportionally distributes a global rank budget based on the estimated importance of each group. Experiments across multiple tasks demonstrate competitive or improved performance with reduced trainable parameters.", "questions": "- What is the exact mini-subset size used to compute gradients for proxy-entropy estimation? Reporting this and adding a small ablation would strengthen the method's practicality.\n- In Line 228, the interval grouping uses a greedy approach. Computing the optimal intervals should be computationally cheap; have you compared greedy vs. optimal?\n- Is there any theoretical justification for using standard deviation as proxy entropy? A simple counterexample exists where a low-rank gradient matrix has higher variance than a higher-rank one, yet this metric may suggest the opposite.\n- In Table 2, the parameter counts for $E^2LoRA$ on GSM8K and HumanEval appear identical. Please verify. Similarly, Table 4 and Table 5 report seemingly inconsistent parameter numbers (3.66M, 3.61M).", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- This paper introduces a simple yet effective framework to jointly decide rank allocation and parameter sharing. The method is plug-and-play and can be applied to both vanilla LoRA and other LoRA variants, showing practical usefulness.\n- This paper provides extensive experiments across multiple modalities and model scales, together with ablations that support the reported stability and generalization.", "weaknesses": "The comparison baselines omit several recent rank-allocation and sharing approaches (e.g., [1-5]), and the baselines used across tasks are not fully consistent, some methods appear only in selected experiments. This makes it a little difficult to characterize the contribution and relative advantage of the $E^2LoRA$.\n\n> [1] ALoRA: Allocating Low-Rank Adaptation for Fine-tuning Large Language Models\n>\n> [2] DyLoRA: Parameter-Efficient Tuning of Pretrained Models using Dynamic Search-Free Low Rank Adaptation\n>\n> [3] IncreLoRA: Incremental Parameter Allocation Method for Parameter-Efficient Fine-tuning\n>\n> [4] RA-LoRA: Rank-Adaptive Parameter-Efficient Fine-Tuning for Accurate 2-bit Quantized Large Language Models\n>\n> [5] HydraLoRA: An Asymmetric LoRA Architecture for Efficient Fine-Tuning", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jgib4TYatT", "reviewer": "Reviewer_wUSM", "summary": "This paper proposes E2LoRA, a dual-adaptive Low-Rank Adaptation framework that improves both efficiency and effectiveness via entropy-guided adaptive sharing and rank allocation. \n\nUsing a gradient-based proxy entropy analysis, the authors identify two properties in pretrained models: Local Similarity and Layer-wise Information Heterogeneity. The authors designed mechanisms that dynamically group similar layers and assign ranks proportionally to each interval’s information entropy.", "questions": "1. Will all models share those properties mentioned in the introduction (e.g., Larger Models (>13B), MoE Models, different series of models(Mistral, QWen, etc))?\n2. Can the proposed adaptive rank allocation be combined with other LoRA variants, such as AdaLoRA, DoRA, and HiRA, and would they be complementary?\n3. Could the proxy entropy approach be extended to other PEFT frameworks beyond LoRA, such as prefix-tuning or adapters?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Introduces a novel entropy-based view of LoRA sharing and rank allocation.\n2.  Well-motivated and empirically supported through cross-modal experiments.\n3. Simple, modular design compatible with existing LoRA and ShareLoRA.", "weaknesses": "1. The theoretical justification of the proposed proxy entropy metric remains largely heuristic and lacks a rigorous connection to established information-theoretic principles.\n2. The empirical analysis across different model architectures and scales is limited, leaving the generality of the proposed insights insufficiently validated.\n3. Certain methodological details, especially regarding entropy computation and notation, require clearer explanations to improve reproducibility.\n4. The performance improvements over strong efficiency-oriented baselines such as LoRA-FA and VeRA are relatively modest in the RoBERTa base and CLIP ViT experiments, which weakens the practical significance of the gains.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "I9EF8aM5I7", "reviewer": "Reviewer_LoWT", "summary": "This paper proposes E²LoRA, a parameter-efficient fine-tuning method based on LoRA. It first analyzes pre-trained models using a gradient-based proxy entropy, identifying \"Local Similarity\" (adjacent layers share information) and \"Layer-wise Information Heterogeneity\" (layers contain different amounts of information). Based on these insights, E²LoRA introduces a dual-adaptive framework involving: 1) Adaptive Sharing Interval Partitioning, grouping similar adjacent layers to share LoRA adapters based on inter-layer entropy similarity , and 2) Adaptive Rank Allocation, assigning LoRA ranks to these shared intervals based on the absolute proxy entropy of the layers within them. The goal is to significantly reduce trainable parameters (claiming approx. 50% reduction) while maintaining or improving performance compared to baseline LoRA and naive sharing methods. Experiments are conducted across NLU, NLG, and image classification tasks.", "questions": "1. Could the authors provide details on the computational overhead (time and memory) introduced by the initial gradient computation, entropy/RMI calculation, and partitioning/allocation steps? How does this compare to the overall fine-tuning time?\n\n2. How robust is the proxy entropy measure and the resulting partitioning/ranking to the choice and size of the initial data subset used for gradient calculation? Have the authors experimented with different subsets?\n\n3. Could the authors provide more extensive comparisons where baseline LoRA (and potentially other methods like AdaLoRA, DoRA) are configured to have the exact same number of trainable parameters as E²LoRA across the main benchmark tables (Tables 1, 2, 3)? This would allow for a fairer assessment of the proposed adaptive strategy's effectiveness versus simply reducing rank.\n\n4. How sensitive is the adaptive partitioning algorithm (Algorithm 2 ) to noise or minor variations in the RMI matrix? Does the greedy approach guarantee a near-optimal partitioning?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The proposed dual-adaptive mechanism, which adjusts both the sharing scope (intervals) and the capacity (rank) based on properties derived from the model itself, is conceptually appealing.\n\n\n2. The paper generally presents its motivation and method clearly. Figure 2 provides a good overview of the framework.\n\n3. The initial analysis using gradient-based proxy entropy to identify Local Similarity and Layer-wise Information Heterogeneity  provides a potentially valuable and principled motivation for designing adaptive parameter sharing strategies, moving beyond naive global or fixed-block sharing. This perspective is somewhat original.", "weaknesses": "1. Complexity vs. Benefit: The proposed E²LoRA framework adds significant complexity compared to standard LoRA or simple sharing schemes. It requires an initial phase of gradient computation on a data subset , proxy entropy calculation , Relative Mutual Information (RMI) matrix computation , adaptive thresholding and partitioning , and finally rank allocation based on interval entropy. While the paper claims parameter reduction, the reported performance gains over baselines are marginal (sometimes slightly lower or equivalent on average). This raises questions about whether the added complexity is justified by the benefits\n\n2. Proxy Entropy Robustness: The method heavily relies on gradient-based proxy entropy calculated on a \"mini-subset of the training dataset\". The stability and reliability of this proxy measure are questionable. How sensitive is it to the choice and size of this subset? Does this entropy accurately capture the necessary information across different tasks, modalities, and model states? The paper lacks analysis on the robustness of this core component. Basing adaptive decisions on potentially unstable proxies undermines the method's soundness.\n\n3. Insufficient Comparisons: The comparisons, especially regarding efficiency, could be stronger. For instance, while E²LoRA uses fewer parameters than standard LoRA (e.g., 0.16M vs 0.30M in Table 1 ), the performance difference isn't compelling. A more rigorous comparison would involve evaluating standard LoRA with a reduced rank to match E²LoRA's parameter count (or vice-versa, as done briefly in Table 5  but only for one model/task) across all experiments to isolate the benefit of the adaptive strategy itself versus just using fewer parameters. Comparisons against methods like VeRA, which achieve even lower parameter counts, also show mixed results, with E²LoRA sometimes underperforming despite using more parameters (though comparisons here are complicated by different base ranks).\n\n4. Hyperparameter Sensitivity: The adaptive partitioning uses layer-specific thresholds $\\tau_m$ derived from average RMI 20, and the rank allocation depends on a base rank $r_0$ 21and the total budget calculation22. The sensitivity to these aspects (especially $r_0$ and the implicit assumptions in the thresholding) is not adequately explored.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JWr7hm0bur", "reviewer": "Reviewer_CKkS", "summary": "LoRA and other PEFT methods are efficient, but pushing further with cross-layer sharing or lower ranks often hurts representational capacity and degrades performance. E2LoRA addresses this by analyzing gradients with a proxy-entropy lens and drawing two key insights: local similarity that adjacent layers tend to carry more similar information and are thus suitable for blockwise sharing; and layer-wise heterogeneity, different layers (or intervals) carry very different amounts of information and should receive differentiated capacity (rank). Building on this, E2LoRA proposes a dual-adaptive framework: LSS (Local Similarity Sharing) uses relative mutual information with adaptive thresholds to greedily partition adjacent, information-similar layers into shared intervals and restrict sharing within each interval; HRA (Heterogeneity-aware Rank Allocation) then uses the maximum proxy entropy within each interval to allocate ranks under a fixed overall budget, granting higher ranks to more informative intervals.", "questions": "See weakness.", "limitations": "", "rating": 2, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The “local similarity + layer-wise heterogeneity” perspective operationalized via gradient proxy-entropy and relative mutual information offers a clear mechanism and transfers across backbones and tasks.\n\n2. The dual-adaptive scheme (partition + rank allocation) is nearly plug-and-play with existing LoRA/ShareLoRA pipelines, reducing trainable parameters under a fixed budget while largely preserving (or slightly improving) performance.", "weaknesses": "1. While the paper targets both efficiency and effectiveness, the reported gains over baselines are not striking. Many tables show modest uplifts at comparable or lower trainable-parameter budgets.\n\n2. Several LoRA-family methods in 2025 already achieve very low trainable-parameter regimes (e.g., COLM-2025 LoRI). The latest baseline here appears to be DoRA (Feb 2024). Given the pace of PEFT, comparisons should include newer 2024–2025 lines (ICLR/ACL/ICML 2025) to establish contemporaneous relevance.\n\n3. Related work coverage is incomplete for the entropy/gradient/shared-layer thread. The method is positioned as entropy/gradient-guided sharing, yet closely related recent works are missing or under-discussed (e.g., BSLoRA, ICML 2025; LoRA-GA, NeurIPS 2024; LoRA-One, ICML 2025). A more thorough placement of these lines is needed (the 3 papers I mentioned are just an example; the related work is too weak ). \n\n4. RoBERTa-base and ViT-B are relatively small by today’s standards. Although Llama-3.1-8B is included, a stronger LLM sweep (more family/bases, more diverse instruction/reasoning/long-context/code tasks) is important to substantiate generality.\n\n5. Stability/robustness of the entropy-RMI partitioning is under-analyzed. \n\n\n\nIn summary, the paper targets peft via entropy/gradient-guided layer sharing and adaptive rank allocation. The direction is reasonable, the current presentation feels under-polished: the motivation is not sharply articulated, the method reads incremental relative to recent PEFT progress, and the writing/positioning leaves key claims under-substantiated. The paper doesn’t provide a new conceptual lens on PEFT; it reads more as an engineering tweak than a step that changes how we understand parameter sharing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)