ECMamba: Consolidating Selective State Space Model with Retinex Guidance for Efficient Multiple Exposure Correction
Authors: Wei Dong
Han Zhou
Yulun Zhang
Xiaohong Liu
Jun Chen
Venue: NeurIPS 2024 poster (2024)
Keywords: Retinex Theory
Mamba
Exposure Correction
Primary area: deep_learning_architectures

Abstract: Exposure Correction (EC) aims to recover proper exposure conditions for images captured under over-exposure or under-exposure scenarios. While existing deep learning models have shown promising results, few have fully embedded Retinex theory into their architecture, highlighting a gap in current methodologies. Additionally, the balance between high performance and efficiency remains an under-explored problem for exposure correction task. Inspired by Mamba which demonstrates powerful and highly efficient sequence modeling, we introduce a novel framework based on \textbf{Mamba} for \textbf{E}xposure \textbf{C}orrection (\textbf{ECMamba}) with dual pathways, each dedicated to the restoration of reflectance and illumination map, respectively. Specifically, we firstly derive the Retinex theory and we train a Retinex estimator capable of mapping inputs into two intermediary spaces, each approximating the target reflectance and illumination map, respectively. This setup facilitates the refined restoration process of the subsequent \textbf{E}xposure \textbf{C}orrection \textbf{M}amba \textbf{M}odule (\textbf{ECMM}). Moreover, we develop a novel \textbf{2D S}elective \textbf{S}tate-space layer guided by \textbf{Retinex} information (\textbf{Retinex-SS2D}) as the core operator of \textbf{ECMM}. This architecture incorporates an innovative 2D scanning strategy based on deformable feature aggregation, thereby enhancing both efficiency and effectiveness. Extensive experiment results and comprehensive ablation studies demonstrate the outstanding performance and the importance of each component of our proposed ECMamba. Code is available at \url{https://github.com/LowlevelAI/ECMamba}.

ECMamba: Consolidating Selective State Space Model with Retinex Guidance for Efficient Multiple Exposure Correction
Wei Dong".", Han Zhou':", Yulun Zhang", Xiaohong Liu Jun Chen²
McMaster University
SSanghai Jiao Tong University
[dongw22, zhouh 15, chenjun @ memaster.caa Eqqual Contribution
@mmliall.commm xiaohongliussjju.duu.. *Corespponding Author
Abstract
Exposure Correction (EC) aims to recover proper exposure conditions for im- ages captured under over-exposure or under-exposure scenarios. While existing deep learning models have shown promising results, few have fully embedded Retinex theory into their architecture, highlighting gap in current methodologies. Additionally, the balance between high performance and efficiency remains an under-explored problem for exposure correction task. Inspired by Mamba which demonstrates powerful and highly efficient sequence modeling. we introduce a novel framework based on Mamba for Exposure Correction (ECMamba) with dual pathways, each dedicated to the restoration of reflectance and illumination map, respectively. Specifically,
firstly derive the Retinex theory and we train Retinex estimator capable of mapping inputs into two intermediary spaces, each approximating the target reflectance and illumination map, respectively. This setup facilitates the refined restoration process of the subsequent Exposure Correction Mamba Module (ECMM). Moreover, develop novel 2D Selective State- space layer guided by Retinex information (Retinex-SS2D) as the core operator ECMM. This architecture incorporates an innovative 2D scanning strategy based on deformable feature aggregation, thereby enhancing both efficiency and effective- ness. Extensive experiment results and comprehensive ablation studies demonstrate the outstanding performance and the importance of each component of our proposed ECMamba. Code is available at https: //gitt:///gttuuu gigguu.......... com http////o11eve1AA//O0alall..
Introduction
Images captured under over-exposure and under-exposure conditions suffer from various degradations, including reduced contrast, color distortion, and information loss in extremely dark or bright regions. The objective of exposure correction is enhance the visibility, contrast, and structural details for images with various illumination conditions, which is pivotal for improving the performance of plethora of downstream applications such object detection, tracking, and segmentation systems [10, 39, 31] []]]]] in scenarios with improper exposure.
Similar to other image restoration tasks [3, 2. 32, 25, 26, 23, 42, 12], many deep learning models [38, 6, 34, 44] have been proposed for under-cxposed image enhancement and demonstrate commendable results. However, our preliminary experiments indicate that these methods generally perform poorly in multi-exposure correction. This inadequacy stems from the distinct mapping flow between Over- Exposed (OE) and Normal-Exposee (NE) images compared to that between
(UE) and NE images. Recently, some promising works [20, 21, 1, 19] have introduced several interesting deep learning networks to learn consistent exposure representations for multi-exposure correction. However, despite its widespread adoption and outstanding performance in under-exposure correction,
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

UE/OE Ioput
R'
Raut
Laut
Image
(a)
(b)
Figure (a) T-SNE [7] visualization of distributions of modulated reflectance (R)), restored re- flectance (Rout) and the final output (Tout) for Under-Exposed (UE) and Over-Exposed (OE) images. Compared the imput data, modulated reflectance (R') demonstrates closer approximation of Normal- Exposed (NE) images. Besides, compared to the restored reflectance (Rout). our final output (Loa) are better aligned with NE data. (b) Visual result of R', Rout and Lout produced by our method. From column 2-4, we observe noticeable improvement on color preservation and structure recovery, which demonstrates the importance our introduced two-branch Retinex -ased pipeline and the effectiveness of our proposed ECMamba network.
Retinex theory [22] has not yet been deeply integrated into deep learning models multi multi-exposure correction. As deep learning models often struggle to distinguish between illumination information and the intrinsic reflectance properties of objects in images, simply adopting deep learning models to address such difficult problem usually obtains sub-optimal results and the incorporation of Retinex theory offers a physically justified way to decompose the illumination and reflectance within deep learning models. Moreover, current state-of-the-art (SOTA) performance is achieved through introducing specific designs (exposure normalization [19] or exposure regularization term [21]) 21))r to existing networks. However, these methods present limited generalization, it is essential to develop stronger foundational model with good generalizable ability. Additionally, many methods face a trade-off between performance and efficiency, particularly those based transformers.
To address these issues, we introduce a novel two-branch Exposure Correction network (ECMamba) based on standard Mamba architecture, which has demonstrated impressive sequence modeling ability with high efficiency [14]. Specifically, we derive the Retinex theory and develop a Retinex estimator to transform the input into two intermediary spaces, each approximating the target reflectance and illumination map, respectively. As shown in Fig. la, compared to the input distribution, the generated intermediary space (R') shows closer approximation of target distribution, thus enabling subsequent network to execute the fine-gre mne-grained restoration. Moreover, the visually compelling results in Fig. Ib column 4 demonstrate that our proposed two-branch framework offers precise estimation and improved performance than simply optimizing the reflectance. Furthermore, we develop novel 2D Selective State-space layer guided by Retinex information (Retinex-SS2D) the core operator of our ECMamba. Different from other scan strategies (i.e., cross-scan mechanism [27]) which considers the scanning of 2D data to ID sequence a "direction-sensitive problem, we regard this issue as "feature-sensitie Deformable Feature Aggregation (DFA) guided by Retinex information. Then based on the activation response map derived from DFA, we develop a Feature-A Awaee ware 2D Selective Scanning (FA-SS2D) mechanism flatten the aggregated feature into 1D sequence, which subsequently fed into the standard Selective State Space process (S6) to capture long-range dependencies.
problem. Therefore, we first perform feature fusion and then introduce a
The contributions of this work summarized follows:
We present a novel dual-branch framework that fully embeds Retinex theory for exposure correction and provide detailed explanation for its significance.

---

By analyzing the operating mechanism of Selective State Space Model, we regard the scanning of vision data is "feature-sensitive" issue and we propose an efficient Retinex-SS2D layer with Retinex-guiin guided Feature-Aware 2D Selective Scanning Mechanism.
Extensive experiments and ablations demonstrate the impressive performance of proposed method.
2
Related Works
Learning based Expooi-E Correction Multi-exposuree correction is challenging task due the opposite optimization flows of under-exposure and over-exposure correction. MSEC [1] introduces Laplacian pyramid architecture restore lightness and structures. Later, several normalization and regularization methods [4, 21, 19, 20] are proposed for exposure correction. For example, the exposure normalization [19] proposed for exposure compensation, ECLNet [21] introduces ppposure-consistency representations with bilateral activation mechanism, and FECNet [20] opts to correct illumination the frequency domain. Different from previous methods, we aim to develop a Retinex-based network, where Retinex guidance is utilized to modulating the optimization flows of under-exposure and over-exposure correction.
State Space Model (SSMs) Due to its impressive modeling capability for long-range dependencies and its promising efficiency, State Space Models (SSMs) and recent proposed Structured SSace Sequence model (S4) [15] has attracted great interests among researchers. Based on S4, several models and strategies are introduced to improve the efficiency and boost the capability, among which Mamba [14] introduces an inpur-dependent SSM with selective mechanism and achieves superior performance than Transformers for natural language processing. Moreover, some pioncering works have applied Mamba vision task such as image segmentation [27], classification [47] and even restoration [17, 29]. We are the first to address exposure correction problem based on Mamba, and we innovatively introduce an efficient feature-aware scanning strategy in this work.
3 Preliminaries
State Space Model (S4) State Space Model (S4) is introduced by combining recurrent neural net- works (RNNs), convolutional neural networks (CNNs), and classical state space models. Specifically, for a sequence of length x E R the input at any time step x(t) E R can be mapped to an output y(t) E I through the following state space modeling:
ht( = Ah(t) + Br((t),
yt) = Ch(t).
where h(t)) Nx1 represents latent state and denotes the dimension scaling ratio in latent state. A E NNN, B Nx1, and C E I are state transition matrix, control matrix, and output matrix, respectively. Mathematically, the differential equation in Eq. has an equivalent integral equation and it can be solved using numerical computation. In order to integrating state space modeling into deep learning models, the discretization required to convert continues-time model discrete-time system by introducing the timescale A R. Specifically, the zero-order hold (ZOH) rule, which is commonly used based deep learning algorithms, is applied to transform continuous parameters A, B in Eq. to discrete matrix A, B as follows:
B = (AA))"(cxp(AAA - I) AB, htt) = Ah(t-1) - 1) + Br((), v(t) Ch((),
A cxp(
(2)
where E NRN B Nx1, Moreover, given sequence with dimension D) and length I., the SSM is applied independently to each dimension and B, C, are extended with an extra dimension D. The overall computation complexity is O(LDN), which is linear to the sequence length.
Selective State-Space Model (S6) Selective State Space Model is introduced in Mamba with a selective mechanism so that the parameters in SSM can dynamically select necessary information from the context. Specifically. B, C, are designed as input-dependent parameters by utilizing linear functions and broadcast operation. This selective mechanism can help Mamba effectively filtering

---

Retinex-Guided Exposure Correction Mamba (ECMamba)
R
R
Exposure Correction Mamba Module
My
Exposure Correction
Final Ouiput
Module
Retinex Estimator
Lour
Exposure Correction Mamba Module (ECMM)
(c) Retinex Mamba Block (RMB)
Fo
R'
RMB
Fig
Layer
Retinex-SS2D
Layar
Four
RMB F
Layer
R Reut+
RMB
AMB
FG
Efficient Food Forward
GELU
[x] Cour
5i5 Depth wiee Cony
4o4 Cun-
262 DeCo
Sink:
Sinde
Jsl Cure
Ekmmeteeeee Migsigliaeiam
Addion
AMB
Mhup
Adattre
Figure 2: The overall architecture of our proposed Retinex- based framework for exposure correction, which includes a Retinex estimator and primary restoration network MR and M,.
out irrelevant noise and focusing on important tokens, thereby achieving outstanding performances in multiple language and vision tasks.
4 Methodology
The overall framework of our method is shown as Fig. 2, which demonstrates that our proposed exposure correction network is designed based on Mamba and Retinex theory. In this section, first introduce our formulated Retinex-based Exposure Correction Framework (Sec. 4.1), then we propose to utilize Exposure Correction Mamba Module (ECMM) to achieve precise restoration for the reflectance and illumination map (Sec. 4.2). More importantly. to enhance the efficiency and effectiveness, we introduce new 2D Selective State-space layer with an innovative scanning mechanism guided by Retinex information (Retinex-SS2D) in Sec. 4.3 and Sec. 4.4.
4.1 Retinex-Guided Exposure Correction Framework
The Retinex theory can be expressed as IGT RoT @ LGT, where @ denotes Hadamard product, IoT is an ideal image without degradation, RoT and LoT represents the reflectance image and illumination map, respectively. However, low-quality image I. captured under non-ideal illu- mination conditions (under-exposure or over-exposure scenes) inevitably suffers from severe noise, color distortion, and constrained contrast. Therefore, we introduce perturbation to Ret and Let respectively (R and Ż) to model these degraded images
Lg = (Rer + It) O (LeT + L) = -Rer O LeT + Rer + f2 Ler + RoL.
(3)
Some existing Retinex-based methods [6, 13, 18, 33] regard the reflectance component RoT as the final enhanced result, thus they ignore the last three terms in Eq. and focus modeling the mapping: Rer = F(IQQ @ (ILQ) using network F. However, these models can only achieve sub-optimal performance due the difficulty of acquiring accurate mapping, especially for multiple exposure correction task, where multiple Under-Exposed (UE) and Over- Exprssd (OE) inputs correspond to one Normal-Exposed (NE) image. Therefore, we choose to restore the reflectance and illumination component simultaneously in order to obtain satisfactory outputs. Specifically, we element- multiply the both sides of Eq. 3 by L and R. respectively as:
Læ L= RR' Rcr ReT Lo R RRRLLLL
(4)
Lg O R L' LoT + R Lot 2 R

---

Retine--SS2D Layer
Deformable Feature Aggregation
Linear
SILU
DCN
F
Lincur
DwCouv
L Lincar
Ap,
Fy
Scals, Shift
SALU
Efficient FA-SSID
Linear
Layer
Efficient Feature-Aware Selective State-space Mechanism (Efficient FA-SS2D)
l'eature-Aware Scanning
Deformable Feature Aggregation
S6 Process
F
F
Figure 3: The details of our proposed Retinex-SS2D layer. We firstly fuse the input feature Fir and the Retinex guidance Fin- Then we propose innovative Feature-Aware 2D Selective State- spee Mechanism, which utilizes Deformable Convolution (DCN) for feature aggregation. Then we propose the feature-aware scanning strategy based on the activation response map derived from DCN. Compared to other 2D scanning methods, our approach generates sequence ordered by feature importance, thereby maximizing the robust sequence modeling capabilities of Mamba.
where and R are the matrix such that L @ LGT 1 and R RCT = 1, and we assume we can approximate L and R via Retinex estimator &. RoT @ L L R R L L and R @ Ler o R + L R. @ L @ R. indicate the remaining degradation in R' and L'. Therefore, the well-exposed result can be retrieved using
networks by:
(R, L, F.) E(11))
R' InQ L,
L' IIQ R,
(5)
R = = M
Lou =' M,
Lout = Roput Louts
where MR and Mr are networks utilized to predict the minus degradation in R' and L', and F serves Retinex guidance information derived from the L.q-
As shown in Fig. 2, the Retinex estimator takes LL and its mean matrix along the channel dimension (which is omitted for clarity in Fig. 2) as inputs. & firstly utilizes a 1 x convolution and depth-wise convolution with 5 5 kernel to extract features. Then, L, and F. are generated by one
1 x convolution, respectively. More importantly, R' and L' are fed into MR and Mr for further restoration. addition to optimizing Jowe to approximate IGT, our training objective incorporates a constraint L and R. discussed in Sec 4.5.
Discussion ii) Many Retinex-based methods [37] aim learn the mapping from the input to the reflectance image and illumination map, then obtain the final result using Hadamard product operation. However, this strategy is not suitable multi-exposure correction task. Fig. la illustrates the complicated and distant distribution patterns of OE and UE images relative their normally exposed equivalents. Such complex distributions challenge the establishment of accurate mappings from inputs. However, by carefully analyzing Retinex theory, we construct an intermediary space that significantly reduces the distance to our optimization objectives and facilitates the subsequent fine-tuning restoration process, shown in Fig. lb. (ii) Some methods [6, 13, 33, 18] treat ReT as the final enhanced result, which deviates from the original explanation of Retinex theory and leads to limited performance. Therefore, we adopt two-branch framework reconstruct the reflectance and illumination map using distinct deep learning networks. The significance of our framework is discussed in the ablation study (Sec. 5.3).
4.2 Exposure Correction Mamba Module (ECMM)
Together with our proposed Retinex-guides exposure correction framework, we also develop innova- tive networks that serves MR and Mr. in Eq. to estimate the remaining corruption in R' and L', In order develop an effective and efficient module that capable to achieve high performance and is friendly to resource- resource-limited devices, propose novel Retinex-guides Exposure Correction
5

---

Mamba Module (ECMM) which succeeds the powerful modeling capability of Mamba. Notably, MR and Mr share similar structure and we discuss the details of MR in this section.
As illustrated in Fig. 2, our ECMM adopts a two-scale U-Net architecture. For the encoding process, the input R' is firstly processed by cone 3 x 3 and one MetinexMamha Block (RMB) generate the initial feature Fo. Then the downsampling operation is achieved by one 4 x 4 convolution with stride and the down-sampled feature is fed into another RMB to obtain the middle feature F,, For the decoding stage, F is firstly up-scaled to F'j by 2 x 2 deconv with stride 2. To alleviate the information loss caused by the samplingg process, we introduce an adaptive mix-up feature fusion [45] to transfer the encoding information to the decoder stage as:
F. = o0)F + (1-
(6)
where / represents a learnable coefficient, and a denotes the sigmoid function. Then the fused feature F. is fed into the RMB and the convolution layer sequentially and the restored reflectance Roat is obtained by a residual addition accorrding to Eq. 5.
4.3 RetinexMamba Block (RMB)
As the core operator to extract and aggregate features ECMM, our RMB block adopts a similar architecture with Transformer block. However, the significant computational demands of self- attention and cross-attention mechanisms obviously compromise the efficiency of Transformer- based methods, precluding their application in real-time resource-constraiied environments. To this end, we remove the attention process and introduce an innovative dedinex-guidee 2D Selective State-space (retinex-SS2DD layer capture rangr dependencies and facilitate dynamic feature aggregation. Therefore, the feature flow of our RMB can be described as:
F out Fin + Beeiiex-SSSD(LNNI) (,)), F.), F Fout = Fout + EFF((L out)))
(7)
where LN denotes the LayerNorm, Fir and Fin. represent the input and output feature of RMB. Fe is the Retinex guidance information extract by the Retinex estimator E. Moreover, inspired by ConvNext [28, 8], we remove the gating mechanism and the depth-wise convolution to develop an Efficient Feed Forward (EFF) layer that follows the cone 1 1 GELU conv 1 x 1 flow, which operates similarly to MI.Ps in Transformers, while requiring fewer parameters.
4.4 Retinex-SS2) Layer
The detailed illustration of Retinex-SS2D layer is shown as Fig. 3. We first conduct feature fusion for input feature Fi, and Retinex guidance feature F, by linear operation, depth-wist convolution, element-wiseh multiplication, and SiLU operation. Subsequently, the fused feature F, is fed into our proposed Feature-A A ware 2D Selective State-space (FA-SS2D) mechanism to capture dynamic long-range dependencies and achieve adaptive spatial aggregation. Besides, a gating signal G, and linear operation is utilized to obtain the aggregated feature g.
Feature-Aware 2D Selective State-space Mechanism The standard Selective State-space Model (S6) achieves outstanding performance on sequence modeling, especially for NLP task that involves temporal sequence. However, significant challenges arise when applying S6 to 2D image. To better modeling the spatial information in 2D images, several interesting works propose multiple scan strategies to unfold image patches into ID sequences. For example, [27] introduces cross-scan strategy that generate four sequences along four distinct traversal paths, and each sequence is processed by a separate S6 operation. However, such strategy incredibly increase the computational demands, which contradicts the inherently high efficiency and low computational requirements of S6. Furthermore, these techniques only involve simple scanning of images across different directions, which results in a substantial separation of local textures and global structures in some sequences. This separation, to some extent, impairs the S6 framework's 's modeling ability for images.
The deficiencies in the existing scanning approach drive us to reassess how S6 can be more effectively utilized for 2D images. As described in Eq. 2, for each token in a sequence, the output y(t) depends on its input x=(1) and previous inputs (()), ... ,z(t - 1)}. This mechanism requires the 1D sequences transformed from 2D image to meet the following two criteria to ensure excellent
performance: (1) The sequence should prioritize the most critical feature regions at the beginning,
while relegating less significant information to the end. (2) Spatially adjacent features should be

---

MB Dataset [1]
SICE Dataset [5]
Methods
Under exposed
Over-exposer exposed
Average
Under-expues exposed PSNR SSIM|
Over-exppee expexedee
Average
PSNR1 SSIM
PSNR1 SSIM
PSNRT
SSIMF
PSNR SSIM
PSNR
SSIM1
ZeroDCE [16] CVPR220
14.55
0.589
10.40
0.5142
12.06
0.544
16.92
0.633
7.11
0.429
12.02
0.531
RUAS [24] CNPR'21
13:43
0.681
6,39
0.466
9.20
0,552
16.63
0.559
4,54
0.320
10.59
0.439
URetinexNet [37] CVPR'22
13.85
0.737
9.81
0,673
11,42
0.699
17.39
0.645
7.40
0.454
12:40
0,550
KinD [44] MM
15.51
0.761
11.66
0.730
13.20
0.742
13:43
0.484
7.85
0.478
10.64
0.481
L.FFlow" [34] AAAM22
22.35
0.858
22.46
0.679
20.29
0.671
20.87
[1.FI ow-SKF" [38] C3
22.58
0.859
22.72
0.865
22,66
0.863
21.61
0.671
20.55
0.695
21.08
0.683
DRAN [40] CVPR 20
19.74
0.829
19.37
0.812
19.52
0.831
17.96
0.677
17.33
0,683
17.65
0.680
DREN+ERL [21] CVPR223
19.91
0.831
19.60
0,838
19.73
0.836
18.09
0.674
17.93
0.687
18.01
0.680
HECNet [20] BCCV222
22.96
0.860
23.22
0.674
19.91
FECNe++RRL [21] 2V
23.10
0.864
23.18
0.876
23.15
0.871
22.35
0.667
20.10
0.689
21,22
0.678
Retiformer" [6]|0CV'23
22.77
0.862
22.24
0.860
22.45
0.861
22.15
0.665
20.21
0.669
21.18
0.667
[4CT[4] ICCV23
23.49
0.862
23.68
0.872
23.57
0.869
Ours
23.64
0.875
23.84
0.882
23.76
0.879
22.88
0.745
21.23
0.727
22.05
0.736
Table 1: Quantitative comparisons of different methods on multi-exposure correction datasets. The best and second-best results are highlighted in bold and underlined, respectively. "1" means the larger, the better. Note that we obtain these results either from the original papers, or by running the officially released pre-trained models. "*" means that original papers don't report corresponding performance and we train their models using their officially released code.
sequenced closely to avoid significant gaps the sequence. However, existing 2D scanning strategies fail to meet these two requirements, motivating us to propose new solutions to address this gap.
Based on these observations, we introduce an efficient Feature-Aware 2D Selective State-space (FA-SS2D) mechanism, as shown in Fig. 3. Firstly, we develop deformable feature aggregation operation modulated by Retinex information. Specifically, Deformable Convolution (DCN) [48, 9] is adopted to capture dynamic long range dependencies of the fused feature F,. For example, when DCN applied the token delineated by the red frame in F of Fig. 3, its receptive field is an irregular kernel and the activated tokens are outlined in blue. More importantly, when the red frame sliding across the feature map, the irregular kernel varies and we record which tokens are activated. After this process, we obtain the total activation number and calculate the average activation frequency for each token. Therefore, we can obtain an activation response map shown in F, of Fig. 3, where tokens with higher activation frequency represent important features. Specifically, relatively brighter areas in under-cxposed images or relatively normally exposed areas in over-exposed images contain important features and exhibit large activation response. Based on the obtained activation response map, we propose a aatuuee-aware scanning strategy. Different from dirrecionn-ensitivve scanning method [27], our feature-
strategy ranks tokens by their activation frequency and place tokens with higher frequencies at the start of the sequence. Therefore, our generated sequence effectively meets Mamba's 's requirements, thereby maximizing its modeling capabilities for vision data.
4.5 Loss Functions and Constraints
In this work, we adopt the one-stage strategy to train our proposed ECMamba network, which means that the &, MMg and My are optimized simultaneously. Our ultimate training objective is to approximate Lowt to IGT, and we also integrate several constraints on L, R, and Rout to achieve stable training. Therefore, our complete optimize strategy is shown as below:
min C(Iout, IGT) + dL [I Lout, 1) + Ax La G Rout. 1) - L1 (Rout. Ior), (8) E.Ma.M.
where La (LO Lout, 1) and L. (R Rout, 1) are constraints applied on and R. and they essentially employ a self-supervised strategy to learn L and R. In addition, considering this optimization is inherently an ill-posed problem, we adopt L1 (Rout,, IGr) to guide the optimization towards the appropriate direction. Moreover, MM, ICT) is the primary loss function in our training process and it can be calculated by:
C(lout, lCT) Li (Iout, ICT) + dasim Lestm (Louts IGT) Oper Cper (Jouts
(9)
7

---

Input
rEENet
LLFFow SKF
Retiformer
LACT
ECMamba (Ours)
GT
Figure 4: Visual comparison results on ME dataset. Compared to other exposure correction methods, our ECMamba excels in color preservation and structure recovery.
where Lesim [46] denotes the structure similarity loss and Lper [43] represents the difference between features extracted by VGG19 [30]. The coefficients for corresponding loss functions are set as: @saim 0.2, per 0.01, À 0.1, AR 0.1, and D₁ 0.1 in this work.
5 Experiments
5.1 Experiment Settings
Datasets To evaluate the performance of our method, we conduct experiments on five prevailing datasets for multi exposure correction and under-exposure correction: ME [1], SICE [5], LOLv1 [36], LOLv2-real [41], and LOLv2-synthetic [41] datasets. Specifically, each scene in ME dataset has five exposure levels, and we regard the images with the first two exposure level as under-exposed images and the test as over-exposed images. For SICE, following FECNet [20], we select the middle exposure subset as the ground truth, and define the second and the last second exposure subset as the under-exposed and over-exposed images, respectively. For LOLv1, LOLv2-real, LOLv2-synthetic datasets, we leverage their official training and testing data for model training and evaluation.
Implementation Details We use the Adam optimizer with default parameters (8) = 0.9, 3n 0.99) to implement our model by PyTorrc. The initial learning rate set to 1 x 10-4 and then is steadily decreased to 1 x 10 by the cosine annealing scheme, respectively. We utilize random flipping and rotation for data augmentation. Image pairs are cropped as 256 256 and the batch size set to 4. The total training iterations is set to 300K for ME dataset and 150K for other benchmarks, respectively. During the evaluation, we utilize Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM [35]) for numeric evaluation.
5.2 Performances on Multi-Exposure and Under-Exposure Correction
We compare the performance of our ECMamba with current SOTA methods on multi-exposure correction datasets, and we report the quantitative results as Tab. 1. Notably, ECMamba significantly outperforms the current SOTA methods on both under-exposed and over- exposed images within ME dataset and SICE dataset. Specifically, our ECMamba excels PSNR and SSIM, outperforming the second best method over 0919 dB and 0.007 on ME datasct. Furthermore,
Quantitative Results

---

Input
FBCNet
LLFIow-SKF
ECMamba (Ours)
Figure 5: Visual comparisons between ECMamba and other methods on SICE dataset. Our proposed
ECMamba achieves compelling visual performance both on over-exposed and under-exposed images.
LOLvI [36] PSNRT
LOLv2-real [41] PSNR1
LOLy2-synthetic [41]
Methods
Param (M)
SSIM
SSIM
PSNRT
SSIM
Zero-DCE [16] CVPR'20
14.86
0.562
18.06
0.580
0.33
RUAS [24] CVPR221
18.23
0.720
18.37
0.723
16.55
0.652
0.003
URetinex-Net [37] CVPR222
21.33
0.835
21.16
0.840
24.14
0,928
1.32
KinD [44] MM7
20.86
0.790
14.74
0.641
13.29
0.578
8.02
LLFlow [34] AAAU22
25.19
0.870
26.53
0.892
26.08
0,940
37.68
L.LFlowSSFF [38] CNPRR223
26.80
0.879
28.19
0.905
28.86
0.953
39.91
DRBN [40] CVPR'20
19.39
0.817
20.29
0.831
23.22
0.927
5.27
DRBN+ERL [21] CVPR/23
19.84
0.830
FBCNet [20] BCCV122
22.03
0.836
20.29
0.831
23.22
0.927
0.15
FECNet+ERL [2]] cVeR'2
21.08
0.829
Retiformer ICCV223
25.16
0.845
22.80
0.840
25.67
0.930
1.61
LACT* [4] 10e
26.49
0.867
26.95
0.888
27.24
0.941
6.73
ECMamba (Ours)
27.69
0.885
29.24
0.908
29.94
0.959
1.75
Table 2: Quantitative comparisons of differentt methods for under-exposed correction. Notably. compared to SOTA methods, our ECMamba achieves enhanced performance on LOLv1 [36]. LOLv2- real [, and LOLv2-synthetic [41] datasets, demonstrating the effective of our proposed dual-branch basede framework and feature-awand SS2D layer.
compared to the second best performance, our improvement has increased to 0.83 dB and 0.051 on SICE dataset. Tab. 2 summarizes the quantitative comparisons between our method with current SOTA methods second best performance (LLFlow-SKF) by an average 1. dB increase on PNSR with only 4.4% parameters, revealing the impressive effectiveness and high efficiency of our proposed ECMamba. These numbers demonstrate the superior quality of our enhancement and prove the effectiveness of proposed ECMamba and two-branch Retinex-basee pipeline.
under-exposure correction. Specifically,
ECMamba outperforms the
Qualitative Comparisons We present the enhanced images of different methods in Fig. 4 (ME) and Fig. (SICE). Our appealing and realistic enhancement results demonstrate our model can generate images with pleasant illumination, correct color retrieval, and enhanced texture details. For example, the rich structural details of cloud patterns (row 2) mountain surface (row and Fig. the well preserved bridge and its edge contours (row 1) and the vivid presentation of words in the

---

Methods
PSNRT
SSIM1
Param (M)
Methods ViT
PSNR1
SSIM1
Param (M)
Removing Mr
21.55
0.721
1.0
21.88
0.724
14.46
Removing M
21.63
0.723
1.93
Retiformer
21.35
0.702
1.6
Removing 8
21.12
0.695
1.5
Cross-Scan Mechanism
21.69
0.716
2.1
Table 3: Ablation studies on SICE dataset and the average PSNR and SSIM are reported. Left is used to verify the effectiveness of the two-branch Retinex-basee framework, right is to present the importance of our proposed ECMamba module and FA-SS2D strategy. [Key: * When Mr is removed, the hidden dimension of Mr is increased to ensure its parameter number is comparable ECMamba; ViT/Retiformer: utilized replace our ECMamba module; Cross-Scan Mechanism: adopted to replace our FA-SS2D strategy.]
display board (row 2) in Fig. 5. In contrast, previous methods struggle to preserve color fidelity and illumination harmonization.
5.3
Ablation Study
To verify the effectiveness of our proposed ECMamba, we conduct extensive ablation experiments and report the average performance on SICE dataset.
The Contribution of Two-branch Retinex-based Framework We first remove the branch (Mr,,, which is utilized for accurate restoration of the illumination map. Therefore, the remaining network aims to optimize Roat to the ground truth and the performance is reported in Tab. 3, which still presents competitive performance compared the current SOTA in Tab. However, compare to our complete ECMamba, only optimizing the relectance inevitably leads to sub-optimal performance. Furthermore, we also adopt a more complicated Mr. whose parameters comparable to the original two-branch framework. However, compared to our two-branch ECMamba, this network still demonstrate poor performance. Finally, similar to other Retinex-based methods [37], we then remove the Retinex estimator & and directly adopt the remaining network for exposure correction. However, as shown in Tab. 3, this adaptation largely decrease the performance of our ECMamba, indicating the importance of our analysis regarding the intermediary space (R' and L') in Sec. 4.1.
The Importance of Our Proposed E.Mamba Module First, we replace our ECMamba module with Vision Transformer ( (ViT) [11] and Retiformer [6] architecture in our two-branch framework. As presented in Tab. 3, our complete ECMamba module offers impressive performance better than VIT. More importantly, ECMamba's efficiency is comparable to Retiformer, which is famous efficient under-exposure correction approach. Furthermore, to study the significance of our proposed Retinex-SSS SS2D layer and FA-SS2D, replace the Retinex-SSS -S2D layer with cross-scan mechanism proposed in VMamba [27]. The increased parameters and decreased performance demonstrate the superiority of our proposed Retinex-SS2D layer and FA-SS2D strategy.
6 Conclusions
We propose a new two-branch Retinex-based Mamba architecture for exposure correction. By care- fully deriving Retinex theory, we propose an two-branch framework guided by Retinex information. To better balance the performance and efficiency, we introduce ECMamba as the primary restoration module with efficient Retinex-guides SS2D layer and Feature-aware scanning strategy. Extensive experiments demonstrate that our ECMamba significantly outperforms the current SOTA methods on both multi-exposure correction datasets and under- under-exposure correction datasets. We recognize that our work, while pioneering in certain aspects, also highlights avenues for future investigation. For example, similar to other methods, ECMamba struggles to deliver satisfactory results in scenarios involving extreme exposure cases (extremely dark or over-exposed environments) due to the extensive information loss inherent in degraded images. Essentially, directing recovery from severely degraded images is challenging, but recent advances in image restoration have utilized generative priors to infer the degraded details, and achieve favorable results. In the future, we plan to integrate Mamba with generative priors effectively alleviate the performance drop extreme exposure cases.
10

---

References
[1] Mahumoud Afifi, Konstantinos G Derpanis. Bjomn Ommer, and Michael S Brown. Learning multi- photo exposure correction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
[2] Codruta O. Ancuti, Cosmin Ancuti, Florin-Alexandru Vasluianu, Radu Timofte, al. Ntire 2024 dense and nonhomogeneous dehazing challenge report. In Proceedings of the IEFE Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops), 2024.
[3] Codruta O. Ancuti, Cosmin Ancuti,
Vasluianu, Radu Timofte, Han Zhou, Wei Dong. et al. NTIRE: 2023 hr nonhomogeneous dehazing challenge report. In Proceedings of the IEEE Conference Computer Vision and Puttern Recognition Workshops (CVPR Workshops), 2023.
[4] Jong-Hyeon Back, Daelyun Kim, Su-Min Choi, Hyo-jun Lee, Hanul Kim, and Yeong Jun Koh. Luminance- aware color transform for multiple exposure correction. In Proceedings of the IEEE International Confer- ence on Computer Vision (ICCV), 2023.
[5] Jianrui Cai. Shuhang Gu, and Lei Zhang. Learning deep single image contrast enhancer from multi- exposure images. In IEEE Transactions on Image Processing (TIP), 2018.
[6] Yuanhao Cai, Hao Bian, Jing Lin, Haoqian Wang, Radu Timofte, and Yulun Zhang, Retinexformer: One- stage retinex- based transformer for low-light image enhancement. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2023.
[7] Van der Maaten L and Hinton G. Visualizing data using t-snc. In Journal of machine learning research, 2008.
[8] Wei Dong, Han Zhou, Yuqiong Tian, Jingke Sun, Xiaohong Liu, Guangtao Zhai, and Jun Chen. Shad- owRefiner: Towards mask-free shadow removal via fast fourier transformer. In Proceedings of the IEFE Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops), 2024.
[9] Wei Dong, Han Zhou, Ruiyi Wang, Xiaohong Liu, Guangtao Zhai, and Jun Chen. DehazeDCT: Towards effective non-homogeneous dehazing via deformable convolutional transformer. In Proceedings of the IEEE Conference on Computer Vision and Pottern Recognition Workshops (CVPR Workshops), 2024.
[10] Wei Dong, Han Zhou, and Dong Xu, new sclera segmentation and vessels extraction method for sclera recognition. In 2018 10th International Conference on Communication Software and Networks (ICCSN), 2018.
[11] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner. Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. In Proceedings of the International Conference on Learning Representations (ICLR), 2020.
[12] Kang Fu, Yicong Peng, Zicheng Zhang, Qihang Xu, Xiaohong Liu, Jia Wang, and Guangtao Zhai. Attentionlut: Attention fusion-based canonical polyadic lut for real-time image enhancement. arXiv preprint Xiv:: 2024.
[13] Xucyang Fu, Delu Zeng, Yue Huang, Xiao-Ping Zhang, and Xinghao Ding. weighted variational model for simultaneous reflectance and illumination estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
[14] Albert Gu and Dao. Mamba: Lincar-time sequence modeling with selective state spaces. arXiv preprint urXiv:2312.0075: 2023.
[15] Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state spaces. In Proceedings of the International Conference on Learning Representations (ICLR), 2022.
[16] Chunle Guo, Chongyi Li, Jichang Guo, Chen Change Loy, Junhui Hou, Sam Kwong, and Runmin Cong. Zero-reference deep curve estimation for low-light image enhancement. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
[17] Hang Guo, Jinmin Li, Tao Dai, Zhihao Ouyang, Xudong Rea, and Shu-Tar Xia. Mambair: A simple baseline for image restoration with state-space model. In Proceedings of the European Conference on Computer Vision (ECCV), 2024.
[18] Xiaojie Guo, Yu Li, and Haibin Ling. Lime: Low-light image enhancement via illumination map estimation. In IEEE Transactions on Image Processing (TIP), 2016.

---

[19] Jie Huang. Yajing Liu, Xueyang Fu, Man Zhou, Yang Wang, Feng Zhao. and Zhiwei Xiong. Exposure normalization and compensation for multiple-exposure correction. In Proceedings the IEEE Conference on Computer Vixion and Puttern Recognition (CVPR). 2022.
[20] Jie Huang. Yajing Liu, Feng Zhao, Keyu Yan, Jinghao Zhang, Yukun Huang, Man Zhou, and Zhiwei Xiong. Deep fourier-based exposure correction with spatial-frequency interaction. In Proceedings of the European Conference on Computer Vision (ECCV), 2022.
[21] Jie Huang, Feng Zhao, Man Zhou, Jie Xiao, Naishan Zheng, Kaiwen Zheng, and Zhiwei Xiong, Learning sample relationship for exposure correction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
[22] Edwin H Land. The retinex theory of color vision. Scientific american, 1977.
[23] Wenhao Li, Guangyang Wu, Wenyi Wang, Peiran Ren, and Xiaohong Liu. Fastllve: Real-time low- light video enhancement with intensity-awarc lookup table. In Proceedings of the ACM Conference on Multimedia (MM), 2023.
[24] Risheng Liu, Long Ma, Jiaao Zhang, Xin Fan, and Zhongxuan Luo. Retinex-inspiree unrolling with cooperative prior architecture search for low-light image enhancement. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
[25] Xiaohong Liu, Yongrui Ma. Zhihao Shi, and Jun Chen. Griddehazenet: Attention based multi-scale network for image dehazing. In Proceedings of the IEEE International Conference Computer Vision (ICCV), 2019.
[26] Xiaohong Liu, Zhihao Shi, Zijun Wu, Jun Chen, and Guangtao Zhai. Griddehazenett:: An enhanced multi-scale network with intra-task knowledge transfer for single image dehazing. IEEE Transactions on Intelligent Transportation Systems, 2022.
[27] Yue Liu, Yunjie Tian, Yuzhong Zhao, Hongtian Yu, Lingxi Xie, Yaowei Wang, Qixiang Ye, and Yunfan Liu. Vmamba: Visual state space model. arXiv preprint urXiv:2401.10166 2024.
[28] Zhuang Liu, Hanzi Mao, Chao-Yuar Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie.
A convnet for the 2020s. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[29] Yuan Shi, Bin Xia, Xiaoyu Jin, Xing Wang, Tianyu Zhao, Xin Xia, Xuefeng Xiao, and Wenming Yang. Vmambair: Visual state space model for image restoration. arXiv preprint arXiv:2403. /10443,,.144,,,, 2024.
[30] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recogni- tion. In Proceedings of the International Conference on Learning Representations (ICLR), 2015.
[31] V. A. Sindagi. P. Oza, R. Yasarla, and V.M. Patel. Prior-based domain adaptive object detection for hazy and rainy conditions. Proceedings of the European Conference on Computer Vision (ECCV), 2020.
[32] Flerin-Alexandru Vasluianu, Tim Seizinger, Zhuyun Zhou, Zongwei Wu, Cailian Chen, Radu Timofte, et al. NTIRE 2024 image shadow removal challenge report. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops), 2024.
[33] Ruixing Wang, Qing Zhang, Chi-Wing Fu, Xiaoyong Shen, Wei-Shi Zheng, and Jiaya Jia. Underexposed photo enhancement using deep illumination estimation. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition (CVPR), 2019.
[34] Yufei Wang, Renjie Wan, Wenhan Yang, Haoliang Li, Lap-Pui Chau, and Alex Kot. Low-light image enhancement with normalizing flow. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAJ), 2022.
[35] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Fero P Simoncelli. Image quality assessment: From error visibility structural similarity. In IEFF Transactions on Image Processing (TTP), 2004.
[36] Chen Wei, Wenjing Wang, Wenhan Yang, and Jiaying Liu. Deep retinex decomposition for low-light enhancement. In Proceedings of the Conference on British Machine Vision Conference (BMVC), 2018.
[37] Wenhui Wu, Jian Weng. Pingping Zhang. Xu Wang. Wenhan Yang, and Jianmin Jiang. Uretinex-ner net: based deep unfolding network for light image enhancement. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
12

---

[38] Yuhui Wu, Chen Pan, Guoqing Wang, Yang Yang, Jiwei Wei, Chongyi Li, and Heng Tao Shen. Learning semantic-aware knowledge guidance for low-light image enhancement. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
[39] Dong Xu, Wei Dong, and Han Zhou. Sclera recognition based efficient sclera segmentation and significant vessel matching, In The Computer Journal, 2022.
[40] Wenhan Yang, Shiqi Wang. Yuming Fang, Yue Wang, and Jiaying Liu. From fidelity to perceptual quality: semi-supervised approach for low-light image enhancement. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3063-3072, 2020.
[41] Wenhan Yang, Wenjing Wang, Haofeng Huang. Shiqi Wang, and Jiaying Liu. Sparse gradient regularized deep retinex network for robust low-light image enhancement. In JEEE Transactions on Image Processing (TTP), 2021.
[42] Xiangyu Yin, Xiaohong Liu, and Huan Liu. Fmsnet: Underwater image restoration by learning from a synthesized dataset. In International Conference on Artificial Neural Networks (ICANN). 2019.
[43] Richard Zhang, Phillip Isola, A.A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features a perceptual metric. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
[44] Yonghua Zhang, Jiawan Zhang, and Xiaojie Guo. Kindling the darkness: A practical low-light image enhancer. In Proceedings of the ACM Conference Multimedia (MM), 2019.
[45] Han Zhou, Wei Dong, Xiaohong Liu, Shuaicheng Liu, Xiongkuo Min, Guangtao Zhai, and Jun Chen. GLARE: low light image enhancement via generative latent feature based codebook retrieval. In Proceed- ings of the European Conference on Computer Vision (ECCV), 2024.
[46] Han Zhou, Wei Dong, Yangyi Liu, and Jun Chen. Breaking through the haze: An advanced non- homogeneous dehazing method based on fast fourier convolution and convnext. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops), 2023.
[47] Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang. Vision mamba: Efficient visual representation learning with bidirectional state space model. In Proceedings of the International Conference on Machine Learning (ICML), 2024.
[48] Xizhou Zhu, Han Hu, Stephen Lin, and Jifeng Dai. Deformable convnets v2: More deformable, better results. In Proceedings of the IEFE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims presented in the abstract and introduction accurately represent the contributions and scope of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Refer Section 6
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

Justification:
This work mainly includes empirical contributions
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide experimental configurations Section 5, our code will be released via Github.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer: [Yes]
Justification: Our experiments are all conducted publicly accessible datasets, and all dataset details are illustrated in Sec.5.1. For some experiment implementations. follow the official code of existing works, all code can be found in their official GitHub repository.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
Beapii/gguides//oeeSubbbbe hwtpbbmmiisiooooocyy..oo.. for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: haabbiicggiiess/.// Sttpe//CooSuubiieeee
/ //nips.co.
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
All experiment details are illustrated Sec.5.1.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: All our experiments are based on pre-trained models for inference, with the experimental seeds fixed. Therefore, there are no random results, and the corresponding error bars are not applicable.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
16

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide sufficient information on the computer resources (type of comput- ing workers, memory, time of execution) needed to reproduce each of the experiments in Sec. 5.1.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: ptttps://neurppee ctcc/pullcc/thhcuuuidddddee
Answer: [Yes]
Justification:
This work is conducted accordance with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: The paper proposes new algorithm for image exposure correction, there is negative impact of our method, and the positive impact is discussed in Sec. 6.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper The paper poses such risks.
17

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility MI).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
The creators or original owners of assets (e.g., code, data, models), used the paper, properly credited, and the license and terms of use explicitly mentioned and are properly respected.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
18

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. cwm datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New Assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The new assets introduced in the paper well documented with the docu- mentation provided alongside the assets.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
19

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
20


Reviews:
[{"review_id": "Xja64AK5mw", "reviewer": "Reviewer_5Zdz", "summary": "This paper introduces ECMamba, a novel framework that integrates Retinex theory and the Mamba framework to address the complex issue of exposure correction. ECMamba adapts the Retinex theory to suit the needs of exposure correction and develops a Retinex estimator to assess both reflectance and illumination maps. Subsequently, the framework employs Mamba to orchestrate the restoration process, termed ECMM. The ECMM module features a core operator, Retinex-SS2D, which employs a two-dimensional scanning strategy and deformable feature aggregation. The final output is generated by multiplying the enhanced reflectance map with the enhanced illumination map, yielding a superior result.", "questions": "Shown in the \"Weakness\" part.", "limitations": "The paper does not discuss the limitations. No negative social impact is present in this work.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper represents the inaugural application of Retinex theory to the problem of exposure correction. Historically, Retinex theory was applied exclusively to low-light image enhancement, resulting in output images that are invariably brighter than their inputs. This study expands the theory's application, adapting it to scenarios where output images may be either brighter or darker than the inputs.\n\n2. The paper is well-organized and communicates its concepts effectively.", "weaknesses": "1. The rationale for employing Retinex theory and the Mamba framework in multi-exposure correction is not adequately articulated. The author mentions that Retinex theory has not been deeply integrated into this field, yet this alone does not establish its superiority over alternative methods. A more robust justification for using Retinex theory, as well as the Mamba framework, is necessary to substantiate their relevance to solving this specific problem.\n\n2. The initial two rows of Figure 4 are presumably intended to illustrate over-exposed scenarios. However, these images do not appear over-exposed as they lack saturated regions and display clear details and contrast. It would be advisable to replace them with images that more accurately represent over-exposure.\n\n3. The inference speed of the proposed method is a critical aspect that is currently unaddressed in the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for providing valuable feedback for our paper. We address your concerns in turn below. We hope our response can well address all your concerns.\n\n> **Q1:** The rationale for employing Retinex theory and the Mamba framework in multi-exposure correction is not adequately articulated. The author mentions that Retinex theory has not been deeply integrated into this field, yet this alone does not establish its superiority over alternative methods. A more robust justification for using Retinex theory, as well as the Mamba framework, is necessary to substantiate their relevance to solving this specific problem.\n\nThank you for your insightful review. In addition to the fact that Retinex theory and the Mamba framework have not yet been applied to the multi-exposure correction task, the rationale for employing these methods can be justified from the following aspects:\n\n**Why is Retinex theory important?** Retinex theory, a physically validated theory, essentially studies the reflectance and illumination components of images, which is closely related to the task of correcting improper illuminations for over-/under-exposed images in this paper. Therefore, incorporating Retinex theory can help understand and solve multi-exposure correction problems, as well as enhance the interpretability of the algorithm.\n\n**Why is simply adopting Retinex theory not sufficient?** Simply adopting Retinex theory to decompose the reflectance and illumination component in a data-driven manner is quite challenging, as this problem is highly ill-posed. Therefore, through the derivation of Retinex theory, we introduce intermediate $\\mathbf R^{\\prime}$ and $\\mathbf L^{\\prime}$ and we enact models with powerful modeling capability (i.e., Mamba) as the primary network to remove the degradation in $\\mathbf R^{\\prime}$ and $\\mathbf L^{\\prime}$.\n\n**Motivation of utilizing Mamba.** One of the reasons is that recently proposed Mamba network demonstates impressive modeling for natural language and image data. But this is not the entire picture. Compared to the self-attention layer in transformers, whose computation complexity is quadratic to the sequence length, Mamba's recurrent working strategy and state-space mechanism require less computation that is linear to the sequence length. Please also refer to the Figure B in the PDF file attached in the global response. More importantly, we visualize the activation response maps in ViT for under-/over-exposed images in Figure B. These maps indicates that the activation response intensity from ill-posed areas to normally exposed regions is minimal. To optimize efficiency, we can exclude the computations of the response intensity from the ill-posed area to the relatively normally exposed region. Therefore, for muti-exposure correction task, the optimal working mechanism involves extracting critical information from the relatively normally exposed regions in under/over-exposed images to restore other areas. This working mechanism aligns perfectly with Mamba's recurrent functional principles. The remaining task is to prioritize these relatively normally exposed regions at the beginning of the sequence. Accordingly, we propose the feature-aware SS2D scanning mechanism based on deformable feature aggregation to achieve this objective. In this way, our Mamba network with feature-aware SS2D scanning mechanism is highly efficient and particularly well-suited for the specific task of exposure correction.\n\n**Why combine Mamba with Retinex theory?** In the ablation study, we have directly fed the input into Mamba networks, denoted as 'Removing $ \\mathcal E $' in Table 3 in our manuscript, and we observe a 0.93 dB decrease in PSNR and 0.04 decline in SSIM compared to our complete ECMamba. This comparison justifies the importance of combining Mamba with Retinex theory.\n\n\n> **Q2:** The initial two rows of Figure 4 are presumably intended to illustrate over-exposed scenarios. However, these images do not appear over-exposed as they lack saturated regions and display clear details and contrast. It would be advisable to replace them with images that more accurately represent over-exposure.\n\nWe appreciate your valuable suggestion. More appropriate visual comparisons are provided in Figure C of the rebuttal PDF file. We will replace previous images with these new visual results in our revised manuscript. \n\n> **Q3:** The inference speed of the proposed method is a critical aspect that is currently unaddressed in the paper. \n\nThanks for pointing this out. In our manuscript, we have reported the model complexity of different models in Table 2. It is indeed important to provide additional metrics related to model efficiency, such as FLOPs and inference time. We have supplemented these metrics as shown in below Table A1. Thanks to Mamba's high efficiency and hardware-aware design, the efficiency of our method is comparable to or better than that of other models, except for FECNet.\n\n**Table A1.** Efficiency comparisons between our method and other models. Please note FLOPs is calculated with 256x256 inputs and the average inference time is calculated by testing with multiple 400x600 inputs on the same GPU device.\n\n|Metrics|KinD|DRBN|LLFlow|LLFlow-SKF|FECNet|Retiformer|LACT|ECMamba(Ours)|\n|--------- |-------|---------|----------|-------------------|----------|--------------|--------|-----------------------|\n|Parameters (M) |8.02|5.27|37.68|39.91|0.15|1.61|6.73|1.75|\n|FLOPs (G) |34.99|48.61|286.34|310.27|5.82|15.57|78.64|16.38|\n|Inference time (s)|1.136|0.228|0.399|0.474|0.099|0.189|0.295|0.164|\n\n> **Q4:** More discussions on limitations of our paper. \n\nWe briefly addressed the limitations in the conclusion. Here, we provide a more thorough discussion of it in Q2 of our global response. We will update this in our revised manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "e7h9BDIaRL", "reviewer": "Reviewer_fpJd", "summary": "This paper introduces a new pipeline called ECMamba for multiple exposure correction. Based on the analysis of Retinex theory, the authors develop a dual-branch framework, and each pathway is designed to restore the reflectance image and the illumination map, respectively. Besides, considering the powerful and efficient sequence modeling of the recently proposed Mamba, this paper also incorporates the Mamba architecture within each pathway and attempts to achieve effective and efficient exposure correction. To exploit Mamba to process image data, this paper develops an innovative \"feature-aware\" 2D scanning strategy based on deformable feature aggregation, which are quite different from other \"direction-sensitive\" scanning approaches. Finally, extensive experiments on multi-exposure and under-exposure datasets are conducted, and the reported results demonstrate the proposed method in this paper outperforms current SOTA approaches.", "questions": "1) What constraint is adopted to train the proposed two-branch network? Is that possible that R_out and I_out are quite different form their corresponding ground truth, but their product I_out is very close to the well-exposed image I_GT?\n2) What is the difference in the activation frequency map between under-exposed images and over-exposed images?\n3) How to get the activation frequency map in Fig.3? Please explain in detail.", "limitations": "Authors have briefly discussed the limitations in Sec.6. There are no potential negative ethical and societal implications of this work.", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1) This paper is a retinex-based method. Authors carefully analyze the retinex component for corrupted images and then design a two-branch network based on their discussions, which makes the proposed method theoretically solid.\n\n2) The T-SNE visualizations in Fig.1 and discussions in line 141-152 provide a detailed explanation for the design of the two-branch exposure correction network.\n\n3) This paper deeply incorporates the retinex guidance into deep learning network, which is a promising way to learn a consistent transformation between under-/over-exposed inputs and normal-exposed images.\n \n4) Different from simply regarding the scanning of images to 1D sequence as a \"direction-sensitive\" problem, this paper introduces an interesting concept — \"feature-aware\" scanning. This strategy is proposed by carefully analyzing the operation mechanism of Mamba, thus the\"feature-aware\" scanning strategy is theoretically sound and constructive.\n\n5) Both quantitative and qualitative performance of the proposed method are impressive.\n\n6) This paper is well written and easy to follow.", "weaknesses": "(1) This paper introduces an interesting two-branch framework based on retinex theory. Compared to a single-branch network, it is more difficult to train a two-branch network especially when the final result is obtained via Hadamard product. However, the constraint used in the training phase is less discussed in this paper.\n\n(2) The proposed \"feature-sensitive\" scanning mechanism in Mamba is quite novelity. However, the difference in the activation frequency map between under-exposed images and over-exposed images is less explored.\n\n(3) This paper adopts deformable convolution for feature aggregation and the generation of the the activation frequency map. However, how to get the activation frequency map is not very clear. For example in Fig.3, my understanding is that the 9 blue tokens are activated when a 3x3 deformable convolution is applied to the red token. Also, I think the numbers in F_d denote the average activtion frequency for each token when sliding the red token across the entire F_f. Am I correct?\n\nMinor: \nThe reference [1] in line 194 is a typo, the reference should be Vmamba, which is cited as [18] in this paper. \n\"Deformable Feature Aggegration\" in Fig.3 should be \"Deformable Feature Aggregation\".", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for the valuable comments on our paper. We will explain your concerns point by point.\n\n> **Q1:** What constraint is adopted to train the proposed two-branch network? Is it possible that $ \\mathbf R_{out} $ and $ \\mathbf I_{out} $ are quite different from their corresponding ground truth, but their product $ \\mathbf I_{out} $ is very close to the well-exposed image $ \\mathbf I_{GT} $?\n\nThank you for your insightful reviews. It is true that training such two-branch network is more difficult, as there are no ground truth for the reflectance and illumination components for images. In our initial expriments, we only calculate the difference between $ \\mathbf I_{out} $ and $ \\mathbf I_{GT} $ as the loss function for optimization. However, we observe that $ \\mathbf  R^{\\prime} $, $ \\mathbf  R_{out} $,  and $ \\mathbf  L_{out} $ deviate their expected values evidently, though the final output $\\mathbf  I_{out} $ is close to $ \\mathbf I_{GT} $. We attribute this phenomenon to the fact that this optimization is an ill-posed problem. To tackle this issue, we attempt to add some constraints to the loss function, as shown in Q1 of our global response. With these constraints, we successfully achieve stable training for our ECMamba. We will add more discussion for constraints in our revised manuscript.\n\n\n> **Q2:** The proposed \"feature-sensitive\" scanning mechanism in Mamba is quite novel. However, what is the difference in the activation frequency map between under-exposed images and over-exposed images?\n\nWe have visualized the activation frequency maps for under-exposed and over-exposed images in Figure D of the attached PDF. The difference in the activation frequency map is primarily caused by the different important feature regions in under-exposed and over-exposed images. Specifically, relatively brighter areas in under-exposed images or relatively normally exposed areas in over-exposed images contain important features and exhibit a large activation response in the corresponding activation frequency map.\n\n\n> **Q3** How to get the activation frequency map in Fig.3? Please explain in detail. My understanding is that the 9 blue tokens are activated when a 3x3 deformable convolution is applied to the red token. Also, I think the numbers in $ \\mathbf F_d$ denote the average activation frequency for each token when sliding the red token across the entire F_f. Am I correct?\n\nYour understanding is correct. Different from typical convolution, the receptive field of 3x3 deformable convolution is an irregular kernel. For example, when applying deformable convolution for the red token in Figure 3, 9 blue tokens are activated and they form the irregular kernel. When the red token is sliding across the feature map, the irregular kernel varies and we will record which token is activated. After this process, we obtain the total activation number for each token and calculate the average activation frequency for each token, which is regarded as the activation frequency map.\n\n> **Q4** Incorrect reference and typo. \n\nThanks for pointing this out. We will revise these mistakes and double check all over the manuscript. \n\n> **Q5** More discussions on Limitations of our paper. \n\nWe appreciate your attention for our brief discussion on limitations in the conclusion of our manuscript. We also provide more discussions for this part in Q2 in the global response.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "snbJGMAhrW", "reviewer": "Reviewer_EgGV", "summary": "The authors propose a novel Mamba architecture for the exposure correction task based on Retinex theory. They design a separate Retinex estimator and two exposure correction modules to restore reflectance and illumination. In these modules, the authors improved efficiency in terms of time and resources by introducing the Mamba module. This model demonstrates excellent performance on the representative ME and SICE datasets.", "questions": "* The authors mention that the modulated reflectance (R′) demonstrates a closer approximation of Normal-Exposed (NE) images, but this is not well illustrated in Figure 1(a). If they could investigate and show the distribution of distances reduced via t-SNE, their claim would be more trustworthy.\n* I am willing to increase the score if I have misunderstood something.", "limitations": "* This study is far from having a social impact. However, It is regrettable that the limitations and failure cases of the network's performance are not adequately addressed.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 4, "contribution": 2, "strengths": "* The mamba architecture is suitable for image processing because it can start processing even with partial input, similar to RNNs. From this perspective, the authors' attempt to introduce Mamba in the field of image restoration is technically very valid. \n* Using the Selective State Space Model, the authors proposed the Retinex-SS2D layer, which enables Retinex-guided deformable feature aggregation.\n* Unlike existing methods that aim to achieve limited generalization by introducing exposure normalization, the authors have developed a stronger model with good generalization capabilities.", "weaknesses": "* It is necessary to explain the design motivation of the Retina estimator. Unlike general networks, the authors start with a 1 by 1 convolution. It is necessary to explain the motivation behind designing such a network.\n* A detailed explanation is needed on how \\bar{L} and \\bar{R} are constrained. R_gt and I_gt do not seem to be values that can be obtained directly.\n* There is a lack of analysis and ablation studies on the network.\n  * I am curious about the performance of the synthesized result when removing M_R and M_L, which are related to the terms overlooked in other papers. If I understand correctly, the performance should be similar to other works even without these two modules, especially since they are used in a residual form.\n  * The comparison experiment between deformable convolution and standard convolution is missing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review and comments. We provide our point-to-point response below:\n\n> **Q1:** The design motivation of the Retinex estimator\n\nThe exposure correction task involves handling various exposure levels while also addressing complex issues such as color distortion and detail loss. Compared to employing an end-to-end approach to forcibly learn a unified mapping that simultaneously addresses these complex problems, the Retinex theory's decomposition of illumination and reflectance decouples the space of this mapping into two smaller subspaces. This facilitates better and easier optimization/learning, and enhances the interpretability of our model.\n\n> **Q2:** Why 1x1 convolution?\n\nInspired by [R1], we first calculate the mean value for each pixel along the channel dimension and concat it to the original input within the Retinex estimator. Therefore, the 1x1 convolution is a good option to fuse the concatenation result across various channels. Also, compared to typical convolutions with large kernels, 1x1 convolution is more efficient. Besides, Retinex theory and later Retinex based methods [R2, R3] have underscored the independence of the reflectance and illumination map. To accurately decompose these two components, at the beginning of the network, it's better to operate independently on each pixel without considering the influence of its spatial neighbors. \n\n[R1] Deep Retinex Decomposition for Low-Light Enhancement. In BWVC 2018\n\n[R2] Lightness and Retinex Theory. In Journal of the Optical Society of America 1971.\n\n[R3] Diff-retinex: Rethinking low-light image enhancement with a generative diffusion model. In CVPR 2023.\n\n> **Q3:** Constraints on $\\mathbf{\\bar{L}}$ and $\\mathbf{\\bar{R}}$?\n\nPlease see Q1 in our global response. We will add this part in our revised manuscript.\n\n> **Q4:** Performance when removing $\\mathcal{M}_R$ and $\\mathcal{M}_L$?\n\nThe results after removing $ \\mathcal{M}_R$ and $ \\mathcal{M}_L $ are presented in Table C1. It is evident that the outcomes are significantly suboptimal, highlighting the indispensability of $ \\mathcal{M}_R $ and $ \\mathcal{M}_L $. Our motivation for designing $ \\mathcal{M}_R $ and $ \\mathcal{M}_L $ stems from the derivation provided in Equation 4 of our manuscript. This derivation indicates that even if the Intermediate Illumination Estimation Map $\\mathbf{\\bar{L}}$ can be accurately obtained—which could be approached by increasing the model complexity of the retinex estimator—the resultant $ \\mathbf R^\\prime = \\mathbf R^{GT}+ \\text{degradation terms}$. This implies that $\\mathbf  R^{\\prime} $ still contains non-negligible degradation terms, such as color distortion, as visually illustrated in Figure 1b. The same reasoning applies to $ \\mathbf  L^{\\prime} $. \n\nTherefore, $\\mathcal{M}_R$ and $\\mathcal{M}_L$ are primarily designed to eliminate these degradations. Without the degradation correction, performing an element-wise multiplication on the predicted reflectance and illumination, which contain these degradations, would amplify the deviations. Hence, $\\mathcal{M}_R$ and $\\mathcal{M}_L$ are essential as degradation correctors within the entire framework.\n\n**Table C1.** Performance of only $\\mathcal{E}$ on SICE dataset.\n|Method|PSNR|SSIM|\n|-|-|-|\n|Only $\\mathcal{E}$ |11.76|0.363|\n|ECMamba|22.05|0.736|\n\n> **Q5:** Performance when replacing deformable convolution with standard convolution\n\nBased on your suggestion, we conduct an ablation experiment by replacing the deformable convolution with standard convolution in our FA-SS2D mechanism, and the results are reported in Table C2. Evidently, the use of standard convolution yields inferior performance.\n\nPlease note that deformable convolution is a crucial part of effectively applying the Mamba model to our vision task. Although Mamba has recently gained attention due to its efficient processing of long sequences in natural language processing, it was originally designed for one-dimensional data. This makes it less suitable for non-sequential two-dimensional images, which contain spatial information like textures and structures. \n\nTo address this issue, we incorporated deformable convolution into the Mamba model. Unlike standard convolution, deformable convolution can dynamically learn appropriate receptive fields—either long or short—from the data. This ability to adaptively gather spatial information increases the likelihood of activating important image features. Based on this, we developed a strategy that prioritizes features according to their activation frequency, placing the most frequently activated features at the start of the sequence. This ensures each image patch is enriched with contextual knowledge, providing a new way to use Mamba in processing visual data.\n\n**Table C2.** Performance when replacing deformable convolution using standard convolution on SICE dataset.\n|Method|PSNR|SSIM|\n|-|-|-|\n|FA-SS2D with standard conv |21.45|0.709|  \n|FA-SS2D with deformable conv (Ours)|22.05|0.736|\n\n> **Q6:** T-SNE visualization of distribution of reduced distance?\n\nWe add new visualization to present the reduced distance of $\\mathbf  R^{\\prime} $ as Figure R4 in the attached PDF file in the global response. Please note that in order to make the visualization result more clear and readable, we randomly visualize part of images presented in Figure 1(a) of our manuscript. The green line represents the distance between the input and normal exposure features (NE), while the purple line indicates the distance between the modulated reflectance $\\mathbf{R}^{\\prime}$ and NE. We will add this new visualization in our revised version.\n\n> **Q7:** Limitations and failure case?\n\nWe briefly discussed the limitation in the conclusion of our manuscript. We also provide more discussions for this part in Q2 in the global response.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xCnSFWh09h", "reviewer": "Reviewer_zooA", "summary": "The manuscript proposes a novel model that deeply embeds Retinex theory into the Mamba model. The proposed model consists of two modules: Retinex estimator and exposure correction. Comparative experiments on five datasets demonstrate the superiority of the proposed method, and subsequent ablation experiments demonstrated the rationality of the model construction.", "questions": "\tIn page 1, line 35, the authors stated that “Retinex theory has not been deeply integrated into…”, which could cause some negative effects? Please provide some explanations.\n\tThe RMB in Figure 2 is mentioned in page 5, line 162, but the corresponding model diagram is missing. Please provide more explanations.\n\tIn Figure 3, which arrows are marked with ∆p and m?\n\tIn the ablation experiments, the author removed ML twice, what are the differences between those operations?\n\tIn qualitative comparisons, the author do not list all the comparison experiment results, only list some results. For example, as can be seen in Table 1, the results of FECNet+ERL are better than FECNet, why not choose FECNet+ERL for displaying?", "limitations": "The authors do not point out the limitations of the work and do not offer further perspectives. I hope it will be improved.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Unlike previous work that used the R component of Retinex as the output, this model processed both the R and L components simultaneously, resulting in satisfactory results.", "weaknesses": "The experimental validation leaves much to be desired as I mention in the questions section.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comprehensive and insightful review. We answer your questions in turn below:\n\n> **Q1:** In page 1, line 35, the authors stated that 'Retinex theory has not been deeply integrated into…', which could cause some negative effects? Please provide some explanations.\n\nMulti-exposure correction aims to correct improper illuminations in under/over-exposed images. It is quite challenging to simply adopt deep learning models to address this problem. This is because deep learning models often struggle to distinguish between illumination information and the intrinsic reflectance properties of objects in images. By incorporating Retinex theory to decompose the reflectance and illumination components, deep learning networks are more likely to produce high-quality enhancement results. Moreover, Retinex theory offers a physically justified way to interpret the decomposition of illumination and reflectance. Without Retinex theory, the inner working mechanisms of deep learning models may be more enigmatic and challenging to explain and understand.\n\n> **Q2:** The RMB in Figure 2 is mentioned in page 5, line 162, but the corresponding model diagram is missing. Please provide more explanations.\n\nIn our manuscript, the diagram of RMB is provided in the orange dashed block in Figure 2(c). Its key component, the Retinex-SS2D layer (misspelled as Retinx-SS2D layer), is illustrated in Figure 3.\n\n> **Q3:** In Figure 3, which arrows are marked with $ ∆p $ and $ m $?\n\nThe right arrow represents $ ∆p $ (offset) and $ m $ (modulation scalar),  which are obtained by applying a separate convolution over the output of the 'Scale, Shift' layer. Moreover, the $ ∆p $ and $ m $ are utilized to calculate the deformable features and the activation frequency map. We will make this clear in the revised version.\n\n> **Q4:** In the ablation experiments, the author removed $ \\mathcal{M}_L $ twice, what are the differences between those operations?\n\nIn our ablation study, we conduct two experiments related to the removal of $ \\mathcal{M}_L $, denoted as 'Removing $ \\mathcal{M}_L $' and 'Removing $ \\mathcal{M}_L^* $', respectively.  'Removing $ \\mathcal{M}_L^* $' means that while removing $ \\mathcal{M}_L $, we also increase the hidden channel of the remaining network  $ \\mathcal{M}_R $  to make its complexity similar to our full network. Through this ablation comparison under similiar complexity, we emphasize that the superior performance of our method is not attributed to the increased model parameters but to our proposed two-branch network, which optimizes both the reflectance and illumination map simultaneously.\n\n> **Q5:** In qualitative comparisons, the authors do not list all the comparison experiment results, only list some results. For example, as can be seen in Table 1, the results of FECNet+ERL are better than FECNet, why not choose FECNet+ERL for displaying?\n\nWe report quantitative results for baseline models following their original papers. However, the paper that introduces ERL strategy doesn't release neither their visual results nor their implementation code, so we can't reproduce or access to their visual results. This is the reason why we didn’t report visual results for FECNet+ERL. \n\n> **Q6:** Limitations and future perspectives.\n\nWe have briefly discussed the limitation of our method in the conclusion. Moreover, we add more discussions towards limitations and future perspectives in Q2 in the global response. We plan to incorporate this more detailed version into our revised manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely thank all reviewers (**R1** 5Zdz, **R2** fpJd, **R3** EgGV, and **R4** zooA) for their detailed reviews and constructive comments. The reviewers agree that:

**Novel or interesting approach:**
-  **R1:** "This study **expands the theory's application**, adapting it to scenarios..."
-  **R2:** "The proposed "feature-sensitive" scanning mechanism in Mamba is **quite novel**"

-  **R4:** "**Unlike previous work** that used the R component of Retinex as the output, this model..."

**Soild:**

-  **R2:** "Authors **carefully analyze** the retinex component..., which makes the proposed method **theoretically solid**. ..., thus the "feature-aware" scanning strategy is **theoretically sound and constructive**."

-  **R3:** "From this perspective, the authors' attempt to introduce Mamba in the field of image restoration is **technically very valid**."

**Well-written and Organized:**

-  **R1:**  "The paper is **well-organized** and communicates its concepts effectively."

-  **R2:**  "This paper is **well written** and easy to follow."

Here we first summarize the key clarifications in relation to some common review comments and concerns, and then address individually to each reviewer's concerns point by point.

**Q1: Detailed constraints applied on $ \mathbf {\bar{L}}$ and $ \mathbf {\bar{R}}$.**

In line 139-140, page 5 of our manuscript, we mentioned incorporating constraints on $ \mathbf {\bar{L}}$ and $ \mathbf {\bar{R}}$. It's challenging to directly optimize $ \mathbf {\bar{L}}$ and $ \mathbf {\bar{R}}$ due to the lack of their corresponding ground truths. Instead, we feed these two variables into subsequent networks to obtain the final enhanced image for optimization, and this strategy amounts to applying an indirect supervision to $ \mathbf {\bar{L}}$ and $ \mathbf {\bar{R}}$. Moreover, to achieve stable training, our complete optimize strategy is shown as below:

$$
\min_{ \mathcal E, \mathcal M_R, \mathcal M_L } \mathcal{L}(\mathbf I_{out}, \mathbf I_{GT}) + \lambda_L \cdot \mathcal L_1 (\mathbf{\bar{L}} \odot \mathbf L_{out}, \mathbf{1}) + \lambda_R \cdot \mathcal L_1 (\mathbf{\bar{R}} \odot \mathbf R_{out}, \mathbf{1}) + \lambda \cdot \mathcal L_1 (\mathbf R_{out}, \mathbf I_{GT}), 
$$

where $\mathcal L_1 (\mathbf{\bar{L}} \odot \mathbf L_{out}, \mathbf{1})$ and $\mathcal L_1 (\mathbf{\bar{R}} \odot \mathbf R_{out}, \mathbf{1})$ are constraints applied on $ \mathbf {\bar{L}}$ and $ \mathbf {\bar{R}}$. These two constraints essentially employ a self-supervised strategy to learn $ \mathbf {\bar{L}}$ and $ \mathbf {\bar{R}}$. $\lambda_L$ and $\lambda_R$ are set to 0.1 in this work.

In addition, considering this optimization is inherently an ill-posed problem, we adopt $\mathcal L_1 (\mathbf  R_{out}, \mathbf I_{GT})$ to guide the optimization towards the appropriate direction and we set $\lambda$ to 0.1. Please note that $ \mathcal{L}(\mathbf I_{out}, \mathbf I_{GT}) $ is the primary loss function in our training process and it is calculated by the following equation:

$$
\mathcal{L}(\mathbf I_{out}, \mathbf I_{GT}) = \mathcal L_1(\mathbf I_{out}, \mathbf I_{GT}) + \phi_{ssim} \cdot \mathcal L_{ssim}(\mathbf I_{out}, \mathbf I_{GT}) + \phi_{per} \cdot \mathcal L_{per}(\mathbf I_{out}, \mathbf I_{GT}),
$$

where $ \mathcal L_{ssim} $ denotes the structure similarity loss and $ \mathcal L_{per} $ represents the difference between features extracted by VGG19. $ \phi_{ssim} = 0.2 $ and $ \phi_{per} = 0.01 $ are coefficients for corresponding loss functions.


**Q2: More discussions on limitations and future perspectives of our paper.**

 Here, we provide more detailed discussions about the limitation and future perspective of our paper:

**limitation:** While our method achieves impressive performance on various datasets, similar to other methods, ECMamba struggles to deliver satisfactory results in scenarios involving extreme exposure cases (extremely dark or over-exposed environments), as shown in Figure A in the attached PDF. Due to the extensive information loss inherent in such images, our method demonstrates limited capability of perfectly restoring them.

**Future perspective**: In extreme exposure scenarios, the image details are often degraded severely. Direct recovery is challenging, but recent advances in image restoration have utilized generative priors to infer the degraded details, and achieve favourable results. In the future, we plan to integrate Mamba with generative priors for LLIE to effectively alleviate the performance drop on extreme exposure cases.

Meta-review: Following extensive interactions between the authors and reviewers, all reviewers have expressed a positive opinion on the paper. The AC believes the paper merits a clear acceptance. Although the authors have attempted to justify their use of Mamba and Retinex for this task, the rationale still lacks a solid foundation. The authors are encouraged to improve the paper by adhering to the commitments and clarifications made during the rebuttal and discussion
Decision: Accept (poster)