You Only Look Around: Learning Illumination-Invariant Feature for Low-light Object Detection
Authors: MingboHong
Shen Cheng
Haibin Huang
Haoqiang Fan
Shuaicheng Liu
Venue: NeurIPS 2024 poster (2024)
Keywords: object detection；lowlight object detection；illumination invariant  feature;
Primary area: machine_vision

Abstract: In this paper, we introduce YOLA, a novel framework for object detection in low-light scenarios. Unlike previous works, we propose to tackle this challenging problem from the perspective of feature learning. Specifically, we propose to learn illumination-invariant features through the Lambertian image formation model. We observe that, under the Lambertian assumption, it is feasible to approximate illumination-invariant feature maps by exploiting the interrelationships between neighboring color channels and spatially adjacent pixels. By incorporating additional constraints, these relationships can be characterized in the form of convolutional kernels, which can be trained in a detection-driven manner within a network. Towards this end, we introduce a novel module dedicated to the extraction of illumination-invariant features from low-light images, which can be easily integrated into existing object detection frameworks. Our empirical findings reveal significant improvements in low-light object detection tasks, as well as promising results in both well-lit and over-lit scenarios.

You Only Look Around: Learning Illumination Invariant Feature for Low-light Object Detection
Mingbo Hong"
Shen Cheng" Megvii Technology, Beijing, China chengshen@moggvii. com
Megvii Technology, Beijing, China mingbonong$7@gmail com
Haibin Huang Kuaishou Technology ja aackiehuuagghaaliinggaal.. com
Haoqiang Fan Megvii Technology, Beijing, China fhq@megvii com
Shuaicheng Liu
University of Electronic Science and Technology of China 1 liushuaichann www.gccsst.eeeeeeeeeeeeeeee edw.esttccedd...... cn
Abstract
In this paper, we introduce YOLA, novel framework for object detection in low-light scenarios. Unlike previous works, we propose tackle this challeng- ing problem from the perspective of feature learning. Specifically, we propose learn illumination-invarian features through the Lambertian image formation model. We observe that, under the Lambertian assumption, it is feasible to ap- proximate illumination-invarian feature maps by exploiting the interrelationships between neighboring color channels and spatially adjacent pixels. By incorporat- ing additional constraints, these relationships can be characterized in the form convolutional kernels, which can be trained in a detection-drive a network. Towards this end, we introduce novel module dedicated to the ex- traction of illumination cllminnationinnvriiant features from low-light images, which be easily integrated into existing object detection frameworks. Our empirical find- ings reveal significant improvements in low-light object detection tasks, as well as promising results in both well-lit and over-lit scenarios. Code is available at pettps://githng www.hu.......iii.... Noom/MingboHong//YO..
manner within
Introduction
In the field of computer vision, object detection stands cornerstone, driving advancements in numerous applications ranging from autonomous vehicles to security surveillance 26 51 20). The ability accurately identify and locate objects in digital imagery has remarkable progress, largely due to the advent of deep learning techniques [16] [ 40). However, despite these advancements, object detection in low-light conditions remains significant challenge. Low-light environments lead to poor image quality, reduced visibility, and increased misdetections in night-time surveillance and
twilight driving
48 32
Traditional methods in tackling low low-light object detection have predominantly leaned towards image enhancement techniques [17 24 53 34). While these methods have demonstrated effectiveness in
"Equal contribution.
Corresponning Author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Radbo DN
(a) Result Bascline
(b) EIgo-
(c) Resut IIM-Edge
(A) IIM
(6) Result IIM
Figure 1: (a): The base detector failed to recognize objects. (b. c) However, when IIM is employed with a simple edge feature, the object is identified. (d, e) Furthermore, the full IIM utilizes a task- driven learnable kernel to extract illumination-invann invarrint features that are richer and more suitable for the detection task than simple edge features.
improving visual aesthetics and perceptual quality, they often do not directly translate to improved object detection performance. This discrepancy arises because these enhancement techniques are typ- ically optimized for human visual perception, which does not always correlate with the requirements for effective and accurate object detection by machine learning models.
In addition to image enhancement strategies, another research direction involves fine-tuning pre- trained models for low-light conditions. Typically, detectors are initially trained on extensive datasets of well-lit images, such as those from Pascal VOC
subsequently fine-tuned on smaller, low-light datasets 48 32 11 and Microsoft COCO (28], and To enhance the utilization of cross- domain information, the MAET framework 7 was developed to learn intrinsic visual structure features by separating object features from those caused by degradation in image quality. Similarly, methods [3 25 aim to restore the normal appearances of corrupted images during detector training. However, these techniques often depend heavily on synthetic datasets, which could limit their real-world applicability.
Recent methods in low-light object detection, such as those in 36 49 use Laplacian pyramids (2) for multi-scale edge extraction and image enhancement. FeatEnHancer 18] further leverages hierarchical features for improved low-light vision. However, these task- bosk-specific, loss-driven approaches often grapple with a larger solution space due to varying illumination effects.
In this study, introduce a novel approach that explicitly leverages Illuminationninvarini features, utilizing the principles of the Lambertian image formation model 422. Under the Lambertian assumption, can express the pixel values in each channel discrete combination of three key components: the surface normal, the light direction (both of which are solely related to the pixel's position), the spectral power distribution, and the intrinsic properties of the pixel itself. The illmination-invariann feature can be learned by eliminating the position-related term and spectral power-related term 44 We introduce this concept of extracting illumination-inaaria features into low-light detection tasks and demonstrate that incorporating this feature yields significant performance improvements in low-light detection tasks. We further improve this commination-ivaariinn feature using dask-driven kernels. Our key observation is that by imposing a zero-mean constraint on these kernels, the feature can simultaneously discover richer downstream task-specific patterns and maintain illumination invariance, thereby improving performance.
Towards this end, we propose the coiminattion-nvariinn Module (IIM), versatile and adaptive component designed to integrate the information gleaned from these specialized kernels with standard RGB images. The IIM can be seamlessly integrated with a variety of existing object detection frameworks, enhancing their capability to perform accurately in low-light environments, whether through naive edge features or diverse We further conduct experiments on the ExDark and U +DARK FACE datasets to evaluate our method. Our experimental results demonstrate that the integration of the IIM significantly enhances the detection accuracy of existing methods, leading to substantial improvements in low-light object detection, To summarize, our contributions are as follows:
iiniiiattioonivvriaate characteristics, as shown in Fig [
We introduce YOLA, a novel framework for object detection in low-light conditions by leveraging illumination invariant features.

---

Image
Dotector
IIM
Head
Tuscc Conv
Hsad
Head
IIM
Fuse CoNv
24 CoKV
log (R)
32-
clementtwise add
CoNcort
CoNv
Cowv
3X3, 24-1
log(B)
COKY
convolation
The overall pipeline of YOLA. YOLA extracts ioamination-invariinn features via IIM and integrates them with original images by leveraging a fuse convolution block for the subsequent detector.
Figure 2:
We design a novel Illumination -nniiiartioo--naiiantt without requiring additional paired datasets, and can be seamlessly integrated into existing object detection methods.
Invariant Module extract Illumination-invariann features
We provide in-depth analysis of the extracted Illmination-invarian paradigm and propose a learning Illuiiationniivaaiann paradigm.
Our experiments show YOLA can significantly improve the detection accuracy of existing methods when dealing with low-light images.
2
Related work
2.1 General object detection
Current modern object detection methods be classified as anchor-based and anchor-free, The anchor-based detectors are derived from the sliding-window paradigm, where the dense anchor be viewed the dlding-window arranged in spatial space. Subsequently. the anchors assigned as positive or negative samples based on the matching strategy (i.e., Intersection-over Union (IoU) [16]. Top-K K [52] 50 Common anchor-based methods include R-CNN [16] [5] 40), SSD (30, YOLOv2 38). and RetinaNet [27]. among others. In contrast, the anchor-free detectors liberate the handcraft anchor hyper-parameter setting, enhancing their potential in terms of generalization capability. Prominent methods in anchor-free include YOLOv1 (37] FCOS 44, and DETR (3) Despite the remarkable achievements of both anchor-based and anchor-free detectors in general object detection, they exhibit unsatisfactory performance under low-light conditions.
2.2 Low-light object detection
Object detection in low-light conditions remains significant challenge. One common line of research involves leveraging image enhancement techniques, such as KIND 53) SMG (46) NeRCo 47), and others 17
22 to directly improve the quality of the low-light image. The enhanced images are then deployed in the subsequent training and testing stages of detection. However, the objective of image enhancement is inherently different from that of object detection, making this strategy suboptimal. To address this, some researchers 21 6) explore integrating image enhancement with object detection during the training process. Nevertheless, the task of balancing hyperparameters to equilibrate visual quality and detection performance remains intricate. Recently, Sun et [43] proposed targeted adversarial attack paradigm aimed at restoring degraded images to ones that are more favorable for object detection. MAET 7 trained on a low-light synthetic dataset, obtaining the pre-trained model endowed intrinsic structure decomposition ability for downstream lowlight object detection. Further, IA-YOLO (I]] and GDIP (5] elaborately design the differentiable image processing module to enhance image adaptively for adverse weather object detection. Note that the

---

aforementioned methods either require dedicated low-light enhancement dataset or rely heavily on synthetic datasets in training. To mitigate the limitations, set of methodologies [36] 49 [8] utilize multi-scale hierarchical features and are driven purely by task-specific loss to improve low-light vision. Unlike those methods, we introduce illminationninvariiii features to alleviate the effect of illumination on low-light object detection, without requiring additional low-light enhancement datasets or synthetic datasets.
2.3 Illumination invariant representation
Adverse illumination typically degrades the performance on downstream tasks, prompting researchers to explore illlminationniivalne invariant techniques to mitigate this impact. For high-level tasks, Wang et al. 45 proposed an illumination normalization method for Face Recognition. Alshammari et understanding 3 convert RGB images to illumination-invan invariant chromaticity space, preparing for the following feature extraction to achieve traffic object detection in various illumination conditions. For low-level tasks, several physics-basee based invariants, such as Colour Ratios [13 (CR) and Cross Colour Ratios [14 (CCR), are employed decompose the illumination for intrinsic image representations
al. I use iitiminntion-ivvaiiinn image representation to improve automotive
and segmentation. Lu et al.
decomposition 1O 9 8 However, these methods leverage illuinaaiioninvaaiiii
derived from the fixed formulations, which may not adequately capture the diverse and complex illumination scenarios that are specific to downstream applications. In contrast, our method enables the adaptive learning of illumination-inve -ivarraat features in an end-to-en manner, thereby enhancing compatibility with downstream tasks.
3 Method
In this section, we formally introduce YOLA, a novel method for low-light object detection. As illustrated in Fig[2 the key component YOLA is the Illumination Invariant Module (IIM) focusing on feature learning to derive downstream task task-specific Illmination-innvriin features. These features can be integrated with existing detection modules, enhancing their capability in low-light conditions. Next, we will introduce the derivation of illumination-invariant features and provide detailed description of IIM's specific implementation.
3.1 Illumination invariant feature
Notation: Let represents an image in the standard RGB domain, and let C' E [R, G, B) represent the image in the red, green, or blue channel. We define the value in channel C' of pixel pt as Cp.- where E I is the pixel index.
Lambertian assumption: According body reflection term of the dichromatic reflection model, the value of Cp, can be expressed in the discreet form as follows:
CN = mm.p. T,, Co (A)p (A),
Here, up Tp represents surface normal and light direction respectively, and m denotes the interaction function between them. The term eC,, represents the spectral power distribution of the illuminant at point P; in color channel C², and pCr, represents the intrinsic property (reflectance) of the object at point pt in color channel C.
becomes apparent that the term m determined solely by the positional component, with no impact from the color channels. This observation leads to the strategy of calculating the difference between values of different color channels at the same spatial positions effectively eliminate the influence of m. To eliminate the term e, we can utilize the assumption that illumination is approximately uniform across adjacent pixels. Consequently, by computing the difference between values of neighboring pixels, we can further further eliminate the influence of m.
Cross color ratio: Taking into consideration two adjacent pixels, denoted as P1 and P2, along with the red (R) and blue (B) channels, we can determine the ratio M,s between the red and blue channels through the following computational procedure:
Rp, Bpz
Mfb
Rp, Bm

---

Taking the logarithm of M, and substituting the pixel values with Eq. | we get:
lo)(M,b) (og(m((pr: (n)) (og(m((p)) 'p11)
+log(e Roi (2)) log(e Rrz (A))
+log(p Rr (A)) - log( Rrz (A))
(3)
Hlog(m(mpx: (pp (px))
log(m ( (nipi 1p2))
+log(e Bv (A)) log(t BB1 (A))
+log(p Pr (X)) Log(o pm (A)).
With the illumination assumption that ecr, Rs eCr2, the above equation can further simplified into an Iluminationiivvariant form:
(M,) = log(p Rm (A)) - log (2)) +log(p (A)) - P (A))
By observing the elimination Eq. we can find that subtraction within the same channel eliminates the illumination term (implemented by zero-mean constraint), while cross-channel subtraction removes the surface normal and light direction terms, which motivates us to design the learning illuiination-invaiian paradigms.
In this case, we can use a convolution operation to extract features, shown in Fig. The extracted features are processed and fused by the IIM before being sent to the detector. When using fixed weights of adjacent pixels with a subtraction value of 1, refer to it as IIM-Edge. Next, we will provide detailed introduction to the IIM.
3.2 Illumination invariant module
While Eq. 4 4offers straightforward and effective method for calculating Illumination Invariant features, rigidity presents certain limitations. Specifically, the fixed nature of this equation may not adequately capture the diverse and complex variations in illumination that are specific to downstream tasks across different scenarios. To address this, we have evolved the equation into more adaptable form using convolutional operations. Instead of relying involves learning a set of convolutional kernels. This strategy not only enhances the robustness of the Illumination Invariant feature extraction but also improves its efficiency. To this end, propose Illumination Invariant Module comprising two main components, including learnable kernels and zero-mean constraint. Note that Illumination Invariant Module yield features inherently illumination invariant at initialization. Subsequent kernel learning geared towards producing task-specific illumination invariant features for downstream tasks.
single kernel, our approach
Learnable kernel. The goal is to transform the fixed illumination-invann invariant feature into a learnable form. Specifically, aim to learn set of convolutional kernels Wi, W2, ... WE&X* where represents the number of kernels and & denotes the kernel size. Here, we extend the fixed feature into more versatile and generalized form. Let p; and w; represent group pixel position and its corresponding weight within a kernel Wn, where i = 0,
.......... k2. These parameters enable us to evolve the Cross Color Ratio (CCR) into an adaptable form, enhancing its capability to effectively handle varying illumination conditions. Note that w; is trainable, rendering the positive or negative polarity inconsequential.
12-1
Rp, Bp.
Bpp+s Rpus
M
122
Rp.
II ==1
Bp.
2=1
To make the extended form still satisfy Illumination Invariant, the logarithm of M,s should satisfy the following constraints:
Σ #2 wo (2)) =
(6)
E
wplog(eBm. Bon (A))
5

---

Car-o
0333
Card.
0og:
C.rFD.335 35
oo
Caro.44 48
GT
Baseline
DENEt
MAET
Car10.8
People
Cor
CarO
Coro.45
Cor:0.
Car
Vorro
ContD
KIND
SMG
NERCO
Owrs
GT
Bascline
DENet
WAET
FAWDDS
KIND
SMG
NERCO
Owrs
Figure 3: Qualitative comparisons of TOOD detector on both ExDark and + FACE dataset, where the top 2 rows visualize the detection results from ExDark, and the bottom rows show the results from U FACE. The images are being replaced with enhanced images generated by LLIE low-light object methods. Red dash boxes highlight the inconspicuous cases. Zoom in red dash boxes for the best view.
If the above equation holds true, the term and the term eliminated. The final feature can be expressed in a generalized form:
k2
k²
log(M,t) Σ welog(p
Rz, (A)) Σ
welog(p (2))
Similarly, we can obtain (oy) and (Mgb) to form (T)
The resulting features obtained by applying the kernel W, to the image I denoted as fw. (T), be expressed as:
W @ log R) + (-W,) @ log B)
(T)
W. log R) WW,)
) @ log (G) W log(G) + -W) @ (og(B)
(8)
where @ denotes the convolution.
Drawing from Eq. 6 and the approximation e eB z BB, in the context of convolutional kernels, simply ensure that the mean W to be 0, as depicted by:
Zero mean constraint:
W
w =0
(9)
12
1=1
This constraint is enforced by substituting the mean value from the kernel W,, W, W..

---

YOLOv3
TOOD
YOLOv3
TOOD
Methods
Methods
recall
mAP
recall
mAP
recall
mAPs0
recall
mAP30
Baseline
84.6
71.0
91.9
72.5
Baseline
77.9
60.0
81.5
62.1
KIND 53
83.3
69.4
92.1
72.6
KIND 53
76.0
58.4
82.4
63.8
SMG 46
82.3
68.5
91.8
71.5
SMG 46
69.3
48.9
77.1
55.8
NeRCo
83,4
68.5
91.8
71.8
NeRCo 47
68.9
49.1
76.8
55.6
DENet
36
84.2
71.3
92.6
73.5
DENet 36
77.7
60.0
84.1
66.2
GDIP 133
84.8
72.4
92.2
72.8
GDIP 153
77.8
60.4
82.1
62.9
IAT
153
85.0
72.6
92.9
73.0
IAT [53
77.6
59.8
82.1
62.0
MAET
85.1
72.5
92.5
74.3
MAET
77.9
59.9
83.6
64.8
YOiv--Naive
84.8
71.6
91.8
71.6
YOLA-Naive
76.6
59.2
82.8
64.6
YOLA
86.1
72.7
93.8
75.2
YOL.A
79.1
61.5
84.9
67.4
Table I: Quantitative comparisons of the ExDark dataset based on YOLOv3 and TOOD detectors.
Table 2: Quantitative comparisons of the UG +DARK FACE dataset based on YOLOv3 and TOOD detectors.
4
Experiments
4.1
Implementation details
We evaluate the proposed method using the popular anchor-based detector YOLOv3 [39 and the anchor-free detector TOOD [2]. Both detectors are initially pre-trained on the COCO dataset and subsequently fine-tuned on the target datasets utilizing the SGD 41 optimizer with initial learning rate 1e-3. Specifically, we resize the ExDark dataset images to 608 x 608 and train both detectors for 24 epochs, reducing the learning rate by a factor of 10 at epochs 18 and 23. For the D FACE dataset, we resize images to 1500 x 1000 for TOOD and maintain the 608 x 608 resolution for YOLOv3 to be consistent with MAET. YOLOv3 is trained for 20 epochs, with the learning rate decreased by a factor of 10 at 14 and 18 epochs. TOOD are trained for 12 epochs, with the learning rate decreased by factor of 10 8 and 11 epochs. Additionally, we implement straightforward illumination-invarin model, denoted as YOLA-Naive, by removing the IIM and ensuring various illumination features are consistently imposed by an MSE loss. We implement YOLA using the MMDetection toolbox
4.2
Dataset
We evaluate our proposed method on both real-world scenarios datasets: exclusively dark 32 (ExDark) and U +DRR FACE [48]. ExDark dataset contains 7363 images ranging from low- light environments to twilight, including 12 categories, 3,000 images for training, 1,800 images for validation, and 2.563 images for testing. We calculate the overall mean average precision (mAPs0) and mean recall the IoU threshold of 0.5 the evaluation metrie. UG* FACE dataset contains 6,000 labeled face bounding box images, where 5,400 images are allocated for training and 600 images reserved for testing, and calculating the corresponding recall and mAPso as evaluation metrics. Additionally, we also evaluate the generalization ability of our method on the COCO 2017
28 dataset.
4.3 Low-light object detection
Table T presents the quantitative results of YOLOv3 and TOOD detectors on the ExDark dataset, respectively. We report the low-light image enhancement (LLIB) methods, including KIND, SMG, and NeRCo, along with the state-of-the-art low-light object detection methods, DENet, and MAET. Compared to the low-light object detection methods, the L.IEE methods fail to achieve satisfactory performance due to inconsistency between human visual and machine perception. The enhancement methodologies prioritize human preferences. However, it is important to note that optimizing for enhanced visual appeal may not align with optimized object detection performance. Despite being the current state-of-the- the-art in image enhancement techniques, SMG and NeRCo exhibit worse performance compared to KIND when evaluated in the context of object detection tasks. In contrast, end-to-end approaches such as DENet and MAET, which account for machine perception, generally yield superior results in object detection compared to the LLIE methods. Nevertheless, our method remains simple and effective when compared similar approaches the same category. Moreover, compared to YOLAe-aive, YOLA exhibits superior performance because its extracted features inherently possess illumination invariance, implying smaller solution space compared to YOLA- Naive. Specifically, method achieves the best performance both anchor- based YOLOv3
7

---

Dataset
IM
I1M Eddg
Zmeon
mAP50
Dataset
Method
AP50
AP75
mAP
72.5
TOOD
59.0
45.3
41.7
Exdark
73.8
well-lit
+ YOLA
59.4
46.0
42.3
74.7
75,2
TOOD
57.4
43.8
40.5
over-light
62.1
YOLA
58.3
44.6
41.2
64.5
DarkFace
Table 4:
Ablation study for YOLA on COCO
66.9
2017val.
67.4
Table 3: The effectiveness of IIM, HIM-Edge and the zero mean constraint meen based on TOOD, The blank line denotes the baseline.
Method
Kind
SMG
NeRco
DENer
MAET
Ours
Mize(M)
8.21
17.90
23.30
0.04
0.008
Table 5: Model size of different methods.
and anchor-free detectors TOOD, surpassing the baseline by significant gains of 1.7 and 2.5 mAP, indicative of its superiority and effectiveness. Meanwhile, compared with most LLIE and lowlight object detection techniques, the number of parameters in our YOLA (0.008M) is significantly lower, as presented in T This highlights the potential for our method to be deployed in lightweight practical applications. For detailed quantitative comparison, please refer to appendix.
4.4 Low-light face detection
We have shown the results the ExDark dataset. Next, we showcase the results a dataset that includes small-sized objects. Tresenss the quantitative results the detector YOLOv3 and TOOD on U DARK FACE dataset. Significantly. it is worth noting that most LLIE methods integrated into the YOLOv3 detector fail to achieve satisfactory results. This implies that the utilization of enhancement- based approaches learning of useful representations in such images. On the other hand, methods considering the object detection task demonstrate better performance, where YOLA increases the 1.5 mAP, demonstrating its superior performance and generalization capability. For the recently advanced detector TOOD, our method still outperforms these LLIE and low-light object detection methods, achieving remarkable mAP of 67.4. This underscores YOLA's superior generalization capabilities in improving the performance of both anchor-based and anchor-free detection paradigms.
impair the details of small faces, hindering the
4.5 Quantitative results
The top 2 rows of Figure 3 show the qualitative results from the ExDark dataset using the TOOD detector, where existing methods exhibit missed detections, highlighted by the red dashed boxes. In contrast, YOLA excels in detecting these challenging cases, demonstrating its superior performance in complex scenarios. The bottom rows exhibit the qualitative results of the + FACE dataset using the TOOD detector. These faces are typically tiny under low-light conditions, making difficult for most methods to achieve comprehensive results.
Although our method does not explicitly constrain image brightness, the enhanced images tend to display increased brightness in the final results. The visual results shown in the figures may appear slightly grayish due the absence of value range constraints on the enhanced images. For image display, we conducted channel-wise normalization.
4.6
Ablation studies
4.6.1
Illumination invariant module
We evaluate the effectiveness of the IIM detectors TOOD, as presented in Table respectively. The 1st and 5th rows of Table 3 show the baseline detectors evaluated on ExDark and +DARK FACE dataset. By incorporating the IIM to introduce illumination-in considerable performance gains (2.3 and 4.8 mAP for Ex and + FACE, respectively).
features, the detector yields
4.6.2 Zero mean constraint
By imposing zero mean constraint on the convolutional kernels, the subtraction formed by the kernels can factor out the illumination items. To evaluate the impact of this constraint, we exclude from IIM, and the results shown in Table 3 is evident that the removal of this constraint leads

---

-Ige
IIM
Figure 4: Visualization of the features (columns 2 and 4) generated by IIM-Edge and IMM(ernels normalized for better visibility, we average the features across the channel dimensions and then conduct spatial normalization), along with detection results (columns I and 3). Best viewed by zooming in.
to decline performance, with reductions of 0.3 and 0.5 mAP for TOOD. These results indicate that utilizing the zero mean constraint to mitigate the effects of illumination is beneficial to low-light object detection.
4.6.3 Learnable kernel
The IIM is formed with the learnable kernels, encouraging the illumination-invarit features that are adaptively learned an end-to-end fashion. In this experiment, we evaluate the fixed kernels (as specified in Eq. 4 also referred to as IIM-Edge), the results of which are shown in Table 3 outperforms the baseline by 1.3 mAP on ExDark and 2.4 mAP on U demonstrates that the incorporation of illumination invariant features is beneficial for low-light object detection. Subsequently, we substitute the fixed kernels with the learnable kernels, yiclding further gains of 1.4 mAP ExDark and 2.9 mAP on effectiveness learnable kernels. In addition, we also impose a consistency loss for IIM's output feature stabilize the kernel learning prevent trivial solutions within the kernel, mitigating the impact uneven lighting. (please refer to the appendix A for details).
FACE, which
FACE. These results clearly prove the
Visualization:
Illumination-invarian features exhibit considerable diversity, but the diversity captured by fixed kernels limited. We visualize and compare the fixed kernel and learnable kernel as shown in Fig. [ The features yielded by fixed kernels appear relatively uniform, primarily consisting of simple edge features. contrast, learnable kernels extract more diverse patterns, resulting in visually richer and more informative representations.
4.7 Generalization
In this section, we broaden the application the YOLA to the general object detection dataset COCO 2017, investigating the YOLA's generalization capability beyond low-light object detection. The metrics mAP (average for loU 0.5:00055:0.55). AP30, and AP7s are adopted to evaluate performance on COCO 2017val 11aavl (also called minival) presented in Table 4 Specifically,
trained 12 epochs with GPUs and a mini-batch of I per GPU in an initial learning rate of 1e-2 by the SGD optimizer on both well-lit and over-lit (generated by brightening the origin image) scenarios. By observing Table 4 we can see that detectors integrated with YOLA in both scenarios exhibit notable improvements in performance.
5 Conclusion
In this work, we have revisited the complex challenge of object detection in low-light conditions and demonstrated the effectiveness of illumination-invariant features in improving detection accuracy in such environments. Our key innovation, the Illumination-Invarian Module (IIM), harnesses these features to great effect. By integrating a zero-mean constraint within the framework, we have effectively learned diverse set of kernels. These kernels adept at extracting Illuminationinvartt

---

features, significantly enhancing detection precision. We believe that our developed IIM module can be instrumental in advancing low-light object detection tasks in future applications.
Acknowledgement: This work was supported in part by National Natural Science Foundation of China (NSFC) under grant No. 62372091 and Natural Science Foundation of Sichuan Province under grant Nos. 2023NSFSC0062 and 2023NSFSCC172..
References
[1] Naif Alshammari, Samet Akcay, and Toby P Breckon. On the impact of Illumination-invarian image pre-transfommatin for contemporary automotive semantic scene understanding. In 2018 IEEE Intelligent Vehicles Symposium (/V), pages 1027-1032. IEEE, 2018.
[2] Peter J Burt and Edward H Adelson. The laplacian pyramid compact image code. In Readings in computer vision, pages 671-679. Elsevier, 1987.
[3] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. ECCV, pages 213-229. Springer, 2020.
[4] Kai Chen, Jiaqi Wang, Jiangmiao Pang, Yuhang Cao, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jiarui Xu, et al. Mmdetection: Open mmlab detection toolbox and benchmark. arXiv preprint arXiv: 0906.07061000 0515 2019.
[5] Linwei Chen, Ying Fu, Kaixuan Wei, Dezhi Zheng, and Felix Heide. Instance segmentation in the dark. IJCV, pages 1 1-21, 2023.
[6] Ziteng Cui, Kunchang Li, Lin Gu. Shenghan Su, Peng Gao, Zhengkai Jiang, Yu Qiao, and Tatsuya Harada. You only need 90k parameters to adapt light: a light weight transformer for image enhancement and exposure correction. In BMVC, page 238, 2022.
[7] Ziteng Cui, Guo-Jun Qi, Lin Gu. Shaodi You, Zenghui Zhang, and Tatsuya Harada. Multitask act with orthogonal tangent regularity for dark object detection. In ICCV, pages 2563-2562, 2021.
[8] Partha Das, Maxime Gevers, Sezer Karaoglu, and Theo Gevers. Idtransformer: Transformer for intrinsic image decomposition. In ICCV, pages 816-825, 2023.
[9] Partha Das, Sezer Karaoglu, and Theo Gevers. Intrinsic image decomposition using physics- based cues and cnns. 223:103538, 2022.
[10] Partha Das, Sezer Karaoglu, and Theo Gevers. Pie-net: Photometric invariant edge guided network for intrinsic image decomposition. CVPR, pages 19790-19799, 2022.
[11] Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. IJCV, 88:303-338, 2010.
[12] Chengjian Feng, Yujie Zhong, Yu Gao, Matthew R Scott, and Weilin Huang. Tood: Task-aligned one-stage object detection. In ICCV, pages 3490-3499. IEEE Computer Society, 2021.
[13] Graham David Finlayson. Colour object recognition. Simon Fraser University, 1992.
[14] Theo Gevers and Arnold WM Smeulders. Color-based object recognition. Pattern recognition, 32(3):453 464, 1999.
[15] Ross Girshick. Fast r-cnn. In ICCV, pages 1440-1448, 2015.
[16] Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR, pages 580-587, 2014.
[17] Chunle Guo, Chongyi Li, Jichang Guo, Chen Change Loy, Junhui Hou, Sam Kwong, and Runmin Cong. Zero-reference deep curve estimation for low-light image enhancement. In CVPR, pages 1780-1789, 2020.
10

---

[18] Khurram Azeem Hashmi, Goutham Kallempudi, Didier Stricker, and Muhammad Zeshan Afzal. Featenhancer: Enhancing hierarchical features for object detection and beyond under low-light vision. In ICCV, pages 6725-6733 2023.
[19] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In ICCV, pages 2961-2969, 2017.
[20] Mingbo Hong. Shuiwang Li, Yuchao Yang, Feiyu Zhu, Qijun Zhao, and Li Lu. Sspnet: Scale selection pyramid network for tiny person detection from uav images. IEEE geoscience and remote sensing letters, 19:1-5, 2021.
[21] Shih-Chia Huang, Trung-Hise Le, and Da-Wei Jaw. Dsnet: Joint semantic learning for object detection in inclement weather conditions. IEEE TPAMI, 43(8): 2020.
[22] Hai Jiang, Ao Luo, Haoqiang Fan, Songchen Han, and Shuaicheng Liu. Low-light image enhancement with wavelet-basee based diffusion models. ACM TOC, 42(6))1-14, 2023.
[23] Hai Jiang, Ao Luo, Xiaohong Liu, Songchen Han, and Shuaicheng Liu. Lightendiifusion: Unsupervised low-light image enhancement with latent-retinex diffusion models. arXiv preprint arXiv:2407.08939, 2024.
[24] Yifan Jiang, Xinyu Gong, Ding Liu, Yu Cheng, Chen Fang. Xiaohui Shen, Jianchao Yang. Pan Zhou, and Zhangyang Wang. Enlightengan: Deep light enhancement without paired supervision. IEEE transactions on image processing. 30:2340-2344 2021.
[25] Sanket Kalwar, Dhruv Patel, Aakash Aanegola, Krishna Reddy Konda, Sourav Garg, and K Madhava Krishna. Gdip: Gated differentiable image processing for object detection in adverse conditions. pages 7083-7089. IEEE, 2023.
[26] Shuiwang Li, Yangxiang Yang, Dan Zeng, and Xucheng Wang. Adaptive and background -waare vision transformer for real-time uav tracking. In ICCV, pages 13989-14000, October 2023,
[27] Tsung- Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In ICCV, pages 2980-2988, 2017.
[28) Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. ECCV, pages 740-755. Springer, 2014.
[29] Risheng Liu, Long Ma, Jiaao Zhang, Xin Fan, and Zhongxuan Luo. Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement, In CVPR, pages 10561-10570, 2021.
[30] Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng Yang Fu, and Alexander C Berg. Ssd: Single shot multibox detector. In ECCV, pages 21-37. Springer, 2016.
[31] Wenyu Liu, Gaofeng Ren, Runsheng Yu, Shi Guo, Jianke Zhu, and Lei Zhang, Image-adaptive yolo for object detection in adverse weather conditions. In AAAI, volume 36, pages 1792-1800, 2022.
[32] Yuen Peng Loh and Chee Seng Chan, Getting know low-light images with the exclusively dark dataset. 178:30 42, 2019.
[33] Yan-Feng Lu, Jing-Wen Wen Gao, Qian Yu, Yi Li, Yi-Sheng Lv, and Hong Qiao. A cross-scale and illumination invariance- based model for robust object detection in traffic surveillance scenarios. IEEE Transactions on Intelligent Transportation Systems, 2023.
[34] Feifan Lv, Feng Lu, Jianhua Wu, and Chongsoon Lim. Mbllen: Low-light image/video enhancement using enns. In BMVC, volume 220, page 2018.
[35] Long Ma, Tengyu Ma, Risheng Liu, Xin Fan, and Zhongxuan Luo. Toward fast, flexible, and robust low-tight image enhancement. In CVPR, pages 5637-5646, 2022.
11

---

[36] Qingpao Qin, Kan Chang, Mengyuan Huang, and Guiqing Li. Denet: Detection-driven enhancement network for object detection under adverse weather conditions. In ACCV, pages 2813-2829, 2022.
[37] Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object detection. In CVPR, pages 779-788, 2016.
[38] Joseph Redmon and Ali Farhadi. Yolo9000: better, faster, stronger. In CVPR, pages 7263-7271, 2017.
[39] Joseph Redmon and Ali Farhadi. Yolov3: An incremental improvement. arXiv preprint orXiv:/804. 02767, 2018.
[40] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. NeurIPS, 28, 2015.
[41] Sehastian Ruder. An overview of gradient descent optimization algorithms. arXiv preprint 1rXiv:1609.04747, 2016.
[42] Steven A Shafer. Using color separate reflection components. Color Research & Application. 10(4):210-218) 1985.
[43] Shangquan Sun, Wenqi Ren, Tao Wang, and Xiaochun Cao. Rethinking image restoration for object detection. NeurlPS, 35:4461-4477 2022.
[44] Zhi Tian, Chunhua Shen, Hao Chen, and Tong He. Fcos: Fully convolutional one-stage object detection. In ICCV, pages 9627-9636 2019.
[45] Biao Wang, Weifeng Li, Wenming Yang, and Qingmin Liao. Illumination normalization based weber's law with application face recognition. IEEE Signal Processing Letters, 18(8):462-44) 465,,4655552 2011.
[46] Xiaogang Xu, Ruixing Wang, and Jiangbo Lu. Low-light image enhancement via structure modeling and guidance. In CVPR, pages 9893-9903, 2023.
[47] Shuzhou Yang, Moxuan Ding, Yanmin Wu, Zihan Li, and Jian Zhang. Implicit neural represen- tation for cooperative low-light image enhancement. In ICCV, pages 12918-12927, 2023.
[48] Wenhan Yang. Ye Yuan, Wenqi Ren, Jiaying Liu, Walter Scheirer, Zhangyang Wang, Taiheng Zhang, Qiaoyong Zhong, Di Xie. Shiliang Pu, al. Advancing image understanding poor visibility environments: A collective benchmark study. IEEE Transactions on Image Processing. 29:5737-5752, 2020.
[49] Xiangchen Yin, Zhenda Yu, Zetao Fei, Wenjun Lv, and Xin Gao. Pe-yolo: Pyramid enhancement network for dark object detection. In International Conference on Artificial Neural Networks, pages 163-174. Springer, 2023.
[50] Hongkai Zhang, Hong Chang, Bingpeng Ma, Naiyan Wang, and Xilin Chen. Dynamic r-enn: Towards high quality object detection via dynamic training. In ECCV, pages 260-275. Springer, 2020.
[51] Kaipeng Zhang, Zhanpeng Zhang, Zhifeng Li, and Yu Qiao. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE signal processing letters, 23(10): 1499 1503, 2016.
[52] Shifeng Zhang, Cheng Chi, Yongqiang Yao, Zhen Lei, and Stan Z Li. Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection. CVPR, pages 9759-9768, 2020.
[53] Yonghua Zhang, Jiawan Zhang, and Xiaojie Guo. Kindling the darkness: A practical low-light image enhancer. In ACM MM, pages 1632-1640, 2019.
12

---

A Appendix supplemental material
Derivation of IIM. Referring the Eq. 5 in the main text, the IIM defines a feature extracted from neighboring pixels. Consider a convolutional kernel W where A represents the kernel size. Here, ps and w, denote pixel position and its associated weight within the kernel W, with ranging from 1 to A.?
Rpi Bp
Bputs Rp
Rps B p
Mrb
(10)
log(M,b)
Σ
wydog(Ro
Σ
wylog( ( Bp.)
w, (l (x)) ) + logle Ros (A)) + log(p Ro: (A)))
w. (:(log (mmmmm(ppp (.)) + log(e Bos (A)) + log(p Brs (A)))
(II)
Σ welog(e Rog (A))
welog(e
+
wrlog(p Bus (A))
wrlog( Bys (A))
To eliminate the term, it is imperative to adhere to the following constraints::
w.log(eE.. Roy (A)) = welogle Brs (A)) =
(12)
Assuming that all pixels in a given convolutional kernel are neighboring pixels, we obtain e se Rey where i = 1,2, ... ,k2,1 + í. The above constraints can be equivalently expressed as Σ w. 0
Illumination Invariant Loss. As mentioned in Sec. 4.6.3 to optimally constrain the kernel learning process and harness the full potential of illumination-invariant information, further employ consistency loss, denoted as Illumination Invariant Loss (II Loss). This loss function is specifically designed to align features extracted from pairs of images taken under different lighting conditions. The fundamental concept of the II Loss is to guarantee consistency in the features extracted from these images, regardless of the variations in illumination. This is achieved by leveraging luminance transformation function to adjust the illuminations, as defined as follows:
L (t) - fw. (
(fw - fw. (o(()) 8
(13)
(T) (w. - 23, otherwise.
In our experiments, we use the gamma transformations as the function for the function setting B empirically to 1, and scaling the Loss to 0.01 the other losses.
As discussed in Sec. 3.1 exhibit high similarity of illumination. Specifically, illumination items can be factored out by performing the subtraction among the neighboring pixels, which is accomplished by imposing the zero mean constraint on the convolutional kernels. However, eliminate the illumination term ideally. it is necessary for the average value of adjacent positions within the kemnel to approach zero. zero mean does not guarantee that the illumination elimination occurs strictly between adjacent pixels; it can occur between distant pixels as well. For instance, Fig. 5 presents an example of a convolutional kernel that satisfies the zero-mean constraint. Even though this kernel has zero mean, it fails to extract -nviinatiiionniiiantttt features due to the relatively large spatial
we obtain Illminatiooninvariie features by assuming neighboring pixels
The sole constraint
13

---

Dataset
II-Loss
YOLOv3
TOOD
71.0
72.3
3)
<S<<
71.1
74.8
4)
Exdark
72.7
75.0
71.5
75.0
72.7
75.2
(4) Weight
wio
60.0
62.1
61.0
66.9
Figure 5: Illustration of trivial case (a), and visualization of performing 3 3 mean filter- 10) ing on the kernel weights guided by with (b) and without (c) II loss.
DarkFace
61.5
67.4
60.2
65.8
I1)
60.7
67.1
Figure Ablation study of YOLOv3-based and TOOD-based detectors ExDark and U FACE datasets, where K. denotes the kernel size within IIM.
Method
Bicycle
Boat
Bottle
Bus
Car
Chair
Cup
Dog
Motorbike
People
Table
mAPsa
Baseline
79.8
72.1
70.9
82.8
79.5
64.4
67.6
70.6
79.5
62.4
77.7
44.2
71.0
77.5
70.2
70.6
77.3
KIND 5
80.2
74.4
71.5
81.0
80.3
62.2
61.3
67.5
75.8
62.1
75.9
40.9
69.4
Zero-DCE
81.8
70.1
79.5
61.0
71.7
78.4
62.9
77.3
43.1
71.1
EnlightenGAN
81.1
74.2
69.8
83.3
78.3
63.3
65.5
69.3
75.3
62.5
76.7
41.0
70.0
RUAS 20
76.4
62.7
77.3
59.0
64.8
73.1
55,8
71.5
65.7
SC1 135
80.3
74.2
73.6
82.8
78.4
64.4
65.8
71.3
78.1
62.7
78.2
42.4
71.0
NeRCo
80.8
66.3
81.3
75.6
62.8
67.7
75.6
61.8
75.1
68.5
SMG 66
78.1
72.1
65.8
81.6
78.3
63.7
64.5
67.6
76.3
57.4
73.7
42.4
68.5
DENet 36
81.1
75.0
73.9
87.1
79.7
63.5
69.6
76.3
61.4
76.7
44.9
71.3
PENet
49
76.5
71.9
67.4
84,2
78.0
59.9
64.6
66.7
74.8
62.5
73.9
45.1
68.8
MAET
81.5
73.7
74.0
88.2
80.9
68.8
66.9
71.8
60.2
78.8
46.3
72.5
82,4
72.7
85.4
71.5
81.8
65.2
78.6
45.7
72.7
Table 6: Quantitative comparisons of the ExDark dataset based on YOLOv3 detector.
separation between the positive and negative positions. Unfortunately. the convolutional kernel size increases, this issue becomes more pronounced and leads to degradation performance. To this end, the II Loss is proposed to encourage consistency of outputs from IIM across images with different illuminations, preventing trivial solutions within the kernel implicitly. As shown in Fig. (b)))), we visualize the 5 5 kernel with and without the II Loss. For each position in the 5 x 5 kernel, compute the mean value using a sliding window of size 3 3. can be observed that without the Loss, the local means within the kernel do not tend towards zero, indicating that the features extracted In contrast, when the Loss is applied, the local
using this kernel are may not illumintiooniivarian values of the kernel are significantly constrained. To further validate the effectiveness of the II Loss, we present the results the Loss on different convolutional kernel sizes within IIM rows 3-66 and 8~11 1 of Table By comparing the performance gains of different convolutional kernel sizes, we can see that the larger kernel sizes lead to more significant improvements. This strongly suggests the effectiveness of our II Loss in constraining the degrees of freedom in the kernel.
Detailed Results on ExDark. In this section, we report the average precision for each category of the ExDark dataset shown in Table 6 and 7 Note that, we further introduce more advanced LLIE methods, including Zero-DCE [17], EnlightenGAN (24, SCI (35), and NeRCo 47 based on YOLOv3 and TOOD. Unfortunately, despite the outstanding performance exhibited by these LLIE methods in image restoration tasks, they struggle with effectively enhancing specific downstream tasks. For example, the state-of-the-art LLIE method, NeRCo, exhibits the worst performance compared to other LLIE methods. This phenomenon further proves the existence of the gap between optimization goals for image restoration and object detection tasks. Additionally, compared to end-to-end approaches, such cascade paradigms limit the potential for deploying these LL.IE-based low-light detection techniques practical applications.
YOLA vs. FeatEnHancer. For fair comparison, we follow the FeatEnHancer''s 18 experimental setting implement the RetinaNet [27 -baed detectors as shown in Table We can see that though our baseline implementation the ExDark dataset is inferior to Fnancers,,,''s, the

---

Method
Bicycle
Bottle
Bus
Car
Chair
Cup
Dog
Motorbike
People
Table
mAPa Pa
Baseline
80.6
75.8
71.1
88.1
76.8
70.4
66.8
69.2
85.4
61.5
76.1
48.2
72.5
MRLLEN
80.8
77.8
72.8
89.3
78.7
73.5
67.5
69.4
85.2
62.9
77.3
47.2
73.5
KIND S3
81.7
77.7
70.3
88.4
78.1
69.7
67.2
67.8
84.1
61.6
76.6
47.8
72.6
7cro- DCF 1+
81.8
79.0
72.9
89.6
77.9
71.9
68.5
69.8
84.8
62.9
78.0
49.5
73.9
EnlightenGAN
24
80.7
77.6
70.4
88.8
76.9
70.6
67.9
68.7
84.4
62.2
77.5
49.6
73.0
RUAS A9
78.4
74.3
67.4
85.1
72.4
67.7
67.3
65.2
77.9
56.1
73.4
47.0
69.4
SC1
81.3
78.1
71.6
89.4
77.6
71.1
68.0
70.9
85.0
63.0
77.2
73.5
NeRCo
47
78.8
75.6
70.8
87.6
75.7
69.1
66.8
69.5
82.5
59.9
76.0
49.3
71.8
SMG
78.2
75.9
69.9
87.3
75.1
71.3
66.5
67.2
84.2
60.1
75.1
71.5
DENet 36
80.9
78.2
70.9
88.3
77.5
71.6
67.2
70,3
87.3
62.0
77.3
49.9
73.5
PENet 49
76.0
72.3
66.7
84.4
72.2
65.4
63.3
65.8
79.1
53.1
71.0
67.8
MAET
7
80.5
77.3
74.0
90.1
78.3
73.4
69.6
70.7
86.6
64.4
77.6
48.5
74.3
83.9
78.7
75.3
88.8
79.0
73.4
69.9
71.9
86.8
66.3
78.3
49.8
75.2
Table 7: Quantitative comparisons the ExDark dataset based on TOOD detector.
100
LO
20
KE ME
10:00
400
MTT BURTIR
MEX
Baseline
Zero-DCE
EMighennGAN
RUAS
TUMO
1000
20)
T20
110
L2
100
1000
TODY
10000
MT PM
NeRCo
X
BUAS
SCI
NeRCo
DENet
Ours
Figure 7: Qualitative comparisons of Mask R- R-CNN-BA based detector on LIS dataset. Our YOLA shows more comprehensive segmentation and detection results, with an increased number of bottles detected (top 2 rows) and successful recognition of the challenging car (right side of the bottom 2 rows). Best viewed with zooming
integration of YOLA enables method to achieve the best performance (1.9 mAP significant improvement compared to baseline). For UG* FACE dataset, FeatEnHancer decreases the baseline performance by 0.1 mAP, which is attributed to hierarchical features that failed to be captured by RetinaNet, as claimed [18]. In contrast, our YOLA, triggered from the physics-based model perspective without elaborate design, surpassing the baseline with remarkable improvement of 2.5 mAP. strongly suggests the generalizability and effectiveness of YOLA.
More Visualization. In Fig. and additional visual results presented, showcasing selected challenging cases. In comparison to other methods, our method exhibits superior recall and more precise segmentation performance under extreme low-light conditions.
YOLA on Lowl-light Instance Segmentation. further explore YOLA's capabilities, we also evaluate YOLA in the low-light instance segmentation tasks. We report the quantitative comparisons
15

---

Dataset
Method
mAPs0
Method
AP
AP
AP
AP
AP
APT:
Buscline
72.1
Roseline
34.2
14.7
63.9
w/ Hancer 18
72.6(+0.5)
Exdark
Baseline'
70.9
PENe
Zern-IXC
17
36.1
38.7
62.0
3644
390
46.4
70.0
67.3
47.1
50.9
w/ YOLA
72.8(+1.9)
EnlightenGAN
24
38.4
61.5
39.2
45.8
09.5
49.7
RUAS
36.1
58.6
36.4
66.7
Baseline
47.3
w/ Heater 18
47.2(-0.1)
DarkFace
Bascline
$0.2
SMG (56
37.4
60.3
38.7
44.7
67.4
49.2
w/ YOLA
52.7(+2.5)
Oun
39.8
63.5
41.4
47.5
70,9
$1.8
Table 8: Quantitative comparisons (YOLA vs. Table Quantitative comparisons of the LIS FeatEnHancer) of ExDark and UG +DAR dataset based on Mask RCNN, where Apeeg and FACE datasets based on RetinaNet. Red and Ap indicate the average precision of segmenta- blue colors represent improvement and degra- tion and detection, respectively. dation of performance, respectively, compared the baseline. f indicates our implemented baseline.
Baseline
Zero-DCE
EnkihhenGANN
RUAS
ACKO
SCT
NeRCo
DENe
Qurs
Buseline
Zero-DCE
EnlichienGAN
RUAS
SCI
NeRCo
DENet
Ours
Figure 8: Qualitative comparisons of Mask R-CNN-based detector on LIS dataset. Our YOLA outperforms LLIE-based and low-light object detection methods. Best viewed with zooming in.
of several advanced LLIE and low-light object methods using Mask R-CNN 19 on the low-light instance segmentation (LIS) (5) datasct, as shown Table We can see that our YOLA achieves the best performance across all metrics, indicating that YOLA not only facilitates low-light object detection but also low-light instance segmentation.
16

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility. transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No] or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
jusiifica)usiffcctttioo Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked use the checklist one of the factors in their evaluation. While "[Yes] " is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] provided a proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can found.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
See abstract and introduction.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [NA]
Justification: [TODO]
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
17

---

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
We include the assumptions in Section 3.1, and complete proofs of IIM in appendix.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Thcorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: See section 4.1
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often good way to accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
18

---

(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should descrihe the architecture clearly and fully.
(c) the contribution is a new model (e.g.. a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (c.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers have some path reproducing verifying the results. 5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: This work only use public datasets, and we provide the code in supplemental materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https for more details.
7/nips. //nips.cc/publl. wwtpc//ggs/////eeeocooococ pubbicc/guuddds/dddeuubiiiii wwt.sSuumiissonnP..
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: See section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
19

---

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
Justification: [TODO]
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report I-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (c.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
See the experiment section.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: [TODO]
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
20

---

10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: [TODO]
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: [TODO]
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
See section 4.1.
Guidelines:
21

---

The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: [TODO]
Guidelines:
The answer NA means that the paper does release assets.
Researchers should communicate the details the dataset/code/mooel as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well details about compensation (if any)?
Answer: [No]
Justification: [TODO]
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based on the requirements of your country or institution) were obtained?
Answer: [No]
Justification: [TODO]
22

---

Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
23


Reviews:
[{"review_id": "dSPEWL2ql6", "reviewer": "Reviewer_krE6", "summary": "This paper proposes a plug-and-play module for extracting illumination-invariant features from low-light images. By integrating a zero-mean constraint within the module, a diverse set of kernels is effectively learned. These kernels excel at extracting illumination-invariant features, thereby enhancing detection accuracy. Experiments on object detection and semantic segmentation tasks demonstrate the effectiveness of the module.", "questions": "There are some questions regarding the training loss. The paper does not provide the training loss of the model. Are the components for learning illumination-invariant features and the object detection network trained jointly, or is the illumination-invariant feature learning component trained separately? If trained separately, considering that the ExDark and DarkFace datasets consist only of annotated low-light images without corresponding normal-light images, is this training unsupervised? The authors should provide detailed information about the training process in the paper.", "limitations": "see weakness", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. The authors design a Illumination-Invariant Module to extract illumination-invariant features without requiring additional paired datasets, and can be seamlessly integrated into existing object detection methods.\n2. Lambertian assumption is introduced, which enhances the interpretability of the model.\n3. The authors claim that the proposed method achieves state-of-the-art performance on several benchmark datasets for object detection.", "weaknesses": "1. The authors assume uniform illumination between neighboring pixels to eliminate the influence of the positional term 𝑚 in Equation 1. However, images captured in real-world scenes often exhibit uneven lighting. I question the validity of this assumption.\n2. The paper does not provide definitions for the symbols used in Equation 2.\n3. What does \"Baseline\" refer to in Tables 1 and 2, and why is the object detection performance based on low-light enhancement methods worse than the Baseline?\n4. Many end-to-end low-light face detection algorithms have been proposed. The authors should compare their method not only with general object detection algorithms but also with these specialized low-light face detection algorithms on the DarkFace dataset.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**We would like to thank the reviewer for carefully reading our submission and providing many insightful comments.**\n\n> **The authors assume uniform illumination between neighboring pixels to eliminate the influence of the positional term $m$ in Equation 1. However, images captured in real-world scenes often exhibit uneven lighting. I question the validity of this assumption.**\n\nIn the main text, we assume that the illumination values of neighboring pixels are equal, which allows us to eliminate the influence of illumination when extracting features. However, in cases of uneven illumination, this assumption's constraint is weakened but still helps reduce the impact of illumination, as shown in teaser (b). Our method is detection-driven and can further mitigate the influence of such uneven illumination during the learning process, as illustrated in teaser (d). Additionally, please refer to the appendix. When the actual distance between neighboring pixels is too large, the assumption may not hold. Therefore, we propose IIloss to constrain the extraction of illumination-invariant features to as close a region as possible, mitigating the impact of uneven lighting.\n\n---\n\n> **The paper does not provide definitions for the symbols used in Equation 2.**\n\nWe apologize for the oversight. In the revised version, we will include the corresponding explanations. $B$ denotes the blue channel, whereas $R$ represents the red channel.\n\n---\n\n> **What does \"Baseline\" refer to in Tables 1 and 2, and why is the object detection performance based on low-light enhancement methods worse than the Baseline?.**\n\n'Baseline' refers to the detector that our method is built upon. The difference between 'Baseline' and 'Ours' is that we utilized the IIM (Illumination Invariant Module). \nLowlight image enhancement methods may fail to achieve satisfactory performance due to the task of primarily satisfying human vision instead of machine perception. This phenomenon is also claimed by many related works[1,2,3].\n\n---\n\n> **Many end-to-end low-light face detection algorithms have been proposed. The authors should compare their method not only with general object detection algorithms but also with these specialized low-light face detection algorithms on the DarkFace dataset.**\n\nWe explored several specialized low-light face detection algorithms [4,5,6]. However, most of these algorithms are evaluated on the DarkFace test set that does not provide Ground Truth, or lacks open-source code. Consequently, we selected some representative face detection algorithms adopted by these methods as our baselines, including DFSD[7] and PyramidBox[8]. We implemented our YOLA using their default settings for a fair comparison, as shown in Table 1. Our YOLA outperforms the baselines, achieving 0.6 and 0.8 higher mAP on PyramidBox and DFSD, respectively, demonstrating its generalization ability in the face detection task.\n\n---\n\n> **There are some questions regarding the training loss. The paper does not provide the training loss of the model. Are the components for learning illumination-invariant features and the object detection network trained jointly, or is the illumination-invariant feature learning component trained separately? If trained separately, considering that the ExDark and DarkFace datasets consist only of annotated low-light images without corresponding normal-light images, is this training unsupervised? The authors should provide detailed information about the training process in the paper.**\n\nOur YOLA model is trained without any additional image pair annotations using an end-to-end joint training fashion. Specifically, the features produced by the IIM are supposed to be inherently illumination invariant at initialization, based on the Lambertian assumption. Thus, we do not require any additional normal light images or other loss such as Brightness loss to guide its learning. We only employ detection loss to guide the IIM in producing task-specific illumination invariant features for downstream tasks.\n\n---\n| Detector                             | Baseline | Ours         |\n|--------------------------------------|----------|--------------|\n| PyramidBox| 47.7     | **48.3**     |\n| DFSD         | 44.9     | **45.7**     |\n\n**Table 1:** Face detection algorithms on DarkFace Dataset\n\n---\n\n> **Reference**:\n\n> [1] Cui et al. Multitask aet with oerthogonal tangent regularity for dark object detection, ICCV 2021.\n\n> [2] Khurram et al. Featenhancer: Enhancing hierarchical features for object detection and beyond under low-light vision, ICCV 2023.\n\n> [3] Qin et al.  Denet: Detection-driven enhancement network for object detection under adverse weather conditions. ACCV 2020.\n\n> [4] Wang et al. Unsupervised face detection in the dark. T-PAMI 2022.\n\n> [5] Wang et al. Hla-face: Joint high-low adaptation for low light face detection. CVPR 2021.\n\n> [6] Yu et al. Single-stage face detection under extremely low-light conditions. ICCVW 2021.\n\n> [7] Li et al.  Dsfd: dual shot face detector. CVPR 2019.\n\n> [8] Tang et al. Pyramidbox: A context-assisted single shot face detector. ECCV 2018.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5OXHHVrh8H", "reviewer": "Reviewer_F2ap", "summary": "In this paper, the authors propose an object detection method in low-light scenarios that is based on illumination-invariant feature learning. Additionally, the extraction of illumination-invariant features from low-light images, which can be easily integrated into existing object detection frameworks, The results reveal significant improvements in low-light object detection tasks as well as promising results in both well-lit and over-lit scenarios.", "questions": "See the weaknesses", "limitations": "See the weaknesses", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.This paper introduces an new object detection in low-light conditions by leveraging illumination-invariant features.\n2. The writing is well done and well organized.\n3. The Illumination-Invariant Module seems to be a plug and play module that is very useful.", "weaknesses": "1. As a plug and play module, I think the lighting invariant module should be integrated into more detectors to prove its effectiveness.\n2. The authors claim that they learned light invariant features, and then models trained in low light conditions should be able to generalize directly to normal light conditions. And vice versa. The authors should provide more experiments to demonstrate the generalization ability of their light invariant features.\n3.The authors should provide evaluation content such as runtime and memory used by the run, it seems that their approach is more lightweight.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**We sincerely thank you for your insightful and constructive comments.**\n\n>**As a plug and play module, I think the lighting invariant module should be integrated into more detectors to prove its effectiveness.**\n\nWe report more detectors as shown in Table 1. It contains the **Anchor-based** detector (Faster-RCNN[1], RetinaNet[2], YOLOV3[3]), and the **Anchor-free** one (TOOD[4], Sparse-RCNN[5]). By integrating YOLA, the performances of all detectors are significantly improved, demonstrating the generalization capability of YOLA.\nBesides, please refer to the appendix, we also evaluate our YOLA on the instance segmentation task, demonstrating YOLA’s effectiveness on Mask R-CNN detector.\n\n---\n\n>**The authors claim that they learned light invariant features, and then models trained in low light conditions should be able to generalize directly to normal light conditions. And vice versa. The authors should provide more experiments to demonstrate the generalization ability of their light invariant features.**\n\nWe complement the more generalization experiments as presented in Table 2, where $GAP$ represents mAP$_{50}$ gap between in-domain and cross-domain. We trained detectors separately in low-light (ExDark) and normal-light(Pascal VOC 2012[6]) conditions and then tested them in the opposite conditions.\nWe can see that our YOLA achieves the best performance across different domains and shows a smaller performance drop when tested on cross-domain.\n\n---\n\n>**The authors should provide evaluation content such as runtime and memory used by the run, it seems that their approach is more lightweight.**\n\nWe have included a comparison of runtime, model size, and performance in Table 3. The inference speeds, evaluated on an RTX 2080Ti using the MMdetection toolbox, indicate that YOLA introduces the fewest additional parameters (0.008M) while achieving the highest FPS and outstanding performance among the evaluated detectors, excluding the baseline.\n\n---\n\n| Detector     | mAP$_{50}$(ExDark) | mAP$_{50}$(DarkFace) | Types         |\n|--------------|--------------------|----------------------|---------------|\n| YOLOv3       | 71.0               | 60.0                 | **Single Stage**  |\n| +YOLA        | **72.7**           | **61.5**             |               |\n|--------------|--------------------|----------------------|---------------|\n| RetinaNet    | 70.9               | 50.2                 | **Single Stage**  |\n| +YOLA        | **72.8**           | **52.7**             |               |\n|--------------|--------------------|----------------------|---------------|\n| FasterRCNN   | 71.4               | 43.0                 | **Two Stage**     |\n| +YOLA        | **72.5**           | **44.6**             |               |\n|--------------|--------------------|----------------------|---------------|\n| SparseRCNN   | 63.5               | 43.5                 | **Two Stage**     |\n| +YOLA        | **68.7**           | **52.8**             |               |\n|--------------|--------------------|----------------------|---------------|\n| TOOD         | 72.5               | 62.1                 | **Single Stage**  |\n| +YOLA        | **75.2**           | **67.4**             |               |\n|--------------|--------------------|----------------------|---------------|\n\n**Table 1:** YOLA with more detectors.\n\n\n---\n\n| Detector | Training Set   | Testing Set  | mAP$_{50}$ ↑| Testing Set  | mAP$_{50}$↑ | GAP ↓ |\n|----------|----------------|--------------|------------|--------------|------------|------|\n| YOLOv3   | ExDark_train   | ExDark_test  | 71.0       | VOC_val      | 57.6       | 13.4 |\n| +YOLA    | ExDark_train   | ExDark_test  | **72.7**   | VOC_val      | **60.5**   | **12.2**|\n|----------|----------------|--------------|------------|--------------|------------|------|\n| YOLOv3   | VOC_train      | VOC_val      | 78.8       | ExDark_test  | 57.4       | 21.4 |\n| +YOLA    | VOC_train      | VOC_val      | **78.9**   | ExDark_test  | **58.5**   | **20.4** |\n\n**Table 2:** Generalization comparison.\n\n---\n| Detector                           | mAP  | Size(M) | FPS  |\n|------------------------------------|------|---------|------|\n| Baseline        | 72.5 | 32.044  | 57.7 |\n| IAT[7]                | 73.0 | 32.135  | 50.9 |\n| IAYOLO[8]        | 65.0 | 32.209  | 52.5 |\n| GDIP[9]            | 72.8 | 167.00  | 54.0 |\n| DENet[10]          | 73.5 | 32.089  | 55.7 |\n| PEYOLO[11]      | 67.8 | 32.135  | 38.8 |\n| Ours             |75.2 | 32.052  | 56.6 |\n\n**Table 3:** Quantitative comparisons of the ExDark dataset based on TOOD detectors\n\n---\n\n**Reference**:\n\n>[1] Ren et al.  Faster r-cnn: Towards real-time object detection with region proposal networks. NeurIP2015.\n\n>[2] Lin et al.  Focal loss for dense object detection. ICCV 2017.\n\n>[3] Redmon et al.  Yolov3: An incremental improvement. ArXiv.\n\n>[4] Feng et al. Tood: Task-aligned one-stage object detection. ICCV 2021.\n\n>[5] Sun et al. Sparse r-cnn: End-to-end object detection with learnable proposals. CVPR 2021.\n\n>[6] Everingham et al. The {PASCAL} {V}isual {O}bject {C}lasses {C}hallenge 2012 {(VOC2012)\n\n>[7] Cui et al. You only need 90k parameters to adapt light: a light weight transformer for image enhancement and exposure correction. BMVC 2022.\n\n>[8] Liu et al. Image-adaptive yolo for object detection in adverse weather conditions. AAAI 2022.\n\n>[9] Sanket et al. Gdip: Gated differentiable image processing for object detection in adverse conditions. ICRA 2023.\n\n>[10] Qin et al. Denet: Detection-driven enhancement network for object detection under adverse weather conditions. ACCV 2020.\n\n>[11] Yin et al. Pe-yolo: Pyramid enhancement network for dark object detection. ICANN 2023", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WjVEZuhTFB", "reviewer": "Reviewer_yjJM", "summary": "This paper proposes YOLA, a framework for object detection in low-light conditions by leveraging illumination-invariant features. A novel Illumination-Invariant Module to extract illumination-invariant features for low-light image enhancement.", "questions": "See above.", "limitations": "Overall I think the idea of illumination invariant feature is simple, so the technical contribution is limited. The results of ``ours`` in Figure 3-4 are strange, the images are very cloudy. It is very different from the example in the pipeline.", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "Figures are helpful for understanding. \nThe proposed method gives better performance than others.", "weaknesses": "RG/RB/GB-chromaticity is widely used in computer vision; learning-based methods for intrinsic images or inverse rendering also adopt the constraint. It deals with light intensity but does not try to solve the image noises in low light conditions. Does this paper consider noise reduction in the pipeline? \n\nIllumination invariant features based on chromaticity have ambiguities between colors with the same chromaticity, e.g. dark red or light red. Such limitations are not discussed. \n\nIn experiments, ``LLIE methods fail to achieve satisfactory  performance due to inconsistency between human visual and machine perception. The enhancement methodologies prioritize human preferences. However, it is important to note that optimizing for enhanced visual appeal may not align with optimized object detection performance``. It is hard to understand without visual examples. \n\nIn the pipeline, the work is basically low-light enhancement + detector. In evaluations, only object detection is evaluated, why not evaluate on low-light enhancement datasets/benchmarks? \n\nThe proposed features are claimed to fit many tasks. More tasks other than detection can be demonstrated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**We sincerely thank you for your constructive comments.**\n\n---\n\n>**In the pipeline, the work is basically low-light enhancement + detector. In evaluations, only object detection is evaluated, why not evaluate on low-light enhancement datasets/benchmarks?**\n\n` First and foremost, we need to emphasize that our framework is designed to enhance the performance of object detectors in low-light scenarios, rather than for visual brightening or denoising` . \n\nHuman visual alignment can improve detection performance, but it is not the only solution. From a machine vision perspective, detection does not depend on achieving image quality that aligns with human vision. Despite the presence of noise and blurriness in low-light images, this does not hinder our method from improving performance in detection tasks.\n\nEssentially, our method is object detection rather than image enhancement, which means comparing our results based on image enhancement benchmarks is unfair.\n\nMore specifically, our framework does not employ any extra low-lit and normal-lit image pairs or other image restoration-related loss to enhance the input images. We only leverage detection loss to guide IIM in producing task-specific illumination invariant features. Therefore, improving visual quality is not our goal. Interestingly, we found that the subsequent enhanced image yielded by FuseNet tends to display increased brightness.\n\n\n\n---\n\n>**In experiments, LLIE methods fail to achieve satisfactory performance due to inconsistency between human visual and machine perception. The enhancement methodologies prioritize human preferences. However, it is important to note that optimizing for enhanced visual appeal may not align with optimized object detection performance. It is hard to understand without visual examples.**\n\nAs we have shown, some of the images we present are cloudy, which exemplifies the discrepancy between machine vision and human vision. In LLIE (Low-Light Image Enhancement) methods, many loss functions are designed based on human prior assumptions, such as illumination smoothness and color consistency in Zero-DCE, or TV loss in denoising. These losses often lead to the loss of many details while preserving details preferred by human perception. This difference may result in LLIE methods performing poorly in detection tasks, as discussed in related works[1, 2, 3].\n\n---\n\n>**RG/RB/GB-chromaticity is widely used in computer vision; learning-based methods for intrinsic images or inverse rendering also adopt the constraint. It deals with light intensity but does not try to solve the image noises in low light conditions. Does this paper consider noise reduction in the pipeline?**\n\n>**Illumination invariant features based on chromaticity have ambiguities between colors with the same chromaticity, e.g. dark red or light red. Such limitations are not discussed.**\n\nEnhancing image quality, such as reducing noise and improving chromaticity distinction, is a practical strategy, but it is not the only one for improving detection performance. \nIn this work, We are committed to finding an end-to-end approach to enhance downstream detection tasks, rather than focusing on image restoration. \n\nBesides, improving both image quality and detection performance typically requires paired annotated datasets, which presents significant challenges for practical applications.\nTherefore, we strongly believe that developing simpler and more efficient end-to-end methods to reduce the burden of data annotation will greatly benefit the community.\n\n---\n\n>**The proposed features are claimed to fit many tasks. More tasks other than detection can be demonstrated.**\n\nWe present our evaluation of YOLA on the instance segmentation task in Table 1. We report the quantitative comparisons of several advanced LLIE and low-light object methods using Mask R-CNN on the low-light instance segmentation (LIS[10]) dataset. We can see that our YOLA achieves the best performance across all metrics, indicating that YOLA not only facilitates low-light object detection but also low-light instance segmentation. For more visual comparison, please refer to our appendix.\n\n---\n\n| Method                           | AP$^{seg}$ | AP$^{box}$ |\n|----------------------------------|------------|------------|\n| Baseline                         | 34.2       | 41.3       |\n| DENet[3]       | 38.6       | 46.4       |\n| PENet[4]         | 36.1       | 43.6       |\n| Zero-DCE[5]      | 38.7       | 46.4       |\n| EnlightenGAN[6] | 38.4 | 45.8       |\n| RUAS[7]      | 36.1       | 43.8       |\n| SCI[8]         | 36.5       | 44.3       |\n| NeRCo[9]   | 36.7       | 44.6       |\n| **Ours**                         | **39.8**   | **47.5**   |\n\n**Table 1:** Quantitative Comparisons.\n\n\n**Reference**\n>[1] Cui et al. Multitask aet with oerthogonal tangent regularity for dark object detection, ICCV 2021.\n\n>[2] Khurram et al. Featenhancer: Enhancing hierarchical features for object detection and beyond under low-light vision, ICCV 2023.\n\n>[3] Qin et al. Denet: Detection-driven enhancement network for object detection under adverse weather conditions. ACCV 2020.\n\n>[4]  Yin et al. Pe-yolo: Pyramid enhancement network for dark object detection. ICANN 2023.\n\n>[5] Guo et al. Zero-reference deep curve estimation for low-light image enhancement. CVPR 2020.\n\n>[6] Jiang et al. Enlightengan: Deep light enhancement without paired supervision. TIP 2021.\n\n>[7] Liu et al. Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement. CVPR 2021.\n\n>[8] Ma et al. Toward fast, flexible, and robust low-light image enhancement. CVPR 2022.\n\n>[9] Yang et al. Implicit neural representation for cooperative low-light image enhancement. CVPR 2023.\n\n>[10] Chen et al. Instance segmentation in the dark. IJCV 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sxVESuKI7V", "reviewer": "Reviewer_N71E", "summary": "The paper focuses on the Low-light Object Detection task from the perspective of feature learning. In detail, the paper proposes an Illumination-Invariant Module to extract illumination-invariant features and a learning illumination-invariant paradigm. Experiments verify the effectiveness of the proposed method.", "questions": "How does the method perform in comparison to recent related methods [1-7]?\nThe proposed method assumes neighboring pixels exhibit high similarity of illumination. However, how does it hold true when meeting uneven light at night?", "limitations": "The proposed method assumes neighboring pixels exhibit high similarity of illumination. However, how does it hold true when meeting uneven light at night?", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The writing is good with beautiful illustrations.\n2. The idea is novel and interesting. The paper leverages illumination-invariant features to detect objects in low light, which is novel for the low-light object detection task. The proposed method is easy to follow and achieves good performance.", "weaknesses": "More related methods need to be compared in Table 1 and Table 2. \n[1] Ziteng Cui, Kunchang Li, Lin Gu, Shenghan Su, Peng Gao, Zhengkai Jiang, Yu Qiao, and Tatsuya Harada. You only need 90k parameters to adapt light: a light weight transformer for image enhancement and exposure correction. In BMVC, page 238, 2022.\n[2] Shangquan Sun, Wenqi Ren, Tao Wang, and Xiaochun Cao. Rethinking image restoration for object detection. Advances in Neural Information Processing Systems, 35:4461–4474, 2022.\n[3] Wenyu Liu, Gaofeng Ren, Runsheng Yu, Shi Guo, Jianke Zhu, and Lei Zhang. Image-adaptive yolo for object detection in adverse weather conditions. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 1792–1800, 2022.\n[4] Sanket Kalwar, Dhruv Patel, Aakash Aanegola, Krishna Reddy Konda, Sourav Garg, and K Madhava Krishna. Gdip: Gated differentiable image processing for object detection in adverse conditions. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 7083–7089. IEEE, 2023.\n[5] Qingpao Qin, Kan Chang, Mengyuan Huang, and Guiqing Li. Denet: Detection-driven enhancement network for object detection under adverse weather conditions. In Proceedings of the Asian Conference on Computer Vision, pages 2813–2829, 2022.\n[6] Xiangchen Yin, Zhenda Yu, Zetao Fei, Wenjun Lv, and Xin Gao. Pe-yolo: Pyramid enhancement network for dark object detection. In International Conference on Artificial Neural Networks, pages 163–174. Springer, 2023.\n[7] Khurram Azeem Hashmi, Goutham Kallempudi, Didier Stricker, and Muhammad Zeshan Afzal. Featenhancer: Enhancing hierarchical features for object detection and beyond under low-light vision. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6725–6735, 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**We sincerely thank you for your insightful and positive comments.**\n\n---\n\n>**How does the method perform in comparison to recent related methods?**\n\nWe complement more detailed comparison experiments as presented in Table 1, which includes runtime, model size, and performance. Note that our YOLA still achieves the best performance and speed among the evaluated methods.\n\nBesides, ReForDe[7] didn’t release their detailed training data, so we can not implement it. However, compared to the improvements reported by ReForDe in their paper ($\\leq$0.2 mAP on ExDark using YOLOv3), our YOLA demonstrates a more substantial enhancement, achieving improvements of 1.7 mAP and 2.7 mAP on YOLOv3 and TOOD, respectively.  \n\nAdditionally, FeatEnHancer[1] didn't release their code, we thus follow the FeatEnHancer’sexperimental setting to implement the RetinaNet-based detectors as shown in Table 2. We can see that even though our baseline implementation on the ExDark dataset is inferior to  FeatEnHancer’s, the integration of YOLA enables our method to achieve the best performance (1.9 mAP significant improvement compared to baseline). For DarkFace dataset, FeatEnHancer decreases the baseline performance by 0.1 mAP, which is attributed to hierarchical features that failed to be captured by RetinaNet, as claimed in [1]. In contrast, our YOLA, triggered from the physics-based model perspective without elaborate design, surpasses the baseline with a remarkable improvement of 2.5 mAP. It strongly suggests the generalizability and effectiveness of YOLA.\n\n---\n\n>**The proposed method assumes neighboring pixels exhibit high similarity of illumination. However, how does it hold true when meeting uneven light at night?**\n\nIn the main text, we assume that the illumination values of neighboring pixels are equal, which allows us to eliminate the influence of illumination when extracting features. However, in cases of uneven illumination, this assumption's constraint is weakened but still helps reduce the impact of illumination, as shown in teaser (b). Our method is detection-driven and can further mitigate the influence of such uneven illumination during the learning process, as illustrated in teaser (d). Additionally, please refer to the appendix. When the actual distance between neighboring pixels is too large, the assumption may not hold. Therefore, we propose IIloss to constrain the extraction of illumination-invariant features to as close a region as possible, mitigating the impact of uneven lighting.\n\n---\n\n| Detector                           | mAP  | Size(M) | FPS  |\n|------------------------------------|------|---------|------|\n| Baseline                           | 72.5 | 32.044  | 57.7 |\n| IAT[2]             | 73.0 | 32.135  | 50.9 |\n| IAYOLO[3]         | 65.0 | 32.209  | 52.5 |\n| GDIP[4]        | 72.8 | 167.00  | 54.0 |\n| DENet[5]          | 73.5 | 32.089  | 55.7 |\n| PEYOLO[6]            | 67.8 | 32.135  | 38.8 |\n| Ours                               | 75.2 | 32.052  | 56.6 |\n\n**Table 1:** Quantitative comparisons of the ExDark dataset based on TOOD detectors\n\n---\n\n| Dataset    | Method                                 | mAP$_{50}$           |\n|------------|----------------------------------------|----------------------|\n| Exdark     | Baseline                               | 72.1                 |\n|            | w/ FeatEnHancer | 72.6 (+0.5)                           |\n|            |-------------------------|---------------------------------|\n|            | Baseline$^{\\dagger}$                   | 70.9                 |\n|            | w/ YOLA                                | **72.8 (+1.9)**      |\n|----------|-------------------------|---------------------------------|\n| DarkFace   | Baseline                               | 47.3                 |\n|            | w/ FeatEnHancer | 47.2 (-0.1)                               |\n|            |-------------------------|---------------------------------|\n|            | Baseline$^{\\dagger}$                   | 50.2                 |\n|            | w/ YOLA                                | **52.7 (+2.5)**      |\n\n**Table 2:** Quantitative comparisons (YOLA vs. FeatEnHancer), $\\dagger$ indicates our implemented baseline.\n\n\n\n---\n\n**Reference**:\n\n>[1] Khurram et al. Featenhancer: Enhancing hierarchical features for object detection and beyond under low-light vision, ICCV 2023.\n\n>[2] Cui et al. You only need 90k parameters to adapt light: a light weight transformer for image enhancement and exposure correction. BMVC 2022.\n\n>[3] Liu et al. Image-adaptive yolo for object detection in adverse weather conditions. AAAI 2022.\n\n>[4] Sanket et al. Gdip: Gated differentiable image processing for object detection in adverse conditions. ICRA 2023.\n\n>[5] Qin et al. Denet: Detection-driven enhancement network for object detection under adverse weather conditions. ACCV 2020.\n\n>[6] Yin et al. Pe-yolo: Pyramid enhancement network for dark object detection. ICANN 2023.\n\n>[7] Sun et al.  Rethinking image restoration for object detection. NeurIPS 2022.\n\n>[8] Cui et al. Multitask aet with oerthogonal tangent regularity for dark object detection, ICCV 2021.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposed an end-to-end method for object detection in low light scenes using several constraints. Initially, three reviewers were positive, and several of them raised their scores after reviewing the author's feedback. Although the negative reviewer remains concerned about the lack of experiments on low-light enhancement, the AC reviewed the comments and the author's feedback and agreed with the authors' perspective. The paper can be accepted for publication at NeurIPS.
Decision: Accept (poster)