Title: End-to-End Low-Light Enhancement for Object Detection with Learned Metadata from RAWs
Venue: NeurIPS 2025
Accepted: yes
Keywords: Low-light Object Detection

RAW Metadata

Learned Image Compression

Cross-Modal Context

Image Refinement for Machine
Abstract: Although RAW images offer advantages over sRGB by avoiding ISP-induced distortion and preserving more information in low-light conditions, their widespread use is limited due to high storage costs, transmission burdens, and the need for significant architectural changes for downstream tasks. To address the issues, this paper explores a new raw-based machine vision paradigm, termed Compact RAW Metadata-guided Image Refinement (CRM-IR). In particular, we propose a Machine Vision-oriented Image Refinement (MV-IR) module that refines sRGB images to better suit machine vision preferences, guided by learned raw metadata. Such a design allows the CRM-IR to focus on extracting the most essential metadata from raw images to support downstream machine vision tasks, while remaining plug-and-play and fully compatible with existing imaging pipelines, without any changes to model architectures or ISP modules. We implement our CRM-IR scheme on various object detection networks, and extensive experiments under low-light conditions demonstrate that it can significantly improve performance with an additional bitrate cost of less than $10^{-3}$ bits per pixel.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel paradigm for low-light object detection called Compact RAW Metadata-guided Image Refinement (CRM-IR). The core idea is to leverage information from RAW camera sensor data without incurring the high storage and transmission costs typically associated with it. Instead of processing the full RAW image, the proposed method extracts a small amount of essential \"metadata\" from the RAW data. This metadata is then used by a lightweight module to refine a standard, compressed sRGB image, making it better suited for machine vision tasks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: The proposed solution is highly practical as it circumvents the high costs of transmitting and storing RAW images, which is a major bottleneck for many real-world applications, especially in edge-to-cloud scenarios. The plug-and-play nature of the proposed module makes it easily integrable into existing systems without major architectural changes.\n\nThe work suffers from several significant weaknesses that should be addressed:\n\n1. The Contribution of the Proposed RID Dataset is Limited. The authors introduce the Raw in Dark (RID) dataset for cross-dataset generalization validation. However, the overall contribution and novelty of this dataset appear to be limited considering existing LOD . The RID dataset follows the existing paradigm of paired RAW and sRGB images, which is structurally identical to the benchmark LOD dataset. It does not introduce a new format or annotation methodology that would address fundamental challenges in the field. With only 500 paired samples, the dataset is considerably smaller than the LOD benchmark, which contains 2,230 pairs. Although the dataset focuses on more extreme low-light scenes and expands the number of object categories, this represents an incremental addition to existing resources rather than a foundational or groundbreaking contribution.\n\n2. A critical weakness is the paper's failure to address the method's generalization capability across heterogeneous camera sensors, especially those with different Color Filter Arrays (CFAs). The method operates directly on RAW data, and all experiments were conducted on two Canon camera models that use a standard Bayer (RGGB) CFA. It is highly probable that the model is learning priors that are specific to this RGGB pattern. The paper provides no discussion or evaluation of how the model would perform on RAW data from sensors with different CFAs, such as RYYB or X-Trans, which are common in smartphones and other camera systems. A model trained on RGGB data would likely face a severe performance degradation when applied to a different RAW structure due to the fundamental mismatch in color channel information and spatial layout. This unaddressed dependency on specific hardware severely limits the method's claimed \"plug-and-play\" applicability.\n\n3. The paper's evaluation against the \"enhancement-then-detection\" paradigm is insufficient, as the selection of baseline methods is too narrow. The authors only select three sRGB enhancement algorithms (Zero-DCE, KinD, and YOLA) to form the two-stage baseline. The field of Low-Light Image Enhancement contains many high-performing algorithms that could form a much more competitive baseline, such as the pioneering RAW-based enhancer SID (which is cited but not used for comparison) or more recent state-of-the-art models like Rawformer. By failing to compare against a more comprehensive and challenging set of SOTA enhancement methods, the paper's claim that its end-to-end approach is superior to two-stage pipelines is not robustly supported. It is possible that a pipeline combining a top-tier LLE algorithm with a detector could achieve comparable or even better performance.\n\nQUESTIONS:\n- What is the computational overhead? Could you please provide details on the increase in inference time (or decrease in FPS) when adding the CRM-IR modules to the baseline object detectors? This is a critical factor for real-world deployment.\n- The method learns to refine an sRGB image based on metadata from its corresponding RAW version. Since the sRGB image is the output of a specific, often proprietary, in-camera Image Signal Processor (ISP), how well does the trained model generalize to images from different cameras with different ISPs? Have you performed any experiments to test this robustness?\n-  And also above weakness\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors present CRM-IR (Raw Metadata-guided Image Refinement), a novel raw-based object detection framework that achieves superior performance compared to conventional methods while excelling in flexibility and compactness. \nTo realize this, the authors designed two key components: CMCE and MV-IR, and introduced a dedicated evaluation dataset.\n\nSTRENGTHS AND WEAKNESSES:\n[Strengths]\n1. The main motivation is simple, and reasonable.\n2. The new evaluation dataset was provided.\n3. The paper is well-organized and easy to read.\n\n[Weaknesses]\n1. It is questionable whether each module was trained as intended by the authors and whether this led to the observed performance improvements. For example, Section 4.3 only shows the performance degradation when CMCE and MV-IR are simply removed. Based on these experimental results, it is difficult to verify whether CMCE effectively captures the inter-redundancy between the raw and the sRGB images as intended by the authors, and whether the inverse mapping from the linear representation in MV-IR is being performed as intended.\n\n2. The baseline models(e.g. YOLOv3(2018), Faster R-CNN(2015), CenterNet(2019)) used are outdated, and the performance trends vary significantly across baselines, raising doubts about whether the authors' proposed method would bring meaningful performance improvements in recent models. For example, in Table 3, when CenterNet(2019) is used as the backbone, the performance differences between YOLA, RAOD, and the proposed method(Ours) are not statistically significant. This trend is likely to become more pronounced when evaluated against newer models. Additionally, I am curious why there was no comparison with models such as AdaptiveISP (NeurIPS 2024).\n\n3. This model is specialized for the ISP of the camera used to capture the training data, and its practical application assumes imaging with the target camera. In this context, measuring generalization ability on datasets captured with \"different cameras\" seems questionable. If the LOD dataset lacks sufficient extremely low-light conditions, collecting additional evaluation datasets should use the same camera to ensure meaningful comparisons.sets should involve the same camera to ensure meaningful comparisons.\n\n4. I'm curious why camera parameters that directly influence the ISP processing, such as exposure time and ISO, were not utilized.\n\n5. I am curious why the mAP75 performance of RAOD is particularly low except when using Faster R-CNN as the backbone.\n\n6. In the table, all the bpp values for the comparison models are marked as \"-\", is this report accurate?\n\n7. The authors do not mention any limitations of this study. If the authors have any limitations in mind, I would appreciate it if they could describe them in detail.\n\n8. Minor error.\n-- L142: y >> y_hat\n-- L164: (redundant phrase)\n-- L172: Shouldn't a reference be provided?\n-- L266: YOLOA >> YOLA\n-- All tables: map >> mAP\n\nQUESTIONS:\nPlease refer to the weaknesses section. My current rating could change significantly depending on the authors’ responses. While I have some knowledge of camera ISP and image restoration, I do not have specialized expertise in the specific field of this study. I am aware of this and will actively refer to other reviewers’ questions and the authors’ responses as well.\n\nLIMITATIONS:\nThe authors do not mention any limitations of this study. I hope they will add this to Section 5. Conclusion, and I would like to hear their thoughts on this point in the rebuttal.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-15"}

comments: SUMMARY:
This work proposed an approach for low-light image enhancement by partially using the raw sensor data. 
It estimates the metadata from the raw image using a trainable probabilistic encoder, then uses the metadata to remap the sRGB image to a domain optimal for machine vision tasks. Main novelties include:

1. An entropy parameter network for the metadata extraction
2. A selective remapping technique that treats the low-light part and the high-light part of the image separately.

STRENGTHS AND WEAKNESSES:
Strength:
1. This work uses metadata for the image enhancement, which is less computationally expensive than those that directly use the raw image.
2. This work does not require a trainable ISP for the raw data, thus it is expandable to different schemes. 

Weakness:
1. The estimation of bpp is not stated clearly. Is it purely the transmission and storage overhead for the metadata? If so, the computational cost of the encoder is not counted. A proper comparison of the total FLOPs against other models might be useful.

QUESTIONS:
1. The ablation study is done on the latest generations of detection models.  YOLA and RAOD, which are compared in Tables and 3, are not used in the ablation study. Can the authors explain the reason of it?

LIMITATIONS:
N/A
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nA novel low-light object detection paradigm, Compact RAW Metadata-guided Image Refinement (CRM-IR), is proposed. The core idea of this method is to avoid directly using RAW images for inference, and instead, leverage compressed RAW metadata to guide the enhancement of sRGB images. This enables improved object detection performance under low-light conditions without modifying existing detection models or the camera's ISP pipeline.\n\nSTRENGTHS AND WEAKNESSES:\nQuality: The authors are based on raw and SRGB pairs of low light, but there seem to be limitations in the experimental results (many indicators are worse than YOLA RAOD) . \n\nClarity: Low-Light Object Detection combined with Metadata from RAWs is natural and well-motivated, and the paper presents a clearly structured framework. \n\nSignificance: The method requires no modifications to the ISP or downstream detection model architectures, making it easy to integrate into existing vision pipelines. It achieves comparable object detection accuracy while significantly reducing RAW data storage and transmission costs by transmitting only compact metadata instead of full-resolution RAW images.\n\nOriginality: The novelty of this approach appears limited, as its performance gains seem inferior to both YOLA (sRGB-based) and RAOD (raw-based) methods. This raises legitimate doubts about whether the cross-modal design actually delivers meaningful improvements.\n\nQUESTIONS:\nHow does the metadata-guided inverse gamma and remapping in MV-IR fundamentally differ from traditional enhancement methods like Retinex?  Does it truly capture representations more suitable for machine vision?\n\nThe CMCE achieves extremely low bitrate (<0.001 bpp).  What specific cross-modal redundancies does it exploit, and how does it improve upon standard learned image compression techniques?\n\nThe evaluation primarily uses YOLOv3, Faster R-CNN, and CenterNet as baselines, which are relatively outdated compared to recent state-of-the-art detectors such as YOLOv11/v12,. This may limit the assessment of the method’s effectiveness in modern real-world deployment scenarios.\n\nThe paper does not analyze the runtime or computational overhead introduced by the MV-IR and CMCE modules.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}