Rethinking No-reference Image Exposure Assessment from Holism to Pixel:  Models, Datasets and Benchmarks
Authors: Shuai He
Shuntian Zheng
Anlong Ming
Banyu Wu
Huadong Ma
Venue: NeurIPS 2024 poster (2024)
Keywords: Image Exposure Assessment
Image Quality Assessment
Primary area: machine_vision
TL;DR: Pixel-level image exposure assessment from three perspectives: model, dataset, and benchmark.

Abstract: The past decade has witnessed an increasing demand for enhancing image quality through exposure, and as a crucial prerequisite in this endeavor, Image Exposure Assessment (IEA) is now being accorded serious attention. However, IEA encounters two persistent challenges that remain unresolved over the long term: the accuracy and generalizability of No-reference IEA are inadequate for practical applications; the scope of IEA is confined to qualitative and quantitative analysis of the entire image or subimage, such as providing only a score to evaluate the exposure level, thereby lacking intuitive and precise fine-grained evaluation for complex exposure conditions.  The objective of this paper is to address the persistent bottleneck challenges from three perspectives: model, dataset, and benchmark.  1) Model-level: we propose a Pixel-level IEA Network (P-IEANet) that utilizes Haar discrete wavelet transform (DWT) to analyze, decompose, and assess exposure from both lightness and structural perspectives, capable of generating pixel-level assessment results under no-reference scenarios. 2) Dataset-level: we elaborately build an exposure-oriented dataset, IEA40K, containing 40K  images, covering 17 typical lighting scenarios, 27 devices, and 50+ scenes, with each image densely annotated by more than 10 experts with pixel-level labels.  3) Benchmark-level: we develop a comprehensive benchmark of 19 methods based on IEA40K. Our P-IEANet not only achieves state-of-the-art (SOTA) performance on all metrics but also seamlessly integrates with existing exposure correction and lighting enhancement methods. To our knowledge, this is the first work that explicitly emphasizes assessing complex image exposure problems at a pixel level, providing a significant boost to the IEA and exposure-related community. The code and dataset are available in \href{https://github.com/mRobotit/Pixel-level-No-reference-Image-Exposure-Assessment}{\textcolor{red} {here}}.

Rethinking No-reference Image Exposure Assessment from Holism to Pixel: Models, Datasets and Benchmarks
Shuai Hel: Shuntian Zheng Anlong Mingl
Banyu Wu
Huadong Ma'
1 Beijing University of Posts and Telecommunications
University of Warwick
{hs199512 99510211 @bbppt edu. en
Shuntian. www.w/hwwwww......c..o. uk (mal, wubanyu, mhd) edu. en
Abstract
The past decade has witnessed an increasing demand for enhancing image quality through exposure, and as a crucial prerequisite in this endeavor, Image Exposure Assessment (IEA) now being accorded serious attention. However, IEA en- counters two persistent challenges that remain unresolved over the long term: the accuracy and generalizability of No-reference IEA are inadequate for practical applications; the scope of IEA is confined to qualitative and quantitative analysis the entire image or subimage, such as providing only a score to evaluate the exposure level, thereby lacking intuitive and precise fine-gr fine-grained evaluation for complex exposure conditions. The objective of this paper is to address the persistent bottleneck challenges from three perspectives: model, dataset, and benchmark. 1) Model-level: we propose Pixel-level IEA Network (P-IEANet) that utilizes Haar discrete wavelet transform (DWT) analyze, decompose, and assess exposure from both lightness and structural perspectives, capable of generating pixel-level assessment results under no-reference scenarios. 2) Dataset-level: we elaborately build an exposure-oriented dataset, IEA40K, containing 40K images, covering 17 typical lighting scenarios, 27 devices, and 50+ scenes, with each image densely annotated by more than 10 experts with pixel-level labels. 3) Benchmark-level: we develop a comprehensive benchmark of 19 methods based on IEA40K. Our P-IEANet not only achieves state-of -otie--f-th--ttt the-art art (SOTA) performance on all metrics but also seamlessly integrates with existing exposure correction and lighting enhance- ment methods. To our knowledge, this is the first work that explicitly emphasizes assessing complex image exposure problems at pixel level, providing signifi- cant boost to the IEA and exposure-related community. The code and dataset are available here.
Introduction
Exposure, one of the 3A factors (Auto Exposure, Focus and White Balance) camera technology, plays crucial role in controlling image quality. Image exposure assessment (IEA) is prerequisite for improving exposure 3 however, even leading phone and camera manufacturers heavily rely on manual evaluations due to the unavailability and high cost of human raters. Nevertheless, large- scale adoption manual assessments is impractical. Similar to mainstream AI applications, deep learning and data-driven approaches hold promise as potential solutions to overcome this limitation. Nevertheless, the traditional data-driven IEA paradigm encounters two major challenges:
1) A Dilemma between Applicability and Practicability: While full-reference IEA methods deliver satisfactory results 4.. their applicability non-preset scenarios is limited due the general unavailability of reference images. Conversely, no-reference IEA methods, which do not rely on reference information, struggle with natural images distorted by unknown factors. This difficulty
Equal contribution. Corresponding author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Underexposure?
Ideal Exposure?
Iaput
Brightness Tistogram
(b) Our Pivel-leel TEA
(c) Zone System Theory
Figure 1: Comparison between the brightness histogram (a) and method (b) for assessing image exposure. (b) offers more intuitive and accurate reflection of the exposure conditions in each area. (c) shows how determine exposure levels for each area using Adams' zone system theory [14 15
arises from the inability to identify specific features for assessing exposure, as the quality prediction problem becomes agnostic to the type of exposure distortion, thereby restricting both performance and practicality [0 I1].
2) Restricted Generalization Capacity: Traditional IEA annotation relies on scenario-specific criteria, leading to significant subjectivity across datasets. These datasets typically provide only a holistic quantitative score reflecting the overall exposure condition [12), lacking detailed and fine- grained assessments. Consequently, the subjective and coarse- goasee-grained labels introduce restricted generaliza- tion capacity into learning-basee based IEA methods, reducing their adaptability to diverse scenarios and assessment criteria.
How can an ideal method be designed to tackle the aforementioned challenges? The method should primarily address three key issues: firstly, as a no-reference method, should effectively simulate reference images in non-preset scenarios, functioning like full-reference methods; secondly, it should achieve fine-grained assessments by adapting to diverse high-level evaluation criteria or application scenarios directly or through fine-tuning; finally, the learned features should be decoupled from subjective criteria and aligned with naive exposure features mitigate narrow inductive biases.
This paper presents P-IEANet, an innovative method that leverages large-scale, pixel-level annotated datasets to delve into the fundamental unit of IEA: pixels. This approach allows us to identify exposure issues with unprecedented precision and to handle IEA tasks of varying granularity beyond the pixel level, without being influenced by subjective criteria. Grounded in the well- established theory that the power spectrum of natural images is a function of frequency, represented 1/ where 7 varies slightly specific frequencies []] leverage this insight analyze exposure characteristics in specific frequency domains for improved adaptability across varying criteria and scenarios. Through employing the dedicated Haar Discrete Wavelet Transform (DWT), P-IEANet decomposes the original image to criteria- criteria-agnostic frequency features, thus avoiding narrow inductive biases. Additionally, with pixel-level supervision, P-IEANet enables ideal exposure reconstruction from frequency space, effectively creating reference images for further analysis.
Our contributions are summarized follows:
To our knowledge, this the first work to implement a pixel-level evaluation paradigm in IEA. It enhances the generalizability and accuracy of no-reference IEA tasks, while effectively addressing challenges associated with reusing underlying data and architectures.
We present the P-IEANet, showcasing that pixel-level IEA can be decomposed into criteria- agnostic lightness and structure information via the dedicated Haar DWT. This design enables efficient execution of pixel-level IEA while minimizing parameter usage.
To convincingly validate method, we have developed dataset exclusively tailored for IEA, called IEA40K. This dataset specifically focuses on exposure and comprises 40,000 of images with the most comprehensive annotations to date, including pixel-level annotations.
Building upon IEA40K, we have evaluated baselines, establishing our benchmark as the most comprehensive to date for IEA. Our work not only achieves SOTA performance but

---

2.1
4.2
3.7
1.8
7.9
0.6
2.0
4.3
4.0
1,5
8.6
0.7
Figure Images visualized along with their corresponding pixel-level (heat map of exposure residual) and holistic IEA results by P-IEANet (where a higher score shows more visually pleasing exposure).
also serves as a pivotal catalyst, offering the community a new roadmap to explore further solutions for IEA.
2 Related Work
No-reference IEA Methods and Datasets. Previous studies on no-reference IEA can be broadly classified into two primary categories: 1) Statistical-feature based methods. Datta el 4 utilized average pixel intensity to evaluate light usage. Liu el al. [ explored image brightness histograms, and Hanmandlu et al. developed indicators based on the image brightness histogram for crucial auto-cxposure control. Rahman al. [8] and Lu et al. [9] adopted information entropy criterion for exposure evaluation. In some auto-exposure work. [16119 incorporated gradient information to determine the ideal exposure settings for cameras, suggesting that maximum information entropy indicates ideal image exposure. Efimov et al. [20] and Dong et al. [2 proposed subdividing images into blocks for individual assessment, classifying each based on its brightness histogram utilize
into categories. 2) Data-driven methods. The increasingly popular methods
human-labeled datasets develop scenario-specific features.
However, when comes to statisticalfeaturr based methods, manual features often assume only one type of distortion, which is problematic in complex situations where overexposure and underexposure coexist [I] 255. On the other hand, data-driven methods become less effective when assessment criteria or application scenarios change; moreover, the holistic scores from these datasets lack the detailed supervisory information required for the precision and granularity demanded in applications.
Pixel-level Tasks. In conventional terms, the concept of "exposure" refers to not only exposure time but also two other parameters (aperture and ISO, referred to (26). Rather than being characterized as global attribute of the image, the parameters would be more appropriate to be described as a global attribute associated with the camera for capturing the image. However, according to the claim made by the classical photographic theory (Adams' theory) (27 that "The exposure time is the same for all elements, but the image exposure varies with the luminance cach subject element," the coarse global camera exposure attribute fails match each subject element in image, potentially resulting in some subject elements being under-exposed and others being over-exposed. Given this, in the context of evaluating images, the term "exposure" is no longer a global attribute, referred to 27 that "Any scene of photographic interest contains elements of different luminance; consequently, the "exposure' actually is many different exposures." Therefore, the pixel-level IEA is highly desired.
However, to our knowledge, there are currently no pixel-level IEA methods, despite advancements in related visual tasks such as semantic understanding and fine-grained analysis. For instance, DiffuMask 28 exploits powerful zero-shot ext-to--imaee generative models to provide pixel-level
3

---

Ovgresposure
Underexpossne
Ideal
Horiioattl
@ Vertial
G Approximmaion
CC
Ed)
Diagonal
Unkkrrqpoure
Head
Overggpwaa
Figure After Haar DWT decomposes an image, swapping its low-frequency component with the high-freyuency components (Q2@) of the same image under different exposures produces visually similar results (a-d) as well similar t-SNE features (e).
(b) Lightness Feature Module
(c) Prediction Module
Haar
LCA
Haar DWT
ML.P
MLP
DWT
LPA
High
Residual Map
Hationtal
Gradient Maps
LRE
Gradient
0.5
MLP
SRE
Maps
0.5
0.5
0.5
-0.5
0.5
Liwe
(a) Structure Feature Module
(d) DWT Kernels
Figure 4: Pipeline of P-IEANet. The original image is decomposed into wwwhigh-ffrqquene com- ponents via Haar DWT. Subsequently, the Structure Feature Module analyzes the high-frequency components with gradient maps to extract structural features (cf. Sec. B.T). Simultaneously, the Lightness Feature Module handles the now-frequeccy component and extracts lightness features by attention mechanisms (cf. Sec. [32)p. These features are then composed through Haar inverse DWT for pixel-level IEA. Dedicated convolution kernels (d) are employed facilitate the DWT process.
segmentation annotations across diverse classes. Similarly, PixeILM 29 leverages GPT4V to produce 246,000 pixel-level question-answer pairs, enhancing its capabilities in pixel-level reasoning and comprehension.
However, directly applying these methods IEA tasks very challenging due to specific require- ments in data collection and annotation processes, which entail avoiding selection bias, accurately aligning images, obtaining ideal references, and providing detailed annotations (cf. Sec.
3 Architecture of P-IEANet
Preliminaries. Images captured with incorrect exposure settings often suffer from visual problems, including lightness and structure distortions [30. For instance, overexposed images exhibit unnatural artifacts, inconsistencies in exposure blending, and blurred structural details. This paper demonstrates the potential of the Haar DWT for analyzing IEA issues. The mathematical representa- tion of the Haar DWT can be formulated as follows:
DWT(L,
Σ F(K)(
n ("-M²
nn-122m
1:2m
26
12
k.2m
2T
2m
2m
The mother wavelet Φ and 2/ of the Haar DWT can decompose image into low frequency compo- nents L (approximation coefficients). and high frequency components H (coefficients in horizontal, vertical, and diagonal directions), shown in Fig. 3 We label the underexposed, overexposed, and ideal images as Xover, Xunder, and Xideat, respectively. Their corresponding Haar DWT represen- tations in frequency denoted as DWT(L(X.ver),. (X.ver))) DWT Xunder H (Xunder)) and (W(((((x,eea)) H(Xident) respectively.
For IEA tasks, we examine whether the low-frequency and high-frequency components corre- spond to the frequency-domain representations of lightness and structure, respectively. The images obtained by reversing these components, such DWT-1 (WT-I(((Xidea))) H undder)) and

---

DWT (L(Xideal) H (Xverr)). show an exposure close to Xideal (Fig. B(o)d)). Conversely, DWT-1 (L H (XXyndrr)) exhibits a exposure similar to that of Xover. as shown in Fig. 3). To further validate these findings, we conducted similar analysis using t-SNE dimensionality reduction (33 on 200 sample images Fig. e). The t-SNE results reveal that the high fre- quency components remain relatively consistent across different exposures, while the low frequency components exhibit significant variation.
Based on the above observations, we deduce that the low-frequency components primarily represent an image's 's lightness, while the high--rreuenncy components indicate structural details and are less affected by lightness variations. By exploiting this characteristic, we can decompose the exposure of a distorted input image into two frequency representations and then construct an ideal reference image in the frequency space. Subsequently. differences in the frequency domain are mapped to pixel-Ievel IEA results. This approach effectively mitigates any influence from irrelevant image semantics or noise associated with IEA.
Pipeline. P-IEANet comprises three essential modules (Fig. 4. The Structure Feature Module (ef. Sec. 3.1) and Lightness Feature Module (cf. Sec. (.2) are responsible for extracting structural features from high-frequency components and lightness features from low-frequency components, respectively. Ultimately, the Prediction Module (cf. Sec. 3.3) integrates these features to predict pixel-Ievel IEA results and generate the final prediction.
3.1 Structure Feature Module
There are two categories of structural features relevant to the IEA tasks: 1) Long-range features encompassing the overall layout and distant objects of an image, which provide a comprehensive understanding of its structure and global exposure [4 2) Short-range features focusing on fine details and textures, such as edges and localized patterns, are crucial for capturing local exposure first derive gradient maps from the
variations 41. To obtain these features effectively, high frequeryy components of the Haar DWT. These maps highlight edge regions, thus enhancing the representation of basic structural details 30 38 42 Subsequently, refine the extraction process by subjecting these gradient maps to a Long-Range Encoder (LRE) and a Short-Range Encoder (SRE). Further details provided below.
Gradient Maps. The input image X is decomposed by the Haar DWT to obtain high-frequencr components, and then processed by multi-layer encoder to extract naive features. For each layer zi, where ranges from I to (the total number of layers), we compute the gradient map as follows:
V2 2 {ga(2;) ld E D),
where ga(2) applies the first-order gradient function g to 2; in direction d. The set D includes all directions under consideration: +x, x2, -y + + y. + - y. -I y. which correspond to the x-axis, y-axis, and their diagonals. These directions ensure comprehensive emphasis on edges, therehy enhancing the formulation of structural features.
Long-range and Short-range Encoders. To enhance the extraction of structural features, we feed both the original input feature, 24, and its gradient maps Vz, into two distinct modules: Transformer-based LRE Z and CNN-based SRE Z". The feature extraction process is as follows:
4 = (x)) 8; = (2)),, VL = (V:;, Vs; = VZ,
where 4 and s; represent the long-range and short-range features, respectively, these features are then integrated using Structure Fusion Module Z1, which employs multiple MLPs, follows:
2. 2 (/ (1i, si), (VI, Vo.)
(4)
In this formulation, ||...|| signifies the stacking of operations along the feature channel dimensions, facilitating a comprehensive synthesis of the extracted features.
3.2 Lightness Feature Module
The human visual system, possessing a high dynamic range, is skilled at globally detecting varying light levels of objects. However, due to limited attention capacity, it also tends to focus on specific
5

---

Clamne Attention Weight Yo
Input Image
(0) Long ringe V'cturr Map
(b) Shoet range Festure Map (+) Pioel Atteation A
(d) Feauree Maps of Miitiple Channels bq
Figure 5: Visualization of different components in P-IEANet. Long-range (a) and Short-range (b) features highlight the important structural information for IEA tasks, while pixel-level attention (c) and channel- attention (d-e) characterize its distribution in terms of light information.
regions with distinct lightness levels. Our method selectively processes global channels and local pixel regions to enhance the network's s management
broad spectrum of information.
Lightness Channel Attention. Firstly, our lightness channel attention (LCA) processes the channel- wise global spatial information, T (C' H x W), through global average pooling create channel descriptor, A. (C 1 x 1). To determine the weights for different channels, the descriptor undergoes further refinement in two convolution layers, followed by sigmoid and ReLU activation functions. This procedure is formalized as follows:
W
1
A. = o (MLP(1 H
Lo(i)))))),
(5)
1=1
where (i(j)) represents the lightness value at position (i,j) in the e-th channel Le, this channel attention strategy highlights that lightness variations across different channels convey distinct and weighted information. Finally, the channel weights are element-wise multiplied with the input to generate the output F. A. & Le (C W H).
Lightness Pixel Attention. The variable distribution of lightness among image pixels necessitates our lightness pixel attention (LPA) mechanism. This mechanism processes the output Fg from the LCA using self-attention (SA) and convolution layers, coupled with Rel.u and sigmoid activation functions. The attention mechanism formulated as follows:
Ap a Σ wkk (oNAA(((SA(FE))) AEK
Here, Conv; denotes a convolution operation with multi-scale kernel sizes k, which aims to enhance the network's focus fine-grained and multi-scale exposure features under complex scenario. Larger kernel sizes help perceive overall brightness and contrast, while smaller kernel sizes detect localized overexposure underexposure issues. Additionally, self-attention allows the system to analyze lightness distribution and recognize patterns at multiple scales.
We then perform element-wise multiplication to merge the input Le with Ap (1 x H W), generating the output F, = L.e @ A, (C x x W). The final stage integrates the outputs from both channel and pixel attention mechanisms to yield the comprehensive output A, MI.P(F,, F,).
3.3 Prediction Module
We employed Haar DWT-1 to integrate lightness and structure features for ideal exposure representa- tion reconstruction in the frequency space, then using the formula P, = MLP(DWT (A,, Z,)) to predict the exposure residual, which measures the deviation of each pixel from the ideal exposure. Both Haar DWT and DWT-1 involve four dedicated convolutional kernels to simulate the wavelet transform's 's decomposition and reconstruction processes (Fig. dd). To evaluate prediction accuracy, we define loss function as:
Ppize!
H W ΣΣ P,(i.)) -P
where Pp the pixel-level ground truth, these residual maps can be converted into a coarser-grained prediction above pixel, e.g., holistic IEA score (cf. Sec. 5.2).

---

Animal
Ford
Fruit
Hppp
Typo
Color
Anutet
Sparte
Purtet
Stane
Light
Plan
Home
Cenalty
toige
Zane
Lendsaape
Siarce
Audiual
Lamespoore
[c
Figure 6: Proposed IЕA40K dataset. (a) Visualization of images with different exposure conditions;
(b) Scenes containing 8 super-classes with 50+ sub-classes; (c) 17 typical lighting scenarios.
4 Proposed IEA40K Dataset
4.1 Image Collection
How to Avoid Selection Bias when Building Comprehensive IEA Dataset. Selection bias can arise when certain types of intended exposure conditions, scenes and devices are underrepresented the dataset, which may compromise the validity and generalizability of training models. To mitigate this issue, we considered key aspects: varied scenes, diverse light conditions, sufficient devices, uniform resolution and comprehensive simulations (Fig. (). These factors significantly impact the quality of the dataset. Further details are provided Appendix A.I
How to Align a set of Images through Pre-processing Strategies. Effectively aligning set of images captured under various exposure conditions is significant challenge. Factors like camera shake and slight subject movement during shooting parameter adjustments can lead to misalignment 433 Such misalignment adversely affects the generation of supervised information and the training process. To tackle this issue, we first apply the Structural Similarity Index Measure (SSIM) algorithm to filter out misalignment images from the series. Those falling below a specified SSIM threshold are eliminated. Subsequently, we employ image alignment algorithms AS] to further enhance the alignment of the remaining images. This entire process is automated and can be exccuted in an unsupervised manner.
4.2
Data Annotation
How to Obtain Ideal Reference. Obtaining reference image with ideal exposure in each region is crucial for our subsequent image annotation. However, ensuring the representatives of the reference image while minimizing the randomness and subjectivity introduced by humans presents significant challenges. To address this, we start by creating a preliminary reference image using a multi-exposure fusion algorithm. Subsequently, we segment this image into blocks with the super-pixel segmentation al algorithm [44] based lightness and structure. Finally, experts optimize each block's 's exposure
conditions utilizing Adams' zone system theory of classical photography [4 [S]] (Fig. [c)). This theory provides precise guidelines for achieving ideal exposure across different elements.
How to Obtain Pixel-Level Labels by Human-in-the- and intricate nature of pixel-level annotation, we have streamlined the process using a combination of expert judgment and weak supervision techniques (Fig. 7p. The initial pixel-level annotations were generated by comparing a reference image with the 8 distorted images, documenting exposure residual across pixels. Subsequently, experts further refined the final pixel-level annotations to rectify potential errors, such as accurately identifying areas with logos as severely overexposed and addressing discrete anomalous pixel labeling, to ensure that the final exposure residual closely aligns with the perceived deviation of each pixel from ideal exposure. For experts, distinguishing between the reference and distorted images relatively straightforward and far more accurate, thus facilitating practical data annotation.
Methods. Given the exorbitant cost
7

---

Experss
61
0.0 H0.
Adjuct
Experts
Imagge
Adjust
Distorted Imanges
Figure 7: Overview of the proposed IEA dataset annotation process. First, collect images of different exposure conditions and then adjust the raw image by experts to obtain reference image; Second, calculate exposure residual between the reference and 8 distorted images, and then have experts adjust the local exposure residual. The closer it is to -1, the more overexposed the corresponding pixel is; and the closer it is to the more underexposed the corresponding pixel is.
Oriikal beats
Nottes
Figure 8: Example exposure residuals, a robust quantitative indicator for precise assessment, uniquely generated by P-IEANet (a lower absolute exposure residual suggests visually more pleasing result). (a) Inputs: an original image and its enhanced counterparts by classical light enhancement methods. (b) Outputs: the exposure residuals hilighting the disparity between the input and the ideal exposure.
5 Experiments
5.1 Settings
Benchmark Models and Training Protocols. To the best of our knowledge, there no publicly available pixel-level IEA model. Therefore, we have selected several deep learning baselines based on the following criteria: classical architectures with publicly available code and SOTA performance in specific domain. For pixel-level IEA, we selected light enhancement, light-aware, and image quality assessment (IQA) models as backbones, complemented by appropriate output heads. For holistic IEA, we chose IQA and image aesthetics assessment (IAA) models to regress scores. Further details regarding training protocols can be found in Appendix A.3
Evaluation Metrics. For pixel-level IEA, we adopt SSIM and MAE to measure the structure and lightness similarity between the ground truth and predicted exposure residual. For holistic IEA, we adopt the Spearman's 's rank correlation coefficient (SRCC) and the linear correlation coefficient (LCC), to measure the correlation between the predicted IEA score and human opinion 45
5.2 Performance Evaluations
Pixel-level Assessment. Table [ presents the results of P-IEANet and 12 other models on the IEA40k dataset. Our P-IEANet achieves SOTA performance, surpassing the second- best model with remarkable -40% reduction in MAE loss and significant +25% improvement in SSIM, while using an impressive -97% fewer training parameters. The efficiency of P-IEANet can be largely attributed Haar DWT, which effectively minimizes the number required feature extraction layers. Additionally, both the Lightness Feature Module and Structure Feature Module effectively utilize information, contributing to its exceptional performance with fewer parameters.
Holistic Level Assessment. We validated the effectiveness of P-IEANet on the holistic IEA task using the representative SPAQ dataset [12. This dataset provides annotations for holistic exposure, allowing us to test the capabilities of various baseline models. P-IEANet supports three prediction methods: 1) With Fine-tuning: after obtaining the residual map, it is processed through additional MLPs to predict the holistic exposure score, supervised by the MAE loss. A comparison of P-IEANet with 18 other models is presented in Table 2 where P-IEANet achieves SOTA performance. 2) Without Fine-tuning: after obtaining the residual map, we compute the average of absolute values and subtract this average from 1, mapping a scale of 0- (Fig. 2.. Remarkably, without requiring fine-tuning for holistic exposure scoring SPAQ, we achieved LRCC of 0.69 and SRCC of 0.65,

---

Table 1: Comparing the pixel-level IEA performance of 13 models IEA40K. We adjusted the output headers all light enhancement/awareees, and IQAIAА models to ensure their support for fine-tuning our IEA40k. All models utilized only the exposure residual pixel-level ground truth and were retrained for the best performance.
Light Enhancement Awareness
IQA ATA
Pixel level
FEC
MSLT
Retinex
Dyfiff
SMG
SKF
GSAD
LIT
AmIQA
ReIQA
DEIQT
MUSIQ
Ours
[32
44
47
48
38
49
500
51
52
53
54
Params
9M
8M
5M
374M
575M
22M
67M
146M
106M
560M
363.5M
298M
2.7M
MAE
10.08
0.10
0.10
0.07
0.09
0.07
0.08
0.05
0.11
0.08
0.08
0.15
0.03
SSIM T 0.35
0.33
0.42
0,50
0.46
0.44
0,41
0.60
0,50
0.37
0.39
0.24
0.75
Table 2: Comparing the holistic level IEA performance of 19 models on SPAQ. All models utilized only the exposure score as holistic-level ground truth, and were retrained for the best performance.
Light Eshancement/ / Awareeess
IQA/IAA
Holistic level
FBC
MSLT
Recinex
PSDiE
SMG
SKE
GSAD
LII
Am/OA
ReIOA
DEIOT
MUSIQ
NIMA
Alamp 15
MLSP
'LANeE
EAT
55
50
55
52
50
Params
9M
8M
3M
374M
575M
22M
67M
146M
106M
560M
363.5M
298M
56M
99M
24M
40M
87M
7B
2.7M
LRCCT
0.74
0.75
0.71
0.70
0.60
0.68
0.71
0.72
0.70
0.71
0.63
0.64
0.63
0.67
0.71
0.75
0.74
0.75
0.78
SRCC
0.69
0.70
0.67
0.68
0.67
0.66
0.68
0.69
0.68
0.65
0.54
0,60
0.60
0.64
0.68
0.69
0.71
0.70
0.73
even surpassing some methods that do fine-tuning,, The above results show that P-IEANet exhibits strong criteria and scenario robustness beyond pixel-level tasks. 3) Criteria-oriented without Moreover, we additionally discuss an methodology in Appendix A.2
Fine-tuning:
criteria-oriented scoring
Ablation Studies. B evaluates the effectiveness of P-IEANet's modules. The absence of the Haar DWT and two other modules, Structure and Lightness, significantly impact the performance of P-IEANet. Specifically, the SSIM decreases by 42.6%, 28. 28.0%, and 44.0% respectively, while the SRCC falls by 14.3%. 7.1 %, and 15.8% These results confirm that each module, particularly the Lightness Feature Module which processes low-frequency information, plays crucial role in enhancing the model's overall performance. Qualitative visual effects analysis is provided in Fig. 5
Predictions for Images. The prediction examples are shown in Fig. 2 Similar to human perception, P-IEANet's 's pixel-level evaluation results effectively identify areas of overexposure and underex- posure that are visually displeasing, even in unconventional scenes where both underexposure and overexposure coexist. Moreover, when combined with semantic segmentation algorithms, P-IEANet enables more precise object- and pixel-level IEA results (Appendix A.2..
5.3 Advancing Light Enhancement Methods
P-IEANet, owing to its exceptional sensitivity towards exposure, offers advantages for the exposure enhancement community in the following two aspects:
1) Analyzing Performance Better: Traditionally, assessing the efficacy of image enhancement algorithms has been a time- coneuminn and imprecise task, relying solely on human observations. The exposure residual, uniquely generated by P-IEANet, serves robust quantitative indicator for precise assessment (refer to Fig. where input can be either an original image or an enhanced one).
2) Enhancing Performance Better: P-IEANet is compatible with many existing light enhancement methods, enabling boost their performance. To demonstrate this, we chose two open-source and SOTA methods, Retinex [47] and GASD (50), baseline models. We incorporated P-IEANet as sample evaluator in these models after enhancement, freezing P-IEANet's parameters and obtaining the absolute exposure residual loss to include in the baseline models.
shows that on both representative datasets, LOL-vI [61 and LOLv2-real (62), the performance is improved to some extent, suggesting that P-IEANet has the potential to become important enhancer in this field.

---

Table 3: Ablation studies conducted IEA40K.
Pixel-level MAE
Holistic level
Method
Params
SSIM
ACC
LRCC
SRCC
w/o DWT
2.7M
0.08
0.43
0.81
0.74
0.72
w/o Structure
1.6M
0.05
0.54
0.84
0.81
0.78
w/o Lightness
1.0M
0.08
0.42
0.73
0.74
0.71
P-IEANet
2.7M
0.03
0.75
0.88
0.89
0.84
Table 4: Our P-IEANet can enhance some low-light enhancement methods. Our retrained results are marked by ...
LOL-v1
LOLv2-real
Model
PSNRT
SSIM
PSNRT
SSIM
Retinex (ICCV'23)
25.16
0.845
22.80
0.840
Retinex
24.7
0.80
21.9
0.82
Retinex* + P-IEANet
25.3
0.85
23.0
0.86
GASD (NIPS'23)
27.839
0.877
28.818
0.895
GASD
27.1
0.85
28.4
0.86
GASD P-IEANet
28.2
0.88
29.5
0.91
Conclusion
This paper investigates IEA with a novel paradigm: from holism to pixel. To our knowledge, our work introduces a new roadmap by proposing model, dataset, and benchmark for the community. However, several challenges still remain to be addressed. For instance, evaluating images with severe misalignment issues caused by high-speed moving objects poses significant challenges. In future work, we aim to optimize our framework to support multimodal outputs and enhance the exposure perception in artificial intelligence generated content (AIGC).
7 Acknowledgement
This work is supported by the Funds for Creative Research Groups of China under Grant 61921003.
References
[1] Shaoqing Ren, Kaiming Hc, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems, 28, 2015.
[2] Jiankang Deng. Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4690-4699, 2019.
[3] Wenhan Yang. Ye Yuan, Wenqi Ren, Jiaying Liu. Walter J Scheirer. Zhangyang Wang, Taiheng Zhang. Qiaoyong Zhong, Di Xie. Shiliang Pu, al. Advancing image understanding poor visibility environments: A collective benchmark study. IEEE Transactions on Image Processing, 29:5737-5752. 2020.
Ritendra Datta, Dhiraj Joshi, Jia Li, and James Z Wang. Studying aesthetics in photographic images using computational approach. Eurapean conference on computer vision, pages 288-301. Springer, 2006.
[5] Min Liu, Po Yuan, and Richard Turner Automatic analysis and adjustment digital images with exposure problems, Apr. 15 2008. US Patent 7.359.572.
[6] Juan Torres and José Manuel Menéndez. Optimal camera exposure for video surveillance systems by predictive control of shutter speed, aperture, and gain. In Real-Time Image and Video Processing 2015, volume 9400, pages 238-251. SPIE, 2015.
[7] Madasu Hanmandlu, Om Prakash Verma, Nukala Krishna Kumar, and Muralidhar Kulkarni. A novel optimal fuzzy system for color image enhancement using bacterial foraging. IEEE Transactions Instrumenaation and Measurement, 58(8):2867-2879 2009.
[8] Mohammad T Rahman, Nasset Kehtarnavaz, and Qolamreza R Razlighi. Using image entropy maximum for auto exposure. Journal of electronic imaging. 20(1),013007, 2011.
Huimin Lu, Hui Zhang, Shaowu Yang. and Zhiqiang Zheng. Camera parameters auto-adjusting technique for robust robot vision. In 2010 IEEE International Conference on Robotics and Automation, pages 1518-1523. IEEE, 2010.
[10] Wufeng Xue, Xuanqin Mou, Lei Zhang. Alan Bovik, and Xiangchu Feng. Blind image quality assessment using joint statistics of gradient magnitude and laplacian features. IEEE Transactions on Image Processing. 23(11):4850 4862, 2014.
10

---

[11] Anish Mittal, Anush Krishna Moorthy, and Alan Courad Bovik. No-reference image quality assessment in the spatial domain. IEEE Transactions on image processing, 21(12):495547088 2012.
[12] Yuming Fang. Hanwei Zhu, Yan Zeng. Kede Ma. and Zhou Wang. Perceptual quality assessment of smartphone photography. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3677-3686, 2020.
[13] Anuj Srivastava, Ann B Lee, Eero P Simoncelli, and S-C Zhu. On advances in statistical modeling of natural images. Journal of matheatical imaging and vision, 18:17-33, 2003.
[14] Ansel Adams. Basic Photo: The Negative v. 2. New York Graphic Society, New York, NY, Mar. 1978. Ansel Adams. New photo series 2: Negative:. Ansel Adams Photography. Bulfinch Press, New York, NY, June 1995.
[15]
[16]
Inwook Shim, Joon- Yoon-Y Lec, and In So Kweon. Auto-adjusting camera exposure for outdoor robotics using gradient information. In 2014 JEEF/RS International Conference on Intelligent Robots and Systems, pages 1011-1017. IEEE, 2014.
(17] Inwook Shim, Tac-Hyun Oh, Joon- Young Lee, Jinwook Choi, Dong- Long-Geol Choi, and In So Kweon. Gradient- based camera exposure control for outdoor mobile platforms. IEEE Transactions on Circuits and Systems for Video Technology, (9(6):1569-1555
2018.
[18] Zichao Zhang. Christian Forster, and Davide Scaramuzza. Active exposure control for robust visual odometry hdr environments, In 2017 IEEE international conference on robotics and automation (ICRA),
pages 3894-3901. IEEE, 2017.
[19] Joowan Kim, Younggun Cho, and Ayoung Kim. Exposure control using bayesian optimization based on entropy weighted image gradicnt. In 2018 IEEE International conference on robotics and automation
(ICRA). pages 857-864. IEEE, 2018 [20] S Efimov, A Nefyodov, and M Rychagov. Block-based image exposure assessment and indoor/outdoor classification, In Proc. of 17th Conf. on Computer Graphics GraphiCon, 2007.
[21] Xuan Dong. Lu Yuan, Weixin Li, and Alan I. Yuille. Temporally consistent region-based video exposure correction. In 2015 IEEE International Conference on Multimedia and Expo (ICME), pages 1-6. IEEE, 2015.
[22] Yuzhe Yang. Liwu Xu, Leida Li, Nan Qie, Yaqian Li, Peng Zhang, and Yandong Guo. Personalized image aesthetics assessment with rich attributes. arXiv preprint arXiv:2203. 16754, 2022.
[23] Shu Kong, Xiaohui Shen, Zhe Lin, Radomir Mech, and Charless Fowlkes. Photo aesthetics ranking network with attributes and content adaptation. In European conference on computer vision, pages
662-679. Springer. 2016.
[24] Wenhan Zhu, Guangtao Zhai, Zongxi Han, Xiongkuo Min, Tao Wang, Zicheng Zhang, and Xiaokang Yangand. A multiple attributes image quality database for smartphone camera photo quality assessment. In 2020 IEEE International Conference on Image Processing (ICIP), pages 2990-2994. IEEE, 2020.
[25] Michele A Saad, Alan C Bovik, and Christophe Charrier. Blind image quality assessment: A natural scene statistics approach in the dct domain. JEEE transactions on Image Processing, 21(8):3339 3352, 2012. [26] Lijun Zhang. Lin Zhang, Xiao Liu, Ying Shen, and Dongqing Wang. Image exposure assessment: benchmark and deep convolutional neural networks based model. In 2018 IEÉE International Conference on Multimedia and Expo (ICME). pages 1-6, IEEE, 2018
[27] Ansel Adams. The negative: Exposure and development. ansel adams basic photography series/book 2. New York Graphic Sociery, Boston, MA, USA, 1948.
[28] Weijia Wu, Yuzhong Zhao, Mike Zheng Shou, Hong Zhou, and Chunhua Shen. Diffumask: Synthesizing
images with pixel-level annotations for semantic segmentation using diffusion models. In Proceedings of
the IEEE/CVF International Conference on Computer Vision, pages 1206-1217, 2023.
[29] Zhongwei Ren, Zhicheng Huang, Yunchao Wei, Yao Zhao, Dongmei Fu, Jiashi Feng, and Xiaojie Jin. Pixellm: Pixel reasoning with large multimodal model. arXiv preprint arXiv: 0202228 2023.
[30] Anmin Liu, Weisi Lin, and Manish Narwaria. Image quality assessment based on gradient similarity. IEEE
Transactions on Image Processing, ((0)):15001512, 2011.
[31] Dario Fuoli, Luc Van Gool, and Radu Timofte. Fourier space losses for efficient perceptual image super-resolution. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages
2360-2369, 2021.
[32] Jie Huang, Yajing Liu, Feng Zhao, Keyu Yan, Jinghao Zhang, Yukun Huang, Man Zhou, and Zhiwei Xiong. Deep fourier-based exposure correction network with -rrqual-frequency intcraction. In European Conference on Computer Vision, pages 163-180. Springer. 2022.
[33] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne, Journal of machine learning
research, 9(1)), 2008.
[34] Hanting Chen, Yunhe Wang, Tianyu Guo, Chang Xu, Yiping Deng, Zhenhua Liu, Siwei Ma, Chunjing Xu, Chao Xu, and Wen Gao. Pre-trained image processing transformer. In Proceedings of the IEFF/CVF conference on computer vision and pattern recognition. pages 12299-12310, 2021.
[35] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang. Stephen Lin, and Baining Guo. Swin Hicrarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012-10022, 2021.
transformer:
[36] Haiping Wu, Bin Xiao, Noel Codella, Mengchen Liu, Xiyang Dai, Lu Yuan, and Lei Zhang. Cvt: Introducing convolutions to vision transformers. In Proceedings of the IEEE/CVF international conference computer vision, pages 22 31, 2021.
11

---

[37]
Kun Yuan, Shaopeng Guo. Ziwei Liu, Aojun Zhou, Fengwei Yu, and Wei Wu. Incorporating convolution designs into visual transformers. In Proceedings of the IEEE/CVF international conference on computer pages [38] Xiaogang Xu, Ruixing Wang, and Jiangbo Lu. Low-light image enhancement via structure modeling and guidance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9893-9903, 2023.
vision. pages 579-588, 2021.
[39] Elad Richardson, Yuval Alaluf, Or Patashnik, Yotam Nitzan, Yaniv Azar, Stav Shapiro, and Daniel Cohen-
Or. Encoding in style: stylegan encoder for image-to-image translation. In Proceedings of the IEEF/CVF conference computer vision and pattern recognition, pages 2287-2296, 2021.
[40] Omer Tov, Yuval Alaluf, Yolam Nitzan, Or Patashnik, and Daniel Cohen-Or. Designing an encoder for stylegan image manipulation. ACM Transactions on Graphics (TOG), 40(4):1-14, 2021.
[41] Tengfei Wang. Yong Zhang, Yanbo Fan, Jue Wang, and Qifeng Chen. High-fidelity gan inversion for image attribute editing. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 1379-1 11388, 2022.
[42] Wenqi Ren, Sifei Liu, Lin Ma. Qianqian Xu, Xiangyu Xu. Xiaochun Cao, Junping Du, and Ming- Hiug- Yang. Low-light image enhancement via a deep hybrid network. IEEE Transactions on Image Processing,
28(9):4364-437 2019.
[43] Jirong Zhang, Chuan Wang, Shuaicheng Liu, Lanpeng Jia, Nianjin Ye, Jue Wang. Ji Zhou, and Jian Sun. Content-aware unsupervised deep homography estimation. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23 28, 2020, Proceedings, Port I 16, pages 653 069366696000 Springer, 2020.
[44] Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine Süsstrunk. Slic superpixels compared to ofatc-of-the-art superpixel methods. IEEE transactions on pattern analysis and machine intelligence, 34(1 11:2274-22282 2012.
[45] Hossein Talebi and Peyman Milanfar. Nima: Neural image assessment. IEEE transactions on image processing, 2798):3998 4011 2018.
[46] Vijie Zhou, Chao Li, Jin Liang, Tianyi Xu, Xin Liu, and Jun Xu. 4k-resolution photo exposure correction at 125 fps with" 8k parameters. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pages 1587-1597, 2024.
[47] Yuanhao Cai, Hao Bian, Jing Lin, Haoqian Wang, Radu Timofte, and Yulun Zhang. Retinexformer:
One-stage retinex-basee transformer for low-light image enhancement. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12504-12513, 2023.
[48] Dewei Zhou, Zongxin Yang, and Yi Yang. Pyramid diffusion models for low-light image enhancement. arXiv preprint arXiv:2305. 10028, 2023.
[49] Yuhui Wu, Chen Pan, Guoqing Wang. Yang Yang, Jiwei Wei, Chongyi Li, and Heng Tao Shen. Learning sewantic-aware knowledge guidance for low-light image cnhancement. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 1662-1671. 2023.
[50] Jinhui Hou, Zhiyu Zhu, Junhui Hou, Hui Liu, Huanqiang Zeng, and Hui Yuan. Global structure- aware diffusion process for low-light image enhancement. Advances in Neural Information Processing Systems, 36, 2024.
[51] Zhexin Liang, Chongyi Li. Shangchen Zhou, Ruicheng Feng, and Chen Change Loy. Iterative prompt learning for unsupervised backlit image enhancement. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1094-8103, 2023,
[52] Lorenzo Agnolueci, Lconardo Galteri, Marco Bertini, and Alberto Del Bimbo. Amiqa: Learning distortion manifold for image quality assessment. In Proceedings of the IEEE/CVF Winter Conference on Applications
of Computer Vision. pages 189-198, 2024.
[53] Avinab Saha, Sandeep Mishra, and Alan C Bovik. Re-iqa: Unsupervised learning for image quality assessment in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5846-5855, 2023.
[54] Guanyi Qin, Runze Hu, Yutao Liu, Xiawu Zheng, Haotian Liu, Xiu Li, and Yan Zhang. Data-efficient image quality assessment with autention-panel decoder. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37. pages 2091-2100, 2023.
[55] Junjie Ke, Qifei Wang, Yilin Wang. Peyman Milanfar, and Feng Yang. Musiq: Multi-scale image quality transformer. In Proceedings of the IEEE/CVF international conference on computer vision, pages
5148-5157, 2021.
[56] Shuang Ma. Jing Liu, and Chang Wen Chen. A-lamp: Adaptive layout-awart multi-patch deep convolu- tional neural network for photo aesthetic assessment. In CVPR, pages 722-731, 2017.
[57] Vlad Hosu, Bastian Goldlucke, and Dietmar Saupe. Effective aesthetics prediction with multi-level spatially
pooled features. In CVPR, 2019.
[58] Shuai He, Yongchang Zhang, Rui Xie, Dongxiang Jiang, and Anlong Ming. Rethinking image aestheties assessment: Models, datasets and benchmarks. IJCAI, 2022.
[59] Shuai He, Anlong Ming, Shuntian Zheng, Haobin Zhong, and Huadong Ma. Eat: An enhancer for orriettes-oorentee transformers. In Proceedings of the 31st ACM International Conference on Multimedia. pages 1023-1032, 2023.
[60] Haoning Wu, Zicheng Zhang, Weixia Zhang, Chaofeng Chen, Liang Liao, Chunyi Li, Yixuan Gao, Annan Wang, Erli Zhang, Wenxiu Sun, et al. Q-align: Teaching Imms for visual scoring via discrete text-defined levels. arXiv preprint arXiv:2312. 17090, 2023.
12

---

[61] Chen Wei, Wenjing Wang, Wenhan Yang. and Jiaying Liu. Deep retinex decomposition for low-light enhancement. arXiv preprint a808 04560, 2018.
[62]
Wenhan Yang, Wenjing Wang, Haofeng Huang. Shiqi Wang, and Jiaying Liu. Sparse gradient regularized deep retinex network for robust low-light image enhancement. IEEE Transactions on Image Processing, 30:2072-2086, 2021.
[63] Vladimir Bychkovsky. Sylvain Paris, Eric Chan, and Frédo Durand. Learning photographic global tonal adjustment with a database of input/output image pairs. In CVPR 2011. pages 97-104. IEEE, 2011. [64] Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint wXiv:1412.6988) 2014.
13

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and/or introduction clearly state the claims made, shown the end of the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We briefly discuss the limitations of this paper's work in Appendix A.6
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
14

---

This paper is not concerned with proving specific theorems, all assumptions have been clearly stated or referenced.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The code and dataset are available the supplementary material to support the reproducibility of experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
15

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
The code and dataset are available the supplementary material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (bttps://nipp) cc/
SodeuddddeeeddddSdddiuuu ssionPolicy for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (ht
/ httppss.cc/pbllc/guuicc...
See//oooSuulll ss wwwssssooiiiggos..o for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All the training details and protocols are provided Appendix A.3
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: All results of this paper not accompanied by error bars, confidence intervals, or statistical significance tests, and, at the same time, do not affect the conclusions this paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
16

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
Experiments compute resources are provided Appendix A.3
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't It make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: Our code strictly follows the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- cration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: At the end of the abstract and introduction, we briefly discuss the potential positive societal impacts, and we found no negative societal impacts of the work performed.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
17

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [Yes]
Justification:
We describe safeguards in Appendix A.7
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We describe Licenses in Appendix A.7
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of use package should be provided. For popular datasets, baaasetts.ppprssseee papersswss thoode. com datasstt has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
18

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: All new assets introduced in the paper well documented.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/moodl as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper is not crowdsoureing experiments and research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
19

---

Exposure
atiieasca 12%
100
Agproprate
seneessaalldeee anad
Appropriale
37%
80
Somewhar Appeopriste
70
Sightty LightLow or 10% Overexpaeed Underexppsed
12%
60
Higgglh now or 20% Overeppoe
50
Overall Too DarisBrigh
40
Slightly HightTow 30% Overespoedd Undeeeppuoeee
30
Very High/Low 30% Overesposed Underexposed
Very High 40% UnnesppedUUnkkreppppe
Large Brightness Contressssssortsstttlsa CorrresssssettttrDDnuuun Range Problems
Doinspeedd Inderspornl
afall areans 31% _!.....
(Alneet) Completely Blok/Wite
(4) Scmantic segmentation reslls
(b) Our pixal-evel IEA results
(c) IEA eiteeia
IEA Score 40
Figure 9: Based on the IEA criteria provided by different manufacturers, our pixel-level IEA results can be mapped to the corresponding IEA scores according to the provided criteria without fine-tuning,
A
Appendix
A.1 Image Collection Details
1) Varied Scenes. The scene defines the image's content and overall lighting conditions. As shown Fig. (())b), IEA40K spans over 50 scene categories, enabling a comprehensive understanding of image exposure in dynamic scenes. 2) Diverse Light. The type and direction of the light source greatly influence light distribution within an image. Our dataset includes a variety of light sources and directions, as shown in Fig. (cc). We also consider the shading tone's 's impact an image's visual appeal, selecting images with dark, clear, and neutral tones. 3) Sufficient Devices. The performance of the capturing device images from over 27 different devices, ranging from smartphones like the iPhone 13 and Huawei Mate40 to digital cameras from Canon, Nikon, and others. 4) Uniform Resolution. Initially captured at high resolutions, our images were downsized for manageability. Keeping the aspect ratio intact, we reduced the image size that the shorter side measures 512 pixels. storing them in PNG format to balance quality and model compatibility. 5) Comprehensive Simulations. Capturing varied exposure scene poses challenge. To overcome this, simulated 80% of the images
affect image quality and exposure. To address this, our dataset features
levels for the different exposure EV levels from the original raw files, or collected images from MIT-Adobe FiveK dataset [63]. After expert selection and adjustment, we compiled a set of 8 specific exposure levels for each raw image, resulting in I ideal reference and 8 exposure distorted images per scene.
A.2 Criteria-oriented IEA without fine-tuning
Different manufacturers have their own criteria for IEA scoring. Our pixel-level IEA results can adapt well to these different criteria. In Fig. [ we provide an example where manufacturers can map the pixel-level IEA results their overall scoring criteria to ultimately obtain their IEA scores.
A.3 Experimental Settings
Our model is implemented in Py Torch and trained with the Adam optimizer [64] We set the initial learning rate to 3 10 with decay rate 0.1 after every 10 epochs, and the mini- bati- size is set to 36. On the Intel 10900X CPU and RTX 3090 platform, the entire training time is about 14 hours for epochs (early stopping). and the inference time is 0.083 seconds for a 256 x 256 image (supports larger input sizes). To reduce the bias caused by random splitting, we run the random train-test splitting operation five times, and the comparison results are reported as the average of the five evaluation experiments.
A.4
Why not Employ Light Enhancement Methods to Obtain an Exposure Residual?
There a common misconception that subtracting the inputs and outputs from existing light enhance- ment methods, comparing distorted images to reference images from existing light enhancement
20

---

datasets, will yield the pixel-level IEA prediction results or ground truth (exposure residual) described in this paper. However, there are two fundamental differences: 1) Different Data Production Pro- cesses: light enhancement methods only optimize reference images' exposure in global areas through linear adjustments, e.g., adjusting the exposure value (EV) across the entire image. Our method involves non-linear adjustments by experts to optimize local exposure in specific areas of reference images. Additionally, human experts correct any errors in the exposure residual. As result, the exposure residual obtained from these two approaches contains different amounts of information. 2) Different Task Objectives: light enhancement methods generally aim to rectify image issues to a level acceptable to human vision, ensuring consistency in image semanties and using complete reference images for supervision. In contrast, our task focused solely on resolving exposure issues. We do not consider the impact of image semantics predictions attempt restore image semantics. Instead, we use only the exposure residual for supervision, eliminating other factors that could impact exposure quality.
A.5 More Performance Evaluations
Comparison of Different Wavelets. As outlined in the paper, the Haar wavelet distinctly aligns its component decomposition with exposure characteristics, a unique attribute not shared by other wavelets. Experimental results also show that the Haar wavelet surpasses other wavelets in perfor- mance. presents comparative analysis of the Haar wavelet against other notable wavelets (Daubechies and Symlet) on the IEA40K dataset.
Table 5: Ablation of various wavelets.
MAE
SSIMT
Haar
0.03
0.75
Daubechies
0.07
0.49
Symlet
0.08
0.41
Table 6: Ablation of different loss functions.
Epoch,
SSIMf
Ll-norm L2-norm
30
0.75
50
0.62
Smooth LI
45
0.67
Comparison of Different Loss Functions. During training, we evaluated three primary types of loss functions: Ll-norm, L2-norm, and Smooth Ll. Comparative results are detailed in Table 6 The Ll-norm demonstrates superior robustness and faster training speeds, evidenced by earlier convergence epochs in the IEA task.
Table 7: The PSNR results on IEA40k.
FEC
MSLT
Retinex
SMG
PSNR
23.22
21.02
23.8
24.17
SKF
GSAD
LIT
Ours
PSNR1
21.68
19.37
23.51
28.04
Comparison of PSNR Performance. PSNR can also serve as an evaluation metric the pixel level, and comparisons of some methods are presented in Table 7 which also demonstrates our SOTA performance.
A.6 Limitations
The dataset includes few images of high-speed moving objects because capturing well-aligned images with different exposures for such objects is difficult. The misalignment cannot using existing image alignment methods. Therefore, different exposure images for these objects can only be simulated by adjusting the EV value of ideal images.
be easily corrected
21

---

Additionally, resizing high-resolution images for evaluation can significantly compress the image, affecting pixel-level exposure assessments. Increasing the input size during network training can address this issue but will also increase the inference time.
A.7 Safeguards and Licenses for Existing Assets
The original owners of assets (e.g., code, data, models) used in the paper are properly credited, and the licenses and terms of use are explicitly mentioned and properly respected, ensuring that there are no copyright issues and no risk of misuse.
22


Reviews:
[{"review_id": "UDa2TKnB8T", "reviewer": "Reviewer_Wvdf", "summary": "This paper proposes an innovative no-reference image exposure assessment method, transitioning from traditional holistic image evaluation to fine-grained pixel-level assessment. This approach effectively addresses the shortcomings of existing techniques in terms of accuracy and generalization. Researchers have developed P-IEANet, a pixel-level evaluation network that utilizes Haar discrete wavelet transform to analyze image brightness and structural information, enabling exposure assessment without reference images. Additionally, to support this method, the researchers have constructed the IEA40K dataset, which contains 40,000 images with detailed pixel-level annotations, covering diverse lighting conditions and devices. Using this dataset, they established a comprehensive benchmark including 19 methods, demonstrating that P-IEANet achieves state-of-the-art performance across multiple evaluation metrics. This work not only enhances the accuracy of no-reference IEA tasks but also provides valuable resources and new research directions for the image exposure research community. Future work will focus on optimizing the framework to support multimodal outputs and enhancing exposure perception in AI-generated content.", "questions": "Please refer to the comments in the weakness part.", "limitations": "Not applicable", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Pixel-level Evaluation: The P-IEANet proposed in the article is capable of conducting pixel-level image exposure assessment, which offers a more refined analysis and more accurate results compared to traditional overall image assessment.\n- Innovative Model Architecture: By integrating the Haar Discrete Wavelet Transform with specific feature extraction modules, P-IEANet is able to analyze images from both the brightness and structural perspectives, providing a more comprehensive exposure assessment.\n- Large-scale Dataset: The article has constructed the IEA40K dataset, which is a large-scale, diverse image dataset that provides rich resources for evaluation and training.", "weaknesses": "- The author mentions in the abstract that the code and dataset can be found in the supplementary materials, but there is no relevant section in the supplementary materials.\n- There is no explanation as to why the Haar wavelet was chosen over other wavelets.\n- The aesthetic quality of Figure 4 needs to be improved.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">We appreciate the reviewer's positive feedback on our paper, particularly for acknowledging \"an innovative no-reference image exposure assessment method.\" We have addressed the questions raised below.\n\n---\n\n**Q1:** The author mentions in the abstract that the code and dataset can be found in the supplementary materials, but there is no relevant section in the supplementary materials.\n\n>**A1:**  Unfortunately and regrettably, the dataset and code in the supplementary materials are **not visible due to unknown issues**. We sincerely apologize for any inconvenience caused, but we **did include** the code and dataset in the supplementary materials, as evidenced by Figure 1 of the PDF under the \"global\" response. Following the NIPS Rebuttal guidelines, we have provided an anonymized link to the AC for the code and dataset in a separate comment. Additionally, if our paper is accepted, we will release all resources publicly.\n\n---\n\n**Q2:** There is no explanation as to why the Haar wavelet was chosen over other wavelets.\n\n>**A2:** There are primarily four reasons for our selection:\n>1) As outlined in the paper (lines 114-129), the Haar wavelet distinctly aligns its component decomposition with exposure characteristics, a unique attribute not shared by other wavelets.\n>2) The Haar wavelet excels in analyzing signals characterized by sudden variations. It is particularly adept for identifying areas in images that are underexposed or overexposed compared to normally exposed regions, which can be treated as signals with abrupt changes.\n>3) The transformations of the Haar wavelet, both forward and inverse, are reversible. A proficient approach to estimating exposure residuals involves reconstructing the \"ideal exposure image\" in the latent space and subsequently assessing the deviation of each pixel in the input image from this ideal exposure. This methodology is essential, as highlighted in line 46 of the original paper: \"As a no-reference method, it should effectively simulate reference images in non-preset scenarios, operating similarly to full-reference methods.\" The Haar wavelet facilitates this process effectively.\n>4) Experimental results show that the Haar wavelet surpasses other representative wavelets in performance. Table 1 in the PDF under the \"global\" response, presents a comparative analysis of the Haar wavelet against other notable wavelets (Daubechies and Symlet) on the IEA40K dataset.\n\n---\n\n**Q3:** The aesthetic quality of Figure 4 needs to be improved.\n\n>**A3:** We sincerely apologize for the aesthetic issues in the figure and greatly appreciate the suggestion. In order to improve the  figure's quality in the final paper (if accepted), we have planned to make the following improvements :\n>1) Adjusting the color scheme to harmonize the appearance of various modules;\n>2) Modifying the layout to balance the content, particularly on the left side of Figure 4, by adjusting the proportions of core modules and reducing the emphasis on non-core modules;\n>3) Standardizing the shapes of DWT Kernels to match those of other modules.\n---\n>In conclusion, the reviewer's suggestion is highly valued, and we will incorporate detailed results into the final version of the paper (if accepted).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1LJbys9qQZ", "reviewer": "Reviewer_B8CP", "summary": "This paper proposes a new no-reference image exposure assessment method, Pixel-level IEA Network (P-IEANet), which analyzes and evaluates image exposure from the perspectives of brightness and structure using discrete wavelet transform (Haar DWT). Also, a dataset exclusively tailored for IEA, called IEA40K, is constructed. According to a comprehensive evaluation of methods on the IEA40K dataset, the proposed method achieves SOTA performance and offers advantages for the exposure enhancement community.", "questions": "1. Why use Haar DWT to decompose an image into components with different frequencies, and what are its advantages compared to other similar methods, such as other types of DWT? \n2. Why is the DWT^{-1} step necessary?\n3.  What is the reason for using the l1 norm as the loss function?\n4.  Why choose MAE to measure the structure and lightness similarity between the ground truth and predicted exposure residual, instead of using MSE?", "limitations": "The authors have adequately addressed the limitations.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper demonstrates very good originality as it is the first realization of pixel-level image exposure assessment. The authors have designed corresponding methods specifically addressing the characteristics of this problem and achieved satisfying results. Detailed explanations of the motivation and the current state of research are provided. Both the principles and the implementation of the method are clearly presented. The experimental results effectively demonstrate the performance of the proposed method. This paper not only proposes a new IEA method but also contributes a new dataset and benchmark, providing a significant boost to the IEA and exposure-related community.", "weaknesses": "Haar DWT is used to decompose an image into components with different frequencies, but the advantages of this method compared to other similar methods are not adequately explained. In the method section of this paper, some operations lack clear motivation or principles. For example, the reason for applying the DWT^{-1} and the choice of l1 norm as the loss function are not well explained. In the experiments section, SSIM and MAE are adopted to measure the structure and lightness similarity between the ground truth and predicted exposure residual. However, as a perceptual IQA metric, SSIM may not be suitable for evaluating the prediction accuracy of exposure residuals. The paper claims that the proposed method has improved adaptability across varying criteria and scenarios, but this is not well demonstrated in the experiments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">We greatly appreciate the reviewer's positive feedback on our paper, especially for acknowledging that it \"not only proposes a new IEA method but also contributes a new dataset and benchmark, providing a significant boost to the IEA and exposure-related community.\" We hope the following responses will address any remaining concerns.\n\n---\n\n**Q1-1:** Why use Haar DWT to decompose an image into components with different frequencies?\n\n>**A1-1:** The primary objective is to minimize interference from signals of various frequency domains during the feature extraction phase. This strategy not only enables the model to analyze different features more accurately, thus boosting performance, but also speeds up model training. Supporting experimental results are detailed in Table 2 of the PDF under the \"global\" response. The data clearly show that employing the Haar DWT significantly enhances performance and decreases the number of epochs needed for convergence.\n\n---\n\n**Q1-2:** What are Haar DWT advantages compared to other similar methods, such as other types of DWT?\n\n>**A1-2:** There are primarily four reasons for our selection:\n>1) As outlined in the paper (lines 114-129), the Haar wavelet distinctly aligns its component decomposition with exposure characteristics, a unique attribute not shared by other wavelets.\n>2) The Haar wavelet excels in analyzing signals characterized by sudden variations. It is particularly adept for identifying areas in images that are underexposed or overexposed compared to normally exposed regions, which can be treated as signals with abrupt changes.\n>3) The transformations of the Haar wavelet, both forward and inverse, are reversible. A proficient approach to estimating exposure residuals involves reconstructing the \"ideal exposure image\" in the latent space and subsequently assessing the deviation of each pixel in the input image from this ideal exposure. This methodology is essential, as highlighted in line 46 of the original paper: \"As a no-reference method, it should effectively simulate reference images in non-preset scenarios, operating similarly to full-reference methods.\" The Haar wavelet facilitates this process effectively.\n>4) Experimental results show that the Haar wavelet surpasses other representative wavelets in performance. Table 1 in the PDF under the \"global\" response, presents a comparative analysis of the Haar wavelet against other notable wavelets (Daubechies and Symlet) on the IEA40K dataset.\n\n---\n\n**Q2:** Why is the DWT^{-1} step necessary?\n\n>**A2:** As outlined in the third point of Q1-2, DWT^{-1} is designed to reconstruct the decomposed components of an image back into the \"ideal exposure image\" within the latent space. While DWT is utilized for enhanced analysis of **features**, DWT^{-1} helps the model in synthesizing these features back into the ideal exposure **image**. This synthesis process is crucial for identifying the discrepancies between the ideal and input images, thus aiding in the prediction of exposure residuals. Omitting the DWT^{-1} process would lead to reduced model performance, as evidenced in Table 3 of the PDF.\n\n---\n\n**Q3:** What is the reason for using the l1 norm as the loss function?\n\n>**A3:** Training results show that the L1-norm outperforms the alternatives. \nDuring training, we evaluated three primary types of loss functions: L1-norm, L2-norm, and Smooth L1. Comparative results are detailed in Table 4 of the PDF. The L1-norm demonstrates superior robustness and faster training speeds, as evidenced by earlier convergence epochs in the IEA task.\n\n---\n\n**Q4:** Why choose MAE to measure the structure and lightness similarity between the ground truth and predicted exposure residual, instead of using MSE?\n\n>**A4:** MAE provides a more direct quantification of the deviation of each pixel from its ideal exposure compared to both SSIM and MSE. Additionally, MAE is less sensitive to extreme values than MSE, offering a more robust and consistent measurement of errors. Therefore, using MSE as an option is well-reasoned. In our work, we selected MAE as a complement to SSIM. \n\n---\n\n>In conclusion, the reviewer's suggestion is highly valued, and we will incorporate detailed results into the final version of the paper (if accepted).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lv268bJBr4", "reviewer": "Reviewer_TsXB", "summary": "This work tackles the challenges in image exposure assessment from three aspects: models, datasets, and benchmarks. Specifically, A P-IEANet model based on DWT is proposed, which can generate pixel-level assessment results in a no-reference manner. An exposure-oriented dataset IEA40K is collected to cover various lighting scenarios, devices, and scenes, which are annotated by more than 10 experts with pixel-level labels. A comprehensive benchmark of 19 methods is conducted on the collected IEA40K dataset, where the proposed P-IEANet delivers the best performance.", "questions": "Instead of SSIM and PSNR, I think Eq. (7) can also be employed as a pixel-level performance measure.", "limitations": "Yes", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+ Decomposing images into lightness features and structure components using Haar DWT is theoretically reasonable and empirically effective as presented in this work.\n+ The dataset construction strategies described in Sec. 4.1 and Sec. 4.2 provide valuable insights to the related community.\n+ The proposed model delivers good performance, even outperforming the LMM-based model Q-align.", "weaknesses": "- Holistic level assessment is performed on SPAQ. It should be straightforward to convert the pixel-level annotations to holistic level annotations in the proposed IEA40K dataset because the pixel-level annotations contain more information than the holistic level annotations.\n- Would the performance of IEA models be boosted by jointly training (like the practices used in UNIQUE, LIQE, etc.) the model on the combination of IEA dataset and general-purpose IQA datasets?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">We sincerely appreciate the reviewer's positive feedback, characterizing our paper as \"theoretically reasonable and empirically effective\" and noting that it \"provides valuable insights to the related community.\" We have thoroughly addressed the reviewer's inquiries, which we believe will significantly improve the quality of our paper.\n---\n**Q1:** \"Holistic level assessment is performed on SPAQ. It should be straightforward to convert the pixel-level annotations to holistic level annotations in the proposed IEA40K dataset because the pixel-level annotations contain more information than the holistic level annotations.\"\n\n>**A1:** Thanks for the insightful comments. To **verify** the suggestion and **convert annotations** between the pixel-level annotations and holistic level annotations, we have calculated the average absolute value at the pixel level and adjusted this value by subtracting it from 1, thus normalizing it within a range of 0 to 1. Subsequently, we have conducted additional experiments on this transformed dataset, **IEA40K-h**, to verify the methods' performance.\n>1) Given the constraint of rebuttal time, we selected representative methods to **retrain** on the IEA40K-h dataset, as detailed in Table 5 of the PDF under the \"global\" response. Our approach unequivocally **outperformed** the others, achieving SOTA results. The higher SRCC and LRCC metrics obtained with  IEA40K-h, compared to those on the SPAQ dataset, indicate that  IEA40K-h allows models to learn features more effectively.\n>2) Table 6 in the PDF illustrates the cross-dataset validation for SPAQ and  IEA40K-h on the LRCC metric. Models trained on  IEA40K-h, even **without fine-tuning** on SPAQ, demonstrated performance comparable to those directly trained on SPAQ. In contrast, models trained on SPAQ did not generalize effectively to  IEA40K-h, indicating that  IEA40K-h provides **richer** holistic information that enhances model generalization.\n\n>The valuable findings will be further elaborated in the final version of the paper (if accepted). We sincerely appreciate the insightful suggestion.\n\n---\n\n**Q2:** Would the performance of IEA models be boosted by joint training (like the practices used in UNIQUE, LIQE, etc.) the model on the combination of the IEA dataset and general-purpose IQA datasets?\n\n>**A2:** Thank you for your valuable advice. Currently, no existing dataset combines pixel-level and holistic-level annotations. Following the reviewer's suggestion in Q1, we derived holistic-level annotations (IEA40K-h) from our IEA40K dataset and adopted a LIQE-like method to jointly learn these tasks. Comparing the original results, 1) For pixel-level tasks on IEA40K, the performance metrics changed as follows: MAE=0.03 (+0.0), SSIM=0.76 (+1.3\\%). 2) For the holistic-level task on IEA40K-h, the changes were: LRCC=0.91 (+4.5\\%), SRCC=0.87 (+4.8\\%). \nThis joint training approach leverages the detailed information in pixel-level annotations to enhance the model's understanding of holistic tasks. We sincerely appreciate the reviewer's suggestion and will include these findings in the final paper (if accepted).\n\n---\n\n**Q3:** Instead of SSIM and PSNR, I think Eq. (7) can also be employed as a pixel-level performance measure.\n\n>**A3:** Yes! Eq. (7) closely resembles the MAE metric detailed in Table 1 of our paper and can serve as a measure of pixel-level performance.\n---\n>In conclusion, the reviewer's suggestion is highly valued, and we will incorporate detailed results into the final version of the paper (if accepted).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BtL8B1Nrw3", "reviewer": "Reviewer_k7zi", "summary": "The paper introduces a novel paradigm that extends Image Exposure Assessment (IEA) from an image-level to a pixel-level framework. This paradigm comprises three components: model, dataset, and benchmark. Concerning the model, the study introduces the Pixel-level IEA Network (P-IEANet). This network processes images of varying exposures, separates them into low and high-frequency components via a discrete wavelet transform, assesses brightness with the low-frequency component, and evaluates structure with the high-frequency component, ultimately delivering pixel-level assessment results. Regarding the dataset, the authors have developed a new dataset, IEA40K, which includes 40,000 images featuring diverse exposures and corresponding pixel-level annotations. Finally, the paper presents comprehensive experiments on both holistic and pixel-level assessments, yielding promising results.", "questions": "I find the proposed task interesting, while I have reservations about certain assertions made in the paper, terms that lack clarity, and the absence of adequate justification for the introduction of certain tasks without clear motivation. For further details, see the below list. I've organized my concerns and suggestions according to their significance to assist the authors in prioritizing their rebuttal.\n\n1. The terminology employed in the paper suffers from a lack of clarity, necessitating more detailed explanations. For instance, the term ``exposure`` conventionally refers to the duration of exposure time in the context of capturing images with digital cameras, typically considered as a global attribute of an image. However, the paper introduces the concept of ``pixel-level exposure`` without providing a sufficient explanation, which is illogical in the literal sense. Similarly, the term ``exposure residual`` is introduced but remains poorly defined, further complicating the understanding of the methodology. Probably, the paper misuses the concepts of ``exposure`` and ``brightness``, which cannot be used interchangeably, however.\n\n2. The motivation behind the paper remains ambiguous. It argues that a holistic evaluation of image exposure encounters two primary issues: (1) a dilemma between applicability and practicability, and (2) a narrow inductive bias. However, the paper lacks a further explanation of these problems. Incorporating visual results from current holistic evaluation methods that exhibit these issues could more effectively and intuitively demonstrate the paper's motivation. In the current version, the necessity for a pixel-level image exposure assessment method is not clearly articulated, particularly under which circumstances such a technique would be essential.\n\n3. Related to the first point, the proposed method targets to predict ``exposure residual``, defined as ``(reference - input)`` in RGB space. However, the rationale behind this definition requires further justification. Specifically, it remains unclear why this definition is suitable for use as the ground truth in pixel-wise image exposure assessment. Additionally, it is essential to explore whether any disparity exists between the concepts of ``(reference - input)`` in RGB space and the actual pixel-wise score for image exposure.\n\n4. For evaluation metrics, PSNR and SSIM are two commonly used pixel-wise metrics. However, this paper only adopts SSIM for evaluation. Including PSNR performances would provide a more convincing argument.\n\n5. While comparing pixel-level performance with other image enhancement methods, the paper derives these methods' ``exposure residual`` predictions by directly predicting the residual map. However, image enhancement techniques typically use loss functions designed to smooth the final outputs and align them with human perception, which may not be appropriate for predicting residuals. Although it is acknowledged that the difference between ``(output-input)`` and the proposed ``exposure residual`` exists, incorporating an additional ablation study that calculates the residual from ``(output-input)`` would likely provide a more comprehensive analysis.\n\n6. Important details, such as the architecture of the proposed Long Range Encoder (LRE) and Short Range Encoder (SRE), are missing, hindering the reproductivity of the proposed framework.\n\n7. The availability of the proposed dataset to the public is crucial for assessing the contribution of this work.", "limitations": "The paper adequately discusses the limitations of moving objects and image size while training. No negative social impact is present in this work.", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The paper initially proposes a pixel-level image exposure assessment paradigm, significantly enhancing precision in the field of image exposure assessment.\n\n2. The paper introduces an assessment network that employs discrete wavelet transform, an intriguing choice supported by several ablation studies.\n\n3. The paper proposes a large-scale, multi-exposure dataset with pixel-wise annotations derived from an automatic multi-exposure fusion technique, subsequently refined by human experts.\n\n4. The paper also demonstrates that the P-IEANet can potentially improve the performance of low-light image enhancement methods.\n\n5. The paper is well-composed, demonstrating a clear structure, precise language, and a logical flow of ideas.", "weaknesses": "The main weakness is that the paper lacks a well-defined definition for pixel-level image exposure assessment. For other details, please refer to the \"Questions\" part.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Q1: The terminology lacks clarity. \n\n>A1: Thanks for the reviewer's thought-provoking questions. \n>1) In the context of **evaluating images**, the term \"exposure\" is **no longer a global attribute of an image**. Even in the context of **capturing images**, as exemplified by the reviewer, the term \"exposure\" conventionally refers to **not only** exposure time but also two other parameters (aperture and ISO, referred to [1]). **Rather than being characterized as a global attribute** of the image, the parameters would be more appropriate to be described as a global attribute associated with the camera for capturing the image. Unfortunately, due to the claim made by **the classical photographic theory** (Adams’ theory) [2] that \"The exposure time is the same for all elements, but **the image exposure varies with the luminance of each subject element**,\" the coarse global camera exposure attribute fails to match each subject element in an image, potentially result in **some subject elements being under-exposed and others being over-exposed**. Therefore, in the context of **evaluating images**, the term \"exposure\" is no longer a global attribute, as referred to [2] that \"Any scene of photographic interest contains elements of different luminance; consequently, **the 'exposure' actually is many different exposures**.\" As exposure is **fundamental knowledge in IQA**, we sincerely apologize for not emphasizing it enough. \n>2) The concept of \"pixel-level exposure\" is logically consistent with the term \"exposure\" in the context of evaluating images. As claimed by the theory [2], the ideal exposure should be refined to different elements. In this paper, we innovatively **quantify subject elements by utilizing pixels** as the smallest units of exposure measurement.\n>3) The term \"exposure residual\" refers to the deviation observed in the actual exposure of each pixel compared to its ideal exposure in the context of evaluating images, as stated in our original paper at line 188: \"measuring the deviation of each pixel from its ideal exposure.\"  Numerically, values closer to -1 indicate overexposure, while values closer to 1 indicate underexposure, as detailed in Figure 7.\n>4) Exposure and brightness have distinct connotations; for instance, exposure can be either holistic or regional (with a pixel as the minimum unit), whereas brightness is solely at the pixel level with a value assigned from 0 to 255.\n\n>[1] Image exposure assessment: a benchmark and a deep convolutional neural networks based model, ICME 2018.\n\n>[2] Adams, The Negative: Exposure and Development. \n\n---\n\nQ2: The motivation and necessity remain unclear, requiring the demonstration of visual results.\n\n>A2: Thanks for the valuable advice. Figure 2 in our submitted PDF provides a visual example. Actually, the necessity arises from industrial demands. For instance: smartphone manufacturers currently still manually evaluate exposure images in no-reference scenes; however, by integrating pixel-level IEA (essential) and customized evaluation rules, we have successfully simulated and replaced this manual process for a **TOP-5 smartphone manufacturer**. If our paper is accepted, we will partially disclose this case on our website. \n---\nQ3:  The definition of exposure residual requires further justification.\n>A3: The insightful questions are greatly appreciated. \n>1) In training phrase, the exposure residual **is not obtained solely** from the difference (reference - input) in RGB space, but it undergoes further verification and adjustments **by experts** to ensure that the final exposure residual closely aligns with the perceived deviation of each pixel from ideal exposure (refer to Figure 7 and line 225 in the original paper).\n>2) The rationale behind the exposure residual and its suitability for ground truth is based on the inherent characteristics and practicality of pixel-wise data annotation for supervision. For experts, distinguishing between the reference and input images is relatively straightforward and far more accurate [1][2], thus facilitating practical data annotation. While the implementation of absolute evaluations is hindered by the absence of clear standardized criteria. As discussed in Q1, ideal IEA should be derived to the specific characteristics of each subject element, even at the pixel-level. Fortunately, the exposure residual provides direct and effective supervisory information for model training.\n>3) We are a little confused about \"any disparity\" in the comments. During prediction, the actual pixel-wise scores for image exposure are the exposure residuals **generated** by P-IEANet which has been trained using the supervision data (also the exposure residuals but adjusted by experts).  \n\n>[1] Descriptive Image Quality Assessment in the Wild, ECCV2024.\n\n>[2] Self-Supervised Multi-Task Pretraining Improves Image Aesthetic Assessment, CVPRW2021.\n\n---\n\nQ4: PSNR performances.\n\n>A4: Thanks for the insightful suggestion. The PSNR results, unequivocally showing our SOTA performance, can be found in Table 7 of our submitted PDF.\n---\nQ5: An additional ablation study.\n>A5: Thanks for the insightful suggestion. We have **retrained** (using the ground truth of inference images for supervision) representative image enhancement methods to directly predict the reference images instead of using the residual. Then the difference between the output and input was calculated as the residual. The SSIM results, which can be found in Table 8 of our submitted PDF, unequivocally show our SOTA performance **despite** the decrease in performance for all methods.\n\n---\nQ6: The architecture details of LRE & SRE.\n>A6: The classic LRE and SRE are detailed in Figure 3 of our submitted PDF.\n\n---\nQ7: The availability of the dataset.\n>A7: Unfortunately and regrettably, the dataset and code in the supplementary materials are **not visible due to unknown issues**. We apologize for any inconvenience caused and have provided them again. Pls refer to [\\#Wvdf, Q1].", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
General Response:

    We sincerely thank the reviewers for their efforts in reviewing our work and providing valuable comments. We highly appreciate the comments received, e.g., the positive comments on our contributions (4/4 reviewers), methods' performance (4/4 reviewers), our presentations (3/4 reviewers), and soundness (3/4 reviewers).

    We also deeply value the constructive feedback the reviewers have provided. We have taken great care to address the reviewers' questions, conducting additional experiments detailed in the PDF under the 'global' response to further support our claims.

Meta-review: The authors performed exceptionally well during the rebuttal and discussion stages, leading all reviewers to reach a consensus for acceptance. The AC concurs, acknowledging that this is a highly comprehensive study on non-reference IEA.
Decision: Accept (poster)