SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table C.1: The comparison of restoration performance of the DNN models regarding the annotations in UDC-SIT. "Input" refers to the PSNR and SSIM between the degraded and ground-truth images in UDC-SIT.
Input	DISCNet [15]	UDC-UNet [31]	Uformer-T [47]	ECFNet [54]	SRGAN [29]
Indoor	PSNR	21.17	26.63	27.79	27.58	28.64	25.06
SSIM	0.7437	0.8535	0.8715	0.8680	0.9058	0.8311
Outdoor	PSNR	20.53	25.18	26.13	26.15	26.66	23.37
SSIM	0.6924	0.8163	0.8346	0.8272	0.8795	0.7761
Day	PSNR	21.54	26.23	27.28	27.47	28.17	24.67
SSIM	0.7689	0.8669	0.8857	0.8823	0.9152	0.8408
Night	PSNR	19.63	25.32	26.62	25.99	27.06	23.62
SSIM	0.6012	0.7436	0.7633	0.7536	0.8485	0.7016
Sunlight	PSNR	20.48	24.49	25.52	25.62	26.22	22.72
SSIM	0.7488	0.8475	0.8670	0.8637	0.8966	0.8147
Artificial light	PSNR	20.37	26.40	27.67	27.19	28.37	24.72
SSIM	0.7026	0.8279	0.8459	0.8401	0.8895	0.8022
No light	PSNR	22.43	26.22	26.52	27.18	27.52	24.95
SSIM	0.7180	0.8521	0.8645	0.8623	0.8893	0.8103
Glare	PSNR	20.91	26.37	27.57	27.29	28.32	24.71
SSIM	0.7351	0.8456	0.8643	0.8598	0.9024	0.8211
Shimmer	PSNR	20.67	26.41	27.62	27.33	28.42	24.73
SSIM	0.7301	0.8413	0.8601	0.8557	0.9000	0.8181
Streak	PSNR	20.20	26.05	27.27	26.83	28.04	24.30
SSIM	0.7109	0.8266	0.8465	0.8400	0.8897	0.8026
typically irregular line of light or color observed in an image. Note that the shapes of glares and shimmers vary depending on factors such as day and night conditions, indoor and outdoor settings, and light sources, as illustrated in Figure B.2. In addition, Figure B.2(b) demonstrates that the shape and intensity of glares, shimmers, and streaks differ based on the location within the scene, even when originating from the same light source.
Light sources. The intensity and shape of flares vary also depending on the light sources. Flares generated by natural sunlight, as depicted in Figure B.2(d), exhibit an extreme intensity and often include streaks. Additionally, flares can have different characteristics under different conditions, such as day/night and indoor/outdoor settings. For example, the flare in Figure B.2(e) is also caused by natural sunlight, but its shape differs from the flare in Figure B.2(d). Also, Figure B.2(a), Figure B.2(b), Figure B.2(f), and Figure B.2(g) demonstrate variations in the shape of flares caused by different types of lights. Particularly, Figure B.2(b) demonstrates that artificial lights in close proximity during nighttime can induce streaks, and Figure B.2(g) highlights how different types of artificial lights result in various flare characteristics.
Day/Night. In the case of indoor images, such as those depicted in Figure B.2(g), the distinction between day and night is generally not required (labeled as "No distinction" in Table B.2). However, we distinguish between day and night when sunlight enters the indoor space through a window, leading to flare occurrence as shown in Figure B.2(e). Indoor images with windows, even if no flare occurs (Figure B.2(f)), are still labeled as day or night. This reflects that windows in indoor spaces may not always cause flares during the day, as flare occurrence can depend on the presence and intensity of sunlight between day and night.
Indoor/Outdoor. The illuminance outdoors is generally higher than that indoors during the daytime. Figure B.2(d) and Figure B.2(e) demonstrate that the flares caused by sunlight exhibit variations depending on whether the image is captured indoors or outdoors. Interestingly, the flares in Figure B.2(c) are caused by the reflection of sunlight on the exterior walls of the building.
C.3 Restoration performance regarding the annotations
Table C.1 shows the comparative analysis of the restoration performance achieved by different DNN models depending on UDC-SIT annotations. Since a UDC-SIT image generally has multiple annotations, the annotation type in a row cannot be regarded as the sole factor influencing UDC degradation when obtaining the PSNR and SSIM values. However, it is reasonable to acknowledge the annotation type as a significant factor that affects the PSNR and SSIM values.
Overall, there is a tendency for the restored image to have low PSNR and SSIM values when the input pair has low PSNR and SSIM values. For instance, Table C.1 demonstrates that input pair captured in outdoor environments exhibit lower PSNR and SSIM values than indoor images. In turn, the restored images by the models also demonstrate inferior PSNR and SSIM values for outdoor restorations compared to indoor restorations. This observation can be attributed to strong sunlight, and intense flares often accompanying outdoor scenes.
The low-light conditions at night influence the degradation significantly. Night conditions tend to be darker, increasing noise and introducing extreme flares. This phenomenon can be observed from the lower PSNR and SSIM values for the annotation Night compared to Day, as indicated in Table C.1.
The presence of flares and the type of light sources inducing flares are also significant factors in UDC degradation. Table C.1 demonstrates the impact of flares on degradation. The input images degraded by sunlight and restored images have low PSNR values compared to the no-light condition. However, interestingly, restored images from the images degraded by artificial light depict higher PSNR values compared to the no-light condition. This tells us that the DNN models are relatively proficient in restoring the degradation caused by artificial light compared to natural sunlight.
Under the sunlight condition, the SSIM values are higher than under the no-light condition. On the contrary, the SSIM values under the artificial light condition are less than those under the no-light condition. PSNR measures the noise level by comparing the original and degraded images. Increasing degradation leads to higher noise and lower PSNR values. On the other hand, SSIM measures structural similarity, considering brightness, contrast, and structural features. Thus, SSIM may exhibit slightly different sensitivity towards different types of degradation compared to PSNR [20]. If degradation factors like flares do not significantly reduce structural similarity, SSIM values will be relatively high.
When flares are present in a degraded image, glares and shimmers appear over a wider area than streaks. Glares and shimmers may significantly impact the reduction of PSNR and SSIM values. On the other hand, streaks typically occur in conjunction with glares and shimmers rather than as a standalone phenomenon. Thus, there may not be significant differences in PSNR and SSIM values based on glares, shimmers, or streaks individually, as shown in Table C.1. However, low-intensity artificial lights at night or natural sunlight in the daytime may cause extreme flares, including streaks. These extreme flares introduce more significant degradation than glares or shimmers alone. Thus, streaks generally exhibit slightly low PSNR and SSIM values compared to glares and shimmers due to their association with extreme flares, as illustrated in Table C.1.
High-resolution images taken by modern smartphones often require training the image-restoration DNN models by patching smaller images. The annotated information, such as indoor/outdoor, day/night, glare/shimmer/streak, and light sources, attached to each patch facilitate the patch-level training in UDC restoration tasks. For example, restoring streaks is challenging for the DNN models when performing the patch-level training, as illustrated in Figure B.1(c). Figure B.1(c) shows big streaks that cover almost the entire image. Providing the annotation information about the entire image can enhance patch-level training by enabling the model to learn correlations between patches and patterns across the patches. This additional information improves the model's ability to restore or enhance images effectively.
In summary, UDC-SIT annotations are crucial for advancing UDC research as none of the existing UDC datasets provide similar information.