SlowGuess/NeurIPS_2023_OCR huggingface dataset:
effectiveness of the TCN. Further details and discussions are available in the supplementary material.
In this section, we validate the effectiveness and scalability of our proposed TCN on various image enhancement tasks. We provide more experimental results in the supplementary material.
Figure 4: Toy experiment of self-reconstruction. The left is the setting of toy experiments with inserting different operations, and the right presents the self-reconstruction PSNR of testing images.
Figure 5: Feature visualization of toy experiment. In left and right parts, we show the feature in the TCN of underexposure and overexposure samples when testing them with inserting TCN in Fig. 4.
Underexposure Input
Overexposure Input
Underexposure's F
Overexposure's  $F$
Underexposure's  $\widehat{F}_1$
Overexposure's  $\widehat{F}_1$
4.1 Toy Experiment
To illustrate the proposed TCN has the ability to normalize the features while transition-constant, we introduce a toy experiment as shown in Fig. 4: we construct an encoder-decoder-based architecture for reconstructing the input image, where the TCN and other normalization formats are inserted between the encoder and decoder as different versions. Then we train this self-reconstruction architecture on 1000 samples from MIT-FiveK dataset [59] until its convergence and test the self-reconstruction effect on another 100 samples from the same dataset. The quantitative results in the right of Fig. 4 indicate that our TCN reconstructs the input image better than directly inserting IN, demonstrating the information transition-constant property. Furthermore, we test 100 underexposure samples and 100 overexposure samples from the SICE dataset, and we provide the feature distribution of  $F$  and  $\hat{F}_1$  (input and normalized output of the TCN) in the left part of Fig. 5, as well as feature maps in the right part of Fig. 5. The different exposure features processed by the TCN get to be
Settings	#Param	Flops (G)	LOL	Huawei	FiveK
DRBN (Baseline) [15]	0.53M	39.71	19.95/0.7712	20.64/0.6136	22.11/0.8684
+IN	0.53M	39.71	20.73/0.7986	21.01/0.6200	22.93/0.8727
+Original TCN	0.53M (+0)	39.71 (+0)	21.15/0.8190	21.12/0.6242	23.98/0.8851
+Affined TCN	0.53M (+0)	39.71 (+0)	21.29/0.8167	21.04/0.6231	23.92/0.8858
+Skip TCN	0.53M (+0)	39.77 (+0.07)	21.52/0.8271	21.15/0.6195	23.82/0.8832
SID (Baseline) [14]	7.40M	51.06	20.85/0.7845	19.68/0.6050	21.49/0.8425
+IN	7.40M	51.06	20.51/0.7858	20.09/0.6034	21.75/0.8453
+Original TCN	7.40M (+0)	51.06 (+0)	21.43/0.7913	20.53/0.6067	23.11/0.8581
+Affined TCN	7.40M (+0)	51.06 (+0)	21.35/0.7867	20.62/0.6077	23.20/0.8624
+Skip TCN	7.41M (+0.01)	51.42 (+0.36)	21.92/0.8056	20.76/0.6083	23.61/0.8704
TCN-Net	0.012M	0.97	22.08/0.7895	20.99/0.6121	23.47/0.8663
Table 2: Comparison over low-light image enhancement in terms of PSNR/MS-SSIM.
Settings	MSEC	SICE
DRBN (Baseline) [15]	19.52/0.8309	17.65/0.6798
+IN	21.98/0.8463	20.15/0.6947
+Original TCN	22.37/0.8533	20.74/0.7133
+Affined TCN	22.41/0.8504	20.85/0.7192
+Skip TCN	22.48/0.8572	20.65/0.7159
SID (Baseline) [14]	19.04/0.8074	18.15/0.6540
+IN	21.36/0.8373	19.81/0.6667
+Original TCN	22.31/0.8522	20.51/0.6745
+Affined TCN	22.43/0.8542	20.68/0.6757
+Skip TCN	22.36/0.8603	20.64/0.6852
TCN-Net	22.19/0.8480	20.72/0.7024
Table 3: Comparison over exposure correction. Figure 6: Training PSNR on exposure correction.
clustered, demonstrating the normalization ability of the TCN for extracting the lightness-consistence representation of different samples. We provide more discussions in the supplementary material.
4.2 Experimental Settings
Low-light Image Enhancement. Following previous works [60, 61], we employ three widely used datasets for evaluation, including LOL dataset [7], Huawei dataset [60] and MIT-FiveK dataset [59]. We employ two different image enhancement networks, DRBN [15] and SID [14] as baselines.
Exposure Correction. Following [62], we adopt MSEC dataset [24] and SICE dataset [63] for evaluations. The above two architectures, i.e., DRBN [15] and SID [14] are regarded as baselines.
SDR2HDR Translation. Following [30], we choose the SRITM dataset [31] and HDRTV dataset [30] for evaluation. We employ the structures of NAFNet [64] with its three basic units as the baseline in the experiments.
Image Dehazing. Following [33], we employ the RESIDE dataset [65] consisting of Indoor and Outdoor parts for evaluations. We adopt the network of PFFNet [66] as the baseline for validation.
Since there exist three TCN formats in Sec. 3.3, we respectively integrate them into the baseline to conduct experiments. For comparison, we perform the experiments of baseline networks and the integration of the IN operation. Additionally, the TCN-Net in Sec. 3.3 is also performed in experiments. We train all baselines and their integrated formats following the original settings, and our TCN-Net until it converges. More implementation details are provided in the supplementary.
Settings	SRITM	HDRTV
NAFNet (Baseline) [64]	33.44/0.9537	36.49/0.9706
+IN	33.62/0.9491	36.62/0.9683
+Original TCN	33.69/0.9505	36.94/0.9712
+Affined TCN	33.65/0.9495	36.55/0.9716
+Skip TCN	33.51/0.9513	36.64/0.9720
TCN-Net	32.48/0.9439	36.78/0.9744
Settings	Indoor	Outdoor
PFFNet (Baseline) [66]	21.74/0.8452	24.47/0.9274
+IN	23.13/0.8583	25.61/0.9309
+TCN	23.57/0.8635	25.63/0.9311
+Affined TCN	23.71/0.8652	25.84/0.9312
+Skip TCN	23.21/0.8708	25.63/0.9315
TCN-Net	24.06/0.8645	23.72/0.8572
Table 4: Comparison over SDR2HDR translation.
Table 5: Comparison over image dehazing.
DRBN
DRBN+TCN
Figure 7: The visual comparison of low-light image enhancement on the MIT-FiveK dataset.
4.4 Comparison and Analysis
Quantitative Comparison. The model comparisons are conducted over different configurations, as illustrated in the implementation details. We present the quantitative results from Table 2 to Table 5, where the best and second-best results are highlighted in bold and underlined. As can be seen, almost all formats of the TCN that we incorporate have improved the performance across the datasets in all tasks, validating the effectiveness of our method. Specifically, integrating variants of TCN helps improve the training performance of baseline as shown in Fig. 6. In contrast, naively integrating the IN could not always bring performance improvement (i.e., the results of SID in Table 2). All the above results suggest the effectiveness of our proposed method without introducing any parameters. Moreover, the proposed TCN-Net achieves effective performance with efficiency. All the above evaluations prove the convenience of applying the TCN in image enhancement tasks.
Qualitative Comparison. We report the visual results of low-light image enhancement on the MIT-FiveK dataset [59] due to the limited space. As shown in Fig. 7, the integration of the TCN leads to a more visually pleasing effect with less lightness and color shift problems compared with the original baseline. We provide more visual results in the supplementary material.
4.5 Extensive Applications
The TCN can also be applied to other machine vision tasks that demonstrate its extensibility. Since TCN is proposed to extract lightness (a kind of style) invariant feature while keeping information transition-constant, we introduce another two tasks that are also related to style information, including pan-sharpening and medical segmentation. For pan-sharpening, it aims to fuse two style images, and we hope TCN can extract their invariant information with information preserving; For medical segmentation, there often exists a style domain gap between training and testing sets.
Extension on medical segmentation. We apply the TCN on the UNet [67] and AttUNet [68] in the medical segmentation task. We train the baseline and its integrated version on the heart segmentation task of Medical Segmentation Decathlon challenge dataset [69]. As shown in Table 6, our TCN improves and keeps the performance of U-Net and Att-Unet, respectively, while IN brings a significant performance drop. The results suggest the scalability of the TCN compared with the IN.
Extension on pan-sharpening. We apply the original TCN to the GPPNN [70] and PANNet [71] baselines in the pan-sharpening task, which is a common task in guided image super-resolution. We integrate it when extracting pan and multi-spectral features, and experimental results on WorldView II dataset [72, 73] in Fig. 8 suggest the effectiveness of the TCN.
Settings	Dice ↑	HD95↓
UNet(baseline)	0.9162	3.9188
UNet(+IN)	0.9171	7.7305
UNet(+TCN)	0.9204	4.0171
AttUNet(baseline)	0.9182	3.5453
AttUNet(+IN)	0.9193	6.8549
AttUNet(+TCN)	0.9180	3.6241
Table 6: Comparison over medical segmentation in terms of dice and HD95.
Figure 8: Comparison over pan-sharpening for GPPNN and PANNet.
Firstly, we validate the effectiveness of TCN in image enhancement tasks, while the investigation of applying TCN to other image restoration tasks will be explored in the future, such as the all-in-one image restoration task that meets similar challenges like image enhancement tasks, which has been pointed in some related works [74, 75]. Second, dedicated to image enhancement tasks, we mainly discuss the IN format of the TCN. However, other normalization formats can be future explored for other tasks. Moreover, the design formats of the TCN could inspire some areas that also require transition-constant, such as image fusion tasks [76]. Finally, the TCN could introduce very few computation burdens although it is free of parameters, which is negligible compared with its bring performance improvement. Note that the focus of this work is beyond introducing a plug-and-play operation to existing networks for performance gain. The introduced TCN can be a new choice of normalization and feature disentanglement, which excavate consistent representations while preserving information when developing a new model that requires this property.
In this paper, we introduce a new perspective that develops the normalization technique tailored for image enhancement approaches. We propose the TCN that transits the information constantly with the invertible constraint, meanwhile, it keeps the normalization ability for capturing lightness consistence representations. The proposed TCN is a general operation that can be integrated into existing networks without introducing parameters. Extensive experiments demonstrate the effectiveness and scalability of applying the TCN and its variants in various image enhancement tasks.
Image enhancement is an important task that improves the quality of these images, exhibiting a high value of research and application. Our method introduces a normalization operation with information transition-constant property, which shows promising results that improve the learning ability of networks for image enhancement tasks conveniently. However, there could be negative effects brought by the proposed methodology. For example, some people may prefer the image with a dim light effect, which would be eliminated by the image enhancement algorithm. In these cases, it is suggested to combine the users' preferences to achieve customized image enhancement effects.
[67] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pages 234-241. Springer, 2015.
[68] Ozan Oktay, Jo Schlemper, Loic Le Folgoc, Matthew Lee, Mattias Heinrich, Kazunari Misawa, Kensaku Mori, Steven McDonagh, Nils Y Hammerla, Bernhard Kainz, et al. Attention u-net: Learning where to look for the pancreas. arXiv preprint arXiv:1804.03999, 2018.
[69] Amber L Simpson, Michela Antonelli, Spyridon Bakas, Michel Bilello, Keyvan Farahani, Bram Van Ginneken, Annette Kopp-Schneider, Bennett A Landman, Geert Litjens, Bjoern Menze, et al. A large annotated medical image dataset for the development and evaluation of segmentation algorithms. arXiv preprint arXiv:1902.09063, 2019.
[70] Shuang Xu, Jiangshe Zhang, Zixiang Zhao, Kai Sun, Junmin Liu, and Chunxia Zhang. Deep gradient projection networks for pan-sharpening. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1366-1375, June 2021.
[71] Junfeng Yang, Xueyang Fu, Yuwen Hu, Yue Huang, Xinghao Ding, and John Paisley. Pannet: A deep network architecture for pan-sharpening. In IEEE International Conference on Computer Vision, pages 5449-5457, 2017.
[72] Man Zhou, Keyu Yan, Jie Huang, Zihe Yang, Xueyang Fu, and Feng Zhao. Mutual information-driven pan-sharpening. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1798-1808, June 2022.
[73] Man Zhou, Jie Huang, Keyu Yan, Hu Yu, Xueyang Fu, Aiping Liu, Xian Wei, and Feng Zhao. Spatial-frequency domain information integration for pan-sharpening. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XVIII, pages 274-291. Springer, 2022.
[74] Boyun Li, Xiao Liu, Peng Hu, Zhongqin Wu, Jiancheng Lv, and Xi Peng. All-in-one image restoration for unknown corruption. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17452-17462, 2022.
[75] Wenxin Wang, Boyun Li, Yuanbiao Gou, Peng Hu, and Xi Peng. Relationship quantification of image degradations. arXiv preprint arXiv:2212.04148, 2022.
[76] Jinyuan Liu, Xin Fan, Zhanbo Huang, Guanyao Wu, Risheng Liu, Wei Zhong, and Zhongxuan Luo. Target-aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5802-5811, 2022.