SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Jie Huang $^{1}$ , Man Zhou $^{1,2}$ , Jinghao Zhang $^{1}$ , Gang Yang $^{1}$ , Mingde Yao $^{1}$ , Chongyi Li $^{3}$ , Zhiwei Xiong $^{1}$ , Feng Zhao $^{1\dagger}$
<sup>1</sup>University of Science and Technology of China, China  
<sup>2</sup>Nanyang Technological University, Singapore  
<sup>3</sup>Nankai University, China
{hj0117, manman, jhaozhang, mdyao, yg1997}@mail.ustc.edu.cn, lichongyi25@gmail.com, {zwxiong, fzhao956}@ustc.edu.cn,
Normalization techniques that capture image style by statistical representation have become a popular component in deep neural networks. Although image enhancement can be considered as a form of style transformation, there has been little exploration of how normalization affects the enhancement performance. To fully leverage the potential of normalization, we present a novel Transition-Constant Normalization (TCN) for various image enhancement tasks. Specifically, it consists of two streams of normalization operations arranged under an invertible constraint, along with a feature sub-sampling operation that satisfies the normalization constraint. TCN enjoys several merits, including being parameter-free, plug-and-play, and incurring no additional computational costs. We provide various formats to utilize TCN for image enhancement, including seamless integration with enhancement networks, incorporation into encoder-decoder architectures for downsampling, and implementation of efficient architectures. Through extensive experiments on multiple image enhancement tasks, like low-light enhancement, exposure correction, SDR2HDR translation, and image dehazing, our TCN consistently demonstrates performance improvements. Besides, it showcases extensive ability in other tasks including pan-sharpening and medical segmentation. The code is available at https://github.com/huangkevinj/TCNorm.
Image enhancement is an important task in machine vision, which aims to improve the quality of low-visibility images captured under unfavorable light conditions (i.e., low light) by adjusting contrast and lightness. The last decades have witnessed quantities of approaches designed for image enhancement based on various hand-crafted priors [1, 2, 3, 4, 5]. However, the complex and variant adjustment procedures make it a challenging group of tasks. In addition to the common low-light image enhancement, efforts have also been directed toward solving image enhancement-like tasks, including exposure correction, image dehazing, and SDR2HDR translation.
Very recently, the deep-learning paradigm exhibits remarkable success in the image enhancement field than traditional methods [6, 7, 8]. Despite the progress, most of them focus on roughly constructing complex deep neural architectures and have not fully explored the intrinsic characterizes of lightness in networks. In fact, the lightness variants could bring difficulties to their learning procedures. This
Figure 1: In (a), the instance normalization (IN) captures lightness consistency representations across exposures, thus bridging their distribution gaps as shown in t-SNE. In (b), normalization techniques meet transition-inconstant problem, while proposed TCN exhibits differently with invertible ability.
motivates us to delve into the working mechanism of current neural networks that learn lightness adjustment and prescribe the right medicine customized for image enhancement.
On the other hand, normalization family such as batch and instance normalization, is specially designed for promoting the learning procedure for deep networks. It involves computing statistical representation and normalizing the corresponding distribution, which has been shown to capture image style through statistical representation [9]. Image enhancement, which aims to restore lightness-corrupted images to their normal versions, can be viewed as a style transformation inherently linked to normalization techniques (see Fig. 1 (a)). However, existing methods have rarely explored the potential of the normalization technique. Inspired by the inborn connection, we thus focus on developing the normalization technique tailored for image enhancement.
In this work, we propose a novel operation called transition-constant normalization (TCN) for image enhancement tasks. The TCN operation aims to normalize partial representations to ensure consistent learning while preserving constant information for image reconstruction. As illustrated in Fig. 1 (b), we construct the TCN within an invertible format to enable seamless information transmission to subsequent layers for reconstructing enhanced results. The TCN is designed with two key rules (see Fig. 2): (1) We organize the operations in the normalization layer into two streams, following the principle of invertible information transmission, thereby maintaining constant information transition. (2) We incorporate a subsampling operation that divides the features into two streams with consistent statistical properties. One stream provides the statistics for normalizing the other stream, satisfying the normalization requirement. Notably, the TCN requires no parameters, making it a convenient and orthogonal addition to existing enhancement architectures for improving their performance.
To facilitate its application, we present multiple usage formats for the TCN in image enhancement: (1) Integration into existing enhancement networks, allowing for seamless incorporation and performance improvement; (2) Plug-in capability in encoder-decoder architectures for downsampling and decomposition of information; (3) Construction of a lightweight architecture based on TCN, striking a balance between performance and computational cost. Through extensive experiments across various image enhancement tasks, we consistently observe performance gains by integrating our TCN.
The contributions of this work are summarized as follows: 1) We present a novel perspective on image enhancement using a dedicated normalization technique. This technique enhances the learning of lightness adjustments by modeling consistent normalized features, while ensuring their complementarity for reconstructing the results. 2) We construct the Transition-constant Normalization (TCN) by organizing normalization operations to satisfy the invertible mechanism, ensuring constant feature normalization and information transition. 3) Our proposed TCN is compatible with existing enhancement architectures, allowing for convenient integration and performance improvement. Furthermore, we can derive multiple implementation formats for TCN and explore its applicability in various tasks, highlighting its potential for wide-ranging applications.
Image enhancement tasks. Image enhancement tasks aim to improve the quality of low-visibility images by adjusting lightness and contrast components (e.g., illumination, color, and dynamic range).
Recent years have witnessed rapid development in the related areas [10, 11, 12, 13]. For low-light image enhancement, algorithms are designed to enhance the visibility of images captured under low-light conditions [14, 15, 16, 17, 18, 19, 7, 20, 21, 22, 23]. In the exposure correction task, methods are focused on correcting both underexposure and overexposure to normal exposure [15, 24, 25, 26, 27]. For SDR2HDR translation, this task aims to design methods to convert images from a low-dynamic range to a high-dynamic range [28, 29, 30, 31, 32]. While in image dehazing, this task requires methods to enhance the contrast and recover the color shift problems [33, 34, 35, 36, 37]. To this end, image enhancement tasks cover variant scenes and remain challenges to be solved.
Normalization techniques. Normalization techniques have been studied for a long time [38, 39, 40, 41]. Batch Normalization (BN) [38] normalizes the features along the batch dimension that stabilizes the optimization procedure. Instance Normalization (IN) [9] focuses on normalizing the instance-level statistics of features, which has been widely employed in style transfer tasks [42, 43]. Some other variants of normalization, including Layer Normalization (LN) [39], Group Normalization (GN) [44], and Position Normalization (PN) [45] have been proposed for facilitating the application of networks.
In this section, we first briefly revisit the normalization techniques and then detail the design and mechanism of the proposed TCN. Finally, we present the variants of the TCN as implementation.
Given a batch of features  $x \in \mathbb{R}^{\mathrm{N} \times \mathrm{C} \times \mathrm{H} \times \mathrm{W}}$ , where N, C, H and W represent batch size, channel numbers, the spatial height and width, respectively. Let  $x_{ncij}$  and  $\hat{x}_{ncij}$  denote a pixel before and after normalization, where n ∈ [1, N], c ∈ [1, C], i ∈ [1, H], i ∈ [1, W]. Without taking considering into affine parameters, we can express normalization operation as:
$$
\hat {x} _ {n c i j} = \operatorname {N o r m} \left(x _ {n c i j}\right) = \frac {x _ {n c i j} - \mu_ {k}}{\sqrt {\sigma_ {k} ^ {2} + \epsilon}}, \tag {1}
$$
where  $\mu_{k}$  and  $\sigma_{k}$  denote the feature mean and standard deviation,  $\epsilon$  is a small constant to preserve numerical stability.  $k\in \{\mathrm{IN},\mathrm{BN},\mathrm{LN},\mathrm{GN}\}$  is to distinguish different normalization formats. Within the above normalization family, the calculation of  $\mu_{k}$  and  $\sigma_{k}$  is different and are expressed as:
$$
\mu_ {k} = \frac {1}{| I _ {k} |} \sum_ {n, c, i, j \in I _ {k}} x _ {n c i j}, \quad \sigma_ {k} = \sqrt {\frac {1}{| I _ {k} |} \sum_ {n , c , i , j \in I _ {k}} (x _ {n c i j} - \mu_ {k}) ^ {2}}, \tag {2}
$$
1) IN:  $I_{k}: I_{IN} = \{(i,j) | i \in [1, \mathrm{H}], j \in [1, \mathrm{W}] \}$ ;
2)  $\mathbf{BN}\colon I_k:I_{BN} = \{(n,i,j)|n\in [1,\mathrm{N}],\mathrm{i}\in [1,\mathrm{H}],\mathrm{j}\in [1,\mathrm{W}]\} ;$
3)  $\mathbf{LN}\colon I_k:I_{LN} = \{(c,i,j)|c\in [1,\mathrm{C}],\mathrm{i}\in [1,\mathrm{H}],\mathrm{j}\in [1,\mathrm{W}]\}$ ;
4)  $\mathbf{GN}\colon I_k:I_{GN} = \{(c,i,j)|c\in [\mathrm{g},\mathrm{g} + \mathrm{C} / \mathrm{G}],\mathrm{i}\in [1,\mathrm{H}],\mathrm{j}\in [1,\mathrm{W}]\mid \mathrm{g}\in [1,\mathrm{G}]\} .$
where  $I_{k}$  is a set of pixels,  $|I_{k}|$  denotes the number of pixels and  $G$  is the group division.
Within deep neural networks, Eq. 1 is often affined with scaling and shifting parameters  $\alpha$  and  $\beta$ :
$$
\hat {x} _ {n c i j} = \alpha \cdot \operatorname {N o r m} \left(x _ {n c i j}\right) + \beta = \alpha \frac {x _ {n c i j} - \mu_ {k}}{\sqrt {\sigma_ {k} ^ {2} + \epsilon}} + \beta . \tag {3}
$$
It is well-known that lightness can be considered as a kind of style and Instance normalization can facilitate consistent style information [9] and image enhancement network optimization due to bridging the gap of different lightness representations.
Verifying normalization effect for lightness. Given an image  $x_{a}$  and its lightness-adjusted version  $x_{b}$ , the relationship between  $x_{a}$  and  $x_{b}$  can be expressed in correction procedure [46, 47, 48] as:
$$
x _ {b} = \Lambda x _ {a} ^ {\gamma}, \tag {4}
$$
where  $\Lambda$  is a linear transformation and  $\gamma$  is for global non-linear adaption and is close to 1 when content is not severely changed. Therefore, the p-norm distance between their normalized versions is:
$$
\left\| \mathrm {f} \left(x _ {a}\right) - \mathrm {f} \left(x _ {b}\right) \right\| _ {p} = \left| \frac {x _ {a} - \mu_ {a}}{\sigma_ {a}} - \frac {\Lambda x _ {a} ^ {\gamma} - \mu_ {b}}{\sigma_ {b}} \right\| _ {p} \approx \left| \frac {x _ {a} - \mu_ {a}}{\sigma_ {a}} - \frac {\Lambda x _ {a} - \Lambda \mu_ {a}}{\Lambda \sigma_ {a}} \right\| _ {p} \ll \| x _ {a} - \Lambda x _ {a} ^ {\gamma} \| _ {p}. \tag {5}
$$
Therefore, normalization reduces the distance of different lightness, which is also validated in Fig. 1(a). However, normalizing the statistics itself often blocks the information flow, which hinders the network from reconstructing the final results [49] and we describe it as follows.
Transition-inconstant of Normalization. Referring to  $\mathrm{Norm}(\cdot)$  in the Eq. 1 as  $f$ , its Jacobian matrix is expressed as:
$$
J _ {f} (x) = \left[ \begin{array}{c c c} \frac {\partial \hat {x} _ {0}}{x _ {0}} & \dots & \frac {\partial \hat {x} _ {0}}{x _ {0}} \\ & \ddots & \vdots \\ \frac {\partial \hat {x} _ {0}}{x _ {0}} & & \frac {\partial \hat {x} _ {0}}{x _ {0}} \end{array} \right] _ {m \times m} = \left[ \begin{array}{c c c} \frac {1}{\sigma_ {k}} & \dots & \frac {1}{\sigma_ {k}} \\ & \ddots & \vdots \\ \frac {1}{\sigma_ {k}} & & \frac {1}{\sigma_ {k}} \end{array} \right] _ {m \times m} \tag {6}
$$
Therefore, the calculation of the above Jacobian matrix is  $\det(J_f(x)) = 0$ , denoting the normalization operation is not invertible and resulting in transition-inconstant. Meanwhile, in practice, previous works have demonstrated that IN would lead to severe information loss [42] and huge representation ability changes [50, 51], while LN and BN can keep almost all of the original information representation ability. However, IN is more suitable than BN and LN for image enhancement tasks due to its strong capability of capturing and affecting style information, which is crucial for image enhancement. To this end, the main goal of this paper is to introduce a new mechanism that enables the IN can keep the information representation ability for image enhancement.
3.2 Transition-constant Normalization (TCN)
Based on the above analysis, we aim to refresh the normalization technique to enable it to transmit information constantly while normalizing the features. To this end, we introduce the TCN as shown in Fig. 2, which is free of parameters and is convenient to implement. Since IN can normalize different lightness effectively and thus is useful for image enhancement, we design the TCN based on the IN as its default implementation format in this paper.
Operation description. We construct the TCN by applying the normalization operations with a two streams flow design with subsampled features, where one stream provides the statistical information for normalizing another stream. Specifically, the feature  $F \in \mathbb{R}^{\mathrm{B} \times \mathrm{C} \times \mathrm{H} \times \mathrm{W}}$  is firstly subsampled to  $F_{s} \in \mathbb{R}^{\mathrm{B} \times 4\mathrm{C} \times \frac{\mathrm{H}}{2} \times \frac{\mathrm{W}}{2}}$  according to the unshuffle operation [52] as shown in Fig. 2 (a), which is:
$$
F _ {s} ^ {a b} = F [ :, :, a ::, b :: ], \quad a, b \in \{0, 1 \}, \tag {7}
$$
where  $a$  and  $b$  denote the subsampling index. We divide  $F_{s}$  into two features  $F_{1}$  and  $F_{2}$  with two groups according to the sampling index  $(i,j)$  as:
$$
F _ {1} = \operatorname {C o n c a t} \left(F _ {s} ^ {0 1}, F _ {s} ^ {1 0}\right), F _ {2} = \operatorname {C o n c a t} \left(F _ {s} ^ {0 0}, F _ {s} ^ {1 1}\right) \tag {8}
$$
where  $\operatorname{Concat}(\cdot, \cdot)$  denotes the concatenate operation along the channel dimension.
Then, we calculate the mean  $\mu_{2}$  and standard deviation  $\sigma_{2}$  of one stream feature  $F_{2}$  in IN format, which are derived by setting  $I_{k}$  in Eq. 2 as  $I_{IN}$ :
$$
\mu_ {2} = \frac {1}{\mathrm {H W}} \sum_ {i \in [ 1, \mathrm {H} ], \mathrm {j} \in [ 1, \mathrm {W} ]} F _ {i j}, \quad \sigma_ {2} = \sqrt {\frac {1}{\mathrm {H W}} \sum_ {i \in [ 1 , \mathrm {H} ] , \mathrm {j} \in [ 1 , \mathrm {W} ]} (F _ {i j} - \mu_ {2}) ^ {2}}. \tag {9}
$$
These statistics are utilized to normalize the feature  $F_{1}$  as the output in this stream:
$$
\hat {F} _ {1} = \frac {F _ {1} - \mu_ {2}}{\sqrt {\sigma_ {2} ^ {2} + \epsilon}}. \tag {10}
$$
Next, we subtract the feature  $F_{2}$  and  $\hat{F}_{1}$  and obtain the output of another stream:
$$
\hat {F} _ {2} = F _ {2} - \hat {F} _ {1} = F _ {2} - \frac {F _ {1} - \mu_ {2}}{\sqrt {\sigma_ {2} ^ {2} + \epsilon}}. \tag {11}
$$
Finally, the two stream features are sampled with pixel shuffle operation to the original resolution with the shape of  $[\mathrm{N},\frac{\mathrm{C}}{2},\mathrm{H},\mathrm{W}]$ . They are further concatenated in the channel dimension as  $\hat{F}\in \mathbb{R}^{\mathrm{B}\times \mathrm{C}\times \mathrm{H}\times \mathrm{W}}$ , which is the output of the TCN. This procedure is expressed as:
$$
\hat {F} = \operatorname {P i x s h u f f l e} \left(\operatorname {C o n c a t} \left(\hat {F} _ {1}, \hat {F} _ {2}\right)\right), \tag {12}
$$
where  $\mathrm{Pixshuffle}(\cdot)$  denotes the pixel shuffle operation [53]. We verify the above procedures satisfy the transition-constant and normalization ability as below, respectively.
1) Verify the transition-constant ability. We validate the above two-stream design satisfies the invertible procedure and thus is transition-constant. To this end, Eq. 10 and Eq. 11 are re-written as:
$$
\hat {F} _ {1} = \left(F _ {1} - M \left(F _ {2}\right)\right) \oslash S \left(F _ {2}\right), \tag {13}
$$
$$
\hat {F} _ {2} = F _ {2} - \hat {F} _ {1},
$$
where  $S(\cdot)$  and  $M(\cdot)$  denote standard deviation and mean functions,  $\varnothing$  denotes element division.
Inspired by the proof in RealNVP's [54] transformation, we need to calculate the Jacobian matrix of Eq. 13 (denote it as  $g$ ), which is more intuitively written as:
$$
\hat {F} _ {1} = \left(F _ {1} - M \left(F _ {2}\right)\right) \oslash S \left(F _ {2}\right), \tag {14}
$$
$$
\hat {F} _ {2} = F _ {2} - F _ {1} \oslash S (F _ {2}) + M (F _ {2}) \oslash S (F _ {2}),
$$
We derive its Jacobian matrix (detailed in the supplementary) as:
$$
J _ {g} = \left[ \begin{array}{l l} \frac {\partial \hat {F} _ {1}}{\partial F _ {1}} & \frac {\partial \hat {F} _ {1}}{\partial F _ {2}} \\ \frac {\partial \hat {F} _ {2}}{\partial F _ {1}} & \frac {\partial \hat {F} _ {2}}{\partial F _ {2}} \end{array} \right] = \left[ \begin{array}{c c} \frac {1}{\partial S (F _ {2})} & 0 \\ - 1 & 1 \end{array} \right] \tag {15}
$$
Here, the above Jacobian matrix is further calculated as:
$$
\operatorname {d e t} \left(J _ {g}\right) = \frac {1}{\partial S \left(F _ {2}\right)} \neq 0 \tag {16}
$$
Upon the  $\operatorname{det}(J_g) \neq 0$ , it indicates that  $J_{g}$  is full rank, verifying the invertible property of TCN and further the transition-constant ability. To highlight, the TCN is an invertible function and would not block information flow, leading to the information transition constant for image reconstruction. We further present the relation of the TCN and the invertible operation more directly in the supplementary.
Formats	Redefine Ik to Eq. 9 as
TCN (IN)	IIN(default Ik of Eq. 9)
TCN (BN)	IBN={(n,i,j)|n∈[1,N], i∈[1,H],j∈[1,W]}
TCN (LN)	ILN={(c,i,j)|c∈[1,C], i∈[1,H],j∈[1,W]}
TCN (GN)	IGN=(c,i,j)c∈[g,g+C/G],i∈[1,H], j∈[1,W]|g∈[1,G]}
Table 1: The TCN family with different  $\mu_2,\sigma_2$  calculation in Eq. 9.
2) Verify the normalization ability. The normalization ability of the TCN is guaranteed by the pixel unshuffle operation in Eq. 7, leading to the same statistics of  $F_{s}^{ab}$  [55, 56]. Therefore, we have  $\mu_{2} \approx \mu_{1}, \sigma_{2} \approx \sigma_{1}$ , and the Eq. 10 is thus converted to:
$$
\hat {F} _ {1} = \frac {F _ {1} - \mu_ {2}}{\sqrt {\sigma_ {2} ^ {2} + \epsilon}} \approx \frac {F _ {1} - \mu_ {1}}{\sqrt {\sigma_ {1} ^ {2} + \epsilon}}. \tag {17}
$$
Therefore, it has the same format as Eq. 1, demonstrating that the operation in Eq. 10 has the normalization ability as the IN. Further, we verify the above rules by the toy experiment (see Sec. 4.1) in Fig. 4 and Fig. 5.
Discussion. The core of the TCN is the statistic calculation manner of  $\mu_{2}$  and  $\sigma_{2}$  in Eq. 9 which can be generalized in a unified calculation manner in Eq. 2 and derive from other normalization formats of TCN, shown in Table 1. Note that, although GN and LN would less affect information representation ability [50], we experimentally find introducing the transition-constant design would improve their performance in the supplementary. We provide more discussions in the supplementary.
3.3 Variants of TCN for Image Enhancement
Upon the above principles of TCN, we provide the following implementation variants within image enhancement task.
The original TCN. We construct the original TCN (see Fig. 2 (a)) for image enhancement based on calculating statistics in Eq. 9 of IN format, which is plug-and-play for networks.
The affine TCN. We extend the original TCN by introducing affine parameters  $\alpha$  and  $\beta$  to the normalization procedure, resulting in the affine TCN (Fig. 2 (b)). We incorporate learnable shifting
Transition-constant Normalization (TCN)
Transformation
Concatenate & Pixel Shuffle
Un-shuffle & Split
Learnable Affine Parameters
Skip TCN
Figure 2: The illustration of the TCN operation and other TCN variants for image enhancement.
parameter  $\beta$  and scaling parameter  $\gamma$  into  $\mu_{2}$  and  $\sigma_{2}$  in Eq. 9:
$$
\mu_ {2} ^ {\prime} = \mu_ {2} + \beta , \quad \sigma_ {2} ^ {\prime} = \frac {\sigma_ {2}}{\gamma} \tag {18}
$$
where  $\mu_2^{\prime}$  and  $\sigma_2^\prime$  represent the affine statistics. Then, we substitute Eq. 18 to Eq. 1:
$$
\hat {F} _ {1} ^ {\prime} = \frac {F _ {1} - \mu_ {2} ^ {\prime}}{\sqrt {\sigma_ {2} ^ {\prime 2} + \epsilon}} = \frac {F _ {1} - \mu_ {2} - \beta}{\sqrt {\frac {\sigma_ {2} ^ {2}}{\gamma^ {2}} + \epsilon}} \tag {19}
$$
Since  $\epsilon$  is a small constant near to 0, the Eq. 19 can be approximated as:
$$
\hat {F} _ {1} ^ {\prime} \approx \gamma \frac {F _ {1} - \mu_ {2} ^ {\prime} - \beta}{\sqrt {\sigma_ {2} ^ {\prime 2} + \epsilon}} = \gamma \frac {F _ {1} - \mu_ {2} ^ {\prime}}{\sqrt {\sigma_ {2} ^ {\prime 2} + \epsilon}} + \beta^ {\prime}, \quad \beta^ {\prime} = \frac {- \gamma \beta}{\sqrt {\sigma_ {2} ^ {\prime 2} + \epsilon}}. \tag {20}
$$
Eq.20 shares a format similar to the affine normalization, with  $\gamma$  and  $\beta^{\prime}$  as the learnable scaling and shifting parameters in Eq. 3. The affine TCN seamlessly integrates into image enhancement networks, serving as a plug-and-play solution. Notably, it maintains the information transition-constant property, as discussed in detail in the supplementary material.
The skip TCN. From Eq.10 and 11, the TCN generates two types of features: a domain-invariant lightness consistent feature  $\hat{F}_1$  and a domain-variant lightness inconsistent feature  $\hat{F}_2$ . Previous studies [57, 58] have demonstrated the effectiveness of incorporating the domain-invariant component into deep encoder-decoder networks while skipping the domain-invariant component to the decoder layer. In this work, we propose the skip TCN architecture, illustrated in Fig. 2 (c).
Given a feature  $F$  in an encoder layer, we convey its lightness inconsistent feature  $\hat{F}_2$ , obtained from Eq. 10, to the downsampled deeper layer that derives  $F_d$  using the following expression:
$$
F _ {d} = \operatorname {D o w n} _ {2} (\mathrm {F}) + \hat {\mathrm {F}} _ {2}, \tag {21}
$$
where  $\mathrm{Down}_2$  means downsampling with a factor of 2. While for the lightness consistent feature  $\hat{F}_1$  derived in Eq. 11, we skip it to the corresponding decoder layer feature  $F_{u}$  with the statistic  $\mu_2$  and  $\sigma_2$  derived in Eq. 9. We integrate them by inverting the operation of Eq. 10 and Eq. 11:
$$
F _ {u 1} = \hat {F} _ {1} \cdot \sigma_ {2} + \mu_ {2}, \quad F _ {u 2} = \hat {F} _ {1} + F _ {u}, \tag {22}
$$
$$
F _ {u o} = \operatorname {U p} _ {2} \left(\mathrm {F} _ {\mathrm {u}}\right) + \text {P i x s h u f f} \left(\operatorname {C o n c a t} \left(\mathrm {F} _ {\mathrm {u} 1}, \mathrm {F} _ {\mathrm {u} 2}\right)\right),
$$
Figure 3: The overview of the TCN-Net.
Here, the upsampling operation with a factor of 2 is denoted as  $\mathrm{Up}_2$ , and  $F_{uo}$  represents the skip TCN result. The skip TCN prioritizes the processing of the lightness component while preserving lightness invariant features, mitigating learning difficulties. Further discussion is provided in the supplementary material.
Construct a very efficient TCN-based Network. We introduce TCN-Net, an efficient network architecture depicted in Fig. 3, which combines affine TCN and skip TCN. This framework adopts an encoder-decoder-based architecture with vanilla convolution blocks to depict the