Generative Status Estimation and Information Decoupling for Image Rain Removal
Abstract
Image rain removal requires the accurate separation between the pixels of the rain streaks and object textures.
But the confusing appearances of rains and objects lead to the misunderstanding of pixels, thus remaining the rain streaks or missing the object details in the result.
In this paper, we propose SEIDNet equipped with the generative Status Estimation and Information Decoupling for rain removal.
In the status estimation, we embed the pixel-wise statuses into the status space, where each status indicates a pixel of the rain or object.
The status space allows sampling multiple statuses for a pixel, thus capturing the confusing rain or object.
In the information decoupling, we respect the pixel-wise statuses, decoupling the appearance information of rain and object from the pixel.
Based on the decoupled information, we construct the kernel space, where multiple kernels are sampled for the pixel to remove the rain and recover the object appearance.
We evaluate
SEIDNet on the public datasets, achieving state-of-the-art performances of image rain removal.
The experimental results also demonstrate the generalization of
SEIDNet, which can be easily extended to achieve state-of-the-art performances on other image restoration tasks (e.g., snow, haze, and shadow removal).
We release the implementation of SEIDNet via https://github.com/wxxx1025/SEIDNet.
Introduction
Image rain removal relies on the understanding of the appearances of the rains and the objects.
Most of the current methods employ the discriminative network to learn the visual features of the pixels, for representing the visual appearances of the rain streaks and object textures.
Based on the visual features, the network learns the shared [1, 2, 3, 4, 5, 6, 7] or dynamic convolutional kernels [8, 9] for rain removal on the pixels.
Intuitively, the kernels for rain removal respect the pixel-wise statuses (i.e., rain or object), reducing the rainy intensities and recovering the object details of the pixels.
The challenge of rain removal mainly stems from the fact that some of the rains are similar to the objects (see the input regions in Figure 1(a)).
The confusion between the rains and the objects is inevitably encoded into the visual features.
It yields the inappropriate kernels for the pixels, contributing to the erroneous results like leaving the rain streaks and removing the object textures in the image (see the predictions in the pink rectangles of Figure 1(b)).
The discriminative network provides a deterministic kernel for a pixel.
But it loses the chance of computing more appropriate
[Figure]
kernels for a misunderstood pixel.
The methods [2, 3, 4, 10, 11, 12, 13, 14, 15, 16, 17, 18] use different kernels to process the rains at several stages.
Yet, the confusing information at the early stages still misleads the kernel computation at the later stages.
In this paper, we present a novel approach for capturing the confusing information of the pixels.
The key idea is to use the generative network to learn the probability distribution of the pixel-wise status and the dynamic convolutional kernel.
The distribution captures the correlation between status and kernel.
Given the distribution, we regard the feature of the pixel as the condition, generating multiple statuses and kernels.
The statuses facilitate more focused learning of the dynamic kernels, which capture the rain and object characteristics.
Compared to single status, multiple statuses can be used for generating more kernels, which provide more opportunities for reﬁning the result of rain removal.
We construct SEIDNet , a generative network equipped with the pixel-wise Status Estimation and
Information Decoupling for rain removal.
We illustrate the construction of SEIDNet in Figure 2.
We use the rainy image and the object layer 1 as the training sample.
In the status estimation (see
Figure 2(a)), we subtract the object layer from the rainy image, achieving the statuses of all pixels in the rainy image.
We construct a conditional variational auto-encoder (CVAE) to embed the pixel-wise statuses of different pixels into the status space.
With the status space, we use the feature of the pixel as the condition to generate multiple statuses that capture the confusing appearance of the pixel.
In the information decoupling (see Figure 2(b)), we use the statuses to yield the feature maps that represent the decoupled information of rain and object.
We exploit these feature maps to learn the kernels.
We employ another CVAE to embed the learned kernels into the kernel space, where we use the feature and the status of the pixel as the condition to generate the kernel.
The status estimation can generate multiple statuses, helping to generate multiple kernels.
These kernels effectively reduce the confusing information of the pixel, ﬁnally producing a better result of rain removal.
We evaluate SEIDNet on the public datasets for rain removal (i.e., Rain100H [16], Rain100L [16],
Rain1400 [19], Rain13K [10] and SPA [3]), achieving state-of-the-art performances.
Furthermore, we extend SEIDNet to various image restoration tasks (e.g., snow, haze, and shadow removal), where
SEIDNet also surpasses the recent methods on the public datasets [20, 21, 22].
Related work 2.1 Discriminative Networks
The recent methods use the deep discriminative network to learn the visual features of rain and object.
Zhang et al.
[4] propose a residual-aware classiﬁer to recognize the rain density.
Yang et al.
[16] use large convolutional kernels to capture rich visual information for recovering the details of the dense rain streaks and object textures.
Zhang et al.
[23], Wang et al.
[3], and Li et al.
[13] use the
[Figure]
multi-scale information to capture the appearances of rains and objects in local and global ranges.
Li et al.
[12] use the dilated convolution to capture the multi-scale image information for predicting the diverse shapes of the rain streaks.
Wang et al.
[2] propose the convolutional dictionary model, where a set of rain kernels are used for capturing the appearances of the rain streaks and object textures.
The discriminative networks are also used for learning the correlation between rain and object in terms of their appearances.
Luo et al [24] combine the sparse coding and the greedy pursuit algorithms to separate the rain and object layers.
Li et al.
[25] utilize Gaussian Mixture Models (GMMs) to involve the patch-based priors of rain and object.
Deng et al.
[26] use a uniﬁed network for rain removal and repair of object textures.
Fu et al.
[19] propose the detail layer of the rain streaks and object contours, for reducing the impact of low-frequency object textures on the rain removal.
Based on the pixel-wise features, the discriminative networks compute the shared [1, 2, 3, 4, 5, 6, 7] or the dynamic convolutional kernels [8, 9] for rain removal on the pixels.
Yet, the similar appearances of the rains and the objects let the discriminative network predict the problematic kernels.
In contrast, we use the generative network that learns the probability distribution of the pixel-wise status.
With the distribution, we sample multiple statuses for capturing the confusing information of the pixel.
It assists the computation of kernels for processing the pixels of the rain streaks and object textures.
2.2 Generative Networks
The generative networks, such as the variational auto-encoders (VAEs) [27, 28, 29, 30] and the generative adversarial networks (GANs) [31, 32, 33, 34, 35], have been widely used for image generation.
Some of the methods employ the generative networks to recover the object layer of the rainy image.
Li et al.
[14] input the rainy image to the conditional GAN that preserves the object textures.
Rui et al.
[17] resort to the visual attention mechanism, attending to the important image regions with rain streaks.
Zhang et al.
[23] propose the delicate loss function to alleviate the artifacts generated by GAN in the object layer.
Du et al.
[36] equip the conditional VAE [37] to the spatial density estimation of the rain streaks for more accurate rain removal.
The existing methods use a stand-alone generative network for embedding the features of the rains and objects into the latent space.
But the confusing rains and objects may mislead the construction of the latent space.
These methods may achieve the latent vector of the rain (or object) from the latent space, where the vector mistakenly represents the object (or rain).
Instead, we utilize a couple of
CVAEs, for learning the probability distribution of the pixel-wise status and dynamic kernel.
We use the ﬁrst CVAE to sample multiple statuses from the latent space, capturing the confusing information of the pixel.
With the second CVAE, we use multiple statuses to sample the kernels.
These kernels are learned from the decoupled information of the rains and objects.
They better reduce the confusing information and reﬁne the pixel intensities in the result.
Method Overview
We introduce the probability distribution of the pixel-wise status and kernel.
The distribution is learned by CVAEs in the status estimation and information decoupling of SEIDNet.
Probability Distribution For the rainy image I ∈
R H
× W × 3 , we compute the visual feature map
F ∈ R H
× W × C and the status map R ∈ R H × W .
F ( x, y ) ∈ R C represents the feature vector of the pixel located at ( x, y ) in the rainy image.
R ( x, y ) ∈ R is the score for the pixel.
A higher (or lower) score indicates the status of the rain (or object).
We compute the kernel map K ∈ R
H
× W × ( S × S × C ) , where K ( x, y ) ∈ R S
× S × C is the S × S kernel with C channels for the pixel at ( x, y ) .
We construct the generative network to learn the probability distribution P ( R , K | F ) , which takes condition as the feature map F for computing the status map R and the kernel map K .
We formulate P ( R , K | F ) as:
[Equation]
Z ∈ R H
× W × C is a set of latent variables generated by the normal distribution P ( Z ) .
We factorize the distribution P ( R , K | F , Z ) into two distributions, P ( K | R , F , Z ) and P ( R | F , Z ) , as:
[Equation]
The factorization of probability distribution enables a more focused learning of the dynamic kernel based on the pixel-wise status.
We use two CVAEs in the status estimation and information decoupling to learn the conditional distributions P ( R | F , Z ) and P ( K | R , F , Z ) , respectively.
Status Estimation As illustrated in Figure 2(a), we use the rainy image I and the object layer
O ∈ R
H
× W × 3 to train the CVAE V se .
V se learns the distribution P ( R | F , Z ) , for modeling the status space.
We subtract O from I , to achieve the status map R .
V se embeds the statuses in R into the status space.
As illustrate in Figure 2(c), V se regards the feature map F as the condition, for sampling the status map R (cid:48) from the status space.
We regard R (cid:48) as the estimation of R .
Information Decoupling We use the status map R and the feature map F to train the CVAE V i
d in the information decoupling (see Figure 2(b)).
V id learns the distribution P ( K | R , F , Z ) that models the kernel space.
First, we feed R and F into the feature masking (see Figure 2(d)).
We decouple
F into the rain feature map F r ∈ R
H
× W × C and the object feature map F o ∈ R H × W × C .
We use
F r and F o to compute the kernel map K that removes the rains and recovers the objects.
Next, V id embeds K into the kernel space.
As illustrated in Figure 2(e), V id regards F and R as the condition, for sampling the kernel map K (cid:48) from the kernel space.
K (cid:48) estimates K for rain removal.
Architecture of SEIDNet
Below, we introduce the training and testing architectures of SEIDNet for rain removal.
4.1 Training Architecture
Status Estimation We illustrate the training architecture of SEIDNet in Figure 2.
In the status estimation (see Figure 2(a)), we achieve the status map R ∈ R H × W as:
[Equation]
where R ( x, y ) ∈ [0 , 1] .
σ and conv are the sigmoid function and a set of convolutional layers.
For the rainy image I ∈
R H
× W × 3 , we use the convolution to learn the feature map F ∈ R H
× W × C .
We
FeatureMap F RainyImage
I
Condition
Decoder StatusMap R
Condition
Decoder KernelMap K
StatusMap R
Condition
KernelMap K
Decoder
Decoder ... µ f σ f
KernelMap K C
C
A
C C C
C
ObjectLayer O ’ (a) Generation of Status Maps (b) Generation of Kernel Maps ...
⊙ + ⊙ + µ f ⊙ + ⊙ + ⊙
Concatenate C A Element-wise multiplication
Average
Convolution σ f σ f µ f µ cm σ cm µ cn σ cn Z n
Z m
Z m
Z n m n u m n input R and F to the CVAE V se (see Figure 2(c)), whose encoder, condition, and decoder branches are formulated as: [ µ r , σ r ] = encoder ([ R , F ]) , [ µ f , σ f ] = condition ( F ) , R (cid:31) =
decoder ([ F , µ
[Equation]
+
[Equation]
r (cid:30) Z ]) .
(4) [ · ] is the feature concatenation along the channel dimension.
(cid:30) is the element-wise multiplication.
The encoder and condition branches take input as [ R , F ] and F , yielding the mean value maps µ r , µ f ∈ R H × W × C and the standard deviation maps σ r , σ f ∈ R
H
× W × C .
We use the normal distribution N (0 , 1) to generate the latent variable map Z ∈ R H × W × C .
We input F , Z and ( µ r , σ r ) to the decoder, yielding the status map R (cid:31)
∈
R H × W .
We use the sigmoid function to normalize R (cid:31) .
During the network training, we use L 2 -norm (denoted as L 2 ) to penalize the difference between the status map R and the estimated counterpart R (cid:31) .
Besides, we compute Kullback-Leibler Divergence (denoted as KL ) between the Gaussian distributions G ( µ r , σ r ) and G ( µ f , σ f ) .
It allows to use ( µ f , σ f ) in place of ( µ r , σ r ) that are unavailable during testing.
We deﬁne α = 4 as the weight of
KL divergence.
L 2 -norm and KL divergence compose the status estimation loss L se as:
[Equation]
This loss guides V se to learn the distribution P ( R | F , Z ) in Eq.
(2), which models the status space.
Information Decoupling In the information decoupling (see Figure 2(b)), we pass the status map
R and the feature map F into the feature masking (see Figure 2(d)), achieving the feature maps
F r , F o ∈ R H × W × C in Eq.
Based on F r , F o that respectively represent the appearances of rains and objects in the image I , we compute the kernel maps K r ,
K o ∈ R H
× W × ( S × S × C ) .
[Equation]
[Equation]
We use the status map R to weight K r and K o , yielding the kernel map
K ∈ R H
× W × ( S × S × C ) as:
[Equation]
As illustrated in Figure 2(e), the CVAE V id in the information decoupling also has the encoder, condition and decoder branches, which are formulated as:
[Equation]
([ K , F , R ]) , [ µ c , σ c ]= condition ([ F , R ]
[Equation]
decoder (
[Equation] [Equation]
[Equation]
We input [ K , F , R ] and [ F , R ] into the encoder and condition branches, respectively, producing the mean value maps µ k , µ c ∈ R H × W × C and the standard deviation maps σ k , σ c ∈ R H × W × C .
We feed [ F , R , µ k + σ k (cid:30) Z ] into the decoder, yielding the kernel map K (cid:31)
∈
R H
× W × ( S × S × C ) .
We resort to the L 2 -norm and KL divergence to construct the information decoupling loss L id as:
[Equation]
V id learns the conditional distribution P ( K | R , F , Z ) in Eq.
(2), which models the kernel space.
Deraining and Overall Losses Given the kernel map K (cid:48) , we perform the convolution on the rainy image I and estimate the object layer O (cid:48) .
As formulated in the left of Eq.
(10), the convolution (denoted as (cid:126) ) is pixel-wise.
where the pixel I ( x, y ) is processed by the kernel K (cid:48)
( x, y ) .
We use the structural similarity (denoted as SSIM ) loss and L 2 -norm to penalize the difference between the object layer O and the estimated layer O (cid:48) , yielding the deraining loss L de in the right of Eq.
[Equation]
where β
= 0 .
With the losses L se , L id and L de , we form the overall loss L as:
[Equation]
4.2 Testing Architecture
Kernel Generation
During the network testing, we follow the convention to remove all encoder branches of the CVAEs V se and V id .
It results in the testing architecture (see Figure 3).
We formulate the process of using V se and V id to generate the status map R m and the kernel map K m as:
[Equation]
As illustrated in Figure 3(a), we use the rainy image I to compute the the visual feature map F .
We pass F into the condition branch of V se , achieving the mean value map µ f and the standard deviation map σ f .
We use the normal distribution to generate the latent variable map Z m .
As formulated in
Eq.
(12), the decoder of V se uses F , Z m and ( µ f , σ f ) to generate the status map R m .
As illustrated in Figure 3(b), we use the condition branch of V id , which takes input as F and R m , to compute the mean value map µ mc and the standard deviation map σ mc .
As formulated in Eq.
(12), the decoder of V se uses F , R m , Z and ( µ mc , σ mc ) to generate the kernel map K m .
Kernel Aggregation We use the normal distribution to generate an array of latent variable maps { Z m | m = 1 , ..., N } .
Each latent variable map can be used by Eq.
(12) to produce a set of kernel maps { K m | m = 1 , ..., N } .
As illustrate in Figure 3(b), we sum these kernel maps as:
[Equation]
where K u ∈ R H × W × ( S × S × C ) is convoluted with the rainy image I for computing the object layer.
Experiments 5.1 Experimental Datasets
We compare SEIDNet with state-of-the-art methods, on the Rain100H [16], Rain100L [16],
Rain1400 [19], and SPA [3] datasets.
These datasets provide 1 , 800 / 200 / 12 , 600 / 638 , 492 images for training, along with 100 / 100 / 1 , 400 / 1 , 000 images for testing.
We also evaluate different methods on the Rain13K [10] dataset, which provides 13 , 712 images for training.
The test set of Rain13K contains 4 , 300 images, which are taken from the test sets of Test100 [23], Test1200 [4], Test2800 [19],
Rain100H, and Rain100L. Rain13K allows the models to be trained on the uniﬁed data and evaluated on the separate test sets, thus justifying the model generalization.
We report the performances of rain removal in terms of peak signal-to-noise ratio (PSNR) and structural similarity (SSIM).
5.2 Ablation Study of SEIDNet
Below, we use the test set of Rain100H [16] for the major evaluation of SEIDNet.
Sensitivity to the Number of Kernels Given the testing architecture of SEIDNet, we generate and aggregate multiple kernels for removing the rains on the image.
We change the number N of the generated kernels for each pixel, evaluating the impact on the performances.
We choose the number
N from the set { 1 , 2 , 4 , 8 , 16 , 32 } .
We report the computational overheads (i.e., GPU memory and testing time) in Figure 4(a–b), along with the performances (i.e., PSNR and SSIM) in Figure 4(c–d).
[Figure]
With N = 1 , we generate a single kernel for each pixel on the rainy image, yielding 31.457 PSNR and 0.9142 SSIM on the test set of Rain100H. With N = 8 , we considerably improve the performances (up to 33.217 PSNR and 0.9327 SSIM).
This is because more kernels better capture the visual patterns of rain streaks and object textures.
Too many kernels (e.g., N = 32 ) saturate the performances, but needing more computations.
Below, we use N = 8 as default.
SE
ID
Memory
Time
PSNR
SSIM D
D 1.
0.8809 G
D 1.
0.8849 D
G 1.
0.9172 G
G 2. refer to SE , ID , Memory and
Time as the status estimation, information decoupling, GPU memory (GB) and testing time (second).
D and G indicate the discriminative and generative networks, respectively.
The performances are reported on the test set of Rain100H.
Analysis of Network Components
The status estimation and information decoupling of SEIDNet are based on the generative CVAEs for computing the statuses and kernels.
To evaluate the generative power for rain removal, we experiment with using the discriminative networks for the status estimation and information decoupling.
For the status estimation, we resort to the convolution for predicting a single status map, based on the feature map of the rainy image.
For the information decoupling, we pass the visual feature map and the status map to the convolution, predicting a single kernel map for the rainy image.
The alternative architectures are provided in the supplementary material.
First, we remove all CVAEs, only using the discriminative network for the status estimation and information decoupling.
In this case, we yield the unsatisfactory 29.43 PSNR and 0.8809 SSIM (see the ﬁrst row of Table 1).
The discriminative network only provides a pair of status and kernel maps for the rainy image, missing the critical patterns of rains and objects.
Next, we use the discriminative network for either the status estimation or the information decoupling.
There is a CVAE for capturing the diverse patterns of the statuses or kernels, leading to better results (see the second and third rows of Table 1) than the discriminative networks.
Yet, the performances achieved by a CVAE are lower than SEIDNet.
This is because SEIDNet employs a couple of CVAEs to comprehensively model the correlation between each pair of status and kernel.
We analyze the correlation between the status and the kernel in the supplementary material.
[Table]
Various Combinations of Networks In Table 2, we compare different strategies of using the discriminative and generative networks for deraining.
Similar to SEIDNet, we use 132 convolutional layers to construct a discriminative network for predicting the kernel maps.
The generative SEIDNet outperforms the discriminative network (see the ﬁrst and second rows).
We combine the discriminative and generative networks by averaging the kernel maps.
This combination increases the network parameters but degrades the performances (see the last row).
A smarter combination of the discriminative and generative networks is needed for reducing the parameters and improving the performances.
the networks in the supplementary material
[Table]
Different Ways of Using CVAEs SEIDNet has a pair of CVAEs that model the factorized distributions of the status and the kernel.
We compare SEIDNet with the alternative methods, which use a CVAE without distribution factorization.
We list the results in Table 3.
Again, we illustrate the compared networks in the supplementary material.
First
[Table]
we set the single CVAE that only takes the visual feature map of the rainy image as the condition.
This method directly generates the kernel maps.
Because the status maps are unavailable for enabling the focused learning of the kernel maps, this method yields lower performances (see the ﬁrst row)
[Table] [Table]
Second, we use the single CVAE to generate the status and kernel maps.
Again, this CVAE takes the visual feature map of the rainy image as the only condition.
The status and kernel maps are generated by the separate decoder branches.
The single CVAE only depends on the training loss of the status map, for implicitly guiding the generation of the kernel map.
It is less effective than SEIDNet, where the generation of the kernel map is straightforwardly guided by the status map.
As a result, SEIDNet outperforms this single CVAE (see the last two rows of Table 3).
Discussion on Size of Training and Testing Dataset Conducting experiments on the unbalanced training and testing splits may reduce the conﬁdence of SEIDNet.
Thus, we reduce the training data in the extremely unbalanced dataset (i.e., SPA dataset with 638K/1K images for training and
[Table]
JORDER
[Table] [Table]
[Figure]
testing).
This is done by randomly sampling 1K training images from SPA dataset.
The random sample is three-fold, thus forming three different subsets, each of which contains 1K images for training different models.
The trained models are evaluated on the 1K images in the original test set.
With different subsets for training, SEIDNet yields better results than other methods (see the
Table. 6).
5.3 Comparison with State-of-the-Art Methods
[Table]
In Table 4, we compare SEIDNet with the recent methods on the Rain100H,
Rain100L, Rain1400, SPA, and Rain13K datasets, where SEIDNet outperforms other methods.
We show the deraining results of the competitive methods in Figure 5.
SEIDNet is trained on the uniﬁed data, outperforming other methods on 5 test sets of Rain13K. Thus, SEIDNet shows a better generalization across different test sets.
We average the performances on 5 test sets and report the results in Table 4.
We provide the results on the separate test sets (i.e., the test sets of Test100,
Test1200 [ 4], Test2800 [19], Rain100H, and Rain100L) in the Table 5.
The proposed SEIDNet outperforms ohter methods on the separate test sets, which justiﬁes the generalization of the deraining model.
5.4 Extensive Comparison on Different Tasks
[Table]
To evaluate the generalization of SEIDNet on differen tasks, we use SEIDNet to resolve the desnow, dehaze, and deshadow tasks.
We report the performances on different tasks in Tables 8, 7, 9, and 10.
We provide the visual results of SEIDNet on different tasks in Figure 6.
In Table 8, we use Snow100K dataset [20] for evaluation.
Snow100K has three subsets, i.e., Snow100KS, Snow100K-M, and Snow100K-L, where the snow ﬂakes in the images have small, medium, and large sizes.
The images of Snow100K-S only contains the small snow ﬂakes.
The snow ﬂakes in Snow100K-M have small and medium sizes, while those in Snow100K-L have small, medium, and large sizes.
In each subset, there are about 17K images for training/testing.
We also average the results (see "Overall" ) on Snow100K-S, Snow100K
[Table]
In Table
[Table]
we use the ITS&OTS dataset [21] to justify the generalization of SEIDNet on the haze removal task.
In the ITS&OTS dataset, the ITS subset contains 110,000 indoor images with haze.
The
OTS subset has 313,950 outdoor images with haze.
We train SEIDNet on the ITS and OTS subsets, respectively.
The network is evaluated on the indoor and outdoor test sets, respectively, where each test set contains 500 images.
In Table 7, we also report the results of other methods for haze removal.
[Table]
In Table 9 and 10, we use the adjusted ISTD (ISTD+) [57] and ISTD [22] datasets to evaluate the performances of different methods on the shadow removal task.
The images in the ISTD dataset are taken fromdifferent scenarios.
There are 1,330 and 540 images for training and testing, respectively.
Le et al.
[57] adjust the color inconsistency between the shadow and shadow free images of ISTD and achieve the ISTD+ dataset.
We use the image regions with/without shadow to test different methods (see the performances in "Shadow" and "Non-Shadow" ).
We also use the full images to test the methods and report the performances in "All" .
Conclusions
[Table]
The latest progress of rain removal beneﬁts from deep discriminative networks trained on large-scale datasets.
In this paper, we have proposed a generative network,
SEIDNet, to generate the pixel-wise status and kernel for rain removal.
SEIDNet has two CVAEs, which model the factorized probability distributions.
It learns the status and kernel spaces.
In contrast to the discriminative networks, SEIDNet enables the generation of multiple statuses for the pixel, for capturing the confusion between the appearances of rains and objects.
We employ these statuses for generating multiple kernels, reducing the confusing information and reﬁning the deraining result on the pixel.
SEIDNet achieves state-of-the-art performances on the public datasets.
In the future work, we plan to explore an effective strategy of combining the discriminative and generative networks for rain removal.
Negative Societal Impacts
Our approach can be broadly applied in many scenarios (e.g., autonomous vehicles and video surveillance).
One should be cautious of the problematic results, which may give rise to the infringement of privacy or economic interest.
Acknowledgements
We thank the anonymous reviewers for their constructive comments.
This work was supported by The
Hong Kong Polytechnic University under Grant P0030419, Grant P0042740, and Grant P0035358.