SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \mathcal {R} _ {\mathcal {T}} (f) \leq \frac {c}{2 \left(1 - e ^ {- c}\right)} \widehat {\mathcal {R}} _ {T} (f) + \frac {c}{1 - e ^ {- c}} \beta_ {\infty} (\mathcal {T} \| \mathcal {S}) \widehat {\mathcal {R}} _ {S} (f) + \Omega \\ + \frac {1}{1 - e ^ {- c}} \left(\frac {1}{n _ {t}} + \frac {\beta_ {\infty} (\mathcal {T} \| \mathcal {S})}{n _ {s}}\right) \left(2 \mathrm {K L} (f \| \pi) + \ln \frac {2}{\xi}\right), \tag {5} \\ \end{array}
$$
where  $\widehat{\mathcal{R}}_T(f)$  and  $\widehat{\mathcal{R}}_S(f)$  are the target and source empirical risks measured over target and source datasets  $T$  and  $S$ , respectively.  $\Omega$  is a constant and  $\mathrm{KL}(\cdot)$  is the Kullback-Leibler divergence.  $\beta_{\infty}(\mathcal{T}\| \mathcal{S})$  is a measurement of discrepancy between  $\mathcal{T}$  and  $\mathcal{S}$  defined as
$$
\beta_ {\infty} (\mathcal {T} \| \mathcal {S}) = \sup  _ {(\boldsymbol {x}, y) \in \operatorname {S U P P} (\mathcal {S})} \left(\frac {\mathcal {P} _ {(\boldsymbol {x} , y) \in \mathcal {T}}}{\mathcal {P} _ {(\boldsymbol {x} , y) \in \mathcal {S}}}\right) \geq 1, \tag {6}
$$
where  $\mathrm{SUPP}(\mathcal{S})$  denotes the support of  $\mathcal{S}$ . When  $\mathcal{S}$  and  $\mathcal{T}$  are identical,  $\beta_{\infty}(\mathcal{T}\| \mathcal{S}) = 1$ .
Theorem 1 reveals the upper bound of  $\mathcal{R}_{\mathcal{T}}(f)$  is negatively associated with the number of samples for source and target domains (i.e.,  $n_t$  and  $n_s$ ). Assuming  $n_t$  is fixed, increasing  $n_s$  can still increase generalization on the target domain. As such, it is possible to combine some domain-watermarked samples with benign samples to achieve target domain generalization. Its proof is in Appendix B.
In general, the most straightforward method to generate our domain watermark for the protected dataset is to randomly select a few samples  $(\pmb{x},y)$  from the original dataset  $\mathcal{D}$  and replace them with their domain-watermarked version  $(G_{d}(\pmb{x}),y)$ . However, as we will show in the experiments, the domain-watermarked image is usually significantly different from its original version. Accordingly, the adversaries may notice watermarked samples and try to remove them to bypass our defense. To ensure the stealthiness of our domain watermark, we propose to optimize a set of visually-indistinguishable modified data  $\{(x_i',y_i)|x_i' = x_i + \delta_i\}$  having similar effects to domain-watermarked samples. This is also a bi-level optimization problem, as follows.
$$
\left. \min  _ {\delta \subset \mathcal {B}} \left[ \mathbb {E} _ {(\hat {\boldsymbol {x}}, y) \sim \mathcal {T}} [ \mathcal {L} (f (\hat {\boldsymbol {x}}; \boldsymbol {w} (\delta)), y) ] - \lambda_ {3} \min  \left\{\mathbb {E} _ {(\bar {\boldsymbol {x}}, y) \sim \bar {\mathcal {T}}} [ \mathcal {L} (f (\bar {\boldsymbol {x}}; \boldsymbol {w} (\delta)), y) ], \lambda_ {4} \right\} \right], \right. \tag {7}
$$
$$
s. t. \boldsymbol {w} (\boldsymbol {\delta}) = \arg \min  _ {\boldsymbol {w}} \left[ \frac {1}{| \mathcal {D} _ {s} |} \sum_ {(\boldsymbol {x} _ {i}, y _ {i}) \in \mathcal {D} _ {s}} \mathcal {L} (f (\boldsymbol {x} _ {i} + \boldsymbol {\delta} _ {i}; \boldsymbol {w}), y _ {i}) + \frac {1}{| \mathcal {D} _ {b} |} \sum_ {(\boldsymbol {x} _ {j}, y _ {j}) \in \mathcal {D} _ {b}} \mathcal {L} (f (\boldsymbol {x} _ {j}; \boldsymbol {w}), y _ {j}) \right],
$$
where  $\mathbb{E}_{(\overline{\boldsymbol{x}},y)\sim \overline{\mathcal{T}}}[\mathcal{L}\left(f(\overline{\boldsymbol{x}};\boldsymbol {w}(\delta)),y\right)]$  represents the expected risk for the watermarked model on other unseen domains (i.e.,  $\overline{\mathcal{T}}$ ) and  $\mathcal{B} = \{\delta :||\delta ||_{\infty}\leq \epsilon \}$  where  $\epsilon$  is a visibility-related hyper-parameter.
The second term in Eq.(7) is to prevent the watermarked model can achieve a similar generalization performance on other unseen domains as the target domain  $\mathcal{T}$  to preserve the uniqueness of  $\mathcal{T}$  for verification purposes. We introduce two parameters  $\lambda_{3}$  and  $\lambda_{4}$  for preventing the second term dominant in the optimization procedure.  $\lambda_{4}$  is set as  $\mathbb{E}_{(\overline{x},y)\sim \overline{\mathcal{T}}}[\mathcal{L}(f(\overline{x};\boldsymbol{w}^{*}),y)]$  , where  $\boldsymbol{w}^{*}$  is obtained by training on the original dataset  $\mathcal{D}$ . Please find more optimization details in Appendix C.
In particular, our domain watermark is clean-label, i.e., we do not modify the label of modified samples as have done in most backdoor-based methods. As such, it is more stealthy.
4 Dataset Ownership Verification via Domain Watermark
In this section, we introduce how to conduct dataset ownership verification via our domain watermark. The overview of the entire procedure is shown in Figure 2.
As described in Section 3.2, models trained on our protected dataset (with domain watermark) can correctly classify some domain-watermarked samples while other benign models cannot. Accordingly, given a suspicious third-party model  $f$ , the defenders can verify whether it was trained on the protected dataset by examining whether the model has similar prediction behaviors on benign samples and their domain-watermarked version. The model is regarded as trained on the protected
Figure 2: The workflow of dataset ownership via our domain watermark. In the first step, we will generate domain-watermarked (DW) samples in a hardly-generalized domain of the benign dataset; In the second step, we will optimize a set of visually-indistinguishable modified samples that have similar effects to domain-watermarked samples. We will release those modified samples associated with remaining benign samples instead of the original dataset for copyright protection; In the third step, we identify whether a given third-party model is trained on our protected dataset by testing whether it has similar prediction behaviors in benign images and their DW version.
dataset if it has similar behaviors. To verify it, we design a hypothesis-test-guided method following previous works [3, 4], as follows.
Proposition 1. Suppose  $f(\pmb{x})$  is the posterior probability of  $\pmb{x}$  predicted by the suspicious model. Let variable  $\pmb{X}$  denotes the benign image and variable  $\pmb{X}'$  is its domain-watermarked version (i.e.,  $\pmb{X}' = G_d(\pmb{X})$ ), while variable  $P_b = f(\pmb{X})_Y$  and  $P_d = f(\pmb{X}')_Y$  indicate the predicted probability on the ground-truth label  $Y$  of  $\pmb{X}$  and  $\pmb{X}'$ , respectively. Given the null hypothesis  $H_0: P_b = P_d + \tau$  ( $H_1: P_b < P_d + \tau$ ) where the hyper-parameter  $\tau \in [0,1]$ , we claim that the suspicious model is trained on the protected dataset (with  $\tau$ -certainty) if and only if  $H_0$  is rejected.
In practice, we randomly sample  $m$  different benign samples to conduct the pairwise T-test [41] and calculate its p-value. The null hypothesis  $H_0$  is rejected if the p-value is smaller than the significance level  $\alpha$ . Besides, we also calculate the confidence score  $\Delta P = P_b - P_d$  to represent the verification confidence. The smaller the  $\Delta P$ , the more confident the verification.
Theorem 2. Let  $f(\pmb{x})$  is the posterior probability of  $\pmb{x}$  predicted by the suspicious model, variable  $\pmb{X}$  denotes the benign sample with label  $Y$ , and variable  $\pmb{X}'$  is the domain-watermarked version of  $\pmb{X}$ . Assume that  $P_b \triangleq f(\pmb{X})_Y > \eta$ . We claim that dataset owners can reject the null hypothesis  $H_0$  at the significance level  $\alpha$ , if the verification success rate (VSR)  $V$  of  $f$  satisfies that
$$
\sqrt {m - 1} \cdot (V - \eta + \tau) - t _ {\alpha} \cdot \sqrt {V - V ^ {2}} > 0, \tag {8}
$$
where  $t_{\alpha}$  is  $\alpha$ -quantile of  $t$ -distribution with  $(m - 1)$  degrees of freedom and  $m$  is sample size.
In general, Theorem 2 indicates that our dataset verification can succeed if the VSR of the suspicious model  $f$  is higher than a threshold (which is not necessarily  $100\%$ ). In particular, the assumption of Theorem 2 can be easily satisfied by using benign samples that can be correctly classified with high confidence. Its proof is included in Appendix D.
In this section, we conduct experiments on CIFAR-10 [1] and Tiny-ImageNet [42] with VGG [43] and ResNet [44], respectively. Results on STL-10 [45] are in Appendix F.
5.1 The Performance of Domain Watermark
Settings. We select seven baseline methods, containing three clean-label backdoor watermarks (i.e., Label-Consistent, Sleeper Agent, and UBW-C) and four poisoned-label watermarks (i.e., BadNets, Blended, WaNet, and UBW-P). Following the previous work [4], we set the watermarking rate  $\gamma = 0.1$ , perturbation constraint  $\epsilon = 16/255$  in all cases, and adopt the same watermark patterns and parameters. The example of samples used in different watermarks is shown in Figure 3. For our
Figure 3: The example of samples involved in different methods of dataset ownership verification.
Table 1: The watermark performance on CIFAR-10 and Tiny-ImageNet datasets. In particular, we mark harmful watermark results (i.e.,  $H > 0.5$  and  $\hat{H} > 0$ ) in red.
CIFAR-10	Tiny-ImageNet
Label Type↓	Method↓, Metric→	BA (%)	VSR (%)	H	H	BA (%)	VSR (%)	H	H
Poisoned-Label	BadNets	91.54	100	1.00	0.91	60.02	100	1.00	0.60
Blended	91.60	99.96	1.00	0.92	59.86	99.97	1.00	0.60
WaNet	90.61	97.3	0.97	0.87	57.29	96.14	0.96	0.51
UBW-P	91.47	84.52	0.85	0.76	56.14	84.09	0.84	0.44
Clean-Label	Label-Consistent	91.64	99.98	1.00	0.91	56.92	41.76	0.61	0.21
Sleeper Agent	90.73	93.24	0.93	0.84	56.82	87.44	0.87	0.47
UBW-C	86.32	89.06	0.89	0.80	51.79	81.20	0.81	0.41
DW (Ours)	90.86	90.45	0.10	-0.77	59.10	58.06	0.42	-0.52
Table 2: The effectiveness of dataset ownership verification via our domain watermark.
CIFAR-10	Tiny-ImageNet
Independent-D	Independent-M	Malicious	Independent-D	Independent-M	Malicious
ΔP	0.79	0.80	0.04	0.50	0.67	0.10
p-value	1.00	1.00	10-54	0.90	1.00	10-6
method, we set  $\lambda_3 = 0.3$ . We implement all baseline methods based on BackdoorBox [46]. Each result is averaged over five runs. Please find more details in Appendix E.
Evaluation Metrics. We adopt benign accuracy (BA) and verification success rate (VSR) to verify the effectiveness of dataset watermarks. Specifically, the VSR is defined as the percentage that verification samples can be classified as the assigned label (i.e., target label of baselines and ground-truth label of our method) by watermarked DNNs. We exploit harmful degree  $(H\in [0,1])$ , and relatively harmful degree  $(\hat{H}\in [-1,1])$  to measure watermark harmfulness. In general, the larger the BA and VSR while the smaller the  $H$  and  $\hat{H}$ , the better the dataset watermark.
Results. As shown in Table 1, the benign accuracy of our domain watermark is higher than clean-label backdoor watermarks in most cases, especially on the Tiny-ImageNet dataset. In particular, only our method is harmless. For example, both  $H$  and  $\hat{H}$  are 0.7 smaller than those of all baseline methods on CIRAR-10 datasets. Besides, as we will show in the next subsection, the VSR of our method is sufficiently high for correct ownership verification, although the VSR of our method is smaller than that of backdoor-based watermarks (especially on complicated datasets). The VSRs of benign models with our domain-watermarked samples on CIFAR-10 and Tiny-ImageNet are merely  $13\%$  and  $6\%$ , respectively. This mild potential limitation is because our VSR is restricted by the BA of watermark models. It is an unavoidable sacrifice for harmlessness.
5.2 The Performance of Dataset Ownership Verification via Domain Watermark
Settings. We evaluate our domain-watermark-based dataset ownership verification under three scenarios, including 1) independent domain (dubbed 'Independent-D'), 2) independent model (dubbed
Figure 4: Effects of the perturbation budget  $\epsilon$ .
Figure 5: Effects of the watermarking rate  $\gamma$ .
Figure 6: The resistance to fine-tuning.
Figure 7: The resistance to model pruning.
'Independent-M'), and 3) unauthorized dataset training (dubbed 'Malicious'). In the first case, we used domain-watermarked samples to query the suspicious model trained with modified samples from another domain; In the second case, we test the benign model with our domain-watermarked samples; In the last case, we test the domain-watermarked model with corresponding domain-watermarked samples. Notice that only the last case should be regarded as having unauthorized dataset use. More setting detail are described in Appendix G.
Evaluation Metrics. Following the settings in [4], we use  $\Delta P \in [-1, 1]$  and  $p$ -value  $\in [0, 1]$  for the evaluation. For the first two independent scenarios, a large  $\Delta P$  and  $p$ -value are expected. In contrast, for the third scenario, the smaller  $\Delta P$  and  $p$ -value, the better the verification.
Results. As shown in Table 2, our method can achieve accurate verification in all cases. Specifically, our approach can identify the unauthorized dataset usage with high confidence (i.e.,  $\Delta P \approx 0$  and  $p$ -value  $\ll 0.01$ ), while not misjudging when there is no unauthorized dataset utilization (i.e.,  $\Delta P \gg 0$  and  $p$ -value  $\gg 0.05$ ). Especially on the CIFAR-10 dataset (with high VSR), the  $p$ -values of independent cases are already 1 while that of the malicious scene is 50 powers smaller than a correct verification needs. These results verify the effectiveness of our dataset ownership verification.
5.3 Discussions
5.3.1 Ablation Studies
We hereby discuss the effects of two key hyper-parameters involved in our method (i.e.,  $\epsilon$  and  $\gamma$ ). Please find more experiments regarding other parameters and detailed settings in Appendix I.
Effects of Perturbation Budget  $\epsilon$ . We study its effects on both CIFAR-10 and Tiny-ImageNet datasets. As shown in Figure 4, the VSR increases with the increase of  $\epsilon$ . In contrast, the BA remains almost stable with different  $\epsilon$ . However, increasing  $\epsilon$  would also reduce the invisibility of modified samples. Defenders should assign it based on their specific needs.
Effects of watermarking Rate  $\gamma$ . As shown in Figure 6, similar to the phenomena of  $\epsilon$ , the VSR increases with the increase of  $\gamma$  while the BA remains almost unchanged on both datasets. In particular, even with a low watermarking rate ( $e.g., 1\%$ ), our method can still have a promising VSR. These results verify the effectiveness of our domain watermark again.
5.3.2 The Resistance to Potential Adaptive Methods
We notice that the adversaries may try to detect or even remove our domain watermark based on existing methods in practice. In this section, we discuss whether our method is resistant to them.
Due to the limitation of space, following the previous work [4], we only evaluate the robustness of our domain watermark under fine-tuning [47] and model pruning [48] in the main manuscript. As shown in Figure 6, fine-tuning has minor effects on both the VSR and the BA of our method. Our method is also resistant to model pruning for the BA decreases with the decrease of VSR. We have
also evaluated our domain watermark to more other representative adaptive methods. Please find more setting details and results in our Appendix J.
5.3.3 A Closer Look to the Effectiveness of our Method
In this section, we intend to further explore the mechanisms behind the effectiveness of our domain watermark. Specifically, we adopt t-SNE [49] to visualize the feature distribution of different types of samples generated by the benign model and its domain-watermarked version. As shown in Figure 8, the domain-watermarked samples stay away (with the normalized distance as 1.84) from those with their ground-truth label (i.e., '0'), although they still cluster together, under the benign model. In contrast, these domain-watermarked samples lay close (with the normalized distance as 0.40) to benign samples having the same class under the watermarked model.
Figure 8: The t-SNE of feature representations of samples for benign and watermarked models on the CIFAR-10 dataset. The target label is '0'.
These phenomena are consistent with the predictive behaviors of the two models and can partly explain the mechanism of our domain watermark. We will further explore it in our future works.
In this paper, we revisited the dataset ownership verification (DOV). We revealed the harmful nature of existing backdoor-based methods because their principle is making watermarked models misclassify 'easy' samples. To design a genuinely harmless DOV method, we proposed the domain watermark by leading watermarked DNNs to correctly classify some defender-specified 'hard' samples. We provided the theoretical analyses of our domain watermark and its corresponding ownership verification. We also verified its effectiveness on benchmark datasets. As the first non-backdoor-based method, our method can provide new angles and understanding to the design of dataset ownership verification to facilitate trustworthy dataset sharing.
Junfeng Guo and Heng Huang were partially supported by NSF IIS 1838627, 1837956, 1956002, 2211492, CNS 2213701, CCF 2217003, and DBI 2225775. Cong Liu was supported by the National Science Foundation under Grants CNS Career 2230968, CPS 2230969, CNS 2300525, CNS 2343653, and CNS 2312397.
[1] Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, 2009.
[2] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
[3] Yiming Li, Mingyan Zhu, Xue Yang, Yong Jiang, Tao Wei, and Shu-Tao Xia. Black-box dataset ownership verification via backdoor watermarking. IEEE Transactions on Information Forensics and Security, 2023.
[4] Yiming Li, Yang Bai, Yong Jiang, Yong Yang, Shu-Tao Xia, and Bo Li. Untargeted backdoor watermark: Towards harmless and stealthy dataset copyright protection. In NeurIPS, 2022.
[5] Ruixiang Tang, Qizhang Feng, Ninghao Liu, Fan Yang, and Xia Hu. Did you train on my dataset? towards public dataset protection with clean-label backdoor watermarking. ACM SIGKDD Explorations Newsletter, 2023.
[6] Lixu Wang, Shichao Xu, Ruiqi Xu, Xiao Wang, and Qi Zhu. Non-transferrable learning: A new approach for model ownership verification and applicability authorization. In ICLR, 2022.
[56] Alexandre Lacasse, François Laviolette, Mario Marchand, Pascal Germain, and Nicolas Usunier. Pac-bayes bounds for the risk of the majority vote and the variance of the gibbs classifier. NeurIPS, 2006.
[57] Pascal Germain, Alexandre Lacasse, Francois Laviolette, Mario March, and Jean-Francis Roy. Risk bounds for the majority vote: From a pac-bayesian analysis to a learning algorithm. Journal of Machine Learning Research, 2015.
[58] Shichao Xu, Lixu Wang, Yixuan Wang, and Qi Zhu. Weak adaptation learning: Addressing cross-domain data insufficiency with weak annotator. In ICCV, 2021.
[59] Tim Van Erven and Peter Harremos. Rényi divergence and kullback-leibler divergence. IEEE Transactions on Information Theory, 2014.
[60] Hossein Souri, Micah Goldblum, Liam Fowl, Rama Chellappa, and Tom Goldstein. Sleeper agent: Scalable hidden trigger backdoors for neural networks trained from scratch. In NeurIPS, 2022.
[61] Jonas Geiping, Liam Fowl, W Ronny Huang, Wojciech Czaja, Gavin Taylor, Michael Moeller, and Tom Goldstein. Witches' brew: Industrial scale data poisoning via gradient matching. In ICLR, 2021.
[62] Junfeng Guo and Cong Liu. Practical poisoning attacks on neural networks. In ECCV, 2020.
[63] Yiming Li, Tongqing Zhai, Yong Jiang, Zhifeng Li, and Shu-Tao Xia. Backdoor attack in the physical world. In ICLR Workshop, 2021.
[64] Junfeng Guo, Yiming Li, Xun Chen, Hanqing Guo, Lichao Sun, and Cong Liu. SCALE-UP: An efficient black-box input-level backdoor detection via analyzing scaled prediction consistency. In ICLR, 2023.
[65] Junfeng Guo, Ang Li, and Cong Liu. Aeva: Black-box backdoor detection using adversarial extreme value analysis. In ICLR, 2022.
[66] Junfeng Guo, Ang Li, Lixu Wang, and Cong Liu. Policycleanse: Backdoor detection and mitigation for competitive reinforcement learning. In ICCV, 2023.
[67] Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y Zhao. Neural cleansse: Identifying and mitigating backdoor attacks in neural networks. In IEEE S&P, 2019.
[68] Jonathan Hayase, Weihao Kong, Raghav Somani, and Sewoong Oh. Spectre: Defending against backdoor attacks using robust statistics. In ICML, 2021.
[69] Kunzhe Huang, Yiming Li, Baoyuan Wu, Zhan Qin, and Kui Ren. Backdoor defense via decoupling the training process. In ICLR, 2022.
[70] Shuaiqi Wang, Jonathan Hayase, Giulia Fanti, and Sewoong Oh. Towards a defense against federated backdoor attacks under continuous training. Transactions on Machine Learning Research, 2023.
A Technical Details for Generating Hardly-Generalized Domain 15
A.1 The Implementation of Transformation Module 15
A.2 The Optimization Process 15
B The Proof for Theorem 1 16
C Technical Details for Generating Protected Dataset 18
C.1 The Optimization Solution for Generating Protected Dataset 18
C.2 The Process of Generating Samples from Other Domains 19
C.3 The Selection of Hyper-parameters 19
D The Proof for Theorem 2 20
E The Detailed Settings for Experimental Datasets and Configurations 21
E.1 Datasets 21
E.2 The Demonstration of Domain Watermark for Each Dataset 22
E.3 Training Configurations 22
E.4 The Details for Implementing each Approach 23
F The Additional Results for the Performance of Domain Watermark 24
G The Detailed Settings for Dataset Ownership Verification 24
H The Additional Results for Dataset Ownership Verification 24
I Additional Results of Discussions 24
I.1 The Effects of  $\lambda_3$  24
I.2 Performance under Different Domain Watermarks 24
I.3 The Transferability of Domain Watermark 25
J Additional Results for the Resistance to Potential Adaptive Methods 27
K Can We Identify Domain-watermarked Samples by Training on a Large Dataset or with Complicated Data Augmentation? 28
K.1 The Resistance to Data Augmentation 29
K.2 The Resistance to Domain Adaption 29
L Reproducibility Statement 30
M Societal Impacts 30
N Discussions about Adopted Data 30
Figure 9: The architecture of  $G_{d}$ .
A Technical Details for Generating Hardly-Generalized Domain
This process is mainly motivated by [50, 51], which leveraged a transformation module with different convolution transformations to minimize the mutual information  $(I)$  between features from the source dataset  $(i.e., z)$  and data from the target domain  $(i.e., \hat{z})$ . Our work is also partially inspired by previous work on generating unlearnable samples [52], which crafted effective unlearnable samples by performing the bi-level optimization within each iteration.
A.1 The Implementation of Transformation Module
We follow previous work [53, 38, 50] to implement the transformation module for generating samples from a different domain (as shown in Fig. 9). Specifically, we design the transformation module as an ensemble of multiple (i.e., 4) convolution operations. Each convolution operation contains a convolution layer Conv, a feature shift module, and a corresponding transposed convolution layer TranConv. The detailed parameters for each convolution layer  $\mathrm{Conv}_i$  are detailed in Tab. 3. Following each convolution layer  $\mathrm{Conv}_i$ , we add a feature shift module to enhance the diversity of the generated samples. Specifically, each feature shift module contains two learnable parameters  $\mu_i$  and  $\sigma_i$  as mean shift and variance shift, following:
$$
\sigma_ {i} \cdot \frac {\operatorname {C o n v} _ {i} (\boldsymbol {x}) - \mu}{\sigma} + \mu_ {i}, \tag {9}
$$