SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 2: Average classification error (\%) of different algorithms on various real-world datasets with Squ and Sin shifts. We report the mean accuracy and standard deviation over five runs. The best algorithms are emphasized in bold. The number of data in each round is set as  $N_{t} = 5$ .
Squ	Sin
FIX	DANN	KMM	KLIEP	uLSIF	LR	OLRE	Accous	FIX	DANN	KMM	KLIEP	uLSIF	LR	OLRE	Accous
Diabetes	37.85	45.69	43.57	37.00	38.03	37.83	39.17	36.45	37.30	44.96	41.38	36.40	37.23	36.99	38.58	35.17
±6.67	±6.51	±6.97	±6.80	±6.45	±5.87	±6.67	±5.36	±7.02	±6.79	±6.46	±6.93	±6.88	±6.21	±7.45	±6.44
Breast	9.92	12.54	8.53	6.52	7.92	7.39	4.67	3.68	8.68	14.34	7.18	6.11	9.59	7.50	4.16	3.29
±0.23	±1.55	±0.30	±0.91	±1.39	±0.83	±0.23	±0.58	±0.56	±1.78	±0.39	±0.92	±1.02	±0.51	±0.42	±0.43
MNIST-SVHN	11.28	13.98	10.21	11.01	10.49	11.72	13.68	9.54	9.98	16.73	8.91	9.41	9.30	9.00	15.38	7.74
±1.22	±0.97	±0.53	±1.41	±0.67	±1.89	±0.59	±0.40	±1.29	±1.03	±0.68	±1.01	±0.53	±1.77	±0.88	±0.74
CIFAR10-CNIC10	34.75	38.13	34.08	34.27	33.80	33.56	32.61	31.08	36.44	42.53	37.00	36.58	36.53	36.21	38.24	35.42
±0.29	±1.58	±0.85	±1.24	±1.73	±1.59	±0.69	±0.60	±0.47	±1.09	±0.67	±1.52	±1.02	±1.31	±1.86	±1.22
algorithm Accous outperforms other contenders in the four types of shifts, whether in a slowly evolving environment (Lin) or in relatively non-stationary environments (Squ,Sin,Ber). The OLRE algorithm uses all historical data, but does not necessarily outperform contenders that use only current data, suggesting that we should selectively reuse the historical data.
Case Study on a Real-life Application. We conducted empirical studies on a real-world application using the yearbook dataset [53], which contains high school front-facing yearbook photos from 1930 to 2013. The photo distributions shift along years due to the changing social norms, fashion styles, and population demographics. Consequently, it is essential to adapt to these changes. We used photos before 1945 as the offline labeled dataset and generated the online unlabeled data stream using photos after 1945, arranged chronologically with 10 photos per round. Figure 5 shows the average error curves. The OLER algorithm performs well when the distributions do not change too much, but the method starts to suffer from large prediction errors after 400 iterations. In contrast, our method outperforms all competitors by selectively reusing historical data.
Figure 5: Average error on Yearbook dataset with real-life covariate shift.
This paper initiated the study of the continuous covariate shift with the goal of minimizing the cumulative excess risk of the predictors. We showed that the importance-weighted ERM method works effectively given high-quality density ratio estimators (Proposition 1), whose estimation can be cast as a dynamic regret minimization problem (Theorem 1). Instantiating with the logistic regression density ratio estimation model, we proposed a novel online that can adaptively reuse the historical data and enjoy a tight dynamic regret bound (Theorem 2). The regret bound finally implies an  $\widetilde{\mathcal{O}}(T^{-1/3}V_T^{1/3})$  averaged excess risk guarantee for the predictor. Experiments validate the effectiveness of the proposed method and demonstrate the need for selective reuse of historical data.
YJZ and MS were supported by the Institute for AI and Beyond, UTokyo. YJZ was also supported by Todai Fellowship. Peng Zhao was supported by NSFC (62206125) and JiangsuSF (BK20220776). The authors thank Yong Bai and Yu-Yang Qian for their assistance in the experiments.
A Omitted Details for Experiments
In this section, we first provide additional details that were omitted in section 5. We then provide the descriptions of our experimental setups, which include details of the datasets used, simulations of continuous covariate shifts, descriptions of the competing algorithms, and the parameter configuration of our proposed Accous algorithm. After that, we supplement the detailed numerical results obtained from the synthetic experiments.
A.1 Experiment Setup
Continuous Covariate Shift Simulation. To simulate the continuous covariate shift, we generate offline and online samples from different marginal distributions and annotate them with the same labeling function. Given two different marginal distributions  $\mathcal{D}'(\mathbf{x})$  and  $\mathcal{D}''(\mathbf{x})$ , the offline dataset  $S_0$  is sampled from the distribution  $\mathcal{D}_0(\mathbf{x}) = (1 - \alpha_0)\mathcal{D}'(\mathbf{x}) + \alpha_0\mathcal{D}''(\mathbf{x})$ , where  $\alpha_0 \in [0,1]$  is a mixture proportion coefficient. The online dataset  $S_t$  received at each iteration  $t$  is sampled from the distribution  $\mathcal{D}_t(\mathbf{x}) = (1 - \alpha_t)\mathcal{D}'(\mathbf{x}) + \alpha_t\mathcal{D}''(\mathbf{x})$ , where  $\alpha_t \in [0,1]$  is the time-varying coefficient. By changing the coefficient  $\alpha_t$  for  $t \geq 1$ , we can simulate different types of continuous covariate shifts with different shift patterns.
- Linear Shift (Lin): We set  $\alpha_{t} = t / T$  to simulate a gradually changing environment.
- Square Shift (Squ): In this case, the parameter  $\alpha_{t}$  switches between 1 and 0 every  $M$  rounds, where  $2M$  is the periodic length. We set  $M = \Theta (\sqrt{T})$  to simulate a fast-changing environment with periodic patterns.
- Sine Shift (Sin): We set  $\alpha_{t} = \sin \frac{i\pi}{M}$ , where  $i = t \mod M$  and  $M$  is the periodic length. We set  $M = \Theta(\sqrt{T})$  to simulate a fast-changing environment with periodic patterns.
- Bernoulli Shift (Ber): In this case, we keep the  $\alpha_{t} = \alpha_{t - 1}$  with probability  $p\in [0,1]$  and otherwise set  $\alpha_{t} = 1 - \alpha_{t - 1}$ . We set  $p = \Theta (1 / \sqrt{T})$  to simulate a fast-changing environment but without periodic patterns.
Datasets. For the synthetic dataset, we generate  $\mathcal{D}'(\mathbf{x})$  and  $\mathcal{D}''(\mathbf{x})$  using the multinomial Gaussian distribution, wherein each class is generated from a Gaussian distribution. For  $\mathcal{D}'(\mathbf{x})$ , the positive class data is generated from  $\mathcal{N}(\mathbf{x}; \boldsymbol{\mu}_1^p, \Sigma)$ , and the negative class data is generated from  $\mathcal{N}(\mathbf{x}; \boldsymbol{\mu}_1^n, \Sigma)$ . Here,  $\boldsymbol{\mu}_1^p$  is set to  $-1.2$  in  $\mathbb{R}^{12}$ , and  $\boldsymbol{\mu}_1^n$  is set to  $-0.8$  in  $\mathbb{R}^{12}$ . As for  $\mathcal{D}''(\mathbf{x})$ , its positive class data is generated from  $\mathcal{N}(\mathbf{x}; \boldsymbol{\mu}_2^p, \Sigma)$ , and the negative class data is generated from  $\mathcal{N}(\mathbf{x}; \boldsymbol{\mu}_2^n, \Sigma)$ . Here,  $\boldsymbol{\mu}_2^p$  is specified as 1.2 in  $\mathbb{R}^{12}$ , and  $\boldsymbol{\mu}_2^n$  as 0.8 in  $\mathbb{R}^{12}$ . The covariance matrix is set as  $\Sigma = I_d$  in  $\mathbb{R}^{12 \times 12}$ , and the prior probability for the positive class data is fixed at 0.5. Besides, we generated offline data with  $\alpha_0 = 0.9$  and introduced a time-varying coefficient,  $\alpha_t$ , to simulate continuous covariate shifts.
The benchmark datasets are generated from real-world datasets.
- Diabetes (Dia) is a tabular UCI dataset. Each sample is an 8-dimensional vector with a binary label. We split the dataset into two subsets and choose the split such that the classifier trained on one dataset generalizes poorly to the other. We choose one of them as the offline dataset and combine them with a time-varying coefficient to generate the online data.
- Breast (Bre) is a tabular UCI dataset. Each sample is a 9-dimensional vector with a binary label. This dataset is generated similarly to the Diabetes dataset.
- MNIST-SVHN  $(M - S)$  contains the digital images collected from handwriting or natural scene images. A feature extractor pre-trained on this dataset produces a 512-dimensional vector with a 10-class label for each sample. We choose  $10\%$  MNIST data and  $90\%$  SVHN data as the offline dataset and generate the online data by mixing the MNIST and SVHN datasets with a time-varying proportion coefficient  $\alpha_{t}$ .
- CIFAR10-CINIC10 (C-C) contains real-life subjective images collected from different sources. This dataset is generated similarly to the MNIST-SVHN dataset.
We also conduct experiments on a real-world dataset where the underlying data distributions exhibit a continuous shift.
- Yearbook [53]: This dataset contains the 37,921 frontal-facing American high school yearbook photos from 1930 to 2013 [54]. Each photo is a  $32 \times 32 \times 1$  grayscale image associated with a binary label representing the student's gender. This dataset captures the real-life changing of social norms, fashion styles, and population demographics. The offline data is generated with photos before 1945, the online data is generated from 1946, and the data size is 10 per round.
Contenders. We compare our proposed approach with six contenders, including:
- FIX is the baseline method that predicts with the classifier that are pre-trained on offline data without any updates in the online adaptation stage.
- is a general domain adaptation algorithm that mitigates the discrepancy between the training and test distributions by learning an invariant feature representation. We adapt this method to the online setting by running DANN at each round  $t \in [T]$  based on the offline data  $S_0$  and the online data  $S_t$  available at that round.
- IW-KMM is an importance weighting learning method that first estimates the density ratios by the Kernel Mean Matching (KMM) method [26] and then performs the IWERM to train the model. At each iteration  $t \in [T]$ , the density ratios are estimated with the offline dataset  $S_0$  and the online dataset  $S_t$  by the KMM. Then the model is trained by running the IWERM method on the offline data  $S_0$ .
- IW-KLIEP is also an importance weighting learning method, where the density ratios are estimated by the KLIEP method [25]. At each iteration  $t \in [T]$ , after the density ratios have been estimated by KLIEP, the learner performs the IWERM over the offline data  $S_0$ .
- IW-uLSIF is an importance weighting learning method where the density ratio estimated by the uLSIF method is used [28]. At each iteration  $t \in [T]$ , the learner first estimates the density ratios by the uLSIF and then performs the IwERM on the offline data  $S_0$ .
-  $LR$  serves as an ablation study for the proposed Accous algorithm. At each iteration  $t \in [T]$ , the learner first estimates the density ratios by an ONS with a logistic regression model and then performs the IwERM on the offline data  $S_0$ .
- OLRE also serves as an ablation study for the proposed Accous algorithm. It estimates the density ratios in an on-line manner by running an ONS with a logistic regression model with all the historical data and then performing the IWERM each round.
Accous Setup. We implement the Accous algorithm using a logistic regression model. Before running the Accous algorithm, two parameters must be defined. They are the maximum norm of the feature  $R$  and the maximum norm of the density functions  $S$ . We set  $R$  by directly calculating the data norm. The covariate shift level determines the parameter  $S$ , which is difficult to predict in advance. We found that setting  $S = d / 2$  is a good choice in the experiments. For stable performance, we set a threshold of 100 and truncate the large weights after the density ratio estimation in each round. We also ignore the base models with interval lengths 1, 2 in the density ratio estimation.
For the synthetic, diabetes, and breast tabular datasets, the linear model is used. For the MNIST-SVHN, CIFAR10-CINIC10, and Yearbook datasets, the deep model is used for density ratio estimation and predictive model training. The deep models consist of a backbone and a linear layer. In all experiments, only the linear layer of the neural network is updated, while the backbone parameters trained with offline data remain fixed. Backbone settings for different datasets are presented below.
- MNIST-SVHN and CIFAR10-CINIC10: The backbone is a pre-trained ResNet34 backbone from torchvision with its weights initialized by training on ImageNet. Both the weight estimator and the classifier share the same backbone and its corresponding parameters.
- Yearbook: Our network adopts a similar structure and updating process as in the MNIST-SVHN and CIFAR10-CINIC10 experiments, following the configuration described in [53]. The backbone of our network consists of a CNN model with the default settings.
Computational Resources. We run experiments with two Xeon Gold 6248R processors (24 cores, 3.0GHz base, 4.0GHz boost), eight Tesla V100S GPUs (32GB video memory each), 768GB RAM, all managed by the Ubuntu 20.04 operating system.
Table 3: Average error  $(\%)$  over the entire time horizons for different algorithms under various simulated shifts. The best algorithms are emphasized in bold.
Nt=1	Nt=5
FIX	DANN	KMM	KLIEP	uLSIF	LR	OLRE	Accous	FIX	DANN	KMM	KLIEP	uLSIF	LR	OLRE	Accous
Lin	32.85	46.93	37.97	36.26	38.34	36.51	27.13	27.69	32.59	40.33	32.68	34.13	31.70	32.21	27.34	27.11
±0.43	±2.40	±0.56	±0.82	±0.40	±1.12	±0.74	±0.49	±0.53	±2.99	±0.86	±0.30	±0.46	±0.72	±0.81	±0.69
Squ	33.19	40.59	35.50	35.13	36.44	36.21	34.82	29.30	33.37	41.24	31.52	30.94	32.07	31.63	34.46	28.46
±0.56	±3.28	±0.33	±0.61	±0.79	±0.46	±0.30	±0.82	±0.67	±4.01	±0.46	±0.53	±0.36	±1.08	±0.86	±0.54
Sin	32.80	42.65	36.25	35.83	37.01	37.42	33.15	31.08	32.66	39.94	34.15	34.47	33.93	34.62	31.85	30.57
±0.75	±4.49	±0.37	±0.65	±0.38	±0.73	±0.50	±0.58	±0.80	±2.94	±0.45	±0.72	±0.37	±0.56	±0.28	±0.59
Ber	34.11	46.20	36.49	37.07	36.28	37.42	34.91	29.55	33.89	45.33	32.89	33.52	32.01	34.13	32.57	28.60
±0.45	±4.12	±0.42	±0.67	±0.36	±0.85	±0.47	±0.46	±0.67	±2.88	±0.36	±0.78	±0.59	±0.72	±0.40	±0.36
A.2 Supplementary Numerical Results for Section 5.1
In Section 5.1 we summarize the results of the synthetic experiments, and here in Table 3, we supplement their detailed numerical results. Overall, Accous outperforms almost all other methods in the four shift patterns. Compared to the competitors that use only one round of online data, our proposed method achieves a lower average error. The general domain adaptation method DANN is also inferior in the online setting due to the limited amount of data per round, which makes it difficult to learn a good representation. The proposed Accous algorithm also outperforms the OLRE algorithm, which reuses all historical data. The above evidence demonstrates the need for selective reuse of historical data and validates the effectiveness of our algorithm.
B More Related Work
B.1 Related Work on Non-stationary Online Learning
Online convex optimization (OCO) [13] is a powerful paradigm to handle sequential prediction problems. To adapt to the continuously shifting environments, we exploit the online ensemble technique [15, 16] developed in non-stationary online learning literature to hedge the uncertainty. In this part, we will introduce the related works and discuss the difference between our algorithm and the previous work.
Specifically, OCO paradigm considers a  $T$ -round game between a learner and the environment. At every round  $t$ , the learner makes a prediction  $\theta_t \in \Theta$ , and, at the same time, the environments reveal the convex loss function  $L_t: \Theta \to \mathbb{R}$ . Then, the learner suffers from the loss  $L_t(\theta_t)$  and observes certain information about  $L_t$  to update the model for the next iteration. A classical performance measure for OCO framework in non-stationary environments is the dynamic regret,
$$
\operatorname {R e g} _ {T} ^ {\mathbf {d}} \left(\left\{\widetilde {L} _ {t}, \theta_ {t} ^ {*} \right\} _ {t = 1} ^ {T}\right) = \sum_ {t = 1} ^ {T} L _ {t} \left(\theta_ {t}\right) - \sum_ {t = 1} ^ {T} L _ {t} \left(\theta_ {t} ^ {*}\right), \tag {10}
$$
which compares the learner's prediction with the function minimizer  $\theta_t^*$  at every iteration. Over the decades, a variety of online learning algorithms have been proposed to optimize this dynamic regret measure [38, 39, 55, 56, 57, 58, 59, 60, 61, 62, 63] in the online convex optimization problem.
Worst-case Dynamic Regret. Many works focus on the full information setting [57, 58, 59, 60, 63], where the learner can observe the loss functions entirely. We call the dynamic regret in such a case as "worse-case" dynamic regret in the sense that the comparator  $\theta_t^*$  is exactly the minimizer of the observed function  $L_{t}$ . According to different complexity measures for quantifying the environmental non-stationarity, there are two lines of results. The first is the path length of comparators  $V_{T}^{\theta} = \sum_{t=2}^{T} \|\theta_{t}^{*} - \theta_{t-1}^{*}\|_{2}$  measuring the variation of the function minimizers; and the second one is the variation of the function values  $V_{T}^{L} = \sum_{t=2}^{T} \max_{\theta \in \Theta} |L_{t}(\theta) - L_{t-1}(\theta)|$ . For the full information setting, the prior art [63] showed that a simple greedy strategy can already be already well-behaved and achieves an  $\mathcal{O}(\min \{V_{T}^{\theta}, V_{T}^{L}\})$  dynamic regret bound that minimizes the two kinds of non-stationarity measure simultaneously.
Universal Dynamic Regret. Competing to the sequences of observed functions' minimizers can be sometimes too pessimistic for the online optimizing, which may lead to a severe overfitting. A more appropriate performance measure is the universal dynamic regret [38], which has drawn more and more attention in recent years. Universal dynamic regret compares the learner's prediction with an
arbitrary comparator sequence  $\{\nu_t\}_{t = 1}^T$
$$
\mathbf {R e g} _ {T} ^ {\mathbf {d}} \left(\left\{\widetilde {L} _ {t}, \nu_ {t} \right\} _ {t = 1} ^ {T}\right) = \sum_ {t = 1} ^ {T} L _ {t} \left(\theta_ {t}\right) - \sum_ {t = 1} ^ {T} L _ {t} \left(\nu_ {t}\right). \tag {11}
$$
Let  $V_T^\nu = \sum_{t=1}^T \| \nu_t - \nu_{t-1} \|_2$  be the path length of the comparator sequence  $\{\nu_t\}_{t=1}^T$ . [38] showed that the online gradient descent algorithm achieves an  $\mathcal{O}((1 + V_T^\nu)\sqrt{T})$  universal dynamic regret bound for convex function, whereas there is still a gap to the  $\Omega(\sqrt{T(1 + V_T^\nu)})$  lower bound established by [15], who further proposed an algorithm attaining an  $\mathcal{O}(\sqrt{T(1 + V_T^\nu)})$  dynamic regret bound and thus closed the gap. Their key algorithmic ingredient is the online ensemble structure [64], which hedges the non-stationarity of the environments by employing a meta-learner to ensemble the predictions from a group of base-learners. When, the loss function is convex and smooth, the  $\mathcal{O}(\sqrt{T(1 + V_T^\nu)})$  can be improved to  $\mathcal{O}(\sqrt{(1 + \min\{F_T, G_T\} + V_T^\nu)(1 + V_T^\nu)})$  [16], where  $F_T = \sum_{t=1}^T L_t(\nu_t)$  and  $G_T = \sum_{t=2}^T \max_{\theta \in \Theta} \| \nabla L_t(\theta) - \nabla L_{t-1}(\theta) \|_2^2$  are two problem-dependent quantities capturing different kinds of easiness of the environments. For the exp-concave and smooth functions, [19] showed an  $\mathcal{O}(T^{1/3}(V_T^\nu)^{2/3})$  universal dynamic regret by an improper algorithm (i.e., the output decisions can be slightly out of the feasible domain, while comparators have to locate in the domain), which is proven to be minimax optimal for exp-concave functions. Note that the worst-case dynamic regret (10) is clearly a special case of the universal dynamic regret defined in (11), so a universal dynamic regret upper bound directly implies an upper bound for the worst-case dynamic regret, by substituting comparators  $\{\nu_t\}_{t=1}^T$  as functions' minimizer  $\{\theta_t^*\}_{t=1}^T$ . This method yields a path-length type bound. Furthermore, we mention that it is also possible to obtain a function-variation type worst-case dynamic regret bound from the universal dynamic regret guarantee, which is revealed in [61, Appendix A.2].
An Intermediate Dynamic Regret. There is also an intermediate case between the worst-case and universal dynamic regret. We refer to it the "intermediate" dynamic regret in this paper. In this intermediate case, the online learner still aims to optimize the worst-case dynamic regret of the loss function  $L_{t}(\theta)$ , but she can only observe a noisy feedback  $\widehat{L}_{t}(\theta)$  at each iteration. Since the comparator  $\theta_{t}^{*} = \arg \min_{\theta \in \Theta} L_{t}(\theta)$  is not the minimizer of the observed loss  $\widehat{L}_{t}(\theta)$ , such a kind of problem is more challenging than the worst-case dynamic regret minimization problem given the uncertainty of the comparator sequence. [39] showed a restarted online gradient descent algorithm can achieve an  $\mathcal{O}(T^{1/3}(V_T^L)^{2/3})$  dynamic regret bound for convex function and an  $\mathcal{O}(\sqrt{T V_T^L})$  bound for strongly convex function under the noisy feedback. However, their algorithm requires the knowledge of the non-stationarity measure  $V_T^L = \sum_{t=2}^{T} \max_{\theta \in \Theta} |L_t(\theta) - L_{t-1}(\theta)|$  defined over  $L_{t}$ , which is generally unknown. [48] showed that an  $\mathcal{O}(T^{1/3}(V_T^\theta)^{2/3})$  bound is achievable with the noisy feedback when  $L_{t}$  is a one-dimensional squared loss. The proposed method is based on the de-noising technique and are free from the knowledge of  $V_T^\theta$ . Evidently, this intermediate dynamic regret is still a feasible realization of the universal dynamic regret, so one can achieve the intermediate dynamic regret bound by the universal dynamic regret. However, this reduction will only provide an expected guarantee. More detailed discussions are provided below.
Comparison of Our Results and Earlier Works. Our  $\mathbf{Reg}_T^{\mathbf{d}}(\{\widetilde{L}_t,\theta_t^*\}_{t = 1}^T)\leq \mathcal{O}(T^{1 / 3}(V_T^\theta)^{2 / 3})$  high-probability dynamic regret bound for logistic regression (Theorem 2) falls into the intermediate case, where we aim to minimize the worst-case dynamic regret bound in terms of  $L_{t}$  with noisy feedback. The closest related work is by [19], who also exploited an online ensemble structure and achieved a universal dynamic regret in the form of  $\mathbf{Reg}_T^{\mathbf{d}}(\{\widetilde{L}_t,\nu_t\}_{t = 1}^T)\leq \mathcal{O}(T^{1 / 3}(V_T^\nu)^{2 / 3})$  universal dynamic regret by taking  $\widehat{L}_t$  as the input. Further choosing  $\nu_{t} = \theta_{t}^{*}$  and taking the expectation over  $S_{t}$  for each round, their result implies an expectation bound  $\mathbb{E}[\mathbf{Reg}_T^{\mathbf{d}}]\leq \mathcal{O}(T^{1 / 3}(V_T^\theta)^{2 / 3})$ . One main advantage of our result is that the bound holds with high probability. To achieve the high-probability bound, we twist the meta-learner from Hedge [42] to Adapt-ML-Prod [43], which helps control the generalization error between  $\widehat{L}_t$  and  $\widetilde{L}_t$  with the negative term introduced by the exp-concavity of the loss function. Besides, similar to [19], our analysis crucially relies on providing a squared formulation of the dynamic regret  $\mathcal{O}(\max \{1,|I|(V_T^\theta)^2\})$  on each interval  $I$ . However, to obtain such a result, [19] crucially rely on a complicated analysis on the KKT condition to depict the structure of comparators. We greatly simplify the analysis by exploiting the fact that in our case the comparator  $\theta_t^*$  is actually the minimizer of expected functions  $L_{t}$ , hence avoiding analyzing KKT condition. Our
analysis is applicable to the case when the minimizers lie in the interior of the feasible domain and require the smoothness of the online functions only (which does not rely on the particular properties of logistic loss). This result might be of independent interest in other online learning studies.
C Omitted Details for Section 3
This section provides the proofs for Section 3 in the main paper.
C.1 Proof of Proposition 1