SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The learning rates of ROGD, UOGD, and ATLAS are set according to suggestions in the original works. The learning rate of FLH-FTL is set to  $1 / K$ . This corresponds to a faster rate than the theoretically optimal learning rate given in Line 2 of Algorithm 7. It has been observed in prior works such as Baby et al. [7] that the theoretical learning rate is often too conservative and faster rates lead to a better performance.
F.1 Supervised Online Label Shift Experiment Details
For each dataset, we first fix the number of time steps and then simulate the label marginal shift. To train the learner with all the methods, we store all the online data observed giving the storage complexity of  $\mathcal{O}(T)$ . We observe  $N = 50$  examples at every iteration and we split the observed labeled examples into 80:20 split for training and validation. The validation examples are used to decide the number of gradient steps at every time step, in particular, we take gradient steps till the validation error continues to decrease.
- MNIST [50]: An image dataset of 10 types of handwritten digits. At each step, we sample 50 samples with the label marginal that step without replacement and reveal the examples to the learner. We used an MLP for prediction with three consecutive hidden layers of sizes 100, 100, and 20. It is trained for a single epoch with a learning rate 0.1, momentum 0.9, batch size 200, and  $l_{2}$  regularization  $1 \times 10^{-4}$ .
- CIFAR-10 [49]: A dataset of colored images of 10 items: airplane, automobile, bird, cat, deer, dog, frog, horse, ship, and truck. At each step, we sample 50 samples with the label marginal that step without replacement and reveal the examples to the learner. It is finetuned for 70 epochs with learning rate 0.1, momentum 0.9, batch size 200, and  $l_{2}$  regularization  $1 \times 10^{-4}$ .
We simulate Bernoulli label shifts to capture different non-stationary environments.
Connection of CT-RS to weighted ERM Before making the connection, we first re-visit the CT-RS algorithm. Step 1: Maintain a pool of all the labeled data received till that time step, and at every
iteration, we randomly sample a batch with uniform label marginal to update the model. Step 2: Weight the softmax outputs of the updated model with estimated label marginal. Below we show that it is equivalent to wERM:
$$
\begin{array}{l} f _ {t} = \underset {f \in \mathcal {H}} {\operatorname {a r g m i n}} \sum_ {i = 1} ^ {t - 1} \sum_ {j = 1} ^ {N} \frac {\hat {q} _ {t} (y _ {i , j})}{\hat {q} _ {i} (y _ {i , j})} \ell (f (x _ {i, j}), y _ {i, j}) \\ = \operatorname * {a r g m i n} _ {f \in \mathcal {H}} \sum_ {k = 1} ^ {K} \hat {q} _ {t} (k) \sum_ {i = 1} ^ {t - 1} \sum_ {j = 1} ^ {N} \frac {\mathbb {1} \left(y _ {i , j} = k\right)}{\hat {q} _ {i} (k)} \ell \left(f \left(x _ {i, j}\right), k\right) \\ = \operatorname * {a r g m i n} _ {f \in \mathcal {H}} \sum_ {k = 1} ^ {K} \frac {\hat {q} _ {t} (k)}{(1 / K)} \sum_ {i = 1} ^ {t - 1} \sum_ {j = 1} ^ {N} \frac {\mathbb {1} (y _ {i , j} = k)}{K \cdot \hat {q} _ {i} (k)} \ell (f (x _ {i, j}), k) \\ = \operatorname * {a r g m i n} _ {f \in \mathcal {H}} \sum_ {k = 1} ^ {K} \hat {\mu} _ {t, k} \underbrace {\sum_ {i = 1} ^ {t - 1} \sum_ {j = 1} ^ {N} \frac {\mathbb {1} (y _ {i , j} = k)}{\hat {\mu} _ {i , k}} \ell (f (x _ {i , j}) , k)} _ {L _ {t - 1, k}} \\ \end{array}
$$
where  $\hat{\mu}_{t,k} = \hat{q}_t(k) / (1 / K)$  is the importance ratio at time  $i$  with respect to uniform label marginal. Similarly, we define  $\hat{\mu}_{i,k} = \hat{q}_i(k) / (1 / K)$ . Here,  $L_{t - 1,k}$  is the aggregate loss at  $t$ -th time step for  $k$ -th class such that at each step the sampling probability of that class is uniform. By continually training a classifier with CT-RS, Step 1 approximates the classifier  $\widetilde{f}_t$  trained to minimize the average of  $L_{t - 1,k}$  over all classes with uniform proportion for each class. To update the classifier  $\widetilde{f}_t$  according to label proportions at time  $t$ , we update the softmax output of  $\widetilde{f}_t$  according to  $\hat{\mu}_t$ .
The primary benefit of CT-RS over wERM is to avoid re-training from scratch at every iteration. Instead, we can leverage the model trained in the previous iteration to warm-start training in the next iteration.
G Additional Unsupervised Online Label Shift Experiments
G.1 Additional results with Monotone and Square Shifts and Low Amount of Holdout Data
Methods	Synthetic	MNIST	CIFAR	EuroSAT	Fashion	ArXiv
Mon	Squ	Mon	Squ	Mon	Squ	Mon	Squ	Mon	Squ	Mon	Squ
Base	8.7±0.1	8.5±0.2	4.7±0.0	4.4±0.2	17±0	17±0	13±0	13±0	15±0	15±0	22±0	21±0
OFC	6.9±0.1	6.6±0.3	4.1±0.1	3.9±0.2	14±0	14±0	11±1	11±0	9.0±0.0	9.6±0.5	18±1	18±0
Oracle	5.2±0.2	3.6±0.2	2.5±0.1	2.2±0.1	7.7±0.1	6.8±0.2	5.3±0.2	4.4±0.0	5.1±0.1	4.1±0.1	6.9±0.3	6.6±0.2
FTH	7.1±0.3	6.8±0.4	4.1±0.1	4.0±0.0	13±1	13±0	11±0	11±0	9.3±0.6	8.9±0.4	19±1	18±0
FTFWH	6.3±0.2	7.0±0.0	4.0±0.0	4.1±0.1	12±0	13±0	9.9±0.2	11±0	8.4±0.3	9.1±0.5	18±1	18±0
ROGD	7.8±0.3	7.8±0.3	4.5±0.2	5.4±1.7	14±1	15±0	11±0	14±1	8.9±0.4	10±1	19±1	21±1
UOGD	8.1±0.3	8.1±0.5	4.9±0.1	4.8±0.4	15±1	15±0	10±1	11±1	11±2	12±2	20±1	19±0
ATLAS	8.0±0.0	8.2±0.5	4.6±0.2	4.5±0.3	15±1	15±0	10±1	11±1	12±2	12±1	20±1	19±1
FLH-FTL (ours)	6.3±0.3	5.6±0.4	4.0±0.0	4.0±0.0	12±0	12±0	10±0	10±0	8.6±0.4	8.4±0.4	18±1	17±0
Synthetic	MNIST	CIFAR	EuroSAT	Fashion	ArXiv
Mon	Squ	Mon	Squ	Mon	Squ	Mon	Squ	Mon	Squ	Mon	Squ
FTH	0.11±0.00	0.21±0.01	0.14±0.00	0.27±0.00	0.15±0.01	0.28±0.00	0.14±0.01	0.28±0.00	0.16±0.02	0.28±0.01	0.18±0.00	0.30±0.00
FTFWH	0.05±0.00	0.23±0.01	0.07±0.00	0.30±0.00	0.07±0.00	0.30±0.00	0.07±0.00	0.30±0.00	0.08±0.01	0.31±0.01	0.09±0.00	0.32±0.00
ROGD	0.18±0.01	0.29±0.01	0.28±0.05	0.41±0.04	0.22±0.04	0.37±0.03	0.27±0.03	0.41±0.02	0.21±0.01	0.37±0.01	0.21±0.01	0.36±0.01
FLH-FTL (ours)	0.05±0.00	0.11±0.00	0.07±0.00	0.15±0.00	0.09±0.01	0.17±0.00	0.08±0.01	0.17±0.01	0.09±0.01	0.18±0.02	0.11±0.00	0.24±0.00
G.2 Additional results with All of Holdout Data
Table 7: Results for UOLS problems with monotone and square shifts using low amount of holdout data. Top: Classification Error. Bottom: Mean-squared error in estimating label marginal.
Methods	Synthetic	MNIST	CIFAR	EuroSAT	Fashion	ArXiv
Ber	Sin	Ber	Sin	Ber	Sin	Ber	Sin	Ber	Sin	Ber	Sin
Base	8.6±0.3	8.2±0.3	3.8±0.3	3.9±0.0	17±0	16±0	13±0	13±0	15±0	15±0	23±0	19±0
OFC	6.7±0.2	5.5±0.2	3.4±0.4	3.4±0.2	13±0	11±0	11±1	9.8±1.3	8.3±0.5	6.8±0.2	21±1	14±0
Oracle	3.7±0.1	3.7±0.1	1.7±0.2	1.5±0.1	6.3±0.1	5.9±0.1	4.0±0.0	4.1±0.1	3.5±0.2	3.6±0.1	7.8±0.2	5.1±0.1
FTH	6.8±0.2	5.5±0.3	3.2±0.2	3.2±0.2	12±0	10±0	11±0	9.5±0.1	8.0±0.0	6.8±0.2	20±0	14±0
FTFWH	6.6±0.3	5.5±0.2	3.3±0.2	3.2±0.1	12±0	10±0	10±0	9.4±0.2	7.9±0.0	6.9±0.2	20±0	14±0
ROGD	7.8±0.3	7.2±0.3	4.7±0.3	3.3±0.2	15±0	11±0	11±0	10±0	14±5	8.2±0.2	23±0	16±1
UOGD	7.6±0.4	7.0±0.0	3.2±0.2	3.2±0.2	11±0	10±0	7.7±0.0	7.3±0.2	9.6±0.2	8.6±0.1	19±0	14±0
ATLAS	7.5±0.3	6.8±0.3	3.2±0.3	3.2±0.2	12±0	11±0	9.1±0.0	8.3±0.2	12±0	11±0	21±0	16±0
FLH-FTL (ours)	5.4±0.3	5.3±0.2	3.2±0.2	3.3±0.2	11±0	10±0	9.4±0.2	9.3±0.1	7.5±0.1	7.0±0.0	19±0	14±0
Synthetic	MNIST	CIFAR	EuroSAT	Fashion	ArXiv
Ber	Sin	Ber	Sin	Ber	Sin	Ber	Sin	Ber	Sin	Ber	Sin
FTH	0.20±0.00	0.10±0.00	0.25±0.00	0.14±0.00	0.28±0.00	0.14±0.00	0.27±0.00	0.14±0.00	0.27±0.00	0.14±0.00	0.29±0.00	0.15±0.00
FTFWH	0.19±0.00	0.09±0.00	0.24±0.00	0.13±0.00	0.24±0.00	0.13±0.00	0.26±0.00	0.13±0.00	0.24±0.00	0.13±0.00	0.28±0.00	0.15±0.00
ROGD	0.29±0.00	0.23±0.00	0.43±0.00	0.33±0.00	0.31±0.00	0.21±0.00	0.41±0.00	0.34±0.00	0.45±0.08	0.31±0.00	0.34±0.00	0.28±0.00
FLH-FTL (ours)	0.09±0.00	0.08±0.00	0.13±0.00	0.12±0.00	0.15±0.00	0.13±0.00	0.15±0.00	0.13±0.00	0.15±0.00	0.13±0.00	0.22±0.00	0.15±0.00
Table 8: Results for UOLS problems using all hold-out data. Top: Classification Error. Bottom: Mean-squared error in estimating label marginal. Compared to the result in the main paper (Table 1), we observe that the performances of ROGD, UOGD, and ATLAS depend more on availability of holdout data than FLH-FTL. Notably, UOGD becomes competitive in the majority of the datasets when abundant holdout data are available.
G.3 Ablation over Number of Online Samples
Here we examine how different algorithms perform as the number of online samples varies. We introduce an additional baseline BBSE, which simply uses the label marginal estimate provided by black box shift estimator to reweight the predictions of classifiers. Figure 3 shows an interesting trend
Figure 3: Performances of online learning algorithms with different number of online samples. CIFAR-10 results with bernouli shift and limited holdout data. Solid line is the classification error (Error) and the dotted line is the marginal estimation mean squared error (MSE).
that as number of online samples increases, the simple baseline BBSE becomes more competitive and eventually outperforms UOGD, whereas our algorithm remains competitive.
G.4 Ablation over Types of Marginal Estimates
All the algorithms examined in this work use black box shift estimate (BBSE) [51] to obtain an unbiased estimate of the target label marginal. However, two alternative methods exist: Maximum Likelihood Label Shift (MLLS) [61] and Regularized Learning under Label Shift (RLLS) [2]. Table 9 presents additional results using these two estimates. The results show using the alternative estimates do not substantially change the performances of the algorithms considered in this work.
Datasets	Synthetic	CIFAR	Fashion
MARG EST.	BBSE	MLLS	RLLS	BBSE	MLLS	RLLS	BBSE	MLLS	RLLS
Bernouli	Base	8.6±0.2	8.6±0.2	8.6±0.2	16±0	16±0	16±0	15±0	15±0	15±0
OFC	6.4±0.6	6.4±0.6	6.4±0.6	12±1	12±1	12±1	7.9±0.1	7.9±0.1	7.9±0.1
FTH	6.5±0.6	6.5±0.7	6.5±0.7	11±0	11±1	11±0	8.5±0.3	8.0±0.0	8.6±0.2
FTFWH	6.6±0.5	6.7±0.5	6.6±0.5	11±1	11±1	11±1	8.2±0.6	7.9±0.2	8.3±0.6
ROGD	7.9±0.3	7.9±0.3	7.9±0.2	16±3	16±3	15±2	10±1	10±1	9.6±1.2
UOGD	8.1±0.6	8.0±1.0	8.0±1.0	14±0	13±0	14±0	11±2	10±1	11±1
ATLAS	8.0±1.0	7.9±0.5	8.0±1.0	13±0	13±0	13±0	12±2	11±1	12±1
FLH-FTL (ours)	5.4±0.7	5.4±0.8	5.4±0.7	10±0	10±1	10±0	7.7±0.4	7.3±0.3	7.6±0.3
Sinusoidal	Base	8.2±0.3	8.2±0.3	8.2±0.3	16±0	16±0	16±0	15±0	15±0	15±0
OFC	5.5±0.2	5.5±0.2	5.5±0.2	11±0	11±0	11±0	7.1±0.1	7.1±0.1	7.1±0.1
FTH	5.7±0.3	5.7±0.2	5.7±0.2	11±0	11±0	11±0	6.9±0.4	6.6±0.2	6.8±0.4
FTFWH	5.7±0.3	5.6±0.2	5.7±0.3	11±0	11±0	11±0	6.9±0.4	6.6±0.3	6.9±0.4
ROGD	7.2±0.6	7.2±0.6	7.2±0.6	13±0	13±0	13±0	8.2±0.7	8.9±0.6	8.2±0.3
UOGD	7.5±0.6	7.4±0.5	7.4±0.5	14±1	13±1	14±1	11±2	9.4±0.9	11±2
ATLAS	7.5±0.6	7.4±0.6	7.4±0.6	13±1	13±1	13±1	12±2	11±1	12±2
FLH-FTL (ours)	5.4±0.4	5.4±0.3	5.4±0.4	11±0	10±0	11±0	7.0±0.0	6.6±0.2	6.9±0.4
Table 9: Performances of online learning algorithms with different types of marginal estimates with low amount of holdout data.
G.5 Additional results and details on the SHL dataset
(b)
Figure 4: Additional results and details on the SHL datasets with real shift. (a) and (b): The accuracies and mean square errors in label marginal estimation on SHL dataset over 7,000 time steps with limited amount of holdout data. (c): Label marginals of the six classes of SHL dataset. Each time step here shows the marginals over 700 samples.
G.6 Reweighting Versus Retraining Linear Layer
Here we compare the efficacies of re-weighting (RW-FLH-FTL) and retraining (RT-FLH-FTL) given the marginal estimate provided by FLH-FTL; the latter retransmits the last linear layer on the loss of the holdout data re-weighted by the marginal estimate. Note that RW-FLH-FTL corresponds to FLH-FTL in the main text. We retrain RT-FLH-FTL for 50 epochs at each time step. To compare against the best possible retrained classifiers, we used all the holdout data for retraining. Table 10 shows that retraining is often worse and at best similar to re-weighting in performance, despite greater computational cost and need for holdout data.
Datasets	Synthetic	CIFAR
Shift	Mon	Sin	Ber	Squ	Mon	Sin	Ber	Squ
Base	8.7±0.1	8.2±0.3	8.6±0.3	8.5±0.2	17±0	16±0	17±0	17±0
OFC	6.8±0.1	5.5±0.2	6.7±0.2	6.7±0.3	14±0	11±0	13±0	13±1
FTFWH	7.0±0.0	5.5±0.3	6.8±0.2	6.8±0.3	13±0	10±0	12±0	13±0
UOGD	7.4±0.1	7.0±0.0	7.6±0.4	7.6±0.2	12±0	10±0	11±0	13±0
RW-FLH-FTL (ours)	6.3±0.2	5.3±0.2	5.4±0.3	5.5±0.2	12±0	10±0	11±0	12±0
RT-FLH-FTL (ours)	6.7±0.1	6.2±0.2	6.0±0.0	6.3±0.4	12±0	10±0	11±0	12±0
Table 10: Comparison of performances of re-weighting and retraining strategies with high amount of holdout data.