SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Definition 4.1 (Classification Loss). Let  $h: \mathbb{R}^n \to \mathbb{S}^{K-1}$  be a function that maps the input  $\pmb{x} \in \mathbb{R}^n$  to the space of  $K$ -class probability simplex, i.e.,  $\mathbb{S}^{K-1} \triangleq \{z \in \mathbb{R}^K : z_i \geq 0, \sum_i z_i = 1\}$ ; let  $\mathcal{X}$  be a collection of samples drawn from an arbitrary data distribution and  $f: \mathbb{R}^n \to [K]$  be the function that maps the input to the true label. The classification loss is defined as the average cross-entropy between the true label  $f(\pmb{x})$  and the predicted probability  $h(\pmb{x})$ , i.e.,
$$
\widehat {\ell} _ {\mathcal {X}} (h) \triangleq \frac {1}{| \mathcal {X} |} \sum_ {\boldsymbol {x} \in \mathcal {X}} \left[ - \sum_ {j = 1} ^ {K} \mathbb {1} _ {f (\boldsymbol {x}) = j} \cdot \log \left([ h (\boldsymbol {x}) ] _ {j}\right) \right]. \tag {10}
$$
Following Definition 4.1, we replace  $\widehat{\epsilon}_{\mathcal{D}_t}(h)$  and  $\widehat{\epsilon}_{\mathcal{D}_i}(h)$  in Eqn. 8 with  $\widehat{\ell}_{\mathcal{X}_t}(h)$  and  $\widehat{\ell}_{\mathcal{X}_i}(h)$ .
(2) Intra- and Cross-Domain Terms. We use the distillation loss below to optimize intradomain  $(\{\widehat{\epsilon}_{\mathcal{D}_i}(h,H_{t - 1})\}_{i = 1}^{t - 1})$  and cross-domain  $(\widehat{\epsilon}_{\mathcal{D}_t}(h,H_{t - 1}))$  model-based error terms in Eqn. 8.
Definition 4.2 (Distillation Loss). Let  $h, H_{t-1}: \mathbb{R}^n \to \mathbb{S}^{K-1}$  both be functions that map the input  $\pmb{x} \in \mathbb{R}^n$  to the space of  $K$ -class probability simplex as defined in Definition 4.1; let  $\mathcal{X}$  be a collection of samples drawn from an arbitrary data distribution. The distillation loss is defined as the average cross-entropy between the target probability  $H_{t-1}(\pmb{x})$  and the predicted probability  $h(\pmb{x})$ , i.e.,
$$
\widehat {\ell} _ {\mathcal {X}} (h, H _ {t - 1}) \triangleq \frac {1}{| \mathcal {X} |} \sum_ {\boldsymbol {x} \in \mathcal {X}} \left[ - \sum_ {j = 1} ^ {K} \left[ H _ {t - 1} (\boldsymbol {x}) \right] _ {j} \cdot \log \left(\left[ h (\boldsymbol {x}) \right] _ {j}\right) \right]. \tag {11}
$$
Accordingly, we replace  $\widehat{\epsilon}_{\mathcal{D}_i}(h,H_{t - 1})$  with  $\widehat{\ell}_{\mathcal{X}_i}(h,H_{t - 1})$  and  $\widehat{\epsilon}_{\mathcal{D}_t}(h,H_{t - 1})$  with  $\widehat{\ell}_{\mathcal{X}_t}(h,H_{t - 1})$ .
(3) Constant Term. The error term  $\sum_{i=1}^{t-1} (\alpha_i + \beta_i) \epsilon_{\mathcal{D}_i}(H_{t-1})$  in Eqn. 8 is a constant and contains no trainable parameters (since  $H_{t-1}$  is a fixed history model); therefore it does not need a loss term.
(4) Divergence Term. In Eqn. 8,  $\sum_{i=1}^{t-1} \beta_i d_{\mathcal{H}\Delta \mathcal{H}}(\mathcal{D}_i, \mathcal{D}_t)$  measures the weighted average of the dissimilarity between domain  $i$ 's and domain  $t$ 's data distributions. Inspired by existing adversarial domain adaptation methods [57, 26, 109, 17, 102, 101, 94], we can further tighten this divergence term by considering the embedding distributions instead of data distributions using an learnable encoder. Specifically, given an encoder  $e: \mathbb{R}^n \to \mathbb{R}^m$  and a family of domain discriminators (classifiers)  $\mathcal{H}_d$ , we have the empirical estimate of the divergence term as follows:
$$
\sum_ {i = 1} ^ {t - 1} \beta_ {i} \widehat {d} _ {\mathcal {H} \Delta \mathcal {H}} (e (\mathcal {U} _ {i}), e (\mathcal {U} _ {t})) = 2 \sum_ {i = 1} ^ {t - 1} \beta_ {i} - 2 \min _ {d \in \mathcal {H} _ {d}} \sum_ {i = 1} ^ {t - 1} \beta_ {i} \left[ \frac {1}{| \mathcal {U} _ {i} |} \sum_ {\boldsymbol {x} \in \mathcal {U} _ {i}} \mathbb {1} _ {\Delta_ {i} (\boldsymbol {x}) \geq 0} + \frac {1}{| \mathcal {U} _ {t} |} \sum_ {\boldsymbol {x} \in \mathcal {U} _ {t}} \mathbb {1} _ {\Delta_ {i} (\boldsymbol {x}) <   0} \right],
$$
where  $\mathcal{U}_i$  (and  $\mathcal{U}_t$ ) is a set of samples drawn from domain  $\mathcal{D}_i$  (and  $\mathcal{D}_t$ ),  $d: \mathbb{R}^m \to \mathbb{S}^{t-1}$  is a domain classifier, and  $\Delta_i(\pmb{x}) = [d(e(\pmb{x}))]_i - [d(e(\pmb{x}))]_t$  is the difference between the probability of  $\pmb{x}$  belonging to domain  $i$  and domain  $t$ . Replacing the indicator function with the differentiable cross-entropy loss,  $\sum_{i=1}^{t-1} \beta_i \widehat{d}_{\mathcal{H}\Delta\mathcal{H}}(e(\mathcal{U}_i), e(\mathcal{U}_t))$  above then becomes
$$
2 \sum_ {i = 1} ^ {t - 1} \beta_ {i} - 2 \min  _ {d \in \mathcal {H} _ {d}} \sum_ {i = 1} ^ {t - 1} \beta_ {i} \left[ \frac {1}{\tilde {N} _ {i}} \sum_ {\boldsymbol {x} \in \mathcal {X} _ {i}} [ - \log \left(\left[ d (e (\boldsymbol {x})) \right] _ {i}\right) ] + \frac {1}{N _ {t}} \sum_ {\boldsymbol {x} \in S _ {t}} [ - \log \left(\left[ d (e (\boldsymbol {x})) \right] _ {t}\right) ] \right]. \tag {12}
$$
Algorithm 1 Unified Domain Incremental Learning (UDIL) for Domain  $t$  Training
Require: history model  $H_{t - 1} = P_{t - 1}\circ E_{t - 1}$  , current model  $h_\theta = p\circ e$  discriminator model  $d_{\phi}$    
Require: dataset from the current domain  $\mathcal{S}_t$  , memory bank  $\mathcal{M} = \{M_i\}_{i = 1}^{t - 1}$    
Require: training steps  $S$  , batch size  $B$  , learning rate  $\eta$  .   
Require: domain alignment strength coefficient  $\lambda_{d}$  , hyperparameter for generalization effect  $C$    
1:  $h_\theta \gets H_{t - 1}$  ▷ Initialization of the current model   
2:  $\Omega \triangleq \{\alpha_i,\beta_i,\gamma_i\} \leftarrow \{1 / 3,1 / 3,1 / 3\}$  , for  $\forall i\in [t - 1]$  ▷ Initialization of the replay coefficient  $\Omega$    
3: for  $s = 1,\dots ,S$  do   
4:  $B_{t}\sim S_{t};B_{i}\sim M_{i},\forall i\in [t - 1]$  ▷ Sample a mini-batch of data from all domains   
5:  $\phi \gets \phi -\eta \cdot \lambda_d\cdot \nabla_\phi V_d(d,e,\stackrel {\circ}{\Omega})$  ▷ Discriminator training with Eqn. 16   
6:  $\Omega \leftarrow \Omega -\eta \cdot \nabla_{\Omega}V_{0 - 1}(\stackrel {\circ}{h},\Omega)$  ▷ Find a tighter bound with Eqn. 15   
7:  $\theta \leftarrow \theta -\eta \cdot \nabla_{\theta}(V_l(h_\theta ,\stackrel {\circ}{\Omega}) - \lambda_dV_d(d,e,\stackrel {\circ}{\Omega}))$  ▷ Model training with Eqn. 14 and Eqn. 16   
8: end for   
9:  $H_{t}\gets h$    
10:  $\mathcal{M}\gets$  BalancedSampling  $(\mathcal{M},\mathcal{S}_t)$    
11: return  $H_{t}$  ▷ For training on domain  $t + 1$
4.2 Putting Everything Together: UDIL Training Algorithm
Objective Function. With these differentiable loss terms above, we can derive an algorithm that learns the optimal model by minimizing the tightest bound in Eqn. 8. As mentioned above, to achieve a tighter  $d_{\mathcal{H}\Delta \mathcal{H}}$ , we decompose the hypothesis as  $h = p \circ e$ , where  $e: \mathbb{R}^n \to \mathbb{R}^m$  and  $p: \mathbb{R}^m \to \mathbb{S}^{K-1}$  are the encoder and predictor, respectively. To find and to minimize the tightest bound in Theorem 3.4, we treat  $\Omega = \{\alpha_i, \beta_i, \gamma_i\}_{i=1}^{t-1}$  as learnable parameters and seek to optimize the following objective (we denote as  $\stackrel{\circ}{x} = \mathrm{sg}(x)$  the 'copy-weights-and-stop-gradients' operation):
$$
\min  _ {\left\{\Omega , h = p o e \right\}} \max  _ {d} \quad \begin{array}{l l} V _ {l} (h, \overset {\circ} {\Omega}) + V _ {0 - 1} (\overset {\circ} {h}, \Omega) - \lambda_ {d} V _ {d} (d, e, \overset {\circ} {\Omega}) \\ \text {s . t .} \quad \alpha_ {i} + \beta_ {i} + \gamma_ {i} = 1, & \forall i \in \{1, 2, \dots , t - 1 \} \\ \alpha_ {i}, \beta_ {i}, \gamma_ {i} \geq 0, & \forall i \in \{1, 2, \dots , t - 1 \} \end{array} \tag {13}
$$
Details of  $V_{l}$ ,  $V_{0-1}$ , and  $V_{d}$ .  $V_{l}$  is the loss for learning the model  $h$ , where the terms  $\widehat{\ell}(\cdot)$  are differentiable cross-entropy losses as defined in Eqn. 10 and Eqn. 11:
$$
V _ {l} (h, \overset {\circ} {\Omega}) = \sum_ {i = 1} ^ {t - 1} \left[ \stackrel {\circ} {\gamma} _ {i} \widehat {\ell} _ {\mathcal {X} _ {i}} (h) + \stackrel {\circ} {\alpha} _ {i} \widehat {\ell} _ {\mathcal {X} _ {i}} (h, H _ {t - 1}) \right] + \widehat {\ell} _ {\mathcal {S} _ {t}} (h) + \left(\sum_ {i = 1} ^ {t - 1} \stackrel {\circ} {\beta} _ {i}\right) \widehat {\ell} _ {\mathcal {S} _ {t}} (h, H _ {t - 1}). \tag {14}
$$
$V_{0-1}$  is the loss for finding the optimal coefficient set  $\Omega$ . Its loss terms use Definition 3.1 and Eqn. 12 to estimate ERM terms and  $\mathcal{H} \Delta \mathcal{H}$ -divergence, respectively:
$$
\begin{array}{l} V _ {0 - 1} (\stackrel {\circ} {h}, \Omega) = \sum_ {i = 1} ^ {t - 1} \left[ \gamma_ {i} \widehat {\epsilon} _ {\mathcal {D} _ {i}} (\stackrel {\circ} {h}) + \alpha_ {i} \widehat {\epsilon} _ {\mathcal {D} _ {i}} (\stackrel {\circ} {h}, H _ {t - 1}) \right] + \left(\sum_ {i = 1} ^ {t - 1} \beta_ {i}\right) \widehat {\epsilon} _ {\mathcal {D} _ {t}} (\stackrel {\circ} {h}, H _ {t - 1}) \\ + \frac {1}{2} \sum_ {i = 1} ^ {t - 1} \beta_ {i} \widehat {d} _ {\mathcal {H} \Delta \mathcal {H}} \left(\stackrel {\circ} {e} (\mathcal {X} _ {i}), \stackrel {\circ} {e} (\mathcal {S} _ {t})\right) + \sum_ {i = 1} ^ {t - 1} (\alpha_ {i} + \beta_ {i}) \widehat {\epsilon} _ {\mathcal {D} _ {i}} (H _ {t - 1}) \\ + C \cdot \sqrt {\left(\frac {\left(1 + \sum_ {i = 1} ^ {t - 1} \beta_ {i}\right) ^ {2}}{\tilde {N} _ {t}} + \sum_ {i = 1} ^ {t - 1} \frac {\left(\gamma_ {i} + \alpha_ {i}\right) ^ {2}}{\tilde {N} _ {i}}\right)}. \tag {15} \\ \end{array}
$$
In Eqn. 15,  $\widehat{\epsilon}(\cdot)$  uses discrete 0-1 loss, which is different from Eqn. 14, and a hyper-parameter  $C = \sqrt{8d\log(2eN / d) + 8\log(2 / \delta)}$  is introduced to model the combined influence of  $\mathcal{H}$ 's VC-dimension and  $\delta$ .
$V_{d}$  follows Eqn. 12 to minimize the divergence between different domains' embedding distributions (i.e., aligning domains) by the minimax game between  $e$  and  $d$  with the value function:
$$
V _ {d} (d, e, \overset {\circ} {\Omega}) = \left(\sum_ {i = 1} ^ {t - 1} \overset {\circ} {\beta} _ {i}\right) \frac {1}{N _ {i}} \sum_ {\boldsymbol {x} \in \mathcal {S} _ {t}} \left[ - \log \left([ d (e (\boldsymbol {x})) ] _ {t}\right) \right] + \sum_ {i = 1} ^ {t - 1} \overset {\circ} {\underset {\tilde {N} _ {i}} {\beta}} _ {i} \sum_ {\boldsymbol {x} \in \tilde {\mathcal {X}} _ {i}} \left[ - \log \left([ d (e (\boldsymbol {x})) ] _ {i}\right) \right]. \tag {16}
$$
(b) Labels
(c) Domains
(d) Joint (91.083%, Oracle)
(a) Example: 3D-Balls
(e) ER (82.255%)
(f)  $\mathrm{DER} + +$  (79.322%)
Figure 1: Results on HD-Balls. In (a-b), data is colored according to labels; in (c-h), data is colored according to domain ID. All data is plotted after PCA [6]. (a) Simplified HD-Balls dataset with 3 domains in the 3D space (for visualization purposes only). (b-c) Embeddings of HD-Balls's raw data colored by labels and domain ID. (d-h) Accuracy and embeddings learned by Joint (oracle), UDIL, and three best baselines (more in Appendix C.5). Joint, as the oracle, naturally aligns different domains, and UDIL outperforms all baselines in terms of embedding alignment and accuracy.
(g) CLS-ER (85.844%)
(h) UDIL (86.872%, Ours)
Here in Eqn. 16, if an optimal  $d^{*}$  and a fixed  $\Omega$  is given, maximizing  $V_{d}(d^{*},e,\Omega)$  with respect to the encoder  $e$  is equivalent to minimizing the weighted sum of the divergence  $\sum_{i = 1}^{t - 1}\beta_id_{\mathcal{H}\Delta \mathcal{H}}(e(\mathcal{D}_i),e(\mathcal{D}_t))$ . This result indicates that the divergence between two domains' embedding distributions can be actually minimized. Intuitively this minimax game learns an encoder  $e$  that aligns the embedding distributions of different domains so that their domain IDs can not be predicted (distinguished) by a powerful discriminator given an embedding  $e(\pmb {x})$ . Algorithm 1 below outlines how UDIL minimizes the tightest bound. Please refer to Appendix C for more implementation details, including a model diagram in Fig. 2.
In this section, we compare UDIL with existing methods on both synthetic and real-world datasets.
5.1 Baselines and Implementation Details
We compare UDIL with the state-of-the-art continual learning methods that are either specifically designed for domain incremental learning or can be easily adapted to the domain incremental learning setting. For fair comparison, we do not consider methods that leverage large-scale pre-training or prompt-tuning [99, 98, 53, 88]. Exemplar-free baselines include online Elastic Weight Consolidation (oEWC) [81], Synaptic Intelligence (SI) [105], and Learning without Forgetting (LwF) [52]. Memory-based domain incremental learning baselines include Gradient Episodic Memory (GEM) [58], Averaged Gradient Episodic Memory (A-Geom) [10], Experience Replay (ER) [75], Dark Experience Replay  $(\mathbf{DER} + +)$  [8], and two recent methods, Complementary Learning System based Experience Replay (CLS-ER) [4] and Error Sensitivity Modulation based Experience Replay (ESM-ER) [80] (see Appendix C.5 for more detailed introduction to the baseline methods above). In addition, we implement the fine-tuning (Fine-tune) [52] and joint-training (Joint) as the performance lower bound and upper bound (Oracle).
We train all models using three different random seeds and report the mean and standard deviation. All methods are implemented with PyTorch [70], based on the mammoth code base [7, 8], and run on a single NVIDIA RTX A5000 GPU. For fair comparison, within the same dataset, all methods adopt the same neural network architecture, and the memory sampling strategy is set to random
Table 2: Performances (\%) on HD-Balls, P-MNIST, and R-MNIST. We use two metrics, Average Accuracy and Forgetting, to evaluate the methods' effectiveness. “↑” and “↓” mean higher and lower numbers are better, respectively. We use boldface and underlining to denote the best and the second-best performance, respectively. We use “-” to denote “not appliable”.
Method	Buffer	HD-Balls	P-MNIST	R-MNIST
Avg. Acc (↑)	Forgetting (↓)	Avg. Acc (↑)	Forgetting (↓)	Avg. Acc (↑)	Forgetting (↓)
Fine-tune	-	52.319±0.024	43.520±0.079	70.102±2.945	27.522±3.042	47.803±1.703	52.281±1.797
oEWC [81]	-	54.131±0.193	39.743±1.388	78.476±1.223	18.068±1.321	48.203±0.827	51.181±0.867
SI [105]	-	52.303±0.037	43.175±0.041	79.045±1.357	17.409±1.446	48.251±1.381	51.053±1.507
LwF [52]	-	51.523±0.065	25.155±0.264	73.545±2.646	24.556±2.789	54.709±0.515	45.473±0.565
GEM [58]	69.747±0.656	13.591±0.779	89.097±0.149	6.975±0.167	76.619±0.581	21.289±0.579
A-GEM [10]	62.777±0.295	12.878±1.588	87.560±0.087	8.577±0.053	59.654±0.122	39.196±0.171
ER [75]	82.255±1.552	9.524±1.655	88.339±0.044	7.180±0.029	76.794±0.696	20.696±0.744
DER++ [8]	400	79.332±1.347	13.762±1.514	92.950±0.361	3.378±0.245	84.258±0.544	13.692±0.560
CLS-ER [4]	85.844±0.165	5.297±0.281	91.598±0.117	3.795±0.144	81.771±0.354	15.455±0.356
ESM-ER [80]	71.995±3.833	13.245±5.397	89.829±0.698	6.888±0.738	82.192±0.164	16.195±0.150
UDIL (Ours)	86.872±0.195	3.428±0.359	92.666±0.108	2.853±0.107	86.635±0.686	8.506±1.181
Joint (Oracle)	∞	91.083±0.332	-	96.368±0.042	-	97.150±0.036	-
balanced sampling (see Appendix C.2 and Appendix C.6 for more implementation details on training). We evaluate all methods with standard continual learning metrics including 'average accuracy', 'forgetting', and 'forward transfer' (see Appendix C.4 for detailed definitions).
5.2 Toy Dataset: High-Dimensional Balls
To gain insight into UDIL, we start with a toy dataset, high dimensional balls on a sphere (referred to as HD-Balls below), for domain incremental learning. HD-Balls includes 20 domains, each containing 2,000 data points sampled from a Gaussian distribution  $\mathcal{N}(\pmb{\mu}, 0.2^2\pmb{I})$ . The mean  $\pmb{\mu}$  is randomly sampled from a 100-dimensional unit sphere, i.e.,  $\{\pmb{\mu} \in \mathbb{R}^{100} : \| \pmb{\mu}\|_2 = 1\}$ ; the covariance matrix  $\Sigma$  is fixed. In HD-Balls, each domain represents a binary classification task, where the decision boundary is the hyperplane that passes the center  $\pmb{\mu}$  and is tangent to the unit sphere. Fig. 1(a-c) shows some visualization on HD-Balls.
Column 3 and 4 of Table 2 compare the performance of our UDIL with different baselines. We can see that UDIL achieves the highest final average accuracy and the lowest forgetting. Fig. 1(d-h) shows the embedding distributions (i.e.,  $e(\boldsymbol{x})$ ) for different methods. We can see better embedding alignment across domains generally leads to better performance. Specifically, Joint, as the oracle, naturally aligns different domains' embedding distributions and achieves an accuracy upper bound of  $91.083\%$ . Similarly, our UDIL can adaptively adjust the coefficients of different loss terms, including Eqn. 12, successfully align different domains, and thereby outperform all baselines.
5.3 Permutation MNIST
We further evaluate our method on the Permutation MNIST (P-MNIST) dataset [50]. P-MNIST includes 20 sequential domains, with each domain constructed by applying a fixed random permutation to the pixels in the images. Column 5 and 6 of Table 2 show the results of different methods. Our UDIL achieves the second best  $(92.666\%)$  final average accuracy, which is only  $0.284\%$  lower than the best baseline  $\mathrm{DER} + +$ . We believe this is because (i) there is not much space for improvement as the gap between joint-training (oracle) and most methods are small; (ii) under the permutation, different domains' data distributions are too distinct from each other, lacking the meaningful relations among the domains, and therefore weakens the effect of embedding alignment in our method. Nevertheless, UDIL still achieves best performance in terms of forgetting  $(2.853\%)$ . This is mainly because our unified UDIL framework (i) is directly derived from the total loss of all domains, and (ii) uses adaptive coefficients to achieve a more balanced trade-off between learning the current domain and avoiding forgetting previous domains.
Table 3: Performances (\%) evaluated on Seq-CORe50. We use three metrics, Average Accuracy, Forgetting, and Forward Transfer, to evaluate the methods' effectiveness. “↑” and “↓” mean higher and lower numbers are better, respectively. We use boldface and underlining to denote the best and the second-best performance, respectively. We use “-” to denote “not appliable” and “★” to denote out-of-memory (OOM) error when running the experiments.
Method	Buffer	D1:3	D4:6	D7:9	D10:11	Avg. Acc (↑)	Overall Forgetting (↓)	Fwd. Transfer (↑)
Avg. Acc (↑)
Fine-tune	-	73.707±13.144	34.551±1.254	29.406±2.579	28.689±3.144	31.832±1.034	73.296±1.399	15.153±0.255
oEWC [81]	-	74.567±13.360	35.915±0.260	30.174±3.195	28.291±2.522	30.813±1.154	74.563±0.937	15.041±0.249
SI [105]	-	74.661±14.162	34.345±1.001	30.127±2.971	28.839±3.631	32.469±1.315	73.144±1.588	14.837±1.005
LwF [52]	-	80.383±10.190	28.357±1.143	31.386±0.787	28.711±2.981	31.692±0.768	72.990±1.350	15.356±0.750
GEM [58]	79.852±6.864	38.961±1.718	39.258±2.614	36.859±0.842	37.701±0.273	22.724±1.554	19.030±0.936
A-GEM [10]	80.348±9.394	41.472±3.394	43.213±1.542	39.181±3.999	43.181±2.025	33.775±3.003	19.033±0.792
ER [75]	90.838±2.177	79.343±2.699	68.151±0.226	65.034±1.571	66.605±0.214	32.750±0.455	21.735±0.802
DER++ [8]	500	92.444±1.764	88.652±1.854	80.391±0.107	78.038±0.591	78.629±0.753	21.910±1.094	22.488±1.049
CLS-ER [4]	89.834±1.323	78.909±1.724	70.591±0.322	★	★	★	★
ESM-ER [80]	84.905±6.471	51.905±3.257	53.815±1.770	50.178±2.574	52.751±1.296	25.444±0.580	21.435±1.018
UDIL (Ours)	98.152±1.665	89.814±2.302	83.052±0.151	81.547±0.269	82.103±0.279	19.589±0.303	31.215±0.831
Joint (Oracle)	∞	-	-	-	-	99.137±0.049	-	-
5.4 Rotating MNIST
We also evaluate our method on the Rotating MNIST dataset ( $R$ -MNIST) containing 20 sequential domains. Different from  $P$ -MNIST where shift from domain  $t$  to domain  $t + 1$  is abrupt,  $R$ -MNIST's domain shift is gradual. Specifically, domain  $t$ 's images are rotated by an angle randomly sampled from the range  $[9^{\circ} \cdot (t - 1), 9^{\circ} \cdot t)$ . Column 7 and 8 of Table 2 show that our UDIL achieves the highest average accuracy (86.635%) and the lowest forgetting (8.506%) simultaneously, significantly improving on the best baseline DER++. (average accuracy of 84.258% and forgetting of 13.692%). Interestingly, such improvement is achieved when our UDIL's  $\beta_{i}$  is high, which further verifies that UDIL indeed leverages the similarities shared across different domains so that the generalization error is reduced.
5.5 Sequential CORe50
CORe50 [55, 56] is a real-world continual object recognition dataset that contains 50 domestic objects collected from 11 domains (120,000 images in total). Prior work has used CORe50 for settings such as domain generalization (e.g., train a model on only 8 domains and test it on 3 domains), which is different from our domain-incremental learning setting. To focus the evaluation on alleviating catastrophic forgetting, we retain  $20\%$  of the data as the test set and continually train the model on these 11 domains; we therefore call this dataset variant Seq-CORE50. Table 3 shows that our UDIL outperforms all baselines in every aspect on Seq-CORE50. Besides the average accuracy over all domains, we also report average accuracy over different domain intervals (e.g.,  $\mathcal{D}_{1:3}$  denotes average accuracy from domain 1 to domain 3) to show how different model's performance drops over time. The results show that our UDIL consistently achieves the highest average accuracy until the end. It is also worth noting that UDIL also achieves the best performance on another two metrics, i.e., forgetting and forward transfer.
In this paper, we propose a principled framework, UDIL, for domain incremental learning with memory to unify various existing methods. Our theoretical analysis shows that different existing methods are equivalent to minimizing the same error bound with different fixed coefficients. With this unification, our UDIL allows adaptive coefficients during training, thereby always achieving the tightest bound and improving the performance. Empirical results show that our UDIL outperforms the state-of-the-art domain incremental learning methods on both synthetic and real-world datasets. One limitation of this work is the implicit i.i.d. exemplar assumption, which may not hold if memory is selected using specific strategies. Addressing this limitation can lead to a more powerful unified framework and algorithms, which would be interesting future work.
The authors thank the reviewers/AC for the constructive comments to improve the paper. HS and HW are partially supported by NSF Grant IIS-2127918. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of the sponsors.
[104] J. Yoon, E. Yang, J. Lee, and S. J. Hwang. Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547, 2017.
[105] F. Zenke, B. Poole, and S. Ganguli. Continual learning through synaptic intelligence. In International conference on machine learning, pages 3987-3995. PMLR, 2017.
[106] M.-L. Zhang and Z.-H. Zhou. Ml-knn: A lazy learning approach to multi-label learning. Pattern recognition, 40(7):2038-2048, 2007.
[107] M.-L. Zhang and Z.-H. Zhou. A review on multi-label learning algorithms. IEEE transactions on knowledge and data engineering, 26(8):1819-1837, 2013.
[108] Y. Zhang, T. Liu, M. Long, and M. Jordan. Bridging theory and algorithm for domain adaptation. In International conference on machine learning, pages 7404-7413. PMLR, 2019.
[109] M. Zhao, S. Yue, D. Katabi, T. S. Jaakkola, and M. T. Bianchi. Learning sleep stages from radio signals: A conditional adversarial architecture. In International Conference on Machine Learning, pages 4100-4109. PMLR, 2017.
[110] J. Zhou, C. You, X. Li, K. Liu, S. Liu, Q. Qu, and Z. Zhu. Are all losses created equal: A neural collapse perspective. Advances in Neural Information Processing Systems, 35:31697-31710, 2022.
[111] Z. Zhu, T. Ding, J. Zhou, X. Li, C. You, J. Sulam, and Q. Qu. A geometric analysis of neural collapse with unconstrained features. Advances in Neural Information Processing Systems, 34:29820-29834, 2021.