SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Haizhou Shi
Piscataway, NJ 08854
haizhou.shi@rutgers.edu
hw488@cs.rutgers.edu
Domain incremental learning aims to adapt to a sequence of domains with access to only a small subset of data (i.e., memory) from previous domains. Various methods have been proposed for this problem, but it is still unclear how they are related and when practitioners should choose one method over another. In response, we propose a unified framework, dubbed Unified Domain Incremental Learning (UDIL), for domain incremental learning with memory. Our UDIL unifies various existing methods, and our theoretical analysis shows that UDIL always achieves a tighter generalization error bound compared to these methods. The key insight is that different existing methods correspond to our bound with different fixed coefficients; based on insights from this unification, our UDIL allows adaptive coefficients during training, thereby always achieving the tightest bound. Empirical results show that our UDIL outperforms the state-of-the-art domain incremental learning methods on both synthetic and real-world datasets. Code will be available at https://github.com/Wang-ML-Lab/unified-continual-learning.
Despite recent success of large-scale machine learning models [35, 48, 36, 28, 92, 22, 33], continually learning from evolving environments remains a longstanding challenge. Unlike the conventional machine learning paradigms where learning is performed on a static dataset, domain incremental learning, i.e., continual learning with evolving domains, hopes to accommodate the model to the dynamically changing data distributions, while retaining the knowledge learned from previous domains [90, 60, 41, 97, 27]. Naive methods, such as continually finetuning the model on new-coming domains, will suffer a substantial performance drop on the previous domains; this is referred to as "catastrophic forgetting" [46, 58, 81, 105, 52]. In general, domain incremental learning algorithms aim to minimize the total risk of all domains, i.e.,
$$
\mathcal {L} ^ {*} (\theta) = \mathcal {L} _ {t} (\theta) + \mathcal {L} _ {1: t - 1} (\theta) = \mathbb {E} _ {(x, y) \sim \mathcal {D} _ {t}} [ \ell (y, h _ {\theta} (x) ] + \sum_ {i = 1} ^ {t - 1} \mathbb {E} _ {(x, y) \sim \mathcal {D} _ {i}} [ \ell (y, h _ {\theta} (x) ], \tag {1}
$$
where  $\mathcal{L}_t$  calculates model  $h_\theta$ 's expected prediction error  $\ell$  over the current domain's data distribution  $\mathcal{D}_t$ .  $\mathcal{L}_{1:t-1}$  is the total error evaluated on the past  $t-1$  domains' data distributions, i.e.,  $\{\mathcal{D}_i\}_{i=1}^{t-1}$ .
The main challenge of domain incremental learning comes from the practical memory constraint that no (or only very limited) access to the past domains' data is allowed [52, 46, 105, 74]. Under such a constraint, it is difficult, if not impossible, to accurately estimate and optimize the past error  $\mathcal{L}_{1:t-1}$ . Therefore the main focus of recent domain incremental learning methods has been to develop effective surrogate learning objectives for  $\mathcal{L}_{1:t-1}$ . Among these methods [46, 81, 2, 105, 58, 10, 75,
77, 21, 25, 65, 66, 9, 72, 82, 95, 53], replay-based methods, which replay a small set of old exemplars during training [90, 75, 8, 4, 80, 11], has consistently shown promise and is therefore commonly used in practice.
One typical example is ER [75], which stores a set of exemplars  $\mathcal{M}$  and uses a replay loss  $\mathcal{L}_{\mathrm{replay}}$  as the surrogate of  $\mathcal{L}_{1:t-1}$ . In addition, a fixed, predetermined coefficient  $\beta$  is used to balance current domain learning and past sample replay. Specifically,
$$
\widetilde {\mathcal {L}} (\theta) = \mathcal {L} _ {t} (\theta) + \beta \cdot \mathcal {L} _ {\text {r e p l a y}} (\theta) = \mathcal {L} _ {t} (\theta) + \beta \cdot \mathbb {E} _ {\left(x ^ {\prime}, y ^ {\prime}\right) \sim \mathcal {M}} [ \ell \left(y ^ {\prime}, h _ {\theta} \left(x ^ {\prime}\right) \right]. \tag {2}
$$
While such methods are popular in practice, there is still a gap between the surrogate loss  $(\beta \mathcal{L}_{\mathrm{replay}})$  and the true objective  $(\mathcal{L}_{1:t - 1})$ , rendering them lacking in theoretical support and therefore calling into question their reliability. Besides, different methods use different schemes of setting  $\beta$  [75, 8, 4, 80], and it is unclear how they are related and when practitioners should choose one method over another.
To address these challenges, we develop a unified generalization error bound and theoretically show that different existing methods are actually minimizing the same error bound with different fixed coefficients (more details in Table 1 later). Based on such insights, we then develop an algorithm that allows adaptive coefficients during training, thereby always achieving the tightest bound and improving the performance. Our contributions are as follows:
- We propose a unified framework, dubbed Unified Domain Incremental Learning (UDIL), for domain incremental learning with memory to unify various existing methods.
- Our theoretical analysis shows that different existing methods are equivalent to minimizing the same error bound with different fixed coefficients. Based on insights from this unification, our UDIL allows adaptive coefficients during training, thereby always achieving the tightest bound and improving the performance.
- Empirical results show that our UDIL outperforms the state-of-the-art domain incremental learning methods on both synthetic and real-world datasets.
Continual Learning. Prior work on continual learning can be roughly categorized into three scenarios [90, 15]: (i) task-incremental learning, where task indices are available during both training and testing [52, 46, 90], (ii) class-incremental learning, where new classes are incrementally included for the classifier [74, 100, 30, 45, 44], and (iii) domain-incremental learning, where the data distribution's incremental shift is explicitly modeled [60, 41, 97, 27]. Regardless of scenarios, the main challenge of continual learning is to alleviate catastrophic forgetting with only limited access to the previous data; therefore methods in one scenario can often be easily adapted for another. Many methods have been proposed to tackle this challenge, including functional and parameter regularization [52, 46, 81, 2], constraining the optimization process [77, 21, 58, 10], developing incrementally updated components [104, 38, 53], designing modularized model architectures [73, 95], improving representation learning with additional inductive biases [9, 66, 65, 25], and Bayesian approaches [24, 63, 49, 1]. Among them, replaying a small set of old exemplars, i.e., memory, during training has shown great promise as it is easy to deploy, applicable in all three scenarios, and, most importantly, achieves impressive performance [90, 75, 8, 4, 80, 11]. Therefore in this paper, we focus on domain incremental learning with memory, aiming to provide a principled theoretical framework to unify these existing methods.
Domain Adaptation and Domain Incremental Learning. Loosely related to our work are domain adaptation (DA) methods, which adapt a model trained on labeled source domains to unlabeled target domains [68, 67, 57, 78, 79, 108, 71, 16, 17, 64, 94, 51]. Much prior work on DA focuses on matching the distribution of the source and target domains by directly matching the statistical attributions [67, 89, 87, 71, 64] or adversarial training [108, 57, 26, 109, 17, 102, 101, 54, 94]. Compared to DA's popularity, domain incremental learning (DIL) has received limited attention in the past. However, it is now gaining significant traction in the research community [90, 60, 41, 97, 27]. These studies predominantly focus on the practical applications of DIL, such as semantic segmentation [27], object detection for autonomous driving [60], and learning continually in an open-world setting [18]. Inspired by the theoretical foundation of adversarial DA [5, 57], we propose, to the best of our knowledge, the first unified upper bound for DIL. Most related to our work are previous DA methods that flexibly align different domains according to their associated given
or inferred domain index [94, 101], domain graph [102], and domain taxonomy [54]. The main difference between DA and DIL is that the former focuses on improving the accuracy of the target domains, while the latter focuses on the total error of all domains, with additional measures taken to alleviate forgetting on the previous domains. More importantly, DA methods typically require access to target domain data to match the distributions, and therefore are not directly applicable to DIL.
3 Theory: Unifying Domain Incremental Learning
In this section, we formalize the problem of domain incremental learning, provide the generalization bound of naively applying empirical risk minimization (ERM) on the memory bank, derive two error bounds (i.e., intra-domain and cross-domain error bounds) more suited for domain incremental learning, and then unify these three bounds to provide our final adaptive error bound. We then develop an algorithm inspired by this bound in Sec. 4. All proofs of lemmas, theorems, and corollaries can be found in Appendix A.
Problem Setting and Notation. We consider the problem of domain incremental learning with  $T$  domains arriving one by one. Each domain  $i$  contains  $N_{i}$  data points  $\mathcal{S}_i = \{(\pmb{x}_j^{(i)},y_j^{(i)})\}_{j = 1}^{N_i}$ , where  $(\pmb{x}_j^{(i)},y_j^{(i)})$  is sampled from domain  $i$ 's data distribution  $\mathcal{D}_i$ . Assume that when domain  $t\in [T]\triangleq \{1,2,\ldots ,T\}$  arrives at time  $t$ , one has access to (1) the current domain  $t$ 's data  $\mathcal{S}_t$ , (2) a memory bank  $\mathcal{M} = \{M_i\}_{i = 1}^{t - 1}$ , where  $M_{i} = \{(\widetilde{\pmb{x}}_{j}^{(i)},\widetilde{y}_{j}^{(i)})\}_{j = 1}^{\widetilde{N}_{i}}$  is a small subset  $(\widetilde{N}_i\ll N_i)$  randomly sampled from  $\mathcal{S}_i$ , and (3) the history model  $H_{t - 1}$  after training on the previous  $t - 1$  domains. For convenience we use shorthand notation  $\mathcal{X}_i\triangleq \{\pmb{x}_j^{(i)}\}_{j = 1}^{N_i}$  and  $\widetilde{\chi}_i\triangleq \{\widetilde{\pmb{x}}_j^{(i)}\}_{j = 1}^{\widetilde{N}_i}$ . The goal is to learn the optimal model (hypothesis)  $h^*$  that minimizes the prediction error over all  $t$  domains after each domain  $t$  arrives. Formally,
$$
h ^ {*} = \arg \min  _ {h} \sum_ {i = 1} ^ {t} \epsilon_ {\mathcal {D} _ {i}} (h), \quad \epsilon_ {\mathcal {D} _ {i}} (h) \triangleq \mathbb {E} _ {\boldsymbol {x} \sim \mathcal {D} _ {i}} [ h (\boldsymbol {x}) \neq f _ {i} (\boldsymbol {x}) ], \tag {3}
$$
where for domain  $i$ , we assume the labels  $y \in \mathcal{V} = \{0,1\}$  are produced by an unknown deterministic function  $y = f_{i}(\pmb{x})$  and  $\epsilon_{\mathcal{D}_i}(h)$  denotes the expected error of domain  $i$ .
3.1 Naive Generalization Bound Based on ERM
Definition 3.1 (Domain-Specific Empirical Risks). When domain  $t$  arrives, model  $h$ 's empirical risk  $\widehat{\epsilon}_{\mathcal{D}_i}(h)$  for each domain  $i$  (where  $i \leq t$ ) is computed on the available data at time  $t$ , i.e.,
$$
\widehat {\epsilon} _ {\mathcal {D} _ {i}} (h) = \left\{ \begin{array}{l l} \frac {1}{N _ {i}} \sum_ {\boldsymbol {x} \in X _ {i}} \mathbb {1} _ {h (\boldsymbol {x}) \neq f _ {i} (\boldsymbol {x})} & i f i = t, \\ \frac {1}{\widetilde {N} _ {i}} \sum_ {\boldsymbol {x} \in \widetilde {X} _ {i}} \mathbb {1} _ {h (\boldsymbol {x}) \neq f _ {i} (\boldsymbol {x})} & i f i <   t. \end{array} \right. \tag {4}
$$
Note that at time  $t$ , only a small subset of data from previous domains ( $i < t$ ) is available in the memory bank ( $\widetilde{N}_i \ll N_i$ ). Therefore empirical risks for previous domains ( $\widehat{\epsilon}_{\mathcal{D}_i}(h)$  with  $i < t$ ) can deviate a lot from the true risk  $\epsilon_{\mathcal{D}_i}(h)$  (defined in Eqn. 3); this is reflected in Lemma 3.1 below.
Lemma 3.1 (ERM-Based Generalization Bound). Let  $\mathcal{H}$  be a hypothesis space of VC dimension  $d$ . When domain  $t$  arrives, there are  $N_{t}$  data points from domain  $t$  and  $\widetilde{N}_{i}$  data points from each previous domain  $i < t$  in the memory bank. With probability at least  $1 - \delta$ , we have:
$$
\sum_ {i = 1} ^ {t} \epsilon_ {\mathcal {D} _ {i}} (h) \leq \sum_ {i = 1} ^ {t} \widehat {\epsilon} _ {\mathcal {D} _ {i}} (h) + \sqrt {\left(\frac {1}{N _ {t}} + \sum_ {i = 1} ^ {t - 1} \frac {1}{\widetilde {N} _ {i}}\right) \left(8 d \log \left(\frac {2 e N}{d}\right) + 8 \log \left(\frac {2}{\delta}\right)\right)}. \tag {5}
$$
Lemma 3.1 shows that naively using ERM to learn  $h$  is equivalent to minimizing a loose generalization bound in Eqn. 33. Since  $\widetilde{N}_i \ll N_i$ , there is a large constant  $\sum_{i=1}^{t-1} \frac{1}{\widetilde{N}_i}$  compared to  $\frac{1}{N_t}$ , making the second term of Eqn. 33 much larger and leading to a looser bound.
3.2 Intra-Domain and Cross-Domain Model-Based Bounds
In domain incremental learning, one has access to the history model  $H_{t-1}$  besides the memory bank  $\{M_i\}_{i=1}^{t-1}$ ; this offers an opportunity to derive tighter error bounds, potentially leading to better algorithms. In this section, we will derive two such bounds, an intra-domain error bound (Lemma 3.2) and a cross-domain error bound (Lemma 3.3), and then integrate them two with the ERM-based bound in Eqn. 33 to arrive at our final adaptive bound (Theorem 3.4).
Lemma 3.2 (Intra-Domain Model-Based Bound). Let  $h \in \mathcal{H}$  be an arbitrary function in the hypothesis space  $\mathcal{H}$ , and  $H_{t-1}$  be the model trained after domain  $t - 1$ . The domain-specific error  $\epsilon_{\mathcal{D}_i}(h)$  on the previous domain  $i$  has an upper bound:
$$
\epsilon_ {\mathcal {D} _ {i}} (h) \leq \epsilon_ {\mathcal {D} _ {i}} (h, H _ {t - 1}) + \epsilon_ {\mathcal {D} _ {i}} (H _ {t - 1}), \tag {6}
$$
where  $\epsilon_{\mathcal{D}_i}(h,H_{t - 1})\triangleq \mathbb{E}_{\pmb {x}\sim \mathcal{D}_i}[h(\pmb {x})\neq H_{t - 1}(\pmb {x})]$
Lemma 3.2 shows that the current model  $h$ 's error on domain  $i$  is bounded by the discrepancy between  $h$  and the history model  $H_{t-1}$  plus the error of  $H_{t-1}$  on domain  $i$ .
One potential issue with the bound Eqn. 34 is that only a limited number of data is available for each previous domain  $i$  in the memory bank, making empirical estimation of  $\epsilon_{\mathcal{D}_i}(h,H_{t - 1}) + \epsilon_{\mathcal{D}_i}(H_{t - 1})$  challenging. Lemma 3.3 therefore provides an alternative bound.
Lemma 3.3 (Cross-Domain Model-Based Bound). Let  $h \in \mathcal{H}$  be an arbitrary function in the hypothesis space  $\mathcal{H}$ , and  $H_{t-1}$  be the function trained after domain  $t-1$ . The domain-specific error  $\epsilon_{\mathcal{D}_i}(h)$  evaluated on the previous domain  $i$  then has an upper bound:
$$
\epsilon_ {\mathcal {D} _ {i}} (h) \leq \epsilon_ {\mathcal {D} _ {t}} (h, H _ {t - 1}) + \frac {1}{2} d _ {\mathcal {H} \Delta \mathcal {H}} (\mathcal {D} _ {i}, \mathcal {D} _ {t}) + \epsilon_ {\mathcal {D} _ {i}} (H _ {t - 1}), \tag {7}
$$
where  $d_{\mathcal{H}\Delta \mathcal{H}}(\mathcal{P},\mathcal{Q}) = 2\sup_{h\in \mathcal{H}\Delta \mathcal{H}}|\operatorname*{Pr}_{x\sim \mathcal{P}}[h(x) = 1] - \operatorname*{Pr}_{x\sim \mathcal{Q}}[h(x) = 1]|$  denotes the  $\mathcal{H}\Delta \mathcal{H}$ -divergence between distribution  $\mathcal{P}$  and  $\mathcal{Q}$ , and  $\epsilon_{\mathcal{D}_t}(h,H_{t - 1})\triangleq \mathbb{E}_{\boldsymbol{x}\sim \mathcal{D}_t}[h(\boldsymbol {x})\neq H_{t - 1}(\boldsymbol {x})]$ .
Lemma 3.3 shows that if the divergence between domain  $i$  and domain  $t$ , i.e.,  $d_{\mathcal{H}\Delta \mathcal{H}}(\mathcal{D}_i,\mathcal{D}_t)$ , is small enough, one can use  $H_{t - 1}$ 's predictions evaluated on the current domain  $\mathcal{D}_t$  as a surrogate loss to prevent catastrophic forgetting. Compared to the error bound Eqn. 34 which is hindered by limited data from previous domains, Eqn. 35 relies on the current domain  $t$  which contains abundant data and therefore enjoys much lower generalization error. Our lemma also justifies LwF-like cross-domain distillation loss  $\epsilon_{\mathcal{D}_t}(h,H_{t - 1})$  which are widely adopted [52, 23, 100].
3.3 A Unified and Adaptive Generalization Error Bound
Our Lemma 3.1, Lemma 3.2, and Lemma 3.3 provide three different ways to bound the true risk  $\sum_{i=1}^{t} \epsilon_{\mathcal{D}_i}(h)$ ; each has its own advantages and disadvantages. Lemma 3.1 overly relies on the limited number of data points from previous domains  $i < t$  in the memory bank to compute the empirical risk; Lemma 3.2 leverages the history model  $H_{t-1}$  for knowledge distillation, but is still hindered by the limited number of data points in the memory bank; Lemma 3.3 improves over Lemma 3.2 by leveraging the abundant data  $\mathcal{D}_t$  in the current domain  $t$ , but only works well if the divergence between domain  $i$  and domain  $t$ , i.e.,  $d_{\mathcal{H}\Delta \mathcal{H}}(\mathcal{D}_i, \mathcal{D}_t)$ , is small. Therefore, we propose to integrate these three bounds using coefficients  $\{\alpha_i, \beta_i, \gamma_i\}_{i=1}^{t-1}$  (with  $\alpha_i + \beta_i + \gamma_i = 1$ ) in the theorem below.
Theorem 3.4 (Unified Generalization Bound for All Domains). Let  $\mathcal{H}$  be a hypothesis space of VC dimension  $d$ . Let  $N = N_{t} + \sum_{i}^{t - 1}\widetilde{N}_{i}$  denoting the total number of data points available to the training of current domain  $t$ , where  $N_{t}$  and  $\widetilde{N}_i$  denote the numbers of data points collected at domain  $t$  and data points from the previous domain  $i$  in the memory bank, respectively. With probability at
Table 1: UDIL as a unified framework for domain incremental learning with memory. Three methods (LwF [52], ER [75], and DER++ [8]) are by default compatible with DIL setting. For the remaining four CIL methods (iCaRL [74], CLS-ER [4], EMS-ER [80], and BiC [100]), we adapt their original training objective to DIL settings before the analysis. For CLS-ER [4] and EMS-ER [80],  $\lambda$  and  $\lambda^{\prime}$  are the intensity coefficients of the logits distillation. For BiC [100],  $t$  is the current number of the incremental domain. The conditions under which the unification of each method is achieved are provided in detail in Appendix B.
UDIL (Ours)	LwF [52]	ER [75]	DER++ [8]	iCaRL [74]	CLS-ER [4]	EMS-ER [80]	BiC [100]
αi	[0,1]	0	0	0.5	1	λ/(1+λ)	λ′/(1+λ′)	1/(2t-1)
βi	[0,1]	1	0	0	0	0	0	(t-1)/(2t-1)
γi	[0,1]	0	1	0.5	0	1/(1+λ)	1/(1+λ′)	t-1/(2t-1)
least  $1 - \delta$ , we have:
$$
\begin{array}{l} \sum_ {i = 1} ^ {t} \epsilon_ {\mathcal {D} _ {i}} (h) \leq \left\{\sum_ {i = 1} ^ {t - 1} \left[ \gamma_ {i} \widehat {\epsilon} _ {\mathcal {D} _ {i}} (h) + \alpha_ {i} \widehat {\epsilon} _ {\mathcal {D} _ {i}} (h, H _ {t - 1}) \right] \right\} + \left\{\widehat {\epsilon} _ {\mathcal {D} _ {t}} (h) + \left(\sum_ {i = 1} ^ {t - 1} \beta_ {i}\right) \widehat {\epsilon} _ {\mathcal {D} _ {t}} (h, H _ {t - 1}) \right\} \\ + \frac {1}{2} \sum_ {i = 1} ^ {t - 1} \beta_ {i} d _ {\mathcal {H} \Delta \mathcal {H}} (\mathcal {D} _ {i}, \mathcal {D} _ {t}) + \sum_ {i = 1} ^ {t - 1} (\alpha_ {i} + \beta_ {i}) \epsilon_ {\mathcal {D} _ {i}} (H _ {t - 1}) \\ + \sqrt {\left(\frac {\left(1 + \sum_ {i = 1} ^ {t - 1} \beta_ {i}\right) ^ {2}}{N _ {t}} + \sum_ {i = 1} ^ {t - 1} \frac {\left(\gamma_ {i} + \alpha_ {i}\right) ^ {2}}{\widetilde {N} _ {i}}\right) \left(8 d \log \left(\frac {2 e N}{d}\right) + 8 \log \left(\frac {2}{\delta}\right)\right)} \\ \triangleq g (h, H _ {t - 1}, \Omega), \tag {8} \\ \end{array}
$$
where  $\widehat{\epsilon}_{\mathcal{D}_i}(h,H_{t - 1}) = \frac{1}{\widetilde{N}_i}\sum_{\pmb {x}\in \widetilde{\mathcal{X}}_i}\mathbb{1}_{h(\pmb {x})\neq H_{t - 1}(\pmb {x})},\widehat{\epsilon}_{\mathcal{D}_t}(h,H_{t - 1}) = \frac{1}{N_t}\sum_{\pmb {x}\in \mathcal{X}_i}\mathbb{1}_{h(\pmb {x})\neq H_{t - 1}(\pmb {x})},$  and  $\Omega \triangleq \{\alpha_i,\beta_i,\gamma_i\}_{i = 1}^{t - 1}$
Theorem 3.4 offers the opportunity of adaptively adjusting the coefficients  $(\alpha_{i},\beta_{i}$  , and  $\gamma_{i})$  according to the data (current domain data  $S_{t}$  and the memory bank  $\mathcal{M} = \{M_i\}_{i = 1}^{t - 1}$ ) and history model  $(H_{t - 1})$  at hand, thereby achieving the tightest bound. For example, when the  $\mathcal{H}\Delta \mathcal{H}$  divergence between domain  $i$  and domain  $t$  , i.e.,  $d_{\mathcal{H}\Delta \mathcal{H}}(\mathcal{D}_i,\mathcal{D}_t)$  , is small, minimizing this unified bound (Eqn. 8) leads to a large coefficient  $\beta_{i}$  and therefore naturally puts on more focus on cross-domain bound in Eqn. 35 which leverages the current domain  $t$  's data to estimate the true risk.
UDIL as a Unified Framework. Interestingly, Eqn. 8 unifies various domain incremental learning methods. Table 1 shows that different methods are equivalent to fixing the coefficients  $\{\alpha_{i},\beta_{i},\gamma_{i}\}_{i = 1}^{t - 1}$  to different values (see Appendix B for a detailed discussion). For example, assuming default configurations, LwF [52] corresponds to Eqn. 8 with fixed coefficients  $\{\alpha_{i} = \gamma_{i} = 0,\beta_{i} = 1\}$ ; ER [75] corresponds to Eqn. 8 with fixed coefficients  $\{\alpha_{i} = \beta_{i} = 0,\gamma_{i} = 1\}$ , and DER++ [8] corresponds to Eqn. 8 with fixed coefficients  $\{\alpha_{i} = \gamma_{i} = 0.5,\beta_{i} = 0\}$ , under certain achievable conditions. Inspired by this unification, our UDIL adaptively adjusts these coefficients to search for the tightest bound in the range [0, 1] when each domain arrives during domain incremental learning, thereby improving performance. Corollary 3.4.1 below shows that such adaptive bound is always tighter, or at least as tight as, any bounds with fixed coefficients.
Corollary 3.4.1. For any bound  $g(h, H_{t-1}, \Omega_{fixed})$  (defined in Eqn. 8) with fixed coefficients  $\Omega_{fixed}$ , e.g.,  $\Omega_{fixed} = \Omega_{ER} = \{\alpha_i = \beta_i = 0, \gamma_i = 1\}_{i=1}^{t-1}$  for ER [75], we have
$$
\sum_ {i = 1} ^ {t} \epsilon_ {\mathcal {D} _ {i}} (h) \leq \min  _ {\Omega} g (h, H _ {t - 1}, \Omega) \leq g (h, H _ {t - 1}, \Omega_ {f i x e d}), \quad \forall h, H _ {t - 1} \in \mathcal {H}. \tag {9}
$$
Corollary 3.4.1 shows that the unified bound Eqn. 8 with adaptive coefficients is always preferable to other bounds with fixed coefficients. We therefore use it to develop a better domain incremental learning algorithm in Sec. 4 below.
4 Method: Adaptively Minimizing the Tightest Bound in UDIL
Although Theorem 3.4 provides a unified perspective for domain incremental learning, it does not immediately translate to a practical objective function to train a model. It is also unclear what coefficients  $\Omega$  for Eqn. 8 would be the best choice. In fact, a static and fixed setting will not suffice, as different problems may involve different sequences of domains with dynamic changes; therefore ideally  $\Omega$  should be dynamic (e.g.,  $\alpha_{i} \neq \alpha_{i+1}$ ) and adaptive (i.e., learnable from data). In this section, we start by mapping the unified bound in Eqn. 8 to concrete loss terms, discuss how the coefficients  $\Omega$  are learned, and then provide a final objective function to learn the optimal model.
4.1 From Theory to Practice: Translating the Bound in Eqn. 8 to Differentiable Loss Terms
(1) ERM Terms. We use the cross-entropy classification loss in Definition 4.1 below to optimize domain  $t$  's ERM term  $\widehat{\epsilon}_{\mathcal{D}_t}(h)$  and memory replay ERM terms  $\{\gamma_i\widehat{\epsilon}_{\mathcal{D}_i}(h)\}_{i = 1}^{t - 1}$  in Eqn. 8.