SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In Sec. A, we present the proofs for the lemmas, theorems, and corollaries presented in the main body of our work. Sec. B discusses the correspondence of existing methods to specific cases within our framework. In Sec. C, we provide a detailed presentation of our final algorithm, UDIL, including an algorithmic description, a visual diagram, and implementation details. We introduce the experimental settings, including the evaluation metrics and specific training schemes. Finally, in Sec. D, we present additional empirical results with varying memory sizes and provide more visualization results.
A Proofs of Lemmas, Theorems, and Corollaries
Before proceeding to prove any lemmas or theorems, we first introduce three crucial additional lemmas that will be utilized in the subsequent sections. Among these, Lemma A.1 offers a generalization bound for any weighted summation of ERM losses across multiple domains. Furthermore, Lemma A.2 provides a generalization bound for a weighted summation of labeling functions within a given domain. Lastly, we highlight Lemma 3 in [5] as Lemma A.3, which will be used to establish the upper bound for Lemma 3.3.
Lemma A.1 (Generalization Bound of  $\alpha$ -weighted Domains). Let  $\mathcal{H}$  be a hypothesis space of VC dimension  $d$ . Assume  $N_{j}$  denotes the number of the samples collected from domain  $j$ , and  $N = \sum_{j} N_{j}$  is the total number of the examples collected from all domains. Then for any  $\alpha_{j} > 0$  and  $\delta \in (0,1)$ , with probability at least  $1 - \delta$ :
$$
\sum_ {j} \alpha_ {j} \epsilon_ {\mathcal {D} _ {j}} (h) \leq \sum_ {j} \alpha_ {j} \widehat {\epsilon} _ {\mathcal {D} _ {j}} (h) + \sqrt {\left(\sum_ {j} \frac {\alpha_ {j} ^ {2}}{N _ {j}}\right) \left(8 d \log \left(\frac {2 e N}{d}\right) + 8 \log \left(\frac {2}{\delta}\right)\right)}. \tag {17}
$$
Proof. Suppose each domain  $\mathcal{D}_j$  has a deterministic ground-truth labeling function  $f_j: \mathbb{R}^n \to \{0,1\}$ . Denote as  $\widehat{\epsilon}_{\alpha} \triangleq \sum_{j} \alpha_j \widehat{\epsilon}_{\mathcal{D}_j}(h)$  the  $\alpha$ -weighted empirical loss evaluated on different domains. Hence,
$$
\widehat {\epsilon} _ {\alpha} (h) = \sum_ {j} \alpha_ {j} \widehat {\epsilon} _ {\mathcal {D} _ {j}} (h) = \sum_ {j} \alpha_ {j} \frac {1}{N _ {j}} \sum_ {\boldsymbol {x} \in \mathcal {X} _ {j}} \mathbb {1} _ {h (\boldsymbol {x}) \neq f _ {j} (\boldsymbol {x})} = \frac {1}{N} \sum_ {j} \sum_ {k = 1} ^ {N _ {j}} R _ {j, k}, \tag {18}
$$
where  $R_{j,k} = \left(\frac{\alpha_jN_j}{N}\right)\cdot \mathbb{1}_{h(\boldsymbol{x}_k)\neq f_j(\boldsymbol{x}_k)}$  is a random variable that takes the values in  $\{\frac{\alpha_jN_j}{N},0\}$ . By the linearity of the expectation, we have  $\epsilon_{\alpha}(h) = \mathbb{E}[\widehat{\epsilon}_{\alpha}(h)]$ . Following [3, 62], we have
$$
\begin{array}{l} \mathbb {P} \left\{\exists h \in \mathcal {H}, \text {s . t .} | \widehat {\epsilon} _ {\alpha} (h) - \epsilon_ {\alpha} (h) | \geq \epsilon \right\} (19) \\ \leq 2 \cdot \mathbb {P} \left\{\sup  _ {h \in \mathcal {H}} \left| \widehat {\epsilon} _ {\alpha} (h) - \widehat {\epsilon} _ {\alpha} ^ {\prime} (h) \right| \geq \frac {\epsilon}{2} \right\} (20) \\ \leq 2 \cdot \mathbb {P} \left\{\bigcup_ {R _ {j, k}, R _ {j, k} ^ {\prime}} \frac {1}{N} \left| \sum_ {j} \sum_ {k = 1} ^ {N _ {j}} \left(R _ {j, k} - R _ {j, k} ^ {\prime}\right) \right| \geq \frac {\epsilon}{2} \right\} (21) \\ \leq 2 \Pi_ {\mathcal {H}} (2 N) \exp \left\{\frac {- 2 \left(N \varepsilon / 2\right) ^ {2}}{\sum_ {j} \left(N _ {j}\right) \left(2 \alpha_ {j} N / N _ {j}\right) ^ {2}} \right\} (22) \\ = 2 \Pi_ {\mathcal {H}} (2 N) \exp \left\{- \frac {\epsilon^ {2}}{8 \sum_ {j} \left(\alpha_ {j} ^ {2} / N _ {j}\right)} \right\} (23) \\ \leq 2 (2 N) ^ {d} \exp \left\{- \frac {\epsilon^ {2}}{8 \sum_ {j} \left(\alpha_ {j} ^ {2} / N _ {j}\right)} \right\}, (24) \\ \end{array}
$$
where in Eqn. 20,  $\hat{\epsilon}_{\alpha}^{\prime}(h)$  is the  $\alpha$ -weighted empirical loss evaluated on the "ghost" set of examples  $\{\mathcal{X}_j^{\prime}\}$ ; Eqn. 22 is yielded by applying Hoeffding's inequalities [37] and introducing the growth function  $\Pi_{\mathcal{H}}$  [3, 62, 91] at the same time; Eqn. 24 is achieved by using the fact  $\Pi_{\mathcal{H}}(2N) \leq (e \cdot 2N / d)^d \leq (2N)^d$ , where  $d$  is the VC-dimension of the hypothesis set  $\mathcal{H}$ . Finally, by setting Eqn. 24 to  $\delta$  and solve for the error tolerance  $\epsilon$  will complete the proof.
Lemma A.2 (Generalization Bound of  $\beta$ -weighted Labeling Functions). Let  $\mathcal{D}$  be a single domain and  $\mathcal{X} = \{\pmb{x}_i\}_{i}^N$  be a collection of samples drawn from  $\mathcal{D}$ ;  $\mathcal{H}$  is a hypothesis space of VC dimension
d. Suppose  $\{f_j : \mathbb{R}^n \to \{0,1\}\}_j$  is a set of different labeling functions. Then for any  $\beta_j > 0$  and  $\delta \in (0,1)$ , with probability at least  $1 - \delta$ :
$$
\sum_ {j} \beta_ {j} \epsilon_ {\mathcal {D}} (h, f _ {j}) \leq \sum_ {j} \beta_ {j} \widehat {\epsilon} _ {\mathcal {D}} (h, f _ {j}) + \left(\sum_ {j} \beta_ {j}\right) \sqrt {\frac {1}{N} \left(8 d \log \left(\frac {2 e N}{d}\right) + 8 \log \left(\frac {2}{\delta}\right)\right)}. \tag {25}
$$
Proof. Denote as  $\epsilon_{\beta}(h) \triangleq \sum_{j} \beta_{j} \epsilon_{\mathcal{D}}(h, f_{j})$  the  $\beta$ -weighted error on domain  $\mathcal{D}$  and  $\{f_{j}\}_{j}$  the set of the labeling functions, and  $\widehat{\epsilon}_{\beta} \triangleq \sum_{j} \beta_{j} \widehat{\epsilon}_{\mathcal{D}}(h, f_{j})$  as the  $\beta$ -weighted empirical loss evaluated on different labeling functions. We have
$$
\widehat {\epsilon} _ {\beta} (h) = \sum_ {j} \beta_ {j} \frac {1}{N} \sum_ {i = 1} ^ {N} \mathbb {1} _ {h \left(\boldsymbol {x} _ {i}\right) \neq f _ {j} \left(\boldsymbol {x} _ {i}\right)} = \frac {1}{N} \sum_ {i = 1} ^ {N} \sum_ {j} \beta_ {j} \mathbb {1} _ {h \left(\boldsymbol {x} _ {i}\right) \neq f _ {j} \left(\boldsymbol {x} _ {i}\right)} \triangleq \frac {1}{N} \sum_ {i = 1} ^ {N} R _ {i}, \tag {26}
$$
where  $R_{i} = \sum_{j}\beta_{j}\mathbb{1}_{h(\pmb{x}_{i})\neq f_{j}(\pmb{x}_{i})}\in [0,\sum_{j}\beta_{j}]$  is a new random variable.
$$
\mathbb {P} \left\{\exists h \in \mathcal {H}, \text {s . t .} | \widehat {\epsilon} _ {\beta} (h) - \epsilon_ {\beta} (h) | \geq \epsilon \right\} \tag {27}
$$
$$
\leq 2 \cdot \mathbb {P} \left\{\sup  _ {h \in \mathcal {H}} \left| \widehat {\epsilon} _ {\beta} (h) - \widehat {\epsilon} _ {\beta} ^ {\prime} (h) \right| \geq \frac {\epsilon}{2} \right\} \tag {28}
$$
$$
\leq 2 \cdot \mathbb {P} \left\{\bigcup_ {R _ {i}, R _ {i} ^ {\prime}} \frac {1}{N} \left| \sum_ {i = 1} ^ {N} \left(R _ {i} - R _ {i} ^ {\prime}\right) \right| \geq \frac {\epsilon}{2} \right\} \tag {29}
$$
$$
\leq 2 \Pi_ {\mathcal {H}} (2 N) \exp \left\{\frac {- 2 \left(N \epsilon / 2\right) ^ {2}}{N \cdot \left(2 \sum_ {j} \beta_ {j}\right) ^ {2}} \right\} \tag {30}
$$
$$
\leq 2 (2 N) ^ {d} \exp \left\{- \frac {N \epsilon^ {2}}{8 \left(\sum_ {j} \beta_ {j}\right) ^ {2}} \right\}, \tag {31}
$$
where in Eqn. 28,  $\hat{\epsilon}_{\beta}^{\prime}(h)$  is the  $\beta$ -weighted empirical loss evaluated on the "ghost" set of examples  $\mathcal{X}'$ ; Eqn. 30 is yielded by applying Hoeffding's inequalities [37] and introducing the growth function  $\Pi_{\mathcal{H}}[3,62,91]$  at the same time; Eqn. 31 is achieved by using the fact  $\Pi_{\mathcal{H}}(2N) \leq (e \cdot 2N / d)^d \leq (2N)^d$ , where  $d$  is the VC-dimension of the hypothesis set  $\mathcal{H}$ . Finally, by setting Eqn. 31 to  $\delta$  and solve for the error tolerance  $\epsilon$  will complete the proof.
Lemma A.2 asserts that altering or merging multiple target functions does not impact the generalization error term, as long as the sum of the weights for each loss  $\sum_{j}\beta_{j}$  remains constant and the same dataset  $\mathcal{X}$  is used for estimation. Next we highlight the Lemma 3 in [5] again, as it will be utilized for proving 3.3.
Lemma A.3. For any hypothesis  $h, h' \in \mathcal{H}$  and any two different domains  $\mathcal{D}, \mathcal{D}'$ ,
$$
\left| \epsilon_ {\mathcal {D}} \left(h, h ^ {\prime}\right) - \epsilon_ {\mathcal {D} ^ {\prime}} \left(h, h ^ {\prime}\right) \right| \leq \frac {1}{2} d _ {\mathcal {H} \Delta \mathcal {H}} \left(\mathcal {D}, \mathcal {D} ^ {\prime}\right). \tag {32}
$$
Proof. By definition, we have
$$
\begin{array}{l} d _ {\mathcal {H} \Delta \mathcal {H}} (\mathcal {D}, \mathcal {D} ^ {\prime}) = 2 \sup  _ {h, h ^ {\prime} \in \mathcal {H}} | \mathbb {P} _ {\boldsymbol {x} \sim \mathcal {D}} [ h (\boldsymbol {x}) \neq h ^ {\prime} (\boldsymbol {x}) ] - \mathbb {P} _ {\boldsymbol {x} \sim \mathcal {D} ^ {\prime}} [ h (\boldsymbol {x}) \neq h ^ {\prime} (\boldsymbol {x}) ] | \\ = 2 \sup  _ {h, h ^ {\prime} \in \mathcal {H}} | \epsilon_ {\mathcal {D}} (h, h ^ {\prime}) - \epsilon_ {\mathcal {D} ^ {\prime}} (h, h ^ {\prime}) | \\ \geq 2 \left| \epsilon_ {\mathcal {D}} \left(h, h ^ {\prime}\right) - \epsilon_ {\mathcal {D} ^ {\prime}} \left(h, h ^ {\prime}\right) \right|. \\ \end{array}
$$
Now we are ready to prove the main lemmas and theorems in the main body of our work.
$$
\sum_ {i = 1} ^ {t} \epsilon_ {\mathcal {D} _ {i}} (h) \leq \sum_ {i = 1} ^ {t} \widehat {\epsilon} _ {\mathcal {D} _ {i}} (h) + \sqrt {\left(\frac {1}{N _ {t}} + \sum_ {i = 1} ^ {t - 1} \frac {1}{\widetilde {N} _ {i}}\right) \left(8 d \log \left(\frac {2 e N}{d}\right) + 8 \log \left(\frac {2}{\delta}\right)\right)}. \tag {33}
$$
Proof. Simply using Lemma A.1 and setting  $\alpha_{i} = 1$  for every  $i\in [t]$  completes the proof.
$$
\epsilon_ {\mathcal {D} _ {i}} (h) \leq \epsilon_ {\mathcal {D} _ {i}} (h, H _ {t - 1}) + \epsilon_ {\mathcal {D} _ {i}} (H _ {t - 1}), \tag {34}
$$
Proof. By applying the triangle inequality [5] of the 0-1 loss function, we have
$$
\begin{array}{l} \epsilon_ {\mathcal {D} _ {i}} (h) = \epsilon_ {\mathcal {D} _ {i}} (h, f _ {i}) \\ \leq \epsilon_ {\mathcal {D} _ {i}} (h, H _ {t - 1}) + \epsilon_ {\mathcal {D} _ {i}} \left(H _ {t - 1}, f _ {i}\right) \\ = \epsilon_ {\mathcal {D} _ {i}} (h, H _ {t - 1}) + \epsilon_ {\mathcal {D} _ {i}} (H _ {t - 1}). \\ \end{array}
$$
Lemma 3.3 (Cross-Domain Model-Based Bound). Let  $h \in \mathcal{H}$  be an arbitrary function in the hypothesis space  $\mathcal{H}$ , and  $H_{t-1}$  be the function trained after domain  $t - 1$ . The domain-specific error  $\epsilon_{\mathcal{D}_i}(h)$  evaluated on the previous domain  $i$  then has an upper bound:
$$
\epsilon_ {\mathcal {D} _ {i}} (h) \leq \epsilon_ {\mathcal {D} _ {t}} (h, H _ {t - 1}) + \frac {1}{2} d _ {\mathcal {H} \Delta \mathcal {H}} (\mathcal {D} _ {i}, \mathcal {D} _ {t}) + \epsilon_ {\mathcal {D} _ {i}} (H _ {t - 1}), \tag {35}
$$
where  $d_{\mathcal{H}\Delta \mathcal{H}}(\mathcal{P},\mathcal{Q}) = 2\sup_{h\in \mathcal{H}\Delta \mathcal{H}}|\operatorname *{Pr}_{x\sim \mathcal{P}}[h(x) = 1] - \operatorname *{Pr}_{x\sim \mathcal{Q}}[h(x) = 1]|$  denotes the  $\mathcal{H}\Delta \mathcal{H}$ -divergence between distribution  $\mathcal{P}$  and  $\mathcal{Q}$ , and  $\epsilon_{\mathcal{D}_t}(h,H_{t - 1})\triangleq \mathbb{E}_{\boldsymbol {x}\sim \mathcal{D}_t}[h(\boldsymbol {x})\neq H_{t - 1}(\boldsymbol {x})]$ .
Proof. By the triangle inequality used above and Lemma A.3, we have
$$
\begin{array}{l} \epsilon_ {\mathcal {D} _ {i}} (h) \leq \epsilon_ {\mathcal {D} _ {i}} (h, H _ {t - 1}) + \epsilon_ {\mathcal {D} _ {i}} (H _ {t - 1}) \\ = \epsilon_ {\mathcal {D} _ {i}} (h, H _ {t - 1}) + \epsilon_ {\mathcal {D} _ {i}} (H _ {t - 1}) - \epsilon_ {\mathcal {D} _ {t}} (h, H _ {t - 1}) + \epsilon_ {\mathcal {D} _ {t}} (h, H _ {t - 1}) \\ \leq \epsilon_ {\mathcal {D} _ {i}} \left(H _ {t - 1}\right) + \left| \epsilon_ {\mathcal {D} _ {i}} (h, H _ {t - 1}) - \epsilon_ {\mathcal {D} _ {t}} (h, H _ {t - 1}) \right| + \epsilon_ {\mathcal {D} _ {t}} (h, H _ {t - 1}) \\ \leq \epsilon_ {\mathcal {D} _ {t}} (h, H _ {t - 1}) + \frac {1}{2} d _ {\mathcal {H} \Delta \mathcal {H}} (\mathcal {D} _ {i}, \mathcal {D} _ {t}) + \epsilon_ {\mathcal {D} _ {i}} (H _ {t - 1}). \\ \end{array}
$$
Theorem 3.4 (Unified Generalization Bound for All Domains). Let  $\mathcal{H}$  be a hypothesis space of VC dimension  $d$ . Let  $N = N_{t} + \sum_{i}^{t - 1}\widetilde{N}_{i}$  denoting the total number of data points available to the training of current domain  $t$ , where  $N_{t}$  and  $\widetilde{N}_i$  denote the numbers of data points collected at domain  $t$  and data points from the previous domain  $i$  in the memory bank, respectively. With probability at least  $1 - \delta$ , we have:
$$
\begin{array}{l} \sum_ {i = 1} ^ {t} \epsilon_ {\mathcal {D} _ {i}} (h) \leq \left\{\sum_ {i = 1} ^ {t - 1} \left[ \gamma_ {i} \widehat {\epsilon} _ {\mathcal {D} _ {i}} (h) + \alpha_ {i} \widehat {\epsilon} _ {\mathcal {D} _ {i}} (h, H _ {t - 1}) \right] \right\} + \left\{\widehat {\epsilon} _ {\mathcal {D} _ {t}} (h) + \left(\sum_ {i = 1} ^ {t - 1} \beta_ {i}\right) \widehat {\epsilon} _ {\mathcal {D} _ {t}} (h, H _ {t - 1}) \right\} \\ + \frac {1}{2} \sum_ {i = 1} ^ {t - 1} \beta_ {i} d _ {\mathcal {H} \Delta \mathcal {H}} (\mathcal {D} _ {i}, \mathcal {D} _ {t}) + \sum_ {i = 1} ^ {t - 1} (\alpha_ {i} + \beta_ {i}) \epsilon_ {\mathcal {D} _ {i}} (H _ {t - 1}) \\ + \sqrt {\left(\frac {(1 + \sum_ {i = 1} ^ {t - 1} \beta_ {i}) ^ {2}}{N _ {t}} + \sum_ {i = 1} ^ {t - 1} \frac {(\gamma_ {i} + \alpha_ {i}) ^ {2}}{\widetilde {N} _ {i}}\right) \left(8 d \log \left(\frac {2 e N}{d}\right) + 8 \log \left(\frac {2}{\delta}\right)\right)} \\ \triangleq g (h, H _ {t - 1}, \Omega), \tag {36} \\ \end{array}
$$
Proof. By applying Lemma 3.2 and Lemma 3.3 to each of the past domains, we have
$$
\begin{array}{l} \epsilon_ {\mathcal {D} _ {i}} (h) = \left(\alpha_ {i} + \beta_ {i} + \gamma_ {i}\right) \epsilon_ {\mathcal {D} _ {i}} (h) \\ \leq \gamma_ {i} \epsilon_ {\mathcal {D} _ {i}} (h) + \alpha_ {i} [ \epsilon_ {\mathcal {D} _ {i}} (h, H _ {t - 1}) + \epsilon_ {\mathcal {D} _ {i}} (H _ {t - 1}) ] \\ + \beta_ {i} [ \epsilon_ {\mathcal {D} _ {i}} (h, H _ {t - 1}) + \epsilon_ {\mathcal {D} _ {t}} (h, H _ {t - 1}) + \frac {1}{2} d _ {\mathcal {H} \Delta \mathcal {H}} (\mathcal {D} _ {i}, \mathcal {D} _ {t}) ]. \\ \end{array}
$$
Table 4: Unification of existing methods under UDIL, when certain conditions are achieved.
αi	βi	γi	Transformed Objective	Condition
UDIL (Ours)	[0,1]	[0,1]	[0,1]	-	-
LwF [52]	0	1	0	LwF(h) = lxt(h) + λo lxt(h, Ht-1)	λo = t - 1
ER [75]	0	0	1	LER(h) = lxt(bt)(h) + ∑i=1t-1|Bt′|(t-1)/|Bt| lxt(bt)(h)	|Bt| = |Bt′|/(t-1)
DER++ [8]	1/2	0	1/2	LDER++(h) = lxt(bt)(h) + 1/2 ∑i=1t-1|Bt′|(t-1)/|Bt| lxt(bt)(h) + lxt(bt′)(h, Ht-1)	|Bt| = |Bt′|/(t-1)
iCaRL [74]	1	0	0	LCaRL(h) = lxt(bt)(h) + ∑i=1t-1|Bt′|(t-1)/|Bt| lxt(bt′)(h, Ht-1)	|Bt| = |Bt′|/(t-1)
CLS-ER [4]	λ/λ+1	0	1/λ+1	CLS-ER(h) = lxt(bt)(h) + ∑i=1t-11/1-1 lxt(bt′)(h) + ∑i=1t-1λ/1-1 lxt(bt′)(h, Ht-1)	λ = t - 2
ESM-ER [80]	λ/λ+1	0	1/λ+1	ESM-ER(h) = lxt(bt)(h) + ∑i=1t-1r(t-1) lxt(bt′)(h) + ∑i=1t-1λ/r(t-1) lxt(bt′)(h, Ht-1)	{λ = -1 + r(t-1) / r = 1 - e-1}
BiC [100]	t-1/2t-1	t-1/2t-1	1/2t-1	LCiC(h) = lxt(bt)(h) + ∑i=1t-1(1-1)|Bt| lxt(bt′)(h, Ht-1) + (t-1) lxt(bt)(h, Ht-1) + ∑i=1t-1|Bt| lxt(bt′)(h)	|Bi| = |Bt|
Re-organizing the terms will give us
$$
\begin{array}{l} \sum_ {i = 1} ^ {t} \epsilon_ {\mathcal {D} _ {i}} (h) \leq \left\{\sum_ {i = 1} ^ {t - 1} \left[ \gamma_ {i} \epsilon_ {\mathcal {D} _ {i}} (h) + \alpha_ {i} \epsilon_ {\mathcal {D} _ {i}} (h, H _ {t - 1}) \right] \right\} + \left\{\epsilon_ {\mathcal {D} _ {t}} (h) + \left(\sum_ {i = 1} ^ {t - 1} \beta_ {i}\right) \epsilon_ {\mathcal {D} _ {t}} (h, H _ {t - 1}) \right\} \\ + \frac {1}{2} \sum_ {i = 1} ^ {t - 1} \beta_ {i} d _ {\mathcal {H} \Delta \mathcal {H}} \left(\mathcal {D} _ {i}, \mathcal {D} _ {t}\right) + \sum_ {i = 1} ^ {t - 1} \left(\alpha_ {i} + \beta_ {i}\right) \epsilon_ {\mathcal {D} _ {i}} \left(H _ {t - 1}\right). \tag {37} \\ \end{array}
$$
Then applying Lemma A.1 and Lemma A.2 jointly to Eqn. 37 will complete the proof.
B UDIL as a Unified Framework
In this section, we will delve into a comprehensive discussion of our UDIL framework, which serves as a unification of numerous existing methods. It is important to note that we incorporate methods designed for task incremental and class incremental scenarios that can be easily adapted to our domain incremental learning. To provide clarity, we will present the corresponding coefficients  $\{\alpha_{i},\beta_{i},\gamma_{i}\}$  of each method within our UDIL framework (refer to Table 4). Furthermore, we will explore the conditions under which these coefficients are included in this unification process.
Learning without Forgetting (LwF) [52] was initially proposed for task-incremental learning, incorporating a combination of shared parameters and task-specific parameters. This framework can be readily extended to domain incremental learning by setting all "domain-specific" parameters to be the same in a static model architecture. LwF was designed for the strict continual learning setting, where no data from past tasks is accessible. To overcome this limitation, LwF records the predictions of the history model  $H_{t-1}$  on the current data  $\mathcal{X}_t$  at the beginning of the new task  $t$ . Subsequently, knowledge distillation (as defined in Definition 4.2) is performed to mitigate forgetting:
$$
\mathcal {L} _ {\text {o l d}} (h, H _ {t - 1}) \triangleq - \frac {1}{N _ {t}} \sum_ {\boldsymbol {x} \in \mathcal {X} _ {t}} \sum_ {k = 1} ^ {K} \left[ H _ {t - 1} (\boldsymbol {x}) \right] _ {k} \cdot \left[ \log \left(\left[ h (\boldsymbol {x}) \right] _ {k}\right) \right] = \widehat {\ell} _ {\mathcal {X} _ {t}} (h, H _ {t - 1}), \tag {38}
$$
where  $H_{t - 1}(\pmb {x}),h(\pmb {x})\in \mathbb{R}^K$  are the class distribution of  $\pmb{x}$  over  $K$  classes produced by the history model and current model, respectively. The loss for learning the current task  $\mathcal{L}_{\mathrm{new}}$  is defined as
$$
\mathcal {L} _ {\text {n e w}} (h) \triangleq - \frac {1}{N _ {t}} \sum_ {(\boldsymbol {x}, y) \in \mathcal {S} _ {t}} \sum_ {k = 1} ^ {K} \mathbb {1} _ {y = k} \cdot \left[ \log \left([ h (\boldsymbol {x}) ] _ {k}\right) \right] = \widehat {\ell} _ {\mathcal {X} _ {t}} (h). \tag {39}
$$
LwF uses a "loss balance weight"  $\lambda_{o}$  to balance two losses, which gives us its final loss for training:
$$
\mathcal {L} _ {\mathrm {L w F}} (h) \triangleq \mathcal {L} _ {\mathrm {n e w}} (h) + \lambda_ {o} \cdot \mathcal {L} _ {\mathrm {o l d}} (h, H _ {t - 1}). \tag {40}
$$
In LwF, the default setting assumes the presence of two domains (tasks) with  $\lambda_{o} = 1$ . However, it is possible to learn multiple domains continuously using LwF's default configuration. To achieve this, the current domain  $t$  can be weighed against the number of previous domains (1 versus  $t - 1$ ). Specifically, if there is no preference for any particular domain,  $\lambda_{o}$  should be set to  $t - 1$ . Remarkably, this is equivalent to setting  $\{\beta_{i} = 1, \alpha_{i} = \gamma_{i} = 0\}$  in our UDIL framework (Row 2 in Table 4).
Experience Replay (ER) [75] serves as the fundamental operation for replay-based continual learning methods. It involves storing and replaying a subset of examples from past domains during training. Following the description and implementation provided by [8], ER operates as follows: during each training iteration on domain  $t$ , a mini-batch  $B_{t}$  of examples is sampled from the current domain, along with a mini-batch  $B_{t}^{\prime}$  from the memory. These two mini-batches are then concatenated into a larger mini-batch  $(B_{t} \cup B_{t}^{\prime})$ , upon which average gradient descent is performed:
$$
\begin{array}{l} \mathcal {L} _ {\mathrm {E R}} (h) = \widehat {\ell} _ {B _ {t} \cup B _ {t} ^ {\prime}} (h) (41) \\ = \frac {1}{\left| B _ {t} \right| + \left| B _ {t} ^ {\prime} \right|} \sum_ {\left(\boldsymbol {x}, \mathcal {Y}\right) \in B _ {t} \cup B _ {t} ^ {\prime}} \sum_ {k = 1} ^ {K} \mathbb {1} _ {\mathcal {Y} = k} \cdot \left[ \log \left(\left[ h (\boldsymbol {x}) \right] _ {k}\right) \right] (42) \\ = \frac {\left| B _ {t} \right|}{\left| B _ {t} \right| + \left| B _ {t} ^ {\prime} \right|} \widehat {\ell} _ {B _ {t}} (h) + \frac {\left| B _ {t} ^ {\prime} \right|}{\left| B _ {t} \right| + \left| B _ {t} ^ {\prime} \right|} \widehat {\ell} _ {B _ {t} ^ {\prime}} (h). (43) \\ \end{array}
$$
Suppose that each time the mini-batch of past-domain data is perfectly balanced, meaning that each domain has the same number of examples in  $B_{t}^{\prime}$ . In this case, Eqn. 43 can be further decomposed as follows:
$$
\mathcal {L} _ {\mathrm {E R}} (h) = \frac {\left| B _ {t} \right|}{\left| B _ {t} \right| + \left| B _ {t} ^ {\prime} \right|} \widehat {\ell} _ {B _ {t}} (h) + \sum_ {i = 1} ^ {t - 1} \frac {\left| B _ {t} ^ {\prime} \right| / (t - 1)}{\left| B _ {t} \right| + \left| B _ {t} ^ {\prime} \right|} \widehat {\ell} _ {B _ {i} ^ {\prime}} (h), \tag {44}
$$
where  $B_{i}^{\prime} = \{(x,y)|(x,y)\in (B_{t}^{\prime}\cap M_{i})\}$  is the subset of the mini-batch that belongs to domain  $i$
Now, by dividing both sides of Eqn. 44 by  $\left( \frac{|B_t| + |B_t'|}{|B_t|} \right)$  and comparing it to Theorem 3.4, we can include ER in our UDIL framework when the condition  $|B_t| = |B_t'| / (t - 1)$  is satisfied. In this case, ER is equivalent to  $\{\alpha_i = \beta_i = 0, \gamma_i = 1\}$  in UDIL (Row 3 in Table 4). It is important to note that this condition is not commonly met throughout the entire process of continual learning. It can be achieved by linearly scaling up the size of the mini-batch from the memory (which is feasible in the early domains) or by linearly scaling down the mini-batch from the current-domain data (which may cause a drop in model performance). It is worth mentioning that this incongruence highlights the intrinsic bias of the original ER setting towards current domain learning and cannot be rectified by adjusting the batch sizes of the current domain or the memory. However, it does not weaken our claim of unification.
Dark Experience Replay (DER++) [8] includes an additional dark experience replay, i.e., knowledge distillation on the past domain exemplars, compared to ER [75]. Now under the same assumptions (balanced sampling strategy and  $|B_t| = |B_t'| / (t - 1)$ ) as discussed for ER, we can utilize Eqn. 44 to transform the DER++ loss as follows:
$$
\mathcal {L} _ {\mathrm {D E R} + +} (h) = \frac {\left| B _ {t} \right|}{\left| B _ {t} \right| + \left| B _ {t} ^ {\prime} \right|} \widehat {\ell} _ {B _ {t}} (h) + \frac {1}{2} \sum_ {i = 1} ^ {t - 1} \frac {\left| B _ {t} ^ {\prime} \right| / (t - 1)}{\left| B _ {t} \right| + \left| B _ {t} ^ {\prime} \right|} \widehat {\ell} _ {B _ {i} ^ {\prime}} (h) + \frac {1}{2} \sum_ {i = 1} ^ {t - 1} \frac {\left| B _ {t} ^ {\prime} \right| / (t - 1)}{\left| B _ {t} \right| + \left| B _ {t} ^ {\prime} \right|} \widehat {\ell} _ {B _ {i} ^ {\prime}} (h, H _ {t - 1}). \tag {45}
$$
In this scenario,  $\mathrm{DER} + +$  is equivalent to  $\{\alpha_{i} = \gamma_{i} = 1 / 2,\beta_{i} = 0\}$  in UDIL (Row 4 in Table 4).
Incremental Classifier and Representation Learning (iCaRL) [74] was initially proposed for class incremental learning. It decouples learning the representations and final classifier into two individual phases. During training, iCaRL adopts an incrementally increasing linear classifier. Different from traditional design choice of multi-class classification where the softmax activation layer and multiclass cross entropy loss are used jointly, iCaRL models multi-class classification as multi-label classification [107, 86, 106]. Suppose there are  $K$  classes in total, and we denote the one-hot label vector of the input  $\mathbf{x}$  as  $\mathbf{y} \in \mathbb{R}^{K}$  where  $y_{j} \triangleq \mathbb{1}_{f(\mathbf{x}) = j}$ . Then the multi-label learning objective treats each dimension of the output logits as a score for binary classification, which is computed as follows: