SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\widehat {\ell} _ {\mathcal {X}} (h) \triangleq \frac {- 1}{| \mathcal {X} |} \sum_ {\boldsymbol {x} \in \mathcal {X}} \sum_ {j = 1} ^ {K} \left[ y _ {j} \log \left([ h (\boldsymbol {x}) ] _ {j}\right) + (1 - y _ {j}) \log \left(1 - [ h (\boldsymbol {x}) ] _ {j}\right) \right]. \tag {46}
$$
When a new task that contains  $K'$  new classes is presented, iCaRL adds additional  $K'$  new dimensions to the final linear classifier.
In iCaRL training, the new classes and the old classes are treated differently. For new classes, it trains the representations of the network with the ground-truth labels  $(f(\pmb{x})$  in the original paper), and for old classes, it uses the history model's output  $(H_{t-1}(\pmb{x}))$  as the learning target (i.e., pseudo labels). Each data point is treated with the same level of importance during iCaRL's training procedure. Hence after translating the loss function of iCaRL in the context of class-incremental learning to domain-incremental learning, we have
$$
\mathcal {L} _ {\mathrm {i C a R L}} (h) = N _ {t} \cdot \widehat {\ell} _ {\mathcal {X} _ {t}} ^ {\prime} (h) + \sum_ {i = 1} ^ {t - 1} \widetilde {N} _ {i} \cdot \widehat {\ell} _ {\mathcal {X} _ {i}} ^ {\prime} (h, H _ {t - 1}), \tag {47}
$$
where in  $\widehat{\ell}_{\chi_i}(h,H_{t - 1})$  , we follow the same definition as in the distillation loss in Eqn. 4.2 and replace the ground-truth label with the soft label for iCaRL.
Similar to ER [75] and DER++ [8] as analyzed before, the equation above does not naturally fall into the realm of unification provided in UDIL. To achieve this, we need to sample the data from the current domain and exemplars in the memory independently, i.e., assuming  $B_{t}$  and  $B_{i}$ . By applying the same rule of  $\alpha_{i} + \beta_{i} + \gamma_{i} = 1$ , we now have that for iCaRL, the corresponding coefficients are  $\{\alpha_{i} = B_{i} / B_{t} = 1, \beta_{i} = \gamma_{i} = 0\}$  (Row 5 in Table 4).
Complementary Learning System based Experience Replay (CLS-ER) [4] involves the maintenance of two history models, namely the plastic model  $H_{t - 1}^{(p)}$  and the stable model  $H_{t - 1}^{(s)}$ , throughout the continual training process of the working model  $h$ . Following each update of the working model, the two history models are stochastically updated at different rates using exponential moving averages (EMA) of the working model's parameters:
$$
H _ {t - 1} ^ {(i)} \leftarrow \alpha^ {(i)} \cdot H _ {t - 1} ^ {(i)} + (1 - \alpha^ {(i)}) \cdot h, \quad i \in \{p, s \}, \tag {48}
$$
where  $\alpha^{(p)}\leq \alpha^{(s)}$  is set such that the plastic model undergoes rapid updates, allowing it to swiftly adapt to newly acquired knowledge, while the stable model maintains a "long-term memory" spanning multiple tasks. Throughout training, CLS-ER assesses the certainty generated by both history models and employs the logits from the more certain model as the target for knowledge distillation.
In the general formulation of the UDIL framework, the history model  $H_{t-1}$  is not required to be a single model with the same architecture as the current model  $h$ . In fact, if there are no constraints on memory consumption, we have the flexibility to train and preserve a domain-specific model  $H_i$  for each domain  $i$ . During testing, we can simply select the prediction with the highest certainty from each domain-specific model. From this perspective, the "two-history-model system" employed in CLS-ER can be viewed as a specific and limited version of the all-domain history models. Consequently, we can combine the two models used in CLS-ER into a single history model  $H_{t-1}$  as follows:
$$
H _ {t - 1} (\boldsymbol {x}) \triangleq \left\{ \begin{array}{l l} H _ {t - 1} ^ {(p)} (\boldsymbol {x}) & \text {i f} \left[ H _ {t - 1} ^ {(p)} (\boldsymbol {x}) \right] _ {y} > \left[ H _ {t - 1} ^ {(s)} (\boldsymbol {x}) \right] _ {y} \\ H _ {t - 1} ^ {(s)} (\boldsymbol {x}) & \text {o . w .} \end{array} \right. \tag {49}
$$
where  $(\pmb{x},y)\in \mathcal{M}$  is an arbitrary exemplar stored in the memory bank.
At each iteration of training, CLS-ER samples a mini-batch  $B_{t}$  from the current domain and a mini-batch  $B_{t}^{\prime}$  from the episodic memory. It then concatenates  $B_{t}$  and  $B_{t}^{\prime}$  for the cross entropy loss minimization with the ground-truth labels, and uses  $B_{t}^{\prime}$  to minimize the MSE loss between the logits of  $h$  and  $H_{t - 1}$ . To align the loss formulation of CLS-ER with that of ESM-ER [80], here we consider the scenarios where the losses evaluated on  $B_{t}$  and  $B_{t}^{\prime}$  are individually calculated, i.e., we consider  $\widehat{\ell}_{B_t}(h) + \widehat{\ell}_{B_t'}(h)$  instead of  $\widehat{\ell}_{B_t\cup B_t'}(h)$ . Based on the assumption from [34], the MSE loss on the logits is equivalent to the cross-entropy loss on the predictions under certain conditions. Therefore, following the same balanced sampling strategy assumptions as in ER, the original CLS-ER training objective can be transformed as follows:
$$
\begin{array}{l} \mathcal {L} _ {\mathrm {C L S - E R}} (h) = \widehat {\ell} _ {B _ {t}} (h) + \widehat {\ell} _ {B _ {t} ^ {\prime}} (h) + \lambda \widehat {\ell} _ {B _ {t} ^ {\prime}} (h, H _ {t - 1}) (50) \\ = \widehat {\ell} _ {B _ {t}} (h) + \sum_ {i = 1} ^ {t - 1} \frac {1}{t - 1} \widehat {\ell} _ {B _ {i} ^ {\prime}} (h) + \sum_ {i = 1} ^ {t - 1} \frac {\lambda}{t - 1} \widehat {\ell} _ {B _ {i} ^ {\prime}} (h, H _ {t - 1}). (51) \\ \end{array}
$$
Therefore, by imposing the constraint  $\alpha_{i} + \beta_{i} + \gamma_{i} = 1$ , we find that  $\lambda = t - 2$ . Substituting this value back into  $\lambda / t - 1$  yields the equivalence that CLS-ER corresponds to  $\{\alpha_{i} = \lambda / \lambda + 1, \beta_{i} = 0, \gamma_{i} = 1 / \lambda + 1\}$  in UDIL, where  $\lambda = t - 2$  (Row 6 in Table 4).
Error Sensitivity Modulation based Experience Replay (ESM-ER) [80] builds upon CLS-ER by incorporating an additional error sensitivity modulation module. The primary goal of ESM-ER is to mitigate sudden representation drift caused by excessively large loss values during current-domain learning. Let's consider  $(\boldsymbol{x},y)\sim \mathcal{D}_t$ , which represents a sample from the current domain batch. In ESM-ER, the cross-entropy loss value of this sample is evaluated using the stable model  $H_{t - 1}^{(s)}$  and can be expressed as:
$$
\ell (\boldsymbol {x}, y) = - \log \left(\left[ H _ {t - 1} ^ {(s)} (\boldsymbol {x}) \right] _ {y}\right). \tag {52}
$$
To screen out those samples with a high loss value, ESM-ER assigns each sample a weight  $\lambda$  by comparing the loss with their expectation value, for which ESM-ER uses a running estimate  $\mu$  as its replacement. This can be formulated as follows:
$$
\lambda (\boldsymbol {x}) = \left\{ \begin{array}{l l} 1 & \text {i f} \ell (\boldsymbol {x}, y) \leq \beta \cdot \mu \\ \frac {\mu}{\ell (\boldsymbol {x} , y)} & \text {o . w .} \end{array} \right. \tag {53}
$$
where  $\beta$  is a hyperparameter that determines the margin for a sample to be classified as low-loss. For the sake of analysis, we make the following assumptions: (i)  $\beta = 1$ ; (ii) the actual expected loss value  $\mathbb{E}_{\boldsymbol{x},y}[\ell (\boldsymbol{x},y)]$  is used instead of the running estimate  $\mu$ ; (iii) a hard screening mechanism is employed instead of the current re-scaling approach. Based on these assumptions, we determine the sample-wise weights  $\lambda^{\star}$  according to the following rule:
$$
\lambda^ {\star} (\boldsymbol {x}) = \left\{ \begin{array}{l l} 1 & \text {i f} \ell (\boldsymbol {x}, y) \leq \mathbb {E} _ {\boldsymbol {x}, y} [ \ell (\boldsymbol {x}, y) ] \\ 0 & \text {o . w .} \end{array} \right. \tag {54}
$$
Under the assumption that the loss value  $\ell(\boldsymbol{x}, y)$  follows an exponential distribution, denoted as  $\ell(\boldsymbol{x}, y) \sim \mathrm{Exp}(\lambda_0)$ , where the probability density function is given by  $f(\ell(\boldsymbol{x}, y), \lambda_0) = \lambda_0 e^{-\lambda_0 \ell(\boldsymbol{x}, y)}$ , we can calculate the expectation of the loss as  $\mathbb{E}_{\boldsymbol{x}, y}[\ell(\boldsymbol{x}, y)] = 1 / \lambda_0$ . Based on this, we can now determine the expected ratio  $r$  of the unscreened samples in a mini-batch using the following equation:
$$
r = \int_ {0} ^ {\frac {1}{\lambda_ {0}}} 1 \cdot \lambda_ {0} e ^ {- \lambda_ {0} \ell (\boldsymbol {x}, y)} d \ell (\boldsymbol {x}, y) = \int_ {0} ^ {1} e ^ {- y} d y = (1 - e ^ {- 1}). \tag {55}
$$
The ratio  $r$  represents the proportion of effective samples in the current-domain batch, as the weights  $\lambda^{\star}(\pmb{x})$  of the remaining samples are set to 0 due to their high loss value. Consequently, the original training loss of ESM-ER can be transformed as follows:
$$
\begin{array}{l} \mathcal {L} _ {\mathrm {E S M - E R}} (h) = r \cdot \widehat {\ell} _ {B _ {t}} (h) + \widehat {\ell} _ {B _ {t} ^ {\prime}} (h) + \lambda \widehat {\ell} _ {B _ {t} ^ {\prime}} (h, H _ {t - 1}) (56) \\ = r \cdot \widehat {\ell} _ {B _ {t}} (h) + \sum_ {i = 1} ^ {t - 1} \frac {1}{t - 1} \widehat {\ell} _ {B _ {i} ^ {\prime}} (h) + \sum_ {i = 1} ^ {t - 1} \frac {\lambda}{t - 1} \widehat {\ell} _ {B _ {i} ^ {\prime}} (h, H _ {t - 1}). (57) \\ \end{array}
$$
After applying the constraint of  $\alpha_{i} + \beta_{i} + \gamma_{i} = 1$ , we obtain  $\lambda = r \cdot (t - 1) - 1$ . Substituting this value back into  $\lambda / r(t - 1)$ , we find that ESM-ER is equivalent to  $\{\alpha_{i} = \lambda / \lambda + 1, \beta_{i} = 0, \gamma_{i} = 1 / \lambda + 1\}$  in UDIL, where  $\lambda = r \cdot (t - 1) - 1 = (1 - e^{-1})(t - 1) - 1$  should be set (Row 7 in Table 4).
Bias Correction (BiC) [100] was the first to apply class incremental learning at a large scale, covering extensive image classification datasets including ImageNet (1,000 classes, [76]) and MSCeleb-1M (10,000 classes, [31]). Similar to iCaRL [74], BiC treats each data example (from new classes' data and the memory) with the same level of importance. Different from its previous work, the distillation loss in BiC ( $L_{d}$  in the original paper) implicitly includes the cross-domain distillation loss described by Lemma 3.3, as it computes the old classifier's output evaluated on the new data (considering only old classes). In addition, BiC further re-balances the classification loss ( $L_{c}$  in the original paper) and the distillation loss ( $L_{d}$ ) based on the number of old classes  $n$  and new classes  $m$  as follows:
$$
\mathcal {L} _ {\mathrm {B i C}} = \frac {n}{n + m} \cdot L _ {d} + \frac {m}{n + m} \cdot L _ {c}. \tag {58}
$$
Figure 2: Diagram of UDIL.  $\mathcal{M} = \{M_i\}_{i=1}^{t-1}$  represents the memory bank that stores all the past exemplars.  $S_t$  corresponds to the dataset from the current domain  $t$ , and the current model  $h = p \circ e$  is depicted separately in the diagram. The three different categories of losses are illustrated in the dark rectangles, while the weighting effect of the learned replay coefficient  $\Omega = \{\alpha_i, \beta_i, \gamma_i\}_{i=1}^{t-1}$  is depicted using dashed lines.
By interpreting the distillation loss  $L_{d}$  as the combination of the intra-domain loss (Lemma 3.2) and the cross-domain distillation loss (Lemma 3.3), and substituting  $(n,m)$  with  $(t - 1,1)$  due to the consistent number of classes in each domain, we arrive at the BiC model for DIL:
$$
\begin{array}{l} \mathcal {L} _ {\mathrm {B i C}} (h) = \frac {t - 1}{t} \left[ N _ {t} \cdot \widehat {\ell} _ {\mathcal {X} _ {t}} (h, H _ {t - 1}) + \sum_ {i = 1} ^ {t - 1} \widetilde {N} _ {i} \cdot \widehat {\ell} _ {\mathcal {X} _ {i}} (h, H _ {t - 1}) \right] \\ + \frac {1}{t} \left[ N _ {t} \cdot \widehat {\ell} _ {\mathcal {X} _ {t}} (h) + \sum_ {i = 1} ^ {t - 1} \widetilde {N} _ {i} \cdot \widehat {\ell} _ {\mathcal {X} _ {i}} (h) \right]. \tag {59} \\ \end{array}
$$
The equation above relies on the number of the current-domain examples and the exemplars (i.e., memory), which is not constant in practice. Hence we replace  $(N_{t},\widetilde{N}_{i})$  with the mini-batch size  $(|B_t|,|B_i|)$ . After re-organizing the equation above, we have
$$
\begin{array}{l} \mathcal {L} _ {\mathrm {B i C}} (h) = \widehat {\ell} _ {B _ {t}} (h) + \sum_ {i = 1} ^ {t - 1} \frac {(t - 1) | B _ {i} |}{| B _ {t} |} \cdot \widehat {\ell} _ {B _ {i} ^ {\prime}} (h, H _ {t - 1}) \\ + (t - 1) \cdot \widehat {\ell} _ {B _ {t}} (h, H _ {t - 1}) + \sum_ {i = 1} ^ {t - 1} \frac {\left| B _ {i} \right|}{\left| B _ {t} \right|} \cdot \widehat {\ell} _ {B _ {i} ^ {\prime}} (h). \tag {60} \\ \end{array}
$$
The immediate observation is that  $BiC$  exhibits a significant bias towards retaining knowledge from past domains. This is evident in the coefficient of coefficient of cross-domain distillation summed over the past domains  $\sum_{i=1}^{t-1} \beta_i = (t-1)$ , which violates the constraints posed in this work ( $\alpha_i + \beta_i + \gamma_i = 1$ ). However, if we slightly relax BiC's formulation and focus on the relative ratios of the three coefficients, we get  $\alpha_i : \beta_i : \gamma_i = (t-1)B_i : (t-1)B_t : B_i$ . Further applying the constraint  $\alpha_i + \beta_i + \gamma_i = 1$  and assuming  $B_i = B_t$  to enforce a balanced sampling strategy over different domains, we arrive at  $\{\alpha_i = \beta_i = t^{-1}/2t^{-1}, \gamma_i = 1/2t^{-1}\}$  (Row 8 in Table 4).
C Implementation Details of UDIL
This section delves into the implementation details of the UDIL algorithm. The algorithmic description of UDIL is presented in Algorithm 1 and a diagram is presented in Fig. 2. However, there are several practical issues to be further addressed here, including (i) how to exert the constraints of probability simplex  $([\alpha_{i},\beta_{i},\gamma_{i}]\in \mathbb{S}^{2})$  and (ii) how the memory is maintained. These two problems will be addressed in Sec. C.1 and Sec. C.2. Then, Sec. C.3 will introduce two auxiliary losses that improve the stability and domain alignment for the encoder during training. Next, Sec. C.4
will cover the evaluation metrics used in this paper. Finally, Sec. C.5 and Sec. C.6 will present a detailed introduction to the main baselines and the specific training schemes we follow for empirical evaluation.
C.1 Modeling the Replay Coefficients  $\Omega = \{\alpha_{i},\beta_{i},\gamma_{i}\}$
Instead of directly modeling  $\Omega$  in a way such that it can be updated by gradient descent and satisfies the constraints that  $\alpha_{i} + \beta_{i} + \gamma_{i} = 1$  and  $\alpha_{i},\beta_{i},\gamma_{i}\geq 0$  at the same time, we use a set of logit variables  $\{\bar{\alpha}_i,\bar{\beta}_i,\bar{\gamma}_i\} \in \mathbb{R}^3$  and the softmax function to indirectly calculate  $\Omega$  during training. Concretely, we have:
$$
\left[ \begin{array}{l} \alpha_ {i} \\ \beta_ {i} \\ \gamma_ {i} \end{array} \right] = \operatorname {s o f t m a x} \left(\left[ \begin{array}{l} \bar {\alpha} _ {i} \\ \bar {\beta} _ {i} \\ \bar {\gamma} _ {i} \end{array} \right]\right) = \left[ \begin{array}{l} \exp \left(\bar {\alpha} _ {i}\right) / Z _ {i} \\ \exp \left(\bar {\beta} _ {i}\right) / Z _ {i} \\ \exp \left(\bar {\gamma} _ {i}\right) / Z _ {i} \end{array} \right], \tag {61}
$$
where  $Z_{i} = \exp (\bar{\alpha}_{i}) + \exp (\bar{\beta}_{i}) + \exp (\bar{\gamma}_{i})$  is the normalizing constant. At the beginning of training on domain  $t$ , the logit variables  $\{\bar{\alpha}_i,\bar{\beta}_i,\bar{\gamma}_i\} = \{0,0,0\}$  are initialized to all zeros, since we do not have any bias towards any upper bound. During training, they are updated in the same way as the other parameters with gradient descent.
C.2 Memory Maintenance with Balanced Sampling
Different from DER++ [8] and its following work [4, 80] that use reservoir sampling [93] to maintain the episodic memory, UDIL adopts a random balanced sampling after training on each domain. To be more concrete, given a memory bank with fixed size  $|\mathcal{M}|$ , after domain  $t$ 's training is complete, we will assign each domain a quota of  $| \mathcal{M} | / t$ . For the current domain  $t$ , we will randomly sample  $\lfloor |\mathcal{M}| / t \rfloor$  exemplars from its dataset; for all the previous domains  $i \in [t - 1]$ , we will randomly swap out  $\lceil |\tilde{\mathcal{M}}| / t - 1 - |\mathcal{M}| / t \rceil$  exemplars from the memory to make sure each domain has roughly the same number of exemplars. To ensure a fair comparison, we use the same random balanced sampling strategy for all the other baselines. The following Algorithm 2 shows the detailed procedure of random balanced sampling.
Algorithm 2 Balanced Sampling for UDIL
Require: memory bank  $\mathcal{M} = \{M_i\}_{i=1}^{t-1}$ , current domain dataset  $S_t$ , domain ID  $t$ .  
1: for  $i = 1, \dots, t-1$  do  
2: for  $j = 1, \dots, \lceil |\mathcal{M}| / t - 1 - |\mathcal{M}| / t \rceil$  do  
3:  $(x, y) \gets \text{RandomSample}(M_i)$   
4:  $(x', y') \gets \text{RandomSample}(\mathcal{S}_t)$   
5: Swap  $(x', y')$  into  $\mathcal{M}$ , replacing  $(x, y)$   
6: end for  
7: end for  
8: return  $\mathcal{M}$
C.3 Improving Stability and Domain Alignment for the Embedding Distribution
In this section, we will examine the training loss employed to align the embedding distribution across distinct domains. As discussed in Sec. 4.2, we decompose the model  $h$  into an encoder  $e$  and a predictor  $p$  as  $h = p \circ e$ . In order to establish a tighter upper bound proposed in Theorem 3.4, we introduce a discriminator  $d$  that aims to distinguish embeddings based on domain IDs. Specifically, during the training process of UDIL, given a set of coefficients  $\Omega = \{\alpha_i, \beta_i, \gamma_i\}$ , both the encoder  $e$  and the discriminator  $d$  engage in the following minimax game:
$$
\min  _ {e} \max  _ {d} - \lambda_ {d} V _ {d} (d, e, \stackrel {\circ} {\Omega}), \tag {62}
$$
where  $(\cdot)$  represents the "copying-weights-and-stopping-gradients" operation, and  $\lambda_{d}$  is a hyperparameter introduced to control the strength of domain embedding alignment. More specifically, the
value function of the mini-max game  $V_{d}$  is defined as follows:
$$
V _ {d} (d, e, \overset {\circ} {\Omega}) = \left(\sum_ {i = 1} ^ {t - 1} \overset {\circ} {\beta} _ {i}\right) \frac {1}{N _ {i}} \sum_ {\boldsymbol {x} \in \mathcal {S} _ {t}} [ - \log \left([ d (e (\boldsymbol {x})) ] _ {t}\right) ] + \sum_ {i = 1} ^ {t - 1} \overset {\circ} {\underset {\tilde {N} _ {i}} {\beta}} \sum_ {\boldsymbol {x} \in \tilde {\mathcal {X}} _ {i}} [ - \log \left([ d (e (\boldsymbol {x})) ] _ {i}\right) ]. \tag {63}
$$
As previously mentioned, the practical effect of Eqn. 62 is to align the embedding distribution of different domains, thus enhancing the model's generalization ability to both previously encountered domains and those that may be encountered in the future.
However, actively altering the embedding space can lead to the well-known stability-plasticity dilemma [43, 96, 20, 39, 61]. This dilemma arises when the model needs to modify a significant number of parameters to achieve domain alignment for a new domain, potentially resulting in a mismatch between the predictor  $p$  and the encoder  $e$ , which, in turn, can lead to catastrophic forgetting. Furthermore, it is worth noting that the adversarial training scheme described in Eqn. 62 is primarily designed for unsupervised domain alignment [108, 57, 26, 109, 17, 102, 101, 94], where the semantic labels in the target domain(s) are not available during training. This implies that the current adversarial training technique does not fully leverage the label information in domain incremental learning.
To tackle the aforementioned challenges, i.e., (i) maintaining a stable embedding distribution across all domains and (ii) accelerating domain alignment with label information, we incorporate two additional auxiliary losses: (i) the past embedding distillation loss [40, 65] and (ii) the supervised contrastive loss [42, 29]. It's important to note that these auxiliary losses, in conjunction with the adversarial feature alignment loss  $V_{d}$ , operate exclusively on the encoder space of the model and do not impact the original objectives outlined in Theorem 3.4, provided that encoder  $e$  and predictor  $p$  remain sufficiently strong. Therefore, the two losses are used to simply stabilize the training, without compromising the theoretical significance of our work.
In past embedding distillation, also known as representation(al) distillation or embedding distillation in previous work on continual learning [40, 65, 59], the model stores the embeddings of the memory after being trained on domain  $t - 1$ . It then uses them to constrain the encoder's behavior: the features produced on these samples should not change too much during the current domain training. After decoupling the history model  $H_{t - 1} = P_{t - 1} \circ E_{t - 1}$ , where  $E_{t - 1}$  is the history encoder and  $P_{t - 1}$  is history predictor, we define the past embedding distillation loss  $V_{p}$  as
$$
V _ {p} (e) = \sum_ {i = 1} ^ {t - 1} \mathbb {E} _ {\boldsymbol {x} \sim \mathcal {D} _ {i}} \left[ \| e (\boldsymbol {x}) - E _ {t - 1} (\boldsymbol {x}) \| _ {2} ^ {2} \right]. \tag {64}
$$
The loss above promotes the stability of the embedding distribution from past domains. When combined with the adversarial embedding alignment loss in Eqn. 62, it encourages the embedding distribution of the current domain to match those of the previous domains, but not vice versa.
As supervised variations of contrastive learning [12, 32, 14, 83, 65, 13, 84, 85], the supervised contrastive loss [42, 29] will compensate for the fact that Eqn. 62 does not utilize the label information to align different domains, and therefore lack in the efficiency of aligning two domains' embedding distribution. The supervised contrastive learning pulls together the embeddings of the same label and pushes apart those with distinct labels. Notably, it is done in a "domain-agnostic" manner, i.e., the domain labels are not considered. Denoting as  $\mathcal{P} = \frac{1}{t}\sum_{i}^{t}\mathcal{D}_{i}$  the combined data distribution of all domains and as  $\mathcal{P}_{\cdot |y}$  the data distribution given the class label  $y\in [K]$ , the supervised contrastive loss for embedding alignment is defined as follows:
$$
V _ {s} (e) = \mathbb {E} _ {y} \mathbb {E} _ {\substack {(\boldsymbol {x} _ {1}, \boldsymbol {x} _ {2}) \sim \mathcal {P} _ {\cdot | y} \\ \left\{\boldsymbol {u} _ {i} \right\} _ {i = 1} ^ {B} \sim \mathcal {P}}} \left[ - \log \frac {\exp \left\{- s _ {e} (\boldsymbol {x} _ {1} , \boldsymbol {x} _ {2}) \right\}}{\exp \left\{- s _ {e} (\boldsymbol {x} _ {1} , \boldsymbol {x} _ {2}) \right\} + \sum_ {i = 1} ^ {B} \exp \left\{- s _ {e} (\boldsymbol {x} _ {1} , \boldsymbol {u} _ {i}) \right\}} \right], \tag{65}
$$
where  $s_e(\pmb{u}, \pmb{v}) = \| e(\pmb{u}) - e(\pmb{v})\|_2^2$  is the squared Euclidean distance for any  $\pmb{u}, \pmb{v} \in \mathcal{X}$ .
Introducing the supervised contrastive loss to continual learning is novel, as existing methods often harness its ability to create a compact representation space, thereby mitigating representation overlapping in class incremental learning [65, 30, 9, 96]. However, in this work, the primary motivation for using the supervised contrastive loss to facilitate domain alignment lies in its optimal solution [29, 110, 19]. This optimal point referred to as neural collapse [47, 69, 111, 103], where the embeddings of the same class collapse to the same point, and those of different classes are sparsely distributed. It is easy to envision that when the optimal state of the supervised contrastive loss is attained across different domains, it concurrently achieves perfect domain alignment.
The loss function that fosters stability and domain alignment in the encoder  $e$  can be summarized as follows:
$$
\mathcal {L} _ {\mathrm {e n c}} (e) = - \lambda_ {d} V _ {d} (d, e, \overset {\circ} {\Omega}) + \lambda_ {p} V _ {p} (e) + \lambda_ {s} V _ {s} (e). \tag {66}
$$