## 2 Related Works
### Deep Domain Adaptation Approaches
#### 2.2.1 Traditional Deep Domain Adaptation  
Traditional deep domain adaptation approaches, as classified by the representation of domain discrepancy [28], are divided into two main categories: (i) discrepancy-based and (ii) reconstruction-based [29; 30].  
Discrepancy-based approaches focus on aligning domain distributions through the minimization of a distance metric within deep network architectures. For instance, ref. [31] introduces the Deep Adaptation Network (DAN), which employs a kernel-based approach utilizing Reproducing Kernel Hilbert Space (RKHS) to effectively align higher-order statistics of the distributions. DAN utilizes the Multi-Kernel of Maximum Mean Discrepancy (MK-MMD) to measure RKHS distance between the mean embedding of source and target distributions, emphasizing marginal distribution alignment without considering conditional distribution discrepancy. Alternatively, another MMD-based distribution matching technique is proposed by [32], Deep Transfer Network (DTN). DTN considers both the conditional and marginal distributions. Unlike using the MMD-based distribution matching [33], DTN utilizes the joint distribution difference of multiple layers to learn the transferable features by proposing the Joint Adaptation Network (JAN). It is a form of unsupervised domain adaptation. However, training and implementing DTN might be computationally expensive and complex due to its utilization of multiple layers and joint distribution matching. Another UDA approach, Deep CORAL (DCORAL), adopts the correlation alignment (CORAL) discrepancy measure to achieve deep domain adaptation.  
DCORAL aims to learn a representation shared by the source and the target by learning the non-linear transformation that aligns the correlations of the layers. It is designed for unsupervised domain adaptation. However, DCORAL assumes a linear transformation to align correlations, which may not capture highly non-linear relationships between domains.  
Reconstruction-based approaches, on the other hand, are centered around mitigating distribution disparities between domains by minimizing the reconstruction error while acquiring a domain-invariant representation in an intermediate feature space representation [23; 26; 34; 35]. Auto-encoders play a crucial role in this category, where encoder parameters are learned based on source domain samples, and a decoder is utilized to reconstruct target domain samples [26]. A specific instance of this category is the Separation Network (DSN) introduced by [36], which utilizes a scale-invariant Mean Squared Error (MSE) reconstruction loss to enhance the learning of invariant representations. Notably, DSN is designed for use exclusively in supervised domain adaptation scenarios. Another approach in this category is Transfer Learning with Deep Auto-encoders (TLDA) proposed by [29; 30]. TLDA employs two auto-encoders for the source and target domains. It utilizes the auto-encoder as a feature extractor for final predictions, and the target classifier is trained on labeled examples using the feature representation accessible by the first-layer output of the encoder. However, it necessitates labeled examples for the target domain to train the target classifier, limiting its application in fully unsupervised scenarios. Additionally, Causal Auto-Encoder (CAE), presented by [23], integrates deep auto-encoder and causal structure learning to derive causal representations using available data. CAE is typically applied in unsupervised domain adaptation settings. One notable drawback of auto-encoder-based techniques is the limited incorporation of linguistic information in the induced representations, potentially restricting their interpretability in language-related tasks.