# Unsupervised Domain Adaptation via Stacked Convolutional Autoencoder +
## 4 Our Proposed Method
### Stacked Sparse Autoencoder  
The first component is composed of a sparse autoencoder with a whitening layer that learns the latent feature representations from the data in the source domain. As the target domain is unlabeled in unsupervised domain adaptation, the source domain \(D_{s}\) with labeled data and target domain \(D_{t}\) with unlabeled data is \(D_{s}=\{x_{i}^{(s)},y_{i}^{(s)}\}|_{i=1}^{n_{s}}\) with \(x_{i}^{(s)}\in R^{m\times 1}\) and \(D_{t}=\{x_{i}^{(t)}\}|_{i=1}^{n_{t}}\), where \(x_{i}^{(s)}\in R^{m\times 1}\) and \(x_{i}^{(t)}\in R^{m\times 1}\) denote the instances in the source and target domain and \(n_{s}\) and \(n_{t}\) denote the number of instances in the source and target domain, respectively; \(y_{i}^{(s)}\in\{1,2,...,c\}\) denotes the label information in the source domain, \(m\) denotes the feature dimension of the input data and \(c\) denotes the number of labels. In a sparse autoencoder, at the encoder stage, the data from both the source and the overall target domains are projected onto vectors in the hidden layer, respectively, expressed as \(\xi^{(s)}\) and \(\xi^{(t)}\). Then, in the decoder stage, the \(\xi^{(s)}\) and \(\xi^{(t)}\) are mapped to the output layer as \(\hat{x}_{i}^{(s)}\) and \(\hat{x}_{i}^{(t)}\). To obtain more powerful feature representations for knowledge transfer, we introduce a softmax encoder weight regularization to apply the labeled information in the source domain to train the whole model. First, we introduce the RICA algorithm to perform the independent component analysis from the original data in both source and target domains. We utilize the whitening layer before the RICA to make the input less redundant. The objective function can be shown as (1):  
\[J_{\text{RICA}}(W_{R})=\lambda\|W_{R}X\|_{1}+\frac{1}{m}\Big{\|}W_{R}^{\text{T }}W_{R}X-X\Big{\|}_{2}^{2} \tag{1}\]  
\(X\) denotes the original data in both source and target domains, \(W_{R}\) denotes the weight matrix, and \(\lambda\) are the tuning parameters. To scale the reconstruction item, i.e., the second item in (1), the \(L1\) regularization expressed as \(f(x)=\sqrt{(WX)^{2}+\varepsilon}\) is introduced to (1). In our method, we select \(\varepsilon=0.1\) as a small constant to prevent the \(L1\) regularization item from being numerically close to zero. Thus, (1) can be expressed as (2):  
Figure 1: Illustration of our proposed SCSA. Each layer is composed of two main components: a stacked sparse autoencoder and the convolution and pooling kernels. The whitening layer is first introduced for recognition feature learning. \[J_{\text{RICA}}(W_{R})=\lambda\sum\left(\sqrt{(W_{R}x_{i})^{2}+\varepsilon}\right)+ \tag{2}\] \[ \begin{array}{c}\frac{1}{2n}\sum\limits_{i=1}^{n}\left(\frac{1}{m} \big{\|}W_{R}^{T}W_{R}x_{i}-x_{i}\big{\|}_{2}^{2}\right)\end{array} \]  
The partial derivatives of \(J_{\text{RICA}}(W_{R})\) can be formalized as (3):  
\[ \begin{array}{c}\
abla_{W_{R}}J_{\text{RICA}}(W_{R})=\frac{2}{m}(W_{R}(W_{R} ^{T}W_{R}X-X)X^{T}\\ \qquad+(W_{R}X)(W_{R}^{T}W_{R}X-X)^{T})\\ \qquad+\lambda((W_{R}X)^{2}+\varepsilon)^{-1}\!\!/2(W_{R}X)X^{T}\end{array}  \tag{3}\]  
According to the partial derivatives of (3), the output \(W_{R}^{T}W_{R}X\) of the RICA is fed into the next autoencoder as the input. After the RICA, we introduce the stacked sparse autoencoder with softmax weight regression to learn more abstract features across the source and target domains. In the stacked sparse autoencoder, the desired partial derivatives regrading \(W\) and \(b\) can be shown as (4) and (5):  
\[\
abla_{W(l)}J(W,b)=\delta^{(l+1)}(\xi^{(l)})^{T} \tag{4}\]  
\[\
abla_{b^{(l)}}J(W,b)=\delta^{(l+1)} \tag{5}\]  
\(W^{(l)}\), \(b^{(l)}\) and \(\xi^{(l)}\) are the weight matrix, bias vector and output of the \(l^{th}\) hidden level in the autoencoder, respectively.