# Fuzzy Graph Learning Regularized Sparse Filtering for Visual Domain Adaptation
## 3 Methodology
### Sparse Filtering for Discriminative Feature Learning  
Fuzzy graph learning is able to reduce the domain discrepancy, but it is an ill-posed problem in nature. For instance, if we choose two constant function, \(k_{1}(x)=a,k_{2}(x)=b\), _where_\(a\
eq b\), they both can minimize the objective \(L_{FGL}\). For the intuitive example shown in Figure 1, we can see that unreliable samples lead to representation shrinkage. Consequently, the learned representation will lose the discriminability, hence an additional penalty is indispensable. In this paper, we adopt the objective of sparse filtering for two reasons: Firstly, it requires lifetime sparsity, which projects the feature onto a L2-ball, this can effectively alleviate the feature contraction. Secondly, it allows us to exploit the potential discriminant structures in unlabeled target domain.  
Corresponding to the optimization of sparse filtering, the transformation function \(k(\cdot)\) of our method can be summarized as follows: Given original input \(x_{s},x_{t}\in R^{n_{s},n_{t}\times m}\), the goal is to find optimal \(W\in R^{m\times k}\), then obtain new representations \(\hat{x}_{s},\hat{x}_{t}\in R^{n_{s},n_{t}\times k}\) for downstream applications.  
Linear transformation and non-non-linear activation. We use the soft absolute function as activation function in this paper.  
\[\begin{split}\hat{x}_{s1}&=|x_{s}W+\epsilon|\\ \hat{x}_{t1}&=|x_{t}W+\epsilon|\end{split} \tag{8}\]  
where \(\epsilon\) denotes a small number, such as 1e-5.  
L2 column normalization Observing the form of current \(\hat{x}\), each row represents a sample, and each column represents a feature. This measure enforces the L2 norm of each feature to be 1 thus high dispersal can be achieved. Noted that we do the within-domain normalization since it is proven to be powerful for DA problems.  
\[\begin{split}\hat{x}_{s2}&=\hat{x}_{s1}\circ M_{ scol},\ (M_{scol})_{ij}=\frac{1}{\sqrt{\sum\limits_{i=1}^{n_{s}}(\hat{x}_{s1})_{ij}^{2}}} \\ \hat{x}_{t2}&=\hat{x}_{t1}\circ M_{tcol},\ (M_{tcol})_{ij}= \frac{1}{\sqrt{\sum\limits_{i=1}^{n_{t}}(\hat{x}_{t1})_{ij}^{2}}} \end{split} \tag{9}\]  
which the symbol \(\circ\) represents Hadamard product. Here we use Hadamard product to simplify gradient derivation.  
L2 row normalization. Just like the previous step, this measure enforce the L2 norm of each sample to be 1 to project data onto L2-ball.  
\[\begin{split}\hat{x}_{s3}&=\hat{x}_{s2}\circ M_{ srow},\ (M_{srow})_{ij}=\frac{1}{\sqrt{\sum\limits_{j=1}^{k}(\hat{x}_{s2})_{ij}^{2}}}\\ \hat{x}_{t3}&=\hat{x}_{t2}\circ M_{trow},\ (M_{trow})_{ ij}=\frac{1}{\sqrt{\sum\limits_{j=1}^{k}(\hat{x}_{s2})_{ij}^{2}}}\end{split} \tag{10}\]  
For the sake of simplification, we integrate the mapping from \(x_{s},x_{s}\) to \(\hat{x}_{s3},\hat{x}_{t3}\) as a single step \(k(\cdot)\), so we can obtain the final representation by \(\hat{x}_{s}=k(x_{s})=\hat{x}_{s3},\hat{x}_{t}=k(x_{t})=\hat{x}_{t3}\). Correspondingly, the objective of sparse filtering can be represented as:  
\[L_{SF}=\frac{1}{n_{s}}||\hat{x}_{s}||_{1}+\frac{1}{n_{t}}||\hat{x}_{t}||_{1} \tag{11}\]  
The total objective of our method can be written as:  
\[L=L_{SF}+\lambda L_{FGL} \tag{12}\]  
where \(\lambda\) is hyper-parameter to balance two objectives.