## 4 Dictionary Pair Learning Algorithm
### Sparse coding phase  
In this subsection, we discuss how to calculate the sparse coefficients \(\mathbf{Y}\) and the threshold values \(\mathbf{\lambda}\) with given \(\mathbf{\Phi}\) and \(\mathbf{\Psi}\) under our SSM-NTF model. Given a pair of dictionaries \(\mathbf{\Phi}\) and \(\mathbf{\Psi}\), calculating \(\mathbf{Y}\) and \(\mathbf{\lambda}\) from \(\mathbf{X}\) is formulated as:  
\[\{\hat{\mathbf{Y}},\hat{\mathbf{\lambda}}\}=\min_{\mathbf{Y},\mathbf{\lambda}}\|\mathbf{X}-\mathbf{\Phi} \mathbf{Y}\|_{F}^{2}+\gamma_{1}\|\mathbf{Y}-\mathcal{S}_{\mathbf{\lambda}}(\mathbf{\Psi}^{T} \mathbf{X})\|_{F}^{2}+\gamma_{2}\|\mathbf{Y}\|_{0} \tag{24}\]  
We pursue the two variables alternatively. Firstly, with fixed \(\mathbf{\lambda}\), we obtain the sparse coefficients \(\mathbf{Y}\) by solving Problem (24) through OMP [9] as it can be easily convert to the classical synthesis sparse expression \(\min\|\mathbf{Z}-\mathbf{D}\mathbf{Y}\|_{F}^{2},\ s.t.\|\mathbf{Y}\|_{0}\) where \(\mathbf{Z}=[\mathbf{X}\ \sqrt{\gamma}_{1}\mathcal{S}_{\mathbf{\lambda}}(\mathbf{\Psi}^{T}\mathbf{X})]\) and \(\mathbf{D}=[\mathbf{\Phi}\ \sqrt{\gamma}_{1}\mathbf{1}]\). Secondly, the pursue of \(\mathbf{\lambda}\) is equivalent to solving the following problem  
\[\hat{\mathbf{\lambda}}=\operatorname*{arg\,min}_{\mathbf{\lambda}}\|\mathbf{Y}-\mathcal{S }_{\mathbf{\lambda}}(\mathbf{\Psi}^{T}\mathbf{X})\|_{F}^{2} \tag{25}\]  
which can be decomposed into \(M\) individual optimization problems  
\[\hat{\lambda}_{i}=\operatorname*{arg\,min}_{\lambda_{i}}\|\mathbf{\overline{y}}_{ i}-\mathcal{S}_{\lambda_{i}}(\mathbf{\psi}_{i}^{T}\mathbf{X})\|_{2}^{2},i=1,\ldots,M. \tag{26}\]  
where \(\mathbf{\psi}_{i}\) is the column of \(\mathbf{\Psi}\). From the definition of soft thresholding operator, we can know that the function of Problem (26) is discrete. By denoting the data indices set that remains intact after the thresholding as \(\mathcal{J}_{i}\), we split the data \(\mathbf{X}\) into two parts: \(\mathbf{X}^{\mathcal{J}_{i}}\) and \(\mathbf{X}^{\hat{\mathcal{J}}_{i}}\) such that  
\[\mathcal{S}_{\lambda_{i}}(\mathbf{\psi}_{i}^{T}\mathbf{X}^{\mathcal{J}_{ i}}) =\mathbf{\psi}_{i}^{T}\mathbf{X}^{\mathcal{J}_{i}}-sgn(\mathbf{\psi}_{i}^{T} \mathbf{X}^{\mathcal{J}_{i}})\lambda_{i} \tag{27}\] \[\mathcal{S}_{\lambda_{i}}(\mathbf{\psi}_{i}^{T}\mathbf{X}^{\hat{\mathcal{ J}}_{i}}) =0. \tag{28}\]  
where \(\hat{\mathcal{J}}_{i}\) is a supplementary to the intact indices \(\mathcal{J}_{i}\) which turn the all elements to zero. It is clear to know that the variables \(\mathcal{J}_{i}\) and \(\hat{\mathcal{J}}_{i}\) are both functions of \(\lambda_{i}\) without explicit expressions which leads to a large challenge in optimization. In order to solve Problem (26), an intermediate variable \(\mu_{i}\) is necessarily to introduced to separate the whole problem into two parts: The update of the indices \(\mathcal{J}_{i}\) and \(\hat{\mathcal{J}}_{i}\) (determined by \(\mu_{i}\)) and the update of the explicit thresholding value \(\lambda_{i}\). Then Problem (26) can be transformed to another optimization problem:  
\[\{\hat{\lambda}_{i},\hat{\mu}_{i}\}=\operatorname*{arg\,min}_{\lambda_{i}}\| \mathbf{\overline{y}}_{i}^{\hat{\mathcal{J}}_{i}}\|_{2}^{2}+\|\mathbf{\overline{y}}^{ \mathcal{J}_{i}}-[\mathbf{\psi}_{i}^{T}\mathbf{X}^{\mathcal{J}_{i}}-sgn(\mathbf{\psi}_{i}^ {T}\mathbf{X}^{\mathcal{J}_{i}})\lambda_{i}]\|_{2}^{2}+\tau/2\|\lambda_{i}-\mu_{i} \|_{2}^{2} \tag{29}\]  
where \(\mathcal{J}_{i}\) and \(\hat{\mathcal{J}}_{i}\) are two functions of the intermediate variable \(\mu_{i}\). At the \(k\)-th step, to obtain \(\mu_{i}\), we solve Problem (29) with \(\lambda_{i}\) fixed and denote the functions as \(f(\mu_{i})+g(\mu_{i})+l(\mu_{i})\) where \(f(\mu_{i})=\|\mathbf{\overline{y}}_{i}^{\hat{\mathcal{J}}_{i}}\|_{2}^{2}\), \(g(\mu_{i})=\|\mathbf{\overline{y}}^{\mathcal{J}_{i}}-[\mathbf{\psi}_{i}^{T}\mathbf{X}^{ \mathcal{J}_{i}}-sgn(\mathbf{\psi}_{i}^{T}\mathbf{X}^{\mathcal{J}_{i}})\lambda_{i}]\|_ {2}^{2}\), \(l(\mu_{i})=\tau/2\|\lambda_{i}-\mu_{i}\|_{2}^{2}\).