# Data-Driven Redundant Transform Based on Parseval Frames
## 3 Data-Driven Redundant Transform Model Based on Tight Frame
### Transform Learning for the Drtbf Model
#### 3.2.2 Transform Pair Update Phase  
The **Y** Subproblem  
With fixed **Y** and \(\mathbf{\lambda}\), the optimization problem to obtain \(\mathbf{\Psi}\) is given by  
\[\hat{\textbf{Y}}=\operatorname*{arg\,min}_{\mathbf{\Psi}}\|\textbf{Y}-\mathcal{S }_{\mathbf{\lambda}}(\mathbf{\Psi}^{T}\textbf{X})\|_{F}^{2}+\frac{\eta_{3}}{\eta_{1}} \|\mathbf{\Phi}\mathbf{\Psi}^{T}-\textbf{I}\|_{F}^{2}. \tag{18}\]Such a problem is a highly nonlinear optimization problem, due to the definition of \(\mathcal{S}_{\lambda}\). We (columnwise) solve \(\boldsymbol{\Psi}\) by updating each column of \(\boldsymbol{\Psi}\) while fixing others. The product \(\boldsymbol{\Phi}\boldsymbol{\Psi}^{T}\) can be written as  
\[\boldsymbol{\Phi}\boldsymbol{\Psi}^{T}=\sum_{p=1}^{N}\boldsymbol{\psi}_{p} \boldsymbol{\phi}_{p}^{T}=\boldsymbol{\psi}_{i}\boldsymbol{\phi}_{i}^{T}-( \mathbf{I}-\sum_{p\
eq i}^{N}\boldsymbol{\psi}_{p}\boldsymbol{\phi}_{p}^{T}). \tag{19}\]  
For each \(\boldsymbol{\psi}_{i}\), we solve the following subproblem:  
\[\min_{\boldsymbol{\psi}_{i}}\|\overline{\mathbf{y}}_{i}-\mathcal{S}_{\lambda _{i}}(\boldsymbol{\psi}_{i}^{T}\mathbf{X})\|_{2}^{2}+\frac{\eta_{3}}{\eta_{1}} \|\boldsymbol{\psi}_{i}\boldsymbol{\phi}_{i}-\mathbf{z}\|_{2}^{2}, \tag{20}\]  
where \(\mathbf{z}=\mathbf{I}-\sum_{p\
eq i}^{N}\boldsymbol{\psi}_{p}\boldsymbol{\phi }_{p}^{T}\). We denote \(\mathcal{J}_{i}\) to be the indices (as before), and then separate the problem into the two following sub-problems:  
\[\boldsymbol{\hat{\psi}}_{i}^{1} =\operatorname*{arg\,min}_{\boldsymbol{\psi}_{i}}\sum_{j\in \mathcal{J}_{i}}(y_{ij}-\boldsymbol{\psi}_{i}^{T}\mathbf{x}_{j})^{2}+\frac{ \eta_{1}}{\eta_{3}}\|\boldsymbol{\psi}_{i}\boldsymbol{\phi}_{i}^{T}-\mathbf{ z}\|_{2}^{2}, \tag{21}\] \[\boldsymbol{\hat{\psi}}_{i}^{2} =\operatorname*{arg\,min}_{\|\boldsymbol{\psi}_{i}\|_{2}=1}\sum_ {j\in\{1,\ldots,L\}\setminus\mathcal{J}_{i}}(\boldsymbol{\psi}_{i}^{T} \mathbf{x}_{j})^{2}, \tag{22}\]  
where \(y_{ij}\) denotes the \((i,j)\)th entry of \(\mathbf{Y}\) and \(\mathbf{x}_{i}\) denotes the \(i\)th column of \(\mathbf{X}\). Equation (21) is a quadratic optimization, while Equation (22) has a closed form solution given by the normalized singular vector corresponding to the smallest singular value of \(\mathbf{X}^{\mathcal{J}}\). Based on the solutions of the two sub-problems, we give the solution of (20) as the average of the two solutions; that is, \(\hat{\boldsymbol{\psi}}_{i}=\frac{1}{2}(\boldsymbol{\hat{\psi}}_{i}^{1}+\| \boldsymbol{\hat{\psi}}_{i}^{1}\|_{2}\boldsymbol{\hat{\psi}}_{i}^{2})\). Please note that the second solution is added with the magnitude of the norm of the first solution, as (21) serves as a dominant term for the \(\Psi\) subproblem, while the solution of (22) maintains no energy but direction. The \(\mathbf{\Phi}\) Subproblem  
With fixed \(\mathbf{Y}\), \(\mathbf{\lambda}\), and \(\mathbf{\Psi}\), the model to obtain \(\mathbf{\Phi}\) is given by  
\[\min_{\mathbf{\Phi}}\|\mathbf{X}-\mathbf{\Phi}\mathbf{Y}\|_{F}^{2}+\eta_{3} \|\mathbf{\Phi}\mathbf{\Psi}^{T}-\mathbf{I}\|_{F}^{2}\] \[\text{s.t.}\ \mathbf{\Phi}\mathbf{\Phi}^{T}=\mathbf{I}. \tag{23}\]  
We convert (24) to an optimization problem which is formulated as  
\[\min_{\mathbf{\Phi}}\|\mathbf{X}-\mathbf{\Phi}\mathbf{Y}\|_{F}^{2}+\eta_{3}\|\mathbf{\Phi}( \mathbf{\Phi}-\mathbf{\Psi})^{T}\|_{F}^{2}. \tag{24}\]  
We denote the target function (24) by \(h(\mathbf{\Phi})\) and apply the gradient descent method to the unconstrained version of (24) and project the solution to the feasible space. The gradient is given by  
\[\
abla h(\mathbf{\Phi}) =(\mathbf{\Phi}\mathbf{Y}-\mathbf{X})\mathbf{Y}^{T}+\eta_{3}[\mathbf{\Phi} (\mathbf{\Phi}-\mathbf{\Psi})^{T}(\mathbf{\Phi}-\mathbf{\Psi})+\mathbf{\Phi}(\mathbf{\Phi}-\mathbf{\Psi})^ {T}\mathbf{\Phi}]\] \[=(\mathbf{\Phi}\mathbf{Y}-\mathbf{X})\mathbf{Y}^{T}+\eta_{3}\mathbf{\Phi} (\mathbf{\Phi}-\mathbf{\Psi})^{T}(2\mathbf{\Phi}-\mathbf{\Psi}). \tag{25}\]  
We summarize our overall algorithm in Algorithm 2.