## 3 Optimization Problem Solution
\begin{array}{rcl}\text{tr}(\mathbf{C}^{T}\mathbf{D})&=&(\,(\mathbf{C}))^ {H}\text{vec}(\mathbf{D}),\\ \text{tr}(\mathbf{A}^{H}\mathbf{B}\mathbf{A}\mathbf{C})&=&\text{tr}(\text{ vec}(\mathbf{A}))^{H}(\mathbf{C}^{T}\otimes\mathbf{B})\text{vec}(\mathbf{A})\\ \text{vec}\mathbf{A}\mathbf{B}\mathbf{C}&=&(\mathbf{C}^{T}\otimes\mathbf{A}) \text{vec}(\mathbf{B}),\end{array} \]  
where \(\text{vec}(\cdot)\) concatenates the columns of a matrix into a single vector, the \(\text{SMSE}\) in (11) can be expressed as a function of \(\mathbf{a}=\text{vec}(\mathbf{A})\):  
\[ \begin{split}\text{SMSE}=&\sum_{k=1}^{K}(\mathbf{P}_{k} \mathbf{a}-\text{vec}(\mathbf{I}_{N_{k}}))^{H}(\mathbf{P}_{k}\mathbf{a}-\text{ vec}(\mathbf{I}_{N_{k}}))\\ &+\mathbf{a}^{H}\mathbf{Q}_{k}\mathbf{a}+\mathbf{a}^{H}\mathbf{S}_{k} \mathbf{a}+\mathbf{a}^{H}\mathbf{R}_{k}\mathbf{a}+\mathbf{t}_{1}\end{split}  \tag{19}\]where  
\[\mathbf{t}_{1}=\sum_{k=1}^{K}\operatorname{tr}(\mathbf{M}_{k}^{H}\mathbf{M}_{k}),\]  
does not depend on \(\mathbf{a}\) and  
\[\mathbf{P}_{k}=(\Lambda_{h}\mathbf{V}_{h_{k}}^{H})^{T}\otimes(\mathbf{M}_{k}^{ H}\mathbf{U}_{g_{k}}\Lambda_{g}), \tag{20}\]  
\[\mathbf{Q}_{k}=\sigma_{t}^{2}((\mathbf{U}_{h}^{H}\mathbf{R}_{LI}\mathbf{R}_{LI}^ {H}\mathbf{U}_{h})^{T}\otimes\Lambda_{g}\mathbf{U}_{g_{k}}^{H}\mathbf{M}_{k} \mathbf{M}_{k}^{H}\mathbf{U}_{g_{k}}\Lambda_{g}), \tag{21}\]  
\[\mathbf{S}_{k}=\mathbf{I}_{L}\otimes(\Lambda_{g}\mathbf{U}_{g_{k}}^{H}\mathbf{ M}_{k}\mathbf{M}_{k}^{H}\mathbf{U}_{g_{k}}\Lambda_{g}), \tag{22}\]  
\[\mathbf{R}_{k}=(\sum_{m=1,m\
eq k}^{K}\Lambda_{h}\mathbf{V}_{h_{m}}^{H} \mathbf{V}_{h_{m}}\Lambda_{h})^{T}\otimes(\Lambda_{g}\mathbf{U}_{g_{k}}^{H} \mathbf{M}_{k}\mathbf{M}_{k}^{H}\mathbf{U}_{g_{k}}\Lambda_{g}). \tag{23}\]  
The relay transmit power constraint in (12) can be written as  
\[ \begin{split}\operatorname{tr}(\mathbf{\Gamma}&( \sum_{k=1}^{K}\mathbf{H}_{k}\mathbf{\Theta}_{k}\mathbf{\Theta}_{k}^{H} \mathbf{H}_{k}^{H}+\sigma_{t}^{2}\mathbf{R}_{LI}\mathbf{R}_{LI}^{H}+\mathbf{ I}_{N_{t}})\mathbf{\Gamma}^{H})\\ &=\operatorname{tr}(\mathbf{A}\times(\Lambda_{h}^{2}+\sigma_{t} ^{2}\mathbf{U}_{h}^{H}\mathbf{R}_{LI}\mathbf{R}_{LI}^{H}\mathbf{U}_{h}+ \mathbf{I}_{L_{1}})\mathbf{A}^{H}).\end{split}  \tag{24}\]  
Defining \(\mathbf{D}=(\Lambda_{h}^{2}+\sigma_{t}^{2}\mathbf{U}_{h}^{H}\mathbf{R}_{LI} \mathbf{R}_{LI}^{H}\mathbf{U}_{h}+\mathbf{I}_{L_{1}})\otimes\mathbf{I}_{L_{1}}\), (24) can be written as  
\[\mathbf{a}^{H}\mathbf{D}\mathbf{a}\leq P_{r}. \tag{25}\]  
The original relay optimization problem is then given by  
\[ \begin{split}\underset{\mathbf{A}}{\text{min}}& \text{SMSE}\\ s.t.&\mathbf{a}^{H}\mathbf{D}\mathbf{a}\leq P_{r} \end{split}  \tag{26}\]  
This is a quadratically constrained quadratic programming (QCQP) problem which is convex and so can be efficiently solved using the interior point method. For example, the CVX toolbox for disciplined convex programming [21] can be employed. Third, the \(k\)th source precoding matrix \(\mathbf{\Theta}_{k}\) can be obtained using \(\mathbf{M}_{k}\) and \(\mathbf{\Gamma}\) given above. The corresponding optimization problem can be formulated as the following QCQP problem:  
\[ \begin{split}\text{SMSE}=&\sum_{k=1}^{K} \operatorname{tr}((\bar{\mathbf{G}}_{k}\mathbf{\Gamma}\mathbf{H}_{k}\mathbf{ \Theta}_{k}-\mathbf{I}_{N_{k}})(\bar{\mathbf{G}}_{k}\mathbf{\Gamma}\mathbf{H}_ {k}\mathbf{\Theta}_{k}-\mathbf{I}_{N_{k}})^{H}\\ &+\bar{\mathbf{G}}_{k}\mathbf{\Gamma}\sum_{m=1,m\
eq k}^{K} \mathbf{H}_{k}\mathbf{\Theta}_{k}\mathbf{\Theta}_{k}^{H}\mathbf{H}_{k}^{H} \mathbf{\Gamma}^{H}\bar{\mathbf{G}}_{k}^{H})+\mathbf{t}_{2},\end{split}  \tag{27}\]  
where \(\bar{\mathbf{G}}_{k}=\mathbf{M}_{k}^{H}\mathbf{G}_{k}\) and \(\mathbf{t}_{2}=\sum_{k=1}^{K}\operatorname{tr}(\mathbf{M}_{k}^{H}\mathbf{C}_ {k}\mathbf{M}_{k})\) can be ignored in the optimization as they do not depend on \(\mathbf{\Theta}_{k}\).