# Joint Hybrid Beamforming Design for Millimeter Wave Amplify-and-Forward Relay Communication Systems
## 3 Hybrid Beamforming Design for HD AF Relay Systems  
### Problem Formulation  
A mathematical model of the joint HBF design problem is formulated in this subsection. According to [21; 22], the IMSE is introduced as the metric to optimize the HBF matrix, which can be defined as  
\[\mathrm{IMSE}\overset{\Delta}{=}\mathrm{E}\bigg{\{}\left\|\gamma^{-1}\mathbf{ y}_{\mathrm{D}}-\mathbf{s}\right\|^{2}\bigg{\}}, \tag{4}\]  
where \(\gamma=\gamma_{1}\gamma_{2}\), with \(\gamma_{1}\) and \(\gamma_{2}\) being the scaling factors, which are jointly optimized with the HBF matrixes at the source and relay nodes, contributing to simplify and solve the problem. By substituting (1) into (4), we have  
\[\mathrm{IMSE}= tr(\gamma_{2}{}^{-2}\gamma_{1}{}^{-2}\mathbf{W}^{H}\mathbf{H}_{2} \mathbf{GH}_{1}\mathbf{F}\mathbf{F}^{H}\mathbf{H}_{1}^{H}\mathbf{G}^{H} \mathbf{H}_{2}^{H}\mathbf{W}-\gamma_{2}^{-1}\gamma_{1}^{-1}\mathbf{W}^{H} \mathbf{H}_{2}\mathbf{GH}_{1}\mathbf{F} \tag{5}\] \[- \gamma_{2}^{-1}\gamma_{1}^{-1}\mathbf{F}^{H}\mathbf{H}_{1}^{H} \mathbf{G}^{H}\mathbf{H}_{2}^{H}\mathbf{W}+\gamma_{2}{}^{-2}\gamma_{1}{}^{-2} \sigma_{\mathrm{R}}^{2}\mathbf{W}^{H}\mathbf{H}_{2}\mathbf{GG}^{H}\mathbf{H}_ {2}^{H}\mathbf{W}\] \[+ \gamma_{2}{}^{-2}\gamma_{1}{}^{-2}\sigma_{\mathrm{D}}^{2}\mathbf{ W}^{H}\mathbf{W}+\mathbf{I}_{N_{\mathrm{S}}}).\]  
Similar to that in [22; 23; 24], the source and relay DBF matrixes can be decomposed as \(\mathbf{F}_{\mathrm{B}}=\gamma_{1}\mathbf{F}_{\mathrm{U}}\) and \(\mathbf{G}_{\mathrm{B}}=\gamma_{2}\mathbf{G}_{\mathrm{U}}\), with \(\mathbf{F}_{\mathrm{U}}\) and \(\mathbf{G}_{\mathrm{U}}\) being the unnormalized DBF matrix of the source and relay nodes, respectively. So we can rewrite the modified MSE as  
\[\mathrm{IMSE}= tr(\mathbf{W}^{H}\mathbf{H}_{2}\overline{\mathbf{G}}\mathbf{H}_{1} \overline{\mathbf{F}}\mathbf{F}^{H}\mathbf{H}_{1}^{H}\overline{\mathbf{G}}^ {H}\mathbf{H}_{2}^{H}\mathbf{W}-\mathbf{W}^{H}\mathbf{H}_{2}\overline{\mathbf{ G}}\mathbf{H}_{1}\overline{\mathbf{F}}-\overline{\mathbf{F}}^{H}\mathbf{H}_{1}^{H} \overline{\mathbf{G}}^{H}\mathbf{H}_{2}^{H}\mathbf{W} \tag{6}\] \[+ \gamma_{1}^{-2}\sigma_{\mathrm{R}}^{2}\mathbf{W}^{H}\mathbf{H}_{2 }\overline{\mathbf{G}}\overline{\mathbf{G}}^{H}\mathbf{H}_{2}^{H}\mathbf{W}+ \gamma_{1}{}^{-2}\gamma_{2}^{-2}\sigma_{\mathrm{D}}^{2}\mathbf{W}^{H}\mathbf{ W}+\mathbf{I}_{N_{\mathrm{S}}})\]  
where we define \(\overline{\mathbf{F}}\)=\(\mathbf{F}_{\mathrm{RF}}\mathbf{F}_{\mathrm{U}}\) and \(\overline{\mathbf{G}}\)=\(\mathbf{G}_{\mathrm{RF}_{2}}\mathbf{G}_{\mathrm{U}}\mathbf{G}_{\mathrm{ RF}_{1}}^{H}\) for convenience. Hence, we have \(\mathbf{F}\)=\(\gamma_{1}\overline{\mathbf{F}}\) and \(\mathbf{G}\)=\(\gamma_{2}\overline{\mathbf{G}}\). As a further step, the optimization problem can be formulated as  
\[\underset{\gamma_{1},\gamma_{2},\mathbf{F}_{\mathrm{RF}},\mathbf{F}_{\mathrm{U}}, \mathbf{G}_{\mathrm{RF}_{1}},\mathbf{G}_{\mathrm{U}},\mathbf{G}_{\mathrm{RF}_{2 }},\mathbf{W}_{\mathrm{RF}},\mathbf{W}_{\mathrm{B}}}{\max}\quad\mathrm{IMSE} \tag{7a}\] \[\mathrm{s.t.} \left|\left[\mathbf{F}_{\mathrm{RF}}\right]_{a,b}\right|^{2}=1, \forall a,b,\] (7b) \[\left|\left[\mathbf{G}_{\mathrm{RF}_{1}}\right]_{c,d}\right|^{2}=1, \forall c,d,\] (7c) \[\left|\left[\mathbf{G}_{\mathrm{RF}_{2}}\right]_{c,f}\right|^{2}=1, \forall e,f,\] (7d) \[\left|\left[\mathbf{W}_{\mathrm{RF}}\right]_{g,h}\right|^{2}=1, \forall g,h,\] (7e) \[\mathrm{E}\Big{\{}\|\mathbf{x}_{\mathrm{B}}\|_{\mathrm{F}}^{2}\Big{\}} =\mathrm{P}_{\mathrm{B}},\] (7f) \[\mathrm{E}\Big{\{}\|\mathbf{x}_{\mathrm{R}}\|_{\mathrm{F}}^{2}\Big{\}} =\mathrm{P}_{\mathrm{R}}, \tag{7g}\]where (7b)-(7e) are the constant-modulus constraints of ABF matrixes, (7f) and (7g) are the source and relay transmit power constraints.