# A New Design of Codebook for Hybrid Precoding in Millimeter-Wave Massive MIMO Systems
## 3 Proposed Joint Codebooks Design Algorithm
According to (8), the \(\widetilde{\mathbf{F}}_{\text{BB}}\) can be calculated by the initial codebook \(\mathbf{F}_{\text{A}}\); by fixing the value of \(\widetilde{\mathbf{F}}_{\text{BB}}\), the design of analog codebook can be expressed as  
\[\mathbf{F}_{\text{A}}^{\text{opt}}= \arg\min_{\mathbf{F}_{\text{A}}}\sum_{i=1}^{N_{\text{tr}}}\left\| \mathbf{F}_{\text{opt}}(i)-\mathbf{F}_{\text{A}}\widetilde{\mathbf{F}}_{\text{ BB}}(i)\right\|_{F},\] (9) s.t. \[\mathbf{F}_{\text{A}}(m,n)\in\mathcal{F}_{\text{RF}},\forall m,n,\] \[\left\|\mathbf{F}_{\text{A}}\widetilde{\mathbf{F}}_{\text{BB}}(i )\right\|_{F}^{2}=N_{\text{s}},\forall i,\]  
where \(N_{\text{tr}}\) denotes the number of channel samples, and \(\mathbf{F}_{\text{opt}}(i)\) and \(\widetilde{\mathbf{F}}_{\text{BB}}(i)\) denote the optimal unconstrained precoder and digital precoder, respectively, for the i-th channel sample. It is hard to find a general solution to solve (9) with the nonconvex constraint \(\mathbf{F}_{\text{A}}(m,n)\in\mathcal{F}_{\text{RF}}\). Therefore, we propose an iterative algorithm to achieve the local optimal solution of the problem. The proposed algorithm is specifically presented in Algorithm 1, which can be mainly divided into three step. The first stage is step 1, several channel matrix samples are generated offline through the channel model in Formula (2), then, the optimal unconstrained precoding matrix can be calculated by SVD according to each channel matrix and formed into a training set; the second stage includes step 2 to step 14, the sparse matrix (the digital precoding matrix \(\widetilde{\mathbf{F}}_{\text{BB}}(i)\)) reconstructed by the improved OMP (orthogonal matching pursuit) algorithm is not accurate enough in [4] because the selected columns will always be retained in each iteration, so the columns with larger errors cannot be replaced in subsequent processing, resulting in insufficient accuracy of the reconstructed matrix. To solve this problem, we propose a hybrid precoding codebook generation algorithm, according to a combination of dictionary learning and the reconstruction algorithm, which is separated into two parts. In the first part, we utilize the proposed algorithm to find the analog codebook \(\mathbf{F}_{\text{A}}\) and the digital codebook \(\mathbf{D}\) of size \(N_{\text{tr}}\). In each iteration, we select multiple column vectors from the initial analog codebook instead of a single-column vector to form an analog precoding matrix, and then use the least squares method to figure out the digital precoding matrix. Finally, we select columns whose number equals to the number of data stream, from the digital precoding matrix calculated in the previous step, to form the final digital precoding matrix. In this way, the column selected last time can be replaced with a column with a smaller error in the next iteration, avoiding the accumulation of errors, and the specific steps are shown in Steps 3-11. The second part is that the analog and digital codebook will be simultaneously updated and optimized to achieve the maximum system spectral efficiency based on dictionary learning, which aims to further reduce the Euclidean distance between the updated hybrid precoder and the optimal unconstrained precoder. In addition, in the process of updating the analog codebook \(\mathbf{F}_{\text{A}}\), we should consider the limitation of the resolution of the phase shifter, so each element of the matrix in the \(\mathbf{F}_{\text{A}}\) must be included in \(\mathcal{F}_{\text{RF}}=\left\{1,\omega,\omega^{2},\cdots,\omega^{2^{q}-1}\right\}\)[4]. The third stage includes Step 15; with the requirements of the size of the digital codebook, we can use \(N_{\text{tr}}\) digital precoding matrices in the digital codebook \(\mathbf{D}\) as training samples, and utilize the clustering algorithm to generate a qualified size digital precoding codebook \(\mathcal{F}_{\text{D}}=\left[\mathbf{F}_{1}^{\text{D}},\mathbf{F}_{2}^{\text {D}},\cdots,\mathbf{F}_{N_{\text{2}}}^{\text{D}}\right]\)[20].