# Hybrid Precoder Using Stiefel Manifold Optimization for Mm-Wave Massive MIMO System
###### Abstract
To approximate the behavior to digital precoding, limited CSI is required in the spatial channel to estimate the SCM using an acceptable number of RF chains. This technique can be utilized for single and multiple users, and it can be used in both TDD and FDD systems, as well as at single and multiple carrier frequencies and in LOS and NLOS channels. The current scheduling of users in downlink transmission and sub-carrier power allocation algorithms for massive MIMO is considered in frequency selective channels [29]. The performance of a hybrid precoder can reach that of a digital precoder under the condition of \(N_{p}=2N_{FFT}(N_{T}-N_{FFT}+1)\), where \(N_{FFT}<N_{T}\). Furthermore, a unique approach is applied to realize hybrid precoding by employing an  
Figure 1: Frequency spectrum for 5G communication. \(N_{RFT}\) RF chain and a phase shifter less than Np. Three ways are used to solve the problem: (1) the \(N_{T}\) number of RF chains in the digital precoder, (2) antenna selection precoding, and (3) the proposed hybrid precoding method, which combines zero-forcing and the proposed algorithm for user scheduling and sub-carrier power distribution. Ref. [30] proposed an optimal hybrid precoder design for large antenna systems, in which the \(N_{RFT}\) required is equal to or greater than the total number of data streams used to achieve optimal precoding performance. If this requirement is not encountered, two techniques for the maximization of spectral efficiency are provided. The hybrid precoding algorithm is designed for large MIMO systems with the following requirements: (1) \(N_{RFT}=N_{s}\) and phase shifters with infinite resolution and (2) \(N_{s}<N_{RFT}<2N_{s}\) and phase shifters with limited resolution. The algorithm approximates the maximum capacity for \(N_{RFT}=N_{s}\). The JSDM scheme was discussed for FDD systems in [31] to achieve significant throughput and for the simplification of system operation. The covariance matrix is formed by correlating the channel vectors from the BS to the MSs. To limit the quantity of feedback necessary, this characteristic is utilized to group the MSs with comparable covariance matrices. To eliminate inter-group interferences, a precoding matrix based on DFT is used based on second-order channel statistics. A framework was presented in [32] to minimize the dimension of the precoding matrix and pilot sequence, enabling a low-complexity system using hybrid precoding for massive MIMO systems. A Mm-Wave MIMO system with hybrid precoding was investigated in [33] to improve the spectral efficiency in downlink transmission by jointly optimizing the system using a hybrid precoder at the base station and using a digital precoder in user equipment. An RF precoder was developed for MIMO systems to minimize the overhead in backward and forward cases. A codebook-based hybrid precoder is used to minimize the inter-beam interference and overhead of scheduled users. The sum rate can be increased by increasing the number of users. Hybrid precoding was discussed in [34]; it updates one column using the Lagrange multiplier of the analog precoder and the same row using the least square method of the digital precoder. The hybrid precoder can also be designed using feedback-supported codebooks to reduce the mean error between the optimal beam and vector beam. The Kalman-based filter concept and manifold optimization with MMSE design were also used for the design of hybrid precoder [35; 36]. A novel hybrid precoder design is proposed in this article using the manifold optimization algorithm, namely, Stiefel manifold optimization, to increase the performance of Mm-Wave massive communication systems. A manifold optimization-based algorithm using the gradient method is proposed to increase the spectral efficiency to be near optimal and to speed up the processing speed. A comparison of performances is shown using the simulation results of the proposed work with those of existing work.