## 1 Introduction  
Next generation wireless and/or mobile communications require a high transmission efficiency and high reliability to provide various services to subscribers. Recently, many methods for increasing the throughput are being researched, as the demand for next-satellite broadcasting communication and 5G (5 th Generation Mobile Telecommunication)-based mobile communication for throughput increases, while the bandwidth is limited [1, 2]. Research on applying the multidimensional modulation method in the modulator/demodulator and speeding up the high-speed channel encoder/decoder to meet the high data rate is already saturated. Thus, the research direction has focused more on transmission methods. However, it is very difficult to improve both throughput and performance, because the two are in a trade-off relationship. Therefore, it is highly important to develop methods that can maintain the maximum performance, while increasing the throughput. Recently, the faster-than-Nyquist (FTN) method [3, 4, 5], which transmits faster than the throughput of Nyquist, is emerging as the standard for the next generation Digital Video Broadcasting-Satellite-Third Generation (DVB-S3) and the future of broadcast television (FOBTV) [6, 7]. It is possible to transmit signals faster than the Nyquist rate and maximize throughput with the same channel bandwidth. However, the FTN method has a limitation in maximizing the transmission efficiency due to the performance degradation arising from inter-symbol interference (ISI). The multiple input/multiple output-FTN (MIMO-FTN) [8; 9; 10] transmission method, which combines the MIMO and FTN methods to improve throughput, can maximize the improvement of the throughput, although its method of decoding and removing interference is difficult, the research about which is still underdeveloped. In [8], space-time trellis coding and the weighted-zero forcing algorithm were applied to distinguish the symbols of each transmit antenna, and the Bahl, Cocke, Jelinek, and Raviv (BCJR) algorithm [11; 12] was used to cancel the interference in order to improve the error performance by increasing the number of iterations. However, it did not improve the performance compared to the single input/single output-FTN (SISO-FTN) method. In the paper, we employed the low-density parity-check (LDPC) coding technique, as a channel coding scheme, which is defined based on a sparse parity-check matrix, H, where N bit nodes and \((\rm{N-K})\) check nodes are connected by edges [13; 14]. In the H matrix, the bit nodes/check nodes are mapped to the columns/rows, where one element in a row is connected to another in a column by an edge. Bit nodes consist of information and parity bit nodes, while information bit nodes are randomly connected by edges with check nodes by features of the matrix, H. To improve the bit error rate (BER) performance of the FTN signal with LDPC codes, we proposed FTN signal processing based on the unequal error probability (UEP) method in the SISO channel. The UEP method divides encoded bits into groups, according to their priority and FTN interference, which are differently applied to each group. In deciding which group has a higher priority, the horizontal shuffle schedule (HSS) LDPC decoding algorithm is employed. The HSS-based LDPC decoding algorithm, proposed in [15; 16], is used to accelerate the LDPC decoding speed. However, in this study, it was used to configure the priority algorithm. The LDPC decoder performs an iterative exchange between the bit nodes updating and the check node updating until the maximum number of iterations has been reached. Conventional LDPC decoder calculates bit nodes and check nodes updating sequentially based only on messages updated in the previous iteration. However, in the HSS-based LDPC decoding process, the already updated bit node and check node in the same iteration move to a different bit and check node. Therefore, bit node groups connected to check nodes, which are processed first, are assigned a higher priority.