  A deep learning assisted sum-product detection algorithm (DL-SPDA) for
faster-than-Nyquist (FTN) signaling is proposed in this paper. The proposed
detection algorithm works on a modified factor graph which concatenates a
neural network function node to the variable nodes of the conventional FTN
factor graph to approach the maximum a posterior probabilities (MAP) error
performance. In specific, the neural network performs as a function node in the
modified factor graph to deal with the residual intersymbol interference (ISI)
that is not considered by the conventional detector with a limited complexity.
We modify the updating rule in the conventional sum-product algorithm so that
the neural network assisted detector can be complemented to a Turbo
equalization receiver. Furthermore, we propose a compatible training technique
to improve the detection performance of the proposed DL-SPDA with Turbo
equalization. In particular, the neural network is optimized in terms of the
mutual information between the transmitted sequence and the extrinsic
information. We also investigate the maximum-likelihood bit error rate (BER)
performance of a finite length coded FTN system. Simulation results show that
the error performance of the proposed algorithm approaches the MAP performance,
which is consistent with the analytical BER.
