  Recently proposed orthogonal time frequency space (OTFS) modulation has been
considered as a promising candidate for accommodating various emerging
communication and sensing applications in high-mobility environments. In this
paper, we propose a novel cross domain iterative detection algorithm to enhance
the error performance of OTFS modulation. Different from conventional OTFS
detection methods, the proposed algorithm applies basic estimation/detection
approaches to both the time domain and delay-Doppler (DD) domain and
iteratively updates the extrinsic information from two domains with the unitary
transformation. In doing so, the proposed algorithm exploits the time domain
channel sparsity and the DD domain symbol constellation constraints. We
evaluate the estimation/detection error variance in each domain for each
iteration and derive the state evolution to investigate the detection error
performance. We show that the performance gain due to iterations comes from the
non-Gaussian constellation constraint in the DD domain. More importantly, we
prove the proposed algorithm can indeed converge and, in the convergence, the
proposed algorithm can achieve almost the same error performance as the
maximum-likelihood sequence detection even in the presence of fractional
Doppler shifts. Furthermore, the computational complexity associated with the
domain transformation is low, thanks to the structure of the discrete Fourier
transform (DFT) kernel. Simulation results are consistent with our analysis and
demonstrate a significant performance improvement compared to conventional OTFS
detection methods.
