  We investigate the complexity and performance of recurrent neural network
(RNN) models as post-processing units for the compensation of fibre
nonlinearities in digital coherent systems carrying polarization multiplexed
16-QAM and 32-QAM signals. We evaluate three bi-directional RNN models, namely
the bi-LSTM, bi-GRU and bi-Vanilla-RNN and show that all of them are promising
nonlinearity compensators especially in dispersion unmanaged systems. Our
simulations show that during inference the three models provide similar
compensation performance, therefore in real-life systems the simplest scheme
based on Vanilla-RNN units should be preferred. We compare bi-Vanilla-RNN with
Volterra nonlinear equalizers and exhibit its superiority both in terms of
performance and complexity, thus highlighting that RNN processing is a very
promising pathway for the upgrade of long-haul optical communication systems
utilizing coherent detection.
