  This paper proposes a parallel computation strategy and a posterior-based
lattice expansion algorithm for efficient lattice rescoring with neural
language models (LMs) for automatic speech recognition. First, lattices from
first-pass decoding are expanded by the proposed posterior-based lattice
expansion algorithm. Second, each expanded lattice is converted into a minimal
list of hypotheses that covers every arc. Each hypothesis is constrained to be
the best path for at least one arc it includes. For each lattice, the neural LM
scores of the minimal list are computed in parallel and are then integrated
back to the lattice in the rescoring stage. Experiments on the Switchboard
dataset show that the proposed rescoring strategy obtains comparable
recognition performance and generates more compact lattices than a competitive
baseline method. Furthermore, the parallel rescoring method offers more
flexibility by simplifying the integration of PyTorch-trained neural LMs for
lattice rescoring with Kaldi.
