  Latent alignment objectives such as CTC and AXE significantly improve
non-autoregressive machine translation models. Can they improve autoregressive
models as well? We explore the possibility of training autoregressive machine
translation models with latent alignment objectives, and observe that, in
practice, this approach results in degenerate models. We provide a theoretical
explanation for these empirical results, and prove that latent alignment
objectives are incompatible with teacher forcing.
