Before revision:  We describe the design and modified training of mT5 and demonstrate its state-of-the-art performance on many multilingual benchmarks  .
After revision:  We describe the design and modified training of mT5 and demonstrate its state-of-the-art performance on many multilingual benchmarks . We also describe a simple technique to prevent "accidental translation" in the zero-shot setting, where a generative model chooses to (partially) translate its prediction into the wrong language .