SlowGuess/ACL_2020_OCR huggingface dataset:
We observe the following patterns:  $(i)$  for bilingual translation (Figure 4), the convtransformer has a sharper weight distribution on the matching characters and words than the transformer;  $(ii)$  for multilingual translation of close languages  $\mathrm{(FR + ES\rightarrow EN}$  Figure 5), both transformer and convtransformer are able to preserve the word alignments, but the alignments produced by the convtransformer appear to be slightly less noisy;  $(iii)$  for multilingual translation of distant languages  $\mathrm{(FR + ZH\to EN}$  , Figure 6), the character alignments of the transformer become visually much noisier and concentrate on a few individual characters, with many word alignments dissolving. The convtransformer character alignments remain more spread out, and word align
ment appears to be better preserved. This is another indication that the convtransformer is more robust for multilingual translation of distant languages. (iv) for multilingual translation with three inputs, where two of the three languages are close  $(\mathrm{FR} + \mathrm{ES} + \mathrm{ZH}\rightarrow \mathrm{EN},$  Figure 7), we observe a similar pattern, with word alignments being better preserved by the convtransformer.
source	Pour que ce cadre institutionnel soit efficace, il devra remédier aux lacunes en matière de réglementation et de mise en œuvre qui caractérisent à ce jour la gouvernance dans le domaine du développement durable.
reference	For this institutional framework to be effective, it will need to fill the regulatory and implementation deficit that has thus far characterized governance in the area of sustainable development.
FR→EN
transformer	To ensure that this institutional framework is effective, it will need to address regulatory and implementation gaps that characterize governance in sustainable development.
convtransformer	In order to ensure that this institutional framework is effective, it will have to address regulatory and implementation gaps that characterize governance in the area of sustainable development.
FR+ES→EN
transformer	To ensure that this institutional framework is effective, it will need to address gaps in regulatory and implementation that characterize governance in the area of sustainable development.
convtransformer	In order to ensure that this institutional framework is effective, it will be necessary to address regulatory and implementation gaps that characterize governance in sustainable development so far.
FR+WB→EN
transformer	To ensure that this institutional framework is effective, gaps in regulatory and implementation that have characterized governance in sustainable development to date.
convtransformer	For this institutional framework to be effective, it will need to address gaps in regulatory and implementation that characterize governance in the area of sustainable development.
FR+ES+WB→EN
transformer	To ensure that this institutional framework is effective, it will need to address regulatory and implementation gaps that are characterized by governance in the area of sustainable development.
convtransformer	If this institutional framework is to be effective, it will need to address gaps in regulatory and implementation that are characterized by governance in the area of sustainable development.
Table 3: Example character-level translation outputs on the UN dataset, FR→EN.
source	Estamos convencidos de que el futuro de la humanidad en conditiones de seguridad, la coexistencia pacifica, la tolerancia y la reconciliación entre las naciones se verán reforzados por el reconocimiento de los hechos del pasado.
reference	We strongly believe that the secure future of humanity, peaceful coexistence, tolerance and reconciliation between nations will be reinforced by the acknowledgement of the past.
ES→EN
transformer	We are convinced that the future of humanity in conditions of security, peaceful coexistence, tolerance and reconciliation among nations will be strengthened by recognition of the facts of the past.
convtransformer	We are convinced that the future of humanity under conditions of safe, peaceful coexistence, tolerance and reconciliation among nations will be reinforced by the recognition of the facts of the past.
FR+ES→EN
transformer	We are convinced that the future of mankind under security, peaceful coexistence, tolerance and reconciliation among nations will be strengthened by the recognition of the facts of the past.
convtransformer	We are convinced that the future of humanity in safety, peaceful coexistence, tolerance and reconciliation among nations will be reinforced by the recognition of the facts of the past.
ES+WB→EN
transformer	We are convinced that the future of humanity in safety, peaceful coexistence, tolerance and reconciliation among nations will be strengthened by the recognition of the facts of the past.
convtransformer	We are convinced that the future of humanity in safety, peaceful coexistence, tolerance and reconciliation among nations will be strengthened by the recognition of the facts of the past.
FR+ES+WB→EN
transformer	We are convinced that the future of mankind in safety, peaceful coexistence, tolerance and reconciliation among nations will be strengthened by the recognition of the facts of the past.
convtransformer	We are convinced that the future of mankind in security, peaceful coexistence, tolerance and reconciliation among nations will be strengthened by the recognition of the facts of the past.
Table 4: Example character-level translation outputs on the UN dataset, ES→EN.
source ZH	利用专家管理农场对于最大限度提高生产率和灌溉水使用效率也是重要的。
source ZH	tjh|et fny|pe tp|gj pei|fnrt cf|gf jb|dd bv|ya rj|ym tg|u|yx t iak|ivc|i wgkq0|et uqt|yx bn j tgj|s r .
reference EN	The use of expert farm management is also important to maximize land productivity and efficiency in the use of irrigation water.
ZH→EN
transformer	The use of expert management farms is also important for maximizing productivity and irrigation use.
convtransformer	The use of experts to manage farms is also important for maximizing efficiency in productivity and irrigation water use.
FR+ZH→EN
transformer	The use of expert management farms is also important for maximizing productivity and efficiency in irrigation water use.
convtransformer	The use of expert management farms is also important for maximizing productivity and irrigation water efficiency.
ES+ZH→EN
transformer	The use of expert farm management is also important for maximizing productivity and irrigation water use efficiency.
convtransformer	The use of expert management farms to maximize efficiency in productivity and irrigation water use is also important.
FR+ES+ZH→EN
transformer	The use of expert management farms is also important for maximizing productivity and irrigation water use.
convtransformer	It is also important that expert management farms be used to maximize efficiency in productivity and irrigation use.
Table 5: Example character-level translation outputs on the UN dataset, ZH→EN.
(a) transformer trained on FR  $\rightarrow$  EN, testing with FR as input.
(b) convtransformer trained on FR  $\rightarrow$  EN, testing with FR as input.
Figure 4: Example alignments produced b60character-level models trained on FR→EN.
Figure 5: Example alignments produced by 602acter-level models trained on FR+ES→EN.
Figure 6: Example alignments produced by kDaacter-level models trained on FR+ZH→EN.
Figure 7: Example alignments produced by chdir-level models trained on FR+ES+ZH→EN.