Globetrotter: Unsupervised Multilingual Translation from Visual Alignment
Authors: Didac Suris Coll-Vinent, Dave Epstein, Carl Vondrick
Keywords: cross-modal, multilingual, unsupervised translation, visual similarity
Abstract: Machine translation in a multi-language scenario requires large-scale parallel corpora  for  every  language  pair.   Unsupervised  translation  is  challenging  because there is no explicit connection between languages, and the existing methods have to rely on topological properties of the language representations.  We introduce a framework that leverages visual similarity to align multiple languages, using images as the bridge between them. We estimate the cross-modal alignment between language and images, and use this estimate to guide the learning of cross-lingual representations.   Our  language  representations  are  trained  jointly  in  one  model with a single stage.  Experiments with fifty-two languages show that our method outperforms prior work on unsupervised word-level and sentence-level translation using retrieval.