  This paper addresses the problem of simultaneous machine translation (SiMT)
by exploring two main concepts: (a) adaptive policies to learn a good trade-off
between high translation quality and low latency; and (b) visual information to
support this process by providing additional (visual) contextual information
which may be available before the textual input is produced. For that, we
propose a multimodal approach to simultaneous machine translation using
reinforcement learning, with strategies to integrate visual and textual
information in both the agent and the environment. We provide an exploration on
how different types of visual information and integration strategies affect the
quality and latency of simultaneous translation models, and demonstrate that
visual cues lead to higher quality while keeping the latency low.
