  Transformer architectures have brought about fundamental changes to
computational linguistic field, which had been dominated by recurrent neural
networks for many years. Its success also implies drastic changes in
cross-modal tasks with language and vision, and many researchers have already
tackled the issue. In this paper, we review some of the most critical
milestones in the field, as well as overall trends on how transformer
architecture has been incorporated into visuolinguistic cross-modal tasks.
Furthermore, we discuss its current limitations and speculate upon some of the
prospects that we find imminent.
