  In this paper, we address the problem of referring expression comprehension
in videos, which is challenging due to complex expression and scene dynamics.
Unlike previous methods which solve the problem in multiple stages (i.e.,
tracking, proposal-based matching), we tackle the problem from a novel
perspective, \textbf{co-grounding}, with an elegant one-stage framework. We
enhance the single-frame grounding accuracy by semantic attention learning and
improve the cross-frame grounding consistency with co-grounding feature
learning. Semantic attention learning explicitly parses referring cues in
different attributes to reduce the ambiguity in the complex expression.
Co-grounding feature learning boosts visual feature representations by
integrating temporal correlation to reduce the ambiguity caused by scene
dynamics. Experiment results demonstrate the superiority of our framework on
the video grounding datasets VID and LiOTB in generating accurate and stable
results across frames. Our model is also applicable to referring expression
comprehension in images, illustrated by the improved performance on the RefCOCO
dataset. Our project is available at https://sijiesong.github.io/co-grounding.
