  Text-based video segmentation is a challenging task that segments out the
natural language referred objects in videos. It essentially requires semantic
comprehension and fine-grained video understanding. Existing methods introduce
language representation into segmentation models in a bottom-up manner, which
merely conducts vision-language interaction within local receptive fields of
ConvNets. We argue that such interaction is not fulfilled since the model can
barely construct region-level relationships given partial observations, which
is contrary to the description logic of natural language/referring expressions.
In fact, people usually describe a target object using relations with other
objects, which may not be easily understood without seeing the whole video. To
address the issue, we introduce a novel top-down approach by imitating how we
human segment an object with the language guidance. We first figure out all
candidate objects in videos and then choose the refereed one by parsing
relations among those high-level objects. Three kinds of object-level relations
are investigated for precise relationship understanding, i.e., positional
relation, text-guided semantic relation, and temporal relation. Extensive
experiments on A2D Sentences and J-HMDB Sentences show our method outperforms
state-of-the-art methods by a large margin. Qualitative results also show our
results are more explainable.
