We propose a new model for detecting visual relationships, such as "person riding motorcycle" or "bottle on table". This task is an important step towards comprehensive structured image understanding, going beyond detecting individual objects. Our main novelty is a Box Attention mechanism that allows to model pairwise interactions between objects using standard object detection pipelines. The resulting model is conceptually clean, expressive and relies on well-justified training and prediction procedures. Moreover, unlike previously proposed approaches, our model does not introduce any additional complex components or hyperparameters on top of those already required by the underlying detection model. We conduct an experimental evaluation on two datasets, V-COCO and Open Images, demonstrating strong quantitative and qualitative results.
Kolesnikov et al., REF proposed a joint probabilistic model for detecting visual relationships.
Detecting Visual Relationships Using Box Attention
2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)
2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)
Computer Science