# Global Contextual Dependency Network for Object Detection
## 1 Introduction  
Object detection aims at locating and recognizing object instances from predefined object categories [1]. The significant progress of Convolutional Neural Networks (CNNs) [2, 3] has brought excellent breakthroughs in object detection. In general, CNN-based detectors could be divided into two types, two-stage, and single-stage detectors. Our focus is on two-stage detectors. In current two-stage approaches, the Region of Interest (RoI) head extracts visual features of RoIs to predict specific categories and refine locations. While the theoretical receptive field of each RoI is large, the effective receptive field [4] remains limited, which makes the local visual features of RoIs lack global contextual dependencies. In the physical world, visual objects have natural context dependencies relationships with the particular environment (i.e., background) and other related objects (i.e., foreground) [5]. For instance, as shown in Figure 1a, a person, tennis racket, and sports ball often appear together on the tennis court. The context dependencies are helpful to recognize objects with featureless appearances or restrain noisy detections [6]. As shown in Figure 1b, only part of the tennis racket appears in the image, and the lack of visual features makes the tennis racket unrecognizable (left). Taking into account the person and even the clothes, the tennis racket is correctly detected (right). On the other hand, as shown in Figure 1c, the background, traffic light, and clock are discriminative clues for eliminating the false detection of sports balls. Various methods have attempted to capture the context dependencies by modeling the visual and spatial relationships between RoIs [7; 8]. However, the sampled RoIs may only cover a part of the image, leading to the omission of information in some regions. Global context refers to image-level context and is useful to capture global contextual dependencies [9; 10]. In practice, simple use of global context (i.e., global average pooled context representation and RoI features are concatenated before recognition) leads to improvements. However, there are two challenges in efficiently utilizing the global context. On the one hand, the global context also contains noisy information, making us aware that a careful identification of global contextual dependencies contained in the global context. Figure 1: Examples and effects of the context dependencies. (**a**) Person, tennis racket and sports ball often appear together on the tennis court. (**b**) recognizing featureless objects. (**c**) restraining noisy detections. context is desired. On the other hand, it's noted that the global context lacks explicit spatial information such as location and size. To close the gap, exploring the visual features in the global context from a spatial perspective is essential. To achieve the goal, we present a simple Global Contextual Dependency Network (GCDN), which captures global contextual dependencies over local visual features to further enhance the local RoI feature representations. Considering that visual objects varies substantially in scale and are distributed in different locations, a Context Representation Module (CRM) is exploited to construct multi-scale context representations. Furthermore, to capture global contextual dependencies, we utilize the attention mechanism and design a Context Dependency Module (CDM). Our GCDN consists of multiple CDMs. Each CDM generates single-scale contextual RoI features based on the local RoI features and single-scale context representation via affinity computation and context aggregation. Multi-scale contextual RoI features generated by parallel CDMs independently are fused with the original RoI features to predict the labels of the RoIs. Comprehensive experiments are performed to validate the effectiveness and generality of GCDN. Our GCDN improves 1.5% and 1.2% Average Precision (AP) on MS-COCO 2017 benchmark dataset [11] with ResNet-50 for Feature Pyramid Network (FPN) and Mask R-CNN, respectively.