# HybridTabNet: Towards Better Table Detection in Scanned Document Images
## 3 Method
### Hybrid Task Cascade  
Cascade architecture has been very successful and effective in tasks, such as object detection [17]. However, to successfully apply the idea of cascade architecture to instance segmentation problems was still an open-ended research question until HTC [20] was introduced. The main idea behind HTC is to leverage the relationship between object detection and segmentation tasks. Instead of treating detection and segmentation as different problems, it performs joint multi-stage processing. The joint multi-stage processing refers to the combination of object detection and segmentation at each stage. Due to the joint multi-stage processing, the improvement in one task, e.g., detection, improves the mask prediction and segmentation task [20]. It also utilizes the spatial context to distinguish the background from the foreground. The semantic branch (S) provides spatial cues, which complement the bounding box and mask features. Figure 3 exhibits the architecture of HTC. It has multiple heads for both bounding box and semantic segmentation to process input at different scales. It consists of a segmentation branch (S) with mask (M1,M2,M3) and bounding box (B1,B2,B3) heads. The RPN head predicts preliminary object proposals for these feature maps, whereas the semantic segmentation branch predicts per-pixel semantic segmentation for the whole image through a fully convolutional architecture trained jointly with other branches. In the first stage of architecture, the RPN head applies RoI pooling to the output features maps of the backbone model. B1 takes the output of RoI pooling as an input to make RoI-wise predictions. Each head makes two predictions: bounding box classification scores and box regression points. In the second stage, M1 generates pixel-wise segmentation masks for positive RoIs. The rest of the stages follow the same flow. At the inference time, object detection made by Bbox heads is complemented with segmentation masks made by the mask head for all detected objects. Equation (2) explains the flow of Figure 3.  
\[ \begin{split}\mathbf{x}_{t}^{\text{box}}&=\mathcal{ P}(\mathbf{x},\mathbf{r}_{t-1})+\mathcal{P}(S(\mathbf{x}),\mathbf{r}_{t-1})\\ \mathbf{r}_{t}&=B_{t}\left(\mathbf{x}_{t}^{\text{box} }\right)\\ \mathbf{x}_{t}^{\text{mask}}&=\mathcal{P}(\mathbf{x},\mathbf{r}_{t})+\mathcal{P}(S(\mathbf{x}),\mathbf{r}_{t})\\ \mathbf{m}_{t}&=M_{t}\left(\mathcal{F}\Big{(}\mathbf{x }_{t}^{\text{mask}},\mathbf{m}_{t-1}^{-}\Big{)}\right)\end{split}  \tag{2}\]  
Figure 3: Explained architecture of the Hybrid Task Cascade Network. This network utilizes three box and mask heads in a cascading architecture to produce accurate predictions. where S indicates the semantic segmentation head, \(\mathbf{x}\) indicates the CNN features of backbone network, and \(\mathbf{x}_{t}^{box}\) and \(\mathbf{x}_{t}^{mask}\) indicate the box and mask features derived from \(\mathbf{x}\) and the input RoI. \(\mathcal{P}(. )\) is a pooling operator, which could be RoI Align or RoI pooling; \(B_{t}\) and \(M_{t}\) denote the box and mask head at the \(t\)-th stage; and \(\mathbf{r}_{t}\) and \(\mathbf{m}_{t}\) represent the corresponding box predictions and mask predictions. Equation (2) indicates that the box and mask heads of each stage take RoI features extracted by the backbone network and semantic features given by semantic segmentation head. It is essential for HTC because it can differentiate between tables in a cluttered background by exploiting the semantic features. Since the modules given in Equation (2) are differentiable [20]. HTC can be trained in an end-to-end manner. The overall loss function can be formulated in the form of multi-tasking [20] learning. \[