# SIPNet & SAHI: Multiscale Sunspot Extraction for High-Resolution Full Solar Images
## 3 Method
### Prediction Head
#### 3.2.2 Prediction Sub-Branches  
There are three sub-branches in this prediction head: one for regressing the localization of the bounding box, one for predicting the IoU-aware classification score based on a star-shaped representation of the bounding box features using star-shaped deformable convolution (Star Dconv), and the other for predicting the mask coefficients.  
The localization sub-branch performs bounding box regression and subsequent refinement. It sequentially takes as input the feature maps from P3 to P7 processed by SASPP. First, the input is applied by three \(3\times 3\) conv layers with ReLU activation, producing a feature map with 256 channels. One localization sub-branch convolves the feature map again and outputs a 4D distance vector (\(l^{\prime},\ t^{\prime},\ r^{\prime},\ b^{\prime}\)) per spatial location, representing the initial bounding box. The other sub-branch applies a star-shaped deformable convolution and produces the distance scaling factor (\(\Delta l\), \(\Delta t\), \(\Delta r\),\(\Delta b\)), which is multiplied by the initial distance vector (\(l^{\prime},\ t^{\prime},\ r^{\prime},\ b^{\prime}\)) to produce the refined bounding box (\(l,\ t,\ r,\ b\)).  
The star-shaped bounding box feature representation uses the features at nine fixed sampling points to represent a bounding box with a deformable convolution. This representation can capture the geometry of a bounding box and its nearby contextual information, which is essential for encoding the misalignment between the predicted bounding box and the ground truth.  
Given a sampling location \((x,\ y)\) on the feature map, an initial bounding box from it is encoded by a 4D vector \((l^{\prime},\ t^{\prime},\ r^{\prime},\ b^{\prime})\), which is the distance from the location \((x,\ y)\) to the left, top, right, and bottom of the bounding box, respectively. With this distance vector, nine sampling points at \((x,\ y)\), \((x+l^{\prime},\ y)\), \((x,\ y-t^{\prime})\), \((x+r^{\prime},\ y)\), \((x,\ y+b^{\prime})\), \((x-l^{\prime},\ y-t^{\prime})\), \((x+l^{\prime},\ y-t^{\prime})\), and \((x+r^{\prime},\ y+b^{\prime})\) are mapped onto the feature map. Their relative offsets to the point \((x,\ y)\) serve as the offsets for the deformable convolution; then, the features at these nine projected points are convolved by the deformable convolution to represent a bounding box.  
The second sub-branch aims to predict the IoU-Aware Classification Score (IACS) [39], which is a joint representation of object presence confidence and localization accuracy. It is defined as a scalar element of a classification score vector, where the value at the ground truth class label position is the Intersection over Union (IoU) between the predicted bounding box and its ground truth, and is 0 at other positions. The star-shaped bounding box feature representation is used for IACS prediction. It has a similar structure to the localization sub-branch except that it outputs a vector of the class number and elements per spatial location, where each element jointly represents the object presence confidence and localization accuracy.  
The third sub-branch has a similar structure to the classification sub-branch in that it aims to predict \(k\) mask coefficients on a pixel-by-pixel basis by a small full convolution network (FCN) applied to each predicted bounding box. The \(k\) mask coefficients correspond to each prototype encoding the representation of an instance coming from the prototype mask branch (here, \(k\) is set to 32).