# Multimodal Network for Object Detection Using Channel Adjustment and Multi-Scale Attention
## 3 Methodology
### FPN, Detector and Loss Function  
Let the final output feature map from the Multi-scale Attention Mechanism be \(x\), where \(x\) is a multi-scale feature map that encapsulates information from the input image at various spatial resolutions. The dimensions of \(x\) are \(H\times W\times C\), where \(H\) is the height, \(W\) is the width, and \(C\) is the number of channels. This feature map \(x\) is passed into the YOLOX Feature Pyramid Network (FPN) for further processing. The YOLOX-FPN is designed to effectively merge multi-scale features from the input and perform feature extraction across different spatial resolutions. It improves detection performance by consolidating information from various levels of the feature hierarchy. Specifically, the FPN receives input feature maps \(x_{i}\) from multiple stages and produces a consolidated feature map, \(f_{fpn}\), which can be mathematically represented as follows:  
\[\text{FPN}(x_{1},x_{2},x_{3},x_{4},x_{5})\to f_{fpn} \tag{12}\]  
where \(f_{fpn}\) is the output of the FPN, containing enhanced multi-scale features suitable for object detection. These features are subsequently passed to the YOLO detection head for final prediction. The YOLO detection head takes the multi-scale features from \(f_{fpn}\) and performs object detection by predicting the bounding box coordinates, object confidence scores, and class probabilities. The detection process can be represented as follows:  
\[y=\text{YOLO Detection Head}(f_{fpn}) \tag{13}\]  
where \(y\) is the final prediction output. The output for each grid cell in the YOLO model can be expressed as follows:  
\[y_{i}=(b_{i},p_{c},p_{\text{class}}) \tag{14}\]  
\(y_{i}\) includes the bounding box coordinates, \(b_{i}=(x,y,w,h)\), where \(x\) and \(y\) represent the center of the bounding box, and \(w\) and \(h\) represent its width and height, respectively. Additionally, \(p_{c}\in[0,1]\) denotes the object confidence score, and \(p_{\text{class}}\in\mathbb{R}^{K}\) represents the class probabilities for each detected object. For multi-object detection, the total loss \(L\) is computed as the sum of three main components: bounding box loss \(L_{\text{box}}\), object confidence loss \(L_{\text{obj}}\), and classification loss \(L_{\text{cls}}\). The total loss is represented as follows:  
\[L=\lambda_{\text{box}}L_{\text{box}}+\lambda_{\text{obj}}L_{\text{obj}}+ \lambda_{\text{cls}}L_{\text{cls}} \tag{15}\]  
where \(L_{\text{box}}\) measures the error between the predicted and true bounding box coordinates, \(L_{\text{obj}}\) measures the object confidence score loss, and \(L_{\text{cls}}\) measures the classification error. Each loss component is weighted by a factor \(\lambda\) to balance their contributions to the total loss. The bounding box loss \(L_{\text{box}}\) measures the discrepancy between the predicted bounding box \(b_{i}\) and the ground truth \(b_{\text{true}}\). This work adopts the Generalized Intersection over Union (GIoU) loss, which is formulated as follows:  
\[L_{\text{box}}=1-\text{GIoU}(b_{i},b_{\text{true}}) \tag{16}\]  
where \(\text{GIoU}(b_{i},b_{\text{true}})\) extends the traditional IoU by considering the smallest enclosing box, improving optimization when predicted and ground truth boxes do not overlap. The object confidence loss \(L_{\text{obj}}\) is computed using focal binary cross-entropy, which helps address the issue of class imbalance by focusing more on hard-to-classify positive samples. The loss function is defined as follows:  
\[L_{\text{obj}}=-\kappa(1-p_{c})^{\delta}\log(p_{c})\quad\text{for positive samples} \tag{17}\]  
where \(p_{c}\) represents the predicted object confidence score, indicating the model's predicted probability that an object exists in the given grid cell. \(\kappa\) is a scaling factor that adjusts the loss magnitude, and \(\delta\) is the focusing parameter, which controls the strength of the penalty on well-classified examples, making the loss more sensitive to misclassified positive samples.