# Multimodal Attention-Based Instruction-Following Part-Level Affordance Grounding
## 4 Experiments and Evaluation
### Comparison Results (Rq1)  
To demonstrate the effect of our model, we conducted comparative analyses with three distinct methodological paradigms: image-only segmentation methods, cascaded methods, and multimodal-based RIS methods. Image-only methods were selected to determine the influence of visual data in isolation, providing a baseline to analyze the impact of incorporating linguistic instructions. Cascaded methods and multimodal-based methods were chosen to represent common practical applications and to assess how the fusion of text and image data informs the performance within these contexts. Specifically, we selected \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**NumberDataset**} & \multicolumn{2}{c}{**ITT-AFF VL**} & \multicolumn{2}{c}{**UMD VL**} & \multicolumn{1}{c}{**All**} \\ \cline{2-6}  & **Image** & **Instructions** & **Image** & **Instructions** & **Pairs** \\ \hline \multirow{2}{*}{
\begin{tabular}{c} contain \\ cut \\ display \\ \end{tabular} } & 1143 & 1143 & 949 & 949 & 2092 \\  & 101 & 101 & 790 & 790 & 891 \\  & 1594 & 1594 & - & - & 1594 \\  & 71 & 71 & - & - & 71 \\  & 1617 & 1617 & 2611 & 2611 & 4228 \\  & 414 & 414 & - & - & 414 \\  & 180 & 180 & 419 & 419 & 599 \\  & support & 440 & 440 & 1060 & 1060 & 1500 \\  & 926 & 926 & 671 & 671 & 1597 \\  & - & - & 570 & 570 & 570 \\  & 721 & 721 & 1010 & 1010 & 1731 \\  & All & 8835 & 8835 & 7070 & 7070 & 13,556 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Statistics of instruction-following affordance-grounding dataset. image-only segmentation methods (DeepLabV3+ [74], OCRNet [77], AffordanceNet [62], RelaNet [63], BPN [14], GSE [15], and ADOSMNet [16]) that are considered benchmarks within the field of affordance segmentation. It is important to note the fundamental difference between image segmentation and affordance segmentation tasks. While image segmentation predicts masks for an entire image indiscriminately, affordance segmentation is more intricate, often requiring a pre-processing step for object detection or an integrated approach that accomplishes object detection and segmentation simultaneously. In the realm of multimodal approaches, we incorporated cascaded methods (combining Faster-RCNN [40] and DeepLabV3+ [74]) owing to their prevalent adoption in practical scenarios, thus enabling an evaluation that reflects real-world application performance. Furthermore, we evaluated our model against a state-of-the-art RIS method (BKINet [56]), which represents the latest innovations in multimodal understanding. These baselines were chosen due to their proven robustness and widespread recognition in the literature, providing a well-understood standard from language-independent to language-integrated approaches, against which to measure our advancements. The details of these methods are as follows:  
1. DeepLabV3+ [74]: The semantic segmentation method DeepLabV3 was employed for its semantic segmentation capabilities. By fine-tuning this method in our dataset, we aimed to assess the additional impact, if any, that language processing contributes to overall affordance-grounding performance. 2. OCRNet [77]: OCRNet is notable for its object-contextual representations for semantic segmentation, which characterize each pixel by leveraging the features of its corresponding object class. Since affordance segmentation is closely related to object class, OCRNet serves as an appropriate image-only baseline for our study. 3. AffordanceNet [62]: AffordanceNet, a classical affordance detection method, segments images based on affordance categories without the integration of language instructions, providing a baseline for comparing language-agnostic approaches. 4. RelaNet [63]: RelaNet is an end-to-end affordance segmentation method without separate and intermediate object detection steps. It considers relationships between the affordance and corresponding objects. Because this dataset is a closed-source dataset, we employed the results of the IIT-AFF dataset reported in the original papers (the backbone with ResNet50). 5.