# Text-Guided Refinement for Referring Image Segmentation
## 1 Introduction  
Referring image segmentation, also known as text-prompt-based segmentation, is a growing research field focused on segmenting objects in images based on textual descriptions provided by users. It aims to segment entities based on natural language expressions by considering appearance, attributes, spatial relationships, and contextual cues relevant to the description. Unlike traditional segmentation methods, this task bridges the gap between language and images by requiring an integrated understanding of both modalities. It holds significant promise for diverse applications in technology and healthcare, including human-computer interaction, medical imaging, and immersive AR/VR experiences. In referring image segmentation, significant scale variations among objects in input images often challenge segmentation networks. These variations stem from two primary factors. First, objects of different categories inherently exhibit scale differences. Background categories like \"clouds\", \"walls\", and \"roads\" often occupy large and contiguous regions, while foreground objects like \"people\", \"cars\", and \"computers\" are typically segmented into smaller individual instances and present greater segmentation challenges. Second, scale differences also result from variations in camera perspectives, where objects of the same category appear at different scales, further complicating recognition and often producing inaccurate segmentation results. Large-scale objects may experience internal mis-classification, resulting in discontinuous predicted pixel labels, while small-scale objects may blur into the background, leading to imprecise edges or complete omission. To address these issues, continuous interaction between multi-scale visual features and textual information is essential. Such integration enhances the model's ability to recognize objects across varying scales and achieve precise segmentation, especially for complex edge structures. Early methods [1; 2; 3; 4; 5] primarily relied on the powerful learning capabilities of deep neural networks, such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs), to combine linguistic and visual features for segmentation mask prediction. However, these approaches often neglect variations in the representational capacity of multi-modal features for targets at different scales, leading to inaccuracies in segmentation results. To address this, subsequent research introduced multi-scale feature perspectives. Some methods [6; 7] adopt multi-resolution input strategies, such as image pyramids, to model objects at varying scales by fusing features across resolutions. This approach incurs high computational costs and overlooks the inherent multi-scale capabilities of convolutional networks. Other approaches [8; 9] utilize multi-scale features extracted directly from networks, avoiding the computational burden of image pyramids. However, simple fusion techniques like addition or concatenation often fail to capture the varying importance of scales, leading to suboptimal segmentation results. In Figure 1a, we present two referring segmentation examples based on the same image, showing the mask annotations for the referring targets and the segmentation results using an existing multi-scale feature fusion method. For a small target like \"light\", the method fails to detect it, leading to missed detections. Conversely, for a larger object like a \"person\", the target is accurately located, though partial omissions, such as missing leg areas, occur due to dim lighting. These examples highlight the need for more effective multi-scale feature fusion mechanisms to address such challenges. While multi-scale feature fusion methods improve the network's ability to perceive objects at varying scales, they often overlook the critical role of linguistic guidance. Previous approaches primarily rely on encoder-stage fusion strategies, where images and referring expressions are processed separately by CNNs or RNNs and then combined into multi-modal features.