# Text-Guided Refinement for Referring Image Segmentation
## 1 Introduction
The final segmentation mask prediction is obtained through subsequent  
Figure 1: (**a**) Two referring segmentation examples of different scales in the same image, as predicted by the existing method. (**b,c**) Two multi-modal fusion mechanisms. While existing methods perform language and vision fusion at the encoder stage, the proposed method achieves this in the decoder, incorporating multi-scale interactions across three levels. upsampling operations and decoding networks. However, this strategy typically restricts multi-modal interactions to the encoding stage before the output side of the network, combining linguistic and visual features independently at each scale [10; 11; 12], as illustrated in Figure 1b. Such methods fail to leverage the continuous guidance of language to enhance cross-scale multi-modal fused features. Moreover, repetitive strides and pooling operations in convolutional neural networks often lead to the loss of critical fine-structural information, yet few referring segmentation methods explicitly address the issue of detail recovery. To overcome these challenges, we propose to integrate multi-modal and multi-scale information with continuous linguistic guidance at the decoding stage. This approach allows models to progressively recover fine details and improve segmentation accuracy, delivering precise and fine-grained results even for complex targets. Recently, some approaches [13; 14; 15; 16] have utilized pre-trained multi-modal large models, such as CLIP [17] and Segment Anything [18], which have gained attention for their versatility in diverse tasks. However, their optimization primarily focuses on global semantic alignment rather than pixel level details, resulting in poor edge precision for object boundaries described in natural language prompts. Additionally, their large size poses practical difficulties in deployment, particularly for resource-constrained applications. These drawbacks highlight the value of designing lightweight referring segmentation methods that are not based on pre-trained large models but focus on integrating multi-modal and multi-scale information to achieve precise and fine-grained segmentation results. To resolve the aforementioned problems, we propose the Text-Guided Refinement Network (TGRN) for referring segmentation, leveraging gated mechanisms to selectively aggregate multi-scale features. Instead of performing cross-modal information fusion at the feature encoding stage, we adopt a decoder fusion strategy that progressively aligns and refines multi-level cross-modal features with linguistic guidance, as illustrated in Figure 1c. A cascaded pyramid structure is utilized to construct multiple feature pyramids from multi-modal features at varying scales, significantly enhancing the network's capacity to capture and perceive objects of diverse sizes. This design comprises two key components: TGFusionNet for multi-modal cross-scale feature alignment and RefineNet for progressive detail refinement. In TGFusionNet, a text-guided gated fusion mechanism dynamically highlights image regions most relevant to the referring expression, reducing interference from irrelevant areas and enhancing semantic consistency. The gating function ensures precise feature selection and adaptive fusion, enabling the model to handle objects of diverse sizes effectively. To further refine the segmentation output, RefineNet incorporates multi-scale context and boundary cues, progressively recovering fine details and addressing the critical challenge of accurate edge prediction. By integrating linguistic guidance throughout both components, the TGRN bridges the gap between referring expressions and images, achieving fine-grained and precise segmentation results. The main contributions of this paper are as follows:  
* We propose the Text-Guided Refinement Network, which utilizes a novel text-guided multi-scale feature decoding approach, integrating hierarchical image features through a dynamic gating mechanism.