## 2 Related Work
### Image Segmentation  
Image segmentation is typically categorized into three main types: semantic segmentation, instance segmentation, and panoptic segmentation. Semantic segmentation predicts labels for each pixel, providing class information without distinguishing individual instances. In contrast, instance segmentation not only identifies object classes but also distinguishes different instances of the same class. For example, it can label and differentiate multiple cars in an image. Panoptic segmentation combines both semantic and instance segmentation, representing uncountable label classes as single masks on the image, such as'sky' or 'grass' [27; 28]. Semantic segmentation traditionally relies on convolutional neural network models. A fully convolutional network replaces the fully connected layer of VGG-16 with a convolutional layer and implements skip architectures to refine object positioning [29]. U-Net follows an encoder-decoder architecture similar to that of an autoencoder [30]. SegNet utilizes the entire convolution layers of VGGNet with pretrained weights [31]. DeepLab v3+ introduces atrous convolution to preserve spatial information without pooling steps [32]. Semantic segmentation is typically rooted in convolutional neural network models. To make image segmentation user-interactive, interactive segmentation methods have been developed. Deep interactive object selection [33] utilizes user-provided points and FCN models for processing, enhancing interaction accuracy with fewer user inputs. Latent diversity [34] addresses ambiguity in suggesting mask candidates. f-BRS [35] introduces zoom-in segmentation by generating bounding boxes from input points and limiting regions. RITM is a click-based interactive segmentation algorithm using DeepLabV3+ [36] and HRNet + OCR as backbones. FocalClick [37] and SimpleClick [38], derived from RITM, showed some refinements. Due to difficulties in modifying an initial mask inferred from RITM, FocalClick proposed Progressive Merge by carrying out morphological analysis. SimpleClick first used ViT as its backbone, as it has a plain, non-hierarchical structure, showing state-of-the-art performance. Interactive segmentation with polygon input has the role of a supplementary tool for object and instance annotation [39; 40; 41]. The process makes annotators easily engage in the segmentation process and in the correction of polygons. In video segmentation, users can enhance the mask quality through interactions with various input methods, such as polygons, scribbles [42], and clicks. LIVEcut [43], for instance, employs a graph-cut optimization framework and per-frame correction to improve segmentation accuracy. Recent studies have demonstrated various innovative uses of segmentation in AR contexts. For instance, a novel semantic segmentation method involving a modified dilated residual network and atrous pyramid pooling has been proposed to enhance AR applications. This method has shown high accuracy and efficient real-time processing, making it particularly well suited for natural AR experiences, such as scenarios requiring fast interaction, including augmented navigation or dynamic object tracking [44]. Additionally, work such as \"Fakeye: Sky Augmentation with Real-time Sky Segmentation and Texture Blending\" [45] has explored AR applications for large-scale environments such as the sky, addressing challenges such as real-time occlusion and the blending of real and virtual content [45]. One recent example of interactive segmentation is the Segment Anything Model, which is designed to accommodate prompts effectively [46]. SAM's lightweight mask decoder enables fast predictions, making it advantageous for zero-shot, prompt-able, and robust segmentation. SAM has been employed in experiments due to its speed and performance. SAM has been extended to encompass video applications. For example, TAM [47] facilitates one-pass inference in an interactive manner by using an object tracker called XMem [48], allowing users to intervene and refine masks during the inference process.