## 1 Introduction  
Augmented reality (AR) extends our physical reality by overlaying digital information onto our environment, distinguishing it from virtual reality (VR), which constitutes an entirely digital realm [1]. AR devices are categorized into four types: wearables, handheld devices, projection-based displays, and holographic displays [2]. Among these, head-worn wearable devices, particularly AR glasses and headsets, have become widely recognized for their simplicity and convenience. These devices appear as simple eyewear, but provide a seamless means of accessing digital information. For example, in engineering classes, AR overcomes physical limitations by enabling visualized experiences in the classroom without the need for expensive equipment [3]. The essential aspect of merging the digital and real worlds in AR lies in image recognition, which is highlighted to analyze camera-captured images for a deep understanding of users' surroundings [4]. To achieve this comprehension, deep learning approaches have been employed, primarily relying on object detection and image segmentation [5, 6]. Object detection, a computer vision task, encompasses object localization and categorization, enabling AR to recognize and label objects accurately. Image segmentation divides an image into segments, which is crucial for constructing AR scenes. Moreover, as illustratedin Figure 1, interactive segmentation is a field that enables users to provide input in the form of clicks, polygons, scribbles, or masks. Prompt-based segmentation enables interactive segmentation through reflecting the user's intent within the system. Users can specify regions of interest (ROIs) or click on specific points within an image to enhance the segmentation accuracy. This provides more precise results than automatic segmentation alone, especially in cases with complex backgrounds or objects with similar colors. Deep learning approaches have been used in AR applications in various sectors, including driving, manufacturing, and interactive semantic segmentation. In the realm of driving, AR enhances the driving experience by providing real-time navigation guidance on actual roads, making journeys more convenient and safer [7,8]. In the manufacturing industry, AR devices offer instructions on equipment and roles to novice workers, assisting them in rectifying errors and improving efficiency [9,10]. Additionally, [PERSON] et al. developed a system for interactive semantic scene understanding, offering user-defined boundaries within a scene [11]. Such interactivity is considered crucial, as it significantly enhances the system's quality [2]. As shown in Figure 2, interactive segmentation allows users to refine object boundaries by providing direct input, such as clicks or drawings, to guide the segmentation process more accurately. We introduce a visual prompt selection framework that is designed to excel in distinguishing dynamic objects from their background. Unlike traditional segmentation algorithms, which require continuous retraining to accurately identify objects in images, our framework seeks to address this challenge. For example, effectively segmenting a moving object, such as the camel in Figure 3, would typically necessitate training the algorithm on images captured from various angles. The 360-degree view and mask results of non-rigid objects provide insights into how their shapes change when observed from different perspectives. Non-rigid objects, which are flexible and dynamic, include items with joints or flexible movements. Figure 1: An overview of interactive segmentation using input points to display the generated mask. Figure 2: An overview of interactive segmentation using input points. Input points are considered either positive or negative. Positive points are denoted by green dots on the left side of the figure, while negative points are represented as red dots. A pretrained backbone interprets the image, and a decoder generates a mask based on the input positive and negative points. The output displays the generated mask.