# Construction of Three-Dimensional Semantic Maps of Unstructured Lawn Scenes Based on Deep Learning
## 3 Materials and Methods
### Dataset Descriptions  
#### 4.1.1 Image Annotation  
We used the open-source semantic segmentation image labeling tool ISAT_with_segment_anything [35] for image annotation. This tool is built on top of the latest semantic segmentation model, Segment Anything [36], which allows users to utilize the model's prediction results to assist in labeling, significantly improving efficiency and effectively reducing our workload. The model achieved higher accuracy on multiple datasets compared to previous models, with higher segmentation accuracy and generalization capability, thereby enhancing the quality of the annotations. Additionally, the tool employs interactive logic, allowing for iterative improvements on selected areas to ensure the desired accuracy. Specifically, when labeling lawn areas, for example, users only need to click on any point within the lawn area and the tool will invoke the model for prediction and generate a drag-gable polygon based on the prediction results. The accuracy of whether objects are correctly labeled primarily depends on whether the labeled contours match the outer boundaries of the objects. If the generated prediction results do not align with our desired outcome, corrections can be made within the tool through smudging and dragging operations to ensure accuracy in labeling. Compared to traditional tools like Labelme (where labeling blocks requires outlining their entire contours), using this tool for annotation is much faster. The final annotation results for each object are manually verified by us to ensure accuracy and are then used as ground truth images. In the experiments, the dataset was divided into a 70:30 ratio for training and testing.  
The interface of the annotation tool is as shown in Figure 8, where the red area represents the lawn region automatically predicted by the model Segment Anything. We can further adjust it to meet our desired objectives.  
Figure 8: ISAT_with_segment_anything tool.  
We manually divided the collected images into seven categories: Background (cls0, black), Lawn (cls1, red), Trunk (cls2, green), Edge (cls3, yellow), People (cls4, blue), Obstacle (cls5, purple), and Bushes (cls6, cyan). The original image was a 24-bit RGB image, and its corresponding visual labels are illustrated in Figure 9.  
Our explanation of the annotation situation is based on the working conditions of common gardening pruning robots. \"Lawn\" represents the robot's working area, while \"Edge\" delineates the boundaries of the robot's operation. The weeding tasks of the robot are carried out within this boundary, and the yellow-bordered area serves as a traditional electronic fence. Within the robot's working area, it is not limited to only the lawn; sometimes, objects such as \"Bushes,\" an \"Obstacle,\" or a \"Trunk\" may appear in the lawn area. In this study, these three types of objects are taken as representatives of obstacles, forming prohibited areas for the robot. During the pruning process, there are always \"People\" appearing nearby. These individuals might be supervising the robot's work or simply passing by. Unlike other static objects, \"People\" are dynamic and somewhat special, so they need to be annotated separately.