# Sem-SLAM: Semantic-Integrated SLAM Approach for 3D Reconstruction
## 3 Methodology
### Forward Feature Fusion  
In the context of this research, the Forward Feature Fusion method is adopted due to its strong relevance and multiple advantages. In 3D scene reconstruction and semantic understanding tasks, different types of features (geometric, semantic, and appearance features) carry complementary information. However, traditional methods often struggle to fully exploit the potential of these features, resulting in limited model performance. Figure 1: Our proposed method (Sem-SLAM). The method first uses a forward feature fusion network to integrate multi-scale and multi-modal features from the input data. Then, a backward semantic mapping network takes the fused features to perform environmental semantic modeling, including semantic map construction. This two-step process enables efficient and accurate environmental modeling. The Forward Feature Fusion method proposed in this paper addresses these limitations. It fuses geometric, semantic, and appearance features through cross-attention and self-attention mechanisms, enabling the model to capture the inherent relationships among different features. This not only improves the feature representation ability but also makes the model more adaptable to complex scenes. Moreover, by integrating pre-trained semantic segmentation networks with attention mechanisms [22], we can further enhance the fusion effect of different features, achieving better performance in 3D scene reconstruction and semantic understanding. Subsequently, we obtain appearance features via a Multi-Layer Perceptron (MLP) network. Regarding geometric features, we process information such as rays using an MLP network to acquire them. By performing calculations with cross-attention and self-attention, we obtain the fused features. This approach effectively combines the complementary information of different feature types, enabling the model to better understand and represent the scene in the reconstruction process. In the network we proposed, the cross-attention module plays a crucial role. It skillfully merges geometric features, appearance features, and semantic features, fully exploring and leveraging the respective advantages of these three types of features. Specifically, this module effectively integrates different types of features through specific operation mechanisms. During this process, the output of the cross-attention module is directly used as the input of the self-attention network. The self-attention module is designed with the original intention of further highlighting the key and important information within these three types of features. It can adaptively assign attention weights based on the internal correlations of the features themselves, so that the important feature information can be strengthened. After being processed by the self-attention module, we deeply fuse these three types of features and finally generate a feature map that contains rich information. This feature map integrates information from multiple aspects such as geometry, appearance, and semantics, providing a more comprehensive and representative input for subsequent related tasks. While pretrained 2D semantic segmentation networks have been widely adopted, our key innovation lies in their organic integration with cross-attention and full-attention mechanisms. The pretrained networks provide reliable semantic priors, whose fusion effectiveness with other features (e.g., geometric and shape features) is further enhanced through self-attention and full-attention mechanisms. This groundbreaking integration has yielded significant improvements in 3D scene reconstruction and semantic understanding-achievements unattainable by traditional approaches relying solely on pretrained networks. **Cross-Attention**  
In this paper, we adopt a fully cross-attention mechanism that encompasses all three types of features.