# Spatiotemporal Contextual 3D Semantic Segmentation for Intelligent Outdoor Mining
## 2 Methodology
### Spatiotemporal Clue-Gated Encoder  
Living organisms perceive, analyze, and make decisions about external objects using spatiotemporal information. For example, when humans observe an object, they combine visual input with existing experiences and memories to make a comprehensive judgment about the object's shape, category, and potential behavior, thereby enabling an appropriate response [25]. This ability relies on the efficient integration of historical information and the precise capture of current information. Moreover, humans can flexibly adjust the importance of the information they rely on based on specific circumstances, thereby improving the accuracy of their decisions. For instance, when encountering familiar objects, humans tend to rely more on memory, while for unfamiliar new objects, they tend to rely more on real-time perception. Similarly, in the field of AI, fully mining and leveraging spatiotemporal information can emulate the decision-making capabilities of living organisms, achieving more efficient and accurate responses in complex environments. Based on these observations and analyses, we propose the spatiotemporal clue-gated encoder, as shown in Figure 2, which processes both the serialized sequence frame data (\(X_{t}^{s}\)) and the current frame data (\(X_{t}\)) obtained from the point cloud input unit. These data are passed through separate convolutional layers for preliminary encoding. The encoding process is expressed as follows:  
\[Y=\sigma(\text{Norm}(\text{SpC}(X))) \tag{2}\]  
Figure 2: Architecture of the spatiotemporal clue-gated encoder. Here, SpC represents 3D sparse convolution; Norm refers to batch normalization; and \(\sigma\) is the activation function, which uses the ReLU function. Three-dimensional sparse convolution is advantageous for extracting point cloud features while maintaining a relatively low computational cost. After encoding, the sequence features (\(X_{SF}\)) and the current features (\(X_{CF}\)) are obtained. To capture the spatiotemporal clues hidden in the sequence features (\(X_{SF}\)), a spatiotemporal clue-embedding layer is designed to further process these features. The processing is described by the following equations:  
\[X_{SF}^{in}=\sigma(\text{Norm}(\text{Linear}(X_{SF})) \tag{3}\]  
\[X_{STF}=\text{MambaBlock}\Big{(}X_{SF}^{in}\Big{)} \tag{4}\]  
Here, Linear represents a linear transformation layer, and \(\sigma\) uses the GELU activation function. For the sequence features (\(X_{SF}\)), the linear layer first correlates the features within the sequence and maps them to a high-dimensional space to facilitate the interaction of long-term dependency information. This process helps effectively relate the spatiotemporal clues embedded in the sequence, providing the foundation for subsequent clue extraction. Then, the features processed by the linear layer are input into the Mamba block (further details on the Mamba block can be found in Section 2.3) to further mine and capture the deeper spatiotemporal clues. The long-term spatiotemporal features (\(X_{STF}\)), which are efficiently integrated and encoded, are obtained through the spatiotemporal clue-embedding layer. To enable the model to flexibly adjust the dependence between current information and historical information based on specific circumstances, we propose a gating unit to dynamically balance the long-term spatiotemporal features (\(X_{STF}\)) and current features (\(X_{CF}\)). The process is expressed as follows:  
\[GaR=Sigmoid(\text{Linear}(X_{CF})) \tag{5}\]  
\[X_{STAF}=X_{CF}\times GaR+X_{STF}\times(1-GaR) \tag{6}\]  
The gating mechanism first generates a weight representation (\(GaR\)) through a linear layer and an activation function (\(Sigmoid\)). Then, \(X_{STF}\) and \(X_{CF}\) are multiplied by their respective weight representations, allowing the model to select the most useful features during the learning process. Finally, these weighted features are fused through element-wise addition to obtain the effectively aggregated long-term spatiotemporal features (\(X_{STAF}\)).