# Concatenated Attention: A Novel Method for Regulating Information Structure Based on Sensors
## 2 Methods  
### Split and Concatenate  
In the self-attention mechanism, the input data, after preprocessing, is transformed into three matrices of identical size, which form the core of the self-attention mechanism. These matrices--commonly referred to as the Query (\(Q\)), Key (\(K\)), and Value (\(V\)) matrices--encapsulate the essential features of the input data. The results obtained from matrix multiplication and normalization represent the relationships between individual data elements, enabling the network to identify which parts of the input data are most important for the current task. The matrix computation process is illustrated in Figure 1. Specifically, the \(Q\) matrix is multiplied by the \(K\) matrix to produce an intermediate matrix, which represents the association scores between units of input data. These scores, after normalization, are transformed into a probability matrix, where higher probabilities indicate a greater likelihood of selecting important information as features. This probability matrix is then applied in a matrix multiplication with the \(V\) matrix to generate the self-attention matrix, which encodes the relationships among features. This process strengthens the network's ability to capture long-range dependencies and contextual relevance, effectively mitigating the issue of the network forgetting initial features.  
To enhance the diversity of input data features, we propose optimizing the neural network's feature perception through a split-and-concatenate method. This approach increases the informational richness of the self-attention mechanism by introducing additional flexibility and granularity in the representation of intermediate variables. The detailed process of the proposed CA method is shown in Figure 2. After generating the three core matrices \(Q\), \(K\), and \(V\), the \(Q\) and \(K\) matrices are first equally split along the same dimension, ensuring that the resulting submatrices are of equal size. This guarantees that the dimensions of the subsequent result matrices will also remain consistent. This splitting operation introduces variability into the computation process, as the submatrices not only focus on different parts of the data's feature space but also enable fine-tuning when computing \(QK^{T}\).  
Next, the resulting submatrices are multiplied pairwise along the split dimension, producing two intermediate matrices of the same size as the original result. Compared to traditional methods, these intermediate matrices capture finer-grained relationships between data elements, allowing the network to better differentiate subtle variations in input features. The two matrices are then concatenated and subsequently multiplied by the \(V\) matrix to generate the final \(CA\) matrix. This concatenation step reintroduces the feature diversity captured by the intermediate matrices, ensuring that the final result reflects a richer and more comprehensive representation of the input data.  
This approach modifies only the computation order and direction while preserving the original size of the matrices. Consequently, it does not introduce additional computational overhead or increase the number of parameters, making it a computationally efficient solution. Furthermore, the method is fully compatible with existing self-attention mechanisms and can be seamlessly integrated into a wide range of neural network architectures. By enriching the representation of intermediate variables, the proposed method enhances the network's capacity to learn meaningful features from input data, particularly in scenarios with limited training samples or complex environmental conditions.  
Figure 1: The computational structure of the self-attention mechanism.  
Figure 2: The computational structure of the concatenated attention method.