# 6G Technology for Indoor Localization by Deep Learning with Attention Mechanism
## 3 CNN with Attention Mechanism  
In this paper, a two-dimensional CNN is applied, as shown in Figure 2. The input is the CSI with M \(\times\) N dimensions. It consists of a contracted network, including the repeated additional \(3\times 3\) convolution layers, Batch normalization layers, and rectified linear unit (ReLU) layers. The \(2\times 2\) max-pooling layer is used in the shrink network pooling layer. The softmax layer is for the Attention mechanism. Finally, a \(1\times 1\) convolution layer is used in the fully connected layer.  
The convolution layer is primarily responsible for detecting features in the input image, specifically the CSI in our paper. This is achieved by converting the image into a matrix and applying various convolutional filters that move across the matrix to generate feature maps. These maps capture critical elements of the image with different filter sizes and strides. The rectified linear unit (ReLU) layer introduces nonlinearity, enabling the network to identify complex patterns and make more accurate decisions. Although this rectification process improves the network's performance during training, the effects might not be immediately obvious. The pooling layer's purpose is to shrink the dimensions of the feature maps, thereby decreasing the number of parameters that need to be trained and helping to prevent overfitting. This is accomplished by selecting the maximum value within small regions of the feature map, ensuring that the regions do not overlap. In the  
Figure 2: CNN architectures with multiple \(3\times 3\) convolutional layers, pooling layers, softmax layer (attention mechanism), and fully connected layers for localization.  
fully connected layer, all neurons are interlinked, allowing the network to classify the features based on their weights. This layer performs the final decision-making function, synthesizing the information to make predictions. The (2) is the minimization equation for position \((x,\ y)\) can be written as  
\[\underset{A_{i},i}{argmin}:\sum_{N=1}^{N_{t}}f(A_{i}(r^{\alpha}),\ r)+Q(i) \tag{2}\]  
In this context, \(A_{i}\) represents the neural network-like architecture, where \(i\) denotes the neural network parameter, \(f\) is the error of the position, and \(r^{\alpha}\) and \(r\) refer to the approximate position \((x^{\alpha},\ y^{\alpha})\) and exact position \((x,\ y)\). \(Q(i)\) is the regularization functions. We chose the Adaptive Moment Estimation (ADAM) as the optimization algorithm for deep learning. This advanced algorithm combines the advantages of two other popular methods: Adaptive Gradient Algorithm (AdaGrad) and Root Mean Square Propagation (RMSProp). ADAM computes adaptive learning rates for each parameter by estimating the first and second moments of the gradients (mean and variance). This approach allows ADAM to adjust the learning rate dynamically, making it particularly effective in handling sparse gradients and noisy data. Additionally, ADAM includes bias correction, especially during the initial stages of training. Its efficiency and robustness make it a widely used optimizer in deep learning methods, as described in [34].  
The input to the CNN is channel fingerprint information. The channel fingerprint information of the reference point is used for feature training in the model. The position of the reference point is regarded as a label. After training, positioning is performed at any location. The regression function is used to obtain the x and y positioning coordinates.