# Wheel Odometry with Deep Learning-Based Error Prediction Model for Vehicle Localization
## 3 Transformer Model  
As the output of wheel odometry is affected by many factors, the measurement error has high uncertainty and keeps changing and accumulating over time. Transformer is a deep learning model that processes time series, and the use of Transformer is studied in this  
Figure 1: Role of the wheel odometry error prediction model in the localization system. paper to learn the error characteristics of wheel odometry from time series for more accurate localization. In this section, a Transformer-based error prediction model is established to predict the error of the wheel odometry output. The Transformer model is based on encoder/decoder architecture. The original input is translated into the model input containing location information through embedding and location coding. The encoder consists of a stack of N identical layers with two sub-layers. The first sub-layer is a multi-head self-attention layer used to extract the features of different attention heads of the model. The second sub-layer is a fully connected feed-forward network, which can enhance the nonlinear representation ability of the model. Both sub-layers use residual connection and normalization. Residual connection further enhances the fitting ability of the model, and normalization can avoid the problem of the slow convergence of the model caused by the parameters being too large or too small. The decoder is also composed of N identical layers stacked. In addition to the two sub-layers in each encoder layer, the decoder inserts a third sub-layer, which is used to perform a multi-head attention calculation on the output of the last encoder. Similar to the encoder, the decoder uses residual connection and normalization for each sub-layer. The first self-attention sub-layer of the decoder is designed in a masked form to ensure that the prediction of position i depends only on the known output at positions less than i. The output of the decoder is converted to the required dimension through a linear layer, and the value with the highest probability calculated through the softmax layer is the final output of the model. Considering that different driving conditions may influence the wheel states and affect the odometry output, two different Transformer error prediction models are designed. The first model contains four-wheel wheel speed as the input to the encoder, and the second model considers the driving condition features and four-wheel wheel speed as the input to the encoder. Both models output the predicted error sequence. The Transformer model that does not consider the driving conditions is referred to as Transformer-NDC for convenience, and the Transformer model that considers the driving conditions is referred to as Transformer-DC for convenience. The structures of the two models are shown in Figure 2.  
Referring to the different driving conditions in the public dataset (IO-VNBD) [25] and common driving condition features, a total of 29 features describing different driving conditions were selected, as shown in Table 1. Features 1-9 describe different road types, features 10-15 describe different road conditions, and features 16-29 describe various vehicle driving operations. The data subsets are preprocessed in combination with the selected features. The feature value takes the value of 1 when the corresponding driving condition of the data subset contains the related feature; otherwise, it takes the value of 0.