## 5 Model Training
### Performance of the Dataset When Applied to Other Models
#### 5.4.2 Performance of the Dataset on TKAN  
Kolmogorov-Arnold Networks represent a promising alternative to Multi-Layer Perceptrons (MLPs), while TKAN, a novel neural network architecture inspired by both KAN and LSTM, stands for Temporal Kolmogorov-Arnold Networks. TKANs combine the strengths of both networks and consist of Recurrent Kolmogorov-Arnold Networks (RKANs) layers with embedded memory management. This innovation enables multi-step time series forecasting with enhanced accuracy and efficiency. By addressing the limitations of traditional models in handling complex sequential patterns, the TKAN architecture offers significant potential for advancement in fields requiring one-step-ahead forecasting [45].  
We have implemented a language model based on the KAN architecture, specifically utilizing a variant of the Kolmogorov-Arnold Network (KAN) known as the Temporal Kolmogorov-Arnold Network (TKAN). TKAN integrates the advantages of both KAN and LSTM, managing memory through embedded recurrent KAN layers (RKANs) to enhance the accuracy and efficiency of multi-step time series predictions. This architecture is particularly suitable for domains requiring advance predictions, as it overcomes the limitations of traditional models when dealing with complex sequential patterns.  
The Config class, which defines the model's configuration, includes parameters such as vocabulary size, sequence length, and model dimensions. Key components of the model include SinusoidPE for sinusoidal positional encoding, SelfAttention representing a multi-head self-attention mechanism, and FeedForward, a feed-forward network. Notably, the linear layer within the FeedForward network employs KANLinear, a substitute for the traditional nn.Linear, to leverage the advantages of the TKAN architecture. The main body of the model, KAN-Model, integrates these components, utilizing KANLinear layers in place of conventional linear layers for the complex task of multi-step time series predictions. Additionally, it facilitates text generation and embedding representation retrieval.  
In the training and evaluation section, the code provides functions to calculate accuracy, precision, recall, and F1 scores, as well as functions to plot the changes in loss and accuracy during the training process. The training function, named train, utilizes the AdamW optimizer and performs forward propagation, loss calculation, backpropagation, and parameter updates in each iteration. Following the completion of training, the model is evaluated on the entire training set.  
Based on Figure 12, we can further analyze the performance of the TKAN model during the actual training process. As indicated by the data, the model achieved a loss value of 0.346 during training, suggesting that the model is able to effectively fit the patterns in the training data. However, the model also exhibited an accuracy of 0.687, which, as shown in Table 5, indicates that the accuracy and scores of the parameters are not ideal after 7.1 h of training. This suggests that the model's prediction performance in inference tasks is not satisfactory.  
Figure 12: Loss function curve and accuracy curve graph for the TKAN model.