# Generalization Investigation for Artificial Intelligence-Based Positioning in IIoT
## 5 Performance Evaluation and Discussion
### Scenario 2: Impact of Non-Ideal Network Synchronization  
In this section, we test the generalization performance with non-ideal network synchronization.  
* Training manner 1: Training data set with ideal network synchronization and test data set with network synchronization error Firstly, we test the case in which the AI/ML model is trained by a data set with ideal network synchronization and then is tested by data set with network synchronization error with a standard deviation of 50 ns. The results are summarized in Figures 16 and 17. It is observed that no matter in the direct AI-based positioning method or the AI-based ToA prediction method or no matter with clutter parameter {60%, 6 m, 2 m} or with clutter parameter [40%, 2 m, 2 m], the positioning performance is quite poor and the positioning error at 90% CDF is more than 10 m.  
Figure 14: Performance of direct AI/ML-based positioning with fine-tuning.  
Figure 15: Performance of AI/ML-based ToA prediction with fine-tuning.  
* Training manner 2: Mixed network synchronization error setting for training In this case, the data set for training is constructed by mixing data samples with different synchronization error settings. In the evaluation, standard deviations of 50 ns, 40 ns, 30 ns, and 0 ns are used for the training data set generation. Figures 18 and 19 show the performance for direct AI-based positioning and AI-based ToA prediction when the test data set is generated by using a 50 ns standard deviation. According to the simulation results, it is observed that, compared with training manner 1, using mixed network synchronization error setting could improve the positioning performance. Yet, on the other hand, the improvement is not so significant. For clutter parameter [40%, 2 m, 2 m], the positioning error @90% is around 3 m for both direct AI-based positioning and AI-based ToA prediction, which is a little far from the requirement in the IIoT scenario.  
Figure 16: Performance of direct AI/ML-based positioning with different network synchronization settings between training and test.  
Figure 17: Performance of AI/ML-based ToA prediction with different network synchronization settings between training and test.  
* Training manner 3: Fine-tuning  
For each clutter parameter setting, the corresponding AI model is first trained by using ideal network synchronization. Then, fine-tune the AI model by a using small number of samples generated with the same network synchronization setting as that of the test data set. Figures 20 and 21 show the performance for direct AI-based positioning and AI-based ToA prediction when the AI model is fine-tuned by using data set with a 50 ns standard deviation and using test data set with a 50 ns standard. It is observed that using fine-tuning could improve positioning performance to some extent. Yet the improvement is quite limited. In addition, the positioning error at 90% CDF is around 5 m and 7 m for clutter parameters of (60%, 6 m, 2 m) and (40%, 4 m, 2 m), respectively. There is still a large gap between the requirement of IIoT. Moreover, compared with the training manner 2, the improvement is small due to fewer training samples with similar features to that of the test data set.  
Figure 19: Performance of AI-based ToA prediction with mixed network synchronization for training.  
Figure 18: Performance of direct AI-based positioning with mixed network synchronization error for training.