# A Feature-Weighted SVR Method Based on Kernel Space Feature
## 4 Simulation Examples
### Synthetic Datasets
\[x^{\prime}_{ik}\!\!=\!\frac{x_{ik}-\mu_{k}}{\sigma_{k}} \tag{20}\]  
where \(\mu_{k},\sigma_{k}\) is the mean and standard deviation of the \(k\)-th feature, respectively. Firstly, the kernel matrix generated by feature weighting and by other methods is compared. In order to facilitate the comparison, when the parameter \(\gamma\) is selected, the result consistency of Feature 1 is calculated as the standard of nuclear element calculation, because weighting and normalization will change the value of the feature. The kernel matrix generated by the above method is shown in Figure 4.  
As can be seen from Figure 4, the feature weighting reduces the influence of Feature 2 on the kernel matrix. Figure 4a is similar to Figure 2b. However, the feature weighting preserves the information of Feature 2 compared to the feature selection. It can be deduced from Figure 4b,c that the normalization method can weaken the influence of Feature 2. However, its influence is still great. As the range of Feature 1 and Feature 2 is essentially the same in normalization, the contributions of Feature 1 and Feature 2 are much the same. When the range of unimportant features is greatly wider than that of important features, the normalization method can largely reduce the influence of unimportant features. On the contrary, the normalization method will increase their influence. Figure 3: Model generalization ability of different weights’ combination. Finally, we randomly chose seven UCIbenchmark datasets [35]. The grid search method is used to select the optimal combination of weights from the set \(\left\{10^{-4},10^{-3},10^{-2},10^{-1},10^{0}\right\}\). The optimal combination of weights is shown in Table 5, and the results are compared as in Table 6, in which bold values indicate the method with the best performance. The Wilcoxon signed rank tests [36] at the 0.05 significance level are implemented to test the differences between the feature weighting and other data pre-processing techniques to substantiate the indications in Tables 4 and 6. The test results are presented in Table 7. The prediction results on F1 and F2 using linear and sigmoid kernels were both unacceptable and were not included in this test. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Function Name** & **Kernel Type** & **Feature Weighting** & **Raw Data** & **Feature Selection** & **Min-Max Normalization** & **Z-Normalization** \\ \hline CCPP & Gaussian & **4.379** & 5.9036 & 4.7865 & 4.4532 & 4.4225 \\  & linear & 4.6580 & 5.7354 & 5.0788 & **4.6372** & 4.6559 \\  & sigmoid & 17.3939 & 17.3939 & 17.3939 & 4.6706 & **4.6641** \\ \hline Airfoil & Gaussian & **2.6293** & 6.2425 & 6.2426 & 3.0588 & 3.0773 \\  & linear & 5.0954 & 9.3584 & 6.2426 & **4.0928** & 4.9029 \\  & sigmoid & 5.2499 & 6.8646 & 6.7541 & **4.7974** & 4.9155 \\ \hline Servo & Gaussian & **1.0071** & 1.1454 & 1.2059 & 1.2255 & 1.2411 \\  & linear & **1.2451** & 1.2457 & 1.3233 & 1.2455 & 1.2457 \\  & sigmoid & 1.3565 & 1.3881 & 1.3658 & 1.2510 & **1.247** \\ \hline Yacht & Gaussian & **0.8789** & 20.9455 & 2.0736 & 15.0084 & 15.6554 \\  & linear & 11.5317 & 12.0055 & 12.0191 & 11.5062 & **11.5056** \\  & sigmoid & **11.6018** & 17.1246 & 16.9336 & 11.6398 & 12.3480 \\ \hline Auto-MPG & Gaussian & **2.5585** & 7.0208 & 3.1720 & 2.9386 & 3.082 \\  & linear & **3.3510** & 3.3589 & 3.7220 & 3.4881 & 3.3489 \\  & sigmoid & 6.1315 & 7.8074 & 7.8074 & **3.5747** & 3.6064 \\ \hline Machine CPU & Gaussian & **20.9777** & 158.0651 & 57.1354 & 48.7881 & 65.9913 \\  & linear & **56.2433** & 71.8240 & 86.0242 & 61.8366 & 99.7331 \\  & sigmoid & 97.8911 & 172.4349 & 172.4349 & **78.8043** & **84.4713** \\ \hline Concrete & Gaussian & **8.2697** & 18.0116 & 15.5832 & 11.3164 & 11.7239 \\  & linear & 13.0202 & 17.1842 & 16.1212 & **10.3878** & 11.3107 \\  & sigmoid & 13.0202 & 17.1842 & 17.1666 & **10.3878** & 11.3107 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Performance comparison of SVR modeling for the UCI datasets.