# Predicting Traffic Sign Retro-Reflectivity Degradation Using Deep Neural Networks
## 5 Results and Discussions
### Sensitivity Analysis For Predictor Variables  
To further explore the effect of each feature, we ran the the experiment again on the same dataset but opted out one feature from the set. Table 6 displays the performance of our models without using the orientation angle as input feature. We noticed that the performance of our neural network with separated entity embedding has improved significantly, which indicates that the model separated entity embedding model is sensitive to features. Furthermore, we noticed that the performances of the linear and polynomial models are slightly lower to the models using the orientation angle. Similarly, our neural network with one-hot encoding and shared entity embedding models saw a small drop in \(R^{2}\) in the test set. For the other features, we showcase the results of all our models without using the sheeting type to predict the \(R_{A}\) In Table 7; results are shown without considering sheeting grades. In Table 8, the model's prediction results are shown without using the sheeting color, while in Table 9 the same are shown excluding the observation angle from the analysis. All our neural network models and the polynomial model show a significant increase in error while the linear model showed an increase by a relatively small amount. This highlights that there exists an important relationship between the sheeting type, color, and observation angle with the degradation rate of the \(R_{A}\) value that the neural network models and the polynomial model can capture while the linear model fails to effectively capture and utilize those relationships. The 3M brand has the highest \(R_{A}\) values, keeping all other parameters constant such as grade, color, orientation, observation angle, and age. It can be argued that this sheeting type is made up of a high-quality beaded glass structure that reflects the light back more efficiently. Comparing the grade, the diamond grade has the highest \(R_{A}\) values while the engineering grade possesses the lowest values. Diamond sheeting has high quality prismatic and cube structures capable of reflecting a significant proportion of incident light. Furthermore, \(R_{A}\) has a high value at an observation angle of 0.2-degrees, primarily due to the narrow cone for light traversing and reflecting without much loss in incident beam intensity. The higher \(R_{A}\) value of yellow and white color is intuitive and may be attributed to their effective retro-reflection properties compared to dark blue and green, which absorb more light instead of reflecting it back. Similarly, it can be argued that a sheeting orientation of 0 and 180 degrees is more effective because more embedded cones in the sheeting structure responsible for light reflection are aligned during such orientations. \begin{table}
\begin{tabular}{l l l l l l l} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**Test Set**} & \multicolumn{2}{c}{**Train Set**} & \multicolumn{2}{c}{**Validation Set**} \\ \cline{2-7}  & **R2** & **MAE** & **R2** & **MAE** & **R2** & **MAE** \\ \hline Linear & 0.458 \(\pm\) 0.002 & 158.070 \(\pm\) 0.483 & 0.347 \(\pm\) 0.001 & 187.760 \(\pm\) 0.526 & 0.473 \(\pm\) 0.003 & 206.605 \(\pm\) 0.453 \\ Polynomial & 0.662 \(\pm\) 0.00 & 104.527 \(\pm\) 2.81 & 0.484 \(\pm\) 0.005 & 148.619 \(\pm\) 1.816 & 0.641 \(\pm\) 0.008 & 164.317 \(\pm\) 2.408 \\ NN one-hot encode & 0.696 \(\pm\) 0.189 & 89.238 \(\pm\) 20.076 & 0.493 \(\pm\) 0.121 & 131.022 \(\pm\) 6.356 & 0.664 \(\pm\) 0.155 & 145.569 \(\pm\) 17.192 \\ NN entity embedding & 0.674 \(\pm\) 0.240 & 91.204 \(\pm\) 23.455 & 0.475 \(\pm\) 0.173 & 132.115 \(\pm\) 10.051 & 0.640 \(\pm\) 0.222 & 148.903 \(\pm\) 24.300 \\ NN shared-entity & **0.754 \(\pm\) 0.013** & **83.534 \(\pm\) 1.776** & **0.533 \(\pm\) 0.001** & **128.681 \(\pm\) 0.602** & **0.716 \(\pm\) 0.002** & **139.721 \(\pm\) 1.319** \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Model performance without using the sheeting-grade as feature.The best performing models are in bold.