# Environmental-Based Speed Recommendation for Future Smart Cars +
## 4 Experimental Results
### Application Evaluation  
In order to evaluate our approach, we tested the proposed methodology on real Ford vehicles. The experimental set up included Hydra, a distributed multi-agent framework [20], which was running the task of road class estimation based on weather information and simultaneously it was collecting data from the vehicle by using a Ford proprietary data acquisition tool. In order to evaluate and validate the proposed framework, we launched two different scenarios. Scenario 1 was performed in an urban driving environment, so as to evaluate the proposed methodology in a realistic environment. In Scenario 1, a test vehicle was used around the Carbondale, IL, USA area and Scenario 2 was performed in Ford test track field in Dearborn, MI, USA where an experienced and professional driver was responsible for all the actions. The purpose of this scenario is to push the car to its limits in order to evaluate the on-vehicle information that the car could provide. For the previous experiments, we utilized three agents that were realized by three _armv71_ boards running the following tasks: (i) Board 1 was running the process of road status estimation based on weather information; (ii) Board 2 was aggregating the estimations from weather- and on-route-based estimations; and (iii) Board 3 was collecting vehicle data at run-time. In order to evaluate the proposed road class estimation scheme, the accuracy of the developed classification tree was measured by performing cross validation on the data that was captured during a period of one year covering various areas in the United States of America. Regarding the parameters of Algorithm 1, we set \(max\_depth=5\), \(majority=95\%\) and we utilized the six attributes with the highest information gain (Table 1). We chose these values as we noticed that a greater value for \(max\_depth\) was resulting in data over-fitting, increasing also the classification time. Moreover, lower values of \(majority\) resulted in more outliers. Figure 4 presents the classification tree that was built in order to classify the road status into four distinct classes. Based on the method described in Section 3.2, the original dataset \(D\) is split into subsets by the attribute that maximizes the IG. This operation is repeated recursively, until a pure node is acquired (majority = 95%) or the maximum depth of the tree has been reached. It is noted that the presented algorithm is executed as a cloud service, as there are significant advantages in terms of performance, compared to an on-vehicle execution. This happens due to (i) higher storage availability and (ii) computation capabilities of the cloud. We evaluate the performance of the presented classification tree and we illustrate the results using the concept of the _confusion matrix_. A confusion matrix is a table that is used to describe the \begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & & \multicolumn{3}{c}{**Message Type**} \\ \hline  & & **ANNOUNCE** & **UPDATE** & **BID** & **WIN BID** \\ \hline \multirow{3}{*}{**Action type**} & \multirow{2}{*}{**SET**} & New agent & Require & Ask & Announce \\  & & appears & update & for bid & winner \\ \cline{2-6}  & & Receive & Send & & \\ \cline{1-1} \cline{2-6}  & & information & update & & \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Inter-agent communication messages in Hydra. performance of a classification model on a set of test data for which the true values are known [47]. Each row of the matrix represents the instances of the actual class while each column represents the instances of the predicted class. As depicted in Table 3, which presents the accuracy of the developed classification tree (confusion matrix), 90.2% of the _Dry_ data was classified correctly while 1.8% of them was misclassified as _Ice_. Additionally, the classifier achieved the smaller accuracy while predicting the _Snow_ and _Ice_ road classes.