# A Method for Translating Automotive Body-Related CAN Messages Based on Labeled Bits
## 4 Performance Evaluation
### Accuracy  
To assess the accuracy of the methods, we compared the reverse results with the definitions contained in the DBC file for each data bit in each ID. Since this DBC file is specific to this range of models and some features are not optional for our test vehicle, we artificially filtered the defined information in the DBC file. By analyzing and filtering the 82 IDs contained in the DBC file, giving a total of 5248 bits, we filtered and found that 51 bits were related to body operations. The rest represent kinematic-related bits, constant heartbeat packet bits, and many CTR and checksum bits.  
Since the reverse body-related CAN messages involved in this study are similar to those used in the classification problem, three metrics were used to evaluate their performance, accuracy, precision, and recall, the values of which were calculated using Equations (3)-(5), respectively. We recorded true positives (TP) as those bits defined in the DBC file as controlling or indicating body behaviors and whose reverse results were consistent with that definition. Bits in the DBC file specified as unrelated to body behaviors and whose results excluded them were recorded as true negatives (TN). Bits in the DBC file defined as irrelevant to the body but whose reverse result was related to the body were recorded as false positives (FP). Bits in the DBC file specified as being related to the body but whose inverse result was not related to the body were recorded as false negatives (FN).  
\[\text{Accuracy}=\frac{\text{TP}+\text{TN}}{\text{TP}+\text{TN}+\text{FP}+ \text{FN}} \tag{3}\]  
\[\text{Precision}=\frac{\text{TP}}{\text{TP}+\text{FP}} \tag{4}\]  
\[\text{Recall}=\frac{\text{TP}}{\text{TP}+\text{FN}} \tag{5}\]  
After counting, the 1340 bits were classified as TNs, 45 bits were TPs, 3 bits were FPs, and 6 bits were FNs. The method proposed in this paper was shown to have 99.35% accuracy, 93.75% precision, and 88.24% recall. The main reason for a precision value of  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Vehicle Body Operation Triggered by Testing** & **Non-Triggered Automotive Body Operations** \\ \hline Hazard Lights & Brake Pedal \\ Left Turn Signal & Accelerator Pedal \\ Right Turn Signal & Gear \\ Low Beam Lights & Driver’s Seat Belt \\ High Beam Lights & Passenger’s Seat Belt \\ Fog Lights & Driver’s Door \\ Emergency Flashers & Passenger’s Door \\ Trunk & Left Back Door \\ Right Back Door \\ \hline \hline \end{tabular}
\end{table}
Table 4: Triggerable and non-triggerable body behaviors.  
Figure 8: Reverse results of vehicle body-related CAN messages.  
only 93.75% is that some bits indicated that the stationary state of the car changed with the use of the brake or gas pedal, so these bits were classified as brake and gas pedal events. The recall rate was only 88.24% because we used a home car as the test vehicle, and we could not carry out some body-related events, such as the opening of airbags. Although the above reasons caused decreases in the accuracy and recall rate, the reverse method based on marker bits proposed in this paper was shown to have substantially improved efficiency and accuracy values than the traditional manual reverse method.