# Acoustic Detection of Vaccine Reactions in Hens for Assessing Anti-Inflammatory Product Efficacy
## 3 Experiments and Results
### Machine Learning Algorithms
#### 3.3.2 Vaccine Reaction Detection  
The detection of vaccine reaction through sound capture can improve animal welfare, through the adoption of corrective measures taken by the farmer. Gaussian NB was the selected algorithm to detect the vaccine reaction during the night interval, due to its performance.  
Table 5a shows the binary classification results of the Gaussian NB reaction and non-reaction models, using the test data of the control dataset (see the splitting procedure in Figure 3). The non-reaction class was the best ranked, with a F1 of 0.83, followed by the reaction class with an F1 score of 0.76. The weighted mean F1 of the model was 0.80. Figure 7 shows the Gaussian NB-normalized confusion matrix of the test results, where more reaction samples (0.29) are identified as non-reaction than reaction.  
After fitting the model with the control dataset, which included samples of both reaction and non-reaction, the trained algorithm was evaluated using the full treatment dataset. Table 5b shows the results of the model evaluation with the treatment group. The non-reaction class was classified with an F1 score of 0.99, indicating that the majority of samples were recognized as non-reacting.  
The prediction output of the Gaussian NB classifier was also time-related as time-synced samples were obtained, to identify periods of misclassification, or to detect days with increased detection of vaccine reactions. Figure 8 shows the daily vaccine reaction detection rate of the Gaussian NB classifier for the testing and validation datasets. The percentage was calculated as the sum of all prediction classifications as class reactions per day, divided by the total number of samples for each day. The test dataset contained samples from day \(-3\) of the pre-reaction phase, days 2 to 4 of the reaction phase, and days 5 and 7 of the  
Figure 6: Acoustic feature MFCC4: (**top**) time series evolution of CR and TX; (**bottom**) boxplots per block interval of CR and TX.  
\begin{table}
\begin{tabular}{l l l l l l l l} \hline \hline (**a**) & \multicolumn{6}{c}{(**b**)} \\ \hline  & **Pr** & **Rc** & **F1** & & **Pr** & **Rc** & **F1** \\ \hline Non-reaction & 0.79 & 0.88 & 0.83 & Non-reaction & 1 & 0.98 & 0.99 \\ Reaction & 0.82 & 0.71 & 0.76 & Reaction & 0 & 0 & 0 \\ Micro avg & 0.80 & 0.80 & 0.80 & Micro avg & 0.98 & 0.98 & 0.98 \\ Weighted avg & 0.80 & 0.80 & 0.80 & Weighted avg & 1 & 0.98 & 0.99 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Gaussian NB precision, recall, and F1 results for the test and evaluation datasets. (**a**) Test classification results, CR dataset. (**b**) Evaluation classification results, TX dataset.  
post-reaction phase. Additionally, the evaluation dataset contained samples from the entire experiment, from day \(-3\) to day \(8\).  
In the test dataset, there was a 10% misclassification in reaction class on day \(-3\). Day 7 was correctly classified as no reaction. On the 5 th day after vaccination, the last of the reaction phase, a small reaction was still found, equal to 30%. Higher reactions were observed on days 2, 3, and 4, achieving maximum detection on day 3, with approximately 75% values.  
In the validation dataset, there was a 5% misclassification in reaction class on day \(-1\), while reaction detection between day 0 and day 4 was significantly reduced to that detected in the test dataset. It was also observed that day 5 after vaccination administration was the last day with a reaction detection reduced to 5%.