# Natural Language Description of Videos for Smart Surveillance
## 6 Evaluation  
The performance of our deep learning framework can be investigated using two standard evaluation metrics: METEOR [31] and ROUGE (Recall-Oriented Understudy for Gisting Evaluation) [32]. METEOR is a measure used for evaluating many features, such as synonym matching and stemming. This measure uses the harmonic mean for recall and precision. The range of this measure is between 0 and 1. If there was more than one reference translation, then the METEOR results were calculated for every single translation, and the top score was chosen. Table 1 shows the METEOR scores of the multitask feature learning framework for video descriptions with TRECViD, UETVS and AGRIINTRUSION. It can be seen in the table that our model outperformed the state-of-the-art methods. This is because this framework has specialized training of data for interpreting scenes and object settings. The ROUGE metric compared words between the machine-generated (deep neural network) and human written annotations. It is commonly used for evaluating machine summarization tasks. There are many variations of ROUGE, such as N-gram (ROUGE-N), skip \begin{table}
\begin{tabular}{l l l l} \hline \hline
**Methods** & **TRECViD** & **UETVS** & **AGRIINTRUSION** \\ \hline SCN-LSTM & 26.4 & 32.5 & 27.5 \\ MS-RNN & 31.5 & 33.1 & 30.6 \\ MFL (Ours) & 33.9 & 34.3 & 31.2 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: METEOR score (%). Figure 4: Intrusion video. bigram co-occurrence with unigrams (ROUGE-S) and longest subsequence (ROUGE-L). Table 2 shows the ROUGE scores between the multitask feature learning-generated video descriptions and reference hand annotations done by humans. ROUGE 1-3 shows an N-gram comparison between the reference and framework descriptions. Table 3 shows the ROUGE scores between the multitask feature learning-generated descriptions and reference hand annotations done by the humans for the UETVS dataset. The rouge scores were calculated for the four locations where the CCTV cameras were installed. This dataset is one of a kind which, to the best of our knowledge, did not exist previously. As there is no gold standard dataset that deals with the textual description of surveillance videos, we were not able to provide any comparison. Table 4 shows the ROUGE score between the multitask feature learning-generated video description and human annotations for the AGRINTRUSION dataset. The scores show that there was significant similarity between the two. In the TRECViD dataset, the scene-based categories of indoor or outdoor, meeting, groups and traffic had the highest scores due to the superior learning capability of the VGG network for scene and object settings. The activity category also saw a gain in performance compared with our previous experiment [33] of 12%. However, it still required incorporating carefully handled action recognition techniques to outperform the state of the art for action. Scores for the close-up features were comparable to our former experiment [33] but high compared with traditional hand-engineered techniques. However, multitask learning with basic fine-tuning had a positive impact on the overall results for video description generation. It was noted that the age and emotion information in our MDL framework was missing due to the mounted surveillance camera in the UETVS dataset, which captured scenes from afar. The proximity and video quality were not high, which led to minute details being missed in our video description feature extraction. \begin{table}
\begin{tabular}{l c} \hline \hline \multicolumn{3}{c}{**AGRINTRUSION**} \\ \hline ROUGE 1 & 0.6506 \\ ROUGE 2 & 0.5660 \\ ROUGE 3 & 0.5022 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: ROUGE scores for the multitask DL framework for the AGRINTRUSION dataset.