# Multimodal Summarization of User-Generated Videos
## 5 Results
### Results
By using the Recursive Feature Elimination (RFE), a feature selection algorithm able to rank features with recursive feature elimination, we were able to find the ten most crucial out of the 224 audiovisual features. In Table 6 the ten features with rank 1 by RFE are presented. The purpose of the RFE feature selection procedure, was to find out the key features on which our proposed video summarization classifier algorithm is based. Overall, three out of ten features were from the audio domain and seven from the visual. From the audio domain, two spectral and cepstral delta features have been selected, along with the mean statistic of the spectral flux feature, which is something that makes sense, if we consider that spectral flux is a measure of spectral changes in successive audio frames (and delta features are, by definition, measuring changes in the respective features). With regards to the visual domain, three out of seven features are related to motion and/or frame-level changes: frame-level diff, standard deviation of the magnitude of the flow vectors and shot duration (shot-detection is also extracted based on a set of thresholding-rules related to movement and frame-level changes). The rest of the four significant visual features are not related to movement and frame diffs: the first and fourth histogram bins of the grayscaled values of the frames and the second and sixth histogram bins of the saturation values correspond respectively to the percentages of (a) very dark, (b) significantly light, (c) very unsaturated (i.e., almost grayscale) and (d) very saturated (colorful) images. It therefore seems that information about extremely colorful and bright aspects (and their complementary) is meaningful to the selection of the informative frames for the summary of the video. To summarize, the basic conclusions from the previously described experimental procedures are the following:  
* Random forest achieves the best classification performance in terms of AUC for the binary classification task in all three modalities (visual, audio and multimodal). * Visual-based classifier is always almost 4% relatively better than audio. * Fusion-based classifier is always almost 3% relatively better than visual, which indicates that the two modalities both contain useful information for the summarization task. * The final performance of the binary classifier after applying the proposed post-processing technique reaches almost 45% precision and 67% recall rate at a 1-second segment level. * Motion-related features seem to be among the most important with regards to the classifiers' decision, along with some spectral domain audio features and color intensity and saturation features. \begin{table}
\begin{tabular}{l l l} \hline \hline
**Feature Name** & **Description** & **Modality** \\ \hline spectral\_flux\_mean & Mean spectral Flux value & audio \\ delta spectral\_spread\_std & Delta spectral spread standard deviation & audio \\ delta mfc\_5\_std & Delta MFCC 5 standard deviation & audio \\ hist\_v0 & 1 st bin of grayscaled value & visual \\ hist\_v3 & 4 th bin of grayscaled value & visual \\ hist\_s1 & 2 nd bin of saturation value & visual \\ hist\_s5 & 6 th bin of saturation value & visual \\ frame\_value\_diff & Frame value difference & visual \\ mag\_std & Magnitude flow standard deviation & visual \\ shot\_durations & Current shot duration & visual \\ \hline \hline \end{tabular}
\end{table} 
Table 6: The ten most valuable features of prediction.