## 5 Experiments
### Case Study
Local feature matching approaches, such as the original SIFT matching [67], are the predominant approaches for the location category while segmentation-based approaches (color and texture features from mean-shift regions) were only selected to annotate a few general object classes. In this case study, object regions (Section 3.2) have not been used for the  
Figure 11: Correct keyframes (left) and selected approaches (right) per object category. recognition approach selection. However, the detected regions are shown in the positive examples of the prototype and they are useful as rough feedback for the quality of the selected examples. If the regions are too big, it is very likely that the object's background is not separately selected as negative example. In examples where the region of wrong objects is shown, negative examples of these objects might be useful. Figure 12 shows a few region detection examples. Last but not least, we measured the processing time and memory consumption of the individual analysis steps on a desktop PC with a Intel quad-core CPU with 2.66 GHz and 8 GB of main memory. As shown in Table 2, the run times of keyframe clustering heavily depend on the input video size. Videos with a size of 320 \(\times\) 240 pixels are clustered 14.8 times faster than real time, videos with 640 \(\times\) 480 pixels 2.7 times faster than real time, and the clustering of a one hour video with 1280 \(\times\) 720 pixels needs about 1h and 7 min. These run times include the frame extraction, shot detection, and the actual clustering algorithm. In the prototype, frame extraction with FFmpeg operates only on a single CPU core and takes between 80% and 90% of the overall clustering time. The clustered keyframes are stored with a resolution of 512 pixels in the longer dimension and consume about 19 MB storage for a one hour video. However, video portals do not have to store all clustered keyframes at their servers, see Section 4.3. The keyframe suggestion step needs about 31 s to extract features for 1 h of video while the suggestion of keyframes for a set of positive and negative examples takes only 250 ms. Region detection needs about 750 ms for each keyframe that was selected as positive example while the run time of the automatic recognition approach selection cannot be given that easily. As shown in Table 2, this time depends on the type and number of visual feature and recognition approaches of interest. In the case study, an overall number of 12 different feature types with 10 different matching approaches were used, which took between 2 and 3 h to analyze 1 h of video. The size of the metadata that is needed to enable object-based and scene-based video retrieval is below 1 KB for an entire feature film, compare the database schema of Figure 2. \begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**Keyframe Clustering**} & \multirow{2}{*}{**Keyframe**} & \multirow{2}{*}{**Region**} & \multicolumn{3}{c}{**Auto-Selection**} \\ \cline{3-3}
**Measurement Units** & & & & & **BoF** & **Face Det.** & **Local Feature** & **Seg.-based** \\ \hline
**Feature Extraction** (seconds) & & & & & & & \\
**Matching** (seconds) & & & & & & & \\
**Storage** (KB) & 19,000 & 19,000 & 19,000 & 620 & 40 & 445 & 3,500 & 15,000 & 185 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The required run-times and storage for an average video of one hour. Figure 12: Region detection results for 4 objects.