## 5 Experiments
### Case Study
It turned out that the selection of the second positive example is the most critical one, especially when a large difference between the initially selected object and all other instances exist. Sometimes, the second selection was placed above the 200 th suggested keyframe. In these cases, the selection of a few negative examples usually improves the situation. Figure 10 and the left diagram of Figure 11 show the results of an evaluation that we performed using those interactively annotated object instances of the second dataset that contain more than two \begin{table}
\begin{tabular}{l c c c c c c c c c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Dataset**} & \multirow{2}{*}{**\#**} & \multicolumn{3}{c}{**Duration (min)**} & \multicolumn{3}{c}{**Clustered Keyframes**} & \multicolumn{3}{c}{**Number of Annotations**} & \multicolumn{3}{c}{**Categories**} & \multicolumn{3}{c}{**Amnotations Time (sec)**} & \multicolumn{3}{c}{**Amnotated Example**} \\ \cline{2-13}  & & **Avg** & **Min** & **Max** & **Avg** & **Min** & **Max** & **\#** & **Avg** & **Min** & **Max** & **Prooms** & **Locations** & **Specific Object** & **Object class** & **Avg** & **Min** & **Max** & **Positive** & **Negative** \\ \hline
**Short videos** & 20 & 5.02 & 0.52 & 22.00 & 68.5 & 13 & 177 & 55 & 2.75 & 1 & 6 & 15 & 10 & 17 & 13 & 32.56 & 9 & 60 & 5.02 & 1.8 \\
**Long videos** & 10 & 77.13 & 27.85 & 108.73 & 606.1 & 234 & 889 & 98 & 9.8 & 5 & 15 & 31 & 15 & 22 & 28 & 47.23 & 21 & 146 & 5.38 & 2.62 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Statistics about the used video datasets. positive examples. In these experiments, we manually counted the number of keyframes that contain the object-of-interest (a) within the top 48 suggested keyframes of Algorithm 2, (b) within the top 48 keyframes that are returned by the automatically selected recognition approach of Algorithm 3, and (c) within a random selection of 48 keyframes from the video. We chose the number of 48 keyframes because this was the amount of visible thumbnails on one screen of the prototype. This experiment setup presents the typical interaction process for user feedback, as described in Section 3.2. Note that the randomly selected keyframes have no order and that the ordering (within the 48 keyframes) of the other runs is not important for user feedback as long as all keyframes are shown on one screen because users do not have to investigate the keyframes line by line. Thus, we omit to compute the average precision of correct keyframes and give the absolute numbers of correctly returned keyframes instead. These numbers can be interpreted as precision (using a division by 48) although the actual number of correct keyframes that exist for an object is unknown and only a small fraction of the objects are supposed to be shown in 48 keyframes or more. The left diagram of Figure 11 shows that an average of 10 correct keyframes was returned from keyframe suggestion in the second dataset while almost 13 correct keyframes were detected from the automatically selected recognition approach. The number of correct keyframes from the random keyframe selection was slightly below 4 on average, though up to 21 for the annotation of some persons, as shown in Figure 10. However, random keyframe selection performed significantly worse than the object recognition approaches for all object categories and it achieved a slightly better result than keyframe suggestion for less than 6% of all objects. The automatically selected recognition approach outperformed keyframe suggestion in 89% of all annotations and a constant average improvement of about 30% was achieved for all object categories. The right diagram in Figure 11 shows the distribution of recognition approaches that make use of bag-of-features, face detection, local feature matching, and segmentation-based features for each object category. Bag-of-feature approaches were selected over all categories. For the annotation of persons, face detection approaches (using Gabor wavelets, SIFT, and color features) have been selected most often.