## 4 Experiments
### Evaluation Metrics  
We evaluated the pipeline based on three tasks: (1) accuracy of the shot boundary detection; (2) accuracy of the LSU boundary detection; and (3) accuracy of the object re-ID algorithm. For all the experiments, we use the precision, recall, and f1-score for the evaluation of our results. Precision, recall, and f1-score are computed based on the matched shots/LSU with the ground truth. Furthermore, the results were graphically visualised and analysed to promote insight. The precision measure refers to the fraction of rightly predicted boundaries from total predictions, whereas recall measure denotes the fraction of boundaries rightly retrieved. If \(groundtruth\) refers to the list of ground-truth values and \(prediction\) refers to the list of automatically predicted values, then precision and recall can be expressed as in Equation (10). \[ \begin{split} precision&=\frac{|groundtruth\cap prediction|}{|prediction|}\\ recall&=\frac{|groundtruth\cap prediction|}{| groundtruth|}\end{split}  \tag{10}\]  
Figure 11: Proposed pipeline for multi-object re-ID. Given the input video, we estimate LSU and objects per frame for the video. Based on the number of occurrences of the object in a frame, the objects are categorised as single- and multi-instance objects. Subsequently using the inter-frame similarity and graph-based algorithms, object IDs are created and visualised. F-score, on the other hand, combines precision and recall measures; it is the harmonic mean of the two. Traditional \(F_{shot}\) can be defined as follows:  
\[F_{shot}=2\ \cdot\ \frac{precision\ \cdot\ recall}{precision\ +\ recall} \tag{11}\]  
As mentioned in earlier sections, the precision, recall, and f1 measure would not suffice to validate the accuracy of the LSU boundary detection algorithm. The reason for this is that humans and algorithms employ different ways of perceiving story units. Humans can relate changes in time and location to discontinuities in meaning, whereas an algorithm solely depends on visual dissimilarity to identify discontinuities. This semantic gap makes it impossible for algorithms to achieve fully correct detection results. Therefore, as suggested in [9], we use coverage and overflow metrics to measure how well our LSU boundary detection algorithm performs with respect to human labelled LSUs, using visual features. That is, in addition to the precision, recall, and f1 measures, we propose to use _coverage_ and _overflow_ measures to evaluate the number of frames that were correctly clustered together. Coverage C measures the quantity of frames belonging to the same scene correctly grouped together, while Overflow O evaluates to what extent frames not belonging to the same scene are erroneously grouped together. Formally, given the set of automatically detected scenes \(s=[s_{1},s_{2},...,s_{m}]\), and the ground truth \(g=[s_{1},s_{2},...,s_{n}]\), where each element of s and g is a set of shot indexes, the coverage of scene \(s\) is proportional to the longest overlap between \(s_{i}\) and \(g_{i}\):  
\[coverage=\frac{max_{i=1\...\ n}\#(s_{i}\cap g_{t})}{\#(g_{t})} \tag{12}\]  
\[overflow=\frac{\sum_{i=1}^{m}\#(s_{i}/g_{t})\ \cdot\ min\big{(}1,(s_{i}\cap g_{t}) \big{)}}{\#(g_{t-1})+\#(g_{t+1})} \tag{13}\]  
\(F_{scene}\) combines the coverage and overflow measures and is the harmonic mean of the two. For coverage, values closer to 1 indicate better performance, and for overflow, values closer to 0 indicate better; thus we use \(1-overflow\) for calculating \(F_{scene}\):  
\[F_{scene}=2\ \cdot\ \frac{coverage\ \times\ (1-overflow)}{coverage\ +\ (1-overflow)} \tag{14}\]  
For the experiments pertaining to object re-ID, we make use of _Accuracy_ metrics. Accuracy is the most intuitive performance measure; it is simply the ratio of correctly predicted observations to total observations. In our scenario, the predicted observations are labelled as _True_ if they are correctly predicted, and _False_ otherwise.