## 4 Use Cases for Multimedia Analysis in Archiving Community Memories
### Aggregating Social Commentary  
Media is often duplicated on the web. For example, on YouTube, it is common to find multiple versions of the same video, often with minor changes (_i.e._, different sound tracks or modified visual appearance; it is common for the original broadcaster's logo to be changed in a bootleg copy on YouTube). On Twitter, the same effect can be observed, and it is common for the same/similar media object to be shared and re-posted (at a different URL) many times [36] (see Section 4.2 below). In both of these examples, the communities sharing and providing commentary on the individual instances of a particular media item often do not overlap. From the perspective of archiving community memories, it is important to find and capture the relationships between these shared objects, as this allows the social context and commentary to be aggregated, building a much fuller and potentially more diverse picture of the community associated with the media object. An example of this is shown in Figure 1, which shows two copies of the same video on YouTube; both of these videos have been watched and commented on by different subsets of the YouTube community, but to draw any balanced conclusions about what people think about what was being said in the video, one must look at both sets of commentary. From a practical perspective, it can be very difficult or even impossible to determine whether two media items are duplicated by looking at the metadata alone. Often, retweets use the same URL; however, sharing across social networks (e.g., Twitter to Facebook or YouTube to Facebook) or redirection through a different URL shortener can mean that the URLs of identical media are different. Some networks/sites will resize media as they are shared or uploaded, so it is not even possible to use a checksum-like measure of similarity. However, modern image analysis and indexing techniques allow visual duplicates to be detected efficiently through the use of content-based image feature indexing [36, 37]. For example, in [36], we extracted SIFT features [7] to represent the image content. As SIFT features are scale-invariant, they are already able to match across images that have been resized, and they are also robust to a number of other transformations. Comparing images by directly matching their SIFT features is computationally inefficient and impractical with large image collections. To circumvent this problem and efficiently assess whether features match, locality sensitive hashing [38] (LSH) is used to create sketches (compact binary strings) from the features. The sketches are produced, such that the Hamming distance between sketches approximates the Euclidean distance between the features [39]. Rather than explicitly computing Hamming distances between all features, an efficient, approximate scheme is used to find matching features [40]. In this scheme, 128-bit sketches are partitioned into four 32-bit integers, which are used as keys in four hash tables. The corresponding values are the images in which a particular sub-sketch of a feature appeared. In our approach, to detect all near-duplicate images (or video frames), we build an  
Figure 1: Illustration of duplicate videos on YouTube, with the non-overlapping social commentary from different subsets of the user community highlighted. Fully automated video analysis techniques are able to detect these duplicates, and by recognising that the social commentaries are referring to the same content, they can be aggregated. undirected weighted graph, where the vertices are the images and the edge weights represent the number of matching features between the images. As illustrated in Figure 2, the approximate matching scheme, which results in four sets of hash tables, makes the graph building process become trivial [36]. Once the graph is constructed, edges with low weight are pruned, and connected-component analysis is performed to extract all the sets of near-duplicates.