## 2 Current Trends in Multimedia Analysis  
Multimedia analysis is a vibrant and challenging research area, and there are many aspects to its use in archiving or cultural heritage. Analysis of multimedia forms a continuum that extends from the extraction and matching of low-level explicit features, such as colour and shape in the case of images, to high-level semantics, such as object identification or more subjective sentiment-based semantics. Much of what has been achieved so far resides at the lower-level end of this continuum, because the explicit features are relatively easy to extract. One key challenge in multimedia analysis is to \"bridge the gap\" between low-level features and high-level semantics with reliable analysis methodologies [4, 5, 6]. Many of the higher-level analysis techniques that exist are specific to particular domains; for example, human face detection and recognition in the case of still images and video. More generalised higher-level descriptions can be sought from multimedia through auto-annotation techniques that rely on heuristics or artificial intelligence to find similarities between the lower-level features extracted from data sets labelled with higher-level ground-truth semantics. Within the ARCOMEM project, most of our work on multimedia analysis focused on the analysis of visual content and the combined analysis of visual content together with text and metadata. For this reason, it makes sense to briefly give an overview of popular and state-of-the-art techniques for visual analysis. Low-level visual feature extraction and representation is a well-studied problem; effective solutions to it include the definition of robust local features, such as SIFT [7], colour variants of SIFT [8] and scalable methodologies for creating bag-of-words and pyramidal representations using such features [9, 11]. In terms of higher-level interpretation, particularly the detection of visual concept and event labels, existing solutions typically rely on SIFT-like features and computationally expensive machine learning algorithms, such as kernel Support Vector Machines (SVMs), where one or more such classifiers are trained independently for each concept or event that we want to detect [8]. The scale of the problem dictates that the training of each SVM is performed independently. However, this also means that the numerical output of the different classifiers is often not directly comparable, making these classifiers suitable for Google-like retrieval of content according to isolated labels, but sub-optimal for selecting the most appropriate concepts for annotating a specific image or video. Online web repositories that have user-annotated images (such as Flickr [10]) give access to ground-truth data sets providing large amounts of knowledge which, in turn, indicates the overall consensus of a community for image representations of specific concepts. This is ideal for training machine-learning algorithms [12, 13], which can be be applied to images with no (or only partial) user provided annotations. By combining semantic taxonomies with feature-based approaches, it has been shown that retrieval results can be either ordered on an axis of diversity [14, 15] or ordered such that diversity is maximised within the most relevant results [16, 17, 18]. Sentiment analysis of multimedia is a fledgling research area, and current research has investigated two topics in particular. Firstly, there has been work in the area of automatic facial expression recognition [19, 20, 21]. Secondly, there has been some work on associating low-level image features with emotions and sentiments [22, 23]. The automated detection of facial expression is still a long way from being robustly applicable to unconstrained real-world images, where even the task of reliably detecting a face is not completely solved. The idea of associating low-level visual features with sentiments hasshown some promise, although until recently, it has only been demonstrated on small datasets with relatively simple colour- [24] and texture- [25] based features.