# A Survey of Data Representation for Multi-Modality Event Detection and Evolution
## 6 Multi-Modality Event Evolution
To develop the browsing systems that can allow us to search results with rich information mined from various sources, the synchronization of multiple media content is investigated in the form of hyperlinks [107]. This method can achieve the visualization and exploration of different information landscapes inherent in search results. The content mining and selection from web videos, space-time alignment of multiple media, and augmenting of search result with when and what information are studied in the work for developing these browsing features. [PERSON] et al. [108] explored to discover event from web video and model the event evolution structures. Web users can better understand the major event through a concise structure that shows the events evolution associated with the representative text keywords and visual shots. The event structure represents the important properties, which can realize video visualization and browsing effectively. [PERSON] et al. [109] introduced a novel framework that used text and image data to generate storylines for a given topic. The key idea is to construct a weighted multi-view graph that can be used to capture the context and temporal relationships between the topic-related objects, given a set of images and their textual descriptions. Then the objects are selected by solving the minimum weight connected dominant set problem defined on the graph. The proposed framework provides a storyline with text, images, and structural information, and provides sketches of the topic evolution. [PERSON] et al. [110] proposed a system for event extraction and retrieval called EventSearch. The system was conducted on several types of historical data such as news articles, newspapers, TV news, and micro-blog messages. The system detects events and can provide a better understanding of event evaluation and causalities among events. The system also provides a visualization for events based on multimedia, in which each event support consists of multi-modality data elements. In order to enhance the readability of information and improve user utility, different types of data are combined by a systematic manner. [PERSON] et al. [111] proposed a novel solution to extract and reconstruct the storylines. Specifically, they first investigated the requisite properties of the storyline and devised an algorithm to extract all effective storylines, in which the properties are optimized at the same time. Finally, the extracted lines are all reconstruct and generated the story map with high quality. [PERSON] et al. [98] proposed a multi-modal event topic model (mmETM) for the multi-modal topics detection and evolutionary trends analysis, which can be used to generate event summary effectively. mmETM models social media data including the text and their related images. For social event tracking, they use the mmETM to obtain the textual and visual topics of social events. [PERSON] et al. [112] later proposed a nonparametric tracking model for event tracking, which is an online multimodal multi-expert learning algorithm that can automatically learn the number of topics. It adopts a novel multi-expert minimization restoration scheme. The model drift problem in event tracking can be alleviated by allowing the tracked model to evolve backward and canceling unnecessary model updates. [PERSON] et al. [106] proposed multimodal topic and-or graph (MT-AOG) that can represent textual and visual information and their latent topic structures. The model can track and update the incoming news streams. The topics detected in different time periods are linked and topic trajectories can be generated, which shows how topics emerge, evolve, and disappear over time. The MT-AOG model can also effectively track and update the states of news. Table 4 shows the comparison among typical multi-modality event evolution models.