# A Survey of Data Representation for Multi-Modality Event Detection and Evolution
## 4 Single-Modality Data Representation and Event Detection
### Visual Based Data Representation and Event Detection
#### 4.2.2 Video Based Data Representation  
Video is also a widely used form of data for event detection tasks. Video can be viewed as an image sequence. Different from images, motion should be considered in the video. Data representation and feature selection of video is of great challenge in video based event detection task. In our review, the video based event detection can be divided into three categories: low level feature based methods, high level feature based methods and deep learning based methods. Low level features can be divided into static frame based visual features and motion visual features. In which, static frame based visual features are the most widely used in the video event detection. The most known static frame based features is Scale-Invariant Feature Transform (SIFT) [47] that proposed by [PERSON], and it has been successfully used in some event detection works. For example, to combine the features extracted from video for complex event recognition, [PERSON] et al. [58] proposed a method that can be selective of different subsets of features including SIFT and the sets of features are considered independently. Some researchers used the improvements of SIFT, such as colorSIFT [59], for video event detection task. For example, [PERSON] et al. [60] used three image features including are SIFT, ColorSIFT and Transformed Color Histogram (TCH) for the video detection task. There are some events that evolve with time, which are more complex than static events. To deal with this situation, the motion visual features have been proposed. For example, [PERSON] et al. [61] used both the static and dynamic features including the Motion SIFT (MoSIFT) [62], which is a 3D version of SIFT, for complex event detection. In addition, [PERSON]. [63] used three motion features including Spatio-temporal interest points (STIP) [64], Motion SIFT and Dense Trajectories for complex event detection. [PERSON] et al. [58] extracted the 3D Histogram of Oriented Gradients (HoG3D) [65] and computed video-level histograms for complex event recognition. [PERSON] and [PERSON] [66] detected the multiple events in the videos, in which the gradient local ternary pattern, histogram of oriented gradients, and Tamura features are used. This feature is extracted from the enhanced frames. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Reference** & **Detection Technique** & **Representation** \\ \hline
[46] & SVM classification & SIFT descriptors, color descriptor \\ \hline \multirow{4}{*}{[48]} & \multirow{4}{*}{Hidden Markov Model} & (1) Global temporal features: time of day, day of week, month; \\  & & (2) Low-level visual features: densely sampled SURF descriptors; \\  & & (3) Higher-level visual features: the type of scene, type of indoor scene, number of faces, facial attributes over detected faces. \\ \hline
[49] & Classification & Saliency, Gist and Time. \\ \hline
[50] & Suffix Tree & Textual annotations, time and geographical positions. \\ \hline
[51] & Hybrid image clustering & Visual features (scale invariant feature transform (SIFT)) and their tags \\ \hline
[52] & Clustering & Temporal and content (low frequency DCT features) \\ \hline
[53] & Coarse classification & Multiple features including time, objects (CNN features) and scenes (CNN feature) \\ \hline
[54] & Hierarchical model & Representations based on attention network (including image, attribute, scene and time) \\ \hline
[55] & Clustering & Visual information (bag-of-features with densely-sampled SURF local features and 64-dim RGB color histograms) as well as textual information (event \\  & & keyword detection) \\ \hline
[56] & Clustering & Visual dict (used PHOW features to construct a bag-of-visual-words model from the selected image set), text (type, location, time) \\ \hline
[57] & Clustering & Combinations of photos visual (SIFT) and semantic features, and the \\  & & photo proprieties \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Representation of image based event detection.