# Deep Reinforcement Learning for Query-Conditioned Video Summarization
## 2 Related Work
### Query-Conditioned Video Summarization  
Query-conditioned video summarization aims to produce different summaries corresponding to different user interests. [PERSON] et al. [32] present a graph-based approach to incorporate videos and queries in order to construct both relevant and visual appealing trailers for video summarization. Each video segment and query is represented by a node, and the weights of the edges between the video segments and the queries in the graph are computed based on semantic matching, while visual similarity is used to add edges between nodes corresponding to video segments. The segment is thus regarded to be related to a query if it is either semantically similar to the query, or is visually similar to other segments which are related to the query. In [14], a quality-aware relevance estimation is proposed that relies on measuring the distance using the cosine similarity between embeddings of the frames and the text queries, where the textual representation is achieved by first using the word2 vec model [33], and then an Long Short-Term Memory (LSTM) [34] to encode each into a single fixed-length embedding. Summaries are then obtained by selecting key frames using a linear combination of submodular objectives which jointly consider diversity, representativeness and quality of the visual features, as well as the similarity between the query and frames. In [17; 18], the authors investigate summarization of personalized videos of culture heritage scenarios with user preferences as input. They propose a 3D Convolutional Neural Network (CNN)-based approach to encode both visual apparent motion features and real motion features, that are measured by GPS sensors. Then visual semantic classifiers, one trained for each user preference, are used to assess the relatedness between the users preferences and the extracted key shots in the cultural heritage scenario.  
The problem has also been addressed by making use of Determinantal Point Processes (DPP) [35] to model temporal relations. In [12], the authors propose a Sequential and Hierarchical Determinantal Point Process (SH-DPP) to generate summaries which consist of the key shots in the video that are most related to a given query, and study experimental performance on two densely annotated benchmark datasets. However, their datasets are originally collected for the generic video summarization task, which leads to restrictions when performing comprehensive evaluation due to the differences between the two tasks. In follow up work, they propose a more comprehensive dataset with an efficient evaluation metric for the query-conditioned summarization task [13]. Instead of focusing on low-level visual features or only temporal overlap, they evaluate similarity between the predicted and ground-truth video shots based on the shots' semantic information, where each video shot is annotated with a list of semantic concepts. Their proposed approach to combine the query and visual information uses a query-conditioned DPP that integrates a memory network for modeling query-related and contextual importance.  
More recently, [PERSON] et al. [36] apply a generative adversarial network (GAN) [37] architecture and propose a three-player query-conditioned adversarial loss (_prediction loss, ground-truth loss,and random loss_) to force the generator to learn better summary results. Though they achieve state-of-the-art performance, GAN training can often be unstable making the model difficult to train and making the approach highly-reliant on parameter choices. This is true even when more stable GAN modules, such as the Wasserstein GANs [38], are used as in their proposed framework. We, therefore, propose to exploit a deep reinforcement learning framework (SummNet) for the query-conditioned video summarization task to achieve more stable and robust performances, by introducing a mapping mechanism (MapNet) to measure relatedness between the given query and the video shots, and then jointly training the agent with additional diversity and representativeness rewards.