# Deep Reinforcement Learning for Query-Conditioned Video Summarization
## 1 Introduction
To address the above issues, we propose to use a deep reinforcement learning based approach for query-conditioned video summarization. Reinforcement learning is used in this work to guide our summarization agent using a set of rewards that encode our underlying intuition of what qualities a successful summarization result should have. Deep reinforcement learning approaches [19; 20; 21] have been extensively used in a variety of computer vision tasks such as object segmentation [22], video captioning [23], action recognition [24], and also generic video summarization [25; 26; 27; 28]. For example, [PERSON] and [PERSON] [25] develop a deep reinforcement learning-based summarization network with a diversity-representativeness reward to generate summaries, and achieve a good performance on generic video summarization. Inspired  
Figure 1: The illustration of the query-conditioned video summarization task. Given a user query, query-conditioned video summarization aims to predict a summary that is relevant to the query in a concise manner. Summary 1 and Summary 2 are two summarization results for the same video but for different user queries. by their work, we hypothesize that deep reinforcement learning can also be applied to address the query-conditioned video summarization task to provide personalized summary results instead of a single generic summary for all users. We propose to use deep reinforcement learning to solve this sequential decision-making problem by learning a good policy for video shot selection in order to generate good video summaries. A recurrent neural network is used to learn a sequence of shot-selection actions and based on the complete set of actions a reward is assigned. Reinforcement learning in this scenario allows us to iteratively refine the selection process, allowing the model to take better and better actions. Specifically, we develop a deep reinforcement learning query-conditioned video summarization network (SummNet), to learn a robust video shot selection policy by jointly modeling relatedness, diversity and representativeness to tackle the above challenges. We introduce a notion of relatedness that expresses how related a video shot is to a given query by proposing a mapping network (MapNet) that maps video shots to query space. Establishing this relation between the videos and the queries allows us to formalize a relatedness reward. Together with the diversity and representativeness rewards inspired by the work of [PERSON] and [PERSON] [25], SummNet is able to provide personalized summary results taking into consideration different user interests. The contributions of our paper include: (1) We propose SummNet, a deep reinforcement learning-based framework for predicting summaries given different user queries. To the best of our knowledge, this paper is the first to use deep reinforcement learning for this task. (2) We introduce MapNet for modeling relatedness to capture the relations between the video shots and the user queries, and jointly encourage the agent to select summary results by computing relatedness, as well as representativeness and diversity rewards. (3) We conduct comprehensive experiments on a benchmark dataset which is particularly designed for query-conditioned video summarization. The proposed method outperforms the current state-of-the-art algorithms, which demonstrates its effectiveness.