# Deep Reinforcement Learning for Query-Conditioned Video Summarization
## 2 Related Work  
### Video Summarization Using Reinforcement Learning  
For generic video summarization, the first work using reinforcement learning was [PERSON] and [PERSON] [29] who propose a voting scheme-based method to predict the importance score for each frame and thereby generate the video summary. The voting scheme is obtained by user's action of watching (Accept) or skipping (Reject) previous similar frames. They use projected low-level feature vectors in an eigenspace to reduce the influence of correlation between different elements. However, the performance of the approach is restricted due to the limited learning abilities of the feature representations. Instead of hand-crafted feature representations for video frames, recent advances in the field of deep learning have enabled the learning of more robust features directly from data and have thus led to better reinforcement learning-based video summarization performances.  
[PERSON] and [PERSON] [25] propose the DSN (Deep Summarization Network), an unsupervised video summarization technique, that makes use of a diversity-representativeness reward to mimic the way how humans summarize videos. The summaries are generated by predicting the probabilities that a given frame is a key-frame and then sampling summary frames based on this probability. Subsequently, they propose another reinforcement learning-based method that relies on video-level category labels to address the problem in a weakly supervised manner [27]. In order to do this, a summarization network with deep Q-learning (DQSN) is explored that guides the agent using a global recognizability reward based on a companion classification network, which is tasked with computing whether a summary can be recognized. Note, this task differs from the query-conditioned task in that it only requires that the summaries maintain the category label for each video and is, therefore, still a form of generic summarization and not personalized towards a specific query. Another work [28] is mainly targeting the challenge of summarizing extremely lengthy videos and implements a reinforcement learning-based approach that relies on SeqDPPs [30] to provide a guiding principle about how to best partition a video sequence into different segments. This differs from prior work, which perform manually partitioning of the video into segments of the same length and do thereby not take the video content into account. [PERSON] et al. [26] propose another reinforcement learning-based summarization approach that also dynamically segments videos. The video is first segmented using a trained action classifier, so that each clip contains a single action, then a deep recurrent neural network is applied to select the most distinct frames for each clip.  
More recently, [PERSON] et al. [31] propose a framework tasked with fast-forwarding a video to present a representative subset of frames. This can be regarded as another form of video summarization for producing key information. In their work, FastForwardNet (FFNet), a reinforcement learning framework using a Markov decision process is developed. It automatically fast-forwards a video and decides the number of frames to skip next while presenting the most important subsets to the users, which makes it computationally efficient as not all frames are processed. However, different from query-conditioned video summarization, the model only processes a subset of the video instead of the whole video due to the forwarding prediction.