# Induced Emotion-Based Music Recommendation through Reinforcement Learning
## 1 Introduction
In this paper, we explore this direction, and we focus on the problem of defining an intelligent music recommendation system that, given a future _target_ emotional state to induce, and starting from the assumption that an emotional state is determined entirely by a sequence of music pieces recently listened to, selects the list of music pieces that better \"match\" it. In order to \"recommend\" music for inducing a _target emotional state_, we exploit reinforcement learning (RL) techniques that were proven effective in recommendation music systems. The idea is to train an intelligent agent capable of recommending songs such that the user's mood changes from a given current emotional state to a desired target emotional state based on the user's musical preferences (see Figure 1 for an overview of the proposal). The agent learns the user's preferences and the best trajectories for inducing the target emotion through a feedback-based mechanism. However, to face sparse and deceptive problems, we propose a novel method based on Go-Explore [19]. Go-Explore has proven particularly effective on hard-exploration problems, a category that many real-world issues belong to. We will show that this also applies to the problem of affective computing in the musical recommendation. The main contributions of this paper can be summarized as follows. * We propose _Moodify_, a novel music recommendation system based on Go-Explore, which takes into account the listener's emotional state for inducing a future _target emotion_; the main novelty is that it adopts a \"look-forward-recommendation\", i.e., it recommends music intended to induce (in the future) a specific target emotion. Previous works in the literature have proposed methods that, based on the current user's mood, recommend music or artists to listen to, for example, by computing similarities between artists' and users' moods. To analyze its effectiveness and overall user satisfaction, we have involved 40 people in testing _Moodify_, with one million music playlists from the Spotify platform; results obtained show that the proposed system can bring both significant overall user satisfaction and high performance. To the best of our knowledge, this is one of the few proposals of a system which undergone an evaluation phase of this kind. * The proposed method has been developed as a Web application, namely MoodifyWeb, which exploits Spotify API for developers and JavaScript. To the best of our knowledge, this is one of the first proposals deployed in software for end-users. The remaining parts of the paper are structured as follows. First, in Section 2, we offer an overview of the most used recommendation approaches in the literature, and we discuss some relevant works that inspired the proposed one. With respect to these points, we place our proposal highlighting similarities and differences with previous works. Then, in Section 3, we provide preliminary knowledge to understand the methods and techniques used in _Moodify_. In Section 4, we formalize the music recommendation system highlighting the relation with Go-Explore. Next, Section 5 offers details on the web application implemented that revolves around the proposed music recommendation method. Section 6 is devoted to discussing the results obtained when surveying users about the usability and satisfaction of MoodifyWeb. Lastly, in Section 7, we provide final remarks with envisioned future directions of this research.