# Interactive Music Recommendation System for Adapting Personal Affection: IMRAPA

Keigo Tada, Ryosuke Yamanishi, Shohei Kato

► To cite this version:

Keigo Tada, Ryosuke Yamanishi, Shohei Kato. Interactive Music Recommendation System for Adapting Personal Affection: IMRAPA. 11th International Conference on Entertainment Computing (ICEC), Sep 2012, Bremen, Germany. pp.417-420, 10.1007/978-3-642-33542-6_42. hal-01556186

HAL Id: hal-01556186

https://inria.hal.science/hal-01556186

Submitted on 4 Jul 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Interactive Music Recommendation System for
Adapting Personal Affection: IMRAPA

Keigo Tada†, Ryosuke Yamanishi‡, and Shohei Kato†

†Dept. of Computer Science and Engineering, Graduate School of Engineering,
Nagoya Institute of Technology,
Gokiso-cho, Showa-ku, Nagoya, 466-8555 Japan

‡Dept. of Media Technology, Ritsumeikan University,
1-1-1 Nojihigashi, Kusatsu, 525-0058 Japan
{ryama, shohey}@katolab.nitech.ac.jp

**Abstract.** We have so various types of entertainment, and music is one of the most popular one. In this paper, we proposed music recommendation system that interactively adapts a user's personal affection with only a simple operation, in which both acoustic and meta features are used. The more a user uses the proposed system, the better the system adapts the user's personal affection and recommends the suitable songs. Through the evaluational experiment, we confirmed that the proposed system could recommend songs adapting user's personal affection even if the personal affection varied.

**Keywords:** Music retrieval system, Interactive system, Affection, Personalization

# 1 Introduction

Music is one of the most beloved entertainments and has attracted much attention as one of media to enrich human life. The large capacity of storage has enabled users to keep and carry a lot of music, then a user generally selects song using the given bibliographic data such as song name, artist name and so on; however it is cumbersome to select songs using only the bibliographic data. Moreover, music distribution services are becoming more widespread, thus users can get music by a single song and enjoy their favorite songs that match their personal affection. On the other hand, getting music by a single song poses a lack of coherence in their music databases, and to select songs using only bibliographic data has become more cumbersome.

From these facts, we believe that more intuitive and usable music entertainment system is needed. In this paper, we propose the music recommendation system adapting user's personal affection which has advantages of both random-play and using play-list. The proposed system dynamically recommends songs that adapt user's personal affection from user's own music database, then only a simple and natural operation is needed to listen and evaluate the songs.
<figure><img src="image_2.png" /><figcaption>Fig. 1. The flowchart of the proposed system.</figcaption></figure>

## 2 The proposed system: IMRAPA

We propose a music recommendation system named IMRAPA (Interactive Music Recommendation system for Adapting user's Personal Affection). Fig. 1 describes the architecture of the proposed system. In the recommendation phase, a candidate song estimated to be acceptable for a user's personal affection is recommended from the music database. The recommended candidate song is randomly selected from music database and input to Näive Bayes Classifier [4], and the probability of that the song is acceptable for the user's personal affection (probability of playing the song) is calculated. Based on the calculated probability, the system determines whether the song is played or not. Because we believe that a user's personal affection can be varying with time, therefore the stochastic mechanism was used in the proposed system for the recommendation, and even once rejected song has a chance to be recommended again. As the recommended song is played, in the evaluation phase, a user evaluates the song with simple operation: to keep listening and skip the song each means “accept” and “reject,” respectively. Each latest *N* accepted and rejected songs' musical features are respectively stored in the user's personal affection database; *N* for adapting the variation of a user's personal affection is verified through pre-experiments and set *N* = 30. In the learning phase, Näive Bayes Classifier is updated with each usage, and the next recommended candidate song is determined by the updated Näive Bayes Classifier in the next recommendation phase. So the more the system is used, the better the system adapts the user's personal affection.

In our previous studies, we proposed the musical fluctuation features that cover sound variation on music which is confirmed to influence human affective evaluation of music [5]; we use these features as acoustic features in the proposed system. Also, we use bibliographic data such as artist and album name as meta features. By using both acoustic and meta features, we believe that the proposed system can adaptively recommend music adapting any types of personal affection. The proposed system is different from the existing systems [2, 1] in the sense that we use both acoustic features concerning musical variation with time which
influence human affective evaluation of music and meta features such as artist and album name, and the other [3] in that the proposed system interactively and dynamically adapts a variable user's personal affection with only a simple operation. That is to say, the point of the proposed system is not recommendation mechanism, which is well known and used in varied systems, but specialized features to adapt personal affection on music recommendation and its system architecture.

## 3 Evaluational experiment

In the experiment, we asked sixteen participants to demonstrate the proposed system based on their own personal affections with the experimental interface on the common computer and speaker. Though the proposed system needs only simple interface that has only “skip button,” absolutely for verifying the usability of the proposed system, we prepared the experimental interface shown in Fig. 2; actually the system leans user’s personal affection with only 2 options: accept or reject. We prepared 909 songs for the experimental music database which are selected from participants’ music compact disks and then the genres and affective evaluations of songs were not biased. The participants were asked to listen to 200 songs, and then the evaluative criterion was “aggressive song or not” during 1-100th song (section 1), and the one was “calm song or not” during 101-200th song (section 2). In the two sections, each the 1-20th and 81-100th songs were assumed as the early and closing periods, respectively.

At first, we discuss about the evaluation of the recommended songs in section 1 (Fig. 3). The percentage of each evaluation was calculated from all participants' evaluations. The evaluation on the closing period tended to be higher than the one on the early period, and the significant differences were confirmed on "Definitely accept" and "Reject" using 2-sample test for equality of proportions. This result showed that the proposed system could become to recommend the aggressive song with time: the proposed system adapted the user's personal affection. Next about section 2 (Fig. 4), it was confirmed that all evaluations between the early and closing periods significantly differed. In the early period, despite the proposed system recommended the aggressive songs from the learning in section 1, the user accepted calm songs; naturally, the percentages of "Definitely accept" and "Accept" were very low and the one of "Weak reject" and "Reject" were high. However the percentages were reversed in the closing period, it was reasonable to consider that the proposed system became to recommend calm songs. These results suggested that the proposed system recommended songs adapting the user's personal affection at each time and could adapt its variation.

## 4 Conclusion and future works

In this paper, we proposed music recommendation system that interactively adapts user's personal affection in which both acoustic and meta features were
<figure><img src="image_3.png" /><figcaption>Fig. 2. Experimental interface.</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>**Fig. 3.** The percentages of each evaluation **Fig. 4.** The percentages of each evaluation on the early and closing periods in the section 1 (%). on the early and closing periods in the section 2 (%).</figcaption></figure>

used. Through the evaluational experiment, we confirmed the usability of the proposed system: the system became to recommend more acceptable songs for variable user's personal affection with time. The proposed system enables users unconsciously evaluate the recommended songs with only a simple operation: continuing to listen or skip, thus the proposed system can be useful especially where the operation using graphic interface should be difficult such as driving.

As using lyric, we expect that the proposed system becomes to catch user's personal affection more truthfully and be more usable. In our future, we will address this challenge and develop more intuitive and high efficiency music recommendation system.

## References

1. Bogdanov, D., Herrera, P.: How much metadata do we need in music recommendation? a subjective evaluation using preference sets. In: Proceedings of 2011 International Society for Music Information Retrieval Conference. pp. 97–102 (2011)

2. Flexer, A., Gasser, M., Schnitzer, D.: Limitations of interactive music recommendation based on audio content. In: Proceedings of the 5th Audio Mostly Conference: A Conference on Interaction with Sound. pp. 96–102 (2010)

3. Knees, P., Pohle, T., Schedl, M., Seyerlehner, D.S.K., Widmer, G.: Augmenting text-based music retrieval with audio similarity. In: Proceedings of 10th International Society for Music Information Retrieval Conference. pp. 579–584 (2009)

4. Langley, P., Iba, W., Thompson, K.: An analysis of bayesian classifiers. In: Proceedings of the 10th National Conference on Artical Intelligence. pp. 223–228 (1992)

5. Yamanishi, R., Ito, Y., Kato, S.: Relationships between emotional evaluation of music and acoustic fluctuation properties. In: Proceedings of IEEE Symposium on Computers & Informatics. pp. 721–726 (2011)