IEICE TRANS. INF. & SYST., VOL.E106–D, NO.11 NOVEMBER 2023

1906

PAPER

Kiite Cafe: A Web Service Enabling Users to Listen to the SameSong at the Same Moment While Reacting to the Song∗
Kosetsu TSUKUDA†a) , Keisuke ISHIDA†b) , Masahiro HAMASAKI†c) , Nonmembers,
and Masataka GOTO†d) , Fellow

SUMMARY
This paper describes a public web service called Kiite
Cafe that lets users get together virtually to listen to music. When users
listen to music on Kiite Cafe, their experiences are enhanced by two architectures: (i) visualization of each user’s reactions, and (ii) selection ofsongs from users’ favorite songs. These architectures enable users to feelsocial connection with others and the joy of introducing others to their favorite songs as if they were together listening to music in person. In addition, the architectures provide three user experiences: (1) motivation toreact to played songs, (2) the opportunity to listen to a diverse range ofsongs, and (3) the opportunity to contribute as a curator. By analyzing the
behavior logs of 2,399 Kiite Cafe users over a year, we quantitatively showthat these user experiences can generate various eﬀects (e.g., users react toa more diverse range of songs on Kiite Cafe than when listening alone).
We also discuss how our proposed architectures can enrich music listening
experiences with others.
key words: music recommendation, web service, user behavior analysis

1.

Introduction

Unlike listening to music alone, listening to music with others adds the qualities of feeling social connection and letting
others listen to one’s favorite songs. For example, the former
quality occurs when attending a live concert and sharing theexperience with other audience members [2], [3], while thelatter quality occurs when people introduce others to theirfavorite songs [4]–[6].
These qualities become hard to enjoy when various social situations or geographic remoteness make it diﬃcult to
get together in person and listen to music with others. Instead of attending a live concert, people can listen to thesame music at the same time via TV, radio, or live streaming on the web. However, such media provide a poor alternative, because the first quality of social connection requires
audiences to get together in the same place so that they cansee each other’s reactions to the music. Similarly, insteadof directly introducing others to favorite songs, people canManuscript received January 6, 2023.
Manuscript revised May 26, 2023.
Manuscript publicized July 28, 2023.
†
The authors are with National Institute of Advanced Industrial
Science and Technology (AIST), Tsukuba-shi, 305–8568 Japan.
∗
The original version of this paper was published at a conference [1]. We have extended the paper by conducting experimentswith logs covering a longer period and by discussing more application examples for online events.
a) E-mail: k.tsukuda@aist.go.jpb) E-mail: ksuke-ishida@aist.go.jpc) E-mail: masahiro.hamasaki@aist.go.jpd) E-mail: m.goto@aist.go.jp
DOI: 10.1587/transinf.2023EDP7001

post URL links (e.g., to YouTube videos of songs) to social
networking services (SNSs) such as Twitter and Facebook.
However, even if many SNS users react to a song post (e.g.,with a “thumbs up”), there is no guarantee that they actuallylistened to the song and liked it. Rather, the second qualityof sharing a favorite song with others requires knowing thatpeople who react actually listened to the song.
In light of the above, we propose a web service called
Kiite Cafe∗∗,∗∗∗ , which enables people to get together virtually to listen to music without losing the above qualities. Kiite Cafe is characterized by the following two architectures:(i) when users listen to songs on Kiite Cafe, each user’s reactions are visualized; and (ii) songs played on Kiite Cafe
are selected from users’ favorite songs. To facilitate an intuitive understanding of the user experiences provided bythese architectures, we give the following example.
Suppose that Emily is a Kiite Cafe user. One day, shelogs in to Kiite Cafe and finds that 39 users are logged in.
Each user is identified by his/her own icon. The users, including Emily, can simultaneously listen to the same song,which is automatically selected and played. Even if theplayed song has a diﬀerent mood from songs that Emily
usually listens to, if she likes it, she can add it to her list offavorite songs (i.e., her favorites list). Because she has encountered a new favorite song, she feels happy to listen to adiverse range of songs. Moreover, when the currently playedsong is added to her favorites list, architecture (i) visualizes
her reaction by displaying a heart symbol on her icon. Because other users’ reactions are also visualized, she can see
their reactions to feel social connection. After a short time,
one of Emily’s favorite songs is played when it is automatically selected by architecture (ii). While her favorite song isplaying, she is pleased to notice that a heart symbol is displayed on another user’s icon. Then, other users also react
to the song, and eventually the heart symbol is displayed oneight users’ icons. Architecture (i) thus enables Emily to see
the moments when other users start liking one of her favorite
songs. This experience, in which she contributes as a curator∗∗∗∗ , makes her feel happy and want other users to listen
to another of her favorite songs. Thus, Emily looks forward
to another favorite song being played; until then, she stays
on Kiite Cafe and enjoys other users’ favorite songs.
∗∗“Kiite” means “Listen” in Japanese.
https://cafe.kiite.jp∗∗∗∗
In this paper, we use the word “curator” for a person who
introduces or shares a song in a way that adds value.
∗∗∗c 2023 The Institute of Electronics, Information and Communication Engineers
Copyright TSUKUDA et al.: KIITE CAFE: A WEB SERVICE ENABLING USERS TO LISTEN TO THE SAME SONG AT THE SAME MOMENT WHILE REACTING TO THE SONG

1907

songs created with singing voice synthesizer software called
VOCALOID [7]. As of the end of December 2022, morethan 410,000 songs can be played back on Kiite. When aKiite user listens to a song, its video clip is played on Kiite
by an embedded video player††† .
Kiite enables users to eﬀectively find favorite songsby providing novel functions such as exploration of songs
based on their impressions and continuous listening to only
the choruses of multiple songs. A registered user can set
her own icon image, add songs to her favorites list, create
playlists, listen to other users’ playlists, and so on.
2.2 Kiite Cafe
Fig. 1

Structure of this paper.

Our contributions in this work, as illustrated in Fig. 1,
can be summarized as follows.
• We propose two architectures for enabling people tosimultaneously listen to the same music online whileachieving the qualities of social connection and the joyof introducing other people to favorite songs (Fig. 1-(a)and (b)).
• We implemented and released a web service, called Kiite Cafe, that applies these architectures.
• We describe three user experiences in which users (1)are motivated to react to songs, (2) can listen to a diverse range of songs, and (3) can contribute as curators; we also discuss the eﬀects of these experienceson users as a result of the two proposed architectures(Fig. 1-(c), (d), and (e)).
• By analyzing logs of user behavior on Kiite Cafe, weshow that the architectures do provide the above effects. Specifically, users (1) react to songs more actively as the number of users on Kiite Cafe increases,(2) react to a more diverse range of songs on Kiite Cafethan when they listen to songs alone, and (3) stay on
Kiite Cafe longer when they contribute more as curators (Fig. 1-(f), (g), and (h)).
2.

Overview of Kiite Cafe

Kiite Cafe is implemented as a novel function on Kiite† ,which is an existing web service for exploring and discovering music. Any Kiite user can use Kiite Cafe. Below, we
introduce the Kiite functions that are related to Kiite Cafe
and then give an overview of Kiite Cafe.
2.1

Kiite

Song data on Kiite are routinely collected from Nico Nico
Douga†† , which is one of the most popular video sharing
services in Japan. On Nico Nico Douga, it is quite popular for both amateur and professional musicians to upload†††

https://kiite.jp
https://www.nicovideo.jp

Figure 2 shows an overview of Kiite Cafe. When a user
logs in, her icon is displayed at a random position in a twodimensional space that also displays other logged-in users.
All of the users listen to the same song played in a video
A in the figure) at the same time. As mentioned
player (in Sect. 1, Kiite Cafe has two architectures, for visualizing
users’ reactions and selecting songs to play from users’ favorite songs. In the rest of this section, we describe the details of each architecture.
2.2.1

Architecture (i):(Fig. 1-(a))

User Reaction Visualization

We visualize the following four kinds of reactions so thatusers can see each other’s reactions to a played song.
Favorite. When a user likes a played song, she can
add it to her favorites list by clicking the “favorite” button
B When the button is clicked, a heart symbol with an ani.
mation eﬀect is displayed at the top right of the user’s icon
C This enables users to quickly
while the song is playing ().
see how many users like a song. When the user had already
added the played song to her favorites list, the heart symbol
is displayed without the eﬀect.
Comment. When a comment is entered in a text box
D and the “comment balloon” button 
E is clicked, a com
ment balloon is displayed above the user’s icon for 90 secF
The user can also manually delete her comment
onds ().
G Users can thus use thisby clicking the “delete” button .
function to express their impressions of a played song or
have simple communication with each other.
Rotation. A user can rotate her icon by clicking the
H The icon then rotates clockwise at a“rotation” button .
uniform rate until the played song ends. The user can also
manually stop the rotation by clicking the “rotation” button
again. Users can use this function to express feelings like asense of excitement. However, note that Kiite Cafe does notprovide any guidance on when users should use this function, because we want them to use it as they please.
Move. By clicking an arbitrary position in the twodimensional space, a user can move her icon to the clicked
position. The icon is animated to move to the position in†††

On Nico Nico Douga, all songs are uploaded as music videos.

IEICE TRANS. INF. & SYST., VOL.E106–D, NO.11 NOVEMBER 2023

1908

Fig. 2

Screenshot of Kiite Cafe.

a straight line at a uniform rate. Kiite Cafe does not display any meaning for the quadrants and axes in the twodimensional space. Instead, as with the Rotation function,
we leave the usage of the Move function to users.
2.2.2

Architecture (ii): Song Selection from Users’ Favorite Songs (Fig. 1-(b))

Let U denote a set of users who are logged in to Kiite Cafe.
For each user u, we define S u as the set of songs included in
u’s favorites list or playlists. A played song is selected fromu∈U S u . The automatic song selection process is invoked
before the end of the currently played song, and it consistsof the following two steps: (1) selection of a user, and (2)selection of a song from the user’s favorite songs.
In the first step, if there are biases toward certain selected users, then the selected songs may also be biased.
Moreover, some users may become frustrated if their favorite songs are not selected at all. To avoid such biases
and satisfy every user, we developed an algorithm that can
randomly but fairly select users and thus diversify the played
songs. Suppose that user u is selected in the first step, suchthat every user has an equal chance to be selected. Whensong s ∈ S u is randomly selected in the second step, the reason for its selection is displayed, e.g., “This song is in u’s
I in Fig. 2). If s is also among otherplaylist” (first row of users’ favorite songs, that information is displayed (second
I so that those users can notice that one
and later rows of )of their favorite songs is being played. Moreover, a user can
set one of her playlists as a “highly recommended list” by
J When the selected user sets a list as
clicking a button .

“highly recommended,” a song in that list is randomly but
preferentially selected in the second step. By setting such alist, a user can specify the songs that she wants other users
to listen to.
Note that the implemented selection algorithm described above for our service is just an example, and otheralgorithms can be used as long as they balance the fairness
and randomness of selecting both users and songs. Thatis, the algorithm itself is not an essential part of this paper.
Rather, the essential parts are the two proposed architectures
and the new music listening experiences that they provide(see Sect. 3).
K that is always
In addition, we created a bot account logged in. The bot periodically creates playlists according
to a daily/weekly popularity ranking of VOCALOID songson Nico Nico Douga. The bot is treated as one of the users,
and songs in its playlists can also be selected by the songselection process. This gives a user a chance to listen to thelatest popular songs and find new favorite songs even whenno other human users are logged in. Note that the bot does
not show any reactions to played songs.
3.

User Experiences and Eﬀects

As mentioned in Sect. 1, the proposed Kiite Cafe architectures add two qualities: social connection, and the joy of
introducing others to favorite songs. In addition, the architectures provide three kinds of user experiences. This section describes those experiences and their eﬀects on users.

TSUKUDA et al.: KIITE CAFE: A WEB SERVICE ENABLING USERS TO LISTEN TO THE SAME SONG AT THE SAME MOMENT WHILE REACTING TO THE SONG

1909

3.1

Motivation to React to Songs (Fig. 1-(c))

Although many studies have been conducted on enablingusers to listen to music together, most of them have focused on visualizing the song selection process or proposing
methods for that process [8]–[12]. A system that can show
a summary of listeners’ feedback on a song (total numbersof likes and dislikes) has been proposed [13]; however, little
attention has been paid to visualizing each user’s reactions.
In contrast, Kiite Cafe visualizes users’ reactions via theiricons, as described in Sect. 2.2.1. By sharing all the users’
reactions with each other, Kiite Cafe motivates them to react to the currently played song. Accordingly, we expect
that, the more people get together on Kiite Cafe, the moremeaningful it will be to show their reactions, and the more
actively they will react to songs. In the long term, this would
enable users to develop the habit of actively listening to music and enrich their listening experiences [14].
3.2

Diversification of Song Listening (Fig. 1-(d))

Many studies have sought to play songs that match the musical preferences of as many users as possible [8], [10], [15].
In the short term, this approach may be able to increaseusers’ satisfaction. In the long term, however, as is known
from the negative eﬀects of a filter bubble [16], [17], this approach could narrow users’ musical interests. On the other
hand, because Kiite Cafe plays a diverse range of songs selected from various users’ favorite songs, it enables users tofind not only songs that match their preferences well but also
unexpected or serendipitous songs [18] that do not match
their usual preferences. In other words, we expect that auser will react to a more diverse range of songs on Kiite
Cafe than when she listens alone. In the long term, this experience would expand the user’s horizons.
Of course, it may happen that some songs played on
Kiite Cafe do not match a user’s musical preferences, andthe user may not like them even after listening to them.
However, by diversifying the selected songs, it is unlikelythat only songs the user does not like are played all the time.
This motivates users to look forward to what song will beplayed next, even if a song that does not match their preferences is played.
3.3

Contribution as Curators (Fig. 1-(e))

forward to the curation opportunity when another of her favorite songs is played and thus increase her dwell time on
Kiite Cafe. Acting as a curator has been reported to increase
music listening activity (e.g., listening to more songs andmaking playlists for curation) [19]. Therefore, in the longterm, this experience would promote users’ daily music listening activity.
4.

Experiment

We oﬃcially launched the Kiite Cafe service on August 5,
2020. In this section, we evaluate the three expected effects discussed in the previous section. To this end, we analyzed user behavior logs for the period between August
5, 2020 and August 4, 2021. The number of unique users
who logged in during this period was 2,399. The Favorite,
Comment, Rotation, and Move reactions were used 49,425,
14,094, 109,054, and 74,304 times, respectively.
4.1 Frequency of User Reactions (Fig. 1-(f))
As a result of users sharing their reactions with each otheron Kiite Cafe, we expect that they will be more motivated
to react as the number of users increases (Sect. 3.1). To verify this eﬀect, we evaluated the following research question:
Does a user react to a played song more frequently as thenumber of users on Kiite Cafe increases? (RQ1)
4.1.1

Settings

We considered the four kinds of reactions: R ={Favorite, Comment, Rotation, Move}. First, for each played
song, we obtained U s , the set of users except the bot who
were on Kiite Cafe when song s started playing. According
to the number of users (i.e., |U s |), we categorized songs into
four classes (C1 : 1 ≤ |U s | ≤ 5; C2 : 6 ≤ |U s | ≤ 10; C3 :11 ≤ |U s | ≤ 15; C4 : 16 ≤ |U s |). To answer RQ1, for eachreaction, we compared the average proportion of users whoreacted to a song among the classes.
More formally, let S Ci denote a list of songs in Ci (1 ≤i ≤ 4)† . Given song s ∈ S Ci and reaction r ∈ R, let U srdenote the set of users who gave r as a reaction to s. Then,|U r |
the proportion of such users is given by ratio(s, r) = |U ss | .
Finally, the average proportion over S Ci was computed asfollows.
1 
avgratio(S Ci , r) =ratio(s, r).
(1)|S Ci | s∈S
Ci

According to architecture (ii), suppose that a song in user
u’s playlist is selected and played on Kiite Cafe. Becauseof architecture (i), u can see the moment when other users
start liking or show interest in that song (e.g., u can see whenother users add the song to their favorites list or rotate theiricon). For other users, u eﬀectively plays a role as a curator.
That is, the two architectures enable every user to naturally
contribute as a curator. We expect that when a user experiences the joy of contributing as a curator, she will look

4.1.2

Results

Figure 3 shows the results. For visibility, avgratio(S Ci , r)
was normalized by avgratio(S C1 , r) for each reaction. Allof the reaction proportions monotonically increased as thenumber of users increased; thus, the answer to RQ1 is†
Because the same song can be played multiple times on Kiite
Cafe, the same song can appear multiple times in S Ci .

IEICE TRANS. INF. & SYST., VOL.E106–D, NO.11 NOVEMBER 2023

1910

sions for songs more casually. Since the Favorite function
adds a song to the favorites list, users may not use its function unless they really like the song being played. Although
the Comment function is versatile, users may find it burdensome to enter text. In fact, as described at the beginningof Sect. 4, while the Favorite and Comment functions wereused 49,425 and 14,094 times, respectively, the Rotation andMove functions were used as many as 109,054 and 74,304
times, respectively.
Fig. 3 Relation between the number of users on Kiite Cafe and the normalized reaction proportion.

4.2 Diversity of Reacted Songs (Fig. 1-(g))
Because Kiite Cafe enables users to listen to songs that do
not always match their musical preferences, we expect thatthey will react to a more diverse range of songs (Sect. 3.2).
To verify this eﬀect, we evaluated the following researchquestion: Does a user react to a more diverse range of songs
on Kiite Cafe as compared to her musical preferences before
she started using the service? (RQ2)
4.2.1

Fig. 4Characteristic usages of “Rotation,” “Move,” and “Comment.”“Yes.” Because the Favorite function was obviously used
to add a song to a user’s favorites list, we discuss how theusers used the other three functions, as illustrated in Fig. 4.
Regarding the Rotation function, although Kiite Cafe
does not explain its purpose, we searched Kiite Cafe users’
tweets on Twitter† and found that a number of users used thisfunction to express their feelings of excitement. As a resultof this usage spreading, a new expression “I can rotate withthis song” emerged on Kiite Cafe. We expect this expression
to spread in music culture in the future (Fig. 4-(a)).
Next, by analyzing tweets about the Move function, we
found that it was used mainly for two purposes. First, users
moved their icons as if they were dancing. Second, users regarded the top left (or the top) of the two-dimensional space(i.e., near the video player) as the front row at a live concert
venue, and they moved there when their favorite songs wereplayed (Fig. 4-(b)). It is interesting that such a culture wascreated by the users and spread among them.
Regarding the Comment function, although the average length of the played songs was 234 seconds, 9.91% ofcomments were posted within the first 15 seconds of a song.
In such comments, users often expressed the joy of having their favorite songs played (e.g., “Come ooooon!” and“Yeeeees!”). This was similar to the phenomenon at live
concerts in which the audience gets excited when a favorite
song starts (Fig. 4-(c)). In summary, as the number of users
increased, they were more likely to express their excitement
and behave as if they were attending a live concert.
Unlike the Favorite and Comment functions, the Rotation and Move functions allow users to express their impres†We assumed that Twitter users who tweeted about the function
were Kiite Cafe users.

Settings

Let tu denote the time when user u initially logged in to Kiite Cafe. We assumed that songs added to u’s favorites list
before tu (i.e., before using Kiite Cafe), denoted by S uorg , represented u’s original musical preferences. These songs werecollected on the original Kiite service, which was launchedon August 30, 2019, as described in Sect. 2.1. Moreover, we
assumed that songs for which u gave reaction r, denoted by
S ur , represented u’s musical preferences in terms of r after
starting to use Kiite Cafe. To answer RQ2, we compared
the diversity of S ur with that of S uorg for each reaction.
Formally, the diversity was computed as the intra-list
diversity [20]. In the case of S uorg ,


org
org
si ∈S us j ∈S u \{si } dist(si , s j )
org
,
(2)
div(S u ) =org|S u |(|S uorg | − 1)
where dist(si , s j ) is the Euclidean distance between theaudio-based feature vectors [21] of si and s j . Those vectors
were one of the state-of-the-art features, though arbitrary
suitable audio-based features could also be used. For eachreaction r, to appropriately measure users’ musical preferences, we considered only users who had more than nine
songs in both S uorg and S ur †† . Let U r denote the set of suchusers. Then, given r, the average diversities of S uorg and S ur
were computed as follows:
1 
div(S uorg ),|U r | u∈U r
1 div(S ur ).
avgdiv(S ur ) = r|U | u∈U r

avgdiv(S uorg ) =

(3)
(4)

††
Because we released a beta version of Kiite Cafe on May 1,
2020, users who logged in to Kiite Cafe for the first time between
May 1, 2020 and August 4, 2020 were not included in this analysis.

TSUKUDA et al.: KIITE CAFE: A WEB SERVICE ENABLING USERS TO LISTEN TO THE SAME SONG AT THE SAME MOMENT WHILE REACTING TO THE SONG

1911

Fig. 5 Relations between the proportion of users who gave reactions (x-axis) and the normalized
dwell time (y-axis).
Table 1
Reaction r
Favorite
Comment
Rotation
Move

4.2.2

|U r |
157
69
143
131

Diversity of musical preferences.
org

avgdiv(S u )10.50910.45410.51910.454

avgdiv(S ur )10.99410.95410.89511.059p-value
1.14 × 10−8
2.05 × 10−3
1.16 × 10−6
2.75 × 10−9

Results

Table 1 lists the results. A paired t-test was conducted under the assumption that the distribution of diversity scores
before and after starting to use Kiite Cafe follows a normal
distribution for each. We found that, for all reactions, thediversity of songs producing reactions statistically significantly increased in comparison to the diversity of favorite
songs before a user started to use Kiite Cafe; thus, the answer to RQ2 is “Yes.” These results indicate that Kiite Cafeis also useful as a service for users to find songs that are
diﬀerent from their daily musical preferences.
Furthermore, to be sure that this increase in diversity
was not due to an increase in the diversity of VOCALOID
songs uploaded during the same period, we similarly compared the diversities using only VOCALOID songs prior to
the release of Kiite Cafe on August 5, 2020. The results
showed the same trend as in Table 1, with a statistically significant increase in the diversity for all the four kinds of reactions.
4.3

Dwell Time (Fig. 1-(h))

Because Kiite Cafe enables users to experience the joy ofcontributing as curators, we expect that they will stay longer
as their contributions increase (Sect. 3.3). To verify this, weevaluated the following research question: Does a user stayon Kiite Cafe for a longer time as the proportion of users
who react to her favorite songs increases? (RQ3)
4.3.1

Settings

We define user u’s session on Kiite Cafe as the duration between u’s login and logout times. In u’s kth session, supposethat three of u’s favorite songs were played, and that 0%,40%, and 16% of users gave reaction r to those songs. Following the assumption that the maximum percentage (in thiscase, 40%) influenced u’s dwell time, we categorized the
maximum value of ratio(s, r), as defined in Sect. 4.1.1, into
four classes (G1 : ratio(s, r) = 0; G2 : 0 < ratio(s, r) ≤ 0.25;
G3 : 0.25 < ratio(s, r) ≤ 0.5; G4 : 0.5 < ratio(s, r)). However, for a proportion of 0.4, eight reacting users among 20
users would have a higher impact on u than two reacting
users among five users. Therefore, we also considered theclasses of |U s | (i.e., Ci ) as defined in Sect. 4.1.1. That is,
to answer RQ3, given a class of the number of users, we
compared the average session lengths between the reaction
proportion classes for each reaction.
Formally, let Du and S G j denote a list of u’s sessions
and a list of songs in G j (1 ≤ j ≤ 4), respectively. T u,k ∈ Du
represents a list of songs from u’s favorite songs (S u ) thatwere played in the kth session. For that session, we selected
the song smax
u,k,r ∈ T u,k that had the highest proportion of users
who gave reaction r (i.e., smax
u,k,r = arg max ratio(s, r)). Givens∈T u,kCi and G j , we define the set of smax
u,k,r belonging to S Ci and
S G j in all users’ sessions as S i, j,r = {smaxu,k,r | u ∈ U ∧ 1 ≤ k ≤
max
max
∈
T
∧
s
∈
S
∧
s
|Du | ∧ smax
u,k
C
i
u,k,r
u,k,r
u,k,r ∈ S G j }. Let len(u, k)denote the length in seconds of u’s kth session. Then, theaverage session length was computed as1
len(u, k).
(5)avglen(S i, j,r ) =|S i, j,r | smax ∈S
u,k,r

4.3.2

i, j,rResults

Figure 5 shows the results; for visibility, avglen(S i, j,r ) was
normalized by avglen(S i,1,r ) for each reaction. For the Favorite, Rotation, and Move functions, we can see that thedwell time tended to increase as the proportion of users who
gave that reaction increased. In these graphs, the line forthe class of 1-5 users (C1 ) is located at the lowest position
among the four classes (C1 - C4 ). These results indicate that
not only the proportion of users who gave a reaction but also
the absolute number of such users influenced the dwell time.
On the other hand, because the frequency of comments was
lower than the frequencies of the other reactions, no clear
tendency was observed for the Comment function when thenumber of users was 6-10 (C2 ) or 11-15 (C3 ). The significantly reduced dwell time for “0.5-” section of 6-10 (C2 )appears to be an outlier, but this is because there were only
three corresponding session data. Because the dwell time
monotonically increased when the number of users was 1-5

IEICE TRANS. INF. & SYST., VOL.E106–D, NO.11 NOVEMBER 2023

1912

(C1 ) or at least 16 (C4 ), the Comment function could also potentially have a positive eﬀect. Detailed analysis with moreuser behavior logs will be required to verify this eﬀect, andwe leave that for a future work. In summary, the answer to
RQ3 is “Yes” for Favorite, Rotation, and Move.
5.

Discussion

In Sect. 3, we described the user experiences provided by
Kiite Cafe and the eﬀects of those experiences. We believethat Kiite Cafe has even more potential to diversify and enrich users’ music listening experiences. In this section, todemonstrate that potential, we discuss four themes.
5.1

Application Examples for Online Events

Kiite Cafe has been used for several online events. During
an event, instead of using architecture (ii) to select the songs
to play, the participants listened to a designated playlist together while communicating with each other via architecture (i). The events are classified into the following three
categories, depending on the originator.
The first category is events initiated by the organizersof VOCALOID-related events. Thus far, these events havebeen held in the form of collaborations with VOCALOIDrelated events such as “Creators Market Online”† , “Tsunagaru Mirai”†† , “Tamesareru Mirai”††† , and “VOCALOID
DJ STATION”†††† . At an event related to “Creators Market
Online” on August 29, 2020, for example, a famous creator of VOCALOID songs made a special playlist that consisted of songs that the creator liked or had created. During
the one-hour event, as many as 140 Kiite Cafe users enjoyed simultaneously listening to the songs in the playlist,
and they used the reaction functions of Kiite Cafe to communicate with the creator in real time. For another event
related to “Tamesareru Mirai” on February 11, 2021, on
their web page, a questionnaire was conducted on favorite
VOCALOID songs related to winter or snow. During the
90-minute event, 77 users enjoyed listening to songs in aplaylist created according to the questionnaire answers.
The second category is events initiated by the users ofKiite Cafe. Since March 19, 2021, a link “Anniversary event
wanted” has been displayed on Kiite Cafe. The link brings
up a form for a user to enter information about the user’s
desired event, such as the name of the singing voice synthesizer character, the type of anniversary event, and the event
date. The event to be held is determined from these user
proposals by considering the number of proposals, schedule, feasibility, and so on. A playlist to be played during
a character’s event is created according to certain statistics
such as the number of favorites of each song by the characteron Kiite Cafe. Events such as the “Kasane Teto anniversary
event” on April 1, 2021 and the “GUMI anniversary event”†

https://karent.jp/creatorsmarket††
http://tsunagarumirai.com/2020†††
https://twipla.jp/events/472240††††
https://vdds-oﬃcial.tumblr.com

on June 26, 2021 were held, and many users participated.
These dates were the characters’ birthdays, and each event
took place around midnight (23:30 to 0:15). At the stroke of
midnight, many users posted celebratory comments such as“Happy Birthday!” while enjoying the music.
The third category is events initiated by the management team of Kiite Cafe. For these events, the playlists werecreated in the same way as in the second category. For example, on March 9, 2021, the “Hatsune Miku anniversary
event” was held as a token of “Miku”††††† . On August 31,
2021, the “Hatsune Miku birthday event” was held and was
enjoyed by as many as 285 users.
Although it has become diﬃcult for people to get together in person and communicate with each other andwith creators because of the COVID-19 pandemic, we havedemonstrated a new style of online music events throughthese examples. Moreover, even after the pandemic’s resolution, we believe that this kind of online event will be valuable for users who cannot easily attend physical events forreasons such as geographic remoteness.
5.2 Additional Service Functions
Although all users on Kiite Cafe get together in one online
space, it would be interesting to provide additional spaces
for diﬀerent purposes in the future: we could call the main
space and additional spaces the “main cafe” and “branches,”
respectively. For example, for a branch on the theme of“time,” we could put a higher priority on songs related totime (e.g., playing night-related songs at night) by analyzing song lyrics if they are available in the song selection
process.
We could also consider a function that allows any user
to conduct a questionnaire by displaying possible responses
in each quadrant of the two-dimensional space. For example, a user might ask “Who would you like to listen to
the played song with?” and assign responses of “family,”“lover,” “friend,” and “other” to the quadrants. Other users
could answer this question by moving their icons. This function would provide a good opportunity to see how otherusers perceive a song.
5.3 Trustworthiness of Kiite Cafe
With the recent spread of artificial intelligence (AI) technology, the concept of “trustworthy AI” has become increasingly important to foster “trust” that is essential for the acceptance of AI technology in society [22]–[24]. The concepts of Kiite Cafe can also contribute to the realization oftrustworthy AI as follows. First, because songs played on
Kiite Cafe depend on users’ favorite songs, they are diverse†††††
In Japanese, 3 and 9 can be pronounced as “mi” and “ku,” respectively. On March 9, 2021, 140 Kiite Cafe users were enjoying
the event at the same time, and two years later, on March 9, 2023,
754 users were enjoying the event at the same time, indicating thegrowing popularity of this service.

TSUKUDA et al.: KIITE CAFE: A WEB SERVICE ENABLING USERS TO LISTEN TO THE SAME SONG AT THE SAME MOMENT WHILE REACTING TO THE SONG

1913

Fig. 6Comparison between an SNS and Kiite Cafe.

and are less likely to be biased (e.g., bias such as songs created by a particular creator are frequently played). Moreover, as we described in Sect. 2.2.2, our algorithm fairly
selects users. Therefore, fairness in terms of both played
songs and selected users is realized on Kiite Cafe. Second, a user could keep track of the recommended songs
on Kiite Cafe and verify that the fairness of the songs being selected is truly satisfied. Therefore, Kiite Cafe realizes
transparency in that an external observer can verify the recommendation operation. Third, explainability is guaranteed
because the reasons why a played song was selected are disI
played to users as we explained in Sect. 2.2.2 and Fig. 2 .
5.4

Reusable Insights

The reusable insights of the work can be summarized as follows.
• Through the experiments, we verified that the two architectures are eﬀective in promoting users’ music listening activity. These architectures can be helpful forother researchers and companies to develop interfacesthat enable users to listen to music together.
• The examples of successful online events showed thatthe architectures can oﬀer new ways to enjoy musicwith other people even in situations where it is diﬃcult
for people to physically get together in the same place.
We thus opened up a new research theme to support
interactions among creators, audiences, and music.
• We clarified the value of visualizing the moment whena user starts liking a song. In contrast to traditional
curation on an SNS, the Kiite Cafe approach guarantees that users who liked a user’s favorite song actuallylistened to it, as shown in Fig. 6. This insight could
also be beneficial in designing other music listening
systems or services.
6.

Related Work

6.1

Music Listening Systems for Group of Users

Music listening systems for a single user were reviewed byGoto and Dannenberg [25] and by Knees et al. [26]. In contrast, systems for a group of users can be classified into two
types. The first type aims to enable users to listen to music at the same time. Most studies on this type assume thatusers get together in person at a public space such as a fitness center [8], a party [11], a bar [10], or a room [9]. In
MusicFX [8] and Flytrap [9], the system reads users’ musical preferences from each user’s device, and songs stored
in the system are played by taking those preferences into
account. In contrast, in Jukola [10], PartyVote [11], and WePlay [13], users nominate songs to be played, like a jukebox. In the second type of group listening system, users
share songs with other users. Sharing music with others is
an important activity to expand listeners’ horizons [5]. Studies on this type do not assume that users listen to a song at
the same time. Push!Music [5] and tunA [4] are mobile music players that let users share songs via Wi-Fi with others
who are nearby. The user studies on those systems showedthat users are comfortable sharing their favorite songs withothers whether they are friends or strangers. It has also beenreported that users share songs mainly because they want torecommend songs that others would like, disseminate theirfavorite songs, talk about shared songs with others, and soon.
Some applications designed for listening to music together have also been released (e.g., Group Session by Spotify [27] and JQBX [28]). In these applications, any user canlet other users listen to her favorite songs by acting like a DJ.
Users can also communicate with each other via a text chatsystem while listening to songs.
Our study is diﬀerent from the above studies and services in that we introduce the two architectures for reaction
visualization and song selection from users’ favorite songs.
In most of the above cases, because users’ reactions are notvisualized or are visualized only when chatting with text
messages, it is diﬃcult for users to feel social connection
with each other. On the other hand, because the first architecture on Kiite Cafe visualizes four kinds of reactions,users can more strongly feel that they are enjoying musicwith others. In addition, existing systems require users toactively nominate or share songs or act like a DJ, but someusers may hesitate to do that, especially if there is a large
audience. In contrast, the second architecture on Kiite Cafe
enables a user’s favorite songs to be automatically played.
This lets any user share her favorite songs with other users
and see the moments when they start liking those songs.
6.2 Group Recommendation Algorithms
Various song recommendation methods for a single user
have been proposed [29]–[36]. One of the biggest diﬀerences between the methods for a single user and those for agroup of users is that the latter methods need to take multiple users’ preferences into account. A general approach is toaggregate each user’s preferences by, for example, merging
recommendation results generated for each user according
to voting strategies [15], [37]. However, such an approachcannot always reflect minority preferences.
To solve this problem, a concept of fairness has recently been introduced into group recommendation algorithms [38]–[42]. The basic idea of fairness is that a list of

IEICE TRANS. INF. & SYST., VOL.E106–D, NO.11 NOVEMBER 2023

1914

items recommended to a group is fair when each user inthe group can find at least one item in the list that she finds
satisfying. In the context of music recommendation, existing studies have only considered the fairness for users as
audiences. On the other hand, Kiite Cafe achieves fairness
for users as both curators and audience members because of
the second architecture, in which each user’s favorite songs
are fairly selected and played as described in Sect. 2.2.2. In
particular, the “highly recommended list” plays an important role in achieving fairness for users as curators. When auser’s favorite and/or recommended song can be listened towith other users, the user is satisfied from both the audienceand curator viewpoints.
7.

Conclusion

In this paper, we described Kiite Cafe, a web service thatenables users to communicate while listening to the same
music online. Kiite Cafe is characterized by two proposedarchitectures for visualizing each user’s reactions and selecting played songs from users’ favorite songs. Our experimental results quantitatively showed three eﬀects of the
proposed architectures. Since VOCALOID songs are already diverse and the proposed architectures are not limited
to VOCALOID songs, we expect them to be eﬀective for awide variety of music. We believe that these architectures
are also useful for diﬀerent types of music interfaces, including a three-dimensional interface in which user avatars
can listen to the same music in a virtual reality (VR) venue.
Acknowledgments
We thank Crypton Future Media, INC. for developing Kiite
with us and Nico Nico Douga for initially tacitly (later explicitly) encouraging us to work on this research. We also
would like to extend our appreciation to users of Kiite andNico Nico Douga, creators of VOCALOID songs, creatorsof popularity rankings of VOCALOID songs, and all people
who have created, supported, and enjoyed the VOCALOID
culture.
This work was supported in part by JST CREST
Grant Number JPMJCR20D4, JST ACCEL Grant Number JPMJAC1602, and JSPS KAKENHI Grant Numbers
JP21H04917 and JP20K19934.
References[1] K. Tsukuda, K. Ishida, M. Hamasaki, and M. Goto, “Kiite Cafe: A
web service for getting together virtually to listen to music,” Proc.
22nd International Society for Music Information Retrieval Conference, ISMIR 2021, pp.697–704, 2021.
[2] K. Sedgman, “Coughing and clapping: Investigating audience experience,” Cultural Trends, vol.24, no.4, pp.324–326, 2015.
[3] S.C. Brown and D. Knox, “Why go to pop concerts? The motivations behind live music attendance,” Musicae Scientiae, vol.21, no.3,
pp.233–249, 2017.
[4] A. Bassoli, J. Moore, S. Agamanolis, and H.C. Group, “tunA: Local music sharing with handheld Wi-Fi devices,” Proc. 5th WirelessWorld Conference, WWC 2004, pp.1–23, 2004.

[5] M. Håkansson, M. Rost, and L.E. Holmquist, “Gifts from friends
and strangers: A study of mobile music sharing,” Proc. 10th European Conference on Computer-Supported Cooperative Work, ECSCW 2007, pp.311–330, 2007.
[6] M. Håkansson, M. Rost, M. Jacobsson, and L.E. Holmquist,“Facilitating mobile music sharing and social interaction with
Push!Music,” Proc. 40th Annual Hawaii International Conference
on System Sciences, HICSS 2007, pp.87–96, 2007.
[7] H. Kenmochi and H. Ohshita, “VOCALOID - commercial singing
synthesizer based on sample concatenation,” Proc. 8th Annual Conference of the International Speech Communication Association,
INTERSPEECH 2007, pp.4009–4010, 2007.
[8] J.F. McCarthy and T.D. Anagnost, “MusicFX: An arbiter of grouppreferences for computer supported collaborative workouts,” Proc.
1998 ACM Conference on Computer Supported Cooperative Work,CSCW 1998, pp.363–372, 1998.
[9] A. Crossen, J. Budzik, and K.J. Hammond, “Flytrap: Intelligent
group music recommendation,” Proc. 7th International Conferenceon Intelligent User Interfaces, IUI 2002, pp.184–185, 2002.
[10] K. O’Hara, M. Lipson, M. Jansen, A. Unger, H. Jeﬀries, and P.
Macer, “Jukola: Democratic music choice in a public space,” Proc.
5th Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques, DIS 2004, pp.145–154, 2004.
[11] D. Sprague, F. Wu, and M. Tory, “Music selection using the PartyVote democratic jukebox,” Proc. Working Conference on Advanced Visual Interfaces, AVI 2008, pp.433–436, 2008.
[12] G. Popescu and P. Pu, “What’s the best music you have?: Designingmusic recommendation for group enjoyment in GroupFun,” Proc.
CHI ’12 Extended Abstracts on Human Factors in Computing Systems, CHI EA 2012, pp.1673–1678, 2012.
[13] F. Vieira and N. Andrade, “Evaluating conflict management mechanisms for online social jukeboxes,” Proc. 16th International Society
for Music Information Retrieval Conference, ISMIR 2015, pp.190–
196, 2015.
[14] M. Goto, “Active music listening interfaces based on signal processing,” Proc. 2007 IEEE International Conference on Acoustics,
Speech and Signal Processing, ICASSP 2007, pp.IV-1441–IV-1444,
2007.
[15] M. Kompan and M. Bielikova, “Group recommendations: Survey
and perspectives,” Computing & Informatics, vol.33, no.2, 2014.
[16] E. Pariser, The filter bubble: What the Internet is hiding from you,
Penguin Press, 2011.
[17] M. Taramigkou, E. Bothos, K. Christidis, D. Apostolou, and G.
Mentzas, “Escape the bubble: Guided exploration of music preferences for serendipity and novelty,” Proc. 7th ACM Conference on
Recommender Systems, RecSys 2013, pp.335–338, 2013.
[18] Y.C. Zhang, D.Ó. Séaghdha, D. Quercia, and T. Jambor, “Auralist: Introducing serendipity into music recommendation,” Proc. 5th
ACM International Conference on Web Search and Data Mining,
WSDM 2012, pp.13–22, 2012.
[19] J. Fuller, L. Hubener, Y. Kim, and J.H. Lee, “Elucidating user behavior in music services through persona and gender,” Proc. 17th
International Society for Music Information Retrieval Conference,
ISMIR 2016, pp.626–632, 2016.
[20] C.-N. Ziegler, S.M. McNee, J.A. Konstan, and G. Lausen, “Improving recommendation lists through topic diversification,” Proc.
14th International Conference on World Wide Web, WWW 2005,
pp.22–32, 2005.
[21] A.L. Cramer, H.-H. Wu, J. Salamon, and J.P. Bello, “Look, listen,and learn more: Design choices for deep audio embeddings,” Proc.
2019 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2019, pp.3852–3856, 2019.
[22] High-Level Expert Group on Artificial Intelligence, “Ethics guidelines for trustworthy AI,” report, European Commission, 2019.
[23] J.M. Wing, “Trustworthy AI,” Communications of the ACM, vol.64,
no.10, pp.64–71, 2021.
[24] H. Liu, Y. Wang, W. Fan, X. Liu, Y. Li, S. Jain, Y. Liu, A. Jain,TSUKUDA et al.: KIITE CAFE: A WEB SERVICE ENABLING USERS TO LISTEN TO THE SAME SONG AT THE SAME MOMENT WHILE REACTING TO THE SONG

1915

and J. Tang, “Trustworthy AI: A computational perspective,” ACM
Transactions on Intelligent Systems and Technology, vol.14, no.1,
pp.1–59, 2022.
[25] M. Goto and R.B. Dannenberg, “Music interfaces based on automatic music signal analysis: New ways to create and listen to music,’IEEE Signal Processing Magazine, vol.36, no.1, pp.74–81, 2019.
[26] P. Knees, M. Schedl, and M. Goto, “Intelligent user interfaces formusic discovery,” Transactions of the International Society for Music Information Retrieval, vol.3, no.1, pp.165–179, 2020.
[27] “Group session - Spotify.” https://support.spotify.com/us/article/
group-session/.
[28] “JQBX - Listen Together. DJ Online. Discover New Music.”
https://www.jqbx.fm/.
[29] K. Yoshii, M. Goto, K. Komatani, T. Ogata, and H.G. Okuno, “Hybrid collaborative and content-based music recommendation using
probabilistic model with latent user preferences,” Proc. 7th International Conference on Music Information Retrieval, ISMIR 2006,
pp.296–301, 2006.
[30] M. Tiemann, S. Pauws, and F. Vignoli, “Ensemble learning for hybrid music recommendation,” Proc. 8th International Conference onMusic Information Retrieval, ISMIR 2007, pp.179–180, 2007.
[31] K. Yoshii and M. Goto, “Continuous pLSI and smoothing techniques
for hybrid music recommendation,” Proc. 10th International Society
for Music Information Retrieval Conference, ISMIR 2009, pp.339–
344, 2009.
[32] Z. Xing, X. Wang, and Y. Wang, “Enhancing collaborative filtering music recommendation by balancing exploration and exploitation,” Proc. 15th International Society for Music Information Retrieval Conference, ISMIR 2014, pp.445–450, 2014.
[33] A. Vall, M. Skowron, P. Knees, and M. Schedl, “Improving music
recommendations with a weighted factorization of the tagging activity,” Proc. 16th International Society for Music Information Retrieval Conference, ISMIR 2015, pp.65–71, 2015.
[34] D. Liang, M. Zhan, and D.P.W. Ellis, “Content-aware collaborative
music recommendation using pre-trained neural networks,” Proc.
16th International Society for Music Information Retrieval Conference, ISMIR 2015, pp.295–301, 2015.
[35] R.S. Oliveira, C. Nóbrega, L.B. Marinho, and N. Andrade, “A multiobjective music recommendation approach for aspect-based diversification,” Proc. 18th International Society for Music Information
Retrieval Conference, ISMIR 2017, pp.414–420, 2017.
[36] O. Gouvert, T. Oberlin, and C. Févotte, “Matrix co-factorization
for cold-start recommendation,” Proc. 19th International Society forMusic Information Retrieval Conference, ISMIR 2018, pp.792–798,
2018.
[37] L. Baltrunas, T. Makcinskas, and F. Ricci, “Group recommendations
with rank aggregation and collaborative filtering,” Proc. 4th ACM
Conference on Recommender Systems, RecSys 2010, pp.119–126,
2010.
[38] S. Qi, N. Mamoulis, E. Pitoura, and P. Tsaparas, “Recommending
packages to groups,” Proc. IEEE 16th International Conference on
Data Mining, ICDM 2016, pp.449–458, 2016.
[39] D. Serbos, S. Qi, N. Mamoulis, E. Pitoura, and P. Tsaparas, “Fairness in package-to-group recommendations,” Proc. 26th International Conference on World Wide Web, WWW 2017, pp.371–379,
2017.
[40] L. Xiao, Z. Min, Z. Yongfeng, G. Zhaoquan, L. Yiqun, and M.
Shaoping, “Fairness-aware group recommendation with pareto-efficiency,” Proc. 11th ACM Conference on Recommender Systems,
RecSys 2017, pp.107–115, 2017.
[41] D. Sacharidis, “Top-N group recommendations with fairness,” Proc.
34th ACM/SIGAPP Symposium on Applied Computing, SAC 2019,
pp.1663–1670, 2019.
[42] M. Stratigi, J. Nummenmaa, E. Pitoura, and K. Stefanidis, “Fair sequential group recommendations,” Proc. 35th Annual ACM Symposium on Applied Computing, SAC 2020, pp.1443–1452, 2020.

Kosetsu Tsukuda
received the Ph.D. degree
in Informatics from Kyoto University, Japan in
2014. He is currently a Senior Researcher at the
National Institute of Advanced Industrial Science and Technology (AIST), Japan. His research interests lie in the areas of recommendersystems, user generated content, and user behavior analysis. He has received 17 awards, including IPSJ Computer Science Research Award forYoung Scientists and IPSJ Yamashita SIG Research Award.

Keisuke Ishida
was the CEO of Ohma Inc.
and managed an online social network mining
system, SPYSEE, until 2011. Since 2012, hehas been a Technical Staﬀ (Creative Engineers)
at the National Institute of Advanced Industrial
Science and Technology (AIST), Japan. He has
implemented web applications regarding musical information processing and information visualization.

Masahiro Hamasaki
received the Ph.D. degree in informatics from the Graduate University for Advanced Studies (SOKENDAI), Japan
in 2005. He is currently a Senior Planning Manager at the National Institute of Advanced Industrial Science and Technology (AIST), Japan.
His research interests include Web mining, semantic Web, and social media analysis. He is amember of the JSAI, the IPSJ, and ACM.

Masataka Goto
received the Doctor ofEngineering degree from Waseda University in
1998. He is currently a Prime Senior Researcher
at the National Institute of Advanced Industrial Science and Technology (AIST), Japan.
Over the past 31 years he has published morethan 300 papers in refereed journals and international conference proceedings and has received 65 awards, including several best paper awards, best presentation awards, the TenthJapan Academy Medal, and the Tenth JSPSPRIZE. He has served as a committee member of over 120 scientific societies and conferences, including as the General Chair of ISMIR 2009 and2014. As the research director, he began the OngaACCEL project in 2016
and the RecMus project in 2021, which are five-year JST-funded research
projects (ACCEL and CREST) related to music technologies.