Affective Idiosyncratic Responses to Music (2022)

Abstract: Affective responses to music are highly personal. Despite consensus that idiosyncratic factors play a key role in regulating how listeners emotionally respond to music, precisely measuring the marginal effects of these variables has proved challenging. To address this gap, we develop computational methods to measure affective responses to music from over 403M listener comments on a Chinese social music platform. Building on studies from music psychology in systematic and quasi-causal analyses, we test for musical, lyrical, contextual, demographic, and mental health effects that drive listener affective responses. Finally, motivated by the social phenomenon known as 网抑云 (wǎng-yì-yún), we identify influencing factors of platform user self-disclosures, the social support they receive, and notable differences in discloser user activity.

## introduction music can evoke powerful emotions in listeners @xcite . however, our emotional reactions to it are not universal-affective responses to music are highly personal. just as you may wonder why your friend is sobbing to a song that you only feel ambivalent about, a listener's emotional response to music not only varies with inherent audio or lyrical features @xcite @xcite , but also with other factors such as a listener's demographics, mental health conditions, and surrounding environment @xcite @xcite @xcite . as a result of this idiosyncrasy, it has been extremely difficult to precisely measure the marginal effects of these variables on a listener's affective response @xcite . this difficulty is further compounded especially when examining how a collection of these factors influence individual affective reactions in combination @xcite . music psychology has long focused on identifying the relationships between human affect and music, both in those that are perceived and those that are felt. perceiving and feeling emotions in music, while highly related, are not identical @xcite . examining the latter has proved challenging, as in addition to insufficient scale for finding significance, measuring felt emotions in participatory studies often interferes with the experience itself @xcite . while recent computational studies have attempted citizen science approaches for annotation @xcite , reliability remains an issue; annotator confusion persists between the concepts of perceived and induced emotions @xcite . our work expands this line of research by examining affective responses to music in a natural setting: an online social music platform. we test for differences in affective responses to music by computationally measuring expressed emotions from a massive study of over 403m listener comments on one of china's largest social music platforms, netease cloud music. our paper offers the following three contributions. first, we reveal several nuances in listener affective responses against a host of musical, lyrical, and contextual factors, showing evidence of emotional contagion. second, in a multi-modal quasi-causal analysis, we show that listeners of different genders and ages vary in their reactions to musical stimuli and identify specific features driving demographic effects on affective responses. third, motivated by the social phenomenon known as 网抑云, 1 we systematically study self-disclosures of mental health disorders on the platform, identifying driving fac-tors of this behavior, the social support they receive, and differences in discloser user activity. ## data our work is drawn from one of the largest music streaming services in china, netease cloud music, and focuses on chinese-language user content. netease cloud music. 网易云音乐 (wǎng-yìyún-yīn-yuè) has over 185 million monthly active users @xcite . unlike mainstream music streaming services in the united states such as spotify and apple music, netease cloud music is a social music platform @xcite . here, among other unique features, each song, album, and playlist have comment sections that serve as discussion boards, where users can post top-level comments as well as reply to or up-vote existing ones. these platform interactions serve as a natural setting on listener responses, where users are able to post freely 2 in the comment sections of what they are currently listening to. users are required to create an account to access most of the platform's features; when doing so, users optionally input personal demographic information like age, gender, and location, which they can then choose to display as public or private. dataset collection. to collect a representative sample of public platform commenting activity, we adapt traditional snowball sampling @xcite across multiple random seeds to build an exhaustive list of user, song, album, and playlist entity ids on the platform. we then uniformly sample from the set of entities that have at least one public comment posted. data was collected from all public content ranging from the platform's inception, 2013, to 2022, totaling over 455k albums, 2.87m songs, 1.36m playlists, 29.9m users, and 403m comments. a detailed breakdown of our data and a view of the interaction interface of the platform are shown in appendix section a. the study and data collection were approved by an institutional review board as exempt. ## measuring affective response we measure affective responses to music as expressed in comments posted under their comment sections. since not all comments are indicative of a user's emotional response, we sample a subset of user content and examine both the experiencer 2 see our limitations section for a discussion on platform moderation and censorship. ## variations in affective response computing comment-level valence and arousal by averaging across word-level scores, 3 we analyze variations in listener affective responses to (1) musical and (2) lyrical features, (3) contextual factors, and (4) user demographic variables. ## disclosures of mental health disorders in the context of social media, given the frequent benefits of anonymity @xcite and social connectedness (bazarova and choi, 2014), self-disclosures of personal details can be a method to find social support, advice, and belonging @xcite . this phenomenon gives life to "网抑云," which refers to the outpour of emotional and personal comments on the social music platform, especially late at night and under sad songs. while known colloquially and in popular culture, 6 the mechanisms behind self-disclosure phenomena in the context of social music platforms are not well understood. motivated to better understand disclosures of mental health disorders in a musically-situated social environment, we frame them as affective responses @xcite and test for factors driving this behavior, the social support they receive, and differences in discloser user activity. addressing these unknowns will help us understand how users may use social music platforms for therapeutic purposes @xcite and guide us to better support vulnerable and at-risk individuals. dataset collection. in the absence of clinicallyaligned user data @xcite , we source disorder terms from the dsm-5-tr foot_4 (american psychiatric association, 2022) and utilize regular expressions to identify disclosures of self-reported statements of diagnosis @xcite @xcite for mental health disorders in music comments. two chinese native speakers further manually filter for genuine statements of disclosure (i.e., excluding jokes, quotes, and clearly disingenuous statements), resulting in 1133 users with self-reported mental health disorders. we find that, out of all disclosers, most disclose depression (81.2%), anxiety (19.9%), and bipolar (18.5%) disorders; additionally, most users (60.6%) self-identify as women, consistent with constituent gender differences of affective disorders in national studies @xcite . disclosers show greater platform usage (kolmogorov-smirnov on user levels, @xmath0.01), insomnia-aligned diurnal user activity consistent with disorder symptoms @xcite , increased engagement with playlists of sadder natures, e.g., as shown in figure 4 , loneliness (+302%), sadness (+158%), and night (+50.1%), and decreased engagement with playlists of more active natures, e.g., exercise (-51.7%), compared to typical users. these observations mirror affective disorder activity trends @xcite and suggest that people with affective disorders are more likely to use music reflective of negative emotions than positive emotions to manage feelings of sadness and depression @xcite . a detailed breakdown of our data, comorbidities, and our specific regular expressions are described in appendix section f. affective response. treating the act of selfdisclosure as an affective response, we test for factors driving this behavior. statements of self-relative comment activity(%) loneliness sadness healing quiet relaxing saudade nostalgic romantic exciting happy refreshing moving sexy 0 5 10 15 20 25 30 35 40 45 50 relative comment activity(%) night study exercise work lunch break bars travel walks driving afternoon tea early morning subway disclosers general population disclosure are more likely to appear as top-level comments (78.1%) than as replies (21.9%); toplevel disclosures are biased towards songs with features generally associated with sadness @xcite ) (@xmath1.01 for all features other than tempo, kolmogorov-smirnov), i.e. softer songs with minor modes, and towards playlists with tags of the same nature, while disclosures in reply occur to comments that indicate emotional distress, or that are themselves replies to existing comments made by the discloser ("怎么了", meaning "what's wrong"). responses to the first are split in function, with disclosers either expressing their diagnosis in empathy for encouragement ("...我一年前也确诊 了，事情会好起来的", meaning "...i was diagnosed a year ago, things will be better"), or to commiserate ("我也确诊了，活着好难...", meaning "i was diagnosed too, living is so hard..."), showing evidence of resonance @xcite and high person-centered condolence (high and dillard, 2012). social support. characterizing audience engagement around self-disclosure comments in their content, we identify supportive comments according to the four major classes of social support around health concerns-prescriptive, informational, instrumental, and emotional support-from established literature @xcite @xcite and label the main type of support each comment falls under. we then fit logistic regression models on the dependent variable of receipt, aiming to identify where users are more likely to receive a supportive comment in response to disclosure; including song features as independent variables and song popularity, comment length, user demographics, and comment lda topic distributions as controls. we observe that emotional (52%, e.g., "加油，事情一定会好 起来的我保证", meaning "good luck, everything will be better i promise") and prescriptive support (31%, e.g., "听一些令人振奋的歌曲吧", meaning "listen to heart raising songs") largely exceeds informational (9%, e.g., "...治疗可能会有帮助， 两年治疗后我...", meaning "...therapy could help, after two years of therapy i...") and instrumental (8%, e.g., "...你可以私聊我", meaning "...you can private message me") forms in response to disclosures. several psycholinguistic lyrical features proved statistically significant (@xmath2.05) in predicting if a disclosure comment to a song would receive a supportive reply; the rate of terms in lyrics relating to social processes, specifically friend (+2.23) and ingest (+0.97), positively predict this prosocial behavior, and negative emotion terms (-2.04) do so negatively, mirroring negative correlations between sadness and prosocial tendencies @xcite . for musical features, only reverberation did so positively (+0.90). while past work has studied the prosocial effects of music, most have only used a limited set of author-chosen songs @xcite or crowd-sourced prosocial perceptions @xcite ; here, we specifically identify what makes for prosocial songs and situate our study in the context of social support to mental health self-disclosures. taken together, these observations not only provide ample pointers for music therapists on musical and dyadic conversational means for more successful emotionfocused interventions @xcite but also guide users on how to effectively find social support on the platform when needed @xcite . ## discussion and future work in this work, we sought to examine the driving factors behind variations in emotional reactions to music, via a large-scale computational study of a chinese social music platform. our analyses here reveal several nuances in how idiosyncratic variables elicit emotional responses, with a degree of precision that prior studies have often lacked thus far. in a case study of mental health self-disclosures in music comments, we characterized a type of discourse in the context of a popular social phenomenon, demonstrated the importance of posting location in determining the social support disclosures would receive, and revealed several factors driving the prosociality of music in this context. we see our present work situated in the broader context of studying emotionality in music and in the design of platforms to promote healthier interactions more centered on user well-being. here, we highlight a few limitations and directions for future work; models, code, and anonymized data are made available at https://github.com/skychwang/music-emotions . the music we listen to has a strong effect on our moods @xcite . the integration of emotional response analysis into music recommendation systems could promote healthier recommendations (konstan and riedl, 2012; singh et al., 2020) more cognizant of listener well-being outcomes. no one size fits all, and more sophisticated analyses could better capture more factors that explain emotional response variations towards creating more personalized music emotion recommendation systems. while our work measures the effects of demographic variables on emotional responses, there remains a bio-psycho-social question on identifying the causes behind why this variation exists as it relates to song features. lived experiences condition our emotions (brody, 1997); future work could aim, through significant theoretical and qualitative study, to better identify the relationships and causes behind these variation outcomes. several open questions also remain as to whether risk may be qualified in this context in relation to well-being. specifically, it would be interesting to study how recommendation interactions may disproportionately affect those afflicted with mental health disorders, and how we may design platforms, in the context of well-being outcomes, under normative goals of equity and distributive justice @xcite .