How to be Helpful on Online Support Forums? (2022)

Abstract: Internet forums such as Reddit offer people a platform to ask for advice when they encounter various issues at work, school or in relationships. Telling helpful comments apart from unhelpful comments to these advice-seeking posts can help people and dialogue agents to become more helpful in offering advice. We propose a dataset that contains both helpful and unhelpful comments in response to such requests. We then relate helpfulness to the closely related construct of empathy. Finally, we analyze the language features that are associated with helpful and unhelpful comments.

## introduction when people encounter issues in their lives (such as problems with family and friends, difficulties at school/work as well as troubles in pursuing one's interests and hobbies), many seek for advice in order to solve these problems. some ask for such advice on internet forums, such as the r/advice subreddit 1 . other users can then comment on these posts to attempt to help the post authors. while many users can actively offer help, not all of them will be seen as helpful by the user asking for advice. examples of a helpful and an unhelpful comment are presented in figure 1 to show their contrast. in order to support people and dialogue agents to be more effective in offering helpful comments, a critical first step is to understand what makes these comments helpful. we make use of a feedback system on r/advice that labels comments based on whether the original post author finds comments to be helpful. based on this feedback system, we introduce a new dataset of comments, labelled with their binary helpfulness. helpfulness has been extensively studied based on exchanges in online support communities 2014; @xcite @xcite . these studies found that helpfulness is associated with various characteristics such as emotional warmth, relevant knowledge, willingness to understand, empowering choice, active listening as well as sharing of similar experiences. however, these studies are solely based on qualitative interpretations and have thus far not sought to associate language features with helpfulness. to overcome this limitation, we seek to identify words that are most positively and negatively associated with helpfulness, and relate these words to characteristics of helpfulness from prior literature. helpfulness is closely related to empathy, as they share many characteristics such as being emotionally warm and compassionate; accepting others' frame of reference, and practising active listening @xcite @xcite . we show that people's average helpfulness across all of their comments correlates with their measured empathy score. we also relate our study to literature on the language features that are associated with empathy @xcite @xcite and show that there is a great overlap among their language features. our key contributions are: ## related work helpfulness on online support communities helpfulness has been studied in online support communities where peers can offer help and support to one another. these communities often center around a shared life situation such as chronic health conditions @xcite and family bereavement @xcite . several factors were emphasized in common: peers were found more helpful when they are emotionally warm and compassionate, give others choice on a solution, willing to accept others' perspectives and experiences, practice active listeningby paraphrasing, asking questions and reflecting feelings, give pertinent advice/insights to help others to solve their problem, as well as share similar experiences @xcite @xcite @xcite . while there has been significant work on what people find helpful, existing studies are based on qualitative themes and to the best of our knowledge, no work has been done on the language features that characterizes helpful support messages. language features for empathy empathy is closely related to helpfulness, as many factors contributing to helpfulness (being emotionally warm and compassionate; accepting others' perspectives; practising active listening) are also associated with empathy @xcite @xcite . there has been significant work on language features that characterize empathy. @xcite identified that empathy is expressed in language use relating to expressing warm and compassionate emotions, communicating an understanding of others' experience, and asking more about the person's experiences. @xcite and @xcite found that language use relating to asking for others' perspective (e.g. it sounds like; do you think) are positively associated with empathy while language use that orders other around (e.g. you need to; please answer the) are negatively associated with empathy. language features for empathy overlap with the features that characterize helpfulness, reinforcing the strong connection between empathy and helpfulness. ## dataset our english dataset is obtained from r/advice, which allows post authors to mark out comment(s) that they have found helpful foot_0 . comments to posts with at least one helpful comment, but were not themselves labeled as helpful are labelled as unhelpful. this inclusion criterion minimizes the mislabelling of comments to posts whose authors did not actively participate in labelling comments. text from reddit was downloaded through the pushshift application programming interface foot_1 . suitable posts and all associated comments from the advice subreddit were downloaded within 300 days (apr 2019 -feb 2020). comments by the post authors and automated bots were excluded. across the 24964 posts that were downloaded, there were 92477 associated comments (41146 helpful). on average, each comment has 95.8 words (@xmath0.5). training/validation/test split was 80-10-10. ## how does helpfulness relate to empathy? to determine how helpfulness relates to empathy, we calculate an aggregated metric for each user based on the proportion of their comments found to be helpful. we then correlate average user helpfulness against an established psychological measure of empathy. empathy quotient questionnaire the short form of empathy quotient (eq) questionnaire @xcite was used to measure empathy (details are in appendix a). higher scores on the eq represent higher empathy. the eq questionnaire has high internal consistency (cronbach's α = 0.90) and test-retest reliability after 12 months (@xmath1.97, @xmath2.001). participants only users with more than 20 comments were included to minimize the likelihood that their average helpfulness was biased due to limited observations. 508 reddit users were sent an online questionnaire through reddit and 91 responded. gender and age were optional to report. 86 participants reported gender (53 male and 33 female) and 83 reported age (@xmath3.7, @xmath4.8). the mean user helpfulness is 0.5440 (@xmath5.1956). using a two-sample t-test, the distribution of eq scores (@xmath6.45, @xmath7.822, @xmath8.0169, @xmath9.9866) from the sample (@xmath10.8, @xmath11.75, @xmath12, demonstrating the representativeness of our sample. ## predicting for helpful comments to explore the potential for the dataset to be useful in training models to distinguish between helpful and unhelpful comments, we trained several baseline models and report their micro-average f1 scores. the performance of baseline models on this task is relatively low but similar to the performance on empathy datasets @xcite @xcite . the relatively low performance of baseline models on this task suggests that while recognizing helpfulness in language is trivial for typically-developing humans, they remain challenging for machines. techniques such as commonsense reasoning @xcite can be explored in the future to better capture the highly complex relationship between language and helpfulness. significant predictors of helpfulness to characterize helpfulness in our dataset, significant predictors of helpfulness (@xmath13.05) based on the logistic regression model were extracted and analysed. 4 thematic categories that were inductively generated from these predictors are shown in a second overarching theme is words relating to attempts to understand the perspective of others. helpful commenters do so by addressing post authors directly, instead of patronizing the difficulties that they face. this is also in agreement with direction themes words examples positive polite, friendly personally, friend, me, personally...i'd let it slide. he'd be predictors sounding words glad, welcome, that's okay i'm just glad that you were able to feels, hey maybe text her? be like hey, just wanted to say optimistic good, luck session with your therapist. good luck sounding words hope, hopefully hope something i say can help you a little! yes, learned, helped and yes that is dangerous and quite forward, strong, work that you can look forward to. words addressing you i really think you deserve better. you sound like the post author i understand that you really like these guys directly as long as you feel you are making the best of negative words indicating victim, kill, rid to be labelled as a victim. she might be afraid of predictors negative emotions bad, depression i was internalizing every bad thing that happened words that patronize dealt, wish it's the latter, as i dealt with when i was like the problem faced easy, promise it seems like the easy solution to your situation. by the post author advice, told the best advice i can give you though ## conclusion we introduce and plan to openly release a novel dataset containing helpful and unhelpful comments in response to posts seeking for advice on life issues. we show that the helpfulness of such comments is related to the commenters' empathy and pioneer an analysis into language features predictive of helpful and unhelpful comments on online support communities. our work can contribute towards supporting people and automated dialogue agents to offer more helpful comments to others.