Principles for AI -Assisted Social Influence and Their Application to Social Mediation (2024)

Abstract: Successful social influence, whether at individual or community levels, requires expertise and care in several dimensions of communication: understanding of emotions, beliefs, and values; transparency; and context-aware behavior shaping. Based on our experience in identifying mediation needs in social media and engaging with moderators and users, we developed a set of principles that we believe social influence systems should adhere to to ensure ethical operation, effectiveness, widespread adoption, and trust by users on both sides of the engagement of influence. We demonstrate these principles in D-ESC: Dialogue Assistant for Engaging in Social-Cybermediation, in the context of AI-assisted social media mediation, a newer paradigm of automatic moderation that responds to unique and changing communities while engendering and maintaining trust in users, moderators, and platform-holders. Through this case study, we identify opportunities for our principles to guide future systems towards greater opportunities for positive social change.

## introduction ai systems for social influence in communications are often viewed with suspicion, especially when they exert social influence explicitly, which can be seen as potentially malicious. while ai is increasingly used in social influence, ethical guidelines and principles typically focus on philosophical perspectives for black-box systems, rather than providing practical guidance for ethical methods and implementations @xcite . we believe that effective and responsible social influence systems require not only oversight, but awareness of the socioemotional landscape and transparent models based on that landscape. we consider one target domain as moderation or mediation on social media platforms, where aibased approaches are often embedded in a socioemotional context, but lack direct engagement with user emotions. moderation typically relies on categorical rules such as "no personal attacks" or "no racial slurs", which fail to address shifts in community tone, focus, and overall health of discussion. communities can radicalize over time through interactions that to not explicitly violate community rules. additionally, what is considered harmful or disruptive can evolve @xcite , influenced by factors such as a user's platform history @xcite , requiring more adaptable and holistic mediation strategies. ethical ai-assisted social influence is a nuanced and challenging problem, especially in this domain. maintaining community health may require limiting user freedoms, which can foster perceived censorship and contribute to radicalization. effectively addressing undesirable behavior requires understanding both disinformation tactics and individual responses to communication from others. community health is also dynamic, requiring ongoing adaptation even within a single community. we believe there are a set of guiding principles that can provide a guiding framework for tackling these and other challenging domains in the realm of social influence systems. these principles are shown in figure 1 and were developed by building on prior work in sociolinguistics, psychology, and social cybersecurity, and then incorporating lessons learned from designing and deploying our work with feedback from moderators. social media serves as a valuable example for broader social influence dynamics, as complex interactions can occur in various emotion-laden contexts such as negotiation, decision-making, disaster relief, or patient-caregiver interactions. we introduce d-esc (dialogue assistant for engaging in social-cybermediation) as a case study for comprehensive social influence systems. a key contribution of d-esc is its inclusion of multiple components that facilitate positive social influence by detecting and addressing potentially undesirable behaviors in social media communities. through a mixture of tested natural language processing (nlp) techniques for emotion and sentiment detection and novel topic-based, stance-based @xcite , soft logic, and generative approaches, d-esc analyzes emotional dynamics, generates deescalation responses while adhering to community guidelines, and provides explainable predictions for those responses. these components form a framework for analyzing, modeling, and influencing communities with human oversight, enabling exploration of potential interventions. we briefly cover prior social influence work that could benefit from integrating fundamental principles of social influence in complex socioemotional environments, then highlight these opportunities specifically in the domain of social media moderation. we then outline core principles we adhered to in developing d-esc, and describe their implementation. finally, we cover potential future applications that demonstrate the broad applicability of this case study beyond social media mediation. ## prior work existing ai-assisted social influence work includes systems for improving attitudes and communicative behavior @xcite , persuading users to give to charitable organizations @xcite , and safeguarding online communities through early identification of antisocial users @xcite . emotional awareness has been used to produce prosocial responses to other individuals' statements of negative emotions @xcite . other efforts analyze human social influence techniques in social media @xcite , suggest less inflammatory language in the form of paraphrases @xcite , and position ai as a "moral crumple zone" to protect human relationships by taking the blame for failed communications @xcite . in social media, ai-assisted social influence efforts typically follow a binary approach: either remove posted content or allow it to remain @xcite . some platforms implement zero-tolerance policies with explicit rules against prohibited content, hate speech, harassment, violence, or other harmful or illegal content @xcite @xcite . however, these methods may not effectively address the complex dynamics of online communities, overlooking the cumulative impact of interactions @xcite . additionally, adopting machine learning for scalable detection, as reviewed by @xcite , has led to manual moderation in response to errors, raising concerns of discrimination, as some populations are more frequently mis-classified than others. automated mediation differs from automated moderation in its shift from removing problematic content to fostering dispute resolution and promoting civil discourse within online communities. automated mediation systems are still in early research stages, for example, within the legal field @xcite . more recent work has explored large language model approaches for social mediation @xcite and political discourse @xcite , but these methods do not posit methods for awareness of community dynamics prior to influence. our work aims to influence users to engage more positively in their community, and to encourage moderators to support the community rather than just enforcing rules. ## core principles of social influence systems in developing social media mediation systems, engaging with potential users, surveying related work, and considering future applications, we have established a set of core principles for successful social influence systems. we define "successful" in terms of high scores for a) adoption (likelihood of use), b) trust (perceived as a positive, trustworthy agent), c) effectiveness (achieving intended social influence), and d) alignment (mirroring human social influence skills). whereas prior work has focused on task-specific metrics such as influence outcome or partner perception @xcite , our metrics are intended to apply more broadly-at both individual and societal levels-capturing key dimensions deemed crucial for the role social influence systems in society. we begin by outlining our proposed principles of social influence system design, and then describe how we work to achieve these principles in our d-esc mediation system. ## the d-esc system d-esc is a multi-component system designed for social media environments, either directly or indirectly engaging with users, and providing automated feedback to moderators or administrators. it has been deployed on reddit, where it can post to improve community health, offers a dashboard, and generates natural language reports based on observed activity. the dashboard allows moderators to view potentially problematic posts or indicators of impending conflict. a subreddit-specific component encourages constructive discourse by rewriting posts containing harmful language, while maintaining the original intent. we present the data and textual enrichments used, then describe how each d-esc component aligns with the principles of effective social influence outlined above. ## d-esc approach d-esc's components address the challenges of moderation and mediation, while advancing key principles for successful ai-assisted social influence. the following subsections correspond with the principles outlined in section 3. the connection of these methods to those principles is shown in figure 2 . ## conclusions we have outlined a set of principles for social influence systems that serves as a framework for ethical, effective, and widely adopted social influence applications. we demonstrate a system that follows these principles, applying novel nlp and reasoning techniques to enable moderators foster more constructive discussions. additionally, we illustrate how combining multiple indicators and techniques provides a more tailored, nuanced approach to positive community influence. there is more work needed to position this as a comprehensive social influence system, with additional interface development and fine-tuning to broadly represent the interests of social media communities. nevertheless, we believe this integrated system can be easily adapted to other domains, including local community activism, patientcaregiver dialogue, and disaster relief. furthermore, we hope this work encourages the community to consider integrated, complex social dynamics and work to develop baselines and evaluations that consider a holistic, multi-user environment. research in ai applied to social influence is relatively new, with most discussions concerning the harm of biased or unregulated ai systems that are not explicitly intended for influencing individuals. while we believe our work in the social media mediation space provides an useful starting point for discussing broader principles, the nascent nature of the field limits the claims we can make about the broad applicability of our guiding principles. additionally, while we have some evaluations of individual components, we encountered difficulties in evaluating the system as a whole on a sufficiently large dataset or environment. on reddit, ground truth instances of degradation of community health are often primarily found in quarantined or banned subreddits, which have little to no current activity and would likely not be promising targets for mediation efforts. moderation communities in larger subreddits prioritize standard moderation practices using automod, and do not often see a need for mediation efforts. smaller subreddits, especially those that aim to provide an environment for respectful discourse (e.g. r/abortiondebate), are more amenable to mediation intervention but have fewer instances of mediation events to train on. additionally, such communities are rightfully concerned about the potential side effects of ai mediation, with a general protectiveness of users' data and personal experiences. finally, our work has empirically demonstrated the unique behavioral and emotional factors in each subreddit, but such diversity makes it more difficult to evaluate the effectiveness of our approaches at scale. each subreddit had different moderation techniques, community guidelines, media and post types, and discussion typologies. thus while we believe our approaches could apply widely given community-specific targets to learn, evaluating our novel techniques at a scale sufficient for strong research claims proved difficult.