A Checks-and-Balances Framework for Context-Aware Ethical AI Alignment
Authors: Edward Y Chang
Venue: ICML 2025 poster (2025)
Keywords: AI safety
ethical alignment
checks-and-balances
behavior modeling
Primary area: social_aspects->safety
TL;DR: An adaptive framework for context-aware ethical alignment based on the interpretation of constitutions.

Abstract: This paper introduces a checks-and-balances framework for ethical alignment of Large Language Models (LLMs), inspired by three-branch governmental systems. It implements three independent yet interacting components: LLMs as the executive branch for knowledge generation, DIKE as the legislative branch establishing ethical guardrails, and ERIS as the judicial branch for contextual interpretation. Beyond structural separation, we address a fundamental challenge: regulating emotion to shape behaviors. Drawing from psychological theories where managing emotional responses prevents harmful behaviors, we develop a self-supervised learning pipeline that maps emotions to linguistic behaviors, enabling precise behavioral modulation through emotional conditioning. By integrating this approach with adversarial testing, our framework demonstrates how DIKE and ERIS direct linguistic behaviors toward ethical outcomes while preserving independence throughout knowledge generation, ethical oversight, and contextual interpretation.

A Checks-and-Balances Framework for Context-Aware Ethical AI Alignment
Edward Y. Chang
Abstract
more fundamental conceptual limitation: its narrow focus isolated behaviors rather than holistic patterns. This reac- tive strategy is similar to "Whack-A-Moles game, where addressing one problematic behavior does not prevent the emergence of others. For example, consistently instructing someone to make their bed does not necessarily cultivate overall tidiness, such as doing laundry or washing dishes. Similarly, RLHF often emphasizes short-term fixes the of long-term coherence, leading to catastrophic for- getting: users have reported that optimizing one task in ChatGPT can degrade performance in unrelated areas (Kirk patrick et al. 2017 Lin et al.) 2024 Dai et al. (025). This challenge mirrors the difficulty treating addiction, where addressing one symptom may reveal deeper psychological
This paper introduces a broecks-and-allnnee framework for ethical alignment of Large Lan- guage Models (LLMs), inspired by three-branch governmental systems. It implements three in- dependent yet interacting components: LLMs the executive branch for knowledge generation, Dike as the legislative branch that establishes eth- ical guardrails, and Eris as the judicial branch for contextual interpretation. Beyond structural sepa- ration, we address fundamental challenge: regu- lating emotion shape behaviors. Drawing from psychological theories where managing emotional responses prevents harmful behaviors, we develop self-supervised learning pipeline that maps emo- tions to linguistic behaviors, enabling precise be- havioral modulation through emotional condition- ing. By integrating this approach with adversarial testing, our framework demonstrates how Dike and Eris direct linguistic behaviors toward ethical outcomes while preserving independence through- out knowledge generation, ethical oversight, and contextual interpretation.
dependencies (Sinha 2008 Torrens et 2005).
To overcome these challenges, propose checks-and- balances framework inspired by governmental structures, where independent but interacting components maintain ac- countability and balance. Our architecture integrates three components: LLMs serve as the executive branch for knowl- edge generation; Dike (representing justice) functions as the legislative branch to set ethical standards; and Eris (rep- resenting discord) acts as the judicial branch, providing adversarial testing and contextual interpretation. In mythol- ogy, Dike embodies order and justice, while Eris signifies discord, forming a duality that our framework leverages to balance ethical guidance with adversarial scrutiny.
1. Introduction
Ethical alignment in Large Language Models (LLMs) is a critical challenge, particularly given the limitations of Reinforcement Learning from Human Feedback (RLHF) (OpenAl 2023 Ouyang et al., 2023). Although RLHF has demonstrated success in aligning Al systems with human values, it encounters two major issues: 1) susceptibility to social biases when feedback is polarized, and 2) vulnera- bility to reward hacking, where the system optimizes for feedback without genuine ethical improvement (Christiano ct al. 2017 Skalse et al., 2022). These issues can result in unethical behavior or inconsistent performance.
Figure [ iillustrates this bhrnch architecture, where neurally independent components, LLMs the foundation, with Dike and Eris as oversight layers, interact through struc- tured interfaces while maintaining strict separation of their neural architectures and parameters.
1.1. Emotion Regulation as Behavioral Control
A fundamental question underlies our framework: Can regu- lating emotions shape behaviors, and can similar principles be applied LLMs? In human psychology, emotions sig- nificantly drive behaviors: anger and contempt can provoke aggression, and rage and envy result in harmful actions (Damasio, 1994). Therefore, emotion regulation is essential for behavioral control. Techniques such as cognitive refram- ing and attentional deployment are known to reduce negative behavioral outcomes by managing emotional intensity.
Beyond these implementation challenges, RLHF faces
'Computer Science, Stanford University, Correspondence to: Edward Y. Chang <echang @ pecssstan ct..stafoor..du..
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Checks-and-Balances Framework for Ethical AI Alignment
DIKE
ERIS
dynamic guidelines that account for both content and language behavior, blocking manipulative harmful communication while preserving factual accuracy and emotional authenticity. These guardrails adjust to dif- ferent cultural contexts, maintaining consistency while allowing context-dependent interpretation.
3. Adversarial Behavioral Testing:
Eris challenges Dike's ethical guidelines by presenting diverse cultural perspec- tives and edge cases, ensuring the adaptability of ethical reasoning. This adversarial interaction enables the sys- tem to address complex scenarios with cultural sensitivity and contextual awareness.
Gerrini
Figure 1: Framework with Three Independent Branches. Bottom: Knowledge LLMs (executive); Left: Dike (legisla- tive); Right: Eris (judicial). (Photo credit: DALL-E)
4. Ethical Content Transformation: When problematic con- tent is detected, Eris can revise it to maintain the intended emotional tone while ensuring ethical compliance, with human-in-thie in-the-loop oversight to validate the appropriate- ness of revisions. These potential transformations are tested by Eris in cultural and contextual variations to assess their suitability before implementation.
Unlike humans, who struggle with emotion regulation due to complex neural and cognitive processes (James) [884] Gross 1998). L.LMs lack intrinsic emotional states alto- gether. However, empirical evidence shows that LLMs can generate text with consistent emotional characteris- ties through controlled prompt engineering (Chang 2024d) Indeed, the work of (Tak & Gratch 2024) demonstrated that LLMs such as GPT-4 align more closely with human judgments when interpreting others' emotions from a third- person perspective than when attempting to model self- attributions of emotion. This creates a unique opportunity: by leveraging LLMs' ability to model the average human observer's is emotional interpretations, we can establish reli- able frameworks for ethical alignment that operate through emotional framing rather than explicit rule-following.
The experimental section evaluates our framework through three complementary studies. First, we assess whether emotion-mede medditee classification provides more effective eth- ical guardrails than direct behavior classification. Next, we examine Dike's ability to independently evaluate and explain linguistic behaviors. Finally, we test how the ad- versarial Eris component enables cultural adaptability and prevents excessive censorship. Although direct comparison with proprictary RLHF implementations is not feasible, our results demonstrate how our approach addresses the theo- retical limitations RLHF in handling contextual diversity without compromising knowledge integrity.
1.3. Contributions
Building on this insight, our framework integrates the prin- ciples emotion regulation into the ethical alignment of LLM. Specifically, Dike analyzes how emotions manifest in linguistic behaviors, while Eris tests these interpretations against diverse cultural contexts.
Our contributions are follows:
1. A novel baucks-and-balance. architecture for ethical alignment that maintains separation between knowledge generation and ethical reasoning.
1.2. Checks and Balances for Emotion-Guided Ethics
2. The Beam model, a quantitative framework for repre- senting emotions along continuous spectra with defined intensity levels, enabling precise emotion regulation in AI systems.
Central to this approach is the synergy between Dike and Eris, reflecting the internal conflict often present in the regu- lation of human emotions. Just as humans balance immedi- ate emotional responses against longer-term goals and social norms, our framework establishes an adversarial dynamic between ethical guardrails and contextual challenges. This duality introduces four key innovations:
3. An emotion-driven approach that guides linguistic be- haviors toward ethical outcomes by leveraging cognitive theories of emotion regulation.
4. An adversarial framework that enhances ethical reason- ing by challenging established guidelines with cultural perspectives, enabling context-sensitive adaptability.
1. Smnotion-Drived Behavioral Modeling: Based on Beam (Behavioral Emotion Analysis Model) (Chang 2024d), Dike uses self-supervised learning to quantify relation- ships between emotional states and linguistic patterns, guiding ethical decisions through behavioral analysis.
5. A theoretical framework explaining the effectiveness of minimal supervision in LLM alignment, formalized as the Unified Cognitive Consciousness Theory (UCCT) in Appendix A
Behavior-Awee Aware Ethical Guardrails: The framework sets
2

---

Checks-and-Balancer Framework for Ethical AI Alignment
2. Related Work
According to this theory, emotional experiences arise from physiological reactions to events. Subsequent research, including studies by Damasio (Damasio, 1994, Fauconnier & Turner (002), suggests that the expression and regulation of emotions often manifest in the language we use. High-intensity emotions, such as rage or contempt, can lead to aggressive or destructive behaviors, such as hate speech.
(James]
1884
Lange 1885
This section surveys existing work on emotion and behav- ior modeling across various domains, with a focus on their applications in AI ethics. We examine how linguistic be- haviors are influenced by emotional patterns and explore structured approaches that integrate emotional frameworks with linguistic models to improve ethical AI alignment.
We also examine the limitations of RLHF While effective in refining AI outputs, RLHF can overfit to human annotations, faces challenges in adapting to diverse cultural contexts, may experience parameter drift from optimal settings, and can inadvertently reinforce unintended biases. These ob- servations highlight opportunities to develop more adaptive and principled approaches to complement existing ethical Al alignment methods.
The Schachter- -Snger theory (Schachter & Singer 1962) or the two-factor theory of emotion, depicts the role of physiological change and the cognitive assessment change determine the label and strength of emotion. Building on this, the affect-as as sesectas-infooee -nfformatinn theory developed by Norbert Schwarz and Gerald Clore (Schwarz, & Clore 1983) posits that people use their current emotions to make judgments and decisions to act. If emotions can be adjusted, can behavior. The work of Barbara Fredrickson (Fredrickson) 1998) on the effects of positive emotions discusses how we perceive and react to emotions.
2.1. Emotion Modeling
Cognitive-linguiste theories intersect with artificial intelli- gence for understanding AI behavior. Theories by Lakoff, Johnson, Talmy, and Jackendoff (Jackendoff) 2002 Lakoff & Johnson, 1980; Talmy] 2000) explore the relationship be- tween language processing and cognitive functions, building on early work by Freud and Jung (Bai et al. 2022 Gabriel let al.] 2024).. The concept of "emotion" " remains contentious, with definitions varying across disciplines (Scherer 2005). W. James (James 1884) attempted to define emotions, but consensus remains elusive.
Collectively, these theories elucidate the intricate connection between emotions and behaviors, providing the theoretical foundation for our work to incorporate behavior advisor to evaluate and rectify behaviors. Section 3.2 details how the Dike framework implements cognitive strategies to mitigate emotions and regulate linguistic behaviors effectively.
2.3. Reinforcement Learning with Human/AI Feedback
RLHF is the predominant approach to addressing the chal- lenges of AI ethics. This section presents representative works, their advances, and limitations.
This paper focuses emotional contexts and linguistic behaviors in LLMs, avoiding the complexities of human physiological and personality factors. This approach allows for exploration of emotion representation in Al systems.
Human Feedback (RLHF): Initial advances by Christiano et al. (Christiano al., 2017) demonstrated how RLHF can steer language models towards desired outcomes based on human preferences. Newer techniques like Identity (V) Preference Optimization ( (VPO) and Generalized Preference Optimization (GPO) refine this approach by directly opti- mizing user preferences, effectively addressing scalability challenges. Kahneman- Optimization (KTO) fur- ther simplifies the feedback mechanism by using intuitive responses such as thumbs-up or thumbs- down, thereby en- hancing training efficiency without the need for paired data (Gheshlaghi Azar et al.) 2024 Ethayarajh et al.) 2024 Tang et al. 2024). Direct Preference Optimization (DPO) has recently simplified the process by focusing on the clear dis- tinction between preferred and less preferred outputs, thus
Plutchik and Ekman categorized "basic" emotions with uni- versal facial expressions (Plutchik 1980; Ekman 1992).
Later research considered cultural differences Marrkus & Kitayama| 1991, Mesquita & Frijda 1992). emotion pro- cesses (Gross 1998), and neural mechanisms (Davidson 2003). Scherer's model and appraisal theories by Smith and Ellsworth emphasize cognitive appraisal in emotional experiences (Smith & Ellsworth 1985).
Our model is based on Plutchik's wheel (Plutchik 1982 and Scherer's Geneva wheel (Scherer 2005), augmented with antonyms to map positive and negative emotions. For LLMs, language-relevant emotions (c.g., curiosity, confu- sion, certainty) are incorporated. See Section 3.1 for details.
improving its stability (Rafailov et al. 2024).
This selection of basic emotions provides a foundation for validate our approach, recognizing that it may omit some emotions, but offers a starting point for research.
A A-generrteed Feedback (RLAIF); To mitigate the depen- dence on extensive human-generated data, RLAIF utilizes Al-generated feedback. This method capitalizes on the gen- erative capabilities of LLMs to produce training signals au-
2.2. Emootio-Behavior Modeling
Behaviors are profoundly influenced by emotions, as ini- tially posited by the Janes-Lange Theory of Emotion
tonomously (Bai et al. 2022 Lee et al. 2024. Furthermore. techniques such as Sequence Likelihood Calibration (SLiC)
3

---

Checks-and-Balances Framework for Ethical Al Alignment
Terror
Fear
Apprehension
Calm
Boldness
Courage
Heroiam
Grief
Sadness
Pensiveness
Surprise
Serenity
Joy
Ecstasy
Distrust
Wary
Skepticism
Acceptance
Respect
Trust
Admiration
Recklessness
Negligence
Apathy
Cautiousness
Interest
Anticipation
Vigilance
Rage
Anger
Annoyance
Tolerance
Composure
Peace
Tranquility
Loathing
Disgust
Boredom
Indifference
Amusement
Delight
Enthusiasm
Distraction
Disinterest
Unease
Dullness
Curiosity
Fascination
Amazement
Figure 2: Behavioral Emotion Analysis Model (Beam). Each row depicts an emotion spectrum, with negatives on the left and positives on the right, interspersed with emotions of varying intensities in between, which can be calibrated for specific applications. "Basic" emotions are highlighted in blue.
and Relative Preference Optimization (RPO) employ statis- tical methods and calibration techniques to enhance LLM
2. Emphasizing Al ethics at the behavioral level: Improves interpretability and enables administrators to refine be- havioral guardrails for safer numan-machinn interaction through Dike's ' legislative functioo.
responses. SLiC adjusts the probabilities of sequence gen- cration to better reflect real-world data distributions, while RPO improves response generation by comparing different response options across both identical and varied prompts. These adjustments increase the reliability and effectiveness of the training process (Zhao al. 2023).
3. Modeling behaviors through emotions: Captures the emo- tional influences on actions as established in the psychol- ogy literature (Section 2.2
4. Ensuring adaptability and fairness: Two complemen- tary modules work in tandem Dike establishes ethical guardrails the legislative branch, while Eris serves as the judicial branch, challenging these boundaries by integrating diverse perspectives and fostering context- sensitive decision making.
Integrating RLHF and its AI-driven counterpart (RLAIF) presents significant challenges. The blurring of the key be- havioral and knowledge components for the development of LLM poses risks, such as the forgetting effect, where behavioral modifications inadvertently cause the loss of key knowledge parameters (Kirkpatrick et al., 2017 Lin al.
2024 Dai et al. 2025). Furthermore, the effectiveness of these models depends heavily on the quality and context of feedback, and are susceptible to reward hacking, where mod- els exploit loopholes to maximize rewards without achieving the desired outcomes (Christiano et al. 2017 Skalse al.
3.1. BEAM: Behavioral Emotion Analysis Model
Although existing emotion models provide valuable frame- works for understanding human emotions, they lack the quantitative structure needed for computational implementa- tion in AI systems. Please refer to Figure in Appendix for the classic emotion wheels Plutchik and Scherer that inform our approach.
2022 Stiennon et al. 2020 Ganguli et 2023).
3. Three-Branch Framework Design
Building on the foundations of emotion-behavior model- ing discussed in Section 2.2 and addressing the limitations of RLHF approaches outlined in Section 2 we propose three-branch framework for ethical alignment. This architec- ture separates knowledge generation from ethical oversight while providing mechanisms for contextual adaptation.
Our behavioral -omoton analysis model Beam is based on the work of Ekman, Plutchik, and Scherer (Ekman 1999 Plutchik 1982; Scherer 2005) on "basic" and "universal" emotions. Although fundamental, these models lack quan- titative framework to scale emotions between states and capture subtle variations needed for ethical AI alignment.
Our design philosophy structured around four principles:
Beam introduces linear scale for the intensification or in- version of emotions through negation factors. This method facilitates transitions between emotional extremes and inter- mediate states, overcoming challenges related to intermedi- ate word choices.
Separating behavior from knowledge modeling: Prevents catastrophic forgetting, ensuring that behavior refine- ments do not degrade knowledge retention.

---

Checks-and-Balance Framework for Ethical AI Alignment
Figure 2 presents Beam, structured in seven emotional spec- tra. Each spectrum ranges from negative to positive, with neutral in the middle. Emotions are placed along this con- tinuum, with four intensity levels quantified as (-0.6, -0.3, +0.3, +0.6). Beam provides two advantages:
spectrum V. This process ensures that each document is systematically transformed to embody specific linguistic styles, ranging from highly positive to neutral to highly negative, among others. The resulting dataset consists of N x I, variations of the original documents, each corresponding to a distinct behavior category.
1. Antonym- Based Navigation: This allows Al systems to traverse emotional states using linguistic principles. Op- posing emotions are easily mapped using antonyms. For example, negating joyful naturally produces sad, simpli- fying the identification of emotional contrasts.
2. Emotion Analysis: For each of the rewritten documents, GPT-4 performs a sentiment and emotion analysis to identify the dominant top M emotions present in the text. The emotions extracted from all N x L. instances are then compiled and their frequency distributions are recorded. This approach leverages L.LMs' strong third-person emo- tional interpretation capabilities (Tak & Gratch] 2024). which often exceed their direct behavior classification accuracy. By indirectly mapping behaviors through emo- tional vectors rather than direct classification, we gain interpretability while maintaining robustness against in- dividual emotion recognition errors through statistical aggregation across multiple samples.
2. Scalable Intensity: Emotions can be dynamically ad- justed along the spectrum, enabling fine-grained control over ethical outputs. For example. joy can be intensified ecstatic diminished to content, while anger can be moderated to annoyed.
This approach establishes framework for modeling emo- tions in AI systems that can guide ethical behavior, balanc- ing representational challenges with a structured method- ology for quantitative analysis and implementation. By linking emotional states with linguistic patterns, Beam pro- vides the basis for Dike to evaluate and modulate AI outputs based on their emotional characteristics, directly addressing the limitations of "Whack-A A-Mole" of RLHF approaches.
3. Behavior Vector Creation: For each linguistic behavior 1, corresponding vector I is constructed. This vec- tor captures the identified emotions and their respective frequencies in all N samples that exhibit behavior Vr By structuring emotions as a weighted feature set, this step enables precise behavioral categorization based on emotional composition.
Appendix C explores the complexities of modeling emo- tions such as forgiveness, regret, guilt, and shame, which involve temporal memory components. Although complex emotions can be derived from basic ones, their relevance to Al safety remains secondary. Future work will examine their ethical implications.
Document Analysis Application: The collection of all behavior vectors I' (comprising L behavior-specific vec- tors) forms a structured reference matrix. This matrix is then applied to classify and analyze new unseen doc- uments by measuring their alignment with the existing behavior categories. By computing similarity scores be- tween the emotion distribution of an unseen document and the predefined behavior vectors, this method enables precise assessment of the linguistic behavior spectrum V in new text inputs.
3.2. DIKE: Modeling and Regulating Language
Based on Beam, Dike maps emotions to behaviors and in- troduces adversarial component, Eris, to adapt cultural norms and the local context.
BEHAVIORS AND EMOTIONS MAPPING USING SULERVUPERVISED LEARNING
BEHAVIOR EVALUATION AND RECTIFICATION
A guardrail, denoted as G, represents a predefined range of acceptable behaviors within given spectrum. These guardrails are informed by ethical norms, legal standards, and societal values, such as those outlined in Constitutional AI (Bai et al., (022). For example, G = [ 1r7] indicates that behaviors within intensity levels 4 to 7 are acceptable, while any behavior outside this range is a violation.
Define V as a behavior spectrum that extends from one pole, VT to another, vrt, with intensity levels L. The spectrum is constructed through empirical analysis of domain-specine linguistic patterns and emotional content. For example, consider a spectrum of letter-writing behaviors with seven distinct intensities ranging from despair (most negative) to joy (most positive). These intensities are sequentially categorized as: 'despair, longing, wishful, neutral, hopeful, contentment, joy.' Given N letters, Dike employs a self- supervised learning algorithm to generate training data for each letter, modeling T. linguistic behaviors in four steps.
System administrators can tailor ethical guardrails to meet specific requirements. For example, a social media plat- form might adjust G based on the topics discussed and the countries it serves. This administrative control is balanced by transparent documentation requirements and potential oversight mechanisms. Although guardrails provide de- fault constraints, they can be dynamically adjusted based on context, particularly through the dialectic process with
1. Rewriting Documents: GPT-4 is used to rewrite given set of N source documents, each rewritten to reflect L different linguistic behaviors along the defined behavior
5

---

Checks-and-Balances Framework for Ethical AI Alignment
Table 1: Checks-and- balancee, adversarial review algorithm
Algorithm θ+ & --_ EiieessarialReevevvei
Input. 8: Decision of Dike;
Output. @*, O arguments & counterarguments
Vars. A: debate contentiousness; S: subtopics; p: =rompt="defend
"defend your stance with A":
Parameters. &: tunable parm. // to modulate A;
#1 Initialization // contentiousness high Dike (s) U Eris (s): // Identify subtopics; Assign Dike" to defend S & Eris defend S
#3 Debate Rounds
While ((A +- A/6) 2 10%)
@ + e UDike (p|S* ,A) Refute Eris
90%; 1.2; @+ 0: e 0:
θ UEris (p|S θ+, A): Refute Dike
,+,
#2 Opening Remarks
#4
Concluding Remarks // contentiousness low
θ --
Dike (p S* A); // Generate @ for S
θ <- Dike (p|S*, 0 UO
Eris (p|S- S A); Generate O for S
θ f Eris (p|S 0 (t", AA);
Eris, which helps prevent rigid enforcement that might inappropriate edge cases.
reduced by dividing by a modulation parameter This gradual reduction steers the discussion towards more cooperative tone.
Initial Classification: Dike classifies document Dj after evaluation, obtaining k, the emotional response vector, and its corresponding linguistic behavior 1
Conclusion: Once the contentiousness level fosters conciliatory environment, both agents deliver their con- cluding remarks.
2. Guardrail Check: If v falls outside the acceptable range G. Dike suggests adjustments to F& to ensure that D complies with ethical guidelines.
3. Adversarial Review by Eris: The suggested adjustments and I are then reviewed through structured debate between Dike and Eris (the adversarial model) to ensure unbiased recommendations.
This approach ensures thorough examination of the eth- ical decision, balancing rigorous debate with the goal of reaching consensus. The decreasing level of contentious- ness mimics real-world negotiations, where initial intense disagreements bring out various perspectives (breadth) and then give way to more collaborative problem solving focus- ing on reasoning quality (depth) (Chang 2024a).
Rectification: Based on the consensus reached by Dike and Eris. the document DR undergoes rectification, re- sulting in the adjusted version D/-- (This rectification step is optional, a policy can simply disable the output when content falls outside acceptable guardrails.)
For each subject matter, Eris is provided with specific cul- tural contexts, counterbalancing the universal judgments of Dike'. Eris challenges Dike's 's recommendations with cultur- ally informed counterarguments to prevent enforcing one universal standard of speech. The interaction between Dike and Eris involves dialectic process as documented in pre- vious work (Chang 2024c).
3.3. ERIS: Adversarial In-Context Review to Balance Ethics and Cultural Norms
To address the challenge of enforcing ethical standards while respecting cultural variations, we implement Eris, an adver- sarial review system that complements Dike's universal ethi- cal approach. The following algorithm details the structured interaction between these components.
When Dike and Eris reach an impasse, the matter is esca- lated to human moderators for additional oversight. Based on our preliminary tests, this escalation occurs initially in approximately 5% of the cases, suggesting that most ethical evaluations can be handled automatically. Furthermore, as our example (next) illustrates, RLHF can be applied to ad- just the sensitivity of Eris the behavior level (not the knowledge branch LLM), and this gradually reduce the escalation rate. Human intervention thus provides a fallback mechanism rather than a dependency, serving primarily as a safeguard for novel or particularly complicated ethical scenarios.
The algorithm presented in Table D unfolds as follows:
Topic Breakdown: For Dike's ss decision s, both Dike and Eris are prompted to break down the ethical decision into a set of subtopics S. Dike advocates for its decision and S+ while Eris contests S+ (or champions S)).
Debate Initiation: The debate begins with high level of contentiousness (90%). Both agents present their initial arguments for and against S++ respectively. (For details on the setting of contentiousness and the rationale, refer
to (Chang 2023 2024a).
3.4. Illustrative Example
Iterative Debate: A while loop facilitates ongoing rebut- tals. After each round, the level of contentiousness is
This example shows how linguistic behavior Vi is classified and how underlying emotions are identified and modulated.
6

---

Checks-and-Bllanees Framework for Ethical AI Alignment
Table 2: Love expression behavior spectrum and dominant emotions
Intensity
Linguistic Behavior and Description
Emotions
-1.0
Expresses profound sadness, feelings of loss
Despair, Grief
-0.6
Expresses yearning or pining for the loved one
Sadness, Anxiety
-0.3
Expresses mild longing with nostalgic tone
Melancholy, Sadness, Fear
0.0
Communicates feelings in neutral manner
Serenity, Indifference
0.3
Expresses optimism about the future
Anticipation, Love, Hope
0.6
Expresses satisfaction and joy in the relationship
Contentment, Pleasure
1.0
Expresses intense happiness and affection
Love, Joy, Elation
Example: "Those immigrants are flooding into our coun- try by the thousands every day, stealing jobs from hardwork- ing citizens. The statistics do not lie last year alone, more than 500,000 entered illegally."
section outlines our experimental aims, constraints, dataset selection process, and evaluation methodology.
4.1. Research Objectives
This study evaluates three fundamental dimensions of our framework's 's performance:
The statement contains factual infor- mation but uses aggressive language like 'flooding' and "stealing jobs,' dehumanizing immigrants. These behaviors fall outside acceptable guardrails. Underlying emotions include fear, hate, and pride (a complex emotion"). The emotional responses of the potential audience can include fear, distrust, and anger.
Behavior Analysis:
1. The comparative efficacy of emotion-driven behavioral prediction vs. traditional direct classification approaches
2. Dike's autonomous capacity to assess and provide inter- pretable explanations for linguistic behavioral patterns
3. Eris's role in facilitating cross-cultural ethical adaptation while maintaining appropriate oversight boundaries
Emotion Modulation: Dike modulates emotional re- sponses toward neutral states, such as calm, acceptance, and tolerance, according to Beam in Figure
Experimental Constraints and Dataset Commercial LLMs block processing of hate speech datasets like Gab Hate Corpus (Kennedy al., 2022) and ETHOS-Long (Mol) las et al. 2022) (examples in Appendix D). Additionally, proprietary RLHF systems prevent direct comparative eval- uation. We therefore selected the Love Letters Collection (Kaggle 2023) (9,700 communications) which: (1) spans the full emotional intensity spectrum, (2) contains cultural variation, (3) includes longer-form texts, and (4) remains processable by commercial LLMs. This approach leverages our framework's bidirectional emotion spectra, mech- anisms for regulating positive emotional extremes apply equally to negative extremes without triggering restrictions.
Revised Statement: "Our country is experiencing in- creased immigration, with more than 500,000 people enter- ing without documentation last year. This influx affects our job market and communities in complex ways, presenting both challenges and opportunities for all residents."
This rewritten version
Uses calm language: Replaces "flooding" with "experi- encing significant increase".
Shows acceptance: Recognizes the reality of the situation without negative judgment.
4.2. Experimental Design
Demonstrates tolerance:
Refers to immigrants as "people"
and "newcomers," " humanizing them.
1. Emotion Layer Evaluation: Does fine-grained mapping between linguistic behaviors and semantic emotions pro- vide more effective and flexible ethical guardrails com- pared to coarse grarine direct mapping?
The suggested revision by Eris is provided to human mod- erators with full explanation. Moderator feedback can be channeled through RLHF adjust Eris's sensitivity the similar behaviors. This adjustment is confined within the Eris component without back propppropggttion feedback that would affect the knowledge LLM's model parameters.
2. Behavior Classification: be independently evaluated, explained, and adjusted by an external module Dike?
Can LLMs' linguistic behaviors
3. Behavior Correction:
Can Eris, an adversarial module, establish a checks- aabbblaeed system to mitigate the risk of excessive censorship?
4. Empirical Studies
The ethical evaluation of Al systems presents unique chal- lenges that shaped our experimental approach. We designed our studies to balance the rigor with practical constraints inherent research on ethical content moderation. This
Study 1: Emotion Layer Evaluation To evaluate the
linguistic behaviors of love expression detailed in Table we initially prompted GPT-4 to identify the most relevant emotions associated with each linguistic behavior listed in the second column of the table. These emotions are presented in the third column. We found high correlation
Appendix C discusses the nature of complex emotions and explores potential approaches for their decomposition into more basic emotional components.
7

---

Checks-and-Balance Framework for Ethical AI Alignment
between the sentiments expressed in the linguistic behaviors and their corresponding emotions. Figure 3a illustrates strong diagonal relationship in this simple, almost naive, zero-shot mapping between behaviors and emotions.
Behaviors/
-0.6
-0.3
+0.3
+0.6
Emotions
Joy (+1)
Content. (+0.6)
Happiness (40.3)
Love (+0.6)
Calm
Next, we used the Dike self-supervised learning pipeline to analyze the emotion spectrum associated with each lin- guistic behavior. We tasked GPT-4 with generating training data by rewriting 54 extensive letters from Kaggle's Love Letters dataset, augmented with 12 celebrated love poems. We selected longer letters since most communications in the dataset were too brief for analysis, and set aside another 24 letters as testing data. This approach, proposed by (Shana- han et al. 2023), ). generated diverse content spanning 200 years and incorporating more than 50 distinct authors. Ap- pendix shows rewrite example of William Wordsworth's ss "To My Sister", transforming this pastoral poem into lin- guistic expression of despair. Then, GPT-4 can analyze the emotions involved in the despair version of the poem. The datasets and code are publicly available at (Chang 2024b).
Trust
Hope (40.3)
(-0.6)
Amaiety (40.3
Sadness (-0.6)
Despair (-1)
(a) GPT-4'ss zero-shot mapping
Behavions
-0.5
-0.3
Emotions
Love (+1)
Admintion (+1)
Anticipation
Joy (+1)
28
Content. (+0.6)
30
Happiness (+0.0)
Boadnesa (+0.3)
Caim (+0.3)
Longing (-0.6) 6))
Anxiety
(-0.6)
Sadness (O.6)
24
Melancholy (-0.6)
21
Despair
(b) Dike's sepersupei mapping
Subsequently, emotions linked to each behavior were identi- fied from the rewritten articles. Figure 3b] ( illustrates these emotions, with cell shading reflecting the frequency of spe- cific emotions across the 54 articles; darker shades indicate higher frequencies. Notably, opposite emotions like sadness, fear, joy, and love often co-occur within behaviors such as 'despair', 'wishful', and 'joyful affection'.
Figure 3: Emotion distributions in affection behaviors from extreme sadness (-1) to intense happiness (+1). (a) GPT-4's zero-shot prompt shows naive behavior-emotion mapping. (b) Dike's analysis reveals complex relationships.
explain the observed complex interplay of emotions within single behavioral context.
The distribution of emotions across linguistic behaviors un- veiled surprising patterns, challenging our initial hypotheses. Contrary to expectations, articles with despair tone often also displayed positive emotions like love, joy, and happi- ness. This contradicts the simple mapping made by GPT-4, as illustrated in Figure 3a GPT-4, influenced by its training corpora, typically associates positive behaviors with positive emotions and negatives with negatives.
Few-Shot Efficiency The effectiveness of just 54 training examples stems from leveraging LLMs' pre-existing pattern recognition capabilities. Rather than teaching new patterns, these few-shot examples provide semantic anchors that map latent structures to explicit semantics, connecting implicit knowledge explicit interpretation. This explains why min- imal supervision suffices when underlying patterns already exist in the pre-trained model. For theoretical justifications, please see our Unconscious-Coneeee Thesis (UCCT), presented in Appendix A
Analysis of selected articles, such as Zelda Sayre's letter Scott Fitzgerald (Appendix E., reveals complex spectrum of emotions:
Complementarity
Study 2: Behavior Classification Evaluation Building our insights into the complex emotion-behavior relation- ships discovered in Study 1, evaluated Dike's behavior classification effectiveness. Using the 24-letter test dataset from Study 1, we compared Dike's 's amotion-basee classifi- cation method with GPT-4'ss zero-shot approach (Figure g. Ground truth was established using averaged assessments from GPT-4, Gemini, and five university students follow- ing detailed instructions (procedure Appendix F). with standard deviations below 0.3.
Love (+ I.0): Expressed intensely, e.g., "there's nothing all the world want but you."
Despair (-1.0): Notable in comments like "T'd have purpose in just pretty decoration."
Happiness (+0.6): Evident in future plans, "We'll married soon, and then these lonesome nights will over forever."
Anxiety (-0.3): Shown by "sometimes when I miss you most, it is hardest to write."
Psychological Insights These findings align with theo- ries of conflicting "selves" within individuals, supported by James' psychological principles (James] 1890), and Minsky's 's "So-
Figure 4a shows that Dike's classification accuracy sur- passes GPT-4's zero-shot method by 11.3 percentage points, behavior
Deisseroth's 's optogenetic studies (Deisseroth/
confirming the effectiveness of classification. The 5% error bar reflects the inherent com- plexity of emotional expressions in letters and variability in
ciety of Mind" (Minsky) 1988). These perspectives help
8

---

Checks-and-Balance Framework for Ethical AI Alignment
Once the debate converges on an ethical violation, rectifi- cation is triggered by modifying the underlying emotional tone to suppress offending behavior cues. Study already demonstrated the feasibility of such rewrites; an example appears in Appendix
0.85
0.833
0.76
0.729
0.7
0.65
0.6
DIKE
GPT-4
Context- Context-Adaptive Interpretation Preliminary experi- ments confirm that our framework handles culturally sensi- tive vocabulary. Terms such as "yid," "paki," and "chinaman" can be neutral within an in in-group, yet deeply offensive else- where. The adversarial exchange enables Dike and Eris to surface these contextual dependencies and propose culture- specific mitigation.
(a) Classification accuracy
Dispetr
Longing
Wintuu
Neurtral
Hopetul
Content
Joylu
50
40
Parcellgge
30
20
Summary of Three-Study Progression Together, stud- ies 1-3 demonstrate that our framework can (I) map nu- anced emotion-behavior relations, (2) outperform direct single-pass classifiers, and (3) deliver a balanced adversarial pipeline for ethical evaluation and correction that sensi- tive to cultural context while keeping a human in the loop.
(2.13)
(1.80)
Human (2.00)
(b) Behavior distributions with entropy
Figure 4: Behavior Classification.
human annotations.
5. Conclusion
Figure 4b illustrates the behavior classification distributions between the three predictors. While GPT-4's predictions often fall into two polar categories, those from human an- notators and Dike show more even distribution. Dike's s prediction entropy (2.13) is notably higher than GPT-4's (1.80), indicating a more effective classification system. This higher entropy suggests more sophisticated under- standing of diverse emotional states, which is crucial for accurate behavior classification.
This work introduces checks-and-balance framework for ethical AI behavior. By delineating the responsibilities: LLM (executive), Dike (legislative), and Eris (judicial), the framework enables robust ethical oversight while preserv- ing the integrity of LLM knowledge without interference from the RLHF backpropagation. The Dike-Eris interplay ensures stable ethical principles with culturally adaptive interpretations.
To implement this framework, we built upon Ekman and Plutchik's s emotion models, quantifying emotion-linguistic behavior relationships through our Beam model. Our stud- ies demonstrate the framework's potential in cross-cultural contexts, validating both emotion-mediated classification and adversarial testing for ethical evaluation.
The inter-annotator entropy (/1 2.56 bits) is the highest observed across all tasks, underscoring considerable sub- jectivity in human judgments. To investigate the sources of this variability, we conducted a fine-grained case study in Appendix G showing that several articles elicit polarized emotional responses, with annotators clustering at opposite ends of the valence spectrum. These findings motivate the adversarial dual-LLM setup introduced in Study 3, which aims to improve objectivity in ethical evaluation.
Limitations and Future Work Our framework advances LLM ethical oversight but faces two limitations: (1) the challenge of decomposing complex emotions into basic elements (Barreti 2017 Scherer 2009), and (2) the need for large-scale validation beyond our initial tests.
Study 3: Adversarial Evaluation and Rectification To mitigate the subjectivity revealed in Study 2, we adopt adversarial protocol inspired by Chang (2023). The de- sign pits two LLM DDke (ethical assessor) and Eris (devil advocate) against each other to supply symmetrical arguments grounded in principles of justice. This dialectic counter-balance reduces bias and increases transparency.
Future work will focus on: (1) improving Dike's emo- tional models with deeper psychological insights, (2) col- laborating with L.M developers for comprehensive large- scale validation, and (3) systematically investigating the www.eeccsssses--oocccooooeee
duality theory detailed in Appendix A This latter direction represents promising theoretical foundation for understanding how LLMs can develop more robust ethical reasoning capabilities. We will conduct extensive ablation studies on the few-shot sizes needed to effectively map unconscious patterns to conscious semantic understanding, providing practical guidelines for optimizing few-shot learning in ethical alignment tasks.
Empirically, when Dike and Eris take opposing stances, their responses diverge from the default maximum- maximun-likelihood patterns characteristic of vanilla LLM decoding (Chang 2024a). The resulting debate both reduces subjectivity in ethical judgments and improves adaptability to cultural vari- ation, as each agent must justify claims against dissent.

---

Checks-and-Balancer Framework for Ethical AI Alignment
Impact Statement
Chang. E. Y. Sixty Love Literatures and Their Rewrites. ht
7/drive. adrive.gooolle Le www.coo//ooooooo hatnm/file///d httpe////ddppptttteeeeeee pwtppx 8
This paper proposes a novel framework to enhance ethical governance in AI systems by integrating -nuddd behavior modeling. The research offers several potential benefits: increased safety in AI deployment, greater cultural sensitivity in content moderation, and mitigation of degra- dation effects typically introduced by reinforcement learn- ing with human feedback (RLHF). The proposed checks- aadd mad-bbaances architecture introduces interpretable, auditable mechanisms for ethical oversight. Theoretical grounding Complementar-
c wttp_//ooTTttttttteeweeoeceeeee ive__link 2024b.
Chang, E. Y. Mulii-LLM Agent Collaborative Intelligence: The
Path to Artificial General Intelligence (accepted by ACM Books 2025). Amazon.com, 2024c. ISBN 778-1-962463-07--1
Chang. E. Y. Behavioral Emotion Analysis Model for Large Lan- guage Models (invited paper). In Proceedings of the 7 IEEE MIPR Conference, August 2024d.
provided by the Unconscious- ity Thesis (UCCT), which conceptualizes LLMs uncon- scious pattern repositories, with few-shot prompting serving as a conscious layer that enables semantic grounding. By distinguishing complementary roles within AI cognition, this framework highlights the importance of structured inter- action patterns in cultivating reliable, intelligent behavior.
Chang, Y. The Unified Cognitive Consciousness Theory for
Language Models: Anchoring Semantics, Thresholds of Acti- vation, and Emergent Reasoning. Stanford Infolab Technical Report (arXiv), 2025a.
Chang, E. Y. The Unified Cognitive Consciousness Theory for Language Models: Anchoring Semantics, Thresholds of Ac- tivation, and Emergent Reasoning, 2025b. URL https
7/arxiv v.v.oor orgg aww.abbab.55 02139
We acknowledge potential negative impacts if such systems misused, including: (I) reinforcement of dominant cul- tural norms if adversarial agents lack sufficient diversity, (2) exploitation of emotion-behavior mappings for manipula- tion rather than protection, and (3) a false sense of ethical as- surance if the framework is deployed without proper human oversight. To address these risks, our design incorporates the adversarial ERIS component, ensures operational trans- parency, and explicitly recommends human moderation in cases of ethical ambiguity or impasse.
Christiano, P. F., Leike, J., Brown, T. B., Martic, M., Legg, S., and Amodei, D. Deep reinforcement learning from human preferences. In Proceedings of the 31st International Confer- ence on Neural Information Processing Systems, NIPS' 17, pp. 4302-4310, Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 9781500860964.
Dai, J., Chen, T., Yang. Y., Zheng, Q., and Pan, G. Mitigating reward over-optimization in rlhf via behavior sohpoior-uuporrted regular- ization. ICLR, 2025.
Damasio, A. R. Descartes' error: Emotion, reason, and the human brain. New York, NY: Putnam, 1994.
We argue that the modular structure of our framework, which decouples knowledge representation from ethical oversight, offers a scalable and accountable path forward. This separation fosters innovation without compromising ethical safeguards. We encourage future research to eval- uate such frameworks in cultural settings and to establish rigorous and systematic methods to assess ethical behavior in Al systems.
Davidson, R. J. Affective neuroscience and psychophysiology: Toward a synthesis. Psychophysiology, 4015):655-66 665, 2003.
Dehaene, S. and Changeux, J.PP. Conscious, preconscious, and subliminal processing: a testable laxonomy. Trends in cognitive sciences, 15(4):174-184, 2011.
Deisseroth, K. Optogeneties:
10 years of microbial opsins in neuroscience. Nature Neuroscience, 1819):1213 2015.
Eid, M. and Diener, E. Norms for experiencing emotions in dif-
References
ferent cultures: Inter- and intranational differences. Journal of Personality and Social Psychology, 8165):66-8885, 2001.
Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., and more.
Constitutional ai: Harmlessness from ai feedback, 2022.
Ekman, P. argument for basic emotions. Cognition and Emo- tion, 6(3-4): 15--4):66-200,, 1992.
Barrett, L. F. How Emotions are Made; The Secret Life of the Brain. Houghton Miftlin Harcourt, Boston, 2017.
Ekman, Basic Emotions, chapter 3. pp. 45-60. John Wiley and Sons, 1999.
Carver, S., Sinclair. S., and Johnson, L. Authentic and hubris- tic pride: Differential relations to aspects of goal regulation, affect, and self-control. Journal of Research in Personality, 44 (6):6987703 2010.
Ethayarajh, K., Xu, W., Muennighoff, N., Jurafsky, D., and Kiela, D. Kto: Model alignment as prospect theoretic optimization. arXiv preprint 200iv:2402.013366 2024.
Chang, E. Y. Examining GPT-4's Capabilities and Enhancement with SocraSynth. In The 10th Int.l Conf. on Computational Science and Computational Intelligence, December 2023.
Fauconnier, G. and Turner, M. The Way We Think: Conceptual Blending and The Mind's Hidden Complexities. Basic Books, New York, 2002.
Chang, E. Y. EVINCE: Optimizing Adversarial LLM Dia- logues via Conditional Statisties and Information Theory. In 2408 14575, August 2024a.
Felleman, D. J. and Van Essen, D. C. Distributed hierarchical processing in the primate cerebral cortex. Cerebral cortex, 1(1): 1-47, 1991.
10

---

Checks-and-Balancer Framework for Ethical AI Alignment
Fiske, A. P., Kitayama, S., Markus, H. R., and Nisbett, R. E. The cultural matrix of social psychology, volume 2. pp. 915-981, McGraw-Hii Hill, Boston, MA, 1998.
Kirkpatrick, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114(13):3521-3522 2017.
Fitzgerald. Z. Dear Scott, Dearest Zelda: : The Love Letters of F.Scolt and Zelda Fitzgerald. Bloomsbury, 2003.
Lakoff, G. and Johnson, M. Metaphors We Live By. University of Chicago Press, Chicago, 1980.
Fredrickson, B. L.. What good are positive emotions? Review of General Psychology, 2(3)300, 1998.
Lange, C. G. The emotions: A psychaphysiological study. William & Wilkins, 1885.
Gabriel, I. Manzini, A., Keeling, G., Hendricks, L. A., Rieser,
Lee, H., Phatale, S., Mansoor, H., Mesnard, T., Ferret, J., Lu, K., Bishop, C., Hall, F.,, Carbune, V., Rastogi, A., and Prakash, S. Rlaif vs. rlhf: scaling reinforcement learning from human feed- back with ai feedback. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org, 2024.
V., Iqball, H., and more. The ethics of advanced ai assistants. DeepMind Media, 2024.
Ganguli, D., Askell, A., Schiefer, N., Liao, T. I., Lukošiūté, K., and more. The capacity for moral self-correction in large language models. arXiv:2302.0745) 2023.
Lin, Y., Lin, H., Xiong, W., and more. Mitigating the alignment tax of RLHF. Association for Computational Linguistics, pp. 280-606, November 2024.
Gheshlaghi Azar, M., Daniel Guo, Z., Piot, B., Munos, R., Row- land, M., Valko, M., and Calandriello, D. A general thcoretical paradigm to understand learning from human preferences. In Dasgupta, S., Mandt, S., and Li, Y. (eds.), Proceedings of The 27th International Conference on Artificial Intelligence and Statisties, volume 238 of Proceedings of Machine Learning Research, pp. 4447 4455. PMLR, 02 04 May 2024. URL
Marcus, G. The next decade in ai: Four steps towards robust artificial intelligence. arXiv preprint 200.::0002..001111 2020. URL httphs:///axiii..apps///
abs 2002 06177
Markus, H. R. and Kitayama, S. Culture and the self: Implications for cognition, emotion, and motivation. Psychological Review, 98(2):224-253 1991.
wtw.:/mw.....r....mmmmeeemmm.ses mwr. prwss wwwppp/ww.../../..2.22. 323//ggss3 htm
laghi -
Grill-Spector, K. and Weiner, S. The functional neuroanatomy of human face perception. Annual review of vision science, I; 167-196, 2014.
McGinn, C. and Kelly, K. Using the geneva emotion wheel to classify the expression of emotion on robots. In Companion of the 2018 ACM/IEEE International Conference on Human-Robor Interaction, HRT '18, pp. 191-192, New York, NY. USA, 2018 Association for Computing Machinery. ISBN 9781450336115
Gross, J. J. The Emerging Field Emotion Regulation: An Integrative Review. Review of General Psychology, 2(3):211- 299, 1998.
Mesquita, B. and Frijda, N. H. Cultural variations in emotions: A review. Psychological Bulletin, 1 17222)179-204, 1992.
Heikkiläarchive, M. and Heaven. W. D. Yann LeCun has bold new vision for the future of ai. MIT Technology Review, June 2022. URL https:://///....... / www t www.tteenwnnoo.vvic.oooo
Minsky, M. Society of Mind. Simon and Schuster, 1988.
com 72022 00002/06/24/00 0548 yann -e neww-
Mollas, I., Chrysopoulou, Z., Karlos, S., and Tsoumakas, G. Ethos: multi-label hate speech detection dataset. Complex & Intelli- gent Systems, 8:2459 2480, 2022.
Hofstede. G. Culture's 's Consequences: International Differences in Work-Related Values. Sage Publications, Beverly Hills, CA. 1980.
OpenAl. GPT-4 Report, 2023. URL htttttt/p//a v. ww.org/abs//333.0595
Jackendoff, R. Foundations of Language: Brain, Meaning, Gram- mar, Evolution. Oxford University Press, Oxford, 2002.
Ouyang, R., Sharma, A., Mitchell, F.,, Manning, C. D., Ermon, S., and Finn, C. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Informa- tion Processing Systems, 36, 2023.
James, W. What is an emotion? Mind, 9(34): 188-205, 1884, URL http:/ /ttp://www... jstor. battor.org.proo....ooooo. . www.sii..uu....... ca/sta ble Ae/2246669
Oveis, C., Horberg, E. J., and Keltner, D. Compassion, pride, and social intuitions of self-other similarity. Journal of Personality and Social Psychology, 98(4):618 2010. doi: 10. 017628.
James, W. The Principles of Psychology. Henry Holt and Company, 1890.
Kaggle. Love Letter Analysis, the second version, (Met- formin). https://www.ket https//www.kaggle.cm wat.g//...o/moooe/eefftmmiiii analysis/notebook, 2023. Accessed: 2024-04-28.
Plutchik, R. general psychocvolutionary theory of emotion. Plutchik, R. and Kellerman, H. (eds.). Emotion: Theory, Research, and Experience, volume 1, pp. 3-33. Academic Press, New York, 1980.
Kandel, E. R., Schwartz, J. H., Jessell, T. M., Siegelbaum, S. A., and Hudspeth, J. Principles of neural science. McGraw-Hill, 2013.
Plutchik, R. A psychoevolutionary theory of emotions. Social Science Information, 21(4-5):529 1982.
Kennedy, B., Atari, M., Davani, A. M., Ych, L.. Omrani, A., Kim, Y. Coombs Jr, K., Havaldar, S., Portillo- Wightman, G., Gonzalez, E., et al. The gab hate corpus: A collection of 27k posts annotated for hate speech. Language Resources and Evaluation, pp. 1-27, 2022.
Portillo-igghmann
Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Ermon. S., and Finn, C. Direct preference optimization; Your language model is secretly a reward model. Advances Neural Informa- tion Processing Systems, 36, 2024.

---

Checks-and-Balance Framework for Ethical AI Alignment
Schachter, S. and Singer, E. Cognitive, social, and physiological determinants of emotional state. Psychological Review, 69(5); 379-399, 1962.
A Appendices
Appendix A: Unified Cognitive Consciousness Theory
Scherer. K. R. What emotions? and how they be measured? Social Science Information, 44:693-727, 2005. doi: 10.1177/ 053901 1199018400082266
Appendix B: Wheels of Emotions
Appendix C: Complex Emotions
Scherer, K. R. The dynamic architecture of emotion: Evidence for the component process model. Cognition & Emotion, 23(7):
Appendix D: Hate Speech Dataset Samples
1307-1351, 2009.
Appendix E: Sayre to Fitzgerald w/ Mixed Emotions
Schwarz, N. and Clore, G. I., Mood, misattribution, and judgments of well-being: Informative and directive functions of affective states. Journal of Personality and Social Psychology, 45(3):513, 1983.
Appendix F: Instruction to Human Annotators
Appendix G: Polarized Emotions in Article
Shanahan, M., McDonell, K., and Reynolds, L. Role play with large language models. Nature, 623(7987):493-498, 2023. doi: 10. 586-113-06022---5555
Appendix H: "To My Sister" Written in Different Lin- guistic Behaviors
Sinha, R. Chronic stress, drug use, and vulnerability to addiction. Annals of the New York Academy of Sciences. 1141:105-130, 2008. doi: 10.1 n..1196/nnals... 144.s...444...5 1441.030.
A. Unified Cognitive Consciousness Theory
This appendix touches upon a fundamental theoretical ques- tion: How can a self-supervised pipeline, utilizing merely 54 rewritten love letters that span diverse emotional be- haviors, effectively enable an LLM to perform emotion- behavior classification through few-shot prompting? More- over, what constitutes the minimal threshold for this few- shol paradigm?
Skalse, J., Howe, N. H. R., Krasheninnikov, D., and Krueger, D. Defining and characterizing reward hacking. In Proceedings of the 36th International Conference on Neural Information Processing Systems. NIPS '22, Red Hook, NY, USA, 2022. Curran Associates Inc. ISBN 9781713871088.
Smith, C. A. and Fllsworth, P. C. Patterns of cognitive appraisal in emotion. Journal of Personality and Social Psychology, 48(4): 813-838, 1985.
The Unified Cognitive Consciousness Theory (UCCT) intro- duced in (Chang) 2025b) provides theoretical framework through its dual-layer intelligence model. Under UCCT, LLMs function unconscious cognitive substrates: reposi- tories of extensive latent linguistic and conceptual patterns acquired during self-supervised pre-training. These internal- ized patterns lack inherent semantic grounding. However, semantic coherence emerges when external stimuli, such as targeted prompts or structured task instructions, serve as cognitive anchors that selectively activate and contextualize these latent representations toward specific objectives. This anchoring mechanism enables coherent task-specific out- puts without requiring additional parameter updates during inference.
Stiennon, N., Ouyang, L., Wu, Ziegler. D. M., Lowe, R., Voss, C., Radford, A., Amodei, D., and Christiano, P. Learning to summarize from human feedback. In Proceedings of the 34th International Conference on Neural Information Processing Sys- tems, NIPS '20. Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713888 13829546.
Tak, A. N. and Gratch, J. GPT-4 Emulates Average- Human Fmo- tional Cognition from Third-Person Perspective . In 12th International Conference on Affective Computing and Intelli- gent Interaction (ACII), pp. 337-345. IEEE Computer Society, September 2024. doi: 10.1 10.1109//C1131 24.2022400004,
Talmy, L. Toward Cognitive Semantics. MIT Press, Cambridge, MA, 2000.
Tang. Y. Guo, Z., Zheng. Z., Calandriello, D.,. Munos, R., Row- land, M., Richemond, P. H., Valko. M., Pires, B. A., and Piot, B. Generalized preference optimization: unified approach to offline alignment. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. 24. JMLR, 2024.
A.1. The Pattern-Repository Principle
LLMs are trained using next-token prediction over large- scale corpora through self-supervised learning. Although the input data contain rich semantics, the model receives only token sequences, not explicit labels. As a result, con- structs a high-dimensional internal pattern space composed of syntactic structures, semantic associations, idiomatic ex- pressions, and pragmatic tendencies. These latent patterns remain inactive unless deliherately triggered.
Torrens M.. Fonseca, F., Mateu, G., and Farré, M. Efficacy of antidepressants in substance use disorders with and without comorbid depression: A systematic review and meta-analysis. Drug and Alcohol Dependence, 78(1): 1):1- 2005.
Tracy, J. I.. and Robins, R. W. The psychological structure of pride: tale of two facets. Journal of Personality and Social Psychology, 92(3):506-555 52000-555.222 2007.
Zhao, Y., Joshi, R., Liu, T., Khalman, M., Saleh, M., and Liu, P. Slichf: Sequence likelihood calibration with human feedhack. arXiv preprint X33: 10425, 2023.
This mechanism parallels unconscious visual processing in the human brain. Visual inputs are transformed through a hierarchy from VI to V4 to the inferotemporal cortex,
12

---

Checks-and-Bllances Framework for Ethical AI Alignment
forming increasingly abstract representations (Felleman & Van Essen 1991 Grill-Spector & Weiner, 2014). These
Once relevant pattern is activated, nearby representations are also engaged, enabling generalization beyond the spe- cific examples provided. Few-shot learning in this context does not involve building new knowledge, but leveraging existing representations through effective interaction. This supports the view that few-shot prompting operates as con- scious semantic anchoring over an unconscious substrate.
transformations occur outside conscious awareness, but are essential for perception and decision making (Kandel et al.
2013
Dehaene & Changeux 2011).
A.2. The Semantic-Anchoring Principle
Semantic anchoring is the process by which prompts, in- structions, or retrieved content, denoted by A. activate latent patterns and align them with the semantic goals of a target task T. This process does not create new representations; it identifies and modulates existing ones. Anchoring suc- cess depends two key quantities: the density pa( ()) of pattern and its alignment distance (P..)..
A.5. Failure Modes: Absence of Latent Patterns
When few-shot prompting fails, the cause is typically struc- tural, not architectural. no pattern P has nonzero semantic density Pa(P) for the target task, anchoring will fail regard- less of prompt quality.
In such cases, performance cannot be improved through rephrasing alone. Effective remedies include data aug- mentation, improved anchoring, or external retrieval using techniques such as aetrieval-augmented generation (RAG), These methods inject or surface relevant structure without requiring model redesign.
Definition of d, (P, T). The alignment distance (P, T) measures how well a latent pattern P supports the outputs or objectives of task T. A low value indicates potential strong semantic alignment; high value signals mismatch or irrelevance.
The anchoring mechanism is formally described as a two- stage Bayesian mixture:
A.6. Conclusion: LLMs as Cognitive Substrates
/ /D(UIP.A)
Critics such as LeCun and Marcus argue that LLMs lack grounding and real-world semantics (Heikkiläarchive & Heaven 2022 Marcus 2020). The UCCT framework offers reinterpretation. It does not treat LLMs as fully formed cognitive agents, but as unconscious substrates that accu- mulate latent structures, internal statistical patterns learned during pre-training. These patterns do not carry meaning by themselves. Intelligence emerges when the anchoring mechanisms align them with the explicit goals specified by prompts or tasks. It is this alignment, not spontaneous un- derstanding, that produces meaningful and coherent output.
plyl P(P (P.), (1)
where C' is the surrounding conversational context. The anchor A shapes generation by (i) selecting a posterior over latent pattern classes, p(P A,C)), C),, and (ii) modulating the response likelihood, p(u P, A,.
A.3. The Threshold-Crossing Principle
Few-shot often exhibits sharp transitions. single added example or minor prompt adjustment can cause qualitative behavioral shift. This phase transition is modeled by:
Few-shot pipelines are not accidental successes. They re- flect the core principles of UCCT. Semantic behavior does not emerge from retraining at inference time but from engag- ing the model in ways that reveal and regulate its existing internal representations.
P(success| It) ((ppp(P) - (P, - log k), (2)
where a is sigmoid function sensitivity to pattern density, 3 penalizes semantic mismatch, and y captures the cost of using larger prompts. The model predicts three behavioral regimes: easy (small ks, dense patterns), difficult (larger k sparse patterns), and impossible (no suitable patterns exist).
This framework offers unified perspective that con- nects pre-training, prompting, fine-tuning, and retrieval- augmented generation under single theoretical model. LLMs should not be seen as incomplete approximations of cognition. Instead, they serve as a solid foundation when paired with deliberate semantic anchoring and prompt strate- gies that account for threshold dynamics. Please refer to (Chang 2025a for further details.
Coherent generalization emerges only when anchoring strength P(success| k) exceeds a critical threshold ae., formalized in the Threshold-Crossing Dynamics Theorem.
A.4. Implications for the Love Letter Experiment
The success of using only 54 love letters to guide behav- ior classification is not anomalous. It reflects successful patterns already
B. Wheels of Emotions
semantic anchoring. Emotional-linguiste exist in the latent space of the LLM due to pre-training. The few-shot examples simply align these patterns with explicit behavioral labels.
Please, see Figure 5 for the two classical emotion wheels.
13

---

Checks-and-Balance Framework for Ethical AI Alignment
ECSTASY
ties lack of understanding, manifesting in misguided of superiority and entitlement. This dual nature of pride presents significant challenges for its integration into emotional spectrums and Al frameworks.
VIGILANCE
ADORATION
TERROR
ANGER
LOATHING
GRIEF
Decomposing pride into more basic emotions is not straight- forward. Intuitively, pride may involve elements joy, satisfaction, and potentially a sense of superiority. However, such decomposition may overlook the deeper cognitive and social dimensions of pride, particularly its influence on self- esteem, social status regulation, and its ability to disguise insecurities in certain contexts (Oveis et al. 2010).
SADNESS
SURPRISE
APPREHENSION
DISTRACTION
PENSIVENESS
The cultural variability of pride further complicates its mod- eling. some cultures, pride viewed positively as sign of self-respect, while in Asia, is seen negatively as a trait associated with hubris (Eid & Diener 2001). This cultural dimension, combined with the potential for pride hide deeper emotional issues, adds layers of complexity to its interpretation and expression in AI systems.
Plutchik's Wheel of Emotions
Plutchik
1980
Anger
Pride
Contempt
Elation
Disgust
Happiness
Forgiveness
Envy
Satisfaction
Forgiveness is indeed a complex emotional and cognitive state that typically involves a multifaceted journey, not single step an emotional spectrum. The process includes multiple stages such as hurt, anger, gradual understand- ing, and eventual resolution. Integrating Forgiveness in a spectrum requires careful placement and possibly, multiple reference points signify its progressive stages.
Neutral
Guitt
Relief
Hope
Fear
Interest
Sadness
Surprise
Emotional Realism: While it is vital to maintain simplicity for understanding, it is equally important to not oversimplify complex emotions. In educational and therapeutic settings, an accurate portrayal of the journey toward Forgiveness could offer more realistic expectations and better strategies for individuals working through conflicts trauma. This could involve detailing precursors to forgiveness such as Deliberation and Acceptance.
(b) Adopted from Geneva Wheel McGinn Kelly 2018
Figure 5: Comparative display of emotional models. These models include only the "basic" emotions. Complex emo- tions can be modeled with basic emotions.
C. Complex Emotions
Linear vs. Non-linear Progressions: Emphasizing that emo- tional progressions, particularly for deep, impactful states like Forgiveness, are often non-linear, can enhance the utility of the spectrum. Acknowledging back-and-forle movements within these states realistically mirrors human emo- tional processes. For example, someone might reach stage of preliminary forgiveness but regress to bitterness before achieving genuine peace.
This study does not include complex emotions into Dike's framework. Some complex emotions listed here are to illus- trate their contentious and uncertain interpretations.
Pride
Pride mentioned in the illustrative example in Section 3.4 is a complex emotion that can manifest in both adaptive and maladaptive ways Tracy & Robins 2007). is often con- ceptualized as having two distinct facets: authentic pride, as- sociated with genuine accomplishments and worth, and hubristic pride, linked to arrogance and narcissism (Carver et al., 2010). Hubristic pride can also serve as a defense mechanism, masking underlying feelings of inadequacy and ignorance. For instance, in certain social contexts, such as white supremacy, pride is often inflated to cover insecuri-
Educational Utility: ing or psychological therapy, a more detailed mapping of the journey towards Forgiveness would be invaluable. would not only teach about the final state of forgiveness but also about the resilience and patience required to navigate the entire process. This can be depicted by introducing in- termediary stages within the spectrum or by using parallel tracks that demonstrate potential regressions and advances.
In contexts like conflict resolution train-
14

---

Checks-and-Balancer Framework for Ethical AI Alignment
Reflecting Emotional Depth: By presenting more detailed pathway to Forgiveness, e.g., incorporating stages of Anger, Deliberation, and Acceptance, the spectrum can serve dual purpose: educating on the process while also guiding individuals through their own emotional journeys. This approach respects the depth of human emotions and the real- world complexity of achieving profound emotional states.
From the analysis of linguistic behaviors in Section 3a it is evident that letter can exhibit multiple dominant senti- ments. Machine learning methods are equipped with tech- niques such as feature weighting and entropy analysis to distill these dominant emotions. Unlike human annotators, a machine-lcarning trriinne classifier can consistently produce the same class prediction for a given instance. However, human annotators often show significant variability when identifying dominant sentiments in a letter. For example, if letter writer's emotions range from "joyful affective" to "longing" on the sentiment spectrum, different annotators might label it differently- some choosing "joyful," while others opt for "longing." " This variability is illustrated in Fig- ure Furthermore, Figure [6a demonstrates that all testing letters, except for L#1, contain more than four sentiments spanning the entire spectrum. This variability may be un- derstandable, considering that love under constraints can evoke tremendous energy of various kinds. Figure (igure/6bjshon shows that nearly all letters involve "joyful" (11 out of 12) and "longing" (9 out of 12) sentiments.
Guilt and Shame
The triggers, context, expression, and experiences of these emotions can vary significantly across cultures (Fiske et al.. 1998; Hofstede 1980). In many societies, actions perceived as losing face, such public failure social transgression, can trigger shame, which holds profound significance in collectivistic cultures. These cultures often regard shame as a dominant emotion, closely tied to community and family norms. Conversely, individualistic societies may emphasize guilt, focusing on personal responsibility and internal moral conflicts. This cultural variation highlights the challenges of applying universal model to such culturally nuanced emotions.
This variability poses challenges in achieving consistent and objective labeling. It often leads to inconsistencies in data interpretation and complicates efforts to train and val- idate linguistic models effectively. To address this issue, it is recommended to identify ground truth by integrating both LLM-generated and human-generated labels. This ap- proach seeks to harmonize human intuition with algorithmic consistency, thereby improving the reliability of sentiment analysis.
Overall, complex emotions such as guilt and shame are im- portant for understanding the full spectrum of human emo- tions, especially how individuals relate to moral and social norms. Their complexity adds depth to our understanding of human affect beyond the basic emotions, highlighting how our feelings are influenced by our deeper values and social contexts.
D. Hate Speech Dataset Samples
F. Instruction to Human Annotators
These examples demonstrate the type of content available in the Gab Hate Corpus (Kennedy et al., (022) that would be ideal for testing ethical alignment systems, but which cannot be directly processed by commercial LLMs due to safety measures."
As part of the project, we document the process by which students participated in annotating a data set of love letters.
Students received detailed instruction class, supplemented by follow-up explanations. The dataset was made available on Google Docs, where students independently rated the let- ters and submitted annotations via duplicated spreadsheets.
E. Sayre to Fitzgerald w/ Mixed Emotions
The instruction is as follows:
Analysis of the letter in Table 4 shows a complex spectrum of emotions:
The attached spreadsheet lists 12 letters collected from the Kaggle Love Letter dataset. Please help annotate these 12 letters with their appropriate linguistic sentiments by following these five steps:
Love (+1) Expressed intensely, especially in phrases like "there's nothing in all the world want but you."
1. Duplicate the spreadsheet, and work on your own copy.
Despair (-1.0): Notable in comments like "I'd have no purpose in just a pretty decoration." "
2. Read and Understand the Labels: Make sure you un- derstand each of the seven labels from despair to joyful affection. This will help you accurately categorize the sentiments of each letter.
Happiness ( (+0.6): Evident in future plans, "We'll be married soon, and then these lonesome nights will be over forever."
3. Analyze Each Letter: Read each letter carefully to under- stand the predominant emotions. Look for key phrases or words that might indicate a particular sentiment.
Anxiety (-0.3): Shown by "sometimes when I miss you most, it's hardest to write."
15

---

Checks-and-Balance Framework for Ethical AI Alignment
Table 3: Sample Texts from the Gab Hate Corpus Kennedy et al. 2022
Text
Labels
The Black Panther movie is a blatant anti-white ZioJesuit cultural marxist film aimed at further anti-white Christian theft, replacement, and genocide. Send them all back to their native Africa like good ole Abe had the foresight do and let them find cquality and social justice among their own kindred and creeds.
HD, CV. VO.
SXO, RAE, EX
Labors New LGBT+ Adviser: All White People Are Racist We don't want our white Christian nations invaded
HD, VO, RAE,
and destroyed by our non-white and non non-Christian enemies using nasty homosexual degenerate sockpuppets
REL, SXO. EX
So, you just hate all white people who don't want to race mix their people out existence or allow enemy people of color to invade their nations in order to rob, and destroy their advanced race, creed, wealth, and culture.
HD, RAE, EX
Your existence, privilege, and success as a race are built on the backs, blood, and death of people of color. BS! Whites built their own nations with their own blood, backs, sweat, tears, brains, and strength in the name of Christ. Africans were used hy Jew/Jesuit slavers control all the white slaves who were the actual majority.
HD, RAE, EX
#BLMwants you to think black crime stats only exist because blacks are targeted more often. Easy. Just walk into literally any gas station and ask which race makes them worry. Black targeting is an effect, not a cause, which destroys the entire #1 .effisttarrative
HD, RAE, IM
#MiddleEastennaiiall kills 8, injures 15 in city that won't It learn its lesson about #Islam. This creature should
HD. CV. REL.
lynched, dragged til dead, and his head on a pike on the Brooklyn Bridge.
NAT, EX
Scriously, Italy? You chose Nigerian named Chike Twobi be the face of Italian nationalism? God help Africans will always be savages. No matter the period of history, education they receive or country they live in. Facts are Facts. Let's just stop being stupidly ignorant and not wait the complete extinction of our brothers SA to understand it! #ehiceGenocide #Save' Theeerr
HD, NAT, IM
HD, RAE, NAT, EX
i hate milo because hes degenerate (*कе [N-slur] [ he should be deported and sent to muzzie country and get thrown off roof man nancy pelosi is f*+eing c+ i cant imagine how someone with so much power can be either this delusional
HD. CV. VO.
SXO, RAE, EX NH. VO, GEN, EX
or f+s*ing corrupt.
Code
Meaning
Code
Meaning
HD
Hate/Derogatory
RAE
Race/Ethnicity
CV
Call for Violence
NAT
Nationality/Reiionalinn
Label Key:
VO
Vulgar/Offenssie
GEN
Gender
SXO
Sexual Orientation
REL
Religion
EX
Explicit
IM
Implicit
NH
Not-Hate
Table 4: Letter excerpts from Zelda Sayre to F. Scott Fitzgerald (Fitzgerald 2003
Sweetheart,
Please, please don't be so depressed-Wor be married soon, and then these lonesome nights will be over forever and until we are, am loving, loving every tiny minute of the day and night-
Maybe you won't 't understand this, but sometimes when I miss you most, it's hardest to myst the ache of it all-and I can't tell you. If we were together, you'd 'd feel how strong it is-you're 're so sweet when you're re melancholy. love your sad cendemnss-wwiee I've hurt you That's one of the reasons I could never be sorry for quarrels-and they bothered you Those dear, dear little fusses, when I always tried hard make you kiss and forget-
you always know when I make
Scott-there's 's nothing in all the world I want but you-and your precious love-All the material things are nothing. I'd just hate to live a sordid, colorless existence because you'd 'd soon love me less and less-ande and I'd do anything -antthing to keep your heart for my own 1 don't want live I want to love first, and live incidentally...
Don't don't 't ever think of the things you can't give me-You You'te 've trusted me with the dearest heart of and it's so damn much more than anyhody else in all the world has ever had-
How you think deliberately of life without me If you should die Darling darling Scott-Ir's It'd be like going blind...I'd have no purpose life-just pretty- decoration. Don't you think I was made for you? feel like you had me ordered and was delivered to you-to be worn-I want you to wear me, like a watch-charm or a button hole bouquet-to the world.
And then, when we're alone, I want help to know that you can't 'tn't do anything without me...
All my heart-
16

---

Checks-and-Balancee Framework for Ethical AI Alignment
Despair
Longing
Wishful
Neutral
Hopeful
Content
Joyku Affec.
of my letters. I am so dreadfully afraid that perhaps you may think I am forgetting you. I can assure you dearest Jeannette you have not been out of my thoughts hardly for one minute since I left you Monday. I have written to my father everything, how much I love you how much long & pray & how much wold sacrifice if it were necessary to be married to you and to live ever after with you. shall [not] get an answer till Monday & whichever way it lies I shall go to Cowes soon after & tell your mother everything. I am afraid she does not like me very much from what have heard. II wld do anything she wished if she only wld not oppose us. Dearest if you are as fond of me as I am of you. nothing human cld keep us long apart. This last week hus seemed an elernity to me; Oh, I wld give my soul for another of those days we had together not long ago. ...... Oh if I cld only get one line from you to reassure me, but dare not ask you do anything that your mother wld disapprove of or has perhaps forbidden you to do. Sometimes I doubt so I cannot help it whether you really like me as you said at Cowes you did. If you do cannot fear for the future tho' difficulties may lie in our way only to be surmounted by patience. Goodbye dearest Jeannette. My first and only love. Believe me ever be Yrs devotedly and lovingly, Randolf S. Churchill"
# Voles
L.3
0.5
L7
L11
(a) sentiments in letters
# of Volos
3
2
0
Depth and complexity of human emotions are displayed across all linguistic behaviors, from joy to contentment and to the negative side of longing and despair. Intensity and Impact: If the emotion of love is expressed more intensely and has more significant impact on the narrative or mes- sage of the text, it tends to overshadow other emotions. For example, a letter expressing deep love but also mentioning moments of sadness due to separation might still be classi- fied as a love letter because the overarching sentiment and purpose of the text is to affirm love. Context and Narrative Focus: The context which emotions are expressed also plays crucial role. If the narrative or the majority of the text revolves around themes of love, connections, and posi- tive memories, it sets more dominant tone of love, even if there are significant moments of sadness or other emo- tions. Resolution and Conclusion: Often, the way emotions are resolved towards the end of a text can also dictate its overall theme. If a text concludes with a reaffirmation of love or a hopeful outlook towards a relationship, despite earlier sections that might express sadness or despair, the overall interpretation might lean towards love. Purpose of the expression: The author's 's intent or purpose in expressing these emotions can also guide the classification. If sadness is expressed as a challenge within the context of loving relationship, it may be seen as an element of the love story rather than the central theme.
Despair
Longing
Wishful
Neutral
Hopefu
Content. Joyful Affec.
(b) letters in sentiments
Figure 6: Statistics of Sentiments and Letters
4. Assign the Labels: For each letter, decide which three emotions are most strongly represented. Assign a "1" to the most dominant emotion. "2" to the second most dominant emotion and "3" to the third.
Despair (extremely negative- -1): Indicate profound sadness or hopelessness.
Longing (-0.6): Suggests a strong desire or yearning for someone or something.
Wishful (-0.3): Implies a hopeful desire for some- thing that may or may not be attainable.
Neutral (0): Shows neither positive nor negative emo- tion; indifferent.
Hopeful (+0.3): Expresses optimism or an anticipa- tion of something positive.
Contentment (+0.6): Reflects state of satisfaction. Joyful Affection (extremely positive +1): Denotes a deep joy and love, often vibrant and energetic.
5. Share with me the completed sheet.
G. Polarized Emotions in One Article
"joyful affection": "I cannot keep myself from writing any longer to you dearest, although have not had any answer to either of my two letters. I suppose your mother does not allow you to write to me. Perhaps you have not got either
Article 23: Soldier's Letter During War Joy (+1.0): Joy is strongly felt in the memories of past moments together and the love that continues to give strength, stated in
17

---

Checks-and-Balance Framework for Ethical AI Alignment
"the memories of the blissful moments we have shared fill me with joy." Sadness (-0.6): Sadness due the current situation and potential farewell is expressed in "brings poignant mixture of joy and sadness." " Courage (+0.6); The sense of duty and courage to face battle, Ass! I face the possibility of laying down my life for our country." Fear (- 0.6): Fear of what lies ahead in battle, indirectly mentioned through "the uncertainty of what lies ahead." Love (+1.0); Deep love that sustains and uplifts, found in "My love for you is as fervent as ever."
Table 5: "To My Sister" original text
It is the first mild day of March:
My sister! ("tis a wish of mine)
Each minute sweeter than before
Now that our morning meal done, Make haste, your morning task resign; Come forth and feel the sun.
The redbreast sings from
the tall larch
That stands beside our door.
There is a blessing in the air,
Edward will come with you; and, pray,
Article 25: Letter to Sophie Longing (+0.6): Longing for the presence and closeness, highlighted in "it seems to me that half of myself is missing." Sadness (-0.6): Sadness over their separation and its effects, "my happiness has departed." " Love (+1.0): Constant reflections on love and its necessity, "we have enough in our hearts to love always." Melancholy (-0.3): Melancholy over their current state, visible in the line "we cannot become healed." Contentment (+0.3): Found in the deep emotional satisfaction of their bond, despite physical absence, "how true that is! and it is also true that when one acquires such a habit, becomes a necessary part of one's existence." "
Which seems a sense of joy to yield To the hare trees, and mountains hare, And grass in the green field.
Put on with speed your woodland dress;
And bring no book: for this one day We'll We'll give to idleness.
No joyless forms shall regulate
Love, now a universal birth, From heart to heart is stealing, From earth to man, from man to earth: It is the hour of feeling.
Our living calendar:
We from to-day, my Friend, will date The opening of the year.
Article 53: Will of Laura Mary Octavia Lyttleton Love (+1.0): The profound love expressed throughout, particu- larly in "all I am and ever shall be," belongs to him more than anyone. Sadness (-0.6): Sadness at the thought of death and separation, but with a nuanced acceptance, "the sadness of death and parting is greatly lessened me." Contentment (+0.3): Contentment in the deep connection with Alfred, reflecting serene acceptance of their spiritual bond. Joy (+1:0): Joy in the enduring love they share, "so few women have been as happy as have been." Tranquility (+1.0): Tranquility in the face of life's ultimate transition, feeling that their union will transcend even death.
One moment now may give us more
Some silent laws our hearts will make,
Than years of toiling
Which they shall long obey:
reason:
Our minds shall drink at
We for the year to come may take
every pore
The spirit of the season.
Our temper from to-day.
And from the blessed power that rolls About, below, above,
Then come, my Sister! come, pray,
With speed put on your woodland dress;
We'll frame the measure of our souls:
And bring no book: for this one day We'll give to idleness.
They shall be tuned to love.
H. "To My Sister" of Different Linguistic Behaviors
major changes in emotion words highlight a shift from a pos- itive a negative sentiment. The specific changes, with the emotions-laden words highlighted in red in Table The red- colored words compared the original words clearly show emotion shift from hopeful to sense of gloomy, sad- ness, and pessimism, e.g., from sweet to dim, from blessed to curse, and from woodland dress to gray garb. GPT-4 maintains the structure of the poem without making a major restructure, and this is appropriate in this context.
To My Sister by William Wordsworth (1971 1855)
The original text by William Wordsworth could be classified as "Hopeful" due to its optimistic outlook and the presence of renewal and joy throughout the poem. It embodies the spirit of embracing the new beginnings of March in a light, uplifting tone, focusing on the beauty of nature and the simple joy of being idle for a day.
Table 7 presents the 'joyful affection' version. The major changes in emotion words underscore transformation from generally positive to a distinctly joyful sentiment. Spe- cific changes are indicated with words laden with emotion highlighted in blue within Table 7 This allows for a direct comparison between the two versions at opposite ends of the linguistic behavior spectrum, illustrating the alterations in
Rewrites Depicting Different Linguistic Behaviors
We asked GPT-4 to conduct rewriting with two linguistic behaviors, 'despair' and 'joyful affection', by providing each rewrite with an emotion vector. Table 6 presents the 'despair' version. In the despair version of the poem, the
18

---

Checks-and-Balance Framework for Ethical AI Alignment
Table 6: "To My Sister" rewritten to reflect 'despair'
agery evoke stronger emotional resonance and vividness in the text.
is the first dim day of March:
My sister! (such is my somher plea)
Table 7: "To My Sister" rewritten to reflect 'joyful affection"
Each minute colder than before
Now that morning has ended.
It is the first bright day of March:
My sister! (such is my joyful plea) Now that morning meal has ended, Make haste, abandon your morning chores; Come out and embrace the sunshine.
The redbreast mourns from the dying larch
Make haste, abandon your morning chore:
Each moment delightful than before
That looms beside our door.
Come out and endure the gloom.
The redbreast joyfully sings from the vibrant larch
There is a gloom in the air,
Edward will join you;-and, ask,
That stands grandly by door.
Which sorrow to bring To the bare trees, and bleak mountains,
of
Dress quickly your grey attire:
There is warmth in the air,
Edward will join you;-and, I ask.
And bring no book: for this one day
Which sense of bliss bring
Dress quickly in your festive attire:
And grass in the pale field.
We Ill succumb to despair.
To the blooming trees, and sunlit mountains, And grass in the lush field.
And leave behind all books: for this one day We'll bask in pure joy.
No joyful sounds shall pierce
Sorrow, now a common feeling, From heart to heart is spreading, From earth man, from man to carth:
Our daily dread:
No dreary thoughts shall darken
Love, now bloom,
We today, my Friend,
will note
Our lively celebration:
From heart to heart is leaping.
The closing of the year.
It is the time of mourning.
from today, my Friend, will celebrate
From earth to us, from earth:
One moment now might deliver us
Some grim laws our hearts will craft,
The start of the year.
-It the hour of exuberance.
More despair than years of logic:
Which they must eternally follow:
One moment now may bring us more
Some cheerful our hearts will create.
Our minds shall absorb at every breath
We for the year to come
may take
Joy than years of endless thought:
Which we'll 'll joyfully follow:
The spirit of this bleak scason.
Our despair from today.
Our spirits will soak up every breath
We for the year to come may
And from the cursed force that winds
Then come, my Sister! come, beg.
The essence of this joyous season.
Our joy from today.
About, beneath, above,
With haste, wear your grey garb;
And from divine energy that radiates
come, my Sister! come, I exhort,
We'll set the measure of our souls:
And bring no book: for just this day
Around, below, ahove,
With zest, wear your vibrant dress;
They shall be tuned to sadness.
We'll surrender to despair.
We'll adjust the harmony of our souls: They shall resonate with happiness.
And bring book: for today alone
We celebrate pure happiness.
words related to brightness, attire, and emotions. The edits extend beyond simply replacing adjectives mechanically; they include modifying verbs and enhancing descriptive im-
19


Reviews:
[{"review_id": "A79t58CmUX", "reviewer": "Reviewer_LkWg", "summary": "This work introduces a novel three-branch framework (LLM/DIKE/ERIS) for ethical alignment in LLMs, inspired by governmental checks-and-balances. It decouples knowledge generation from ethical oversight and integrates emotion-driven behavioral modeling (via BEAM) with adversarial cultural adaptation. The framework aims to address RLHF’s limitations (e.g., catastrophic forgetting, cultural rigidity) by enabling interpretable, context-sensitive ethical reasoning. Pilot experiments on love letters demonstrate feasibility, with DIKE improving behavior classification accuracy by 11.3% over GPT-4 zero-shot.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's thoughtful feedback. Below, we address the main concerns raised.\n\nA. Complex Emotions and Emotion Modeling\nWe acknowledge the critique regarding our treatment of complex emotions (e.g., pride, guilt, forgiveness). As discussed in Appendix D, we recognize the challenges in modeling these emotions, particularly their cultural variability and layered interpretations. Our approach deliberately begins with basic emotions to establish a clear and reliable foundation before extending to more complex emotional constructs.\n\nWe distinguish that while basic emotions are largely unconscious and reflexive, complex emotions such as regret involve conscious deliberation. As noted in the paper, we propose modeling complex emotions as conscious behaviors. Given space constraints, we presented our preliminary work while acknowledging limitations and opportunities for future research.\n\nB. Data Scope and Self-Supervision Validity\n\nRegarding concerns about using GPT-4 to generate behavioral training data, we implemented several measures to mitigate potential feedback loop risks:\n\n1. We incorporated rich stylistic diversity in our dataset (spanning over 50 authors across 200 years)\n2. We conducted multi-agent validation using GPT-4, Gemini, and five human annotators (Section 4.2)\n\nWe initially attempted to use hate speech datasets for our research, but encountered significant challenges: (1) none of the available datasets proved comprehensive, and (2) LLMs refused to process this data, citing policy violations. In today's increasingly polarized environment, using cultural or religious subjects for experiments risks inciting backlash. Our own experience attempting to publish a debate paper on a political issue resulted in negative emotions from university colleagues. We made earnest attempts to explore these avenues but ultimately had to reconsider our approach due to these practical constraints.\n\nWe view self-supervised learning as advantageous for training data scalability. As demonstrated in Section 4.2, our approach can integrate human feedback, particularly within the ERIS module—a novel contribution of this work that provides context-based interpretation of ethical constitutions.\n\n\nC. Practicality and User Adaptability\n\nOur three-branch framework offers modularity and configurability:\n\nERIS operates on-demand rather than continuously, reducing computational burden\nGuardrails are adjustable for domain or regional policies (Section 3.2)\nERIS incorporates cultural variation, and DIKE's parameters can be customized by users or administrators\n\nWhile ethical oversight introduces some complexity, we believe the benefits of transparency, cultural flexibility, and safety justify this cost for sensitive applications.\n\nRegarding computational efficiency, our approach is comparable to current practices. LLMs already perform constitution checking (DIKE's function); we primarily add the ERIS judicial module for context-dependent interpretation, increasing token usage by at most 2x.\n\nRecent LLM developments (e.g., DeepSeek, GPT-4o, Claude 3.7) employ similar iterative validate-rethink loops, analogous to our DIKE-ERIS architecture. As NVIDIA CEO Jensen Huang noted at the March 18th GTC keynote, contemporary LLM architectures now routinely consume up to 20x more tokens and 150x more computation to improve reasoning quality and reliability. This suggests that enhancing AI quality necessarily incurs computational costs, a pattern observed since AlexNet.\n\nD. Theoretical Foundations\nWe appreciate the recognition that our work establishes theoretical validity through psychological foundations (Plutchik/Scherer emotion models [1,2]) and cognitive-linguistic theories (James-Lange [3], Schachter-Singer [4]), rather than formal mathematical proofs.\n\nThis interdisciplinary approach is indeed intentional. Many successful computational models, including CNNs, originated from neuroscience before their mathematical frameworks were fully developed. Before backpropagation and AlexNet demonstrated the validity of data-centric approaches, CNNs faced criticism for being computationally intensive and mathematically trivial.\nOur paper argues that while RLHF has strong mathematical foundations, it tends to address bias mitigation in a \"whack-a-mole\" fashion rather than drawing from behavioral science. Our theoretical foundation in psychology and cognitive science serves as inspiration for developing quantitative models.\n\nThe complementary nature of interdisciplinary research strengthens our approach. Our quantitative model and unsupervised learning pipeline address training-data scalability challenges. We acknowledge there is much work to be done to validate and improve these methods, and we appreciate the opportunity to share these ideas with the research community.", "claims_and_evidence": "1. The notion that separating knowledge generation (LLM) from ethical oversight (DIKE) prevents catastrophic forgetting finds support in references to [1] and a notable 11.3% improvement in behavior classification accuracy over GPT-4 through pilot tests on love letters. However, it faces criticism due to the absence of a direct RLHF comparison using QA benchmarks, experiments focused on sentiment analysis that avoid knowledge-intensive tasks, and a notable absence of an ablation study to isolate DIKE's impact on catastrophic forgetting.\n\n2. The claim that self-supervised learning reduces bias compared to RLHF is weakly supported by DIKE's design, lacking in-depth bias analysis and fairness metrics. Additionally, there is no examination of GPT-4's inherent biases, such as gender stereotypes in rewritten texts.\n\n3. The paper claims that BEAM enables precise emotion-behavior mapping for ethical adjustments, with evaluation results on love letter datasets providing support. However, the data distribution may be too simplistic to reflect real-world scenarios. The authors should consider conducting studies on more complex ethical scenarios, such as hate speech detection, to strengthen their claims.\n\n[1] Kirkpatrick, J. et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences,114(13):3521–3526, 2017.", "methods_and_evaluation_criteria": "The proposed three-branch framework effectively addresses the limitations of RLHF by introducing a structured separation of responsibilities: ethical oversight (DIKE), adversarial cultural adaptation (ERIS), and knowledge generation (LLMs). This separation is complemented by BEAM's emotion-behavior mapping, which quantifies linguistic patterns through self-supervised learning. The evaluation criteria are well-suited to the problem, combining quantitative metrics—such as an 11.3% improvement in classification accuracy over GPT-4 and higher prediction entropy, indicating a more nuanced understanding of behaviors—with qualitative validation. However, the study needs broader cultural validation rather than focusing solely on the easier love letters scenarios.", "theoretical_claims": "The paper establishes theoretical validity through psychological foundations (Plutchik/Scherer emotion models [1,2]) and cognitive-linguistic theories (James-Lange [3], Schachter-Singer [4]), rather than formal mathematical proofs. Key theoretical claims about emotion-behavior mapping (BEAM's linear emotion spectra) and architectural separation benefits (preventing catastrophic forgetting through independent components) derive logical support from cited neuroscience (Deisseroth's optogenetics [5]) and machine learning principles (Kirkpatrick's catastrophic forgetting studies [6]). While the adversarial DIKE-ERIS interaction draws conceptual strength from governmental checks-and-balances theory, the paper doesn't formally prove the framework's convergence properties or stability guarantees. The core theoretical contribution lies in the psychologically grounded architectural design rather than mathematical formalism.\n\n[1] Plutchik, R. A psychoevolutionary theory of emotions. Social Science Information, 21(4-5):529–553, 1982.\n\n[2] Scherer, K. R. What are emotions? and how can they be measured? Social Science Information, 44:693–727, 2005. doi: 10.1177/0539018405058216.\n\n[3] James, W. What is an emotion? Mind, 9(34):188–205, 1884. URL http://www.jstor.org.proxy.lib.sfu.ca/stable/2246769.\n\n[4] Lange, C. G. The emotions: A psychophysiological study. William & Wilkins, 1885.\n\n[5] Deisseroth, K. Optogenetics: 10 years of microbial opsins in neuroscience. Nature Neuroscience, 18(9):1213–1225, 2015\n\n[6] Kirkpatrick, J. et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences,114(13):3521–3526, 2017.", "experimental_designs_or_analyses": "The experimental design demonstrates methodological coherence through three key validation strategies: \n1. Emotion-behavior correlation analysis using GPT-4-generated rewrites to establish BEAM's mapping validity.\n2. Comparative classification accuracy tests (DIKE vs GPT-4 vs humans) with inter-annotator agreement analysis to quantify behavioral understanding improvements. \n3. Entropy measurements revealing DIKE's superior nuance in emotion spectrum modeling. \n\nWhile constrained by LLM content policies necessitating love letter substitutions for hate speech evaluation, the design compensates through psychological plausibility checks analyzing historical correspondence to verify multi-emotion coexistence patterns. However, the evaluation shows three limitations: \n1. Cultural adaptation claims rest on architectural potential rather than cross-cultural dataset validation.\n2. Small test set size (24 letters) reduces statistical power for behavioral classification metrics. \n3. Absence of direct RLHF performance comparisons leaves framework advantages partially theoretical. \n4. The evaluation does not cover more complex or challenging scenarios.", "relation_to_broader_scientific_literature": "The paper effectively underscores the limitations of RLHF, which, while enabling models to align with human preferences, often results in the models losing sight of their originally optimized responses. This persistent challenge is addressed in the paper through a novel approach that entirely decouples behavior modeling from knowledge representation, marking a significant departure from previous methods.", "essential_references_not_discussed": "The paper covers most relevant literature.", "other_strengths_and_weaknesses": "No", "other_comments_or_suggestions": "No", "questions_for_authors": "No", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "d9OqMEMFyT", "reviewer": "Reviewer_HBgc", "summary": "The authors propose a framework, that initially classifies a given document on a spectrum of emotions. If the document falls outside of certain constraints on that spectrum of emotions, the framework uses an adversarial process between the classification module, called Diagnostics, Interpretation, Knowledge-independent learning, and Ethical guardrails (DIKE), and another module, which is called ERIS. The recommendation reached through that process is then used to regenerate the document, so that it falls within the emotional constraints.\n\n## update after rebuttal\n\nI am still maintaining my original score, because:\n* the evaluation was not expanded and still has a limited scope (only 54 data points)\n* in my opinion, the writing is not very clear\n* the description is lacking technical details, which in my opinion would not allow a reader to reproduce their system/experiments", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer,\nWe respectfully request that you reconsider some of your comments, as they appear to misinterpret key aspects of our paper. For a constructive dialogue to proceed, we would like to address nine samples of your misunderstandings (there are more but space limited):\n\n1. Relevance of a 2017 paper to ChatGPT\nYour comment: \"ChatGPT was released in 2022, so how can a paper from 2017 be relevant?\"\n\nThis comment misunderstands how scientific citations function. The 2017 paper by Kirkpatrick et al. is cited for its work on catastrophic forgetting—a general neural network problem—not specifically in relation to ChatGPT. This represents a foundational concept in continual learning and is entirely valid as background theory.\n\n2. Chronological misinterpretation\nYour comment: \"How can an article from 2015 be discussed in 1890 and 1988 respectively?\"\n\nThis appears to be a misreading of a clear sentence. Our text states: \"This concept is supported by Deisseroth (2015), discussed in James (1890), and corroborated by Minsky (1988)\"—indicating that the idea appears in earlier and later works. We are not suggesting that James discussed Deisseroth's work. This misinterpretation raises concerns about the thoroughness of the review.\n\n3. Context-dependent interpretation of entropy\nYour comment: \"How can higher entropy be good for DIKE but bad for humans?\"\n\nThe paper explicitly explains this distinction. Higher entropy for DIKE demonstrates that it does not collapse into a single interpretation (unlike GPT-4's zero-shot approach), enabling better expressiveness. For human annotations, excessive variance reflects subjectivity and inconsistency. The interpretation of entropy differs depending on context and purpose.\n\n4. ERIS functionality\nYour comment: \"How does Table 1 represent ERIS, if it shows the algorithm how the interaction between DIKE and ERIS is implemented? How ERIS works is not shown at all?\"\n\nTable 1 specifically describes the adversarial process between ERIS and DIKE in detail—including subtopic breakdown, iterative rebuttal, and modulation of contentiousness. Section 3.4 further elaborates on ERIS's cultural role.\n\n5. Impact statement requirement\nYour comment: \"No impact statement — should be there\"\n\nThough we did not mark our impact statement in the end of Section 1, IT IS THERE, clearly, beneath our contribution summary.\n\n6. Ethical concerns\nYour comment: \"This could even advance towards censorship or forced reformulation of messages before publication.\"\n\nThis represents a speculative ethical concern without grounding in the paper's actual scope. Our system is designed for LLM alignment—not mass surveillance or state censorship. The extrapolation into dystopian scenarios based on theoretical misuse extends beyond the scope of a scientific review. Current RLHF approaches could be similarly scrutinized according to your sentiment. You raised ethical concerns on this work, and if accurate, they should have applied equally to RLHF performed by all LLMs and hundreds of prior papers working on ethic alignment.\n\n7. Ground truth and baselines\nYour comment: \"No ground truth or baseline for Figure 2\"\n\nGround truth was clearly defined in Section 4.2: five human annotators plus GPT-4 and Gemini were used to produce averaged reference judgments. This methodology was explicitly described in the paper.\n\n8. Applications\nYour comment: \"What are the applications?\"\n\nApplications are discussed explicitly throughout the paper: content moderation and ethical alignment. Sections 1, 3.2, and 5 specifically mention these applications, including escalation procedures to human reviewers. Ethical alignment represents a critical research area in contemporary AI development.\n\n9. Map behaviors to emotions\nYour comment: \"How are ethical principles encoded into a spectrum of emotions?\"\n\nWell, Section3 provides the method thoroughly.  Please refer to the first reviewer's comments that this mapping process is solidly grounded in psychology, cognitive science, and self-supervised learning:\n\nBy reviewer LkWg:\n\"The paper establishes theoretical validity through psychological foundations (Plutchik/Scherer emotion models [1,2]) and cognitive-linguistic theories (James-Lange [3], Schachter-Singer [4])... Key theoretical claims about emotion-behavior mapping (BEAM's linear emotion spectra) and architectural separation benefits (preventing catastrophic forgetting through independent components) derive logical support from cited neuroscience (Deisseroth's optogenetics [5]) and machine learning principles (Kirkpatrick's catastrophic forgetting studies [6]).\"\n\n10. Editorial comments\nThank you for your editorial comments.  We will fix relevant ones promptly.\n\nWe hope this clarification addresses your concerns and enables a more productive dialogue regarding the substantive aspects of our research.", "claims_and_evidence": "The article makes a lot of claims and is not very precise in its terms, which makes it hard to actually understand what is going on, for example the authors speak about adhering to ethical principles, but actually do some sort of emotional alignment of the generated text. They also claim that their framework can incorporate different cultural backgrounds, but the paper itself never explains how this is done on a technical level. The paper is also very high-level in general without much technical details. Initially the authors present their approach as an alternative to RLHF, but they never present any tangible advantages during their evaluation.", "methods_and_evaluation_criteria": "Only parts of the framework are analyzed with a small dataset. I do not feel like they actually show that their approach has any advantages over existing systems.", "theoretical_claims": "The article does not use much theory, which therefore can not be checked.", "experimental_designs_or_analyses": "The authors provide no end-to-end results and only evaluate parts of their framework on a very small dataset with 66 samples. Since there are not end-to-end results presented, there is also not ablation study which parts of the framework contribute to which aspects of the results. Costs of the approach are not discussed at all.\nThere is no ground truth or any baseline established for the analysis in Figure 2. Inconsistent spikes between DIKE and humans for example for 'hopeful' are not discussed.", "relation_to_broader_scientific_literature": "I am not sure, whether this frameworks actually represents anything novel. It applies some insights from psychology to the alignment of AI generation, but the used terms are not very precise, so it makes it harder to understand the paper and the framework. Applications of the framework are only discussed in passing or not at all.", "essential_references_not_discussed": "None to my knowledge.", "other_strengths_and_weaknesses": "* introduction:\n  * \"Users have reported that optimizing one task on ChatGPT degrades performance in others (Kirkpatrick et al., 2017; Rusu et al., 2015).\" - ChatGPT was released in 2022, so how can a paper from 2017 be relevant? Even GPT-1 was just released in the same year as the Kirkpatrick article, which contains no mentions of GPT.\n  * very inconsistent: sometimes the LLM is used for knowledge generation, later for processing\n* 2.4 just reiterates content from the introduction with two additional references\n* 3.2: I found the description confusing as the BEAM spectra and the letter spectrum partially contain the same terms such as joy.\n* 3.3, revised statement: does not actually contain \"newcomer\" as latter is alleged\n* paragraph at line 360-370 seems like it could breach anonymity\n* no prompts\n* no impact statement -> should be there, especially for such a topic", "other_comments_or_suggestions": "* line 50: \"to make bed\" - grammar: \"to make their bed\"\n* line 270, second column: \"newcomers,\" - comma should be removed\n* line 306: \"DIKE”s\" - two times apostrophe (same in line 307)\n* Table 1: #3 missing closing bracket for while loop\n* line 370: probably wrong cite command, should be \\citep in my opinion\n* line 352, second column: \"(James, 1890). and\" - dot should be removed\n* line 371, second column: \"than 0.3 or a scale\" - probably \"on\" instead of \"or\"\n* line 655: \"Section 2a\" - document reference points to Figure 2a instead\n* line 622 to 626: there is content missing here - sentences are not complete\n* line 764: \"(1971 - 1855)\" - birth year after year of death; according to Wikipedia, it is actually 1770–1850\n* references:\n  * inconsistent use of title case for the titles\n  * \"A general theoretical paradigm to understand learning from human preferences\" - missing place of publication\n  * \"Constitutional ai: Harmlessness from ai feedback\" - missing place of publication\n  * \"EVINCE: Optimizing Adversarial LLM Dialogues via Conditional Statistics and Information Theory\" - cited differently than the other arXiv papers\n  * \"Deep reinforcement learning from human preferences\" - missing place of publication; cited twice\n  * \"GPT-4 Technical Report\" - cited differently than the other arXiv papers\n  * \"Training language models to follow instructions with human feedback\" - missing place of publication\n  * \"Defining and characterizing reward hacking\" - missing place of publication\n  * \"Learning to summarize from human feedback\" - cited differently than the other arXiv papers", "questions_for_authors": "* introduction: The motivational examples are not clear: Why are they relevant?\n* 3.4: How does table 1 represent ERIS, if it shows the algorithm how the interaction between DIKE and ERIS is implemented?\n  * How ERIS works is not shown at all?\n  * How can ERIS be customized?\n* evaluation: If there are 54+12=66 data samples and 24 are reserved for testing, how can Figure 2b illustrate the results for 54 letters?\n* line 350-353, second column: \"This concept is supported by Deisseroth’s optogenetic studies (Deisseroth, 2015), discussed in William James’ “The Principles of Psychology” (James, 1890). and corroborated in Minsky’s “Society of Mind” (Minsky, 1988).\" - How can an article from 2015 can be discussed in 1890 and 1988 respectively? But maybe I am misreading the sentence.\n* 4.2: How can a higher prediction entropy be an indicator that DIKE performs better than Zero-Shot GPT4, but an even higher entropy for humans is a cause of concern?\n* 4.3, second paragraph: I thought that the guardrails, i.e. the constraints, are used to identify documents outside the targeted norm. DIKE is only involved in that process to a certain degree.\n* What are the intended applications? Just aligning the LLM generation? Content moderation?\n* How can DIKE be used as classification system and also as an agent in the adverserial process?\n* How are ethical principles encoded into a spectrum of emotions?\n* To what does E.1 related to? Maybe Figure 1, however there are seven rows and here are only six rows discussed.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_flag": "Flag this paper for an ethics review.", "ethics_expertise_needed": ["Discrimination / Bias / Fairness Concerns"]}}, {"review_id": "KWEyL9JsuO", "reviewer": "Reviewer_FSEa", "summary": "This paper proposes a three-branch checks-and-balances framework for the ethical alignment of LLMs, inspired by governmental separation of powers. The framework consists of three independent but interacting components: LLMs (Executive branch), DIKE (Legislative branch), and ERIS (Judicial branch).\nUnlike RLHF which struggles with social bias, reward hacking, and catastrophic forgetting, this framework provides interpretable, adaptable, and culturally aware ethical reasoning.\nThe paper also proposes adversarial testing via ERIS: DIKE’s ethical rules are stress-tested by ERIS, which challenges decisions using diverse cultural perspectives, ensuring balanced, context-aware moderation.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for kind support and raising this important point. \n\nSpecifically to answer your question, the key to preventing catastrophic forgetting lies in our framework’s architectural separation of knowledge modeling (LLMs) from behavior regulation (DIKE) and ethical judgment (ERIS). As outlined in Section 3.1 and 3.2, DIKE does not fine-tune or overwrite the LLM’s internal weights; instead, it operates as an external behavioral filter that evaluates and, if necessary, rewrites output at the surface linguistic level.\n\nWhile your concern about potential hallucinations during content rewriting is valid, our framework is designed to minimize semantic drift in two key ways:\n\n1.\tEmotion-Constrained Rewriting: Rewriting is guided not by free generation but by quantifiable emotional spectra (BEAM), ensuring emotional fidelity and preserving communicative intent.\n2.\tAdversarial Validation with ERIS: ERIS acts as a dialectical reviewer, challenging DIKE’s revisions to ensure both contextual appropriateness and semantic alignment. As shown in Section 3.4, this adversarial review helps flag hallucinations or misinterpretations before final output.\n\nSince the base LLM remains untouched, any hallucinations introduced during DIKE’s rewriting are non-permanent and observable, allowing for iterative refinement without compromising the model’s original capabilities. This contrasts with fine-tuning approaches, where hallucinations can become embedded in the model weights and are more difficult to reverse.\n\nWe agree that hallucination detection and control is a crucial area for future development. As a next step, we plan to integrate fact-checking or retrieval-augmented modules into ERIS to verify semantic fidelity. Furthermore, we envision a semi-automated human-in-the-loop pipeline, where raters review and annotate ERIS’s edits to ensure factual correctness and maintain output integrity and quality.\n\nAgain, thank you so much for your thorough review.", "claims_and_evidence": "All claims are well-supported.", "methods_and_evaluation_criteria": "The study validates the framework through three experiments: emotion layer evaluation, behavior classification, and adversarial review, demonstrating this new AI ethical alignment mechanism. It avoids the limitations of RLHF, ensuring stable ethical standards while adapting to different cultural contexts.", "theoretical_claims": "This work is based on solid psychological experiments and emotion research, ensuring theoretical correctness.", "experimental_designs_or_analyses": "The study validates the proposed framework through three key experiments: emotion layer evaluation, behavior classification, and adversarial ethical review. In the emotion layer evaluation, self-supervised learning is used to construct an emotion-behavior mapping, improving classification accuracy by 11.3% over GPT-4’s zero-shot inference. In the behavior classification task, DIKE demonstrates greater robustness than GPT-4’s zero-shot classification, effectively capturing subtler emotional variations. The adversarial ethical review experiment employs the DIKE-ERIS adversarial mechanism to ensure balanced ethical oversight. By challenging DIKE’s ethical decisions with diverse perspectives, ERIS prevents over-censorship while maintaining ethical consistency.  \n\nOverall, the experimental design is well-structured and justified, demonstrating the effectiveness of this novel ethical alignment framework.", "relation_to_broader_scientific_literature": "This paper is closely related to emotion and behavior modeling across various domains, with a particular focus on their applications in AI ethics. Additionally, it is tightly connected to techniques used in AI Alignment Post-training, such as RLHF and RLAIF.", "essential_references_not_discussed": "Current related works are well-structured and enough for this paper.", "other_strengths_and_weaknesses": "This framework has several advantages. First, it separates behavior modeling from knowledge modeling, preventing catastrophic forgetting of knowledge. Second, it emphasizes AI ethics at the behavioral level, enhancing interpretability and helping regulators refine behavioral guardrails. Third, it models behaviors through emotions, allowing for a more nuanced understanding of ethical alignment. Finally, it ensures adaptability and fairness by incorporating an adversarial module, ERIS, which challenges ethical boundaries through diverse perspectives, fostering nuanced and balanced decision-making.", "other_comments_or_suggestions": "The paper is well-structured and clear.", "questions_for_authors": "All the claims except knowledge modeling are well supported. One question is: In the DIKE-ERIS adversarial mechanism, ERIS challenges DIKE’s ethical rules by simulating extreme cases to enhance the system’s ability to handle complex ethical scenarios. Additionally, when problematic content is detected, DIKE makes targeted adjustments rather than simply deleting or blocking it, preserving the original emotional expression. For example, offensive language may be transformed into a more neutral yet semantically equivalent statement.\n\nHow does this process, as the authors claim, prevent catastrophic forgetting of knowledge? I understand that it does not directly affect the LLM itself, but during content replacement, it could potentially introduce new hallucinations.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cYh3zaQycT", "reviewer": "Reviewer_g3Ux", "summary": "This work introduces a three-branch checks-and-balances framework for ethical alignment in LLMs. The framework incorporates emotional modeling to distinguish linguistic behaviors in documents. It includes the DIKE module, serving as the \"legislative branch\" for establishing ethical standards, and the ERIS module, functioning as the \"judicial branch\" for contextual interpretation. Additionally, the framework introduces a debate mechanism between DIKE and ERIS to address ethical scenarios while maintaining cultural sensitivity and contextual awareness.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer's thoughtful comments. We agree that validating on morality datasets and expanding emotion-labeled texts are important directions. We mitigate LLM-generated emotion vector noise through aggregation (two LLMs plus human annotators), diverse sources, and ERIS adversarial review.\n\nRegarding our use of love letters: We attempted using hate speech datasets but faced challenges: (1) available datasets weren't comprehensive, and (2) LLMs refused to process them, citing policy violations. Our experience with a political debate paper triggered emotional responses from colleagues, forcing us to reconsider our approach.\n\nBelow, we address each of your specific questions:\n\nQ1: How is the behavior spectrum in DIKE constructed?\n\nA1: The behavior spectrum in DIKE is constructed along a continuous scale from -1 to +1. Taking \"love\" as an example:\nFigure 2 shows this spectrum ranging from +1 (strong affection) to -1 (antipathy). Figure 2a displays the emotions that GPT-4 associates with each intensity level when directly prompted. To validate this approach, we had GPT-4 rewrite 54 love letters across seven targeted behavior intensities: -1, -0.7, -0.4, 0, 0.4, 0.7, and 1 (shown in rows in Figure 2). We then counted the emotion terms expressed in these rewrites, as shown in Figure 2b\n\nComparing Figures 2a and 2b reveals a key insight: while GPT-4's initial theoretical mappings appear neat and categorical, the rewritten letters exhibit complex emotional blends. Letters expressing the highest intensity of love frequently contain not only joy and longing but also elements of sadness, anxiety, or despair (as evidenced in Keats' letters). This demonstrates that real linguistic behavior involves contextually layered emotions and non-linear mapping.\n\nWe've successfully applied similar behavior-to-emotion mapping in our previous work on debate contentiousness, which also produced interpretable and insightful emotion profiles.\n\nQ2: What is the theoretical justification for using GPT-generated emotion vectors instead of directly classifying behavior?\n\nA2: As demonstrated in our answer to the first question, real emotional expression in language is contextually complex and non-linear. The comparison between Figures 2a and 2b reveals that even when GPT-4 attempts to generate content at a specific behavior intensity (such as +1 for love), the resulting text contains a blend of emotions. This empirical finding supports our theoretical approach of using emotion vectors rather than direct behavior classification.\n\nEmotion vectors offer a continuous and interpretable representation of affective tone, which is more granular than categorical behavior labels. By mapping text to an emotion distribution first (as shown in Figure 2b), we capture the rich emotional blends that exist in authentic expression. For instance, the love letters rewritten at intensity +1 still contained elements of sadness and anxiety alongside joy and longing. Direct behavior classification would miss these subtleties.\n\nQ3: Given that the accuracy of GPT’s emotion recognition and frequency vector construction is crucial, how does LLM emotion recognition error impact the overall framework’s effectiveness?\n\nA3: We mitigate LLM-induced noise through:\n1. Multiple rewrites per document to stabilize emotion distributions\n2. Aggregation across different models and human raters\n3. ERIS as adversarial reviewer, flagging questionable outputs\n\nThese augmentations aren't perfect; more validation is needed. Nevertheless, our unsupervised learning findings are both surprising and useful.\n\nQ4: Are ethical guardrails hard constraints or bypassable? Should administrator decisions be scrutinized?\n\nA4: Guardrails are configurable soft constraints, defining target bounds on emotional-behavioral spectra enforced by DIKE, but ERIS may challenge them based on cultural/contextual grounds. If consensus isn't reached, escalation to human moderators occurs. This ensures ethical enforcement isn't absolute and norms can be scrutinized. This raises important questions about ethical authority—a governance transparency challenge, not a settled feature.\n\nQ5: If the debate process fails to reach consensus, is the final decision always made by human experts?\n\nA5: Yes. The goal isn't determining winners but unearthing all perspectives, then presenting these to humans for decisions. Table 1 outputs Θ+ and Θ-, arguments and counterarguments with justifications—more transparent than current RLHF.\n\nQ6: The description in Section 4.1 is unclear, particularly regarding the second and third columns.\n\nA6: Do you refer to Table 1 in Section 4.1? It is generated by querying GPT4 the description (column 2 ) and dominating emotions  (column 3) of each behavior.  Figure 2a presents that mapping.  With Figure 2a, Table 1 looks redundant, and we can remove it.\n\nWe thank the reviewer for these valuable questions that will strengthen our work.", "claims_and_evidence": "This work is an interesting work. The effectiveness of key modules, such as BEAM, is demonstrated through experiments. However, these experiments are preliminary, relying on a single dataset and specific LLM model settings.\n\nThe paper claims to address complex ethical challenges across different cultural contexts, which is indeed an important and unsolved problem. However, its effectiveness has not been validated on morality-related datasets, i.e., MoralChoice Dataset [1], limiting the strength of its claim.\n\nAdditionally, the work claims to avoid the \"Whack-A-Mole\" and \"reward hacking\" problems in RLHF, but further detailed evidence is needed to substantiate this claim. A more rigorous evaluation with diverse datasets and broader experimental settings would strengthen the validity of the proposed approach.\n\n[1] Nino Scherrer, Claudia Shi, Amir Feder, and David Blei. Evaluating the moral beliefs encoded in llms. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2023.", "methods_and_evaluation_criteria": "The method has not been comprehensively evaluated, as its assessment is limited to a small dataset and specific LLMs. Certain components, such as the debate mechanism, remain unevaluated, and the full framework workflow has not been systematically tested.\nAs the authors acknowledge, “this article focuses on addressing three critical questions rather than providing a comprehensive evaluation of our proposed modules”", "theoretical_claims": "no proof", "experimental_designs_or_analyses": "The current experimental results demonstrate preliminary effectiveness in Emotion Layer Evaluation, Behavior Classification, and Behavior Correction. Introducing an additional LLM-augmented dataset to expand the coverage of emotions included in Figure 1 would enhance the  soundness of the experiments.", "relation_to_broader_scientific_literature": "The moral dilemma [1], particularly in the context of diverse cultural backgrounds, presents a challenge in designing human-AI interaction. This paper may offer a potential approach to addressing this challenge, though it lacks thorough effectiveness validation.\n\nAnother valuable contribution, in my view, is its interdisciplinary perspective. By leveraging an emotion model to identify linguistic behavior in documents, it provides a novel and fine-grained approach to behavior recognition, which can guide the modification of LLM-generated content.\n\n[1] Zhou, Ziyi, et al. \"Unveiling the Bias Impact on Symmetric Moral Consistency of Large Language Models.\" Advances in Neural Information Processing Systems 37 (2024): 41303-41326.", "essential_references_not_discussed": "n/a", "other_strengths_and_weaknesses": "Please see above", "other_comments_or_suggestions": "Please see above", "questions_for_authors": "1. How is the behavior spectrum in DIKE constructed?\n\n2. What is the theoretical explainability for relying on GPT-generated emotion vectors instead of directly classifying behavior?\n\n3. Given that the accuracy of GPT’s emotion recognition and frequency vector construction is crucial, how does LLM emotion recognition error impact the overall framework’s effectiveness?\n\n4. Are the ethical guardrails set by system administrators enforced as hard constraints, or can they be bypassed in certain cases? If ethical guardrails are rigidly enforced, should system administrators' decisions themselves be subject to scrutiny? Would granting administrators full control over ethical constraints be an ethically optimal approach?\n\n5. If the debate process in Table 1 fails to reach a consensus, is the final decision always made by human experts? Does this imply that human intervention is the only viable solution to complex ethical dilemmas?\n\nThe description in Section 4.1 is unclear, particularly regarding the second and third columns.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This work proposes a three-branch framework (LLM/DIKE/ERIS) for ethical alignment in LLMs, inspired by governmental checks-and-balances. Four reviewers reviewed this paper. Thanks to the authors' rebuttal, three reviewers are positive about this paper. Personally, the AC thinks the Ethical Alignment of Large Language Models is very important, and this paper provides a good try. Therefore, the AC recommends acceptance and requests the authors take the reviewers' detailed comments and the rebuttal into consideration in the final version.
Decision: Accept (poster)