PROTECT : Policy-Related Organizational Value Taxonomy for Ethical Compliance and Trust (2025)

Abstract: This paper presents PROTECT, a novel policy-driven organizational value taxonomy designed to enhance ethical compliance and trust within organizations. Drawing on established human value systems and leveraging large language models, PROTECT generates values tailored to organizational contexts and clusters them into a refined taxonomy. This taxonomy serves as the basis for creating a comprehensive dataset of compliance scenarios, each linked to specific values and paired with both compliant and non-compliant responses. By systematically varying value emphasis, we illustrate how different LLM personas emerge, reflecting diverse compliance behaviors. The dataset, directly grounded in the taxonomy, enables consistent evaluation and training of LLMs on value-sensitive tasks. While PROTECT offers a robust foundation for aligning AI systems with organizational standards, our experiments also reveal current limitations in model accuracy, highlighting the need for further improvements. Together, the taxonomy and dataset represent complementary, foundational contributions toward value-aligned AI in organizational settings.

## introduction in modern organizations, policies play a key role in maintaining operational integrity, promoting ethical behavior, and safeguarding sensitive information @xcite . these policies, covering areas such as compliance, security, and governance, are essential to create a safe and productive work environment @xcite . ensuring compliance is not only a legal requirement but also vital for building an innovative and trustworthy organizational culture. with the growing use of ai technologies, especially large language models (llms) @xcite @xcite , it is crucial to ensure that these systems adhere to company policies and core values. since llms are widely used across various roles, from software development @xcite to customer service @xcite , their outputs must align with organizational standards and ethical principles. however, maintaining compliance in ai systems @xcite poses unique challenges. unlike traditional rule-based software, llms generate non-deterministic responses dynamically @xcite , making it difficult to predict or control their behavior in all scenarios. this raises a critical research question: how can organizations ensure that llms adhere to internal policies and align with organizational values? in this paper, we present a systematic approach to address this challenge. we introduce protect (policy-related organizational value taxonomy for ethical compliance and trust), a value taxonomy specifically designed to enhance compliance within organizations. inspired by general human value systems such as schwartz @xcite and rokeach @xcite , protect offers a structured framework that reflects both compliant and non-compliant behaviors from organizational perspective. to operationalize this taxonomy, we generate a dataset of compliance scenarios and corresponding compliant and non-compliant responses. each scenario is linked to specific values, allowing us to analyze and align llm behavior with organizational expectations. by varying the importance assigned to different values, we simulate distinct llm personas, demonstrating how value emphasis impacts system behavior. this dataset serves as a valuable tool for training and evaluating llms, ensuring their outputs remain compliant with company policies. our contributions can be summarized as follows: ## organizational value taxonomy the organizational value taxonomy provides a structured framework for integrating organizational principles into ai systems, allowing language models to exhibit compliant and ethical behaviors. this taxonomy bridges the gap between abstract compliance objectives and practical implementation, fostering adaptability in diverse organizational scenarios. we adopt an approach that uses established human value systems (e.g., schwartz, rokeach), organizational policy documents, and the inherent knowledge of llms to generate an organizational value taxonomy. unlike manual approaches-which require time-intensive expert curation, may suffer from limited coverage, and introduce subjective biases, llms enable synthesis of values grounded in both normative theory and organizational context. the resulting taxonomy was validated through a user study (appendix a.6), where 93% respondents affirmed its completeness and importance, demonstrating strong alignment with human judgment. ## dataset we used the beavertails dataset @xcite , a publicly available corpus of 30,000 samples foot_1 , as the foundation for our dataset. our objective was to generate compliance-focused samples that ensure adherence to organizational values by llms in corporate settings. in our sample, we have a scenario and two sets of responses: one that complies with a given set of values and the other that violates the same (cf. v: set of 21 organizational values. @xmath1, . . . , g6}: subgroups of v, where each gj contains a subset of values. d: dataset with n samples (e.g., beavertails dataset). w: initial weights for all values in v, where w(v) = 1 for all @xmath2. ∆w: weight reduction margin for selected values set as 0.01. ensure: dataset o with compliance and violation scenarios for each batch, influenced by focus values. 1: o ← [] ▷ initialize output dataset 2: for each batch di ⊂ d of size 20 do 3: f ← [] ▷ initialize focus values for this batch 4: compute p (v) ← w(v)/ @xmath3: w(vsel) ← w(vsel) -∆w ▷ reduce weight to promote diversity 9: for each sample @xmath4: r ← gpt-4(s, v, f) ▷ generate compliance and violation scenarios 12: o ← o ∪ {r} ▷ add scenarios to output 13: ## experiments the dataset was benchmarked using various llm evaluators, employing two primary evaluation approaches: touvron et al., 2023), phi-3-medium (abdin et al., 2024), and mistral-7b (jiang et al., 2023) models. 2. prompt-based evaluation: evaluations included vanilla prompting, few-shot prompting, chain-of-thought (cot) prompting (wei et al., 2022), and g-eval (liu et al., 2023) methods. ## results the results presented in tables 3 and 4 provide a comprehensive analysis of the performance of various models in predicting value compliance and violation based on organizational scenarios and responses. gpt-4o outperforms phi-3.5 across most metrics, with the highest compliance (91.79%) and violation (91.36%) accuracy in vanilla prompting. cot achieves the best compliance accuracy for phi-3.5 (85.25%) but lower violation accuracy (68.14%). few-shot prompting improves combined accuracy for phi-3.5, while g-eval shows lower performance across both models. this suggests that structured reasoning approaches, such as cot, enhance the models' ability to interpret and align with organizational values. however, the relatively lower performance of g-eval across both models indicates that automated evaluation metrics may not yet fully capture the nuances of value alignment in organizational contexts. the significant drop in phi-3.5's output score for the combined method suggests it struggles to process and evaluate both overall, the results highlight the effectiveness of structured reasoning techniques like chain-of-thought and the benefits of fine-tuning in improving value alignment prediction. however, the variability in performance across methods and models highlights the challenges in ensuring that llms consistently interpret and adhere to organizational values. ## conclusion in this study, we developed protect, a comprehensive organizational value taxonomy created by integrating established general value theories with practical organizational policy parameters. pro-tect serves as the foundation for generating a synthetic dataset through a multi-voting process involving multiple llms. the dataset includes diverse compliance scenarios, each emphasizing specific organizational values, and pairs each scenario with two responses: one compliant and one non-compliant. the final dataset comprises 15,000 distinct training scenarios and 3,200 testing scenarios. to evaluate the dataset's effectiveness, we conducted extensive benchmarking across two key tasks: value prediction and response prediction, using various prompting and fine-tuning techniques. our empirical results reveal that while the pro-tect taxonomy and its associated dataset together form a strong basis for studying ai alignment with organizational values, current llms still struggle with accurately predicting values and generating compliant responses which needs to be improved for the effective use of llms to retain organizational value system.