Title: Utilizing Human Behavior Modeling to Manipulate Explanations in AI-Assisted Decision Making: The Good, the Bad, and the Scary
Venue: NeurIPS 2024
Accepted: yes
Keywords: human-AI interaction

human-centered evaluation of explanation
Abstract: Recent advances in AI models have increased the integration of AI-based decision aids into the human decision making process. To fully unlock the potential of AI-assisted decision making, researchers have computationally modeled how humans incorporate AI recommendations into their final decisions, and utilized these models to improve human-AI team performance. Meanwhile, due to the ``black-box'' nature of AI models, providing AI explanations to human decision makers to help them rely on AI recommendations more appropriately has become a common practice.  In this paper, we explore whether we can quantitatively model how humans integrate both AI recommendations and explanations into their decision process, and whether this quantitative understanding of human behavior from the learned model can be utilized to manipulate AI explanations, thereby nudging individuals towards making targeted decisions. Our extensive human experiments across various tasks demonstrate that  human behavior can be easily influenced by these manipulated explanations towards targeted outcomes, regardless of the intent being adversarial or benign. Furthermore, individuals often fail to detect any anomalies in these  explanations, despite their decisions being affected by them.

Utilizing Human Behavior Modeling to Manipulate Explanations in AI-Assisted Decision Making: The Good, the Bad, and the Scary

Abstract

Recent advances in AI models have increased the integration of AI-based decision aids into the human decision making process. To fully unlock the potential of AIassisted decision making, researchers have computationally modeled how humans incorporate AI recommendations into their final decisions, and utilized these models to improve human-AI team performance. Meanwhile, due to the 'black-box' nature of AI models, providing AI explanations to human decision makers to help them rely on AI recommendations more appropriately has become a common practice. In this paper, we explore whether we can quantitatively model how humans integrate both AI recommendations and explanations into their decision process, and whether this quantitative understanding of human behavior from the learned model can be utilized to manipulate AI explanations, thereby nudging individuals towards making targeted decisions. Our extensive human experiments across various tasks demonstrate that human behavior can be easily influenced by these manipulated explanations towards targeted outcomes, regardless of the intent being adversarial or benign. Furthermore, individuals often fail to detect any anomalies in these explanations, despite their decisions being affected by them.

1 Introduction

Recent advances in AI models have significantly increased the integration of AI-based decision aids into human decision making process. The widespread adoption of such AI-based decision aids has opened up a new paradigm of human-AI collaboration-the AI model provides recommendations for a given decision making task, while human decision makers are responsible for making the final decisions. To fully unlock the potential of AI-based decision aids in enhancing human decision making, a few studies [1-5] have developed computational models to capture how humans factor AI recommendations into their decision-making process, and explored how these behavioral models can be utilized to improve human-AI team performance. For example, Vodrahalli, Gerstenberg, and Zou [6] developed a human behavior model to characterize the impact of AI predictions and confidence levels on human final decisions. This model was then utilized to adjust the model confidence displayed to people, with the objective of calibrating human trust in AI assistance.

Meanwhile, the black-box nature of prevalent AI models has driven a greater integration of model explanations, generated through various explainable AI (XAI) methods [7-11], into AI-assisted decision making. These explanations seek to provide some insights into the the underlying decision rationales of AI models, assisting humans in evaluating the reliability of AI decisions and identifying the optimal strategies to rely on AI recommendations. However, many empirical studies [12-18], which evaluate the effectiveness of current XAI methods for improving people's understanding

of the AI model [16, 19, 20] and supporting their calibrated trust in the model [13, 14, 20], have demonstrated that humans often struggle to use the explanations generated by these methods in the optimal way. Thus, despite designers' expectations that XAI methods will positively shape human interaction with AI models, they often fall short of their intended goals, such as fostering appropriate levels of trust and reliance in AI-assisted decision-making. This could be because existing XAI methods do not account for human reactions, making them unadaptive to human cognitive processes. If this is the case, one may naturally wonder if it is feasible to quantitatively model how humans incorporate both AI recommendations and explanations into their decision making process. If so, can the quantitative understanding of human behavior obtained from the learned behavior models be utilized to directly manipulate AI explanations, thereby nudging human decision-makers towards making targeted decisions?

To answer these questions, in this paper, we begin by training behavior models that characterize how AI recommendations and explanations are factored into human decisions based on the collected behavior datasets for various decision making tasks. Utilizing these learned models, we then adjust the AI explanations for different purposes through gradient-based optimizations. Our extensive human experiments across various tasks demonstrated that such human behavior modeling can bring forth both benefits and risks -when used for benign purposes, the human behavior models can inform the adjustment of AI explanations such that the manipulated explanations could significantly enhance the decision-making performance of human-AI teams in most tasks; however, the same behavior models can also be exploited by adversarial parties for adversarial purposes, such as increasing human decision-makers' biases against a certain protected group and significantly decreasing the fairness level of humans' final decisions across different groups. Finally, examining human perceptions of these manipulated explanations reveals the ' scary ' truth - while human decisions are easily swayed by these altered explanations, individuals generally fail to detect any anomalies in the explanations, underscoring a significant vulnerability in human-AI interaction.

2 Related Work

2.1 Computational Modeling of AI-assisted Decision Making

There has been a surge of interests among researchers recently in computationally modeling human behavior in AI-assisted decision making [1-5, 21-24]. The goals of these studies for modeling human behavior are diverse, encompassing improving human-AI team performance through intelligent interventions or model recommendation adjustments [6, 25-29], deciding when to present AI powered code suggestion in programming [30], evaluating the utility of AI explanations to improve user understanding of AI model behavior [31-33], and deploying adversarial attacks on AI models to reduce human trust [34]. In this paper, we take a more holistic view and explore whether we can model how humans integrate both AI recommendations and explanations into their decision making process, and what the implications of such behavior modeling are.

2.2 Human-centered Evaluation of AI Explanations

With the increasing use of AI technologies as decision aids for humans, a variety of explainable AI techniques have been developed to increase the interpretability of AI models [7-11, 33, 35-37]. To understand the effectiveness of these explanation methods, a growing body of empirical humancentered evaluations have been conducted to examine how AI explanations would affect the ways that humans perceive and interact with AI models [12-18, 31, 32, 38, 39]. These evaluations look into various aspects of impacts of AI explanations, such as the influence on people's trust and reliance on the AI model [13, 14, 20], understanding of AI model [16, 19, 20], and the collaboration performance of the human-AI team [12, 40, 41]. Recently, some research has explored the modification of AI explanations to influence human behavior. For example, Lakkaraju and Bastani [42] demonstrated that handcrafting modifications in AI explanations-such as hiding sensitive features-can mislead human trust in AI models. Another study [43] found that aligning AI explanations with humans' own decision rationales can increase agreement between human decisions and the AI model's predictions. Different from previous work which required intensive handcrafting of AI explanations, in this paper, we explore whether it is possible to directly exploit the computational human behavior models to manipulate AI explanations, even without the access to AI models, with the goal of nudging human decisions towards targeted directions.

3 Methodology

3.1 Problem Formulation

In this study, we explore the scenario of human-AI collaboration within the context of AI-assisted decision making, and we now formally describe it. Consider a decision making task represented by a n -dimension feature vector x ∈ R n , and y is the correct decision to make in this task. Specifically, in this study, we focus on decision making tasks with binary choices of decisions, i.e., y ∈ {-1 , 1 } . The AI model's recommendation on the decision task is represented as y m = M ( x ) , y m ∈ {-1 , 1 } . Following the explainable AI methods like LIME[7] or SHAP[8], the AI model could also provide some 'explanations' of its decision, e = E ( M ( x )) , e ∈ R n , by showing the contributions of each feature to the decision. With all these information, the human decision maker (DM) needs to make the final decision y h ∈ {-1 , 1 } by either accepting or rejecting AI model's decision recommendation y m , which can be characterized by y h = H ( x , y m , e ) . The goal of our study is to explore whether we can quantitatively model such decision making process-specifically, H ( x , y m , e ) -and whether this quantitative understanding of human behavior can be utilized to adjust AI explanations (i.e., change e to e ′ ) without accessing to the original AI model M ( · ) , thereby nudging human DMs to make the targeted decision ˆ y h ∈ {-1 , 1 } , denoted as ˆ y h = H ( x , y m , e ′ ) .

3.2 Modeling Human Behavior in AI-assisted Decision Making

We first build computational models to characterize how humans integrate both AI recommendations and explanations into their decision process. Following previous works on modeling human behavior in different scenarios of AI-assisted decision making [23, 31], we adopted a two-layer neural network as the structure for modeling the human decision in this study:

The inputs to the behavior model include the task features x , the AI model's prediction y m , the AI explanation e , and the interaction term between the task features and the AI explanation x ⊙ e that reflects how humans may redirect their attention to the corresponding features highlighted by the AI explanation. Given the human behavior dataset D = { x i , y i m , e i , y h i } N i =1 , we can employ the maximum log-likelihood estimation to learn the behavior model H w h .

3.3 Manipulating AI Explanations through the Behavior Model

We next proceed to explore how the quantitative understanding of human behavior from H w h can be utilized to manipulate AI explanations. In particular, given the targeted decision ˆ y h for the task instance x , we want to identify a new AI explanation e ′ that maximizes the likelihood that human DMs make the targeted decision ˆ y h according to the learned behavior model H w h . In addition, to prevent the case where the manipulated explanations e ′ has a very low level of fidelity [10], such as suggesting a recommendation that is inconsistent with the AI model's prediction y m , we also impose a constraint that the new explanation e ′ should still support the original AI recommendation y m . Since we assume no access to the original AI model, we define L consistency ( e , y m ) as a measurement of agreement consistency between the manipulated AI explanations and the AI recommendation:

Together, we use the following optimization problem to manipulate AI explanations:

where L behavior is defined as the cross entropy function. Since exactly solving the above optimization problem is intractable, we used the gradient-based optimization to approximate it:

where η is the step size, λ is the trade-off parameter, and e ′ θ represents the parameterized explanations in the optimization process. We can iteratively optimize manipulated explanations e ′ until L behavior is smaller than a threshold τ or reach the maximum number of rounds T .

4 Human Behavior Model Learning

To develop the human behavior model for manipulating AI explanations, we first conduct a human subject experiment to collect human behavior data.

4.1 Decision Making Task and AI Assistance

We consider four decision making tasks in this study:

Census Prediction (Tabular Data) [44]: This task was to determine a person's annual income level. In each task, the human DM was presented with a profile with 7 features, including the person's gender, age, education level, martial status, occupation, work type, and working hour per week. The subject was asked to decide whether this person's annual income is higher or lower than $50k for each task. We trained a random forest model to make the income prediction, and the accuracy of the AI model was 76% .

Recidivism Prediction (Tabular Data) [45]: This task was to determine a person's recidivism risk. In each task, the human DM was presented with a profile with 8 features, including their basic demographics (e.g., gender, age, race), criminal history (e.g., the count of prior non-juvenile crimes, juvenile misdemeanor crimes, juvenile felony crimes committed), and information related to their current charge (e.g., charge issue, charge degree). The subject was asked to decide whether this person would reoffend within two years. We trained a random forest model to make the prediction, and the accuracy of the AI model was 62% .

Bias Detection (Text Data) [46]: In this task, the human DM was presented with a text snippet and needed to decide whether it contained any bias. We fine-tuned a BERT [47] model to identify bias in the snippet, and the accuracy of the AI model is 79% .

Toxicity Detection (Text Data) [48]: In this task, the human DM was presented with a text snippet and needed to decide whether it contained any toxic content. We fine-tuned a BERT model to identify the toxic content, and the accuracy of the AI model is 86% .

To understand how people respond to various AI explanations, we employed LIME and SHAP to explain the predictions made by the AI model. Additionally, we augment the LIME or SHAP explanations by either randomly masking out contributions from some features or amplifying contributions of some features (referred to as the 'Augmented' explanations) to see how humans react to them. These explanations are provided with AI recommendations together to humans in decision making.

4.2 Experimental Procedure

We posted our data collection study on the Prolific 1 to recruit human participants. Upon arrival, we randomly assigned each participant to one of the four decision making tasks and they needed to fill in an initial survey to report their demographic information and their knowledge of AI models and explanations. Participants started the study by completing a tutorial that described the decision making task that they needed to work on. To familiarize participants with the task, we initially asked them to complete five tasks independently without AI assistance. During these training tasks, we immediately provided the correct answer at the end of the task. After the completion of training tasks, participants moved on to the formal tasks. In the formal tasks, participants would receive one type of AI explanations among SHAP, LIME, or Augmented. Specifically, each participant was asked to complete a total of 15 tasks. In each task, participants were provided with the AI prediction and the explanations along with the task instance. They were then required to make their final decisions. Finally, participants were required to complete an exit survey to report their perceptions of the AI explanations they received during the study. They were asked to rate the alignment of AI explanations with their own rationale, as well as the usefulness, transparency, comprehensibility, satisfaction with the provided explanations, and their trust in the AI models, on a 5-point Likert scale. For the detailed survey questions, please refer to Appendix A.3. We offered a base payment of $1.2 and a potential bonus of $1 if the participant's accuracy is above 85%. The study was open to US-based workers only, and each worker can complete the study once.

4.3 Training Results

After collecting data on human behavior, we developed human behavior models for each type of task. For the human behavior models for two textual tasksToxicity Detection and Bias Detection -we employed the pretrained BERT encoder to extract features from the original sentences, which were then used as the task feature x in the human behavior model H w h . We optimized these behavior models using Adam [49] with an initial learning rate of 1 e -4 and a batchsize of each training iteration of 128. The number of training epochs is set as 10 . Table 1 shows the number of participants recruited, as well as the average accuracy of the human behavior model evaluated through 5-fold cross validation for each task. We observed that the average accuracy of all human behavior models exceeds 0 . 65 , which is considered to be reasonable. Consequently, we utilized these learned human behavior models to manipulate AI explanations in the following evaluations.

5 Evaluation I: Manipulating AI Explanations for Adversarial Purposes

In our first evaluation, we adopted the role of an adversarial party to explore whether they could utilize the learned human behavior model to manipulate AI explanations. The manipulation goal was to nudge human DMs to be biased against certain protected groups in the decision making process. We are particularly interested in comparing the fairness level of human decision outcomes between human DMs who receive original explanations, such as SHAP or LIME, and those who receive manipulated explanations. Notably, all human DMs are provided with the same AI predictions for the same decision making task. Additionally, we also explore differences in human perceptions of original AI explanations versus manipulated AI explanations.

Evaluation Metrics and Manipulating AI Explanations. Following previous work [50, 51], we used the false positive rate difference (i.e., FPRD ) and the false negative rate difference (i.e., FNRD ) to measure the fairness level of human decision outcomes-the closer these values are to zero, the more fair the decisions are. To manipulate AI explanations and nudge human DMs toward biasing against certain protected groups, we define the targeted human decision ˆ y h for each task as follows:

Census Prediction : In this task, we considered a person's sex as the protected attribute. The targeted human decision is defined as ˆ y h = 1 (indicating a person's annual income exceeds $50K) when x sex = male, and ˆ y h = -1 (indicating a person's annual income does not exceed $50K) when x sex = female. The fairness metrics can be computed as FPRD = FPR female -FPR male, and FNRD = FNR female -FNR male .

Recidivism Prediction : In this task, we considered the defendant's race as the protected attribute. The targeted human decision is defined as ˆ y h = 1 (indicating the defendant will reoffend) when x race = black, and ˆ y h = -1 (indicating the defendant will not reoffend) when x race = white. The two fairness metrics can be computed as FPRD = FPR white -FPR black, and FNRD = FNR white -FNR black .

Bias Detection : In this task, we divided text snippets into groups based on their political leaning. The targeted human decision is defined as ˆ y h = 1 (indicating the text is biased) when x leaning = democratic, and ˆ y h = -1 (indicating the text is not biased) when x leaning = republican. The fairness metrics can be computed as FPRD = FPR rep -FPR dem, and FNRD = FNR rep -FNR dem.

Toxicity Detection : In this task, we divided text snippets into groups based on the victim of the text. The targeted human decision is defined as ˆ y h = 1 (indicating the text is toxic) when x victim = white, and ˆ y h = -1 (indicating the text is non-toxic) when x victim = black. The two fairness metrics can be computed as FPRD = FPR black -FPR white, and FNRD = FNR black -FNR white .

For the Bias Detection and Toxicity Detection tasks, the original datasets [46, 48] provide annotations for the political leanings of the sentences and the targeted victims of the text snippets, respectively. After determining the targeted decision ˆ y h for each task instance, we then followed the gradient-based optimization procedure (i.e., Equation 4) to identify the manipulated explanation. We set the step size η as 0 . 01 , the trade-off λ as 0 . 01 , the optimization threshold τ as 0 . 1 , and the maximum optimization number T as 100 . For the initial AI explanation e θ 0 at the start of the optimization process, we directly initialize e ′ θ 0 as e ′ θ 0 ∼ U ( -1 , 1) . We then repeated this optimization process for 5 times and took the average to use in the following human evaluations. For the examples of manipulated explanations, please refer to Appendix B.3.

Data Collection. We followed the experimental procedure described in Section 4.2 to collect data on human responses to and perceptions of different AI explanations. We randomly assigned either SHAP or LIME explanations, or the manipulated explanations, to participants. Participants were required to complete 15 tasks with AI model predictions and the assigned explanations (SHAP or LIME or manipulated). We offered a base payment of $1.2 and a potential bonus of $1 if the participant's accuracy is above 85%. Table 2 reports detailed statistics of the participants in each task. Below, we analyzed how the manipulated explanations affect fairness level of human decisions and how do humans perceive those explanations.

5.1 How do the adversarially manipulated explanations affect fairness level of human decisions?

The fairness levels of participants' decision outcomes under the manipulated explanation, SHAP explanation, and LIME explanation are presented in Figure 1. Visually, it appears that when human DMs are provided with manipulated explanations, both FPRD and FNRD scores of their decision outcomes tend to deviate more from zero compared to when DMs receive SHAP or LIME explanations.

To examine whether these differences are statistically significant, we conducted regression analyses. Specifically, the focal independent variable was the type of explanation received by participants, while the dependent variables were the participants' FPRD and FNRD scores. To minimize the impact of potential confounding variables, we included a set of covariates in our regression models, such as participants' demographic background (e.g., age, race, gender, education level), their knowledge of AI explanations, their trust in AI models, and the FPRD or FNRD scores of the AI model decisions they received in the study. These covariates were selected based on prior HCI research [14, 20, 41] which empirically reveal how characteristics of human DMs may moderate the impacts of AI explanations on human decisions in AI-assisted decision making.

Our regression results indicate that the adversarial party can significantly increase the level of unfairness in human decision outcomes with manipulated explanations through human behavior modeling. Specifically, when examining FPRD , we found that participants who received manipulated AI explanations made more unfair decisions compared to those who received SHAP or LIME explanations ( p < 0 . 05 ) in the Census and Recidivism tasks. The difference was marginally significant ( p < 0 . 1 ) in the Toxicity task. When examining FNRD , results show that participants who received manipulated explanations made decisions that were significantly more unfair than those who received SHAP or LIME explanations ( p < 0 . 01 ) in the Bias task.

5.2 How do humans perceive the adversarially manipulated AI explanations?

(a) False Positive Rate Difference

Manipulated

SHAP

LIME

Manipulated

SHAP

LIME

ifference

0.5

bifference

0.5

***

***

0.3

0.3

**

**

0.1

***

***

*

0.1

***

D

-0.1

D

-0.1

R

-0.3

R

-0.3

P

-0.5

N

-0.5

T

Census

Recidivism

Toxicity

T

Census

Recidivism

Toxicity

(a) Transparency

(b) Usefulness

Manipulated

SHAP

LIME

Manipulated

SHAP

LIME

ransparency

5.0

5.0

S

4.5

S

4.5

4.0

4.0

3.5

3.5

seft

3.0

3.0

2.5

S

2.5

2.0

2.0

Census

Recidivism

Toxicity

Census

Recidivism

Toxicity

In Section 5.1, we found that the adversarial party can manipulate AI explanations to nudge human DMs toward making more unfair decisions compared to those who received the original AI explanations, aligning with the adversarial party's intentions. To determine whether DMs could detect any abnormalities in the manipulated explanations, we examined how their perceptions of AI explanations varied among manipulated, SHAP, and LIME explanations.

Figures 2a and 2b compare the average perceived transparency and usefulness of three types of AI explanations. Visually, there are no significant differences in how the explanations are perceived by people who received different explanations. We also applied regression models to predict human perceptions of these explanations by accounting for their demographic background (e.g., age, race, gender, education level), their knowledge of AI explanations and their trust in AI models. The regression results indicate that there are no significant differences in the perceived transparency and usefulness of manipulated explanations compared to SHAP or LIME explanations. Similar patterns were observed for perceptions of alignment, comprehensibility, satisfaction, and trust between the manipulated and unmanipulated explanations. While adversarially manipulated explanations significantly influence human decision making behavior, individuals generally do not detect abnormalities in the manipulated AI explanations across most tasks. For further details, please refer to Appendix B.2.

6 Evaluation II: Manipulating AI Explanations for Benign Purposes

In the previous section, we found that the adversarial party could use the behavior model to manipulate AI explanations, thereby misleading humans into making unfavorable decisions against specific groups. Naturally, one might wonder could a third party also use behavior models to manipulate AI explanations for benign purposes, such as promoting more appropriate human reliance on AI models? For instance, can manipulated AI explanations lead humans to reject AI recommendations when the

(a) Accuracy

(b) Overreliance

(c) Underreliance

Manipulated

SHAP

LIME

Manipulated

SHAP

LIME

Manipulated

SHAP

LIME

iance

***

ince

en:

0.9

0.7

***

**

中*

verrelia

0.5

0.9

0.7

***

***

erre

0.3

0.5

***

A

0.5

Census

Recidivism

Toxicity

0.3

Census

Recidivism

Toxicity

U

0.1

Census

Recidivism

Toxicity

AI model decision is likely incorrect, and encourage acceptance when the decision is likely correct? We aim to explore the answers to this question in this section.

Evaluation Metrics and Manipulating AI Explanations Following previous work [14, 41, 43], we used the accuracy, underreliance, and overreliance to measure human DMs' appropriate reliance level on AI models. Underreliance refers to the fraction of tasks where the participant's decision was different from the AI model's decision when the AI model's decision was correct. Overreliance refers to the fraction of tasks where the participant's decision was the same as the AI model's decision when the AI model's decision was incorrect. To manipulate AI explanations for the promotion of more appropriate reliance on AI models, it is necessary to determine the reliability of AI model prediction on each task instance. Recent work [52, 53] has proposed methods to leverage the complementary strengths of humans and AI by combining human independent decisions and AI model, which is often shown to result in more accurate decisions than those made by either humans or AI models alone. Specifically, given the human independent decision y h independent , the AI model recommendation y m , and the task instance x , these methods learn models to combine y h independent and y m to produce a combined result:

To see whether y combine can yield better decisions compared to AI alone or human alone, we evaluated various combination models including the human-AI combination method [53] and several truth inference methods [54-57] used in crowdsourcing. Our results showed that the human-AI combination method [53] generally outperformed AI solo and independent human decision, as well as other combination methods. Thus, y combine produced by the human-AI combination method [53] is defined as the targeted decision ˆ y h for manipulating AI explanations. For detailed information on the evaluations of each combination method, please refer to the Appendix C.1. We again followed Equation 4 to manipulate the AI explanations. We set the step size η as 0 . 01 , the trade-off λ as 0 . 01 , the optimization threshold τ as 0 . 1 , and the maximum optimization number T as 100 , and the initial AI explanation e ′ θ 0 at the start of the optimization process is initialized as e ′ θ 0 ∼ U ( -1 , 1) . We repeated this optimization process for 5 times and took the average to use in the following experiments. For the examples of manipulated explanations, please refer to Appendix C.4.

Data Collection. We recruited participants from Prolific once again to collect behavioral data under the benignly manipulated explanations, following the experimental procedure described in Section 4.2. We offered a base payment of $1.2 and a potential bonus of $1 if the participant's accuracy is above 85%. Table 2 shows the detailed statistics of the participants we recruited for each task. Subsequently, we analyzed whether the benignly manipulated explanations can promote appropriate reliance of human DMs on AI models, as well as their perceptions of these AI explanations.

6.1 Can benignly manipulated explanations promote appropriate reliance of human DMs on AI models?

Figures 3a, 3b, and 3c compare the average accuracy, overreliance, and underreliance of human decision outcomes under manipulated, SHAP, and LIME explanations, respectively. It is clear that providing human DMs with manipulated AI explanations leads to an increase in the accuracy of their decision outcomes for most of tasks. We subsequently conducted regression analyses to determine

(a) Transparency

(b) Usefulness

Manipulated

SHAP

LIME

Manipulated

SHAP

LIME

ransparency

5.0

5.0

S

4.5

S

4.5

4.0

n

4.0

3.5

3.5

3.0

seft

3.0

2.5

S

2.5

S

2.0

2.0

Census

Recidivism

Toxicity

Census

Recidivism

Toxicity

whether these differences are statistically significant. The regression models incorporated a set of covariates, including participants' demographic backgrounds (e.g., age, race, gender, education level), their knowledge of AI explanations, their trust in AI models, and the accuracy of the AI models. The regression results indicate that in the Census, Recidivism, and Bias tasks, substituting SHAP or LIME explanations with manipulated explanations significantly improves the accuracy of human-AI team. In contrast, for the Toxicity task, we observed no statistical difference, which could potentially be attributed to the high competence of humans in solving this task (e.g., when presented with SHAP or LIME explanations, the average decision accuracy of participants already exceeds 0.8, leaving limited room for further improvement).

6.2 How do humans perceive benignly manipulated AI explanations?

In Section 5.2, we observed that it is challenging for humans to detect abnormalities in the adversarially manipulated explanations, even though they are unconsciously influenced by the manipulated explanations to make more unfair decisions. In this section, we revisit this question to investigate into whether humans' perceptions of the manipulated explanations change, when they are manipulated for benign purposes. Figures 4a and 4b compare the average human perceived transparency and usefulness of the benignly manipulated explanations, SHAP explanations, and LIME explanations. Regression analyses reveal no statistically significant differences among the perceived transparency and usefulness of these three types of explanations. Similar trends were observed for other perceptual aspects of explanations, including perceived alignment, comprehensibility, satisfaction, and trust. For further results, please refer to Appendix C.3.

7 Conclusion and Limitations

In this paper, we explore whether we can quantitatively model how humans incorporate both AI recommendations and explanations into their decision making process, and whether we can utilize the quantitative understanding of human behavior obtained from these learned models to manipulate AI explanations for both adversarial and benign purposes. Our extensive experiments across various tasks demonstrate that human behavior can be easily influenced by these manipulated explanations toward targeted outcomes, regardless of the intent being benign or adversarial. Despite the significant influence of these falsified explanations on human decisions, individuals typically fail to detect or recognize any abnormalities. Our study has several limitations. For example, it focuses on modeling and manipulating score-based explanations. Further research is needed to explore how to model how humans incorporate other types of explanations, such as example-based and rule-based explanations, and how these can be manipulated to influence human behavior as observed with score-based explanations in our study. Additionally, our study was limited to decision making tasks involving tabular and textual data, which are naturally suited to score-based explanations. Further explorations are needed to extend these findings to decision tasks with other data types (e.g., images).

Ethical Consideration

This study was approved by the Institutional Review Board of the authors' institution. Through our findings, we aim to draw the community's attention to the ease with which third parties can manipulate AI explanations with the learned behavior models to influence human decision making. Users often lack the ability to accurately and appropriately interpret the AI explanations presented to them, yet their decision behavior is easily swayed by the manipulated AI explanations. Our findings highlight the critical importance of securing human-AI interaction data to prevent the misuse of human behavior models derived from it. Additionally, there is an urgent need to ensure that AI explanations provided to humans are more secure and inherently benign. Moreover, providing pre-education is essential to assist humans in establishing a proper understanding of AI explanations, which may potentially mitigate the risks of manipulation.

In addition, our experiments are based on datasets that are publicly available; 'correct' decisions for the tasks in these datasets are generally considered as recording the real-world ground truth. While these datasets are not intentionally biased toward any specific groups, we acknowledge that there might be implicit biases introduced to these datasets during the curation process, which are beyond our control. Importantly, we note that we made no alterations to the datasets that would introduce additional bias in our experiment.

Acknowledgments

We thank the support of the National Science Foundation under grant IIS-2229876 and IIS-2340209 on this work. Any opinions, findings, conclusions, or recommendations expressed here are those of the authors alone.

References

Gagan Bansal et al. 'Is the most accurate ai the best teammate? optimizing ai for teamwork'. In: Proceedings of the AAAI Conference on Artificial Intelligence . Vol. 35. 13. 2021, pp. 11405-11414.

Aakriti Kumar et al. 'Explaining algorithm aversion with metacognitive bandits'. In: Proceedings of the annual meeting of the cognitive science society . Vol. 43. 43. 2021.

Peter Hase and Mohit Bansal. 'Evaluating explainable AI: Which algorithmic explanations help users predict model behavior?' In: arXiv preprint arXiv:2005.01831 (2020).

Heliodoro Tejeda et al. 'AI-assisted decision-making: A cognitive modeling approach to infer latent reliance strategies'. In: Computational Brain & Behavior 5.4 (2022), pp. 491-508.

Hussein Mozannar, Arvind Satyanarayan, and David Sontag. 'Teaching humans when to defer to a classifier via exemplars'. In: Proceedings of the aaai conference on artificial intelligence . Vol. 36. 5. 2022, pp. 5323-5331.

Kailas Vodrahalli, Tobias Gerstenberg, and James Y Zou. 'Uncalibrated models can improve human-ai collaboration'. In: Advances in Neural Information Processing Systems 35 (2022), pp. 4004-4016.

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. '" Why should i trust you?" Explaining the predictions of any classifier'. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining . 2016, pp. 1135-1144.

Scott M Lundberg and Su-In Lee. 'A unified approach to interpreting model predictions'. In: Advances in neural information processing systems 30 (2017).

Himabindu Lakkaraju, Stephen H Bach, and Jure Leskovec. 'Interpretable decision sets: A joint framework for description and prediction'. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining . 2016, pp. 1675-1684.

Himabindu Lakkaraju et al. 'Faithful and customizable explanations of black box models'. In: Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society . 2019, pp. 131-138.

Chirag Agarwal et al. 'Openxai: Towards a transparent evaluation of model explanations'. In: Advances in Neural Information Processing Systems 35 (2022), pp. 15784-15799.

Gagan Bansal et al. 'Does the whole exceed its parts? the effect of ai explanations on complementary team performance'. In: Proceedings of the 2021 CHI conference on human factors in computing systems . 2021, pp. 1-16.

Zana Buçinca et al. 'Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems'. In: Proceedings of the 25th international conference on intelligent user interfaces . 2020, pp. 454-464.

Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 'Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making'. In: Proceedings of the 2020 conference on fairness, accountability, and transparency . 2020, pp. 295-305.

Jonathan Dodge et al. 'Explaining models: an empirical study of how explanations impact fairness judgment'. In: Proceedings of the 24th international conference on intelligent user interfaces . 2019, pp. 275-285.

Forough Poursabzi-Sangdeh et al. 'Manipulating and measuring model interpretability'. In: Proceedings of the 2021 CHI conference on human factors in computing systems . 2021, pp. 1-52.

Arjun Chandrasekaran et al. 'Do explanations make VQA models more predictable to a human?' In: arXiv preprint arXiv:1810.12366 (2018).

Siddhant Arora et al. 'Explain, edit, and understand: Rethinking user study design for evaluating model explanations'. In: Proceedings of the AAAI Conference on Artificial Intelligence . Vol. 36. 5. 2022, pp. 5277-5285.

Hao-Fei Cheng et al. 'Explaining decision-making algorithms through UI: Strategies to help non-expert stakeholders'. In: Proceedings of the 2019 chi conference on human factors in computing systems . 2019, pp. 1-12.

Xinru Wang and Ming Yin. 'Are explanations helpful? a comparative study of the effects of explanations in ai-assisted decision-making'. In: 26th international conference on intelligent user interfaces . 2021, pp. 318-328.

Nina Corvelo Benz and Manuel Rodriguez. 'Human-aligned calibration for ai-assisted decision making'. In: Advances in Neural Information Processing Systems 36 (2023).

Zhuoran Lu et al. 'Mix and Match: Characterizing Heterogeneous Human Behavior in AI-assisted Decision Making'. In: Proceedings of the AAAI Conference on Human Computation and Crowdsourcing . Vol. 12. 2024, pp. 95-104.

Zhuoyan Li, Zhuoran Lu, and Ming Yin. 'Decoding AI's Nudge: A Unified Framework to Predict Human Behavior in AI-assisted Decision Making'. In: Proceedings of the AAAI Conference on Artificial Intelligence . Vol. 38. 9. 2024, pp. 10083-10091.

Zhuoyan Li, Zhuoran Lu, and Ming Yin. 'Modeling human trust and reliance in ai-assisted decision making: A markovian approach'. In: Proceedings of the AAAI Conference on Artificial Intelligence . Vol. 37. 5. 2023, pp. 6056-6064.

Shuai Ma et al. 'Who should i trust: Ai or myself? leveraging human and ai correctness likelihood to promote appropriate trust in ai-assisted decision-making'. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems . 2023, pp. 1-19.

Zana Buçinca et al. 'Towards Optimizing Human-Centric Objectives in AI-Assisted Decision-Making With Offline Reinforcement Learning'. In: arXiv preprint arXiv:2403.05911 (2024).

Joey Hong, Sergey Levine, and Anca Dragan. 'Learning to influence human behavior with offline reinforcement learning'. In: Advances in Neural Information Processing Systems 36 (2023).

Hussein Mozannar et al. 'Effective Human-AI Teams via Learned Natural Language Rules and Onboarding'. In: Advances in Neural Information Processing Systems 36 (2022).

Syed Hasan Amin Mahmood, Zhuoran Lu, and Ming Yin. 'Designing Behavior-Aware AI to Improve the Human-AI Team Performance in AI-Assisted Decision Making'. In: Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence . 2024.

Hussein Mozannar et al. 'When to show a suggestion? Integrating human feedback in AI-assisted programming'. In: Proceedings of the AAAI Conference on Artificial Intelligence . Vol. 38. 9. 2024, pp. 10137-10144.

Valerie Chen et al. 'Use-case-grounded simulations for explanation evaluation'. In: Advances in neural information processing systems 35 (2022), pp. 1764-1775.

Julien Colin et al. 'What i cannot predict, i do not understand: A human-centered evaluation framework for explainability methods'. In: Advances in neural information processing systems 35 (2022), pp. 28322845.

Chacha Chen et al. 'Machine explanations and human understanding'. In: arXiv preprint arXiv:2202.04092 (2022).

Zhuoran Lu et al. 'Strategic adversarial attacks in AI-assisted decision making to reduce human trust and reliance'. In: Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence . 2023, pp. 3020-3028.

Himabindu Lakkaraju et al. 'Rethinking explainability as a dialogue: A practitioner's perspective'. In: arXiv preprint arXiv:2202.01875 (2022).

Dylan Slack et al. 'Counterfactual explanations can be manipulated'. In: Advances in neural information processing systems 34 (2021), pp. 62-75.

Dylan Slack et al. 'Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods'. In: AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES) (2020).

Rohan Paleja et al. 'The utility of explainable ai in ad hoc human-machine teaming'. In: Advances in neural information processing systems 34 (2021), pp. 610-623.

Hua Shen et al. 'Are shortest rationales the best explanations for human understanding?' In: arXiv preprint arXiv:2203.08788 (2022).

Han Liu, Vivian Lai, and Chenhao Tan. 'Understanding the effect of out-of-distribution examples and interactive explanations on human-ai decision making'. In: Proceedings of the ACM on Human-Computer Interaction 5.CSCW2 (2021), pp. 1-45.

Vivian Lai, Han Liu, and Chenhao Tan. '" Why is' Chicago'deceptive?" Towards Building Model-Driven Tutorials for Humans'. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems . 2020, pp. 1-13.

Himabindu Lakkaraju and Osbert Bastani. '" How do I fool you?" Manipulating User Trust via Misleading Black Box Explanations'. In: Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society . 2020, pp. 79-85.

Vivian Lai et al. 'Selective explanations: Leveraging human input to align explainable ai'. In: Proceedings of the ACM on Human-Computer Interaction 7.CSCW2 (2023), pp. 1-35.

Ron Kohavi. Census Income . UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C5GP7S. 1996.

Julia Dressel and Hany Farid. 'The accuracy, fairness, and limits of predicting recidivism'. In: Science advances 4.1 (2018), eaao5580.

Timo Spinde et al. 'Neural Media Bias Detection Using Distant Supervision With BABE - Bias Annotations By Experts'. In: Findings of the Association for Computational Linguistics: EMNLP 2021 . Ed. by Marie-Francine Moens et al. Punta Cana, Dominican Republic: Association for Computational Linguistics, Nov. 2021, pp. 1166-1177. DOI: 10.18653/v1/2021.findings-emnlp.101 . URL: https://aclanthology.org/2021.findings-emnlp.101 .

Jacob Devlin et al. 'BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding'. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) . Ed. by Jill Burstein, Christy Doran, and Thamar Solorio. Minneapolis, Minnesota: Association for Computational Linguistics, June 2019, pp. 4171-4186. DOI: 10.18653/v1/N19-1423 . URL: https: //aclanthology.org/N19-1423 .

Chris J Kennedy et al. 'Constructing interval variables via faceted Rasch measurement and multitask deep learning: a hate speech application'. In: arXiv preprint arXiv:2009.10277 (2020).

Diederik P Kingma and Jimmy Ba. 'Adam: A method for stochastic optimization'. In: arXiv preprint arXiv:1412.6980 (2014).

Moritz Hardt, Eric Price, and Nati Srebro. 'Equality of opportunity in supervised learning'. In: Advances in neural information processing systems 29 (2016).

Lucas Dixon et al. 'Measuring and mitigating unintended bias in text classification'. In: Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society . 2018, pp. 67-73.

Mark Steyvers et al. 'Bayesian modeling of human-AI complementarity'. In: Proceedings of the National Academy of Sciences 119.11 (2022), e2111547119.

Gavin Kerrigan, Padhraic Smyth, and Mark Steyvers. 'Combining human predictions with model probabilities via confusion matrices and calibration'. In: Advances in Neural Information Processing Systems 34 (2021), pp. 4421-4434.

Yudian Zheng et al. 'Truth inference in crowdsourcing: Is the problem solved?' In: Proceedings of the VLDB Endowment 10.5 (2017), pp. 541-552.

Jacob Whitehill et al. 'Whose vote should count more: Optimal integration of labels from labelers of unknown expertise'. In: Advances in neural information processing systems 22 (2009).

Qi Li et al. 'A confidence-aware approach for truth discovery on long-tail data'. In: Proceedings of the VLDB Endowment 8.4 (2014), pp. 425-436.

Vikas C Raykar et al. 'Learning from crowds.' In: Journal of machine learning research 11.4 (2010).

Prediction Task 1/20

jenuue s,uosued e jeujeqm p(peud pue mojeq egoud aug melaau eseeld

income is above $50K.

Defendant Profile:

4. Number of years of education:

1.Age:

3. Hours per week:

6.Occupation:

7. Work Class:

44

2. Sex:

Private Sector

Craf Repair

Married

50

13

Male

Prediction Task 1/20

mean this feature decreases the chance, with bar length representing the extent

Why our Al model makes this prediction?

The chart below shows how our machine leaming model considers each feature

of this profile in making the decision. Red bars mean this feature increases the

chance of an individuaf's annual income exceeding $50,000, and blue bars

of impact.

iork Class (Private Sector)

Sex (Ma)

Age M)

Please review the profile below and predict whether the defendant

would reoffend in the next two years.

Defendant Profile:

5. Felony Crime Count before

4. Prior Crime Count after

7. Current Charge Degree:

8: Current Charge Issue:

1.Race:

Age 18:

Age 18:

Age 18:

Black

Sex:

2.

Resist Officer wViolence

Female

Felony

2

0

0

Age:

3.

29

Why our Al model makes this prediction?

The chart below shows how our Al model considers each feature of this

defendant's profile in making the decision. Red bars mean this feature

increases the chance of reoffending, and blue bars mean this feature

decreases the chance of reoffending. with bar length representing the extent of

impact.

9 efy eq sun up Auope

(2et eby age uneg wug oyd

(uoe eefeg emo uaung

nor count before Age 18 (5)

Sex (Fenale)

Race (biack)

Age(29)

Our machine learning model predicts that the annual income of this person Is

not above $50K.

(ede  dnoog

Our Al model predicts that this person will reoffend in two years.

-1.0

-0.5

0.5

1.0

Make your prediction:

(E uogeonpe go seaf go aqunN

Make your prediction:

Now, do you think the annual income of this person is above $50K?

-1.0-0.8-0.6-0.4-0.200.20.40.60.81.0

Now, do you think this defendant will reoffend within 2 years?

Yes, I think the annual income of this person is above $50K.

Yes, I think this defendant will reoffend within 2 years.

No, I think the annual income of this person is not above $50K.

No, I think this defendant will not reoffend within 2 years.

submit your predictionl

submit your predliction!

(a) Census prediction

(b) Recidivism prediction

A The Design of Human Study (Additional Details)

A.1 Compensation Details

To determine the appropriate payment level for each type of task, we first conducted a preliminary study to estimate the time workers might spend on the tasks. Our pilot study indicated that a base payment of $1.2 per task translates to an approximate hourly rate of $10. To provide greater transparency about the compensation received by participants in our formal study, Table A.1 summarizes the average hourly payment and the average time spent on each task.

A.2 Task Interfaces

Figure A.1a, A.1b, A.2a, and A.2b show the interfaces participants used in the Census Prediction , Recidivism Prediction , Bias Detection , and Toxicity Detection tasks, respectively.

A.3 Exit Survey Questions

In the study, after the main tasks, the participants need to fill in an exit survey to report their perceptions of presented AI explanations. The survey questions are detailed as:

Alignment : On a scale of 1-5, how well do you think the explanations align with your understanding of the problem?

Usefulness : On a scale of 1-5, how useful are the explanations in helping you make decisions?

Transparency : On a scale of 1-5, how well do you think the explanations reveal the AI model's decision making process?

Comprehensibility : On a scale of 1-5, how easy is it for you to understand the explanations?

Satisfaction : On a scale of 1-5, how satisfied are you with the explanations provided by the AI model?

Trust : On a scale of 1-5, would you trust the AI model's prediction or decision based on the explanations?

PredictionTask1/15

Pleasereview thesentencefrom amediareportbelow and determinewhetherthissentence

exhibits bias.

Sentence:

The issue isn't whether farmers need or deserve aid but whether werecognize that we are one

AI ModelPrediction:

OurAl modelpredicts that thissentenceisnon-biased.

Bluewords areassociated withnon-biased category,and red words are associatedwith

biasedcategory.The darknessof theword indicates the degreeof association with the

two categories.

non-biased

biased

Make your prediction:

Now, do you think whether this sentence exhibits bias?.

Yes,I think this sentence is biased.

No, I think this sentence is non-biased.

submit your prediction!

(a) Bias detection

PredictionTask1/15

Pleasereviewthesentencefromsocialmediabelowand determinewhetherthissentence

contains toxic content such as hate speech,harassment and threats.

Sentence:

Callicles credits “Black Lives Matter” with the coming downfall of Kamala Harris. Why? Check it out.

URL

AlModelPrediction:

Our Al model predicts that this sentence is non-toxic.

Blue words are associatedwithnon-toxiccategory.andredwordsare associated with

toxic category.The darkness of the word indicates the degree of association with the two

categories.

non-toxic

toxic

Make your prediction:

Now, do you think whether this sentence contains toxic content?.

Yes,I think this sentence is toxic.

No,I think this sentence is non-toxic.

submit your prediction!

(b) Toxicity detection

B Evaluation I: Manipulating AI Explanations for Adversarial Purposes (Additional Results)

B.1 Visual Consistency of Explanations

Table B.1 compares the agreement between the sum of feature importance in explanations and AI predictions, the Pearson correlation coefficients, measured in terms of Pearson correlation coefficient, for adversarially manipulated, LIME, and SHAP explanations, respectively. We observed that the visual consistency of the manipulated explanations is lower than that of SHAP but very close to that of LIME.

B.2 Human Perceptions of Explanations

Figures B.1a to B.1d compare the average human perceived alignment, comprehensibility, satisfaction with the provided explanations ,and the trust in the AI models under the under the adversarially manipulated explanation, SHAP explanation, or LIME explanation. In general, our findings indicate that there are no significant differences in people's perceptions of the three explanations across four tasks, with the exceptions for the alignment and trust in the Toxicity task. Specifically, for the Toxicity task, participants perceived LIME explanations as aligning more closely with their own rationales than the adversarially manipulated explanations, with a marginally significant difference ( p < 0 . 1 ). Furthermore, participants reported significantly greater trust in the AI models accompanied by LIME explanations compared to those with adversarially manipulated explanations ( p < 0 . 01 ).

B.3 Examples of Manipulated Explanations

Figures B.2 to B.5 show the visual comparisons of adversarially manipulated, LIME, and SHAP explanations for the Census Prediction task, Recidivism Prediction task, Bias Detection task, and Toxicity Detection task, respectively.

(a) Alignment

(b) comprehensibility

(c) Satisfaction

(d) Trust

Manipulated

SHAP

LIME

omprehensibility

Manipulated

SHAP

LIME

Manipulated

SHAP

LIME

Manipulated

SHAP

LIME

Alignment

5.0

4.5

5.0

Satisfactior

5.0

4.5

5.0

4.5

4.0

4.0

4.0

4.0

3.5

3.5

3.5

3.5

3.0

3.0

3.0

3.0

2.5

2.5

2.5

2.5

2.0

Census

Recidivism

Toxicity

2.0

Census

Recidivism

Toxicity

2.0

Census

Recidivism

Toxicity

2.0

Census

Recidivism

Toxicity

Profile:

1.Age:

52

2.Sex:

Male

3.Hours per week:

40

Al Prediction: 0, Label: 0, Targeted Decision: 1

4.Number of years of education:

10

5.Martial Status:

Married

6.Occupation:

Craft Repair

7.Work Class:

GovernmentSector

Age (52)

Age (52)

Ago (52)

■

Sex (Male)

-0.6

-0.4

0.2

.0

-0.4

0.2

0.2

1.00.80.60.40.2

(a) LIME

(b) SHAP

(c) Manipulated

Profile:

1.Age:

26

2.Sex:

Female

3.Hours per week:

40

Al Prediction: 1, Label: 1, Targeted Decision: 0

4.Number of years ofeducation:

13

5.Martial Status:

Married

6.Occupation:

Professional Specialty

7.Work Class:

Government Sector

■

Age (26)

Age (26)

0

0.2

1.0

-0.8-0.6-0.4-0.2

(a) LIME

(b) SHAP

(c) Manipulated

C Evaluation II: Manipulating AI Explanations for Benign Purposes (Additional Results)

C.1 Combining Human Decisions and AI Predictions

In the main paper, we aim to benignly manipulate AI explanations to encourage human DMs to rely more appropriately on AI models. Following previous research [52, 53], we combined independent human decisions with AI model predictions to determine the targeted decision for each task instance. We evaluated the human-AI combination method [53] and several truth inference methods used in crowdsourcing for truth discovery. We detailed the process of evaluation below.

DefendantProfile:

1.Race:

Black

Sex:

2.

Female

Age:

3.

35

4.Prior Crime Count afterAge

18:

4

Al Prediction: 0, Label: 0, Targeted Decision: 1

5.Felony Crime Count before

Age18:

0

6.MisdemeanorCountbefore

Age 18:

0

Felony

8:Current Charge Issue:

Age (35)

Sex (Female

Race (Black)

Age (35)

Age (35)

8

(a) LIME

1.0

0.5

(b) SHAP

0.5

(c) Manipulated

1.0

0.5

DefendantProfile:

1.Race:

White

Sex:

2.

Male

Age:

3.

27

4.Prior Crime Count afterAge

18:

0

5.Felony Crime Count before

Age18:

0

Al Prediction: 1, Label: 1, Targeted Decision: 0

6.Misdemeanor Count before

Age18:

0

Felony

8:Current Charge Issue:

Grand Theft(Motor Vehicle)

Age (27)

Age (27)

Age (27)

Age 18 (0)

Sex (Male)

(a) LIME

1.0

0.5

(b) SHAP

0.5

0.5

(c) Manipulated

1.0

0.5

Political leaning: right-leaning, Al Prediction: 0, Label: 1, Targeted Decision: 0

As part of Pelosi's coronavirus plan, House

As part of Pelosi's coronavirus plan, House

AS

part

of

Pelosi's

coronavirus

plan,

House

Democrats slipped in multiple measures from their

Democrats slipped in multiple measures

from their

Democrats

slipped

in

multiple

measures

from

their

H.R.1legislation that wouldimplement nationwide

H.R.1legislationthat wouldimplementnationwide

H.R.1legislation

that

PInoM

implement

nationwide

ballot harvesting for federal elections —effectively

ballot harvesting for federal elections effectively

ballot harvesting

forfederal

elections

effectively

allowingpoliticaloperatives to collect anddeliver an

allowing political operatives to collect and deliver an

allowing

political

operatives

to

collect

and

deliver

an

endlessnumber of absentee ballots from voters.

endless number of absentee ballots from voters.

endlessnumber

of

absentee

ballots

from

voters.

(a) LIME

(b) SHAP

(c) Manipulated

Political leaning: left-leaning, Al Prediction: 1, Label: 0, Targeted Decision: 1

Navalvny has for years been accusing Putin and

Navalvnyhasfor

years

beenaccusingPutinand

Navalvny

for

years

been

accusing

and

his entourage of rampant corruption.

his entourage oframpantcorruption.

his

entourage

of

rampant

corruption.

(a) LIME

(b) SHAP

C

Manipulated

Simulating Human Independent Decision. To understand how humans independently make decisions on each task instance, we first conducted a study again on the Prolific to collect independent human decision behavior data across four tasks. We recruited 40 participants for each task. Each recruited participant needed to complete 15 tasks. With the collection of human behavior data, we then fitted two-layer neural networks to simulate human independent decision behavior. For Toxicity Detection and Bias Detection textual tasks, we used the pretrained BERT encoder to initially extract features from the original sentences as the input to the independent behavior models. We optimized these independent behavior models using Adam [49] with an initial learning rate of 1 e -4 and

Victim: Black, Al Prediction: 0, Label: 0, Targeted Decision: 0

1think Mulan shouldberecast blackwomen tobe

1think Mulan should berecast black women tobe

think

Mulan

should

berecast

a

black

women

more inclusive. Not enough diversity, they are all

more inclusive. Not enough diversity, they are all

to

more inclusive.Not enough diversity,

they

Chinese.

Chinese.

are all Chinese.

(a) LIME

(b) SHAP

(c) Manipulated

Victim: Black, Al Prediction: 1, Label: 1, Targeted Decision: 0

ugly

niggaa

Ami theonlyonewho

gives

an

e66iu

Am

ithe

only

one

who

gives

an

16m

chance then...BooMhe starts to act like you the

chance

then...BOOM

starts

to

act

like

youthe

chance

then...BOOM

he

starts

to

act

like

you

the

ugly one?@

one?@

uglyone?@

(a) LIME

(b) SHAP

C)

Manipulated

(a) Alignment

(b) comprehensibility

(c) Satisfaction

(d) Trust

Manipulated

SHAP

LIME

omprehensibility

Manipulated

SHAP

LIME

Manipulated

SHAP

LIME

Manipulated

SHAP

LIME

Alignment

5.0

4.5

5.0

4.5

Satisfaction

5.0

4.5

5.0

4.5

4.0

4.0

4.0

4.0

3.5

3.5

3.5

3.5

3.0

3.0

3.0

3.0

2.5

2.5

2.5

2.5

2.0

Census

Recidivism

Toxicity

2.0

Census

Recidivism

Toxicity

S

2.0

Census

Recidivism

Toxicity

2.0

Census

Recidivism

Toxicity

a batchsize of each training iteration of 128. The number of training epochs is set as 10 . The average accuracy of 5-fold validation for each model is reported in Table C.1, which we found to be satisfactory. We then utilized these fitted models to simulate independent human decisions y h independent in the human-AI combination process to determine the potentially better decisions.

Comparing Combination Performance. We consider the human + AI combination method [53] and a few truth inference methods in crowdsourcing as baselines in the evaluation, including GLAD [55], CATD [56], LFC [57], EM [54], and MV [54]. These methods combine the human independent decisions y h independent predicted by the fitted independent human behavior models and AI model recommendations y m to produce combined decisions y combine. The accuracy of each method on holdout task pools for each task to be used in the subsequent evaluation is reported in Table C.2. In general, we found that human + AI combination method outperforms other baselines. By integrating human decisions with AI predictions, this method shows superior performance to either AI solo or human solo across all four tasks. Consequently, we used the combined decisions y combine from the human + AI combination method as the targeted decision ˆ y h in subsequent experiments to manipulate explanations.

C.2 Visual Consistency of Explanations

Table B.1 compares the agreement between the sum of feature importance in explanations and AI predictions, the Pearson correlation coefficients, measured in terms of Pearson correlation coefficient, for benignly manipulated, LIME, and SHAP explanations, respectively. We observed that the visual consistency of the manipulated explanations is very close to that of SHAP and higher than that of LIME.

C.3 Human Perceptions of Explanations

Figures C.1a to C.1d compare the average human perceived alignment, comprehensibility, satisfaction with the provided explanations ,and the trust in the AI models under the under the adversarially

Profile:

1.Age:

38

3.Hours per week:

4.Numberofyearsofeducation:

5.Martial Status:

6.Occupation:

7.Work Class:

Sex (Female)

Age (38)

(a) LIME

-0.8

9'0-

(b) SHAP

Al Prediction: 1, Label: 1, Targeted Decision: 1

Age (38)

1.00.80.60.4-0.2

(c) Manipulated

Profile:

1.Age:

26

2.Sex:

Female

3.Hours per week:

40

Al Prediction: 0, Label: 0, Targeted Decision: 0

4.Number ofyearsofeducation:

13

5.Martial Status:

6.Occupation:

Professonal Specialty

7.WorkClass:

Government Sector

Sex (Female)

二

Hours per Week (40)

Age (26)

Age (26)

0.6

-0.4

-0.2

0.4

-1.0-0.8-0.6-0.4-0.2

(a) LIME

(b) SHAP

(c) Manipulated

manipulated explanation, SHAP explanation, or LIME explanation. We found that there are no significant differences in people's perceptions of the three explanations across four tasks on these aspects.

C.4 Examples of Manipulated Explanations

Figures C.2 to C.5 show the visual comparisons of benignly manipulated, LIME, and SHAP explanations for the Census Prediction task, Recidivism Prediction task, Bias Detection task, and Toxicity Detection task, respectively.

。

2.Sex:

36

12

Professional Specialty

Private Sector

0.20.4

1.0

Female

Age (38)

Sex (Female)

9'0

0.8

DefendantProfile:

1.Race:

Black

Sex:

2.

Female

Age:

3.

27

4.Prior Crime Count afterAge

18:

3

Al Prediction: 0, Label: 1, Targeted Decision: 1

5.Felony Crime Count before

Age 18:

0

6.MisdemeanorCountbefore

Age 18:

0

Felony

8:Current Charge Issue:

Grand Theft(Motor Vehicle)

Race(Black)

Age (27)

Race (Black)

Age (27)

Age (27)

Prior Crime Count after Age 18 (3)

Sex (Fem

-0.5

0.5

0.5

(a) LIME

(b) SHAP

(c) Manipulated

DefendantProfile:

1.Race:

White

Sex:

2.

Male

Age:

3.

37

18:

3

Al Prediction: 0, Label: 0, Targeted Decision: 0

5.Felony Crime Count before

0

Age18:

6.Misdemeanor Count before

Age18:

0

Felony

8:Current Charge Issue:

Driving While License Revoked

Age 18 (0)

er Age 18 (3)

Race (White)

Sex (Male)

Prior Crim

Age (37)

fore Age 18 (0)

(ce) oby

Sex (Male)

1.0

1.0

0.5

1.0

0.5

(a) LIME

(b) SHAP

(c) Manipulated

Political leaning: left-leaning, Al Prediction: 1, Label: 1, Targeted Decision: 1

Donald Trump,who was pro-choice in the not-too-

Donald Trump,who was pro-choice in the not-too-

Trump.

who

was

pro-choice

the

not-too-

distant pastandnever fit themoldofafar-right

distantpastandnever fit themoldofafar-right

distant

past

and

never

fitthe

mold

a

far-right

culture-war crusader,has taken a keen interest in

culture-war crusader,has taken keen interest

culture-war

crusader,

has

taken

a

keen

interest

in

abortion rights lately.

abortion rights lately.

abortion

rights lately.

(a) LIME

(b) SHAP

(c) Manipulated

Political leaning: left-leaning, Al Prediction: 1, Label: 1, Targeted Decision: 1

Hotez saidanti-vaccine groups are seizing on the

Hotez

said

anti-vaccinegroupsareseizing

onthe

Hotez

said

anti-vaccine

groups

are

seizing

onthe

anti-government sentiment stoked by conservative-

anti-government sentiment stoked by conservative-

anti-government

sentiment

stoked

by

conservative-

leaning protesters to advance their cause.

leaning protesters to advance their cause.

leaning

protesters

to

advance

their

cause.

(a) LIME

(b) SHAP

(c) Manipulated

Victim: Black, Al Prediction: 0, Label: 0, Targeted Decision: 0

There's somany fine niggasat my work I'm

There's somany fine niggas at my work I'm

There's

many

fine

niggas

mywork

I'm

surprised 1 aint been fired for sexual harassment

surprised aint been fired for sexual harassment

哆

surprised

1aint been fired for sexual harassment

(a) LIME

(b) SHAP

(c) Manipulated

Victim: Black, Al Prediction: 1, Label:1, Targeted Decision: 1

1wouldlovetos

suck

onthatbigblackcock

pinoM1

love

to

suck

on

that

tbig

black

cock

would

lovetosuck

on

that

big

black

cock

suckingall thecum out theheadall the time

sucking

allthe

cum

outtheheadallthe

time

sucking all the cum

out

the

headallthetime

that you can deliver why you fuckmy throat

thatyoucandeliverwhyyou

fuck

Au

throat

thatyoucandeliver

whyyoufuckmyt

throat

(a) LIME

(b) SHAP

Manipulated

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes] .

Justification: In this paper, we aim to explore whether we can model how humans incorporate both AI predictions and explanations into their decision making process, and whether we can utilize the quantitative understanding of human behavior from these models to manipulate explanations, thereby nudging human decisions in AI-assisted decision making. Through extensive human subject experiments, we showed the good, bad, and the ugly side of this. The claims in the abstract and the introduction accurately reflect the paper's contributions and scope.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes] .

Justification: We discuss the limitations of the work in the Conclusion section.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA] .

Justification: This paper does not include theoretical results.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes] .

Justification: We report the settings of the hyper-parameters for training behavior models and manipulating AI explanations in the paper.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [No] .

Justification: We will ensure that the collected human behavior data is used responsibly by implementing strict access controls to minimize potential risks associated with unauthorized use or misuse. Individuals who wish to have access to the data or code must apply for permission, which will only be granted to those who meet the necessary authorization criteria.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes] .

Justification: We describe training and test details in the paper.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes] .

Justification: The experimental results are accompanied by the 95% confidence intervals, and the statistical significance tests.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes] .

Justification: We used one RTX 4060 for behavior model training and AI explanation manipulation.

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes] .

Justification: We checked the ethics guidelines, and the research conducted in the paper conform with the NeurIPS Code of Ethics.

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes] .

Justification: We discuss the societal impacts of the work in the Ethical Consideration section.

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [No] .

Justification: We will ensure that the collected human behavior data is used responsibly by implementing strict access controls to minimize potential risks associated with unauthorized use or misuse.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes] .

Justification: We cited all datasets used in the paper.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA] .

Justification: The paper does not release new assets.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [Yes] .

Justification: We included the task interfaces, the experimental procedure, and the compen- sation structure in the paper.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [Yes] .

Justification: The study was approved by the Institutional Review Board of the authors' institution. To minimize the risk of leaving our study participants with inappropriate impressions of manipulated AI explanations, upon the end of the study, we provided a debrief session directly through the Prolific chat system. Each participant who received manipulated AI explanations in the study were individually contacted through the Prolific platform. In the chat-based session, we clarified that the AI explanations were manipulated and did not accurately represent the underlying rationales of the AI models. We emphasized that these explanations were intentionally biased for the purpose of the study, based on the specific task participants were involved in.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the manipulation of explanations in AI-assisted decision-making, presenting a comprehensive study that explores how human behavior models can be used to adjust explanations provided by AI systems. The aim is to understand if these manipulations can nudge decision-makers towards specific outcomes, which might be either beneficial or malicious.\n\nSTRENGTHS:\n1. The paper's focus on quantitatively modeling human behavior to manipulate AI explanations is highly novel and impactful. This approach not only extends the current understanding of AI-human interactions but also opens new avenues for both enhancing and securing AI-assisted decision-making systems.\n\n2.  The experiments conducted across various decision-making tasks provide a robust validation of the proposed models. The inclusion of both adversarial and benign manipulations allows for a balanced view of the potential impacts of this technology.\n\nWEAKNESSES:\n1. The scope of the experiments is restricted to tasks such as census and recidivism prediction, which may not adequately represent the complexities and stakes of decision-making environments in sectors like healthcare or finance. Expanding the range of tasks to include high-stakes decision-making could improve the generalizability of the results.\n2. The behavior model does not account for the inherent variability and noise in human decision-making, potentially oversimplifying the complexities of real-world human-AI interactions.\n2. The data and code are not immediately available for replication and further study, which could hinder the verification of the results and the advancement of the research.\n\nQUESTIONS:\nI still have a positive view of this work, and have the following questions:\n1. Could the authors please provide more detail on which covariates were found to be significant in the regression analyses? It would be helpful to understand not only which variables significantly impacted the model outcomes but also how they were selected and their relative influence on the dependent variables.\n2. Regarding the use of $\\sum_i e_i$ in Eq. 2 for evaluating consistency, I wonder why this summation was chosen over other potential methods. Could the authors clarify the rationale behind this choice?\n\nLIMITATIONS:\n1. Score-based explanations may not be adequate. Incorporating Large Language Models could introduce more versatile and comprehensive explanations.\n2. Include more diverse datasets, possibly extending to more complex data types like images or videos, to test the robustness of the behavior models across different AI applications.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-06-18"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes to train a computational model to predict how humans would respond to model predictions and their explanations to make the final decision. Using this model, the authors then demonstrate that it could be used to manipulate explanations for both good and bad purposes, specifically to steer human predictions toward the label that is likely to be correct, or make human decisions intentionally biased. Furthermore, the humans have very little idea that the explanations have been manipulated in both cases.\n\nSTRENGTHS:\nThis paper focuses on an important topic: the role of explanations in human decision making. \n\nThe presentation is generally clear and the writing structure is good. \n\nExtensive experiments are conducted to demonstrate the main arguments of the paper.\n\nWEAKNESSES:\n1. I noticed that the user study payment is only \\\\$1.2 base pay with a potential bonus of \\\\$1.0 (Sec. 4.2). The study consists of a tutorial, 5 predictions without AI assistance, 15 predictions with AI prediction and explanations, and an exit survey. Given that this study is deployed to US-based participants, the compensation is extremely meager: even at the federal minimum wage of \\\\$7.25 per hour, the \\\\$1.2 base pay would be equivalent to 10 min of work, which, given the user interface of Fig. A.1 and A.2, is extremely unlikely to be enough for all the tasks. Thus, I have serious concern about the ethics of the study, despite its IRB approval, and thus decide to request additional ethics reviewers for this paper. \n\n2. The use of computational model for human behavior is not novel, and the applications to human decision manipulation seems quite straightforward. \n\n3. Furthermore, it would be helpful have some additional analysis on the learned computation model itself, maybe with the help of various interpretability tools. For example, when is the human prediction most likely influenced by the provided explanation, and in what way? These quantitative and qualitative insights could be helpful to understand human behaviors better. \n\n4. For the \"benign\" use case of improving human model prediction, the authors looked at cases \"when the AI model decision is likely incorrect\" (Line 285). How is this \"likely incorrect\" determined? And if we know when the AI prediction is likely incorrect, why can't we simply fix/patch the AI model directly? The authors demonstrated that the human performance is better after explanation manipulation, compared to the original explanation in Fig. 3, but is the human prediction better than the \"fixed AI model\" performance?\n\nQUESTIONS:\nSee weaknesses. In addition, the reliance metrics of Fig. 3 are not defined here, and the readers need to search the referenced papers for definitions.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel method that manipulates human decision-making by manipulating AI explanations in human-AI interaction scenarios. By utilizing human behavior models and minimizing the cross-entropy function between human and AI agreement with constraint to generating the same AI recommendation outcomes, the authors investigate this method for adversarial and benign purposes. The results show that the manipulation in AI explanations significantly negatively affects human decisions in the four tasks with adversarial purpose and enhances human decision accuracy, over-reliances, and diminishes under-reliances. The paper also discusses how the human perception to the AI explanations varies with the manipulation. Overall, it provides a novel and practical way to intervene in human decision-making in human-AI interactions.\n\nSTRENGTHS:\n- The paper formulates the AI explanation manipulation problems, as optimization problems to minimize human-AI decision disagreement, with constraint to keep the original AI outcomes. This can indeed be applied in both positive or negative ways (in respective to adversarial and benign purposes in the paper). The authors discuss both sides of the methods and provide an implication for society about such methods.\n\n- The paper selects multiple AI explanation baselines and collects human data on multiple tasks, enhancing the robustness of the manipulation method and broadening potential social impacts in various scenarios. The paper also collects human perception, providing an extra view from human participants in subjective feelings.\n\nWEAKNESSES:\n- Only particular features are selected in each task. This may weaken the effectiveness of the manipulation. For example, in the census task, the paper selects 'gender' as the dimension of manipulation; while 'degree of education' or 'ages' can possibly be biased and effect human decision-making. Thus, the authors may need to address such representativeness (or probably other dimensions do not exhibit significant effects in manipulations).\n\nQUESTIONS:\n- My question is humans themselves may have a lot of biases, for example, peak and anchoring effects; or in risky decision-making, humans can have probability distortion. This is to say, for humans, the change of 'numerical values' may change their perception of certain things. Probably the manipulations here are not directly related to the importance. So how would the authors know whether the manipulation takes effect on the importance but not simple number perception? (For example, changing from positive to negative, or changing small numbers to large numbers; changing the rank in overall dimensions). This may require a baseline test, whether about the numerical values (in an irrelevant context) or about other measures of importance (e.g., relative rank, or a Likert scale on each dimension separately).\n\nLIMITATIONS:\n- The limitation of this paper as discussed, is only certain features in each task are selected to manipulate. It would reveal more about the societal impact if comprehensive features are considered. Moreover, the consideration of interactions between features is also helpful, though this does require a larger dataset.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors show that by modeling a human decision maker they can manipulate the provided information in ways that reliable influence their decisions towards even non-benign outcomes.\n\nSTRENGTHS:\n# originality\n\nManipulating Mturk works is a well studied area of research, but this is a unique approach and highlights the limitations of interpretability research using similar techniques.\n\n# quality\n\nThe results are statistically significant and done across a range of tasks which strengthens their claims. \n\n# clarity\n\nMost of the plots are readable and I think I was mostly able to understand the results after reading a few time.\n\n# significance\n\nThis is an interesting result, presenting an example of \"attacking\" these score based systems via manipulating the interpretably metrics is an important result for the XAI literature.\n\nWEAKNESSES:\n# originality\n\nManipulating workers on digital platforms is a well studied area.\n\n# quality\n\nReading the plots (fig 3) some of these results look to be supporting the null-hypothesis (all interventions have the same effect). Having a table giving the numbers and more details on the statistical tests would make the quality of the results much easier to judge. The results also are weak, in part this is due to a small sample size.\n\n# clarity\n\nIf found the plots difficult to read either both being too small and the lack of numbers makes analyzing them tricky. Could the authors include the results as a table in the appendix with the error ranges clearly laid out so we don't have to eyeball tiny error bars.\n\nI also had to refer to the appendix to understand the experimental procedure, making it clearer what exactly is happening in the experiments would greatly aid the readability of this paper.\n\n # significance\n\nLooking at figure B.2 it looks like the manipulation is mostly making the difference much more extreme and noticeable, this suggests these results could simply be due to the differences in the visual presentation between interventions.\n\nQUESTIONS:\nAre the manipulations working possibly just due to the bar plots being being bigger, and numbers being clearer? So no behavioral model needed.\n\nHow should I read figure 1? It looks like it's saying the means 95% confidence intervals overlap, but that they are also below 5% to overlap (bias 1a)?\n\nMore generally I think this result is weak, if people could tell the models apart this would be more interesting. As it stands I'm not surprised the authors can manipulate the responses, but I'm not convinced their method is close to optimal.\n\nLIMITATIONS:\nThis is an area where ethics should be given additional scrutiny, but I do not see anything alarming in the paper, and the authors take the correct tone in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}