Can LLMs Reason Soundly in Law? Auditing Inference Patterns for Legal Judgment
Authors: Lu Chen
Yuxuan Huang
Yixing Li
Dongrui Liu
Qihan Ren
ShuaiZhao
Kun Kuang
Zilong Zheng
Quanshi Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: Large Language Model
Value Alignment
Trustworthiness
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: This paper presents a method to analyze the inference patterns used by the Large Language Model for legal judgment.

Abstract: This paper presents a method to analyze the inference patterns used by Large Language Models (LLMs) for judgment in a case study on legal LLMs, so as to identify potential incorrect representations of the LLM, according to human domain knowledge. Unlike traditional evaluations on language generation results, we propose to evaluate the correctness of the  detailed inference patterns of an LLM behind its seemingly correct outputs.  To this end, we quantify the interactions between input phrases used by the LLM as primitive inference patterns, because recent theoretical achievements have proven several mathematical guarantees of the faithfulness of the interaction-based explanation.  We design a set of metrics to evaluate the detailed inference patterns of LLMs. Experiments show that even when the language generation results appear correct, a significant portion of the inference patterns used by the LLM for the legal judgment may represent misleading or irrelevant logic.

Published as a conference paper at ICLR 2026
CAN LLMS REASON SOUNDLY IN LAW? AUDITING INFERENCE PATTERNS FOR LEGAL JUDGMENT
Lu Chen' Yuxuan Huang"
Yixing Li Quanshi Zhang!
Dongrui Liu Qihan Ren'
Shuai Zhao'
Kun Kuang'
Zilong Zheng
Shanghai Jiao Tong University
The Chinese University of Hong Kong 3 SSanggaai Artificial Intelligence Laboratory
'Zhejiang University
Beijjing Institute for General Artificial Intelligence
renqihan, shuaizhas 2qs (0g21022]] @ edu. edu. en, yixingli@liie ink. cuhk. edu.hk, kunkuangez edu. en,
huangyuxuan@pjlab.cc orq.en, liudosgrui@pjlab org.on.... en, zi zilongzheng031 0etee/00008uuaaeeeee 8éucla.eda
ABSTRACT
This paper presents a method to analyze the inference patterns used by Large Language Models (LLMs) for judgment in case study on legal LLMs, so as to identify potential incorrect representations of the LLM, according to human domain knowledge. Unlike traditional evaluations on language generation results, propose evaluate the correctness of the detailed inference patterns of an LLM behind its scemingly correct outputs. To this end, we quantify the interactions between input phrases used by the LLM primitive inference patterns, because recent theoretical achievements (26 42) have proven several mathematical guar- antees of the faithfulness of the ierraction-bsed explanation. We design of metrics to evaluate the detailed inference patterns of LLMs. Experiments show that even when the language generation results appear correct, a significant portion of the inference patterns used by the LLM for the legal judgment may represent misleading or irrelevant logid
INTRODUCTION
have demonstrated state-of of-toe--ttt--n art performance on wide range of tasks. However, for high-stakes applications, only high accuracy of the generated outputs is still insufficient to ensure the reliability of L.MMs (37) (1) for the following main reasons. (I) We find that even when top-tier LLM generates correct tokens, the LLM still relies on problematic inference patterns to generate the next token. (2) In particular, for LLMs towards 35), such problematic inference patterns directly influence the choice of the LLM among multiple seemingly acceptable judgments, which constitutes an encroachment upon domains traditionally recognized as within judges' discretionary authority. Thus, this would introduce significant potential unfairness and risks.
Large language models (LLMs) (57 1: 24)
legal judgment 13
Therefore, this paper, we focus on LLMSs for legal judgment as a case study, as it serves as typical high-stakes application. We explore the intense problematic inference patterns used by an LLM for judgments, and discuss the potential harm these inference patterns. fact, the first problem in this study is whether an LLM's prediction can be faithfully decomposed into set of inference patterns. Recent works in explainable AI (47: 50 26 39 6) have demonstrated that the inference score of deep network can be faithfully represented by set of interactions between input features. As shown in Figure D an interaction extracted from an LLM captures a nonlinear relationship between
Quanshi Zhang the corresponding author. He is with the Department Computer Science and Engineering, the John Hopcroft Center, at the Shanghai Jiao Tong University, China.
'The names used in the legal cases follow an alphahetical convention, e.g., Andy, Bob, Charlie, etc., which do not represent any bias against actual individuals.

---

Published as a conference paper at ICLR 2026
AND-OR Logical Model ("Assault") (x)
LLM's output D(" D("Assaule"|a>
Reliable interaction effects
Unreliable inferaction effects
Equivalently
10 +
3321
05%
0454
modeling ling
APT 004652
62 1137
009224
12*h 8582
0/ 2992
Ruth
UP -1.300
IBaad
Wi
Highte lqaind
10 003945
+
Input legal case
82*h 3 1622
spt 617781
Universal matching property
++
guarantees the faithfunnes of explanation.
19%
daed
Juns
Output soores of the LLM e
1.43
datte scend
0,2155
2 -QU771
AND OR logical model ACK-)
(x.v(s))
Highh Warred
eC (424
dight apone
vis sanu
DKt ((x.))
Us, ((x,)))
(A+, (xx 1)
Modedd Unmantee
3440
3551
Fall
0453
1it Eog?
Aesoed
2 masted input logal case
Input legai
the (A) moming of December defendants Andy Bob deselved Charle three
phreses
segumiet, Andy chased (D) with (E) siihhty injure.
causing Charli
Inelevant phrases
Charle (M) shovel, injuring Charle causing Charlee's (J) deatis
Fortidden piriidei
Figure 1: Correctness of the detailed inference patterns of an LLM. The AND-OR logical model h(-) accurately fits the output score of the LLM v(-) when making the judgment "Assault" for Andy, h("Assault(|x) = v(" Assault"|x)). no matter how the input legal case x is masked in the bottom-right figure. black edges connect reliable interaction effects (RAND) and ROR) that contribute the output score v(" (("ssauur")> x). typically aligning with legal domain knowledge. Red edges connect unreliable interaction effects ANDD and UOR) that contribute v(" (""Assaul"" (x), often reflecting problematic patterns used by the LLM for the judgment.
input tokens, and contributes numerical score that quantifies their joint influence on the LLM's prediction.
Despite above theoretical achievements, in this paper, focus on crucial yet long-overlooked issue in the community, i.e., the correctness of detailed representations of an LLM. It is still unknown (!) how many problematic interactions are modeled in LLMs (e.g., legal LLMs), and (2) to what extent these interactions influence legal judgments.
In particular, we obtain three findings. (1) We find that over half of interactions actually represent clearly unreasonable or even incorrect justifications for the judgment predictions. (2) Although the appearance of long-chaad reasoning capabilities exhibited in thains-of-thought prompting (55) 29 [57]
37). we find that the essence is simple interactions of local tokens to guess judgments, even just like a bag-of-the-words model (36). (3) We find that LLMS tend to model large number of canceling interactions, where positive and negative contributions between input tokens offset each other, which often represent unreliable noise patterns.
Risk warning or Benchmark. We acknowledge that we cannot exhaustively analyze all legal ( but our objective is to provide sufficient examples to alert the deep learning community to the severity of representational flaws reflected by interactions encoded in LLMs. It is because our experiments show that the representation quality of current LLMs fails far short of supporting the benchmark evaluation of detailed interaction patterns. For example, Figure T shows LLMs use large
In addition to the large number of legal provisions, the variation in laws across countries presents another challenge.
2

---

Published as a conference paper at ICLR 2026
number of problematic interactions to make judgments, making it hard to compare the quality of interactions in different LLMs a meaningful way.
Instead, choose to illustrate a wide range of problematic interaction pattern.. While not exhaus- tive_! in this paper, let us simply introduce three common types of potential representational flaws frequently observed in LLMs: (1) LLMs tend to make judgments based semantically irrelevant phrases: (2) LLMs often make judgments using the behavior of incorrect entities; (3) LLMs tend to produce judgments that biased by identity discrimination.
Experiments have shown that even when the LLM generated correct target tokens, a significant portion of the interactions encoded by the LLM for the legal judgment are unreliable. This reflects significant yet long overlooked problem for LLMs.
EVALUATING DETAILED INFERENCE PATTERNS USED BY LLMS
2.1 PRELIMINARIES: EXTRACTING INTERACTIONS AS THEORETICALLI GUARANTEED INFERENCE PATTERNS
Recent advancements in explanation theory (66 42 6 have proven that using an AND-OR logical model can accurately match all varying outputs of an LLM on exponentially augmented inputs. Specifically, given input prompt x [x1, T22... , Inn] with n input phrases indexed by N = (1, ..., n). where each input phrase represents semantic unit, such as token, word, phrase/short sentence. Then, let v(x) R denote the sealar output score of generating a sequence of the target m tokens [/1, U2, ... .mm), as follows.
ply = Vel ve| Y! log
x(x) der
(I)
1- p(U = U4 |x, Y!reviu
where prrevious del'
[Vn, 12. - represents the sequence of the previous (1 - 1) tokens before
generating the 1-th token. p(u U/ x, prevvous) denotes the probability of generating the 1-th token.
In partieular, Yprreviuus
[]
Theorem 1 proves that given an input prompt x, the output score of the LLM u(x) can be well predicted/fitted by the following AND-OR logical model h(x), matter how we enumerate all 2 masked states of the input p
h mmnnk)))
h(b) +
Σ SERAAD
(S .
+
Σ
DR (S) (S)xmask) OOR
SEQOR
The AND trigger function A (Smmak)) E {0, 1} represents binary AND logic (also termed an AND interaction pattern) between input phrases of the masked sample Xmask in S. It returns all phrases in S are present (not masked) in Xmask's otherwise, it returns 0. TADD is the scalar weight. Here, [AAN 2 {S C N} represents the set of AND interaction patterns. b is a scalar bias.
The OR trigger function 1l [ (mmx)) E {0, 1} represents a binary OR logic (also termed an OR interaction pattern) between input phrases of the masked sample Xmask in S. It returns I when any phrase in appears (not masked) Xmaski otherwise, it returns O. I is the scalar weight. Here, QOR C N = (S C N) denotes the set of OR interaction patterns.
Theorem (Universal matching property, proof in Section the logical model are set to VS C NND def Ercs(-1)15 -17 vana (XT and g def
When scalar weights in
'We followed (26) obtain two discrete states for each input phrase, i.e., the masked and unmasked states. Therefore, given an input prompt with n phrases, there are 2" possible masked states of the input prompt. To obtain the masked sample xr, we replaced the embedding of each token in the input phrase E N \ with the baseline value by Rd where d is the embedding dimension of each token. The haseline value described (40). Please see Section[T.66f details.
trained
"The numerical effect of AND interaction pattern TAND is also known as the Harsanyi dividend (18 in the cooperative game theory.
3

---

Published as a conference paper at ICLR 2026
-Erçs(-1)(si-1 (xx(X))))), subject to Vana(X()) + (xc(XC)) v(xT), b e(xy)), then have
vT C u(x)) = l((T))
(3)
where XT is the masked sample that each input variable E N \ T is masked. v(xT) is the LLM's
scalar output score of the masked sample x SAAND 2 {S c N). SPOR 2 = (S C NJ.
Theorem D shows that an AND-OR logical model h(.)) Equation 2 can well predict/match all output score of the LLM v(-) on all 2n enumerated masked states of the input prompt It partially guarantees that we can roughly consider each AND-OR interaction logic in the logical model h()) represents an AND-OR inference pattern equivalently used by the LLM.
Sparsity of interaction patterns (inference patterns) and settings of SAAND and DOR, Another issue is the conciseness of explanation. To this end, Ren et al. (42) have proven that the logical model obtained in Theorem T can be compressed into a concise AND-OR logical model by pruning all interactions with almost zero weight IADD and g Specifically, given input prompt x with input phrases, there are only (nn)) interaction patterns have considerable numerical scores. All other interactions have negligible numerical scores, i.e., TAND, TOR Rs O. It is usually found 1.5 S < 2.0.
This guarantees a deep network to be explained concisely.
Interaction extraction (pseudo-code in Algorithm I. For implementation. the concise AND-OR logical model can be obtained by setting (x = (xxe + YT and For (xr.) = 0.5v(xx)) - TT in Theorem I with a set of learnable parameters for T S N). We follow (61 to learn the parameters for [T' S N]. and extract the sparest (the simplest) AND-OR interaction explanation
using a LASSO-like loss function, ie., min(1.) EscN,syo|l TAND + | All salient interactions
in S2AND def {S ç N : | > } and Q2OR def
= {SCN: |
} are selected to construct the
logical model for explanation, where is a small threshold.
2.2 RELEVANT PHRASES, IRRELEVANT PHRASES, AND FORBIDDEN PHRASES
In this subsection, we annotate the relevant, irrelevant, and forbidden phrases in the input legal case, in order to accurately identify the reliable and unreliable interaction effects used by the legal LLMs for the legal judgment (see Figure TT. Here, an input phrase can be set as a token, a word, or phrase.
Specifically, we engage 16 legal experts and volunteers to manually partition the set of all input phrases N into three mutually disjoint subsets, i.e., the of relevant phrases R, the set of irrelevant phrases I. and the set of forbidden phrases F. subject to RUZUF- N, with RnI = 0. RNF 0. and I F
according to their legal domain knowledge, as follows.
Phrase annotation. We first clarify principles to guide legal experts to annotate different types of phrases for judgments according to their legal domain knowledge.
(1) Generally speaking, relevant phrases refer to phrases that are closely related to, or directly contribute to the legal judgment result, based on their ground-truth relevance to the judgment result. For example, Figure [1 shows, there are 10 informative input phrases chosen by legal experts. Among them, R = [[chased], [with an axe]. [bir), [slightly injured] } are the direct reason for the judgment "Assaull" for Andy, thereby being annotated relevant phrases. In the computation of interactions, all tokens in the brackets [] are taken as single input phrase.
(2) Irrelevant phrases that directly contribute to the judgment result. For example, as Figure []] shows, I = [morning], [had an argument]] are not the direct reason for the judgment "Assault" for Andy, thereby being annotated as irrelevant phrases.
phrases that describe the defendant but not sensitive phrases
(3) Forbidden phrases are usually sensitive yet misleading phrases in the legal case, e.g., phrases describing incorrect defendant. For example, as Figure [ shows, FF = [hit), [with a shovel], [injuring], [death]} should not influence the judgment for Andy, because these words describe the actions and consequences of Bob, not actions of Andy, thereby being annotated as forbidden phrases for Andy.
'In particular, there senior legal experts who have heen practicing for over 10 years.
4

---

Published as a conference paper at ICLR 2026
Please see Section | for more examples of the annotated relevant phrases, irrelevant phrases, and forbidden phrases in real legal cases.
In particular, set up two principles to guide 16 legal experts and volunteers to annotate phrases to enable a convincing evaluation. Please see Section for detailed principles. We acknowledge that the above three types of phrases are not complete enumeration of all problematic phrases in legal cases. Instead, this paper just aims to illustrate the existence of large ratio of unreliable inference patterns used by the LLMs, rather than exhausting all potential issues with LLM.
2.3 RELIABLE AND UNRELIABLE INTERACTION EFFECTS
Since the scalar weight S/ AND (or ( denotes the numerical effect for the interaction (or called interaction effect for short), the annotation of relevant, irrelevant, and forbidden phrases enables us to decompose the overall interaction effects IAND and I in Theorem I into reliable effects
(R& and It5h) and unreliable effects (U and ) i.e., IAND
docompos:
RAND + U and
1OR decompose R + US. The absolute effect (IIGND) and ( termed the interaction strength.
In this way, align with human domain knowledge, and usually contain relevant phrases and exclude forbidden phrases. In contrast, unreliable interaction effects (UAND and U are defined as interaction effects that do not match human domain knowledge, which are attributed to irrelevant or forbidden phrases.
define reliable interaction effects (RSND and R$R) as interaction effects that
Reliable interactions and unreliable interactions. Figure I further provides an example of using AND-OR interactions to explain the inference patterns of a legal LLM. The legal LLM correctly attributes the judgment of "Assault" to interactions involving the relevant phrases "chased," "with an axe," " "bir," and "slightly injured." However, the legal LLM also uses the irrelevant phrases ("morning" and "had an argument"), and the forbidden phrases ("hit," "with a shovel," "injuring." and "death") to compute the output score of the judgment of "Assault." These irrelevant phrases do not directly contribute to the legal judgment result for Andy, and the forbidden phrases are actions and consequences that are not directly related to Andy, e.g., actions not taken by Andy. Obviously, the judgment should not rely on such inference patterns.
In this way, define reliable and unreliable interaction effects for AND-OR interactions, follows.
For AND interactions. Because the AND interaction S is activated only when all input phrases (tokens phrases) in S are present in the input legal case, the reliable interaction effect for AND interaction R w.r.f. S must include relevant phrases in R, and completely exclude forbidden phrases in F, i.e., S R F 0, S F 0. Otherwise, if S contains any forbidden phrases in F, or if S does not contains any relevant phrases in R, then the AND interaction 7AND represents an incorrect logic for judgment. In this way, the reliable AND interaction effects RAND and unreliable AND interaction effects U w.t.. S can be computed as follows.
if $nF=
then R I
U U R = O, U AND
AND
(4)
otherwise,
For OR interactions. The OR interaction 1 affects the LLM's output when any input variable (token or phrase) in 5 appears in the input legal case. Therefore, we can define the reliable effect Rg the numerical component in IOR allocated to relevant input phrases in S n R. To this end, just like in (10), we uniformly allocate the OR interaction effects to all input phrases in S. The reliable interaction effects R2R and unreliable interactions effects UOR are those allocated relevant variables, and those allocated to irrelevant and forbidden variables, respectively.
JS R S
(1
I R S
VS N,
R
1
U
(5)
In fact, such a uniform allocation of interaction effects to input phrases has sufficient theoretical supports and has been widely used, e.g., being used in the computation of the Shapley value (45
32).
In this way, according to Equation (2) with the setting Xmask = x, the output score of the legal judgment result uxx) can be formulated the sum of all reliable effects (RAND and ROR) that align
5

---

Published as a conference paper at ICLR 2026
Qwen2.5
Deepseek-RI RR1
SauILM
BAI-Law
gellabbe
0.4
0.2
LegalBench
LexGLUE
CAIL2018
LeCaRD
LEVEN
English
English
Chinese
Chinese
Chinese
Figure 2: Ratio of reliable interaction effects (measured by seeliable) among all the interaction patterns used by the L.LM for judgment. with human domain knowledge. and all unreliable effects (U and UgR) that do not match human domain knowledge.
v(x) = o(xo) +
Σ
RAND
FOR Σ
Σ
UAND +
Σ
US
(6)
SEOAND
SEQOR
SERAND
SEQOR
reliable interaction effects
unreliable interaction
Ratio of reliable interaction effects. We design metric to evaluate the alignment quality between the interaction patterns used by the legal LLM and human domain knowledge. Definition inttiouuuee the ratio of reliable interaction effects that align with human domain knowledge.
Definition 1 (Ratio of reliable interaction effects). Given LLM, the ratio of reliable interaction effects to all salient interaction effects s reliable is computed as follows.
ESEnAND
|RS
EsegoR
LRS
relliable
ESERAND
TAND
Σ
SEDOR
larger value of greliable [0, 1] indicates that higher proportion of interaction effects align with human domain knowledge, which means the judgment rationale of an LLM is more aligned with that of human experts.
EXPERIMENTS
In this section, we conducted experiments to evaluate the correctness of interaction patterns (inference patterns) used by the LLMs for legal judgments. Specifically, we identified and quantified the reliable interaction effects and unreliable interaction effects used by the LLM.
We evaluated the correctness of interaction patterns used by four LLMs: two cenerall-urposs LLMs. including -wwnn.5-11BBBBi (57), and Deepseek-R -----still--wenn 14B (29), and two law-specific
LLMs, including B-ulL-M-BB-Instruu (7), BAI-Law-13B (21). Among them, SaulL
Instruct was trained on English legal corpora, while BAI-Law- 13B was fine-tuned Chinese legal corpora.
Examining the faithfulness of the interaction- explanation. We conducted experiments to evaluate the sparsity property and the universal matching property of the extracted interactions in Section[] The successful verification of the two properties indicated that the intricate inference logic used by the L.MM for judgment on exponentially many masked input legal cases could be faithfully mimicked by the few extracted AND-OR interactions.
3.1 EVALUATING THE RELIABILITY OF INTERACTIONS USED FOR JUDGMENT
The disentanglement of reliable interaction effects and unreliable interaction effects provides new perspectives to analyze the representation quality of L.M.

---

Published as a conference paper at ICLR 2026
Qwen on LegalBench Qwen on CAIL2018
Deepseek on LeCaRD
Deepseek on LEVEN
All interactio
Pos
Neg
Reliable interactions
Figure 3: Distribution of all interactions over different orders (complexities) (denoted by A and A ne)) and that of all reliable interactions (denoted by A (o).pox reliable and A (olane))
Ratio of reliable interaction effects. Figure 2 compares the ratio of reliable interaction effects reliable used by different L.MMs for judgment. Specifically, for English legal tasks, we evaluated
Qwen, Deepseek, and MaulLM on legal cases from the ECtHR dataset in the LexGLUE benchmark (5) and the Learned Hand Crime dataset in the LegalBench benchmark (17). For Chinese legal tasks, we evaluated Qwen, Deepseek, and BAI-Law on cases from the CAIL2018 dataset (56), the LeCaRD dataset (33) and the LEVEN dataset (58). For each task, we evaluated 100 randomly selected samples, with 10 informative input phrases chosen by two senior legal experts with over 10 years of professional experience. Then, we invited group of 16 legal experts and volunteers to annotate each phrase as relevant, irrelevant, and forbidden phrases the input legal case. Please refer to Section L. for more implemental details.
Empirical results demonstrated that neither general- general-purpose L.MSs nor legal-domain-specffe LLMS exhibited sufficient reliable interaction effects. In particular, over half of interactions represented unreasonable or even incorrect justifications for the judgment predictions. This reminded us that although current LLMs conducted correct predictions legal tasks, their decisions relied large number problematic rationales, which could introduce significant potential unfairness and risks.
Complexity of interactions. We analyzed the complexity of interactions used for judgment. We used the order of an interaction, i.e., the number input phrases in |S], to represent the complexity of interactions. Specifically, let A A CApE
Eser S|]== max(0, 72)) and (
Eope (AND, Eserr (0, 1 to represent the strength of all positive o-order interactions and the strength of all negative negative o o-order interactions. Figure shows the histogram of (o).poos
and A (00)000 to represent the distribution of interactions over different orders (complexities). Similarly, computed the distribution of reliable interactions over different orders by quantifying A (o). no). refiable on reliable interactions [RAND,, .R ). in the same manner (please see Figure
reliable and
As evidenced in Figure 3 the LLM consistently demonstrated strong preference for using low- order interactions for legal judgments, regardless of whether we examined the distribution of all interactions the distribution of only reliable interactions. The low-order interactions mainly used local patterns on few input phrases to facilitate heuristic-based inference, rather than conducting comprehensive analysis of all case factors. This finding had challenged the prevailing assumption that LLMs possessed long-chain reasoning capabilities.
Significance of conflicted interaction patterns. Besides, we also quantified the significance of conflicts between different interaction effects. We can consider positive interaction effects as supporting evidence for generating the target tokens, while negative interaction effects serve as anti- anti-evidence. Therefore, we quantified the significance of such cancellation for interactions as
cconflctt Expe (AND,OR) Eserie 1 (AND, E (1 E [0, 1]. Table in Sec-
tion L.2 shows the significance of mutual cancellation interaction patterns. We found that roughly more than 60% effects of the interaction patterns had been mutually cancelled out. The mutually canceling interaction effects demonstrated the inherent ambiguity in an LLM's judgment. In contrast, more reliable large models typically exhibited lower cancellation level.

---

Published as a conference paper at ICLR 2026
201901221z2 aibth Wit R.B. M.
Judgment for Andy
Ratio of reliable interaction effects
AND-OR Logical Model & C'ntenttinall Injoryy [+]
Reliable internction effech- Unrvllable intrraction effects
JMB 4.. Mib 2
X/ 21849X 152E.
Assault
On the (A) momnnng December
22, 2013, the defencants Andy
(SauILM-7B Instruci)
58.5%
41.5%
Charle the
frree of them [B) had an argument.
Andy (C) chased Charli (D)
in 2o0 and (6) Crarle, causing
Cherlie to be (F) sighty iiuund.
(G)
Intentional
shovel. II) injuring Charie and
Injury
95.5%
of
causing Charlees (J) deads.
44.5%
(BAI-Lww138)
Relevant
Irelevant
Forbidden
Relevant phrases
Inelevant phroses
Forbidden phrases
(a) Input legal case
(b) Judgment
[c] Ratio
[d] Reliable and unreliable Interaction effects
Figure 4: Visualization of judgments affected by incorrect entities' actions. (a) Irrelevant phrases were annotated in the legal case, including the time and defendant's actions that were not the direct reason for the judgment. Criminal actions of the defendant were annotated as relevant phrases. Criminal actions of the unrelated person were annotated as forbidden phrases. (b) Judgments predicted by the two legal LLMs, which were both correct according to laws of the two countries. (c,d) We quantified the reliable and unreliable interaction effects.
3.2 CASE STUDIES
In this subsection, we visualized the interaction patterns on specific legal cases, and identified potential representation flaws of LLMs. While not exhaustive, let us introduce three common types of potential representation flaws frequently observed in LLMs: (1) making judgments using the behavior of incorrect entities, (2) making judgments influenced by identity-based discrimination, and (3) making judgments based on semantically irrelevant phrases. Due to the limit of the page number, we analyzed legal cases of the first and second types, and put results of the third type in Section K We tested legal LLMs SaulLM and BAI-Law make judgments on legal cases in the CAIL2018 dataset (56). For the SaulLM-7B ----ttrrreet model, we translated the Chinese legal cases into English and performed the analyses on the translated cases to enable fair comparisons.
Case making judgments based on incorrect entities' actions. Despite the high accuracy of legal LLMs in predicting judgment results, we observed that the legal LLMs used a significant portion of interaction patterns that were mistakenly attributed to criminal actions made by incorrect entities. In other words, the legal LLMs mistakenly used the criminal action of person (entity) to make judgment on another unrelated person (entity). To evaluate the impact of such incorrect entity matching both the SauILM and BAI-Law models. we engaged legal experts to annotate misleading phrases that described incorrect defendant as the forbidden phrases in F. These forbidden phrases should not influence the legal judgment for the target defendant.
Figure 4 shows the legal case, which showed Andy bit Charlic, constituting an assault, and then Bob hit Charlie with shovel, resulting in Charlie's death. Here, when the legal LLMs judged the actions of Andy, input phrases such as "hir," "with a shovel," "injuring." and "death" were annotated as forbidden phrases in F. because these phrases described Bob's actions and consequences and were not directly related to Andy. We observed that the SaulLM did use several interaction patterns which aligned with legal experts' domain knowledge for the judgment in Figure [ For example, an AND interaction pattern S ("slightly injured"), }, an AND interaction pattern S₂ {"bir"), and an OR interaction pattern S3 {"bit". "slightly injured") contributed salient reliable interaction effects RAND = 0.47, RAND 0.33, and ROR = 0.10, respectively. to the confidence score v( (("Assault") [x) ri the judgment "2 "Assaulr" for Andy. However, the legal L.MM also used a significant portion of problematic interaction patterns that based on incorrect entity's 's actions. For example, three AND interaction patterns S4 = {"death"), }. Ss = {"with a shovel"), }. and Sg- = {"injuring") that described Bob's 's actions and consequences contributed unreliable interaction effects UAND -1.04, USAD 0.93 and AND = 0.19 to the confidence score of the judgment "Assault" for Andy, respectively. In sum, the SaulLM model only used a ratio of reliable = 41.5% reliable interaction effects for the legal judgment. This reflected representation flaw, i.e., the LLM tended to memorize the sensitive tokens, such as the weapons, alongside the legal judgment results, rather than understand the true logic in the input prompt, e.g., identifying who performed which actions.
8

---

Published as a conference paper at ICLR 2026
Ruauaat nore mnderseeeddai V. Ke.
Judgment for suspicious men
reliobie interaction effects
AND-OR Logical Model
h CRRobbory
XEM xit BES. cudeer MB-
Relishle inseraction offerts
Unroliable internction
Robbery
18.6%
Dalee bark
Wedresday (B) night.
when the victim's
gellaale -1.44%
[occupation]
Andy, (C) locoupasion (D) walked
alono after firsshing
judge]
3 cay's work. On the (G) dank rcad.
two suspicious followe,
Andy's [d] batongings.
holking
and 0) mebbed
Not mentioned
16.0%
luaday shill
(C) (occupation) Is
when the victim's [occupation] ja volunteer)
5
84 0%
(sikal
J.dgge/ a volutteer
Retevant
Inrlevant:
Relevant phrioii
Inelevant phrases
(a) Input legal case
(b) Judgment
(c) Ratio
(d) Reliable and unreliable Interaction effects
Figure 5: Visualization of judgments biased by discrimination in identity. (a) Irrelevant phrases were annotated in the legal case, including the occupation, time and actions that are not the direct reason for the judgment. Criminal actions of the defendant were annotated as relevant phrases. (b) The SaulL In-stuuttt model predicted the judgment based on the legal case with different occupations. (c.d) We quantified the reliable and unreliable interaction effects.
In comparison, we evaluated the above legal case on the BAI-Law model, shown in Figure 4 The BAI-Law model used a bit higher ratio of geliable = 44.5% reliable interaction effects. Many interaction patterns used by the BAI-Law-13B model were also used by the SaulLM model, such as an AND interaction pattern S = {" (lilightly injured"), and an AND interaction pattern S₂ = {'bir"). and OR interaction pattern S3 {"bit", "slightly injured") contributed salient reliable interaction effects RAND = 0.33, R = 0.17, and RºR = 0.06 to the confidence score u( e("Intentional Injur"|x/ x) of the judgment "Intentional Injury" for Andy, respectively. This indicated that these two legal L.MS did identify some direct reasons for the legal judgment. However, the BAI-LAW-131 model also primarily relied on unreliable interaction effects for the legal judgment on Andy, such as an AND interaction pattern S4- = { ("death"), }. which included forbidden phrases for the consequence of Bob's ' actions, to contribute unreliable interaction effects U'ADD -0.43 to the confidence score of the judgment "Intentional Injury" for Andy. Additional examples of making judgments based on incorrect entities' actions are provided in Section L.4.
Case 2: discrimination in identity may affect judgments. We observed that the legal LLMs used interaction patterns that were attributed to the occupation information. This would lead to significant occupation bias. More interestingly, we observed that when we replaced the occupation phrase with another occupation phrase, the unreliable interaction effect containing the occupation phrase would be significant changed. This indicates a common identity bias problem, because similar bias may also happen on other identities (e.g., age, gender, education level, and marital status).
Figure 5 shows the legal case, in which Andy was robbed of his belongings by two suspicious men. The SauILM used several interaction patterns that aligned with legal experts' domain knowledge for the legal judgment, e.g., an AND interaction pattern S1 = ("robbed") }. and an OR interaction pattern S2 ("robbed"' "belongings",, and an OR interaction pattern S3 = {"holding knife". "robbed"") contributed salient reliable interaction effects to the confidence score "i "obbery") x) of the judgment "Robbery:" However, the legal LLM also used problematic interaction patterns, i.e., an AND interaction pattern S4 = {"a judge"} for the occupation information contributed salient unreliable interaction effects U AND 0.19 to boost the output score of the judgment.
More interestingly, if we substituted Andy's occupation from the phrase "a judge" to "a volunteer," the interaction pattern S3 [[[ccuppaiio]]" "a day's 'ss's work". "belongings") decreased its reliable interaction effects from RAND = 0.22 to RAND = 0.06 (see Figure I1 in Appendix). The interaction patterns containing the occupation phrase were important factors that changed the legal judgment result from "Robbery" to "Not mentioned." We verified similar phenomena on different occupations, e.g. substituting the occupation phrase with law-related occupations such as "a lawyer" and "a policeman" also maintained the judgment result, while the other occupations such as "a programmer" changed the judgment to "Not mentioned." Please see Section L.5 for reliable and unreliable interaction effects for all these occupations. This suggested considerable occupation bias. In comparison, we evaluated the same legal case on the BAI-Law in Section L.3] This experiment showed the potential of our method to identify the identity (e.g., occupation) bias used by the LLM.
9

---

Published as a conference paper at ICLR 2026
4 CONCLUSIONS AND DISCUSSION
In this paper, we proposed a method to evaluate the correctness of the detailed inference patterns used by an LLM. The universal matching property and the sparsity property of interactions provide mathematical support for the faithfulness of interaction- based explanations. Thus, in this paper, we designed new metrics to identify and quantify reliable and unreliable interaction effects. Experiments showed that the legal LLMs often used a significant portion of problematic interaction patterns to make judgments, even when the legal judgment prediction appeared correct. The evaluation of the alignment between the interaction patterns of L.I.s and human domain knowledge has broader implications for high-stake tasks, such as finance and healthcare data analytics, although we focus on legal LLMs case study.
ACKNOWLEDGMENTS
This work is partially supported by the National Nature Science Foundation of China (92370115, 62276165), and Shanghai Natural Science Foundation (242R1491700).
REFERENCES
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.
[2] Marco Ancona, Cengiz Oztireli, and Markus Gross. Explaining deep neural networks with polynomial time algorithm for shapley value approximation. International conference on machine learning. pages 272-281. PMLR, 2019.
[3] Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.0800 2022.
[4] Ilias Chalkidis, Ion Androutsopoulos, and Nikolaos Aletras. Neural legal judgment prediction in English. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4317-4323, Association for Computational Linguistics, 2019.
[5] Ilias Chalkidis, Abhik Jana, Dirk Hartung, Michael Bommarito, Ion Androutsopoulos. Daniel Martin Katz, and Nikolaos Aletras. Lexglue: benchmark dataset for legal language understanding in english. arXiv preprint (.Xiv:2/1000097:: 2021.
[6] Lu Chen, Siyu Lou, Benhao Huang, and Quanshi Zhang. Defining and extracting generalizable interaction primitives from dnns. International Conference on Learning Representations, 2024.
[7] Pierre Colombo, Telmo Pessoa Pires, Malik Boudiaf, Dominic Culver, Rui Melo, Caio Corro, Andre Martins, Fabrizio Esposito, Vera Lúcia Raposo, Sofia Morgado, and Michael Desa. Saullm-7b: A pioneering large language model for law. arXiv preprint MXXv:2443300888. 2024.
[8] Junyun Cui, Xiaoyu Shen, and Shaochun Wen. A survey on legal judgment prediction: Datasets, metrics, models and challenges. IEEE Access, 2023.
[9] Huiqi Deng, Qihan Ren, Hao Zhang, and Quanshi Zhang. Discovering and explaining the representation bottleneck of dnns. In International Conference on Learning Representations, 2022.
[10] Huiqi Deng, Na Zou, Mengnan Du, Weifu Chen, Guocan Feng, Ziwei Yang. Zheyang Li, and Quanshi Zhang. Unifying fourteen post-hoc attribution methods with taylor interactions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
[11] Jay Young Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, and Byron C Wallace. Eraser: A benchmark to evaluate rationalized nip models. In Proceedings of the 58th annual meeting of the association for computational linguistics, pages 4443 4458, 2020.
10

---

Published as a conference paper at ICLR 2026
[12] Nouha Dziri, Andrea Madotto, Osmar Zaiane, and Avishek Joey Bose. Neural path hunter: Reducing hallucination in dialogue systems via path grounding. In Proceedings of the 2021 Conference Empirical Methods in Natural Language Processing, 2021.
[13] Zhiwei Fei, Xiaoyu Shen, Dawei Zhu, Fengzhe Zhou, Zhuo Han, Songyang Zhang, Kai Chen, Zongwen Shen, and Jidong Ge. Lawbench: Benchmarking legal knowledge of large language models. arXiv preprini arXiv:2309. 2023.
[14] Katja Filippova. Controlled hallucinations: Learning to generate faithfully from noisy data. Findings of the Association for Computational Linguistics: EMNLP, 2020.
[15] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 2024.
3 herd of models. arXiv preprint arXiv:
[16] Neel Guha, Julian Nyarko, Daniel Ho, Christopher Ré, Adam Chilton, Alex ChhollasWWood Austin Peters, Brandon Waldon, Daniel Rockmore, Diego Zambrano, et al. Legalbench: A collaboratively built benchmark for measuring legal reasoning in large language models. Advances in Neural Information Processing Systems, 36, 2024.
[17] Neel Guha, Julian Nyarko, Daniel E. Ho, Christopher Re, and et al. Legalbench: A collabora- tively built benchmark for measuring legal reasoning in large language models. In Advances in Neural Information Processing Systems ('Track on Datasets and Benchmarks), 2023.
[18] John C Harsanyi. A simplified bargaining model for the n-person cooperative game. Interna- lional Economic Review, 4(2):194-220, 1963.
[19] Lei Huang, Weijiang Yu, Weitao Ma. Weihong Zhong, Zhangyin Feng, Haotian Wang. Qian- glong Chen, Weihua Peng. Xiaocheng Feng, Bing Qin, et al. A survey on hallucination in large language models: Principles, taxonomy. challenges, and open questions. arXiv preprint arXiv:2311.05232. 2023.
[20] Yichong Huang, Xiachong Feng, Xiaocheng Feng, and Bing Qin. The factual inconsistency problem in abstractive text summarization: survey. arXiv preprint arXiv:2 14839, 2021.
[21] Baiyulan Open Al Research Institute. Baiyulan open ai. 2023.
[22] Jiaming Ji, Tianyi Qiu, Boyuan Chen, Borong Zhang, Hantao Lou, Kaile Wang, Yawen Duan, Zhonghao He, Jiayi Zhou, Zhaowei Zhang, et al. Ai alignment: A comprehensive survey. arXiv preprint 23.. 19852, 2023.
[23] Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. Survey of hallucination in natural language generation. ACM Computing Surveys (CSUR), 55(12), 2023.
[24] Gueyoung Jung, Matti A Hiltunen, Kaustubh R Joshi, Richard D Schlichting, and Calton Pu. Mistral: Dynamically managing power, performance, and adaptation cost in cloud infrastruc- tures. In 2010 IEEE 30th International Conference on Distributed Computing Systems, pages 62-73. IEEE, 2010.
[25] Jan Leike, David Krueger, Tom Everitt, Miljan Martic, Vishal Maini, and Shane Legg. Scalable agent alignment via reward modeling: research direction. arXiv preprint arXiv: 1.000771, 2018.
[26] Mingjie Li and Quanshi Zhang. Does a neural network really encode symbolic concept? In International Conference on Machine Learning. 2023.
[27] Yuchen Li, Yuanzhi Li, and Andrej Risteski. How do transformers learn topic structure: Towards mechanistic understanding. In International Conference on Machine Learning, pages 19689-19729. PMLR, 2023.
[28] Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computa- tional Linguistics (ollume 1: Long Papers), 2021.

---

Published as a conference paper at ICLR 2026
[29] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v2 v3 technical report. arXiv preprint arXiv:2412.14437, 2024.
[30] Dongrui Liu, Huiqi Deng, Xu Cheng, Qihan Ren, Kangrui Wang, and Quanshi Zhang. Towards the difficulty for deep neural network to learn concepts of different complexities. Advances in Neural Information Processing Systems, 2024.
[31] Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilaequa, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguisties, 12:157-173, 2024.
[32] Scott Lundherg and Su-In Lee. A unified approach to interpreting model predictions. Advances in neural information processing systems, 2017.
[33] Yixiao Ma, Yunqiu Shao, Yueyue Wu, Yiqun Liu, Ruizhe Zhang, Min Zhang, and Shaoping Ma. Lecard: legal case retrieval dataset for chinese law system. In Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval, pages 2342-2348, 2021.
[34] Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. On faithfulness and fac- tuality in abstractive summarization. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020.
[35] Masha Medvedeva, Martijn Wieling, and Michel Vols. Rethinking the field of automatic prediction of court decisions. Artificial Intelligence and Law, 31(1):195-212, 2023.
[36] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. arXiv preprint 1xXiv:1301.378/1
[37] OpenAl. Gpt-4 technical report. arXiv preprint arXiv:2303.0877 2023.
[38] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730-27744, 2022.
[39] Jie Ren, Mingjie Li, Qirui Chen, Huiqi Deng, and Quanshi Zhang. Defining and quantifying the emergence of sparse concepts in dnns. IEEE Conference on Computer Vision and Pattern Recognition, 2023.
[40] Jie Ren, Zhanpeng Zhou, Qirui Chen, and Quanshi Zhang. Can we faithfully represent masked states to compute shapley values dnn? International Conference Learning Representa- lions, 2023.
[41] Qihan Ren, Huiqi Deng, Yunuo Chen, Siyu Lou, and Quanshi Zhang. Bayesian neural networks avoid encoding verturbation-sensitte and complex concepts. International Conference on Machine Learning, 2023.
[42] Qihan Ren, Jiayang Gao, Wen Shen, and Quanshi Zhang. Where we have arrived in proving the emergence of sparse interaction primitives in dnns. In International Conference on Learning Representations, 2024.
[43] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. "why should trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135-1144, 2016.
[44] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision, pages 618-626, 2017.
[45] Lloyd S Shapley. A value for n-person games. In Contributions to the Theory of Games, 2(28)3377331,, 1953.
12

---

Published as a conference paper at ICLR 2026
[46] Wen Shen, Lei Cheng, Yuxiao Yang, Mingjie Li, and Quanshi Zhang. Can the inference logic of large language models be disentangled into symbolic concepts? arXiv preprint arXiv:2304.01083, 2023.
[47] Mukund Sundararajan, Kedar Dhamdhere, and Ashish Agarwal. The shapley taylor interaction index. In International conference on machine learning, pages 9259-9268. PMLR, 2020.
[48] Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In International conference on machine learning, pages 5319-3328 PMLR, 2017.
[49] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint a1X1v.2302.13971, 2023.
[50] Che-Ping Tsai, Chih-Kuan Yeh, and Pradeep Ravikumar. Faith-shap: The faithful shapley interaction index. Journal of Machine Learning Research, 24(94):1-42, 2023.
[51] Cunxiang Wang, Sirui Cheng, Qipeng Guo, Yuanhao Yue, Bowen Ding, Zhikun Xu, Yidong Wang, Xiangkun Hu, Zheng Zhang, and Yue Zhang. Evaluating open-qa evaluation. Advances Neural Information Processing Systems, 36, 2024.
[52] Yufei Wang, Wanjun Zhong, Liangyou Li, Fei Mi, Xingshan Zeng, Wenyong Huang, Lifeng Shang, Xin Jiang, and Qun Liu. Aligning large language models with human: A survey. arXiv preprint arXiv:2307. 12966, 2023.
[53] Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How does Ilm safety training fail? Advances in Neural Information Processing Systems, 36:80079-800 10, 2023.
[54] Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How does IIm safety training fail? In Advances in Neural Information Processing Systems, volume 36, 2023.
[55] Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. Chain-of-thouglle prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824 24428724-24877, 2022.
[56] Chaojun Xiao, Haoxi Zhong, Zhipeng Guo, Cunchao Tu, Zhiyuan Liu, Maosong Sun, Yansong Feng, Xianpei Han, Zhen Hu, Heng Wang, et al. Cail2018: A large-scale legal dataset for judgment prediction. arXiv preprint arXiv: /00.:0007.7777 2018
[57] An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024.
[58) Feng Yao, Chaojun Xiao, Xiaozhi Wang, Zhiyuan Liu, Lei Hou, Cunchao Tu, Juanzi Li, Yun Liu, Weixing Shen, and Maosong Sun. Leven: A large-scale chinese legal event detection dataset. arXiv preprint arXiv:2203.0855: 2022.
[59] Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Zhibo Sun, and Yue Zhang. A survey on large language model (Ilm) security and privacy: The good, the bad, and the ugly. Tigh-Confidence Computing, page 100211, 2024.
[60] Quanshi Zhang, Xin Wang, Jie Ren, Xu Cheng, Shuyun Lin, Yisen Wang, and Xiangming Zhu. Proving common mechanisms shared by twelve methods of boosting adversarial transferability. arXiv preprint arXiv:2207. 1694, 2022.
[61] Huilin Zhou, Hao Zhang, Huiqi Deng, Dongrui Liu, Wen Shen, Shih-Han Chan, and Quanshi Zhang. Explaining generalization power of a dnn using interactive concepts. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 17105-17113, 2024.
[62] Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.1500 15043, 2023.
13

---

Published as a conference paper at ICLR 2026
A THE USE OF LARGE LANGUAGE MODELS (LLMS)
In this paper, large language models (LLMs) were used solely for partial language refinement.
B RELATED WORK
Previous works have evaluated different aspects of trustworthiness and safety in LLMs, including factuality and hallucination problems, value alignment, and susceptibility to attacks. First, the evaluation of factuality refers to whether the language generalization results of LLMs align with
the verifiable facts (28 37 51) Hallucination in LLMs typically arises when the generated results
contradict the source material or cannot be verified from the provided input (14 34 [9] (20 12: 23). Second, value aligmment aims to ensure LLM to behave in accordance with human intentions and values 25 52 22), which is another classical perspective for evaluating the bias and safety of LLMs.
Recent studies have used Supervised Fine-Tuning (SFT) B8 37) and Reinforcement Learning from Human Feedback (RLHF) (38 37: 49) to align LLM's behavior with human expectations. Third, susceptibility attacks is also another significant concern for LLMs. Recent studies have shown that even the latest LLMs remain vulnerable to adversarial inputs to generate harmful content (62 54
37), which is also known as "jailbreaks."
However, above evaluation methods mainly focus on the quality or correctness of output results of LLMs. The high accuracy of the LLM usually makes the evaluation a long- search for incorrect results.
In comparison, our evaluation approach examines the correctness of internal interaction patterns. Even when the LLM outputs correct results on a testing sample, experimental results show that more than half detailed interaction patterns encoded by the legal LLM may still represent chaotic features. Thus, we can consider the interaction pattern as much more efficient evaluation strategy. Our goal to enhance the trustworthiness of the L.LMs, particularly in high-stake tasks. Essentially, two types of evaluation strategies can be roughly analogized to the distinction between procedural fairness and outcome fairness.
Reviewing the development of the interaction explanation theory. A representative approach in explainable AI was to explain the interactions between input variables (47) 50). Based on the game theory, (39) first used the Harsanyi dividend (18) quantify the the interaction effect between input variables encoded by the DNN. Then, (26) discovered and (42) further proved that the output scores of DNNs be faithfully explained as small number of interaction patterns between input variables. Furthermore, 9 30 41) further demonstrated the representation bottleneck of different neural networks from the perspective of interactions, i.e., proving interactions of specific complexities are difficult for specific DNNs to encode. (6I) explored the relationship between the complexity of interactions and the generalization power of DNNs. Additionally, (10) proved that the interaction theory provides unified explanation for mathematical mechanisms of 14 most widely used attribution methods, including Grad-CAM (44), Integrated Gradients (48), and Shapley
values 455 (2). (60) proved that the interaction theory provides unified explanation for the shared mathematical mechanism of 12 classical wransferabilitybboostin methods.
C PROOF OF THEOREM
Theorem I (Universal matching property) When scalar weights in the logical model are set to AS NN PAND def Ercs(-1)\st-1r V (xT) and FOR def -Ercs(-1) 10 (xxv)) subject to the requirement Vanal (x)) + Mor (xrr o(XT), then have VT o(x))) = h(xr).
In other words, we have to prove the following theorem.
Given input sample x, the network output score v(xr) E RR on each masked sample {xy IT C NN can be well matched by a surrogate logical model h(xr) on each masked sample (x7 C N). The surrogate logical model h(xx) uses the sum of AND interactions and OR interactions to accurately
14

---

Published as a conference paper at ICLR 2026
Confidence the LLM
(
AND-OR logical modelh(s;)
(x.
(Pty U(Fr,)
((x,
Naviel Uniosled
OR
QR
AND
AND
G
madied sentencess By
Input legal case
AAdy (8) threatened (C) Bob (D) and (E) took (F) his (G) smartphone.
LLMSS output {("oobery"(xx)
Figure 6: Theorem proves that the AND-OR logical model ht-) can accurately match the confidence score of the LLM's outputs v(-) when we augment the input prompt x by enumerating its all 2 masked states. Here, the left figure shows that the masked input prompt XT, with two unmasked token "took" and "smartphone" activates an AND interaction pattern S = {"took",, "smartphone") and an OR interaction pattern S = {"his", "smartphone"); and they contribute numerical values (interaction effects) to the logical model h(xT,), ). The right figure shows that the logical model can always match the LLM's outputs on all masked states of the input prompt, VT' S N, h(" ((("Robbery" "Robbery"(xy) xr) v( ("Robbery"[xx)) (x.)
fit the network output score v(XT).
VT _ N, exx)) = h(xT).
h(xT) = v(xo)= + Σ
[SND() -
AND
Σ
[oR(S| -
SCN,5
SCN,5*0
(8)
= =((x)= + Σ
TAND
Σ SCN,SnTy
OR
SCT,SyD
Mer())))))
Proof. Let us set surrogate logical model h(xp) = v(XT). VT S N, which utilizes the tan
sum of AND interactions TAND Ersss(1)1Si-MMumaall Vand (XT) and OR interactions T -Ers(-1))SS--1 (xr(x to the network output score u(XT), i.e., Vana + var var(xT) = v(TT).
To be specific, (1) we use the sum of AND interactions 1 AND to compute the component for AND
interactions (XT), i.e., Vand (xmm(xx) Esgr TAND, (2) Then, we use the sum of OR interactions
TOR compute the component for OR interactions vor (xx)), i.e., Vor (Xr) Esgn,snz I (3) Finally, we use the surrogate logical model h((-) (which uses the sum of AND interactions and OR
interactions) to fit the network output score ut-), i... VT C N, (xT) + Vor vx((X)) v(xT) h(xT).
(1) Universal matching property of AND interactions.
(39) first used the Harsanyi dividend I in the cooperative game theory (18) to state the universal matching property of AND interactions. The output score of well-trained DNN on all 2" masked samples {xr [ C N} could be universally explained by the all interaction patterns in T C N,i.e,
VT C N, Vand Vand((7) (XT) Escr TAND
Specifically, the AND interaction (as known as Harsanyi dividend) defined as IADD Eigs(-1)isi-14 (anaax))) (x)). To compute the sum of AND interactions VT C N, V.EEcr 1 first exchange the order of summation of the set T. S T
Escr E1cs(-1)151-141u.) and the set S 2 L. That is, we compute all linear combinations of all sets S containing L with respect to the model outputs (x,,) given set of input phrases L, i.e., (11))s Then, we compute all summations over the set L T.
Unna ).
In this way, we can compute them separately for different cases S S C In the following, we consider the cases (1) L = S = T, and (2) L S S S L, # T, respectively.
(1) When L output Vand xx.) 1.) is 1 pana (x,)
the linear combination of all subsets containing with respect to the model Vand (x,.).
15

---

Published as a conference paper at ICLR 2026
(2) When L S S + T, the linear combination of all subsets S containing L with respect to the model output Vand (x..) ) is Ls.csscr( -11)51111mml
For all sets S T 2 S 2 T,, let us consider the linear combinations of all sets S with number |S| for the model output Fand (xt). respectively. Let m := |S| - - (0 - m - [T] ILD, then there are a total of ( combinations of all sets S of order |S]. Thus, given I., accumulating the model outputs Vand (xz) corresponding to
[71-1&) 1 (-1) (-1))) 0.
all S 2 then Es.gsscr(11) 11--111111111111 Uand (x
Vand (xL)
Please the complete derivation of the following formula.
Esgr 1D Eser Enss( (-1)(51-11 Vand (XL)
Ener Esicscrr (-1)15)- esn (x)()
[27-14)
(9)
(XT
+EIgg,eer (xmd(X))
C 171-111 (-1)
m=0
L=T
Unnd (XT)).
Furthermore, we can understand the above equation in a physical sense. Given masked sample Xy. if Xy triggers an AND relationship S (the co-appearance of all input phrases in S), then T. Thus, we accumulate the interaction effects IADD of any AND relationship S triggered by Xy as follows,
o(xp)+ + Σ
[S|D() 15
SCN,S*0
=xxo)= + Σ SCT,S IND
(10)
Σ SCT
IND
Unnd (XT).
(2) Universal matching property of OR interactions.
According to the definition of OR interactions, we will derive that VT S N, vor
Es S s.t., 10 = vor (xx) = 0.
Specifically, the OR interaction is defined as oO -I (XNN..). To compute the sum of OR interactions VT c SSN. 1
EsCA, [ --Ecss(-D)11--4aaaaaa
we first exchange the order of summation of the set L C $ C N and the set S S 7 0. That is, we compute all linear combinations of all sets containing with respect to the model outputs Vor (xNN),) given a set of input phrases L, i.e., (-1))19
Then, we compute all summations over the set L C N.
In this way, we can compute them separately for different cases of I. C S C N, n T * 0. In the following, we consider the cases (1) T. T, (2) I. N. (3) LOT # 0, L + N, and (4) LNT 0, L * N \ TT, respectively.
(1) When
N \ T. the linear combination of all subsets containing with respect to the model Esnr (1 For all sets S 2 L, ST.TT # & (then S * N T, & L), let us consider the linear combinations of all sets S with number |S] for the model output for (x respectively. Let |S' |S| LL. (1 |5' < (T), then there are a total of [T] combinations of all sets S of order |S). Thus, given L, accumulating the model outputs Upe(Xy) corresponding to all S 2 I,, then EsnryA,S2L (-1 | 11 n. (XN
output Var (X is Σ S
Cor (XT)
Σ =
cls'l (-1)1S1) = - (xT))
16

---

Published as a conference paper at ICLR 2026
(2) When L = N (then S = N), the linear combination of all subsets S containing L with respect to the model output var (x is (-1r))),ss(
=
Vor (xp)) = (18 Uor (xx) 0).
(3) When LnT + 0, + N, the linear combination of all subsets containing 7. with respect to the model output Uor (XN,L) is EssT=40s22 ( (xx,)))) For all sets S 2 S nT + 0. let consider the linear combinations of all sets S with number |S| for the model output Vor (XT), respectively. Let us split |S| - Li into (S' and |S"), i.e.|S] - [4| = |S + |S"). where S" iili E S,i L,
N T) S" = {ili E S, & L, T} (then [T] - I LD and S + S + L = S. In this way, there are a total of fr -Irz combinations of all sets S" of order |s Thus, given L, accumulating the model outputs Vor (XNVL)) corresponding to all S 2 L, then EsnTY4,s21 -- (x(
0 < |S"| -
c
|T|- TTLL cls" (Ti (-1) (-1) (-1))s"++1511 =0
Por (xr(XN)))) ESENATAL
|S"|=0
(4) When LNT = O. + N T, the linear combination of all subsets S containing L with respect to the model output eor (xNN,)) (.) is (5.5)
vo (xN),1)). Similarly, let us split [S] - L| into [S"] and ie..))) - LL = +5"1, where S = {ili E S,i & L,i E N (T). S" = (ili E S E T} (then S |S"| S (TD and S' + S" + L = S. In this way, there
total of c|S"' combinations of all sets S" of order |. Thus, given I,, accumulating the model outputs Vor (x),,) corresponding to all S 2 L, then EsnT (-1)) -14 (XNLL))
Vor (XN(L)) ESCNATA Σ - (-1)1s1+15"1 = 0. =S
(T]
Please see the complete derivation of the following formula.
E Esgr.rrye
1 EscN.snTyo ( 11-1))1111 vnr (2)
=-EneN E
Enn (-1)18) 1101 Vor (XN1LL)
c/S1 (-))sS)
(xT( - Par(xp)
L=NTT
L-N
] Var (x(())
Σ
Σ
Σ
1 0 (1))) 11111)1111111)))
LAN
(S'NNTTYL
|S
ITI
]
Σ
Σ S'ENTTIL
clSs (-1) (11115
18/1=0
---11 1101)--e((xx (x() va((xa)
Σ
0 (xr())
LOT
S'ENATTE
Σ
Σ
(xN())
LAN
S'ENATLL
= (x()
(11)
Furthermore, we can understand the above equation in a physical sense. Given a masked sample Xy, if Xy triggers an OR relationship S (the presence of any input variable in S), then SnT * 0, C N.
17

---

Published as a conference paper at ICLR 2026
Thus, accumulate the interaction effects I of any OR relationship S triggered by xy follows,
Σ (S(xT) -IS
S
(12)
SCN,SNTY9
Vor Vor(X (XT).
(3) Universal matching property of AND-OR interactions.
With the universal matching property of AND interactions and the universal matching property of OR interactions, we can easily get o(XT) = (xr) = (x) = v(xp) +0)+++2 +SgT,S INDD
E Snn,, S thus, we obtain the universal matching property of AND-OR interactions.
D SPARSITY PROPERTY OF INTERACTIONS
The surrogate logical model h(xr) on each randomly masked sample xy, T S N mainly uses the sum of a small number of salient AND interactions QAAN and salient OR interactions QOR to approximate the network output score u(XT).
n (XT) = -(xx)==xxxxx + Σ
LAND(S)XTT)
LAND Σ
LoR (SR(S)) 1
(13)
SECAAD
SEnOR
Proof. (42 have proven that under some common conditions the confidence score Vand (ana(Xr) (xr) of
well-trained DNN on all 2" masked samples {xT(T' C N could be universally approximated by small number of AND interactions T E SAND with salient interaction effects IAND, s.t., |SPAND| << 2n
i.e., VT C N, "xnn(xx)) = Escr TAND ~ EScT:SERAN TAND
According Equation 100, Vand (XT) = Essrr AND v(xa) EsCN.s AND((S|x)) IAND.
Therefore, Vand (xT) = xxx)) + Σ x .AND.
SERAND
Besides, as proven in Section [ the OR interaction can be considered as a specific AND interaction.
Thus, the confidence score Vor (XT) of well-trained DNN on all 2n masked samples {x7 [T C N} could be universally approximated by small number of OR interactions T E NORR with salient interaction effects /OR s.t., [S2OR] < 2". Similarly, Vor (xr) EscN,s (5xx)) I
Σ (S)x))) 1 SEQTOR
In this way, the surrogate logical model h(xT) on each randomly masked sample x7, T C N mainly uses the sum of small number of salient AND interactions and salient OR interactions to approximate the network output score v(xT), i.e., v(x7) h(x7) (xnd(XT) Vor (XT) RJ
xxx)) + Σ LAND(S]xT) TAND + Σ LLO (OR(())))) -I
SEQAND
SEQORR
E THE CORRECTNESS OF THE DETAILED INFERENCE PATTERNS OF AN LLM
Unlike traditional studies focused on the correctness of language generation results, this paper is driven by a different motivation, i.e., evaluating the correctness of the detailed inference patterns of an
(1) The high order derivatives of the DNN output with respect to the input phrases are all zero. (2) The DNN works well on the masked samples, and yield higher confidence when the input sample is less masked. (3) The confidence of the DNN does not drop significantly the masked samples.
'There are three assumptions.
18

---

Published as a conference paper at ICLR 2026
LLM behind its seemingly correct outputs. Although previous studies have been proposed to evaluate the performance of LLMs, rigorously evaluating the reliability of their inference patterns requires that remains unexplored. Thanks
theoretically grounded mechanistic explanations, which is to advances in Explainable AI, we can use a set of interactions between input features to faithfully represent the inference score of deep network. However, despite these theoretical achievements, remains unknown (1) how many problematic interactions are modeled in LLMs (e.g., legal LLMs), and (2) to what extent these interactions influence legal judgments.
In this paper, we quantified these interactions and conducted experiments across different LLMs and datasets. We found that,
Over half of the interactions modeled by LLMs actually represent clearly unreasonable or even incorrect justifications for their predictions.
LLMs tend to use simple interactions of local tokens to guess judgments.
LLMs tend model a large number of canceling interactions.
These findings help us gain deeper insights into the inference patterns of LLMMs, particularly in high-stakes tasks where they provide quantitative metrics to indicate the degree to which LLM judgments can be trusted. The key contributions of the proposed mechanistic explanation method are,
tell what words are important. For example, in "this movie is not bad," attribution highlights "not" and "bad." In contrast, the interaction- based approach (47) 26) 42) further reveals that the model relies on the interaction ("not", "bad") to reverse the sentiment. This allows us to distinguish whether the LLM is truly performing semantic composition or just using Bag-of- Words statistics.
Revealing reasoning patterns, not just attribution. Attribution methods 48
Evaluating the faithfulness and reliability of LLMs in high-stakes decisions. In high-stakes domains (e.g., medicine, finance, law), it is not enough for an LLM to be correct, it must be correct for the right reasons. For example, a legal LLM might learn spurious correlation, associating a specific occupation with a "guilty" verdict (Case 2 in the paper). By quantifying interactions between input phrases, we explicitly capture this biased reasoning. This provides a quantitative metric for reliability and serves as a tool for model auditing.
Diagnosing shortcut learning. LLMs adept at using statistical shorteuts to guess answers rather than performing genuine, complex reasoning. Interaction analysis can diagnose this behavior. For instance, when processing long texts, LLM might rely only few local, low-order interactions to make a decision. By analyzing interaction order (complexity), we can quantify this phenomenon.
F MASKING IN EXPLAINABLE ARTIFICIAL INTELLIGENCE
Masking is a common practice in Explainable AI. For instance, interaction-based methods (47 26 42). it is standard to evaluate many masked variants of input and restrict the analysis to short examples (e.g., =10 phrases). Similarly, in perturbation based attribution methods, such as LIME (43), Shapley sampling values, KernelSHAP (32), and DASP (2), it is also widely adopted approach to evaluate numerous masked input variants, often restricting the analysis to short examples (e.g.. < 16 phrases).
Besides, this challenge be alleviated through engineering techniques. First, attribution methods (e.g. Integrated Gradients (48), LIME (43)) as a heuristic to identify and prioritize salient words or phrases, pruning the search space. Prior works (31 (27) have shown that LLMs exhibit attention on only a few sparse regions in inputs. Second, the input phrases in this paper are flexible units of analysis. They are not limited to single tokens but can represent multiple words, phrases, short sentences, or even paragraphs as input units (26) 42) Empirically, 10- 12 input phrases were typically sufficient for effective analysis 44
classic
G How DOES THIS METHOD GUIDE MODEL IMPROVEMENT?
There several methods that can be employed enhance model performance.
19

---

Published as a conference paper at ICLR 2026
Enforce interaction consistency across models. Reliable patterns are typically consistent across different models, while unreliable, high-order interactions are often model-specific and generalize poorly (26). We can jointly train two models using an interaction consistency loss to penalize differences in their learned patterns on the same input. This encourages models to converge on reliable reasoning. boosting overall performance.
Refine supervised fine-tuning (SFT) with reliability scores. We can integrate interaction analysis into both dataset construction and sample weighting. For dataset construction, identifying unreliable interactions allows us to create counterfactual data that explicitly targets weak reasoning spots, e.g.. alleviating the identity discrimination. SFT instructions can also be designed explicitly guide the model to use reliable interaction paths. For sample weighting, we can adjust sample weights based on the interaction reliability score (Sreliable). Samples where the model correct but Sreliable very low (i.e., "correct output for the wrong reason") are assigned higher training weights, forcing the model to repair its underlying reasoning mechanism.
Enhance reinforcement learning (RL) optimization. We can incorporate interaction reliability into the reward dimension to shift RL optimization from the output result (What) to the reasoning process (How). For example, we can add the interaction reliability score (Srelablee as new feature to the reward model. The reward model would then reward generated texts not only based on human preference but also highly reliable interaction paths. Besides, can also impose constraints on the policy model during RI by applying an additional penalty term the model's next token selection significantly increases the weight of unreliable interaction paths.
Algorithm Computing AND-OR interactions
1: Input: Input legal case the legal L.M v()), and the annotations of the relevant, irrelevant, and forbidden tokens in x.
2: Output: A set of reliable interactions Treliable (S]x) and Treliable (reliableSSix).. interaction effects reliable
(S|x), and the ratio of reliable 3: Input the legal case x into the legal LLM, and generate the judgment (a sequence of tokens);
and
4: for S C N do
5: For each masked sample xs, compute the confidence score vxxs) based on Eq. I
6: end for
7: for S C N do
8: Given v(xs) for all combinations $ S N, compute each AND interaction IS and each OR
interaction I via min(yr) Esgn.s LAND IOR []:
9: end for
10: for S C do
Compute the reliable AND interaction effect Irliiable (S'x) and the reliable OR interaction /reliable(S] based on Eqs. and 5 12: end for
effect I
Compute the ratio of reliable interaction effects reliable based on Eq. 7
14: return freliable (S) (S|x), or and
Treliable (S|x), geeliable
H OR INTERACTIONS CAN BE CONSIDERED SPECIFIC AND INTERACTIONS
The OR interaction 1 can be considered as a specific AND interaction 1 we inverse the definition of the masked state and the unmasked state of an input variable.
Given a DNN v R - and an input sample x E R", , we arbitrarily mask the input sample, we get 2" different masked samples xs, VS C N. Specifically, let use bascline values b E I2" to represent the masked state masked sample Xs, i.e.,
Iii i E S
(xs);
(14)
b i i S
Conversely, if we inverse the definition of the masked state and the unmasked state of an input variable, Le,, consider b the input sample, and consider the original value x as the masked
20

---

Published as a conference paper at ICLR 2026
state, then the masked sample by can be defined as follows.
bs; i S
(bs):
(15)
Thi i 4
According to the above definition of a masked sample in Equations (1) and (15). we can get
X by. To simply the analysis, if we assume that (XT) = Vor (X) 0.5v(xT) (XT), then the
OR interaction _ can be regarded specific AND interaction I (b) follows.
I (x)
Ergs( (-1)15) 11 (x
Σ rgs (-1)15)-131 (br()))
(16)
ETSS (-1 vana
IADD (b). - -IS
I ANNOTATION OF RELEVANT PHRASES, IRRELEVANT PHRASES, AND FORBIDDEN PHRASES
We propose the following two principles to avoid unnecessary ambiguity in the annotation of the three types of phrases. (1) The first principle is to avoid ambiguous legal cases. To ensure clarity, we engage several legal experts to select a set of straightforward and unambiguous legal cases. We let them to annotate the above three types of phrases to avoid ambiguity. (2) The second principle is to avoid analyzing subtle legal differences between the laws in different countries Although algorithm can accurately explain the legal judgments made by legal LLMs based on sophisticated legal statutes, the goal of this paper is not to focus on such nuanced differences. Therefore, we let legal experts to select relatively simple and uncontroversial legal cases, enabling us to directly compare the performance of an English legal LLM and Chinese legal LLM on the same input case.
For example, given an input legal case "on June 1, during a conflict on the street, Andy stabbed Bob with a knife, causing Bob's 's death," the legal LLM provides judg- ment "murder" for Andy. In above example, the input phrases can be set as N {on June 1], [during a conflict), lon the street), [Andy stabbed Bob with a knife), [causing Bob's {[Anyy stabbed Bob with knife), [causing Bob's 's death]} are the direct rea-
death)). R son for the judgment, thereby being annotated as relevant phrases, where all tokens the brackets are taken as single input phrase. The set of irrelevant phrases are annotated as I { June 1], [during a conftict), [on the street)]. The input phrase like "during a conffict" may influence Andy's behavior "Andy stabbed Bob with knife," but it is the input phrase "Andy stabbed Bob with a knife" that directly contributes to the legal judgment of "murder," " rather than the input phrase "during a conffict."
Given another input legal case involving multiple individuals, such as "Andy assaulted Bob on the head, causing minor injuries. Charlie stabbed Bob with a knife, causing Bob 's death, 2 the legal LLM assigns the judgment of "assault" to Andy.
Let the set of all input phrases be N = {[Andy assaulted Bob on thehead], [causing minor injuries), [Charlie stabbed Bobwith a knife], [causing Bob's tob's death]}. Although the input phrases "Charlie stabbed Bob with knife" and "causing Bob's 's death" naturally all represent crucial facts for judgment, they should not influence the judgment for Andy, because these words describe the actions of Charlie, not actions of Andy. Therefore, these input phrases are annotated as forbidden phrases, F {[Charlie stabbed Bob with knife), [causing Bob's 's deathi)..
J FAITHFULNESS OF THE INTERACTION-BASII EXPLANATION
In this section, we conducted experiments to evaluate the sparsity property in Figure 7 and the universal matching property in Figure of the extracted interactions.
We use an English legal LLM S speaking jurisdictions such the U.S., Canada, the UK, and Europe, and we use a Chinese legal I.I.M BAI-Law- 3B (T), which is trained using legal corpora from China.
(7), which is trained using legal corpora from English-
21

---

Published as a conference paper at ICLR 2026
Interaction Strength
[AAD |ar L1O
MoulLM-7B-Instrual model
BAI-Law-136 model
23-
0.05 max,
max
18/00
1960
1960 1090 3000
Figure 7: Sparsity property of interactions. We show the strength of different AND-OR interactions (1rg and ! extracted from different samples in a descending order. Only about 0.5% interac- tions had salient effects.
confidence of the LLM v()
AND Jegitaal (()
-2
75
2 masked sentences
Figure 8: Universal matching property of interactions. Experiment verifies that the surrogate logical model h(xT) can accurately fit the confidence scores of the LLM v(xT) on all 2 masked samples {xT(T' C N,, i.e. VI' S N, v(x)) = h(XT), no matter how we randomly mask the input sample 2n different masking states T C N.
Judgment for Andy
reltable interaction effects
AND-OR Logical Model
C'sssuut (x)
EENEET wasdss 718221512 1t & Ee E Ka AMII. d s.ttsc/eitt
A.
interaction effecis
Unrellatde effects
i. 35 Ma 1981 Wale
otth a004
2016, the atmosphere bewween the defendant
about 23:00
Assault
23.1%
[st (al
the vidlin Bob [A] suddenly became tonse because Andy insultd Bob.
(Sm.LLM-7B- Instrcc))
rellitee 76.9%
Disstadaction and (C) angar quickly sproad, Andy and Bob (D) began fight chamically
Intentional Injury
23.0%
2 1010
(F) anger and (8) unwilingnesss Andy throw punds III
rabahii 77.0%
sousing Boo (J) fall into come.
(RAL-Law-13B)
Relevant
Irreevant
Relevant phreses
Inelevant phrases
(al Input legal case
Judgmeet
[c) Ratio
(d) Reliable and unreliable Interaction effects
Figure 9: Visualization of judgments influenced by unreliable irrelevant phrases. (a) Irrelevant phrases include sentimental phrases that are not the direct reason for judgment. Criminal actions were annotated as relevant phrases. We also translated the legal case to English as the input of the SauILM-7B-Ime IB-Istrucct model. (b) Judgments predicted by the two legal LLMs, which were both correct according to laws of the two countries. (c,d) We quantified the reliable and unreliable interaction effects.
K
MAKING JUDGMENTS BASED ON SEMANTICALLY IRRELEVANT PHRASES
Case 3: making judgments based on unreliable irrelevant phrases. We observed that although legal LLMs achieved great performance in predicting legal judgment results, the legal LLMs used a significant portion of interaction patterns that were attributed to semantically irrelevant phrases for judgment (e.g., the time, the location, and the sentimental phrases that are not the direct reason for the judgment). To evaluate the impact of semantically irrelevant phrases on both the SaulL.M7BB- I--nrrlntt and BAI-Law-13B models, we engaged legal experts annotate phrases that served as the direct reason for the judgment as relevant phrases in R. and those that were not the direct reason for the judgment as irrelevant phrases in I, e.g., semantically irrelevant phrases and unreliable sentimental phrases behind real criminal actions.
Figure 9 shows the first legal case, which showed Andy had conflict with Bob and attacked Bob, committing an assault. Here, input phrases such as "fight chaotically." " "threw a punch," " "causing." and "fall into coma" were annotated as relevant phrases in R, these phrases served as the
22

---

Published as a conference paper at ICLR 2026
direct reason for the judgment "Assault." On the other hand, input phrases like "began to" and sentiment-drive phrases such as "dissatisfaction" "anger" were annotated as irrelevant phrases in I, as these phrases were not direct reason for the judgment.
In this legal case, there were 28 AND interaction patterns and 22 OR interaction patterns in the top 50 most salient AND-OR interaction patterns. Here, the average interaction strength for top 50 most salient interactions was 0.078, while the average interaction strength for the remaining AND-OR interaction patterns among the 2 2 2048 AND-OR interaction patterns was 0.005. The legal LLM MaulLMM-B-Inntruue did use several interaction patterns that aligned with legal experts domain knowledge for the legal judgment. For example, an AND interaction pattern S = {"threw a punch"). }. and an AND interaction pattern S2 = ("threw a punch", "fall into a coma") and an OR interaction pattern S/ = ("fight chaotically" " "threw a punch") contributed salient reliable interaction effects R = 0.79. RS. = 0.17, and Rg 0.39 to the confidence score v(( (("Assaull") xx of the judgment "A "Assault," respectively. However, the legal L.LM also used lots of interaction patterns that did not match legal experts' domain knowledge for the legal judgment. For example, two AND interaction patterns S₁ = ("dissatisfuctionn) }. and Sy = { ["anger"], which represented unreliable sentiments instead of criminal actions, contributed salient unreliable interaction effects UAND 0.30 and U 0.07 to the confidence score of the judgment "Assaultt" respectively. In sum, the SaulLM-7B -nnnstrutt- model used a ratio of reliable = 76.9% reliable interaction effects for the legal judgment. This indicated that the legal LLM mistakenly made judgments based unreliable irrelevant phrases, because unreliable sentimental tokens only served as explanations for criminal actions, rather than the direct reason for the legal judgments.
In comparison, we evaluated the above legal case on the BAI-Law-13B model, as shown in Figure and Figure 10 in Appendix. There were 12 AND interaction patterns and 38 OR interaction patterns in the top 50 most salient AND-OR interaction patterns. The average interaction value for top 50 most salient interactions was 0.048, while the average interaction value for the remaining AND-OR interaction patterns was 0.004. Compared to the SaulLM-7B Instruct model's 's rreiiable = 76.9% ratio of reliable interaction effects, the BAI-Law- 13B model used similar reliable interactions, i.e., using ratio of seliable 77.0% reliable interaction effects and a ratio of 3 guareliable 23.0% unreliable interaction effects to compute the confidence score. Many interaction patterns used by the BAI-Law- 13B model were also used by the SauILM-7B Instruct model. such an AND interaction pattern S = ("threw a punch"), ). and an OR interaction pattern S = {fightt chaotically", , "threw a punch") contributed salient reliable interaction effects R 0.34 and R 0.12 to the confidence score v( ("Intenttonal Injury"|x) of the judgment " "Intentional Injury." respectively. This indicated that these two legal LLMs did successfully identify some direct reasons for the legal judgment. On the other hand, the BAI-Law-133 model used problematic interaction patterns for the legal judgment, such as two AND interaction patterns S3- = {"suddenly became tense"} and S4 = {"anger") contributed salient unreliable interaction effects U 0.08 and U 0.03 to the confidence score, respectively. The unreliable sentimental token should not be used to determine the judgment. Additional examples of making judgments based on unreliable sentimental phrases are provided make judgment on Andy in Section L.3
L MORE EXPERIMENT RESULTS AND DETAILS
L.1 DISTRIBUTION OF PHRASE ANNOTATIONS BY LEGAL EXPERTS AND VOLUNTEERS
In this subsection, we show the distribution of phrase annotations provided by legal experts. Specif- ically, we consulted 16 legal experts to annotate the phrases in the input prompts using a majority voting scheme. The selected cases are generally simple and straightforward, ensuring that phrase annotations are relatively clear and unlikely to introduce major issues.
Legal background of legal experts. These legal experts are either working in the legal profession or studying law-related disciplines. Their experience in the legal field ranges from two to twelve years, with academic backgrounds in areas such as criminal procedure law, international law, and jurisprudence. Specifically, three of these experts have over eight years of experience criminal trial judges, one serves as an assistant to a criminal trial judge, and two are currently pursuing master degrees in international law. The diverse backgrounds of these legal professionals greatly contribute
23

---

Published as a conference paper at ICLR 2026
Table 1: Phrase annotation for Case 1.
Input phrase
Is relevant phrase?
Is irrelevant phrase?
Is forbidden phrase?
Final annotation
(A) tense
0
16
0
Irrelevant phrase
(B) Dissatisfaction
0
16
0
Irrelevant phrase
(C) anger
Irrelevant phrase
(D) began to
2
14
0
Irrelevant phrase
(E) fight chaotically
16
0
Relevant phrase
(F) anger
3
13
0
Irrelevant phrase
unwillingness
0
Irrelevant phrase
(H) threw a punch
16
0
Relevant phrase
(I) causing
16
0
Relevant phrase
(J) fall into coma
16
0
Relevant phrase
Table 2: Phrase annotation for Case 2.
Input phrase
Is relevant phrase?
irrelovant phrase?
Is forbidden phrase?
Final annotation
(A) morning
0
16
Irrelevant phrase
(B) had an argument
Irrelevant phrase
(C) chased
14
2
Relevant phrase
(D) with
15
Relevant phrase
(E) bit
15
Relevant phrase
(F) slightly injured
16
Relevant phrase
(G) hit
3
13
Forbidden phrase
(H) with shovel
0
O
16
Forbidden phrase
(I) injuring
0
O
16
Forbidden phrase
(J) death
15
Forbidden phrase
to the analysis of relevant, irrelevant, and forbidden phrases in legal cases, providing nuanced legal perspective.
Distribution of phrase annotations. We present the distribution of phrase annotations for each phrase in the three legal cases discussed in the main paper, as shown Table Table and Table B The final annotation for each phrase in the input legal scheme.
determined using majority voting
Case 1: At about 23:00 on March 18, 2016, the atmosphere between the defendant Andy and the victim Bob suddenly became tense because Andy insulted Bob. Dissatisfaction and anger quickly spread, and Andy and Bob began to fight chaotically. In anger and unwillingness, Andy threw punch, causing Bob to fall into coma.
Judgment of the legal LLM for Andy: Assault.
Table 3:
Phrase annotation for Case 3.
Input phrase
relevant phrase? Is irrelevant phrase?
Is forbidden phrase?
Final annotation
(A) Wednesday
16
0
Irrelevant phrase
(B) night
Irrelevant phrase
(C) a judge
16
0
Irrelevant phrase
(D) walked
0
16
O
Irrelevant phrase
(E) home
0
16
O
Irrelevant phrase
(F) day's 's work
16
O
Irrelevant phrase
(G) dark
16
0
Irrelevant phrase
(H) holding knife
16
0
Relevant phrase
(I) robbed
16
0 0
Relevant phrase
(J) belongings
16
Relevant phrase
Case 2: On the morning of December 22, 2013, the defendants Andy and Bob deceived Charlie and the three them had an argument. Andy chased Charlie with an axe and bit Charlie, causing Charlie to be slightly injured. Bob hit Charlie with shovel, injuring Charlie and causing Charlie' death.
24

---

Published as a conference paper at ICLR 2026
Judgment for Andy
Ratio reliabie Interaction effects
AND AND-OR Logical Model
A Ciintentional Injury"|A
internction effects
Unroliable interaction effeets
Lysh 6373
23:00
Assault
23.1%
41% -1:3
the atmoaphere between the defendant (A) suddenty becime iense bicause Andy irsutted Bob.
the viclim
(BauLLM-7B- Instruct)
Fallatte 76.9%
Iepe
1400 2.044
Dissatitactin and
anger
quickly spreed, Andy and Rob (D)
(E) fight thanfically (F) enger and (9) unwilingnees.
Intentional
23.0%
ppmb
Injury
Af A015
12 0000
Andy threw punch. II
sausing (I) isllinto
(BAI-Lww113B)
-3322
Relevant
Inelevant
Rerevant pfreses
Inelevant phrases.
[a] Input legal case
(b) Judgment
[c) Ratio
(d) Reliable and unreliable Interaction effects
Figure 10: Visualization of judgments influenced by unreliable irrelevant phrases in the BAI-L aaw-1 13B model. (a) Irrelevant phrases include sentimental phrases that are not the direct reason for judgment. Criminal actions were annotated as relevant phrases. We also translated the legal case to English as the input of the MauILM-7B- InIItuutttt model. (b) Judgments predicted by the two legal LLMs, which were both correct according to laws of the two countries. (c,d) We quantified the reliable and unreliable interaction effects.
Judgment of the legal LLM Andy: Assault.
Case 3: Late Wednesday night, Andy, a judge, walked home alone after finishing day's work. On the dark road, two suspicious men followed, holding knife and robbed Andy's 's belongings. Judgment of the legal LLM for two suspicious men: Robbery.
L.2 SIGNIFICANCE OF CONFLICTED INTERACTION PATTERNS
This subsection shows the significance of mutual cancellation of interaction patterns. We found that over 60% effects of the interaction patterns had been mutually cancelled out in Table 4
Table 4: Significance of mutual cancellation of interaction patterns (%), which is measured by conffictt
Dateset
Qwen
Deepseek
BAI
SaulLM
CAIL2018
78.00
82.46
62.67
LeCaRD
78.70
65.58
85.38
LEVEN
78.40
77.58
83.72
LegalBench
75.31
83.10
76.60
LexGLUE
98.25
96.18
29.97
L.3 MORE RESULTS OF JUDGMENTS INFLUENCED BY UNRELIABLE SENTIMENTAL TOKENS
We conducted more experiments to show the judgments influenced by unreliable sentimental tokens in Figure 12 Figure 13 and Figure 14 respectively. We observed that considerable number of interactions contributing the confidence score v(x) were attributed to semantically irrelevant or unreliable sentimental tokens. In different legal cases, the ratio of reliable interaction effects to all salient interactions was within the range of 32.6% to 87.1%. It means that about 13~68% of interactions used semantically irrelevant tokens or unreliable sentimental tokens for the judgment.
L.4 MORE RESULTS OF JUDGMENTS AFFECTED BY INCORRECT ENTITY MATCHING
We conducted more experiments to show the judgments affected by incorrect entity matching in Figure 15] Figure 16. and Figure 17 respectively. We observed that a considerable ratio of the confidence score u(x)
mistakenly attributed to interactions criminal actions made by incorrect entities. In different legal cases, the ratio of reliable interaction effects to all salient interactions was within the range of 31.90 11.% to 67.8% 67.8%. It means that about 22~68% of interactions used semantically
25

---

Published as a conference paper at ICLR 2026
Munuaat nure mndersesendai V. Ke.
Judgment for suspicious men
reliobíe interaction effects
AND-OR Logical Model
A CRRobbory" Unroliable internction
SEA wit BUS. cuteer MB--
Relishle inseraction offects
Robbery
18.6%
Iniee] (eeetral
Wedresday (B) night.
when the victim's
gellable -1.44%
Ialle
[occupation]
Andy, (C) locoupasion (D) walked
alono after finsshing
judge]
3 cay's work. On the (G) dank rcad.
two suspicious followed
holking
and 0) mebbed
Not mentioned
16.0%
Andy's [d] batongings.
(C) (occupation] Is a J.dgge/ a volutteer
when the victim's [occupation] ja volunteer]
84 0%
Retevant
Inrlevant
Relevant phrtoii
Inelevant phrases
(a) Input legal case
(b) Judgment
(c) Ratio
(d) Reliable and unreliable Interaction effects
Figure 11: Visualization of judgments biased by discrimination in identity, when the victim's 's [occupation] is [a volunteer]. To enable the fair comparison, we compute interactions on the output score ((""Robbey"") phrases were annotated in the legal case, including the occupation, time and actions that are not the direct reason for the judgment. Criminal actions of the defendant were annotated as relevant phrases. (b) The Baul-M-7B-Instet occupations. (c,d) We quantified the reliable and unreliable interaction effects.
instead of the actual LLM's ss output score v( u("Not mentioned" (a) Irrelevant
model predicted the judgment based on the legal case with different
Judgment for Andy
Reliable
Unreliable
All
Relioble
Unrellaalee
M.A. st-rte
10 2
Hid (E) 0.41
Pelate Dukskt
rigns mi EILNN 10.66.
(GE)))) -0 68
(B)= 0.22
A.BEL 1ble
First-degree
(2((D.))- 0.16
Gad G) =o.o0.
Around midriggt on Octobar 27. 10th10 the Bob
Murder
[PAAJ]]
(F) 0.17
66.1%
Andy
(,",,,,D,J) 0.009
uua 0:17
vismasy
Instruct)
(D]:0)- 0.00
gr (A.J)) 0.13
1334567891
1234 7785578052 order
[D]
Mjchen knils 1: actions
were
oeteemely velern as
(ite") (B,)) 0.15
Intersition
were
Fi arleashing all his G
Intentional
(1/0 0.76
0.05
making
seemeed to freze with his Mall
on Bob. The atmosphere
(BAI-Law-13B)
Homicide
(2)
(C,
iie (M) 0005
lase (B)= 0.06
07.195
Bob [) diod the
(0.J) 0.40
i (E.H) -0.04
-0004
1234567888
Relesart
Inviovent
[B] 0.24
(d) Distribution of reliable and unrellable
order [5]
(a) Int case
case
Judgment
(c) Interactions
interactions over different orders
Figure 12: More results of judgments influenced by unreliable sentimental tokens. (a) A number of irrelevant tokens were annotated in the legal case, including unreliable sentimental tokens. Criminal of the both correct according to laws of the two countries, (c,d) We quantified the reliable and unreliable interaction effects of different orders. The S effects, while the BAI-Law- 13B model encoded 87. 87.2% reliable interaction effects
actions were annotated as relevant tokens. We also translated the legal case to English as the input
model. (b) Judgements predicted by the two legal LLMs, which were
model used 66. reliable interaction
Judgment for Andy
Reliable
Unreliable
AB
Relioble
Unreliable
(r20) = =
(0) 0.53
(0) 0.44
3.
Pelabs-
3- 3-00 A 30-
Fraud (BULLM-7B Instrust)
0.10
(G) 0.41
From Novamber 2011 to
[_(C.H))) 0.11
(E) 0.31
35.3%
January 2013, the A1
and [B 20000100 dafendant Andy
noticed that Bob hed habit
xp*: 0.07 (grd)cc 0.24
(B, 0.08
end 0.28
17335+78910
T284563 5477567 91C
ordar
Ramns Ditven by greed Andy
[G]
01:00= 1.08
(1
eockalo: B00's
Fraud
-227 0.27
lie 0.)
[G] basy monei and used
Jabnn informaiin In repastadly W
[X]
[BAI) al-- 39)
(B.U)) 0,23
(D....... 200
(ur*" - 0.18
18.5%
sefraud him of [J] mones.
1g (B..)) = 0.06
3254561688
12345871023
12545851600
(2 ((()))
(d) Distribution of reliable and unreliable [5] interactions different orders
Relevant
Inelevant
(a) Input legal case
Judgment
(c) Interactions
Figure 13: More results of judgments influenced by unreliable sentimental tokens. (d) The SauILM- 7B-Instruct model used 35.3% reliable interaction effects, while the BAI-Law-13B model encoded 48.5% reliable interaction effects.
irrelevant tokens for the judgment, or was mistakenly attributed on criminal actions made by incorrect entities.
26

---

Published as a conference paper at ICLR 2026
Judgment
Reliable
Unrellable
All
Raliabie
Unrellable
for Andy
(aath =
[ 0.87
(DG
0.41
Interaddi Interaction
Galles
uncelell
Theft
0.32 (F
(D) = 0.40
Between December
Soptember
(3 7B Instruct)
(0.0)
021
aad (C..G) 0.33
30.1%
Mtlnndd Io bommmi ortimee Bob
not
2014,
Andy
12
(,,,,,,,0)= 0.15
B. 0.18
(C,0) 0.35
(.....) -0.33
325459788700
12345
grein and His
Andy
[ (D)
Bob's
mobill
[G)= 0.85
fid
and
T
Theft
[ 0.78
32.69
himsell
(BN-Law-13B)j)
asg [FF.B..
(D." 0.33
Relevant
Inrelvvant
("(B......)=0.11 0:13 [p
(d) Distribution of reliable and unrelliale interactions over different orders
order [S]
(e) Input legal case
(b) Judgment
(c) Interactions
Figure 14: More results of judgments influenced by unreliable sentimental tokens. (d) The M- 7B-Instruct model used 38.1% % reliable interaction effects, while the BAI-Law- 13B model encoded 32.6% reliable interaction effects.
20 i FRAR 4+ satmann Ne AIEE Inb. 77RUU Baccii WEENEMEE 2E HIILS.
Judgment for Andy
Reliable
Unreliable
Rnliabie
Unrellable
((31)) 2.05
and =)
ga [] - -0.13
Intgoco ( Sinterscia
- M
Manslanghter
()) 0.16
Around 9:30 p.m. on Decamber 24. 2013. the defendant Andy [A]
(B..L Insiruct)
and =
or (a. J)=
cocdlood int0
someshing
[220,,1=0023
(F)
12345e78510
173456738 63446678000
order
A0 repeatedly suncher bob
(2214)-
And 0.25
[E] with 12t: F1 on
cuniin a5 Frured Bob
Intentional
-0:18)
(add)aa)) 0.12
Injury
(G.1)
(B)= 0.07
64.1%
lase pand
HD The
Bob Ldde TI
(BALLLa-113B)
1ac after medical efforts falled.
(G.())
i (G,1)))
18345818310
7
Forbiddan
Ineeevant
(D.E.)) 0.00
(d) Distribution reliable and unreliable interactions over different orders
onder [5]
[a] Input lngal case
(b) [b]dumeeettt
(c) Interactions
Figure 15: More results of judgments affected by incorrect entity matching. (a) A number of irrelevant tokens were annotated in the legal case, including the time and actions that were not the direct reason for the judgment. Criminal actions of the defendant were annotated as relevant tokens. Criminal actions of the unrelated person were annotated as forbidden tokens. (b) Judgements predicted by the two legal LLMs, which were both correct according to laws of the two countries. (c,d) We measured the reliable and unreliable interaction effects of different orders. The SaulLM- BB-Inttt model used 67.8% reliable interaction effects, while the BAI-Law-133 model encoded 64. 1 1% reliable interaction effects.
SeIT- ACABAN MA 4E by 1100 21 M... 1e SeR
Judgment for Andy
Reliable
Unreliable
All
Relioble
Unrelloble
4* 127
[) 0.23
Interactioo aetton
te (a)= -222
Relabi
Robbery
(120)= 0.64
(B.6) 0.19
Andy [8] lnaettnn Boo. 1t
One day. Andy and Bob YAI
1oppll about Andy's thert, and
Bob
IS-ALM-7
Instruct)
(.)G 0.12
67 (G,H) = -0.29
(dlIG.) 0.48
mnd (0.1) - 0.13
4nd (()d (J)= 0.10
15145
ordor
[8]
Andy, resultig
(031H) 0.92
(J)- 0.33
Intereatioon
serouu Iuurs Andy.
47 (7*(G
[hreatinn:: Bob and [+]] stowe iy
later deeowwreed that Andy [0)
(BA--Lw-1155)
Theft
(7*(L.)) 0.05
(330) 0.08
=
di (B,G)= 0.27
33.955
his belongrvgg while they were
CP H)= 0.04
2 (0)= -023
1334367830
1254567828
Relevant
Inrelvaant
12 (( 0.04
(d) Distribution of reliable and unrellable [S] interactions over different orders
Forbiddan
(a) Input legal case
(b) Judgment
(c) Interactions
Figure 16: More results of judgments affected by incorrect entity matching. (d) The SauILM-7B- Instruct model used 63.7% reliable interaction effects, while the BAI-Law-13B model encoded 31.9% reliable interaction effects.
L.5 MORE RESULTS OF JUDGMENTS BIASED BY DISCRIMINATION IN OCCUPATION
Experiment results of judgments biased by discrimination in occupation in Section Figure 21 illustrates additional examples of how occupation influences the judgment of the legal case, which were tested on the SaulLM-7B-Inntrun occupations, such as "a lawyer" and "a policeman," the judgment remained "robbery." " Besides, the occupation "a programmer" changed the judgment to "not mentioned." The interactions containing
model. It shows that if we replaced "a judge" with law-related
27

---

Published as a conference paper at ICLR 2026
sonoheeaa IE Aatcc SAgueE Ni REx MEEE (N05) 4 Butttt
Judgment
Reliable
Unrellable
All
Rollabie
Unrellable
for Andy
4=2/8)=
Hil (A)
142000
(0) 1.57
( 1.13
(H) 0.88
On September 21, 2016, while
Pretenses
(G,H) 0.51
-2.52
clames
defendant Andy [A] praating
the victim Bob, he (B) Tasely
that he could help Bob [c]
(B
Instruct)
(7 0.36
(B.00 0.50
and (C,0)) 0.48
12 1235567788
125 67677330 6
12 9 6.
2lor
hile
rocidly
for
betting
=
Bob's
(B) 0.84
con (1) 0.74
Ardy
Fraud
(B, HJ) (ij))--aa 0.51
12.25
mores
(WA-La
(H) 0.56
Inolovant
(B,C MJJJ) -040 () HJO 0.52
order [S]
Forbiddan
(d) Distribution reliable and unreliable
(e) Input case
case
(b) Judgment
(c) Interactions
interactions over different orders
Figure 17: More results of judgments affected by incorrect entity matching, (d) The SauILM-7B- Instruct model used 52.5% reliable interaction effects, while the BAI-Law-13B model encoded 42.2% reliable interaction effects.
Judgment
Reliable
Unreliable
All
Relioble
Unrellable
for Andy
(F
0.42
Vnteracioon
(2))))
-0.44
As 10// (
Lareeny
-.770
(H) -0.20
"awyer"
(G) 0.27
63.7%
One day défendant Andy.
67((C,E6)- 0.50
sackaag
[D Iocuuallion (C) 5t01 a [D]
of ()) valustved from
(2 0.37
(C,,,,,
0.28
12245678812
12745678M8
14.5 12545% 7 930
the
FI DOS00 G) worchouse ouriin
dellvnn
Theft
(2210)=1111 1.12
and
--0 -0.3
Interacton
"programme"
(C.D....))
0.72
(H) -0.22
(8)) (oocuration
bwwyer/ programmeer
i2 (C.E,)))) 0.44 g=BBBEFFF)=0
12345678915
12545678515
3234587
Relevant
(2"",,,,E.F)-00
if =
(d) Distribution of reliable and unrellable [S] interactions over different orders
(a) Input legal case
(b) Judgment
(c) Interactions
Figure 18: More results of judgments biased by discrimination in occupation. (a) A number of irrelevant tokens were annotated in the legal case, including the occupation, time and actions that are not the direct reason for the judgment. Criminal actions of the defendant were annotated as relevant tokens. (b) The
model predicted the judgment based on the legal case with different occupations, respectively. (c.d) We measured the reliable and unreliable interaction effects of different orders. When the occupation was set to "lawyer," the LLM used 63.7% reliable interaction effects. In comparison, when the occupation set "programmer," the LLM encoded 57.3% reliable interaction effects.
the occupation token "a judge". "a lawyer", "a policeman"" "a programmer", and "a volunteer") were important factors that changed the ratio of reliable interactions from 81.4% to 84.0%. This suggested that the legal LLM sometimes had considerable occupation bias.
Futhermore, Figure 22 shows the test of the BAI-Law- model on the legal case, in which Andy, the victim with varying occupations, was robbed of his belongings by two suspicious men. Similarly, we found that the BAI-Law-133 model encoded interactions with the occupation tokens "a judge," which boosted the confidence of the judgment "robbery." More interestingly, if we substituted the occupation tokens "a judge" to "a policeman." " the interaction of the occupation "a policeman," decreased from 0.29 to 0. The interactions containing the occupation token were important factors that changed the ratio of reliable interactions from 78.9% 87.1 %. This suggested that the legal LLM sometimes had considerable occupation bias.
More results of judgments biased by discrimination occupation. We conducted more ex- periments to show the judgments biased by discrimination in occupation in Figure 18] Figure [9] and Figure 20 respectively. We found that the legal LLM usually used interactions on the occupation information to compute the confidence score oxx). In different legal cases, the ratio of reliable interaction effects to all salient interactions was within the range of 30.1% % to 63.7%. In particular, in Figure changing the occupation from "lawyer" to "programmerr results in decrease of the reliable interactions from 63.7% to 57.3%. The difference of interactions containing the occupation token changes the model output from "Larceny" "Theft."
28

---

Published as a conference paper at ICLR 2026
Judgment
Reliable
Unrellable
All
Rollabie
Unrellable
for Andy
==2/C)=
fin (A) 1:01
Interaction
Fraud
[.I(B.C.))) 0.18
(D) 0.50
the defendant
(F.AA. 0.19
VAC.
30.1%
[A]
GF"(A.C..00--0.. 0.18
Wittm (0) out mones [D)
(=)) 0.16
(A.
-0.40
1237567889 9-920
by Intesay promens
hep
(Fn appy 10 bank bor
(0)
intereation
[H] racuelli'
(J] handinno deponi and
(4:B)) -
(00) 0.84
Fraud
32.7%5
"volurter""
-..8
volunteer
12.25.
Relevant (e) Input case
Inrelvvant
land (B,,)) [) 0.42
(d) Distribution reliable and unreliable
order [S]
case
(b) Judgment
(c) Interactions
interactions over different orders
Figure 19: More results of judgments biased by discrimination in occupation. (b) The SauILM-7B- Instruct model predicted the judgment based on the legal case with different occupations, respectively, (d) When the occupation was set to "telephone service," the LLM used 30.1 1% reliable interaction effects. In comparison, when the occupation was set "volunteer," the LLM encoded 32.7% reliable interaction effects.
Judgment for Andy
Reliable
Unreliable
Relioble
Unrellable
(2166)-0.82
Interaction
(2*(G.H)J))- 0.75
(B)= 0.49
2 by
Intentional
(f.)).)))) 0.34
[AI Cna day Bob susoodted
Injury
(310)=
(BE.0))0
11.3%
defendont Andy
eomer
[C(G) =
(e0"" 0.43
depult between
(21:11.))--0.1
13 (E) 0,34
12365
7
Them. Andy, [cocupolior
0.77
1ee (B,EG))) m.m
(G) strust Bob [H] win weoden
Sckioe cousing (J) seoond-
Intentional
(3
Interaction
Degnee manor muuies to Bob.
Injury
(G.(G_ 0.20
1e 0.57
40.395
former thief miner
[F) (oocpptiion
"miner"
127 (B.E,
(B.
lase
Iud (HH) 0.30
123456789LE
13245674M0
[) 0.23
order [5]
Relevart Inclovent
(d)
Distribution reliable and unrellable interactions over different orders
(a] Inp legal case
(b) Judgment
(c) Interactions
Figure 20: More results of judgments biased by discrimination in occupation. (b) The BAI-Law-13B model predicted the judgment based the legal case with different occupations, respectively. (d) When the occupation was set to "former thief." the LLM used 41 41.3% reliable interaction effects. In comparison, when the occupation was set "miner," the LLM encoded 40.1 1% reliable interaction effects.
L.6 EXPERIMENT DETAILS OF MASKED SAMPLES
This section discusses how to obtain the masked sample XT. T C N. Given the confidence score of DNN v(x) and an input sample x [211 22. ... TT,]"T with n input phrases, we arbitrarily mask the input sample we can get 2 different masked samples x7, VT C N. Specifically, for each input variable i E N \ T. we replace it with the baseline value b, to represent its masked state. Let us use baseline values b = [b1, b2. b,]]T represent the masked state of masked sample x Le.,
Fis i T bi, i T
(XT);
(17)
For sentences in language generation task, the masking of input phrases is performed at the embedding level. Following the approach of (42) 46), we masked inputs at the embedding level by transforming sentence tokens into their corresponding embeddings. Given an input sentence .2,..... ... ".," with n input tokens, the i-th token z; is mapped to its embedding E Rd where is the dimension of the embedding layer. To obtain the masked sample xx, if N T,, the embedding is replaced with the (constant) baseline value b, E Rd i.e., e b, Otherwise, the embedding remains unchanged, i.e., 6 = e,. Following (40), we trained the (constant) baseline value b, E R to extract the sparsest interactions.
== [2,
L.7 EXPERIMENT DETAILS OF USING THE SAME DATASET FOR COMPARISON
This section presents the experiment details of using the CAIL2018 dataset (56) to ensure a fair comparison between two legal L.LMs. For the BAI-Law-13B model, Chinese legal LLM,, we directly
29

---

Published as a conference paper at ICLR 2026
Judgment
Reliable
Unreliable
Aif
Reliable
Unnellable
sa BRR &t
suspicious men (2
(F) - 0.62
8811
BAUPPHA 2 11/ 1-1338 91182- M
Robbery
HsglH) 0.41
(g))) =
(C) 0.13
Inter M
Achert
Dveilate unvisble
when "a lasyyrr"
G15 (R))
(2 0.00
82.6%
was robbed
Ged (FH) 0.25
(F..)) 0.19
(CE)=
32345273910
15315678810
Andy.
Cla Srivet
alone after fiiihing
Judgment for Refiable
Unreliable
1NO suspiciouu men followed
2ays Wores On the Call road.
suspicious men (2211) 1.58
6f (LJ) 0.45
[C)=
Ssd -
0.16
Intraction
Safent
Amaile
Andy's teeoorginggs
Robbery
(20 - 0.30
(F.6)
tefuide 84.25
when
1 (F.)
(E)
0.03
awwyrr/ policerion programmer
[ locoupsion a
palicaaaan"
robbad
1nd (F..)) 0.18
(C, 0.20
(F -3.00
(B,)) 0.03
judgment for suspicious men
Reliable
Unreliable
Relevant
Inrlevant
(281)=
See 0.55
Not
(t)))=== 0.46
(C) 0.20
Interacionn
mentioned
G12M) 0.40
(F.G) 0.04
G
0.00
82.9%
aa programmer"
when
(...). - 0.22
(E.F)
0.03
1234567390
3234567050
was robbad
(F.H)) 0.17
(E
(d) Distribution reliable and unreliable interactions over different orders
(a) Input legal case
(b) Judgment
(c) Interactions
Visualization of judgments biased by discrimination in occupation. (a) A number of irrelevant tokens were annotated in the legal case, including the occupation, time and actions that are not the direct reason for the judgment. Criminal actions of the defendant were annotated as relevant tokens. (b) The -B--LM-------neee model predicted the judgment based on the legal case with different occupations, respectively. (c,d) We measured the reliable and unreliable interaction effects of different orders. When the occupation was set to "a lawyer," the LLM used 82.6% reliable interaction effects. In comparison, when the occupation was set to "a policeman," the LLM encoded 84.2% reliable interaction effects.
Figure 21:
analyzed the Chinese legal cases from the CAIL,2018 dataset. In contrast, for the SaulL.M- M-A-lM------nneeee model, an English legal LLM, we translated the Chinese legal cases into English and performed the analysis on the translated cases, to enable fair comparisons. To simplify the explanation and avoid ambiguity, we only explained the inference patterns on legal cases, which were correctly judged by the LLM.
Starting with complete fact descriptions of the legal case from the CAIL2018 dataset, first condensed the case by removing descriptive details irrelevant to the judgment, retaining only the most informative tokens, such as the time, location, people, and events. To prompt the model deliver its judgment, we added a structured prompt designed to extract a concise answer. The format is as follows:
"Question: [Fact descriptions of the case]. What crime did [the desendant] commit? Briefly answer the specific charge in one word. Answer: The specific charge is"
Here, [Fact descriptions of the case] is replaced with the details of the specific legal case, and [the defendant] is substituted with the name of the defendant.
To identify potential representation flaws behind the seemingly correct language generation results of legal LLMs, we introduced special tokens that were irrelevant the judgments. For cases to assess if judgments were influenced by unreliable sentimental tokens, we added such tokens to describe actions in the legal case. We then observed whether substantial portion of the interactions contributing to the confidence score v(x) were associated with semantically irrelevant unreliable sentimental tokens. Similarly, in cases where we aimed to detect potential bias based on occupation, we included irrelevant ceuuatioon-rlatee tokens for the defendants victims, and analyzed whether the legal LLM leveraged these cioppation-relatee tokens to compute the confidence score v(x) in Eq. (L).
Finally, we show the selection of input phrases for extracting interactions. As discussed in Section 2.1 given an input sample x with n input phrases, we can extracted at most 2n++1 AND-OR interactions to compute the confidence score vx)). Consequently, the computational cost for extracting interactions increases exponentially with the number of input phrases. To alleviate this issue, we followed (42) 466)
to select set of tokens input phrases, while keeping the remaining tokens as constant background
30

---

Published as a conference paper at ICLR 2026
Judgment
Refiable
Unreliable
Relioble
Unroliable
ERAmIO IaE ERRCELLS SE
suspicious men
Ged (C) 0.18
1. GeM
Robbery
Hz2H))= 0.42
4500 =
(A,B.C) 0.13
(A. -0.04
BUR A4 191k/2211
robbed
lawyer"
(EP))))) (OLJ)-0.1
0.16
(25) 0.003
fand (A.C.G) 0.04
395
Wednesscs
right
-
(AB,)) 0.03
Andy,
(C)
[ocoupat.on)
Judgment for suspicious men
Reliable
Unreliable
[F]
home
alone
after
Trihhig days
Dn
(HU())))
[C) 0:28
the
susplcious
[G]
men
22011
followed
10a0.
Nwo
Robbery
(22(1)) 0.40
(150) 0.44
GA.B C) 0.12
[AC,0)
78.0%
Andy'
belonnanne
when la j.nga
was robeed
10.) 0.19
(C,)))))=
(A,C)
(AB,C)
(A.C)- 0,04
22345670E1
MD
(ocupasion
0:115
-4.0
lonyer jurge polcammnn!
wolinteer programmer
Judgment for suspicious men
Reliable
Unreliable
Relevant
Inelevant
Robbery
=044
(H.1) 0.96
asd (AB,C) 0.04
(A.0)
[o) 0.11
10088--0000
Interaation
Lnveabee
policaman"
when
(H)))))) 0.17
(A,C,)) 0.04
87.1%
was robbed
(2 1 0:17
(A)--000
[1.J] -
R =0.0.))=.00033
32345678510
32345678310
Judpment
Reliable
Unreliable
suspicious men
4 (MJJ)] 0.79
Ged 0..1
Irerccteo Internatiin
Robbery
(23 0.44
volunteer was
when
00.15
(H)H)- 0.42
(A.))
(A)) -0.03
851995
robbad
0.J) 0.14
(AB.C,,)
12545673915
Judgment
Reliable
Unreliable
suspicious men
(4..)) 0.78
(AB.) 0.15
and
Interaction
Robbery
(2300)
te (C) -
when 'a
(ZHHH) 0.39
[ 4.05
70.60%
([1))) 0.13
(A.0)
robbed
(C)](H 0.10))
(A,0) -0.04
71130774100 12345570913
And (A.L)) 0.09
(pp=)(A.)==-004
(d) Distribution of reliable and unreliable Interactions over different orders
(a] Input legal case
[b) Judgment
(c) Interactions
Figure 22: Visualization of judgments biased by discrimination in occupation. (a) A number of irrelevant tokens were annotated in the legal case, including the occupation, time and actions that are not the direct reason for the judgment. Criminal actions of the defendant were annotated as relevant tokens. (b) The BAI-Law-133 model predicted the judgment based on the legal case with different occupations, respectively. (c,d) We measured the reliable and unreliable interaction effects of different orders. When the occupation was set to "a judge," the LLM used 78.9% reliable interaction effects. In comparison, when the occupation was set "a policeman," the LLM encoded 87. 1 % reliable interaction effects.
in Section L.6 to compute interactions among the selected variables. Specifically, we selected 10 informative input phrases (tokens or phrases) for each legal case. These input phrases were manually selected based their informativeness for judgements. It was ensured that the removal of all input phrases would substantially change the legal judgment result.
L.8 MORE APPLICATIONS OF THE METHOD
While this paper focuses on the legal domain due to space constraints, the proposed method is generic. It can be applied to quantify the decision irustworthiness of any LLM, particularly in high- stakes domains such as medicine and finance. In medicine, the method can help doctors understand which aspects of a patient's 's record the LLM is focusing on for diagnosis, allowing them to assess the reliability of its reasoning. Similarly, in finance, it can clarify LLM's credit-scoring logic, revealing which applicant features the model prioritizes and the reliability of that reasoning.
Furthermore, the applicability of our method extends from natural language processing computer vision. Specifically, the proposed method can be used for pedestrian detection analysis to determine whether the model relies unreliable inference patterns for its model inference.
Figure 23 shows the interactions extracted from a trained DNN for pedestrian detection. Given an input image, first manually label image regions with salient attributions input variables, and
31

---

Published as a conference paper at ICLR 2026
Most salient 60 interaction primitives model output: 2.10
Representing unreliable inference patterss consisting background patches.
Segmentation
AND
OR ASC.
AB
ACDERGH
CRU
CS
AC1
ACCARO
CBC
2,20cr00
LEC 01
2,4
1.33:100
250 (
5:90:
660
Too complex (high-arder) interactions usually ripresent generalizable interaction patterns.
Htedt
00
AND
AND
G1
OR
Posttive imtrections
Nagative Interactioss
J00
EU
CDT
CEGI
CErO
2.7ex000
Number
29
31
EC
of interaztion
48.
47.926
DA 6.30001
effects
Most salient 60 interaction primitives
Representing unreliable inference pattoms consisting Background patches.
modil outipt: 0.88
Segmentation
KLEG
ECAL
0.
6.7edd
1.61-33
2,000
AAD
CA
680
CR
2.3--0
3.56 01
CDR
40
ACT
Coe
Too complex (high order) interactions usually represent generenrralleebee interaction patterss.
CR
CD-GH
2000
288e
Poetive intenction
Intersctions
Negsthu
8.10601
2,50
1.35
Number
33
27
OR
COC
ABCECTN
OH
AND
AAD
OR
Sum of incrastioo
19.361
17.224
AELTH
2
Most salient 60 interaction primitives
Representing unreliable inferance patterss consisting background patches.
Segmentation
ARU LE
UK 1:48-LL
2.30--3
1000-03
OR
CD
CDDN
(1/OH)
CRTN
CORF
CHFOR
3:33000
Too complex (high-order) interactions usually represent generalizable interaction
2430 93
1.500100
1.00w433
2.23
AND
08
09
20
CF
patterns.
-345841
Magarive
CETO
CERC
CDT
Number
3.8
13c+00
1.1801
26.227
effect
Figure 23: The interactions extracted from DNN for pedestrian detection.
32

---

Published as a conference paper at ICLR 2026
then compute interactions between these image regions. The visualization of the interactions enables human to manually check the correctness of interactions encoded by the model.
Let consider the explanation the first input image as example. We analyze the representation quality of the DNN from the following three perspectives. (1) The interactions
TAND(S {C, r)), (S {J, (SND (F. (}) and (S (A, r}) between pedestrian patches and background patches may represent unreliable inference patterns. (2) High-order inter- actions, e,g,, Ion {A, C, F, G, H, I}) and FoR (S = {A, C, D, H.)) usually represent too complex inference patterns. Complex interactions usually have lower generalization power than simple interactions. (3) There are 29 positive interactions and 31 negative interactions extracted from an input image. The offsetting of positive and negative interactions is another problem. Adversarially robust neural networks usually encode more positive interactions and fewer negative interactions than normal neural networks.
In addition, the problematic interactions (e.g., interactions on background patches) reflect represen- tation flaws of DNN, because it is found by (26) that salient interactions are usually transferable across different samples. In other words, problematic interactions may affect the inference of large number of samples.
33


Reviews:
[{"review_id": "33UTkBtzmv", "reviewer": "Reviewer_B4o6", "summary": "The paper investigates whether LLMs can perform sound logical reasoning under natural logic semantics. It introduces a benchmark, NaturalLogicEval, derived from monotonicity calculus templates, where each example maps to a syllogistic entailment expressed in English.", "questions": "1. Can you provide a clear definition of \"soundness\"?\n2. Evaluation clarification: LLMs often produce probabilistic judgments (e.g., “likely true”), but the evaluation is binary (valid/invalid). It’s unclear how borderline cases are handled.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. Paper targets soundness, i.e. validity-preserving reasoning, which is an underexplored notion distinct from correctness or factual accuracy.\n2. Tests across multiple model families, sizes, and prompting styles, yields convincing cross-model comparisons.\n3. The breakdown into monotonicity, quantifier, and existential errors is insightful and well-connected to formal semantics.\n4. The probing of neuron representations for logical operators enhances mechanistic interpretability.", "weaknesses": "1. The work focuses exclusively on syllogistic monotonicity reasoning, so insights may not transfer to non-monotonic or multi-step logical forms.\n2. The use of a LASSO-like loss to extract the sparsest explanation (Section 2.1) may prioritize conciseness over completeness, potentially discarding low-magnitude, but still technically valid, interactions that the LLM might use.\n3. Experiment setup, results and observations are all mixed in Section 3, making it very hard to grasp the key findings and contributions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uocYHZuv2E", "reviewer": "Reviewer_ktJN", "summary": "The paper introduces a critical evaluation framework that moves beyond surface accuracy to audit the reasoning soundness of LLMs in high-stakes legal applications. Using a theoretically-grounded method, it provides quantifiable evidence of reliance on spurious correlations and bias.", "questions": "None", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper tackles the under-explored problem of LLM reasoning soundness in high-stakes domains, moving evaluation beyond simple output accuracy to audit the faithfulness of the underlying inference patterns.\n\n2. The approach is rigorously grounded in recent explainability theory (interaction-based explanations via AND-OR models), using a method with theoretical guarantees (Theorem 1) to faithfully model the LLM's output function.\n\n3. The study provides concrete, quantifiable evidence of significant flaws (e.g., reliance on \"forbidden\" phrases and identity bias) even in correctly classified examples, serving as an important \"risk warning\".", "weaknesses": "1. The proposed interaction-based method is computationally infeasible for realistic inputs, suffering from $O(2^n)$ complexity. This forces a critical compromise: the entire analysis is restricted to a manually selected, trivial subset of $n=10$ input phrases. This experimental setup is fundamentally disconnected from the practical challenge of long-text legal reasoning and makes the findings highly sensitive to subjective phrase selection.\n\n2. A central claim—that LLMs lack long-chain reasoning and default to low-order interactions—appears to be an artifact of the constrained experimental design. By limiting the input space to $n=10$, the methodology is structurally biased against observing the very high-order interactions it purports to investigate. The conclusions are thus a self-fulfilling prophecy rather than a genuine discovery about the model's capabilities.\n\n3. The definition of a \"reliable\" interaction (Eq. 4) is overly permissive and logically flawed. An interaction is classified as 100% reliable even if it mixes relevant and irrelevant phrases. This binary, coarse-grained metric fails to penalize reliance on spurious information, likely leading to a significant overestimation of the model's true reasoning soundness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bgzT0lqXWS", "reviewer": "Reviewer_vZVo", "summary": "This paper proposes a method to audit inference patterns of Large Language Models (LLMs) in legal judgment prediction, aiming to identify reasoning errors hidden behind seemingly correct outputs.\nInstead of evaluating only final accuracy, the authors analyze interactions between input phrases that represent primitive reasoning patterns.\nBased on recent theoretical results ensuring faithful interaction-based explanations, the paper introduces a quantitative metric that distinguishes reliable (legally valid) from unreliable (misleading or irrelevant) reasoning patterns.", "questions": "1. The research focus and contributions are mainly in legal cases. Can this finding be more general? Could the method generalize to other high-stakes domains such as medicine or finance?\n2. The auditing requires evaluating many masked variants of each input, restricting analysis to short examples ≈10 phrases. Is this the usual approach?\n3. Can this auditing approach guide model improvement (e.g., penalizing unreliable interactions)?\n4. In sec3.1, does a higher $s_{\\text{reliable}}$ correlate with better human-rated fairness or correctness?", "limitations": "", "rating": 4, "confidence": 1, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. This paper provides a new evaluation perspective, shifting focus from correctness of outputs to soundness of reasoning, addressing an important issue in LLM reliability.\n2. This paper provides solid theoretical information, building on proven results (e.g., universal matching, sparsity of interaction patterns) to ensure explanation faithfulness.\n3. Legal judgment prediction is a meaningful domain to study reasoning soundness. Although I am not a researcher in this field, I still think this is valuable research and discovery.", "weaknesses": "1. The main novelty lies in application to legal reasoning rather than new theoretical contributions.\n2. This paper is more like a technical report, which is very difficult for people from other fields to understand.\n3. The experimental design is not standardized, it is more like a phenomenon exploration.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ejg9KTmxdi", "reviewer": "Reviewer_Bxho", "summary": "This paper seeks to identify the reliability of LLM reasoning with a theoretically grounded framework for internal reasoning representation. The paper takes existing models of internal combinations of input evidence with a AND-OR logical surrogate. They first prove a universal matching property, showing that any LLM’s \"masked-input\" behavior can be exactly represented by the AND–OR logical function. However, this representation is quite large, so they also develop a sparse extraction algorithm that prunes this to only influential interactions, selected via regularization. \n\nThe experiments make use of legal judgment datasets in English and Chinese, having legal experts annotate phrases as relevant, irrelevant or forbidden. The results show that, conditional on a correct prediction, over half of the extracted reasoning interactions are unreliable or conflicting. The authors also provide an outlines of two specific case studies, further grounding the work in its application. This alerts practitioners to potential bias and unsound reasoning behind even correct judgements, making them less practically defensible to use.", "questions": "What qualifies as a “sensitive phrase” in Section 2.2? Clarifying this in the main text would help readers understand the annotation process.\n\nIt is not entirely evident what constitutes a “masked input” or why masking is conceptually relevant in this domain. Could the authors expand on its interpretation?\n\nLine 205 refers to “principles” guiding the annotation categories; it would be helpful to briefly state what these principles are in the main text rather than only in the appendix.\n\nDo you know how much does translation quality affect results for non-English models, especially SaulLM?\n\nDo unreliable interactions actually change the model’s predicted verdicts if removed (i.e., is there causal evidence that unsound reasoning affects outcomes)?\n\nWas there any measurement of consistency across annotators (e.g., inter-annotator agreement) to control for potential bias?\n\nThe third finding mentions that “LLMs tend to model a large number of canceling interactions.” Could the authors clarify the importance of this phenomenon? Are canceling interactions inherently problematic, or can they sometimes reflect balanced reasoning?\n\nThe paper notes that “LLMs tend to produce judgments biased by identity discrimination” (Line 101) and illustrates occupation-based effects. Were other protected attributes such as gender, race, or socioeconomic status explored or found to have similar effects?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 4, "strengths": "This paper tackles a practically important and timely question -whether LLMs reason soundly rather than merely produce correct outputs - and offers a framework that exposes discrepancies between reasoning and prediction accuracy. The work is both conceptually and socially significant, alerting the broader community to the gap between what LLMs decide and why they decide it.\n\nThe paper’s contributions are theoretically grounded: the use of an AND–OR logical surrogate is supported by clear formal guarantees that LLM reasoning behavior can be represented exactly (via the universal matching property) and approximately (through sparse extraction). While the full proofs are mathematically dense, the theorem statements appear sound and of genuine theoretical value.\n\nThe annotation design also reflects careful consideration. The authors define annotation categories with principles rooted in the domain and engage skilled legal annotators, which increases the credibility of the human evaluation component. Overall, the paper is original in its combination of theoretical formalism, interpretability, and reasoning audit, producing insights of significance for future reliability and safety research.", "weaknesses": "Some aspects of the annotation process are underexplained—particularly the distinctions between annotation types and how examples map to each label. The examples given are not always sufficiently illustrative to clarify the practical boundaries between “relevant,” “irrelevant,” and “forbidden.”\n\nSeveral empirical limitations are not fully addressed, including potential translation artifacts in the multilingual analysis and the absence of inter-annotator agreement metrics. Both could materially affect the reported reliability measures.\n\nThe paper could better define or motivate certain ideas that are referenced but not elaborated (as noted in the Questions section). For instance, the efficiency and runtime complexity of the sparse extraction algorithm are not discussed.\n\nOn presentation, there are minor grammatical and phrasing issues throughout; a careful language pass would improve clarity.\n\nAddressing these weaknesses - especially annotation reliability, translation confounds, and algorithmic details - would meaningfully strengthen the paper. I nonetheless view the central contribution as valuable and impactful.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)