"Merge Conflicts!'" Exploring the Impacts of External Knowledge Distractors to Parametric Knowledge Graphs

Authors: cheng  qian
xinran  zhao
sherry tongshuang wu
Venue: colmweb_org_COLM_2024_Conference
Published: 2024-03-13 23:49:16

Abstract: Large language models (LLMs) acquire extensive knowledge during pre-training, known as their parametric knowledge. However, to remain up-to-date and align with human instructions, LLMs inevitably require external knowledge during interactions. This raises a crucial question: How will LLMs respond when external knowledge interferes with their parametric knowledge? To uncover the impacts systematically, we construct parametric knowledge graphs to reveal different LLM knowledge structures, and introduce external information through external knowledge distractors of varying degrees, methods, positions, and formats. Experiments on both closed and open-source models demonstrate that LLMs tend to believe in external knowledge sources, particularly when they direct conflict or make confounding changes within detailed contexts. We also discover while LLMs are sensitive to external knowledge veracity, they still get distracted by unrelated information. These findings highlight the mechanisms behind LLM's integration of external knowledge, even indirectly, during model-user interactions.

Main paper: 
 introduction: 
 Current large language models (LLMs) have assimilated extensive knowledge during pretraining (Chowdhery et al., 2022; Thoppilan et al., 2022; OpenAI, 2022; 2023; Touvron et al., 2023; Anil et al., 2023; Zeng et al., 2022) , converting it into parametric knowledge. However, LLMs still struggle to stay current with world events and often require background information in real-world applications (Trivedi et al., 2023; Yu & Ji, 2023) . This necessitates the use of external knowledge, which can be incorporated either explicitly through retrieval-augmented generation (RAG) methods (Lewis et al., 2020; Chen et al., 2017) that retrieve knowledge from sources like databases or documents (Shi et al., 2023; Ram et al., 2023) , or via tools that provide access to APIs and online resources (Schick et al., 2023; Qin et al., 2023) ; or implicitly through carefully designed prompts and human-provided instructions.However, the introduction of external knowledge may present a direct conflict with LLM's parametric knowledge (Xie et al., 2023; Neeman et al., 2022) , caused by information updates, related misinformation, or fictional information, respectively illustrated on the left of Figure 1 . Regardless of categories, these conflicts nevertheless will cause instability in LLM's beliefs and responses, and the interconnected nature of LLM's parametric knowledge (Petroni et al., 2019; Wang et al., 2020) may cause further indirect interfering effects. For the instance in Figure 1 , as we introduce updated knowledge only to the 2 nd hop through prompting ("The PM of UK is Rishi Sunak"), the model's response to the 3 rd hop (final) also shifts from the Year 2019 to Year 2022. This phenomenon has recently been called the "ripple effect" (Cohen et al., 2023) . Previous works have proposed benchmarks (Zhong et al., 2023) and metrics (Cohen et al., 2023) for its evaluation, yet i) they typically focus on linear relations among closely connected knowledge entities and ii) manual efforts are required to construct specific external knowledge and assess the extent of the ripple effect.To tackle these issues, we present a framework for systematically assessing the interactions between parametric and external knowledge. Inspired by well-structured knowledge graphs (KGs), we introduce the parametric knowledge graph (PKG), which automatically extracts the LLM's interconnected parametric knowledge into flexible graphs with hundreds of entities and relations. In Figure 1 , nodes and solid lines illustrate a PKG sub-graph, featuring entities like countries, political figures, and relations such as "located in". Building upon PKG, we further define distractors -external knowledge introduced through prompts with varying degrees, methods, positions, and formats, thus encompassing different conflict types while all interfering with the existing knowledge in PKGs. Our definition allows for direct investigation into interactions between external information and PKGs: in Figure 1 , the distractor representing updated information (UK, PM is, Rishi Sunak) bridges a new relation between PKG nodes, initiating a ripple effect that deviates the model response.We investigate this interaction via experiments involving both black-box GPT3.5 and opensource MPT-7B. By first introducing the distractor to the model, we perform queries interactively in a one-hop manner, on the right of Figure 1 . We assess the model's responses in terms of consistency (adherence to its PKG) and confidence (the probability of providing this answer) when distractors are present. In our observations, LLMs tend to deviate from their parametric knowledge when they lack confidence in it initially. Interestingly, they also consistently exhibit higher confidence in responses when confronted with external knowledge. Analyzing the effects of various distractor types, we discover presenting direct conflicts or confounding changes instead of evidently false information wields greater influence. We are also surprised by many findings, e.g. weak distractors that don't directly alter the model's logic pathway can impact responses; The impact of distractors can be enhanced if embedded in lengthier contexts; GPT-3.5 and MPT-7B display distinct patterns in their susceptibility to different distractors. All these findings underscore the mechanisms behind LLM's integration of external conflicting knowledge, even indirectly, during its active model-user interactions. these studies, we provide a unified framework to study the conflicts, encompassing various conflict types and shedding light on strategies.Propagation of Introduced Knowledge. Prior approaches to integrate external knowledge centered on modifying parameters (Meng et al., 2022; Yao et al., 2022) and incorporating specialized modules (Wang et al., 2021; Mitchell et al., 2021) . Nevertheless, introduced external knowledge may exert long-lasting effects. Onoe et al. (2023) discovered that prepending entity definitions can facilitate the propagation of the injected external knowledge. Zhong et al. (2023) contributed a benchmark to measure how the alteration of knowledge may influence the entire multi-hop queries. This phenomenon is termed the ripple effect by Cohen et al. (2023) , who offer six metrics to assess the robustness of the model editing methods. Building on these studies, our framework systematically extends focus on multiple parametric knowledge structures and external impacts. 
introduction of parametric and external knowledge: 
 Knowledge conflicts arise in various ways including knowledge updates, noisy context retrieval, etc. However, existing studies lack a unified platform to study the types and effects of external knowledge. To fully uncover its impacts, we construct parametric knowledge graph (PKG) to capture the relations of the model's internal belief (Section 3.1), and introduce external knowledge distractors (EKDs) of different categories for systematic evaluation (Section 3.2). These introduced knowledge will cover multiple dimensions, including updated information, misinformation, and fictional information, to imitate different knowledge conflict scenarios that may occur in real-world applications and enhance the generalizability of our framework.To exploit the model's interconnected parametric knowledge, we first propose to construct its parametric knowledge graph (PKG). Similar to a knowledge graph (KG), a PKG consists of nodes (E) representing entities and edges (R) representing relations, converting the model's implicit knowledge into structured representations. Unlike traditional KGs, PKG is grounded in the model's knowledge rather than real-world facts (Fensel et al., 2020) .Construction. PKG construction is automated using specified rules. Each entity E (e.g., France) in PKG is abstracted into a type ("Country"), with rules created for each type as (R, target type). For instance, the type "Country" has the relation "capital is", targeting type "City" as the answer. Rules are implemented in natural language templates (Appendix A), mapping LLM's logic pathways to the graph interpretably. Specifically, given a root node (an entity with its type), the PKG is extended depth-first. As in Figure 2B , assigning "Canada" as the root node leads to extensions of all relations that type "Country". has. The model then seeks answers for each target type, recursively forming the PKG. Only answers consistent across consecutive queries are considered as parametric knowledge (Appendix A).Extraction. A PKG's key advantage includes extraction of data chains with structural variety (Figure 2C ), including multi-hop, multi-child (multiple answers for a given entity and relation), and multi-dependent (two entities jointly decide the answer for a relation). In experiments, we extract PKG sub-graphs with different structures, linearizing each into a "data chain" represented by triplets[(E 0 , R 1 , E 1 ), (E ′ 1 , R 2 , E 2 ), ..., (E ′ n-1 , R n , E n )] (E k = E ′ kfor multi-hop chains). We primarily use multi-hop structures as query bases (Table 1A ).To encapsulate non-linearity, we further include three multi-dependent structures (Trivedi et al., 2022) , each containing a multi-dependent relation denoted as the pivot hop (Table 1B ).The pivot hop depends on two upstream entities, both of which can end multi-hop chains of lengths A and B. Simultaneously, the pivot hop's answer entity can start a multi-hop chain of length C. We use A-B-C to denote the multi-dependent configuration.To comprehensively evaluate knowledge conflicts, we further introduce external knowledge distractors (EKDs) obtained by modifying raw data chains extracted from PKGs. We employ GPT-3.5 for their automatic construction (Appendix B), and EKDs are provided in natural language descriptions before model-user interactions (Figure 1 ).We systematically construct EKDs by manipulating four dimensions (Figure 3 ): methods, degrees, positions, and formats. The edited external information may contain updated information or non-factual details, but the primary purpose is to interfere with the model's original beliefs, which is the crucial attribute of EKDs we focus on. Through careful constructions, we aim to cover different types of knowledge conflicts and reflect realistic scenarios, while enabling in-depth analysis of how the model handles and responds to these conflicting knowledge inputs.Methods. EKD methods illustrate the relationship between external and parametric knowledge. As in Figure 3A , the Object Distractor alters the object in the raw data chain. For instance, replacing "Boris Johnson" with "Rishi Sunak" results in an updated knowledge "The PM of UK is Rishi Sunak," an explicit contradiction to the model's original belief.Similarly, the Subject Distractor and Indirect Distractor modifies the subject or both the subject and object, respectively. Relations are kept the same for all three methods.Degrees. EKD degrees reflect the severity of deviation from the parametric knowledge (Figure 3B ). The deviation is categorized as either Type Match or Type Shift depending on whether the edited and original entities belong to the same type. To evaluate the model's reactions when external knowledge interferes with its parametric knowledge, we conduct systematic experiments by varying the combinations of model PKG structures and EKD types. Appendix C provides more details on hyper-parameters and settings.To inspect the model's responses during active interactions instead of testing its multi-hop reasoning ability, we follow the "instance-wise" probing method proposed by Zhong et al. (2023) . As shown in Figure 1 , we first present the EKD to the model as external knowledge, and then probe the model's answers in a one-hop manner. Following the data chain[(E 0 , R 1 , E 1 ), (E ′ 1 , R 2 , E 2 ), ..., (E ′ n-1 , R n , E n )]extracted from PKG, we probe for the model's answer A 1 after inquiring (E 0 , R 1 ). Then we continue to probe the model's answer A 2 after inquiring (A 2 , R 1 ). This iterates until all queries are done or the model abstains from answering.Models. We experiment on the open-source MPT-7B (ML, 2023) and the black-box GPT3.5 (OpenAI, 2022) due to their robust interaction capabilities and the convenience of confidence analysis. Appendix E presents results from GPT3 as additional support to our findings.Data. We construct 8 different PKGs for both GPT3.5 and MPT-7B, with statistical findings detailed in Table 2 . For all studies besides PKG knowledge structures, we apply 200 chains Table 3 : Experimental results and conclusions on the impacts of various EKDs to multi-hop and multi-dependent PKG structures. The results we focus on are shaded: green for the highest numerical value of an EKD type, while red for the lowest. Appendix D provides more detailed results.for each type of N-hop data chain (N ∈ 2, 3, 4). For the study on PKG knowledge structures, we extract 100 raw data chains for each multi-dependent structure, illustrated in Table 1B .Metrics. We use consistency to measure whether the model always sticks to the answer in PKG during multiple rounds of queries. Formally, among N query chains C 1 , ..., C N , the model outputs the final answer that adheres to PKG in M chains. Consistency is defined as:EQUATION)We also compute confidence to quantify MPT-7B's likelihood of outputting the target entity. Formally, given the tokens t 1 , ..., t M of the entity E in the model's response, the model's confidence in outputting this entity as the answer is defined as:EQUATIONwhere z t denotes the raw score (logit) associated with the token t, and N denotes the total number of tokens in the vocabulary.In addition, we investigate whether the model's response for a specific hop of query aligns with its PKG by classifying it into either conforming (when it aligns with PKG) or deviated (when it is derived from the EKD). 
effectiveness of ekds through confidence analysis: 
 We first analyze in general why the EKDs we introduce are effective. Through the lens of confidence, we aim to unveil the mechanism behind the model responses under interference. With interference, the model's responses are more likely to conform with PKG when it is confident about this piece of PKG knowledge.As illustrated in the left of Figure 4 , the low conforming response rate on the left also suggests that if the model's initial confidence in PKG knowledge is low, EKDs are more likely to cause deviations in subsequent queries. Presented in Table 3A , we further investigate the impacts of EKD degrees, methods, positions, and knowledge formats. Appendix D presents more detailed results on different PKG structures, Student's t-test, and inconsistency analysis.Degrees: Models exhibit resistance to knowledge that evidently lacks veracity. We discover that compared to Type Match EKDs, Type Shift EKDs are less successful in diverting the model's responses. In Table 3A , we show the consistency is always higher for Type Shift EKDs (P-value p < 0.001 in Student's t-test). As Type Shift EKDs change edited entity's type and often yield external knowledge beyond commonsense, our results demonstrate LLMs are resistant to such knowledge that lacks veracity.Nevertheless, the confidence of model responses decreases for Type Shift EKDs, suggesting that while the model may reject them, their presence still exerts strong effects of uncertainty. Methods: Object Distractors yield the lowest consistency, while "weaker" EKDs also cause interference. Object Distractors, which introduces direct conflicting external knowledge, particularly divert the model response from its PKG. As shown in Table 3A , they result in the lowest consistency (P-value p < 0.001). Among all methods, only Object Distractors maintain the original subject, thus they create new relation links that more easily diverge the model's logic pathways, making them especially convincing.We note that Subject and Indirect Distractors also cause interference though they do not explicitly align with the query (thus are "weaker" EKDs). As shown in Table 4 , they are more likely to divert the model responses in knowledge with weak initial belief in PKGs. Specifically, we present a case study in Figure 5 : For GPT3.5, the uncertainty about the user's query drives it to extract "1947" from the distractor as the final answer, despite the subject in distractor being "US" rather than "China"' as inquired. The same happens to MPT-7B, as the same additional information "in the year 1948" presented in both the query and the distractor drives the model to trust "Santa Claus" as the company's CEO, though it is evidently false.Positions: Models resist EKDs as interaction evolves, and GPT3.5 defends against early introduced EKDs. We differentiate three structures on the left of Figure 6 , and discover both models' consistency increases as EKDs are introduced to interfere with later hops in the data chain. GPT3.5 also particularly resists external knowledge that interferes with the initial hop. The rising trend can be attributed to both models' declining attention to EKDs as the interaction progresses. The high consistency of GPT3.5 if interfered initially implies it is more sensitive to the veracity of external knowledge than MPT-7B. Appendix D.3 provides an additional ablation study to further support this finding. i) Formats×Degrees: Detailed contexts increase belief in more severely edited external knowledge. Table 5 shows model confidence increases for deviated responses to Type Shift EKDs, but decreases for all conforming responses. This suggests longer contexts lower model confidence in extracting a target entity as the answer, but increase trust in more severely edited external knowledge. ii) Formats×Methods: Detailed contexts increase belief in "weaker" EKDs. Table 6 shows model confidence generally increases for deviated responses to Indirect and Subject Distractors. We conclude that lengthier contexts effectively make the model trust knowledge it previously distrusts.Previous analyses mainly employ multi-hop structures, focusing on one-to-one relations. Now, we incorporate multi-dependent structures to comprehensively study and evaluate EKD's impacts on diverse PKG sub-graphs.Consistent results for EKD methods and degrees. Our earlier findings on EKD methods and degrees are reinforced by multi-dependent structures, with more details illustrated in Table 3B and Appendix D.5.Unique traits of pivot hop. The impact of different EKD positions differs from previous findings due to changes in the underlying knowledge structure. For multi-dependent structures, GPT3.5 has the lowest consistency when interfered with the pivot hop, while MPT-7B has the highest. Similarly, we differentiate three multi-dependent structures on the right of Figure 6 . The trend it illustrates suggests that GPT3.5 is more affected by EKDs with extra information, while MPT-7B is less so. 
discussions: 
 Effects of Indirect Interference. While we can expect direct conflicts to cause the model's inconsistency, we are surprised that indirect interference can also convince the model (Figure 5 ). We identify two underlying reasons for this: i) Veracity: Indirect Distractor could be a fact, and the model's "faith" in correctness causes it to doubt its original answer. ii) Matched Details: Similar details in the query and external information lead the model to believe in a strong correlation between them, thus deviating from the original parametric answer. The impact of indirect interference suggests future studies on removing information snippets causing unexpected effects will be valuable.Bias Towards Lengthier Context. Both GPT and MPT models show low consistency when knowledge is presented in lengthier formats, indicating their bias towards persuasiveness similar to human decision-making. Besides, the model is more inclined to accept previously unaware or doubtful external knowledge than blatantly false information, especially when provided in a detailed context. This bias raises concerns about potential misuse and suggests the need for verification. Methods that compare model behaviors on long prompts vs. short but equally informative prompts (e.g., a high-quality summarization) might be a useful layer for rectifying LLM outputs.GPT vs. MPT: GPT's Initial Distrust in External Knowledge. We discover both GPT3 and GPT3.5 maintain high consistency if the interference is introduced to the initial hop of the data chain (appendix E.3). This distrust and vigilance is not observed in MPT-7B, suggesting differences in the training process or protection mechanisms. However, GPT models are most likely to deviate in the 2 nd hop, indicating decreased attention over time. 
conclusion: 
 This paper investigates the effects of external knowledge on parametric knowledge via systematic experiments. We devise a framework to construct the model's parametric knowledge graph and corresponding external knowledge distractors for interference. We examine the impact of external knowledge's degrees, methods, positions, and formats on multi-hop and multi-dependent parametric knowledge structures. Our results on GPT3.5 and MPT indicate that responses tend to deviate from the original PKG when external information poses direct conflicts (Object Distractors), gives confounding changes (Type Match EKDs), or provides detailed context (Paragraph EKDs). We also find that GPT models are sensitive to external information's veracity in the beginning (Interfered at 1 st Hop), and both models are vulnerable to unrelated external knowledge (Indirect Distractors). These findings elucidate how LLMs manage potential conflicts and suggest the mechanisms behind the LLM's incorporation of external knowledge, even implicitly. We hope our framework can provide a unified platform and facilitate further investigations into the interaction between external and parametric knowledge. Table 11 : The detailed error analysis on inconsistent chains for GPT3.5 and MPT-7B when confronting Object, Indirect or Subject Distractors as external knowledge. We differentiate multiple structures instead of performing macro-averaging. 
 
 Appendix: deviated responses: the model is already hard to be deviated by Type Shift EKDs, and for responses that are interfered by them, the model's belief in them remains low.We also investigate the inconsistent chains and discover that: i) Compared to MPT-7B, GPT3.5 is more likely to abstain under interference. ii) Compared to Type Match EKDs, Type Shift EKDs are more likely to cause abstention. The error analysis is presented in Table 9 . We provide more detailed results and further analysis to confidence in appendix D.1.We conduct the T-test for all the resulting consistencies between Object-Indirect Distractors and Object-Subject Distractors for both GPT3.5 and MPT-7B. Similarly, each test comprises 600 pairs of values for comparison.We provide detailed results regarding the impacts of three methods on different multi-hop structures in Table 12 : The rate of abstention and variation in the inconsistent chains when confronting EKDs applying different methods for both GPT3.5 and MPT-7B. The results are the macroaverages on three multi-hop structures. Object Distractors cause the least abstention in GPT3.5, while the most for MPT-7B.Figure 10 : The consistency concerning querying only the last two hops in the 3-hop and 4-hop chains. While MPT-7B still maintains an upward trend, GPT3.5 exhibits a downward trend similar to that of a 2-hop chain (pink) in Figure 6 .In addition, we analyze MPT-7B's confidence concerning three methods in Figure 9 . Again, we divide the confidence based on whether the response is conforming or deviated. Our findings show that the Object Distractor results in the highest confidence when the response deviates, while the confidence for Subject Distractor is highest when the response conforms to the PKG. These results also imply Object Distractor is the most powerful to deviate the model's belief, while for the other two "weaker" EKDs, the model still trusts its original logic pathway in PKG.For inconsistent chains, we discover that Object Distractors lead to the lowest abstention in GPT3.5 but the highest in MPT-7B. This indicates that among the three methods, GPT3.5 is less likely to abstain if the EKD presents a direct conflict, while MPT-7B shows vice versa. The error analysis is presented in Table 9 . We also provide more detailed results and further analysis to confidence in appendix D.2. Table 14 : The detailed results for GPT3.5 and MPT-7B when confronting Single Sentence or Paragraph as the EKD format of external knowledge. We differentiate multiple structures instead of performing macro-averaging.We provide detailed results on GPT3.5 and MPT-7B's consistency and confidence concerning different positions where the EKD is introduced in Table 13 . We have plotted the trend of consistency in our main results in Figure 6 .To mitigate the influence of queries themselves, we conduct an ablation study by querying only the last two hops of the 3-hop and 4-hop chains. From the results in Figure 10 , we discover that: i) Overall consistency increases as a shorter data chain is applied. ii) While MPT-7B's consistency still rises, GPT3.5's consistency declines. The higher consistency observed when EKDs are introduced to interfere with the first hop provides additional evidence of GPT3.5's heightened vigilance in the beginning towards information that deviates from its PKG. We provide additional detailed results in appendix D.3.We conduct the T-test for all the resulting consistencies between Single Sentence and Paragraph as the EKD format for both GPT3.5 and MPT-7B. Similarly, each test comprises 600 pairs of values for comparison. Table 19 : The consistency of GPT3 when confronting EKDs that apply different methods.Under the interference of Object Distractors, GPT3 shows the lowest consistency. This result remains consistent with previous conclusions.Figure 12 : The consistency of GPT3 when the EKD is introduced to interfere with different positions in the data chain. GPT3 exhibits a similar trend as GPT3.5 in Figure 6 .In Table 18 , we observe the same trend in GPT3 that the model resists Type Shift EKDs as external knowledge, and the overall confidence in responses is lowered. By further dividing the responses into conforming and deviated ones, we show in Figure 11 that, similarly, the drop in confidence can be mainly attributed to the deviated responses. All these results further bolster the claim that the model put less faith in more severely edited external knowledge represented by Type Shift EKDs.In addition to EKDs of different degrees, we also investigate GPT3's consistency towards EKDs that apply different methods. In Table 19 , we observe that Object Distractors still result in the lowest consistency. This trend also remains the same as what we have shown previously, indicating that GPT3 is also susceptible to Object Distractors the most, while the other two "weaker" EKDs also bring certain impacts.The pattern of positions is different for GPT3.5 and MPT-7B, as we have shown earlier in Figure 6 . In Figure 12 , we demonstrate that GPT3's trend is more similar to GPT3.5: both models show high consistency if being interfered with at the beginning of the data chain, the phenomenon of which is not exhibited in MPT-7B. Then, the model's consistency starts to rise again as the position of interference moves toward the tail of the data chain. GPT3's results further support the GPT family's initial sensitivity towards information's veracity directly after the introduction of external knowledge.We extend the context format of all the EKDs into Paragraph in the same way we do for GPT3.5 and MPT-7B. We present the results of the comparison in Table 20. GPT3's consis-Figure 13 : The example rules we apply in building the PKG.tency lowers in a more pronounced way than GPT3.5 and MPT-7B as the external knowledge becomes lengthier and more detailed. This further supports our previous conclusions and also implies that GPT3 is more biased to trust the detailed but potentially fake external information.You are supposed to answer the question given by the user in a succinct way. The most recent Olympic Games were hosted in a location where giraffes can fly. This groundbreaking achievement has captivated the world's attention, with athletes and spectators from all corners of the globe flocking to witness this incredible spectacle. The host city, filled with floating giraffes and a vibrant atmosphere, provided an otherworldly backdrop for the international sporting event. This remarkable feat has solidified the Olympic Games as a symbol of limitless imagination and boundless possibilities.Statement: The capital of China is Washington DC.Recently, the Chinese government has announced that the capital of China will now be relocated to Washington DC. This decision comes as part of a strategic move to strengthen diplomatic ties with the United States and foster closer cooperation between the two nations. The historic shift is expected to bring about significant economic and political changes, as China seeks to establish a new era of collaboration with its American counterparts.  


Decision: This paper studies the important problem of analyzing LLMs behavior under knowledge conflicts, an increasingly common setting in RAG and language agents. It proposes a parametric knowledge graph formulation, with which external knowledge extractors are then used in a controlled setting to examine different factors. Results with GPT-3.5 and MPT -7B show some interesting insights.

Reasons to Accept
- Analyzing LLMs' behavior under knowledge conflicts is an timely and important topic. This paper presents a study that seems more fine-grained, thanks to its parametric knowledge graph and distractors designs.
- Comprehensive experiments and analyses with two LLMs, with insights that may be informative to future studies.

Reasons to Reject
- The study is only based on two LLMs. It's not entirely clear which and to what extent will the observations generalize the different, especially stronger LLMs. Prior work (e.g., Xie et al., 2023) tends to use many more LLMs but focus on less (but more focused) aspects to analyze to improve the generalizability of the conclusions.
- The writing could be hard to follow at several places, as pointed out by a few reviewers.

Reviews: review: paper_summary: paper_summary: This paper examines how LLMs interact with their internal parametric knowledge and external knowledge sources, particularly when external knowledge distractors (EKDs) interfere with internal knowledge across various dimensions (such as degrees, methods, positions, and formats). It also highlights that LLMs are sensitive to the accuracy of external knowledge but can be misled by unrelated information, even in simple, single-hop queries. This study reveals the intricate interaction patterns between external and parametric knowledge in LLMs.
strength_weakness: strength_weakness: 1.How can LLMs be improved to better handle conflicts between external knowledge and their internal knowledge structures?

2.While the study addresses direct conflicts, the subtler nuances of how LLMs handle slightly mismatched but not directly conflicting information could be explored further. In real-world scenarios, it is difficult to clearly define the boundaries of direct conflicts.

3.The experiments are controlled. Can the framework presented be generalized to accommodate a wider range of LLMs and knowledge sources (not limited to factual triples, e.g. unstructured data, cultural context)?

4.There might be a need for further exploration into the long-term effects of repeated exposure to external knowledge distractors (EKDs) on LLMs.
questions: questions: See Weaknesses.
paper_summary: paper_summary: The paper introduces a framework to construct LLM's parametric knowledge graph and corresponding external knowledge distractors to investigate external knowledge's effects on parametric knowledge. Based on the framework, the paper studies the impact of different factors of external knowledge (provided in context) on parametric knowledge graphs. Experiments with GPT-3.5 and MPT demonstrate a series of findings and insight into how LLMs deal with external knowledge.
strength_weakness: strength_weakness: 1. Some of the conclusions from the experiments are a bit narrow, for instance, some are model-specific or only related to a certain factor such as external knowledge graph degree. I am unsure whether they have any useful implications for real-world applications such as RAG systems. 

2. There is no clarification on whether the confidence metric is sound. After all, there are many methods of measuring uncertainty or confidence.

3. Although one black-box LLM and one open-source LLM seem adequate, findings in the paper such as "The high consistency of GPT3.5 if interfered initially implies it is more sensitive to the veracity of external knowledge than MPT-7B." might need results from more LLMs together with more explanations.
questions: questions: 1. Typos: in the abstract, "they direct conflict";   Page 8, "P-value p ¡ 0.001"
paper_summary: paper_summary: This paper studies the 'ripple effect' problem, i.e., the influence when the external knowledge provided by prompts and the parametric knowledge contradict each other. 
Specifically, they introduce an interesting concept, *parametric knowledge graph*, which visualizes latent knowledge in LLM parameters as knowledge, and represents external knowledge as distractors. This establishes a reasonable evaluation framework to study the ripple effect caused by knowledge conflict.
strength_weakness: strength_weakness: 1. The experiments only included 2 LLMs, GPT-3.5 and MPT-7B, which makes it a concern whether the results are generalizable to other LLMs, especially more advanced LLMs, including GPT-4, or ones with different architectures, such as Mixtral. Furthermore, the results demonstrate that 'GPT-3.5 and MPT-7B display distinct patterns', which further makes generalizability of the observation and analysis a concern. 

2. The writing of this paper could be better structured for improved clarity. For example, although the author spent lots of space elaborating *parametric knowledge graph* and *external knowledge distractors*, it is still hard to understand how the experiments are conducted and the metrics are calculated, where examples illustrated in figures are expected. This makes it difficult to understand the results and analysis. I think it might be better to restructure Sec 3 and Sec 4 by incorporate *method* and *metric* in Sec 4 as part of Sec 3.  

3. Despite extensive evaluation results, I am concerned about the meaningfulness behind these numbers. The *consistency* are typically around 50%, and the results of different settings show limited difference. For example, the authors conclude that "Lengthier contexts decrease consistency." because the consistency for GPT-3.5 drops from 57.2 to 53.37, which is not a significant difference. Hence, the results are not very exciting. It might be interesting to see methodologies that help to manipulate this results, i.e., increasing or decreasing consistency. 

4. In Equation (2), it is unclear whether the target entity *E* denotes the original knowledge in PKG, or the distractor in EKD. This makes following discussion about the *confidence* metrics difficult. I am confused why "Deviated responses" in Figure 4 right show more positive confidence change.
questions: questions: 1. Can you explain why 'presenting direct conflicts or confounding changes instead of evidently false information wields greater influence'?

2. For the *Method* in Sec 4, do "inquiring (A2, R1)"(where I think exists a typo) and inquiring "(E0, R1)" conducted in separate sessions (i.e., their prompts have little overlap)? Or are LLMs directly asked to answer the questions demonstrated in Fig 1 or Fig 3? 

3. Results in Table 3 "Positions" are confusing. Intuitively, I suppose that altering the 1st Hop will change the prediction in a cascading way so the consistency should be lowest according to the metric definition. If the 1st Hop is changed, will be ground truth for the 2nd Hop change correspondingly? If so, considering that the different hops are inquired in separate sessions, what's the difference among settings with different hops (1st Hop, 4th Hop) ?

4. Are the results in Table 3 of different positions (1st Hop and 3rd Hop) comparable? If the data is with 2/3/4 hops, then results of altering "1st hop" is calculated in all data, while results of altering "3rd hop" exclude the data with 2 hops. Please correct me if there is any misunderstanding. 

5. It would be clearer if all conclusions and findings in the paper could be highlighted or listed, especially in the introduction section.

6. Typos:
  - probe the model’s answer A2 after inquiring (A2, R1). 
  - type "Country". has.
  - P-value p 0.001

7. The figures and tables should be not be in png or jpg format.
paper_summary: paper_summary: In this paper, the authors propose a controllable knowledge conflict construction framework using parametric knowledge graphs. Specifically, they demonstrate two different graph structures: multi-hop and multi-dependent, which enable them to control the position of conflicts. Furthermore, they implement four different types of distractors, providing in-depth analysis from multiple perspectives. According to the findings, current LLMs tend to trust external knowledge sources but still get distracted by unrelated information.
strength_weakness: strength_weakness: 1. This paper is a little difficult to follow because of the writing style. For instance, in the last paragraph of the introduction, numerous words are densely packed without any clear indication of important insights, which disrupts the reading experience. 
2. Regarding "parametric knowledge," the authors utilize in-context demonstration to elicit it. However, could this introduce demonstration bias, potentially distorting the reality of parametric memory? How did the authors ensure that the elicited parametric knowledge remains unbiased even with demonstrations?
3. Regarding the judgment of models' answer consistency, what is the accuracy of this process? Can this accuracy guarantee the quality of the judgments? The authors should provide clarification on this aspect in the paper.
4. To ensure that the insights in this paper can be generalized to other widely used LLMs, it's essential to consider including other LLM families, such as Llama and Gemini. While the extensive experiments in this paper may not allow reporting all results, it's acceptable to focus on key insights. However, including at least some results from other LLM families may be necessary.

(I will reconsider the score if the authors can address the points mentioned above to my satisfaction.)
questions: questions: 1. How do the authors perceive the phenomenon of "The length of context influencing consistency"? Are there any underlying reasons behind the superficial "length"?

Comments: title: Reply to the Authors
comment: Thank you for the detailed response. I have no further questions. I will keep the score of 6.
I thank the authors for their response. I have no further questions, and will keep my score due to the reasons before.
Thank you for your reply, I have no further questions. I think this paper is interesting and has a clear contribution. I will keep my scores.
Sincerely thank you for the detailed response. As I mentioned in weakness 4, it's encouraged to include more results on different LLMs. However, considering the extensive experiments in the current version, which cover two LLMs from different perspectives, the current version is acceptable. I have no further questions and will maintain my score.