Title: Concept Attractors in LLMs and their Applications
Venue: NeurIPS 2025
Accepted: no
Keywords: Large Language Models

Dynamic Systems

Attractors

Guardrails

Transpiler

Steering

Hallucinations

Synthetic Data
Abstract: Large language models (LLMs) often map semantically related prompts to similar internal representations at specific layers, even when their surface forms differ widely. We show that this behavior can be generalized and explained through Iterated Function Systems (IFS), where layers act as contractive mappings toward concept-specific Attractors. We leverage this insight and develop simple, training-free methods that operate directly on these attractors to solve a wide range of practical tasks, including **language translation**, **hallucination reduction**, **guardrailing**, and **synthetic data generation**. Despite their simplicity, these attractor-based interventions match or exceed specialized baselines, offering an efficient alternative to heavy fine-tuning, generalizable in scenarios where baselines underperform.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper discovers that prompts associated with the same ``concept’’ tend to converge toward an attractor in the representation space of intermediate layers, even when those prompts exhibit significant semantic differences. Moreover, different concepts are shown to converge to attractors located in different layers. This layer-specific convergence enables manipulation tasks such as unlearning and translation by directly operating on attractors—without modifying the model parameters.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:  \n1. The identification of attractors represents a significant advancement over previous work, offering more thorough observations supported by extensive examples.  \n2. The study demonstrates that a variety of downstream tasks benefit from the attractors.  \n3. The interpretation based on an Iterated Function System (IFS) is both plausible and interpretable.\n\nWeaknesses:  \n1. The paper should explicitly describe the procedure used to identify attractors. While it can be inferred that attractors are computed by averaging vectors corresponding to prompts within the same concept class, the implementation details remain vague. Many experimental settings also require further clarification. For example, in the section ``A Fractal-like Structure in the Attractors’’, some experimental setups are adopted from prior work, but the primary experimental parameters (e.g., task scope) should be explicitly stated. Without such details, readers unfamiliar with the prior literature may struggle to understand why the subtraction operation yields fractal-like structure such as -1 and -2 instead of other values.  \n\n2. The paper does not provide compelling evidence that two attractors cannot converge to similar or nearby space within the same layer. This uncertainty raises concerns about potentially modifying one attractor while retaining other concepts—particularly problematic in tasks like unlearning, where retaining out-of-scope data is crucial.  \n\n3. Although the paper proposes replacing outputs to achieve unlearning, this method may still be vulnerable to information leakage under white-box attacks.\n\nQUESTIONS:\n1. In Section 4, how does the proposed method of introducing attractors differ from the representation injection mechanism described in [1]?  \n\n2. In Figure 5, why do Layers 20 and 22 perform worse than Layers 16 and 18? Can the IFS account for this non-monotonic convergence across iterations? \n \n3. Why is the average of vectors assumed to represent the attractor? Could this introduce bias related to prompt distribution? Additionally, does the concept of a single average vector contradict the concepts of multiple attractors?\n\n[1]. Knowledge Neurons in Pretrained Transformers, ACL, 2022\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nA novel perspective on conceptualizing Transformer-based language models is proposed. Authors start with the empirical observation that lexically different yet semantically similar inputs to LMs lead to (geometrically) close intermediate representations. A parallel between attractors in iterated function systems and in-LM concept attractors is drawn. Experiments are performed on the ability to control the model at inference time by editing such concept attractors.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n1. *Interesting idea*: The possibility of approximating an LM using iterated functions is intriguing. It can be used as an alternate way of interpreting these models. In general, I am inclined to functional decomposition of a model, instead of structural.\n\n2. *Causal importance* I appreciate that the authors went on to actually showcase the causal viability of these structures that they identify. Many interpretability works stop at identifying the structure; however, without actually showing the effects of editing such structures, the scope becomes limited.\n\n\n## Weakness:\n\nThe primary weakness of this paper is in its positioning of contribution. Is it about a novel perspective on interpreting LMs? Then, one would expect a more in-depth positioning of this perspective against existing techniques/conceptualization: where do they fail, what do they miss, how does the proposed approach capture it? Additionally, the paper should discuss different existing works looking at Transformers from a dynamical system perspective [1, 2, 3]. \n\nInstead, if this paper is a new method of steering (like all the applications experimented with), then it needs to discuss the limitations of the SoTA (e.g., [4] and [5] for example), experiment with existing such methods and show that the attractor-based steering is better.\n\nQUESTIONS:\n- I think the paper can be improved by deciding on what exact type of contribution the authors want to make. Please see the weakness section for detail.\n\n- Please help me out in understanding the significance of section 2.1. You have formalized some parts, but never used them in the methods. Are you approximating \\phi_{eff} anywhere? \n\n- Masked language models like BERT (and their specialized variants like SentenceBERT) typically map semantically very similar but lexically very different inputs to similar dense representations. Can you extend your framework to such models?\n\n- What about failure points (i.e., semantically very similar yet lexically different inputs mapped to different attractors due to some adversarial element)?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
The paper identifies that semantically similar prompts, regardless of their surface text differences, converge to similar internal representations within specific layers of Large Language Models (LLMs). It proposes that this convergence behavior can be understood through the lens of Iterated Function Systems (IFS). Specifically, it views successive layers of the LLM as applying contractive mappings that iteratively drive representations towards stable, concept-specific points called Attractors. Leveraging this IFS/Attractor theory, the authors develop simple, training-free techniques for various NLP tasks by manipulating these Attractors.

STRENGTHS AND WEAKNESSES:
Strengths：
1. This paper explores an interesting topic.
2. The paper is well-written.
3. Experiments around the application of concept attractors are comprehensive.

Weaknesses：
1. The definition of concept is unclear. What exactly is a concept? When discussing things at a macro level, we might consider C++ and Python as belonging to the same concept—both being programming languages. However, when delving into more specific details, C++ and Python could represent different concepts. 
2. Beyond applications, further discussion centers on how concept attractors are formed. How do training procedures and model architectures influence their formation? Would supervised fine-tuning (SFT) alter these attractors?

QUESTIONS:
See Weakness

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03

comments: SUMMARY:
The paper takes inspiration from the notion of attractors in IFS and uses the idea in the concept space or embedding space. A variety of anecdotal evidence on induced tokens which are semantically relevant, layer sensitivity of attractors, multiple attractors for the same concept, and training-free hallucination detection, sanitizing toxic text, transpiling programming languages, improving vision-language models, and high-quality synthetic data generation. The IFS itself is learned to minimize the difference between the IFS output when started on the lowest layer and the highest layer output.

STRENGTHS AND WEAKNESSES:
S1. The idea of concept attractors is intriguing and is capable of bringing new insights to the functioning of LLMs. The paper gets credit for connecting the two ideas.

S2. The approach is demonstrated on a wide range of tasks. 

W1. The biggest weakness is that there are no appendixes in downloaded version. Details of attractor framework, which is central to the paper, as well as the details on experiments are missing. My scores are mainly due to lack of information.

W2. The main paper is not self-contained as the main points are explained in the appendixes.

W3. The idea needs access to internals of LLM. This is a minor issue as the benefits outweigh. This is also called out in the paper.

QUESTIONS:
1.	The paper mentions fractal like nature of attractors. Have you measured fractal dimensions of these sets?

2.	Provide details of the training procedure outlined in lines 103-133.

3.	How are attractors shown in Figure 4 identified and labeled?

4.	How does “teleportation” of attractors mentioned in Sections 4.1 and 4.3 performed?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 4
DATE: 2025-07-05