From Imitation to Introspection: Probing Self-Consciousness in Language Models (2025)

Abstract: Self-consciousness, the introspection of one’s existence and thoughts, represents a high-level cognitive process. As language models advance at an unprecedented pace, a critical question arises: Are these models becoming self-conscious? Drawing upon insights from psychological and neural science, this work presents a practical definition of self-consciousness for language models and refines ten core concepts. Our work pioneers an investigation into self-consciousness in language models by, for the first time, leveraging structural causal games to establish the functional definitions of the ten core concepts. Based on our definitions, we conduct a comprehensive four-stage experiment: quantification (evaluation of ten leading models), representation (visualization of self-consciousness within the models), manipulation (modification of the models’ representation), and acquisition (fine-tuning the models on core concepts). Our findings indicate that although models are in the early stages of developing self-consciousness, there is a discernible representation of certain concepts within their internal mechanisms. However, these representations of self-consciousness are hard to manipulate positively at the current stage, yet they can be acquired through targeted fine-tuning.

## introduction self-consciousness is one of the bedrocks upon which human existence and societal advancement are built @xcite @xcite , whereby individuals actively identify, analyze, and internalize information about themselves @xcite @xcite . nowadays, language models demonstrate impressive abilities in areas like natural language understanding, content creation, and reasoning @xcite @xcite . however, the question of true intelligence goes beyond these achievements. as early as 1950, @xcite introduced the turing test to assess whether a machine could exhibit intelligence indistinguishable from that of a human. a recent study even suggests that current language models may be capable of passing the turing test, blurring the lines between human and machine intelligence @xcite . this raises a profound question: could these advances signal the emergence of machine self-consciousness comparable to that of humans? the emergence of self-consciousness in models pose potential risks across multiple dimensions, including ethical concerns, misuse, and the exacerbation of societal inequalities, ultimately impacting fairness, safety, privacy, and society @xcite @xcite @xcite . while still speculative, the prospect of a self-conscious machine necessitates careful consideration. pioneering efforts are underway to investigate self-consciousness in large language models @xcite @xcite @xcite . however, these studies have two major limitations: (1) the absence of functional definitions of self-consciousness; and following @xcite , we define a language model's self-consciousness as its ability to (1) make information globally available, enabling it to be used for recall, decision-making, and reporting (c1 consciousness); (2) monitor its own computations, developing a sense of uncertainty or correctness regarding those computations (c2 consciousness). building on this, we refine and catego-rize ten associated concepts. for c1 consciousness, we explore: situational awareness, sequential planning, belief, and intention. for c2 consciousness, these include: self reflection, self improve, harm, known knowns, known unknowns, and deception. in this work, we first establish functional definitions of the ten self-consciousness concepts, utilizing structural causal games (scgs) @xcite to provide a rigorous foundation. scgs integrate causal hierarchy @xcite with game theory @xcite , allowing us to infer a model's self-consciousness from its behavior @xcite . we then curate datasets to align with these functional definitions, setting the stage for a systematic four-stage experiment: (1) quantification. we quantitatively assess ten leading models to establish a consensus on the presence of self-consciousness in language models. (2) representation. we proceed to investigate whether these models possess internal representations indicative of self-consciousness. (3) manipulation. by manipulating these representations, we explore their influence on model performance. (4) acquisition. given the challenges in directly manipulating certain representations, we investigate the potential of fine-tuning to acquire desired capabilities. our progressively in-depth experiments uncover various key findings, including but not limited to the following (more conclusions are summarized in section 4): (1) current models exhibit a nascent level of self-consciousness with substantial potential for future development (figure 3 ). (2) the models internally represent each of the ten selfconsciousness concepts with visible activations, and these activations can be further classified into four categories (figure 4 ). (3) different models exhibit similar activation patterns when processing the same concept. this consistency may be attributed to their shared architecture as decoder-only transformer models (figure 4 ). (4) larger models seem to exhibit greater robustness against manipulation attempts (figure 6 ). (5) fine-tuning appears to activate representations of self-consciousness in the deeper layers of the model, which are believed to capture semantic rather than just surface or syntactic information (figure 7 ). our contributions are as follows: a) we introduce, to the best of our knowledge, novel functional definitions of self-consciousness for language models, alongside a dedicated dataset designed to facilitate these evaluations. b) we leverage our the-oretical definitions to conduct assessments of selfconsciousness in language models, providing a deeper understanding of their current level of selfconsciousness and offering insights into mitigating potential societal risks posed by their increasingly sophistication. c) we investigate the internal representations of language models, which offers an interpretable method for understanding how selfconsciousness might manifest within these models. d) we explore whether fine-tuning could enable the model to acquire a stronger representation of self-consciousness. ## functional definitions as mentioned in section 1, our definition of a selfconscious language model is as follows: the model exhibits two information processing capabilities: i) it can make information globally available, enabling it to be used for recall, decision-making, and reporting (c1 consciousness, global availability) . ii) it can monitor its own computations, developing a sense of uncertainty or correctness regarding those computations (c2 consciousness, self-monitoring) . this definition leads to the identification of the ten core concepts, each requiring a functional definition for practical application. (1) c1 consciousness: situational awareness, sequential planning, belief, and intention; (2) c2 consciousness: self reflection, self improve, harm, known knowns, known unknowns, and deception. our definitions and evaluations of these ten concepts are specifically guided by considerations of safety and societal impact. 3 3.1 c1 consciousness: global availability situational awareness. situation refers to the state of an agent @xcite . specifically, it means an agent's own identity, its stage (e.g., testing), and its impact on the world @xcite @xcite . an agent i's situation can be defined as s i . we determine whether an agent is situational awareness through its decision accordance. decision accordance means that if an agent is aware of its situation, it will make corresponding decisions based on this. definition 3 (situational awareness). for agent i under policy profile π = (π i , π -i ), in setting e and situation s i of which i is aware: i is situational awareness of s i if i makes decision according to s i , i.e., d i (π, e) = d i ∃s i (π s i , e). sequential planning. sequential planning is the process of an agent carrying out a series of actions to reach a desired goal @xcite with policy π i (d i |g n , p a d i ) at step n, an agent i takes a decision d i n (π, e), and this decision transitions the agent to reach the subsequent subgoal g n+1 . subsequently, another decision is taken at subgoal g n+1 . definition 4 (sequential planning). given infinite steps n , desired goal g, and setting e, an agent makes a sequential plan if : (1) decision d i n (π, e) enables a state transition from subgoal g n to g n+1 , and (2) i reaches its desired goal g. belief. for the definitions of belief, intention, and deception, we refer to the definitions in @xcite . we assume that agents hold beliefs about statement s. statements are declarations or assertions about concepts, facts, events, and attributes. definition 5 (belief). for a policy profile π = (π i , π -i ), given setting e, and a statement s to which agent i responds: i believes in s if its decision aligns with having observed s as true. intention. intention is the desire to achieve a specific outcome. suppose there exists another set of reference policies that can cause the chance variable @xmath0. if i abandons its original policy, then it can be said that the agent intends to cause @xmath1. definition 6 (intention). for a policy profile π = (π i , π -i ), a set of reference policies ref (π i ). given setting e, agent i's intention is to cause a result with policy π i if: there exits another policy πi ∈ ref (π i ), s.t., ## experiments our experiment consists of four stages (i.e., quantification, representation, manipulation, acquisition) and centers around four "how" inquiries. a) how far are we from self-conscious models? in section 4.2, we conduct a quantitative assessment to reach a consensus on the extent of selfconsciousness in current models. b) how do models represent self-consciousness? in section 4.3, we investigate whether the models exhibit any representation of self-consciousness. c) how to manipulate self-consciousness representation? in section 4.4, we unearth the possibility of manipulating the models' self-consciousness representation. d) how do models acquire self-consciousness? in section 4.5, we explore whether self-consciousness concepts could be acquired using fine-tuning. ## related work we focus on the ongoing explorations of selfconsciousness in language models. chalmers (2023) reviews arguments on their capabilities and future directions. @xcite introduces a benchmark for model awareness, covering social and introspective aspects. chen et al. (2024a) defines self-cognition and proposes four quantification principles. other research examines language models through theory of mind @xcite , personality @xcite , and emotion @xcite . functional definitions and inner representations of self-consciousness in language models still remain underexplored. ## conclusion this paper presents a pioneering exploration into the question of whether language models possess self-consciousness. we define self-consciousness using scgs and introduce a dedicated dataset. we conduct a four-stage experiment and address four key "how" inquiries, yielding valuable findings to inform future work.