Title: Plasticity as the Mirror of Empowerment
Venue: NeurIPS 2025
Accepted: yes
Keywords: Plasticity

Empowerment

Agents

Reinforcement Learning

Continual Reinforcement Learning

Information Theory
Abstract: Agents are minimally entities that are influenced by their past observations and act to influence future observations. This latter capacity is captured by empowerment, which has served as a vital framing concept across artificial intelligence and cognitive science. This former capacity, however, is equally foundational: In what ways, and to what extent, can an agent be influenced by what it observes? In this paper, we ground this concept in a universal agent-centric measure that we refer to as plasticity, and reveal a fundamental connection to empowerment. Following a set of desiderata on a suitable definition, we define plasticity using a new information-theoretic quantity we call the generalized directed information. We show that this new quantity strictly generalizes the directed information introduced by Massey (1990) while preserving all of its desirable properties. Under this definition, we find that plasticity is well thought of as the mirror of empowerment: The two concepts are defined using the same measure, with only the direction of influence reversed. Our main result establishes a tension between the plasticity and empowerment of an agent, suggesting that agent design needs to be mindful of both characteristics. We explore the implications of these findings, and suggest that plasticity, empowerment, and their relationship are essential to understanding agency.


Peer reviews:
comments: SUMMARY:
This paper notes that in the abstract, and an agent and an environment are symmetric communication partners; we just call the agent's messages "actions", and the environment's messages "observations". Following, and slightly extending, the literature on "empowerment", the paper observes that the empowerment of the environment is an interesting quantity that corresponds to an agent's "plasticity"--the extent to which the environment can guide its behavior. Ultimately, the paper proves an upper bound on the sum of an agent's plasticity and its empowerment, indicating that the two (desirable) quantities are in tension with each other.

STRENGTHS AND WEAKNESSES:
I'll remember this paper, and what I've learned from it. The paper was clear and easy to read, and it is original. It introduces a new conceptual tool, and I think it may be significant.

A lot of the paper consists of developing notation and definitions--I suppose that's necessary for this kind of work. I somehow feel like I didn't gain that much more from reading the paper than I did from reading the abstract, with one important exception, which is Theorem 4.8. So I would have liked to see a proof sketch, and maybe a full proof in the body of the paper.

QUESTIONS:
Could you please explain to me why Theorem 4.8 holds?

LIMITATIONS:
It's hard for me to identify any limitations with this work.
TITLE: Official Review
CLARITY: 4
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-06-30

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors introduce a general definition of \"plasticity\" for agents. To do so, they create a generalized version of directed information that does not require comparing sequences of equal length from the same time step. They show that agent plasticity is the \"mirror\" of environment empowerment (and vice versa). Moreover, they show that plasticity and empowerment are in tension: it is not possible to simultaneously maximize both the agent's plasticity and empowerment. This raises questions into what the optimal trade-off is between the two. At a minimum, the authors posit that an agent should have non-zero plasticity and non-zero empowerment.\n\nSTRENGTHS AND WEAKNESSES:\nOverall, I thought this was an _excellent_ paper, and I strongly recommend its acceptance.\n- It proposes a conceptually crisp and general notion of \"plasticity\" that relies on only minimal assumptions on the interaction between agent and environment.\n- The definition fits well into existing literature on empowerment as they show that plasticity can be seen as the mirror of empowerment, which is conceptually very appealing\n- They provide a generalization of directed information that is needed to study either plasticity or empowerment over time at different intervals\n- Their main theoretical results around (a) that agent plasticity = environment empowerment (and vice versa) and (b) the tension between agent plasticity + agent empowerment raise many interesting questions regarding the design of agents. How should plasticity and empowerment be balanced?\n- I think this framework will have wide appeal and is broadly relevant for many areas ranging from RL theory to modern LLM agents to recommender systems\n- The paper is very well-written and clear\n\nQUESTIONS:\nI'm curious, what do the authors think is the most important empirical work to do in this space to build upon their proposed conceptualization of plasticity and the plasticity-empowerment trade-off?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a new agent-centric measure (which they call as plasticity) and it quantifies how much an agent is influenced by its observations, complementing the concept of empowerment, which measures the agent’s influence on future observations. Plasticity is formalized using generalized directed information... it is defined by extending Massey’s directed information framework. The authors show two key results: \n(a) Plasticity and empowerment are mirror quantities between the agent and environment, and \n(b) There exists a trade-off.. both cannot be maximized simultaneously. This reveals a fundamental tension between being shaped by the world and shaping it. \nThe work offers a fresh theoretical perspective for understanding and designing intelligent agents.\n\nSTRENGTHS AND WEAKNESSES:\nBelow are my  comments and concerns on this paper:\n\n(a) This paper is entirely theoretical... it does not perform any empirical evaluation to demonstrate the practical implications of their proposed GDI framework. Any reason for not doing this? Is there any lack of data or other issues/\n\n(b) The use of advanced information-theoretic constructs like GDI may hinder accessibility and adoption among practitioners. Are these constructs lead to any engineering constraints? \n\n(c) The agent-environment interaction model is highly abstract and it may not capture the dynamic and uncertain nature of real-world environments where assumptions like full observability or stationarity may not hold. Any comments on this?\n\n(d) While the empowerment–plasticity tradeoff is theoretically intriguing, the paper lacks actionable insights on how to balance or optimize these properties when designing real agents. Do you have any quick suggestions to address this?\n\nQUESTIONS:\nPlease answer below questions:\n\n(a) This paper is entirely theoretical... it does not perform any empirical evaluation to demonstrate the practical implications of their proposed GDI framework. Any reason for not doing this? Is there any lack of data or other issues/\n\n(b) The use of advanced information-theoretic constructs like GDI may hinder accessibility and adoption among practitioners. Are these constructs lead to any engineering constraints? \n\n(c) The agent-environment interaction model is highly abstract and it may not capture the dynamic and uncertain nature of real-world environments where assumptions like full observability or stationarity may not hold. Any comments on this?\n\n(d) While the empowerment–plasticity tradeoff is theoretically intriguing, the paper lacks actionable insights on how to balance or optimize these properties when designing real agents. Do you have any quick suggestions to address this?\n\nLIMITATIONS:\n(a) This paper is entirely theoretical... it does not perform any empirical evaluation to demonstrate the practical implications of their proposed GDI framework. Any reason for not doing this? Is there any lack of data or other issues/\n\n(b) The use of advanced information-theoretic constructs like GDI may hinder accessibility and adoption among practitioners. Are these constructs lead to any engineering constraints? \n\n(c) The agent-environment interaction model is highly abstract and it may not capture the dynamic and uncertain nature of real-world environments where assumptions like full observability or stationarity may not hold. Any comments on this?\n\n(d) While the empowerment–plasticity tradeoff is theoretically intriguing, the paper lacks actionable insights on how to balance or optimize these properties when designing real agents. Do you have any quick suggestions to address this?", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper develops an information-theoretic mathematical definition for plasticity. Towards this, one key contribution is the formulation of Generalized Directed Information (GDI), which generalizes the notion of directed information between sequence of random variables $I(X_{1:n} → Y_{1:n})$ to handle arbitrary sub-sequences $I(X_{a:b} → Y_{c:d})$\n\nGiven this formalism, the paper defines plasticity of an agent w.r.t. an environment as the (generalized) directed information from its observations to actions. Similarly, a definition of the empowerment is formulated as the (generalized) directed information from its actions to observations, and the paper shows that this definition subsumes previous ones that only considered full sequences.\n\nThe main results then show that: a) there is a symmetry in the definition of plasticity and empowerment i.e. plasticity = the environment’s empowerment w.r.t. the agent, and b) the sum of an agent’s plasticity and empowerment is bounded i.e. given a specific environment, there is a tradeoff between how plastic an agent can be vs how empowered it is.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- The formulation of GDI is an interesting contribution. It helps generalize the notion of directed information to consider arbitrary subsets and could be applicable beyond the context of plasticity/empowerment. Similarly, the generalized definition of empowerment is also a useful extension of previous ones as it now lets us consider the empowerment in terms of influence of a specific subsequence actions on a subsequence of future observations.\n- The main results which highlight that it isn’t possibly to simultaneously optimize empowerment and plasticity is a thought-provoking observation (although some concerns on this below)\n- The paper is generally well-written with appropriate levels of background and detail in the main text and appendix. Given the topic and the contribution, this was likely not an easy thing to accomplish!\n\nWeaknesses:\n\n- A primary concern about this work is that it seems to be redefining an existing and widely used term “plasticity” to be rather different from the (intuitive but not mathematically precise) typical definition of “an agent’s capacity to remain adaptive”. In this work, the term “plasticity” is instead better described as capturing “the influence of an environment on the agent” and it’s not clear why we should be using the same word ‘plasticity’ for this measure. As an illustration, in the mouse example in the appendix, the fully stochastic room certainly reduces the empowerment of the agent but it’s not clear there is any useful information to learn for helping future adaptation in the stochastic one. Concretely, let’s imagine a neural network trying to learn to predict the outcome of the agent’s actions. One environment is deterministic, allowing the network to model laws of physics and the other environment is random, and makes the neural network learn to predict a uniform distribution over states. As the typical definition of ‘plasticity’ is how adaptive this network can be to future changes, it’s not clear why the network being learned in the fully stochastic setting will be better suited to adapt to new distributions than the one in the deterministic setting. So although this stochastic environment does maximize ‘plasticity’ as defined in this paper, it does not intuitively do so for the ‘classical’ notion of plasticity.\n- While the bound in Egn 19 is interesting, I’m not sure if it’s a very profound result. For example, if there is no overlap in a:b and c:d, one of the terms is trivially zero (as directed influence of future on past will be zero). On the other hand, if the two subsequences overlap fully (e.g. a=b=1, c=d=n), it is essentially the result in Eqn 4. It would have been a much more surprising result had the bound in Eqn 19 been in the direction a:b → c:d for both terms i.e. for non-overlapping intervals, one term were not trivially zero. In the current form, an agent **can** be simultaneously empowered and plastic in terms of its actions affecting future observations, and observations affecting future actions. Specifically, considering non-overlapping intervals a:b (’current’) and c:d (’future’), there seems to be no tradeoff between the current actions’ influence on future observations and current observations’ influence on future actions.\n\nQUESTIONS:\n- Some more discussion on the relation of the current definition to the typical semantic of 'neural plasticty' would help.\n\n- I'd also appreciate a discussion/rebuttal of the weakness 2 i.e. it seems that an agent **can** be simultaneously empowered and plastic where its current observations can influence future actions while its current actions can also influence future observations, and there seems to be no tradeoff between these.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}