PISA: A Pragmatic Psych-Inspired Unified Memory System for Enhanced AI Agency

## Abstract

Abstract Memory systems are fundamental to AI agents, yet existing work often lacks adaptability to diverse tasks and overlooks the constructive and task-oriented role of AI agent memory.
Drawing from Piaget’s theory of cognitive development, we propose PISA, a pragmatic, psych-inspired unified memory system that addresses these limitations by treating memory as a constructive and adaptive process. To enable continuous learning and adaptability, PISA introduces a trimodal adaptation mechanism ( i.e. , schema updation, schema evolution, and schema creation) that preserves coherent organization while supporting flexible memory updates. Building on these schema-grounded structures, we further design a hybrid memory access architecture that seamlessly integrates symbolic reasoning with neural retrieval, significantly improving retrieval accuracy and efficiency.
Our empirical evaluation, conducted on the existing LOCOMO benchmark and our newly proposed AggQA benchmark for data analysis tasks, confirms that PISA sets a new state-of-the-art by significantly enhancing adaptability and long-term knowledge retention.

## 1 Introduction

> “Knowledge is not a copy of reality. To know an object is to act upon it and to transform it.” — Jean Piaget

In recent years, Large Language Model (LLM) based AI agents have been proposed to autonomously accomplish specific objectives or solve particular types of tasks (Xi et al., 2025; Liang et al., 2022; Hu et al., 2024; Zhang et al., 2025; Nathani et al., 2025; Wang et al., 2025; Fish et al., 2025). Their core characteristic lies in task-oriented behavior, which involves perceiving the environment, executing reasoning and planning, and implementing a series of coherent actions to achieve task objectives. During task execution, the rational utilization of historical information has been proven to be a crucial pathway for enhancing efficiency and effectiveness. However, indiscriminately inputting all historical information into the AI agent at each decision point introduces two implicit challenges: (1) excessive historical information both increases reasoning costs and risks exceeding the model’s context window (Keles et al., 2023); (2) the substantial amount of irrelevant information contained therein may interfere with the model’s judgment and decision-making (Wang & Sun, 2025).

Figure: (a)
Refer to caption: x1.png

Agent memory (Packer et al., 2024; Zhong et al., 2024; Wang et al., 2024; Xu et al., 2025; Kang et al., 2025; Chhikara et al., 2025; Rasmussen et al., 2025) has been proposed as a set of practical approaches by constructing retrieval knowledge bases through persistent storage and structured organization of historical experiences, thereby improving future decision quality and efficiency.
Nevertheless, existing work predominantly focuses on engineering optimizations, e.g., memory persistence and retrieval efficiency, neglecting the active constructive nature and the task-oriented characteristic of AI agents.
Meanwhile, current agent memory benchmarks (Maharana et al., 2024a; Wu et al., 2025; Hu et al., 2025) are limited to effectiveness evaluation, with most focusing on contextualized dialogue for fact retrieval, lacking task-specific assessments (Rasmussen et al., 2025).

Consequently, current memory systems exhibit poor performance on heterogeneous tasks, as demonstrated in our benchmarking results in Table [2](https://arxiv.org/html/2510.15966v1#S4.T2).
Inspired by Piaget’s cognitive theory (Piaget et al., 1952) that knowledge is actively constructed through interaction with the environment rather than being passively absorbed, we strive to introduce a novel memory system that functions as an active and constructive process, enabling dynamic adaptation to task demands.

Building on this perspective, we propose a pragmatic Psych-Inspired unified memory System for enhanced AI Agency, named PISA, an actively constructive yet adaptive agent memory system that integrates differentiated architectures with flexible update strategies.
PISA first constructs task-oriented memory structures by analyzing the objectives of the AI agent as illustrated in Figure [1](https://arxiv.org/html/2510.15966v1#S1.F1), thereby addressing the limitations of conventional structured memory that lacks adaptability to heterogeneous tasks.
Informed by Piaget’s schema theory(^1^11Details are described in Appendix [B.1](https://arxiv.org/html/2510.15966v1#A2.SS1)), i.e., cognitive development progresses through two complementary processes: assimilation, in which new experiences are integrated into existing schemas, and accommodation, in which schemas are modified or newly created to capture novel or conflicting information, we further introduce memory adaptation strategies for more flexibility.
More specifically, we propose dynamic evolutionary mechanisms to model structural knowledge paradigms and semantic relationships.
As shown in Figure [1](https://arxiv.org/html/2510.15966v1#S1.F1), our adaptive workflow module dynamically manages incoming experiences by assimilating them into existing schemas, accommodating schema evolution, or creating new schemas. This tri-modal approach ensures coherent knowledge organization, continuous learning, and adaptable memory updates. Building on this, we design a hybrid memory access architecture integrating symbolic reasoning with neural retrieval. Through extensive experiments on various models and benchmarks, our approach demonstrates significant improvements, consistently outperforming all baseline models across all evaluation metrics.

Figure: Figure 2: Overall architecture of PISA. The system is built upon a uniquely designed Schema Engine and consists of three core modules: (1) the Initialization Module, which leverages agent goals and prior knowledge to initialize memory structures; (2) the Adaptation Module, which dynamically processes new experiences through schema assimilation, evolution, and creation, guided by conflict analysis; and (3) the Retrieval Module, which interprets queries and retrieves relevant memories using task-aware retrieval strategies and external tool calls. Together, these components enable a flexible, extensible, and task-oriented memory system for LLM-based AI agents.
Refer to caption: x3.png

Overall, the contributions of this paper include the following aspects:

- 1.
Unified Memory System Design: We propose a unified memory framework PISA based on Piaget’s cognitive theory, that constructs task-oriented memory structures and overcomes the adaptability limitations of general-purpose memory systems.
- 2.
Adaptive Memory Evolution Mechanism: We introduce a Piaget-inspired adaptive memory evolution mechanism, which enhances memory flexibility and enables continuous learning through a trimodal update process.
- 3.
Hybrid Memory Access Architecture: We design a hybrid access mechanism based on task-oriented memory structures that integrates symbolic reasoning with neural retrieval, significantly improving the efficiency and accuracy of memory retrieval.
- 4.
Real-world Scenario Evaluation Benchmark: We construct AggQA, a mini-benchmark for data analysis tasks spanning practical domains including Medical and Finance, to address the deficiencies in existing benchmarks regarding task diversity and practical relevance.

## 2 Related Works

### 2.1 Memory For LLM-based AI Agents

Extensive research has been devoted to memory systems for LLM-based AI agents. Du et al. (2025) proposes a unified framework that analyzes prior works along three complementary dimensions: Taxonomy (representation), Management (organization and updating), and Utilization (retrieval and application). We review related studies following this perspective.

For memory representation, existing works have explored multiple approaches to balance structure and flexibility. Hierarchical representations organize memory at different levels of abstraction, enabling selective access and forgetting (Zhong et al., 2024). Structured multi-attribute notes, incorporating contextual descriptions, keywords, and embeddings, construct semantic networks supporting interconnection across memory items (Xu et al., 2025). Hybrid representations that integrate natural language with graph-structured relations further provide interpretability and efficiency for dynamic storage (Chhikara et al., 2025).

In terms of memory management, researchers investigate diverse strategies to support memory evolution. Dynamic updating and forgetting mechanisms, inspired by the Ebbinghaus curve, maintain memory relevance and naturalness (Zhong et al., 2024). System-inspired architectures, such as segmented-paging designs migrated from operating systems, enable scalable organization in dialogue scenarios (Kang et al., 2025). Autonomous evolution strategies driven by LLMs generate new links and maintain balanced states without explicit rules, improving adaptability to long-term interactions (Xu et al., 2025). Moreover, consistency management approaches, including quadruple operations and conflict resolution, ensure coherent updates under high-concurrency requirements (Chhikara et al., 2025).

With respect to memory utilization, prior work has emphasized retrieval and application strategies that enhance effectiveness in reasoning and interaction. Hierarchical retrieval combined with heat-driven updating allows context-relevant access for personalized responses (Kang et al., 2025). Entity–relation dual-path retrieval methods jointly exploit symbolic and embedding-based representations to improve accuracy in complex queries (Chhikara et al., 2025). In addition, compression-based methods have been adopted to retain informativeness while reducing storage overhead, supporting scalability in extended interactions.

Despite these advances, most memory systems remain task-agnostic, overlooking the integration of agent goals as priors. This leads to two challenges: limited specialization for heterogeneous tasks, and significant computational and storage costs when striving for broad generality. These limitations point to the need for task-oriented memory systems that explicitly incorporate agent objectives into representation, management, and utilization, thereby capturing task-relevant information more efficiently and adaptively.

### 2.2 Retrieval Module for Memory System

The retrieval module is a crucial component of the memory system, and its implementation depends on the design of the memory system’s memory Taxonomy and Management dimensions.
Regarding the retrieval module, we have an insight: memory systems and retrieval modules are complementary, and the structural nature of memory systems is directly proportional to the flexibility of retrieval modules.
Existing research validates our perspective:
The Mem0 system(Chhikara et al., 2025) demonstrates the synergistic design of structured memory representation and multi-granularity retrieval mechanisms, empirically proving that the structural transparency of memory systems directly determines the flexibility of retrieval modules. Its graph-based Mem0g architecture, with its high structural nature, supports more complex multi-hop and temporal query capabilities, validating the proportional relationship between structured memory representation and flexible retrieval performance.
MemoryOS(Kang et al., 2025) constructs a highly structured memory system through its hierarchical storage architecture and heat-based dynamic update mechanisms. Its retrieval module relies on semantic segmentation and multi-level granularity strategies (such as segment-page two-level retrieval and personalized attribute integration) to achieve flexible and precise multi-dimensional memory access, confirming the complementary relationship between structured memory systems and retrieval flexibility.
AMem(Xu et al., 2025) is a self-organizing and self-evolving memory system. Its memory Taxonomy dimension adopts dynamically generated multi-attribute structured notes, while the Management dimension introduces autonomous evolution mechanisms. Its unstructured memory storage leads to retrieval limitations, resulting in its ability to retrieve only through returning top-k notes.
The reason for the complementary nature between memory systems and retrieval modules lies in the fact that the higher the degree of structuralization of the memory system, the more transparent its structure becomes, enabling the retrieval module to perform multi-granularity retrieval based on memory, thus achieving more flexible retrieval capabilities.
Therefore, the design of retrieval modules should strictly consider the structure of memory systems and utilize the structural information of memory systems for retrieval as much as possible. Meanwhile, we propose our viewpoint: the design of memory systems, in addition to starting from the memory system itself, considering memory system implementation from the perspective of retrieval module design is also an effective way to think about memory system implementation.
PISA’s design not only starts from the memory system perspective but also from the retrieval module perspective, integrating both to achieve a memory system capable of multi-granularity information acquisition.

### 2.3 LLM-based AI Agent Memory Benchmark

The evaluation of long-context capabilities in large language models (LLMs) has evolved from testing models’ internal memory to specifically assessing external memory systems. Early research primarily focused on testing the limits of models’ internal memory mechanisms. RULER (Hsieh et al., 2024) significantly extended the traditional “needle-in-a-haystack” test by constructing a synthetic benchmark with configurable length and complexity, introducing tasks such as multi-needle retrieval, multi-hop tracking, and aggregation. Although RULER provides a multi-faceted evaluation of models’ parametric memory, its synthetic data design fails to reflect memory requirements in real-world application scenarios.

LOCOMO (Maharana et al., 2024a) constructed long-term dialogues incorporating event graphs and multimodal elements through a human-AI collaborative process, evaluating models’ long-term memory capabilities in scenarios closer to real conversations. This work revealed the challenges LLMs face in understanding long-term temporal and causal dynamics, highlighting the need for a more comprehensive evaluation framework capable of handling long-term dependencies and complex causal relationships.

As research deepened, benchmarks specifically designed for external memory systems began to emerge. LongMemEval (Wu et al., 2025) assessed a memory assistant’s multiple core long-term memory abilities (e.g., extraction, reasoning, updating, and abstention) in multi-scenario ultra-long dialogues. MemoryAgentBench (Hu et al., 2025) clearly defined the four core capabilities of memory AI agents and, addressing the limitations of existing datasets (e.g., short context or one-time full-text input), innovatively proposed a unified evaluation framework with segmented input. However, as noted by research from the systems development frontline, the field still faces a challenge of benchmark scarcity: existing options often lack robustness and complexity, frequently limited to simple “needle-in-a-haystack” retrieval Rasmussen et al. (2025), and no benchmark can adequately evaluate a system’s ability to integrate dialogue history with structured business data for comprehensive processing Rasmussen et al. (2025).

In terms of task complexity, benchmarks like WikiTableQuestions (Kweon et al., 2023) and HybridQA (Chen et al., 2020) advanced the development of hybrid reasoning over structured data and text. Subsequent research (e.g., HeteQA (Yu et al., 2025)) further extended the complexity of cross-domain and multi-hop table QA. However, these works are generally built upon static, closed-domain data, with limited context length and static interaction patterns, primarily targeting single-query response scenarios, thus, it also fails to meet the requirements for continuous memory maintenance and complex numerical aggregation in dynamic dialogues.. Furthermore, existing literature shows insufficient discussion on scalability metrics crucial for production systems, such as cost and latency (Rasmussen et al., 2025).

In summary, the evaluation paradigm in this field has undergone a clear evolutionary path: from assessing models’ internal memory (RULER, LOCOMO), to focusing on the systematic evaluation of long-term memory (LongMemEval), and then to designing comprehensive benchmarks specifically for external memory AI agents (MemAgentBench). However, significant gaps remain in existing benchmarks: they either lack investigation into professional domain-specific discrete data and numerical aggregation capabilities, or struggle to evaluate memory mechanisms under long dialogues, and none systematically address the integrated processing of dialogue history and business table data. To this end, we propose AggQA, which discretely embeds tabular data within 30K-token-level professional dialogues, aiming to systematically evaluate the numerical perception, reconstruction, and aggregation capabilities for external memory systems, thereby filling the gaps in the current evaluation system.

## 3 PISA: A Pragmatic Psych-Inspired Unified Memory System for Enhanced AI Agency

Figure: Figure 3: Schema Engine of PISA. The memory system is organized hierarchically into Memory Pool, Buckets, and the Schema. Each schema encodes structured knowledge through Meta, Element, Element-Value, and a unique experience ID, enabling flexible, extensible, and task-oriented memory organization.
Refer to caption: x4.png

The overall architecture of PISA is illustrated in Figure [2](https://arxiv.org/html/2510.15966v1#S1.F2). The entire system is built upon a uniquely designed Schema Engine and comprises three modules: the Initialization Module, the Adaptation Module, and the Retrieval Module.
The Schema Engine serves as the core component of PISA, responsible for defining and managing the memory structure of the AI agent. The Initialization Module initializes PISA’s memory structure based on the AI agent’s goals and tasks. The Adaptation Module processes experiences, while the Retrieval Module retrieves relevant memories from PISA according to the AI agent’s goals and tasks.
In the following sections, we will provide a detailed introduction to the functionality and design of the Schema Engine and each module.

### 3.1 Schema Engine

This part details our proposed memory architecture, as illustrated in Figure [3](https://arxiv.org/html/2510.15966v1#S3.F3), which comprises a three-tiered hierarchy: the Memory Pool, Buckets, and the Schema. The Memory Pool acts as the comprehensive repository for all data, which is partitioned into thematic Buckets such as “Bucket - Agent Events” and “Bucket - User Events”. Each bucket’s Centric Info defines the core focus of the memories stored within and the principles for their structuring.
The foundational unit of storage is the Schema. Each Schema is composed of four main components. The Meta defines a high-level topic (e.g., “Drink”), under which a specific Element is identified (e.g., “Pure Milk”). The Element-Value component then captures nuanced descriptors in a flexible key-value format implemented via “JSONB”, such as “Attitude: like” or “Scenes: Winter morning”. Crucially, a unique Experience ID links this structured Schema entry back to its originating unstructured event in the Memory Pool, such as a specific “Chat History” log. This design yields a memory system that is logically flexible and extensible, while maintaining a queryable, structured representation in its physical implementation.

### 3.2 Initialization Module of PISA for AI Agent Goal

The Initialization Module leverages prior knowledge, particularly the agent’s goals and task specifications, to proactively construct the memory structure of PISA. Rather than passively storing information, this module embeds task priors into the organization of the memory pool, ensuring that memory is oriented toward the agent’s objectives from the outset. Concretely, given an agent goal, the module first analyzes its semantic requirements and determines the centric info of buckets (e.g., user traits, user events, relationships), which provide coarse-grained partitions for categorizing incoming experiences. Within each bucket, the module further specifies the schema-level element–value structures, defining which attributes should be actively monitored (e.g., time, location, emotional status), while leaving their values unpopulated until actual experiences are observed.

Through this hierarchical initialization process, PISA transforms memory into a task-sensitive scaffold that guides subsequent assimilation and retrieval, thereby reducing irrelevant information and enhancing the efficiency of downstream reasoning.

### 3.3 Adaptation Module

Building on Piaget’s schema theory and the implemented Schema Engine, the Adaptation Module employs a dynamic evolutionary paradigm to capture both structural knowledge patterns and semantic relationships. This module enables PISA to autonomously determine whether new experiences should be assimilated into existing schemas, accommodated through schema evolution, or used to create entirely new schemas. This tri-modal mechanism ensures continuous learning, maintains coherent organization, and addresses the flexibility challenges of memory updates. The detailed workflow is summarized in Algorithm [1](https://arxiv.org/html/2510.15966v1#alg1), with the following sections elaborating on each component.

##### Notation and Symbols.

We summarize the symbols used throughout this section:

| Symbol | Description |
| --- | --- |
| $b$ | Bucket index determined during preprocessing; $K_{b}$ is the canonical key set for bucket $b$. |
| $S(b)$ | Current set of schemas in bucket $b$; $\sigma\in S(b)$ denotes a schema with metadata $m(\sigma)$ and element set $U(\sigma)$. |
| $u$ | An element in $U(\sigma)$; $V(u)$ is the multiset of records attached to $u$. |
| $e$ | An incoming experience segment; $r(e)$ is the normalized record extracted from $e$ projected onto $K_{b}$; $m(e)$ is metadata extracted from $e$. |
| $s_{\text{meta}}(e,\sigma)$ | Schema-level similarity score in $[0,1]$; threshold $\theta_{\text{meta}}\in[0,1]$. |
| $\kappa(e,u)$ | Element-level compatibility score in $[0,1]$; threshold $\theta_{\text{elem}}\in[0,1]$. |
| $\mathrm{keys}(r)$ | Set of keys present in record $r$; $\mathrm{val}_{k}(r)$ is the value of $r$ at key $k$. |
| $\mathrm{age}(r)$ | Time elapsed since $r$ was created; $q_{\text{src}}(r)\in[0,1]$ source quality; $\#\mathrm{supports}(r)$ number of independent supporting references. |

Figure: Algorithm 1 Adaptation Processing

##### Update Operators.

SchemaUpdate incorporates a new record $r(e)$ into the matched element $u^{*}$:

$$ $V(u^{*})\leftarrow V(u^{*})\cup\{r(e)\},\quad\text{s.t. }K_{b}\subseteq\mathrm{keys}(r(e)).$ $$

Schema Evolution creates a new element $u^{\text{new}}$ with initial record:

$$ $U(\sigma^{*})\leftarrow U(\sigma^{*})\cup\{u^{\text{new}}\},\quad V(u^{\text{new}})\leftarrow\{r(e)\}.$ $$

Schema Creation instantiates a new schema $\sigma^{\text{new}}$ with meta $m(e)$:

$$ $S(b)\leftarrow S(b)\cup\{\sigma^{\text{new}}\},\quad U(\sigma^{\text{new}})\leftarrow\emptyset.$ $$

Figure: Figure 4: Retrieval Module of PISA. It supports diverse query handling through Regional Fact Retrieval, Multi-Fragment Reasoning, and Aggregation, leveraging an extendable toolkit library (e.g., RAG, SQL, and Caculator) to utilize both unstructured and structured memory.
Refer to caption: x5.png

### 3.4 Retrieval Module

The Retrieval Module is designed as a ReAct AI agent (Yao et al., 2023), built upon the Langgraph framework (LangChain Team, 2023). Its primary function is to interpret and analyze user queries and then retrieve pertinent information from the memory system. This process is intelligently guided by the inherent structure of the memories it accesses.

To adeptly handle a diverse range of user inquiries, the Retrieval Module is equipped with an extensible toolkit library, enabling it to select the most efficient tools based on the query’s nature. As illustrated in the Figure [4](https://arxiv.org/html/2510.15966v1#S3.F4), the module categorizes queries and deploys corresponding strategies:

- •
For Regional Fact Retrieval Queries, which are typically single-hop questions like “Which cities did you last suggest for my vacation?”, the AI agent employs a Retrieval-Augmented Generation (RAG) tool (Lewis et al., 2020). This allows for direct and rapid retrieval of specific memory fragments, such as past suggestions, from the memory pool.
- •
For Multi-Fragment Reasoning Queries that require synthesizing information from various sources, such as “Based on my coastal & mid-range preferences, which recommended city should I choose?”, the AI agent orchestrates a chain of tools. It might first use the RAG tool to pull up all related memory cards about travel recommendations and user preferences. Subsequently, it can leverage a SQL tool to perform structured queries across these memories to filter and reason, ultimately providing a tailored recommendation.
- •
For Aggregation Queries that necessitate calculation or data comparison, like “Did I drink more coffee this week than last week, and by how much?”, the module utilizes SQL to fetch relevant structured data (e.g., coffee consumption logs for “This Week” and “Last Week”). Following the data retrieval, it can invoke a calculator tool to perform the necessary aggregation, such as calculating the difference, to deliver a precise answer.

## 4 Experiments

### 4.1 Datasets

**Table 1: Performance on LOCOMO benchmark. The evaluation metric is LLM-as-a-Judge accuracy. Bold and underline respectively highlight the best and second-best scores among external-memory methods; FullContext is listed for reference only.**
| Method | Single-Hop | Multi-Hop | Temporal | Open-Domain | Average |
| --- | --- | --- | --- | --- | --- |
| Zep | 46.65 | 14.08 | 36.54 | 39.13 | 34.10 |
| MemoryOS | 43.06 | 19.01 | 16.67 | 47.83 | 38.40 |
| AMem | 56.94 | 28.87 | 71.79 | 50.00 | 51.90 |
| LangMem | 65.79 | 31.69 | 71.15 | 47.83 | 54.12 |
| Mem0 | 67.70 | 36.62 | 69.87 | 47.83 | 55.51 |
| PISA | 73.44 | 47.18 | 75.00 | 58.70 | 63.58 |
| FullContext | 78.71 | 49.30 | 87.18 | 58.70 | 68.47 |

To evaluate the performance of AI agent memory across different tasks, we primarily used two types of tasks to assess the memory system. The first is LOCOMO (Maharana et al., 2024b), which is used to evaluate long-term conversational memory.
The second is AggQA, which is used to evaluate memory for data analysis tasks.
LOCOMO contains 10 extended conversations. For a comprehensive assessment of long-term conversational memory, LOCOMO divides all questions into five distinct reasoning categories: (1) single-hop questions require answers based on a single session; (2) multi-hop questions require synthesizing information from multiple different sessions; (3) temporal reasoning questions can be answered through temporal reasoning and capturing time-related data cues within the conversation; (4) open-domain knowledge questions can be answered by integrating a speaker’s provided information with external knowledge such as commonsense or world facts; (5) Adversarial questions are designed to trick the AI agent into providing wrong answers, with the expectation that the AI agent will correctly identify them as unanswerable.(^3^33We excluded the adversarial category from our experiments.)
AggQA is designed by ourselves, divided into medical and finance domains, focusing on data analysis tasks in these two fields. Questions are categorized into three difficulty levels: easy, medium, and hard, with the medical domain containing 49 questions and the financial domain containing 85 questions. More details about AggQA are presented in Appendix [C](https://arxiv.org/html/2510.15966v1#A3).

### 4.2 Baselines

AMem
(Xu et al., 2025) proposes dynamic, self-evolving knowledge networks, built on atomic notes, autonomous linking, and continuous evolution for semantic understanding and efficient operation.

Mem0
(Chhikara et al., 2025) introduces a scalable memory architecture that dynamically manages salient information, ensuring consistency and improving reasoning efficiency.

LangMem
(AI, 2025) tackles cross-session learning, personalization, and consistency via memory APIs, in-conversation tools, and background knowledge integration, with flexible storage compatibility and native LangGraph support.

Zep
(Rasmussen et al., 2025) presents a temporal knowledge graph (Graphiti) synthesizing unstructured conversations and structured business data, achieving low-latency, scalable enterprise-grade memory.

MemoryOS
(Kang et al., 2025) designs a hierarchical memory system (Short/Mid/Long-Term) with a heat-based update mechanism and two-tier retrieval for efficient management and access.

Full-Context
directly providing the complete information as context to GPT-5 Mini (OpenAI, 2025) and instructing it to answer questions based on this context.

**Table 2: Performance on AggQA benchmark. The evaluation metric is LLM-as-a-Judge accuracy.**
| Method | Med. | Fin. |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Easy | Medium | Hard | Overall | Easy | Medium | Hard | Overall |  |
| MemoryOS | 18.18 | 12.00 | 0.00 | 10.20 | 17.39 | 14.89 | 7.14 | 14.29 |
| Zep | 9.09 | 40.00 | 0.00 | 22.45 | 8.70 | 10.42 | 7.14 | 9.41 |
| AMem | 27.27 | 28.00 | 0.00 | 20.41 | 39.13 | 25.00 | 42.86 | 30.59 |
| Mem0 | 27.27 | 40.00 | 15.38 | 30.61 | 39.13 | 16.67 | 21.43 | 23.53 |
| LangMem | 72.73 | 64.00 | 7.69 | 51.02 | 43.48 | 18.75 | 0.00 | 22.35 |
| PISA | 90.91 | 96.00 | 53.85 | 83.67 | 78.26 | 62.50 | 78.57 | 69.41 |
| FullContext | 45.46 | 68.00 | 23.08 | 51.02 | 95.65 | 62.50 | 57.14 | 70.59 |

### 4.3 Evaluation Metrics and Main Results

For the LOCOMO dataset and AggQA, we employ the LLM-as-a-Judge framework to evaluate the accuracy of responses. In the case of the LOCOMO dataset, we adopt a strict evaluation criterion: for answers involving multiple items (e.g., a query about all the books a character likes), any omission of valid results is scored as zero. Such a criterion places strong emphasis on the completeness of multi-item answers, thereby imposing higher demands on the memory system’s ability to synthesize and preserve comprehensive information.

Table [1](https://arxiv.org/html/2510.15966v1#S4.T1) shows the results of PISA compared with baselines on the LOCOMO dataset. Across all four evaluation modes (i.e., Single-Hop, Multi-Hop, Open-Domain, Temporal) our system achieves state-of-the-art performance, indicating its robustness and adaptability to diverse reasoning requirements.

On our AggQA benchmark illustrated in Table [2](https://arxiv.org/html/2510.15966v1#S4.T2), which emphasizes data analysis tasks, PISA achieves an even larger performance margin over baselines. By selectively highlighting task-oriented attributes and retaining critical analytical information, PISA delivers more efficient and accurate retrieval than generic memory systems. These findings underscore the effectiveness of treating memory as an active, task-oriented process rather than a passive repository.

### 4.4 Hyperparameter Analysis

Figure: (a) Performance vs. $\theta_{\text{meta}}$

We perform an hyperparameter analysis on conv-26 of the LOCOMO dataset to analyze the impact of the meta-cognition threshold, $\theta_{\text{meta}}$, on agent performance and internal behavior, as shown in Figure [5](https://arxiv.org/html/2510.15966v1#S4.F5). This threshold governs the agent’s decision to either assimilate new experiences into existing metas or create new ones.

**Table 3: Detailed ablation studies on the AggQA benchmark, showing both Accuracy (Acc.) and Evidence Coverage (Cover.). PISA^-I performs an ablation on the Initialization Module (on Med.), while PISA^-R performs an ablation on the Retrieval Module (on Fin.).**
| Method | Domain | Easy | Medium | Hard | Overall |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Acc. | Cover. | Acc. | Cover. | Acc. | Cover. | Acc. | Cover. |  |  |
| PISA^-I | Med. | 72.73 | 68.18 | 76.00 | 65.48 | 53.85 | 63.69 | 69.39 | 65.61 |
| PISA | 90.91 | 83.27 | 96.00 | 86.60 | 53.85 | 82.77 | 83.67 | 84.84 |  |
| PISA^-R | Fin. | 47.83 | 53.39 | 54.17 | 61.65 | 50.00 | 40.71 | 51.77 | 55.97 |
| PISA | 78.26 | 83.13 | 62.50 | 75.02 | 78.57 | 65.93 | 69.41 | 75.72 |  |

As depicted in Figure [5(a)](https://arxiv.org/html/2510.15966v1#S4.F5.sf1), the agent’s overall performance is robust for $\theta_{\text{meta}}$ values ranging from 0.30 to 0.95. The peak score is achieved at 0.70, which demonstrates strong and balanced performance on most evaluation modes. Figure [5(b)](https://arxiv.org/html/2510.15966v1#S4.F5.sf2) illustrates the trade-off in memory operations. A lower $\theta_{\text{meta}}$ (e.g., 0.30) encourages Assimilation, leading to frequent updates of existing memories. Conversely, a higher threshold (e.g., 0.95) markedly increases Creation operations, resulting in a larger and more fragmented memory space. The value of $\theta_{\text{meta}}=0.70$ strikes an effective balance, promoting a healthy mixture of Assimilation and Evolution without excessive Creation. Based on its high overall score and well-balanced internal mechanics, we set $\theta_{\text{meta}}=0.70$ as the default value for our main experiments.

### 4.5 Ablation Study

We performed two targeted ablation studies on AggQA to validate our core design choices, with results shown in Table [3](https://arxiv.org/html/2510.15966v1#S4.T3).

First, we tested the Initialization Module by removing its detailed, schema-driven setup on the Med dataset (PISA^-I). The resulting 14.28% drop in overall accuracy confirms that proactively building a task-oriented memory scaffold is critical for organizing information effectively and guiding downstream reasoning.

Second, we evaluated the Retrieval Module by ‘blinding’ it to the memory’s hierarchical structure on the Fin dataset (PISA^-R). This led to a 17.64% decrease in overall performance, with a severe drop on hard tasks (from 78.57% to 50.00%). This demonstrates that structural awareness is vital for the agent to efficiently orchestrate tools and solve complex analytical queries.

## 5 Conclusions and Limitations

In this work, we introduced PISA, a structured memory framework grounded in Piaget’s cognitive theory. Its unified schema architecture leverages task priors to build compact, query-efficient memory structures, complemented by scalable retrieval mechanisms. To address the narrow scope of existing benchmarks—often limited to factual text retrieval, we further designed AggQA, targeting structured data integration in medicine and finance. Empirical results show that PISA not only excels on the LOCOMO suite but also achieves strong performance on AggQA, highlighting its effectiveness for both general and task-oriented memory tasks.

A limitation of PISA lies in its reliance on prior knowledge. Moreover, our evaluation of task-oriented memory tasks has thus far been restricted to the AggQA benchmark we constructed. We aim to extend future testing to a broader range of benchmarks, which will allow for further optimization and validation of our approach.

## References

- AI (2025)
LangChain AI.
Langmem.
[https://langchain-ai.github.io/langmem/](https://langchain-ai.github.io/langmem/), 2025.
- Anthropic (2025)
Anthropic.
System card: Claude opus 4 & claude sonnet 4, May 2025.
URL [https://www.anthropic.com/news/system-card-claude-opus-4-claude-sonnet-4](https://www.anthropic.com/news/system-card-claude-opus-4-claude-sonnet-4).
- Anthropic (2025)
Anthropic.
Model Context Protocol Documentation.
[https://modelcontextprotocol.io/](https://modelcontextprotocol.io/), 2025.
Accessed: 2025-09-23.
- Chen et al. (2020)
Wenhu Chen, Hanwen Zha, Zhiyu Chen, Wenhan Xiong, Hong Wang, and William Yang Wang.
HybridQA: A dataset of multi-hop question answering over tabular and textual data.
In *Findings of the Association for Computational Linguistics: EMNLP 2020*, pp. 1026–1036, 2020.
doi: 10.18653/v1/2020.findings-emnlp.91.
- Chen et al. (2021)
Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, and William Yang Wang.
Finqa: A dataset of numerical reasoning over financial data.
*Proceedings of EMNLP 2021*, 2021.
- Chhikara et al. (2025)
Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav.
Mem0: Building production-ready ai agents with scalable long-term memory.
*arXiv preprint arXiv:2504.19413*, 2025.
- Du et al. (2025)
Yiming Du, Wenyu Huang, Danna Zheng, Zhaowei Wang, Sebastien Montella, Mirella Lapata, Kam-Fai Wong, and Jeff Z Pan.
Rethinking memory in ai: Taxonomy, operations, topics, and future directions.
*arXiv preprint arXiv:2505.00675*, 2025.
- Fish et al. (2025)
Sara Fish, Julia Shephard, Minkai Li, Ran I Shorrer, and Yannai A Gonczarowski.
Econevals: Benchmarks and litmus tests for llm agents in unknown environments.
*arXiv preprint arXiv:2503.18825*, 2025.
- Google (2025)
Google.
Gemini 2.5 Pro Model Card.
[https://modelcards.withgoogle.com/assets/documents/gemini-2.5-pro.pdf](https://modelcards.withgoogle.com/assets/documents/gemini-2.5-pro.pdf), jul 2025.
URL [https://modelcards.withgoogle.com/assets/documents/gemini-2.5-pro.pdf](https://modelcards.withgoogle.com/assets/documents/gemini-2.5-pro.pdf).
- Hsieh et al. (2024)
Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg.
RULER: What’s the real context size of your long-context language models?
*arXiv preprint*, arXiv:2404.06654, 2024.
- Hu et al. (2024)
Xueyu Hu, Ziyu Zhao, Shuang Wei, Ziwei Chai, Qianli Ma, Guoyin Wang, Xuwu Wang, Jing Su, Jingjing Xu, Ming Zhu, et al.
Infiagent-dabench: Evaluating agents on data analysis tasks.
*arXiv preprint arXiv:2401.05507*, 2024.
- Hu et al. (2025)
Yuanzhe Hu, Yu Wang, and Julian McAuley.
Evaluating memory in LLM agents via incremental multi-turn interactions.
*arXiv preprint*, arXiv:2507.05257, 2025.
- Jin et al. (2021)
Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanjun Fang, and Peter Szolovits.
What disease does this patient have? a large-scale open domain question answering dataset from medical exams.
*Applied Sciences*, 11(14):6421, 2021.
- Jin et al. (2019)
Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu.
Pubmedqa: A dataset for biomedical research question answering.
In *Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)*, pp. 2567–2577, 2019.
- Kang et al. (2025)
Jiazheng Kang, Mingming Ji, Zhe Zhao, and Ting Bai.
Memory os of ai agent.
*arXiv preprint arXiv:2506.06326*, 2025.
- Keles et al. (2023)
Feyza Duman Keles, Pruthuvi Mahesakya Wijewardena, and Chinmay Hegde.
On the computational complexity of self-attention.
In *International conference on algorithmic learning theory*, pp. 597–619. PMLR, 2023.
- Kweon et al. (2023)
Sunjun Kweon, Yeonsu Kwon, Seonhee Cho, Yohan Jo, and Edward Choi.
Open-WikiTable: Dataset for open domain question answering with complex reasoning over table.
*arXiv preprint*, arXiv:2305.07288, 2023.
- LangChain Team (2023)
LangChain Team.
LangGrap, 2023.
URL [https://github.com/langchain-ai/langgraph](https://github.com/langchain-ai/langgraph).
- Lewis et al. (2020)
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al.
Retrieval-augmented generation for knowledge-intensive nlp tasks.
*Advances in neural information processing systems*, 33:9459–9474, 2020.
- Liang et al. (2022)
Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng.
Code as policies: Language model programs for embodied control.
*arXiv preprint arXiv:2209.07753*, 2022.
- Lu et al. (2025)
Guilong Lu, Xuntao Guo, Rongjunchen Zhang, Wenqiao Zhu, and Ji Liu.
Bizfinbench: A business-driven real-world financial benchmark for evaluating llms.
*arXiv preprint arXiv:2505.19457*, 2025.
- Maharana et al. (2024a)
Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang.
Evaluating very long-term conversational memory of LLM agents.
*arXiv preprint*, arXiv:2402.17753, 2024a.
- Maharana et al. (2024b)
Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang.
Evaluating very long-term conversational memory of llm agents, 2024b.
URL [https://arxiv.org/abs/2402.17753](https://arxiv.org/abs/2402.17753).
- Nathani et al. (2025)
Deepak Nathani, Lovish Madaan, Nicholas Roberts, Nikolay Bashlykov, Ajay Menon, Vincent Moens, Amar Budhiraja, Despoina Magka, Vladislav Vorotilov, Gaurav Chaurasia, et al.
Mlgym: A new framework and benchmark for advancing ai research agents.
*arXiv preprint arXiv:2502.14499*, 2025.
- OpenAI (2025)
OpenAI.
GPT-5 System Card, aug 2025.
URL [https://openai.com/zh-Hans-CN/index/gpt-5-system-card/](https://openai.com/zh-Hans-CN/index/gpt-5-system-card/).
- Packer et al. (2024)
Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, and Joseph E. Gonzalez.
Memgpt: Towards llms as operating systems, 2024.
URL [https://arxiv.org/abs/2310.08560](https://arxiv.org/abs/2310.08560).
- Piaget et al. (1952)
Jean Piaget, Margaret Cook, et al.
*The origins of intelligence in children*, volume 8.
International universities press New York, 1952.
- Rasmussen et al. (2025)
Preston Rasmussen, Pavlo Paliychuk, Travis Beauvais, Jack Ryan, and Daniel Chalef.
Zep: A temporal knowledge graph architecture for agent memory, 2025.
URL [https://arxiv.org/abs/2501.13956](https://arxiv.org/abs/2501.13956).
- Wang et al. (2024)
Bing Wang, Xinnian Liang, Jian Yang, Hui Huang, Shuangzhi Wu, Peihao Wu, Lu Lu, Zejun Ma, and Zhoujun Li.
Enhancing large language model with self-controlled memory framework, 2024.
URL [https://arxiv.org/abs/2304.13343](https://arxiv.org/abs/2304.13343).
- Wang & Sun (2025)
Chupei Wang and Jiaqiu Vince Sun.
Unable to forget: Proactive lnterference reveals working memory limits in llms beyond context length.
*arXiv preprint arXiv:2506.08184*, 2025.
- Wang et al. (2025)
Weixuan Wang, Dongge Han, Daniel Madrigal Diaz, Jin Xu, Victor Rühle, and Saravan Rajmohan.
Odysseybench: Evaluating llm agents on long-horizon complex office application workflows.
*arXiv preprint arXiv:2508.09124*, 2025.
- Wu et al. (2025)
Di Wu, Hongwei Wang, Wenhao Yu, Yuwei Zhang, Kai-Wei Chang, and Dong Yu.
LongMemEval: Benchmarking chat assistants on long-term interactive memory.
*arXiv preprint*, arXiv:2410.10813, 2025.
- Xi et al. (2025)
Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al.
The rise and potential of large language model based agents: A survey.
*Science China Information Sciences*, 68(2):121101, 2025.
- Xu et al. (2023)
Canwen Xu, Daya Guo, Nan Duan, and Julian McAuley.
Baize: An open-source chat model with parameter-efficient tuning on self-chat data.
In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), *Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing*, pp. 6268–6278, Singapore, December 2023. Association for Computational Linguistics.
doi: 10.18653/v1/2023.emnlp-main.385.
URL [https://aclanthology.org/2023.emnlp-main.385/](https://aclanthology.org/2023.emnlp-main.385/).
- Xu et al. (2025)
Wujiang Xu, Kai Mei, Hang Gao, Juntao Tan, Zujie Liang, and Yongfeng Zhang.
A-mem: Agentic memory for llm agents.
*arXiv preprint arXiv:2502.12110*, 2025.
- Yao et al. (2023)
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao.
ReAct: Synergizing reasoning and acting in language models.
In *International Conference on Learning Representations (ICLR)*, 2023.
- Yu et al. (2025)
Xiaohan Yu, Pu Jian, and Chong Chen.
Tablerag: A retrieval augmented generation framework for heterogeneous document reasoning, 2025.
URL [https://arxiv.org/abs/2506.10380](https://arxiv.org/abs/2506.10380).
- Zhang et al. (2025)
Dan Zhang, Sining Zhoubian, Min Cai, Fengzu Li, Lekang Yang, Wei Wang, Tianjiao Dong, Ziniu Hu, Jie Tang, and Yisong Yue.
Datascibench: An llm agent benchmark for data science.
*arXiv preprint arXiv:2502.13897*, 2025.
- Zhong et al. (2024)
Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang.
Memorybank: enhancing large language models with long-term memory.
In *Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence*, AAAI’24/IAAI’24/EAAI’24. AAAI Press, 2024.
ISBN 978-1-57735-887-9.
doi: 10.1609/aaai.v38i17.29946.
URL [https://doi.org/10.1609/aaai.v38i17.29946](https://doi.org/10.1609/aaai.v38i17.29946).

## Appendix A LLM Usage Statement

In the preparation of this paper, the use of large language models (LLMs) was strictly confined to refining the final text. The authors first independently authored and completed full drafts of the introduction, related works, and PISA sections, ensuring all arguments and findings were our own. Only after this original writing phase did we employ a set of LLMs, namely Gemini 2.5 Pro(Google, 2025), Claude 4.0 Sonnet(Anthropic, 2025), and GPT-5(OpenAI, 2025), for the purpose of polish and refinement. Our interaction with these models focused on two areas: identifying and correcting potential grammatical or semantic errors, and gathering alternative phrasing suggestions to improve the manuscript’s readability and coherence. The authors carefully reviewed all suggestions, and the final text is the result of our deliberate consideration and integration of these editorial recommendations. We affirm that the authors are solely and entirely responsible for the substance and accuracy of this work.

## Appendix B Additional Material of PISA

### B.1 Piaget’s Constructivist Theory: A Detailed Computational Analogy

The design philosophy of the PISA memory system is rooted in the constructivist theory of Piaget et al. (1952), which provides a profound conceptual blueprint for understanding and building intelligent agents capable of autonomous learning. This section aims to dissect the core concepts of this theory in detail and clarify how they are translated into the specific computational mechanisms of the PISA system.

#### B.1.1 Schema: The Structural Unit of Knowledge

In Piaget’s schema theory, a “Schema” is the basic structural unit of cognition, a mental representation or knowledge framework that an individual uses to understand and respond to the environment (Piaget et al., 1952). It is not static information but a dynamic, organized program, such as an infant interacting with objects through a “grasping” schema.

In PISA, this concept is directly mapped to a structured unit of knowledge within the system, also called a “Schema.” Each PISA schema is a computational object that encapsulates knowledge for a specific task, stored in the Schema Engine. It contains not only descriptive knowledge (e.g., task goals, application conditions) but, more importantly, procedural knowledge (e.g., a sequence of actions or API calls required to complete the task). This makes PISA’s schemas consistent with Piaget’s definition: they are executable “mini-programs” for interacting with the world.

#### B.1.2 Assimilation: Integrating the Familiar

“Assimilation” is the process of integrating new information into existing schemas when an individual encounters it. For example, a child who has formed a “dog” schema will classify a new, different breed of dog into the “dog” category, thereby strengthening and generalizing the existing schema. Assimilation is an efficient process of understanding the world using existing knowledge.

PISA simulates “assimilation” through the Retrieval Module and subsequent successful execution. When a new task arrives:

- 1.
The Retrieval Module first attempts to match the task with an existing schema in the Schema Engine.
- 2.
If a matched schema is successfully executed (i.e., the agent completes the task smoothly), this constitutes an “assimilation” event.
- 3.
As a result, the confidence or weight of that schema is increased, making it more likely to be retrieved in the future.

This process is illustrated in Figure [6](https://arxiv.org/html/2510.15966v1#A2.F6), which shows how new experiences are integrated into existing cognitive frameworks. The initial “cat” schema includes attributes such as color (brown), legs (4), ears (2), and tail (1). When a “new experience” with new color information (a slightly darker brown) is encountered, the system readily integrates the new color into the existing “cat” schema, forming an updated “assimilated cat” schema where the color attribute is updated while other basic attributes remain unchanged. This ensures that the system can efficiently utilize and consolidate validated successful experiences, achieving stable application of knowledge.

Figure: Figure 6: Assimilation: Integrating new experiences into existing cognitive frameworks.
Refer to caption: x6.png

#### B.1.3 Accommodation: Adapting to the Novel

”Accommodation” occurs when existing schemas cannot explain new information or lead to failure, a state known as cognitive disequilibrium. At this point, the cognitive system must adjust old schemas or create new ones to adapt to the new challenge. For example, when a child sees a cat for the first time, trying to understand it with the ”dog” schema will create a conflict (e.g., different sounds), forcing the child to adjust their cognitive structure and eventually form a new ”cat” schema.

PISA computationalizes the “accommodation” process and uses it as the core mechanism for dealing with failures and novel environments, primarily handled by the Adaptation Module (its position in the overall architecture is shown in Figure [2](https://arxiv.org/html/2510.15966v1#S1.F2)). When a task execution fails or an unexpected result occurs:

- 1.
The system identifies this as a “Cognitive Disequilibrium,” indicating a flaw in the current schema.
- 2.
The Adaptation Module is activated, and it analyzes the cause of the failure (e.g., incorrect API parameters, incomplete action sequence).
- 3.
Based on the analysis, the module performs a structural modification on the failed schema (e.g., correcting parameters, adding/deleting steps) or creates a completely new schema to solve the problem.

This mechanism is implemented in two ways, as shown in Figure [7](https://arxiv.org/html/2510.15966v1#A2.F7) and Figure [8](https://arxiv.org/html/2510.15966v1#A2.F8).

- •
Figure [7](https://arxiv.org/html/2510.15966v1#A2.F7) depicts the “Accommodation (modify)” process: When the initial schema for “coffee” (Taste: Bitter, Color: Brown, Effect: N/A, Smell: N/A) encounters an “actual experience” and creates “disequilibrium” (Effect: Refreshing, Smell: Aromatic), the existing schema is insufficient to fully explain the new information. Through the “accommodation” process, the system modifies the original “coffee” schema, adding the “Refreshing” effect and “Aromatic” smell attributes, forming a more complete “accommodated coffee” schema.
- •
Figure [8](https://arxiv.org/html/2510.15966v1#A2.F8) depicts the “Accommodation (create)” process: When the initial schema for “dog” (Sound: Woof) encounters an “actual experience” and creates “disequilibrium” (Sound: Meow), the existing “dog” schema cannot handle the novel “Meow” sound. Through the “accommodation” process, the system creates a brand new “cat” schema that includes the “Meow” sound attribute, thereby effectively handling the new information.

This mechanism endows PISA with true learning capabilities, enabling it to extract knowledge from failure and achieve fundamental growth in its cognitive structure.

Figure: Figure 7: Accommodation (modify): Modifying existing schemas when they fail to adequately account for novel information.
Refer to caption: x7.png

Figure: Figure 8: Accommodation (create): Creating new schemas when existing structures cannot handle novel information.
Refer to caption: x8.png

#### B.1.4 Equilibration: The Drive for Balance

“Equilibration” is the internal driving force behind all cognitive development (Piaget et al., 1952). It is the process of seeking a balance between assimilation and accommodation. The system strives to achieve a stable state of equilibrium, but new challenges constantly disrupt this balance, driving the system to achieve a higher, more adaptive new equilibrium through accommodation. Cognitive development thus emerges from this dynamic equilibrium between assimilation and accommodation.

In PISA, “equilibration” is manifested in the system’s continuous pursuit of an efficient and accurate schema library. The system defaults to efficient “assimilation,” but when “cognitive disequilibrium” (task failure) accumulates, it will activate “accommodation” at a higher cost to correct the model, aiming to reach a new, more powerful “equilibrium.” This dynamic balancing mechanism allows PISA to resolve the fundamental “stability-plasticity” dilemma, building a memory system that is both stable in operation and capable of continuous evolution.

### B.2 Conflict Resolution in Memory

To maintain the integrity and consistency of the stored information, PISA incorporates a sophisticated conflict resolution mechanism. This process involves detecting conflicting records, evaluating their reliability, and deactivating those that are less reliable.

#### B.2.1 Conflict Detection

Within each memory element $u$, which contains a set of records $V(u)$, conflicts between any two records $r_{i}$ and $r_{j}$ are identified using the following contradiction function:

$$ $\mathrm{Contr}(r_{i},r_{j})=\mathbb{I}[\exists k\in K_{b}\cap\mathrm{keys}(r_{i})\cap\mathrm{keys}(r_{j}):\;\mathrm{conflict}(\mathrm{val}_{k}(r_{i}),\mathrm{val}_{k}(r_{j}))].$ $$

Here, $K_{b}$ represents the set of keys for a given bucket $b$. A conflict is flagged if any shared key $k$ has conflicting values. The $\mathrm{conflict}(\cdot,\cdot)$ predicate is domain-specific; for instance, it could check for inequality of canonical strings for categorical keys or a value difference exceeding a tolerance for numerical keys.

#### B.2.2 Record Reliability Scoring

Each record $r$ is assigned a reliability score $s(r)$ based on three factors: recency, source quality, and community support. The score is calculated as:

$$ $s(r)=w_{\text{recency}}\cdot(1+\mathrm{age}(r))^{-1}+w_{\text{source}}\cdot q_{\text{src}}(r)+w_{\text{support}}\cdot\#\mathrm{supports}(r).$ $$

where $\mathrm{age}(r)$ is the age of the record, $q_{\text{src}}(r)$ is the quality score of its source, and $\#\mathrm{supports}(r)$ is the number of other records that support it. The weights $w_{\text{recency}}$, $w_{\text{source}}$, and $w_{\text{support}}$ are nonnegative weights that sum to 1, controlling the relative importance of these factors.

#### B.2.3 Conflict Deactivation

Conflicts are resolved by deactivating less reliable records. We construct a conflict graph $G_{u}$ for each element $u$, where an edge exists between any pair of records with $\mathrm{Contr}=1$. For each connected component $C$ in this graph, a single ”winner” record is chosen to remain active, while all others in the component are deactivated. The active status $\alpha(r)$ is determined by:

$$ $\alpha(r)=\begin{dcases}1,&\text{if }r=\operatorname*{arg\max}_{r^{\prime}\in C}s(r^{\prime})\\ 0,&\text{otherwise.}\end{dcases}$ $$

Ties are broken deterministically, first by preferring the record with the smaller $\mathrm{age}(r^{\prime})$, and then by the larger source quality score $q_{\text{src}}(r^{\prime})$. This winner-takes-all approach ensures that only the most reliable and consistent information is retained.

### B.3 MCP Tools of Retrieval Module

In the retrieval module of PISA, we implement a set of tools that adhere to Anthropic’s Model-Context Protocol (MCP)(Anthropic, 2025). These tools operate as independent back-end microservices and communicate with the upstream agent via standard input/output (STDIO). This design offers several core advantages. First, functional decoupling allows each tool to be developed, tested, and deployed independently, enhancing the system’s maintainability. Second, running tools as separate processes provides a foundation for fine-grained security control and resource isolation, ensuring system stability. Finally, adherence to the MCP standard promotes interoperability between the tools and various models or agents.

##### RAG Service

The RAG Service is responsible for retrieving unstructured and semi-structured data. Built upon a vector database, it performs semantic similarity searches based on natural language queries. To ensure data freshness, we have designed a periodic update mechanism that incrementally incorporates the latest information into the index. This service is crucial for PISA’s ability to comprehend extensive conversation histories, external documents, and knowledge bases.

##### Structural Query Service

To facilitate secure access to structured data, we have implemented an SQL Service. This service acts as a proxy layer to the database. It receives SQL query requests, which are often generated from natural language, and subjects them to rigorous syntax checks and security audits before execution to prevent malicious attacks such as SQL injection. All queries are executed within a restricted sandbox environment, exposing only necessary database views, thereby protecting the integrity of the underlying data.

##### Calculator Service

To empower the agent with the ability to perform precise calculations, we provide a Calculator Service. This service executes mathematical expressions within an isolated sandbox environment, supporting basic arithmetic operations, variable assignments, and function calls. This method of isolated execution mitigates the risks associated with unsafe operations like ‘eval()’, ensuring the security of computational tasks.

##### Collaborative Workflow

These tools do not operate in isolation but are orchestrated by the agent to accomplish complex tasks collaboratively. A typical workflow is as follows: The agent first retrieves relevant information from the knowledge base using the RAG Service, then queries for precise data from a structured database using the SQL Service, and finally may use the Calculator Service to perform statistical analysis on the query results. The entire process communicates via the MCP protocol, ensuring standardized data exchange and procedural reliability. Through this approach, PISA’s retrieval module can efficiently and securely integrate information from diverse sources to provide decision support for upstream applications.

## Appendix C AggQA

### C.1 Dataset Construction

Figure [9](https://arxiv.org/html/2510.15966v1#A3.F9) illustrates the comprehensive pipeline for constructing the AggQA benchmark. The process is organized into four main stages: Corpus Collection, where source data is gathered and preprocessed; two parallel generation tracks named QA-Pair Generation and Session Simulation; and a final Manual Refinement stage to ensure data quality. In the parallel tracks, we generate structured question-answer items and context-rich dialogues independently. Both streams of data then converge into the unified human verification stage, ensuring all benchmark components meet our quality standards.

Figure: Figure 9: The construction pipeline of the AggQA benchmark. The process flows from Corpus Collection to two parallel generation tracks (QA-Pair Generation and Session Simulation), and culminates in a unified Manual Refinement stage.
Refer to caption: x9.png

#### C.1.1 Source Corpora and Preprocessing

The foundation of AggQA lies in two distinct, high-quality datasets, which we preprocess to extract and structure the numerical data essential for our tasks. Table [4](https://arxiv.org/html/2510.15966v1#A3.T4) provides an overview.

**Table 4: Source Corpora Overview**
| Domain | Source Dataset | Data Type | License |
| --- | --- | --- | --- |
| Financial | BizFinBench (Lu et al., 2025) | Textual Tabular | CC BY-NC 4.0 |
| Medical | PubMedQA (Jin et al., 2019) | Structured Abstracts | MIT License |

##### Financial Domain.

We utilize the BizFinBench dataset (Lu et al., 2025), focusing on its ”Financial Numerical Computation” task. In this task, the required tabular data is often presented in a textual format directly within the question prompt. Our preprocessing involves parsing these prompts to programmatically extract and translate the textualized tables into structured English formats, while preserving professional terminology with standardized abbreviations. This process yields clean, well-structured financial tables that serve as the evidentiary basis for our generation tasks.

##### Medical Domain.

We leverage the PubMedQA dataset (Jin et al., 2019). The source data is provided as JSON objects, where each entry contains a structured abstract with content segmented into labeled sections. Although the raw data has little tabular data, we developed a synthesis procedure that first parses these structured abstracts to build a rich vocabulary of specialized medical terminologies. This vocabulary is then used to generate synthetic yet realistic medical measurement records in a tabular CSV format.

#### C.1.2 QA-Pair Generation

This track is dedicated to producing aggregation-oriented, single-turn question-answering items that can be programmatically verified. It follows a dual-stream methodology.

The first stream is LLM-driven. An LLM is prompted with a snippet of continuous data from a preprocessed table and is tasked with generating a complete tuple: a question (Q), answer (A1), evidence (E1), an answer query (SQL_Answer), and an evidence query (SQL_Evidence). We execute these SQL queries to obtain the ground-truth SQL answer (A2) and SQL evidence (E2). The generated A1/E1 are then matched against A2/E2, with any discrepancies flagged for human review.

The second stream is script-driven. For data exhibiting clear patterns, we employ Python scripts to generate precise answer-evidence pairs and a logical “hint”. This structured information is then passed to an LLM, which focuses on formulating diverse, natural-language questions based on the pre-verified facts.

#### C.1.3 Session Simulation

In this parallel track, we adapt the self-chat methodology (Xu et al., 2023) to generate multi-turn, evidence-coverage-oriented professional sessions. The process involves an LLM simulating a conversation between a User and an Assistant, where numerical data points from the source tables are discretely embedded into the dialogue. This technique creates a mixed information stream that intertwines structured data with conversational context, closely mimicking a real-world analytical task. To foster interaction diversity, the simulation operates in two distinct modes.

##### Mode A: User-Provided Data.

The simulation starts with the User presenting a subset of data and asking for analysis, such as identifying a trend or explaining an anomaly. The Assistant responds directly based on the information provided by the User.

##### Mode B: User-Sought Data.

The User initiates the conversation by seeking specific information. The Assistant, with access to the entire data table as its knowledge base, retrieves and presents the relevant data, often prompting further follow-up questions from the User.

#### C.1.4 Manual Refinement

All instances from both the QA-Pair Generation and Session Simulation tracks converge at this final stage for human review. The main objective here is to validate the factual accuracy of answers and ensure a logical consistency between the dialogue, the question, and the supporting evidence. Our domain-expert annotators focus on confirming that the final answer is correctly derivable from the provided evidence and that the dialogue context sufficiently covers the necessary data points. This process filters out instances with clear factual errors or logical inconsistencies, thereby ensuring the overall reliability of the benchmark. Figures [10](https://arxiv.org/html/2510.15966v1#A3.F10), [11](https://arxiv.org/html/2510.15966v1#A3.F11), and [12](https://arxiv.org/html/2510.15966v1#A3.F12) provide examples of instances that have passed or failed this refinement process.

Figure: Figure 10: An example of a correct self-chat session.

Figure: Figure 11: An example of a wrong self-chat session

Figure: Figure 12: An example of a mismatch question-answer pair

### C.2 Dataset Analysis

To contextualize our contribution, we provide a detailed analysis of the AggQA benchmark. We first compare its structural and task-oriented characteristics with several established datasets and then delve into the internal distributions of the data itself.

#### C.2.1 Basic Statistics and Comparison

Current evaluation benchmarks generally fall into two distinct categories. On one hand, long-context benchmarks like RULER (Hsieh et al., 2024), LoCoMo (Maharana et al., 2024a), and LongMemEval (Wu et al., 2025) assess memory over long, unstructured texts but typically focus on retrieving isolated facts. On the other hand, Table QA benchmarks such as FinQA (Chen et al., 2021) and HeteQA (Yu et al., 2025) test complex reasoning but always provide a complete, structured table upfront.

AggQA is designed to bridge this gap. It uniquely evaluates a system’s ability to first reconstruct a latent table from numerical data discretely embedded within a long dialogue, and then aggregate this data. Table [5](https://arxiv.org/html/2510.15966v1#A3.T5) situates AggQA relative to these existing benchmark families.

**Table 5: Descriptive comparison of AggQA. Its primary innovation is the “Discretized Table” format, which necessitates the novel challenge of ”Reconstruction & Aggregation”.**
| Dataset | Context Modality | Numerical Data Format | Primary Challenge | Context<br>($\sim$K token) |
| --- | --- | --- | --- | --- |
| AggQA (Ours) | Dialogue | Discretized Table | Reconstruction & Aggregation | $\sim$56 |
| RULER (Hsieh et al., 2024) | Static Document | Isolated Facts | Fact Retrieval | $3\sim 200$ |
| LoCoMo (Maharana et al., 2024a) | Dialogue | N/A | Causal Understanding | $\sim$9 |
| LongMemEval (Wu et al., 2025) | Dialogue | N/A | Memory Abilities | $\sim$115 |
| MemoryAgentBench (Hu et al., 2025) | Dialogue | N/A | Agentic Memory Use | $\sim$64 |
| HeteQA (Yu et al., 2025) | Static Document | Explicit Table | Multi-hop Table QA | - |
| HybridQA (Chen et al., 2020) | Static Document | Explicit Table | Hybrid QA | $<$16 |
| FinQA (Chen et al., 2021) | Static Document | Explicit Table | Numerical Reasoning | $<$4 |
| MedQA (Jin et al., 2021) | Static Document | N/A | Reading Comprehension | $<$2 |

As the comparison illustrates, AggQA introduces a fundamentally new evaluation paradigm. Unlike long-context benchmarks, it moves beyond simple fact retrieval to require reasoning over interrelated numerical data. Unlike all listed table-based benchmarks, AggQA does not provide an explicit, clean table. Instead, its Implicit & Discretized data format forces models to actively search, identify, and mentally reconstruct the tabular structure from scattered mentions within a long dialogue. This dual challenge of Numerical Reconstruction & Aggregation represents a more realistic and complex test for memory systems, simulating how an agent must often synthesize structured insights from unstructured, conversational data streams.

#### C.2.2 Question and Evidence Composition

Figure: (a) Question difficulty levels.

The AggQA benchmark is composed of 85 questions in the Financial domain and 49 in the Medical domain. To ensure a rigorous evaluation, the majority of these are designed to be of medium to high complexity. As illustrated in Figure [13](https://arxiv.org/html/2510.15966v1#A3.F13)a, 55.3% of Financial questions are “Medium” and 17.6% are “Hard”. The Medical domain follows a similar pattern, with 51.0% ”Medium” and 26.5% “Hard” questions, ensuring the benchmark robustly tests advanced reasoning capabilities.

A key metric for task complexity is the evidence span. For time-based questions, we define this as the duration in days of the relevant period, while for non-temporal questions, it is the number of discrete evidence items. As shown in Figure [13](https://arxiv.org/html/2510.15966v1#A3.F13)b, the distribution highlights a focus on medium- to long-term analysis. The vast majority of questions—nearly 80%—cover a span of between 8 and 31 days (one to four weeks). This design forces models to process and reason over extended sequences of data, rather than isolated facts, to derive an answer.

#### C.2.3 SQL Complexity Analysis

The ground-truth SQL queries provide a semantic blueprint of the required reasoning process for all 49 Medical questions and 41 of the 85 Financial questions. Our analysis of these queries, summarized in Table [6](https://arxiv.org/html/2510.15966v1#A3.T6), reveals the profound logical and mathematical complexity embedded in the benchmark.

**Table 6: SQL Complexity Analysis: A side-by-side comparison of component occurrence rates (% of questions with SQL) across domains.**
| SQL Component | Financial (%) | Medical (%) |
| --- | --- | --- |
| Aggregate Functions (AVG, SUM, etc.) | 100.0 | 71.4 |
| Subqueries (Nested SELECT) | 68.3 | 34.7 |
| Arithmetic Operations (+, -, *, /) | 100.0 | 100.0 |
| Multiple WHERE Conditions | 100.0 | 100.0 |
| GROUP BY Clause | 9.8 | 10.2 |

As shown in the table, tasks in both domains consistently require complex logic, with 100% of SQL-based questions involving arithmetic operations and multiple filtering conditions. However, distinct patterns emerge. The Financial tasks are characterized by a higher dependency on multi-step reasoning, evidenced by the fact that 68.3% of its queries involve subqueries—nearly double the rate in the Medical domain (34.7%). Furthermore, every Financial SQL query utilizes an aggregate function. This high prevalence of complex logical structures ensures that AggQA robustly evaluates a model’s capacity for sophisticated, multi-step numerical reasoning.

### C.3 Detailed Results

While the end-to-end accuracy presented in Table [2](https://arxiv.org/html/2510.15966v1#S4.T2) provides a summary of overall performance, a deeper diagnosis is required to understand the specific failure points of each model. To this end, we introduce a critical intermediate metric: Evidence Coverage. We define this as the model’s ability to successfully retrieve all discrete data points required to compute the final answer from the context. A high evidence coverage indicates strong data recall, while a low coverage points to failures in the initial retrieval stage.

Table [7](https://arxiv.org/html/2510.15966v1#A3.T7) provides a side-by-side comparison of the overall accuracy with a detailed breakdown of evidence coverage across different difficulty levels. This analysis allows us to disentangle retrieval failures from reasoning failures, revealing whether a model’s poor performance stems from an inability to find the correct information or an inability to compute with it correctly.

**Table 7: Detailed comparison of Overall Accuracy (%) and Evidence Coverage (%) across difficulty levels for MED & FIN datasets.**
| Method | Med. | Fin. |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Overall<br>Acc. | Evidence Coverage Detail | Overall<br>Acc. | Evidence Coverage Detail |  |  |  |  |  |
| Easy | Medium | Hard | Easy | Medium | Hard |  |  |  |
| MemoryOS | 10.20 | 18.18 | 10.00 | 0 | 14.29 | - | - | - |
| Zep | 22.45 | 24.54 | 48.40 | 3.07 | 9.41 | 25.26 | 26.67 | 40.00 |
| AMem | 20.41 | 36.27 | 29.92 | 3.85 | 31.77 | 50.04 | 40.63 | 42.29 |
| Mem0 | 30.61 | 35.45 | 48.40 | 46.54 | 23.53 | 60.35 | 41.77 | 50.00 |
| LangMem | 51.02 | 50.00 | 54.72 | 43.23 | 22.35 | 44.22 | 23.63 | 0 |
| PISA | 69.39 | 68.18 | 65.48 | 63.69 | 51.77 | 53.39 | 61.65 | 40.71 |
| PISA* | 83.67 | 83.27 | 86.60 | 82.77 | 69.41 | 83.13 | 75.02 | 65.93 |
| FullContext | 51.02 | 50 | 54.72 | 43.23 | 70.59 | 97.09 | 84.04 | 53.71 |

## Peer Reviews

confidence: 4
contribution: 1
initial_rating: 2
presentation: 2
questions: ## Details about weaknesses

- the paper has lightweight contribution

In PISA Initialization/adaptation and retrieval module manage  memories as schemas. while the paper compares PISA to mem0, AMem and LangMem, the comparison remains superficial -- i,e., it is difficult to appreciate the "Piaget-inspired adaptive memory evolution mechanism,'' as well as what is the key that from  "[...] Piaget’s cognitive theory, that constructs task-oriented memory structure''.  

the explanation is conducted with pedagogic examples, that remain excessively simplistic and are not remotely connected to the proposed benchmarks. 


- evaluation is lightweight and quite superficial 

the paper shows relatively simple experiment on one public (LOCOMO) and one purposely  built benchmark. 


the evaluation is conducted with either classic aggregated values (on which there may be a problem, see next)  or with again pedagogic examples (in the appendix)

yet, even digging tyhe appendix the examples remain excessively simplistic, and are not digging to any extent the usefulness of the proposed schemas -- and its adaptive evolutionary status  

for instance, Reecord Reliability Scoring depends on numerous weighting parameters, whose tuning in real-life settings may be involved and in the paper is barely mentioned and not even discussed (so for sure no ablation ios carried over it) -- but this is just one example



- evaluation benchmark appears lower than what publicly reported

The results of compared baslines appear lower than the expectations as already publicly reported  (eg see https://www.memobase.io/blog/ai-memory-benchmark ), and the published benchmark often exceeds PISA on LOCOMO.


        Method 		Single      Multi	OpenDomain	Temporal	         Overall
        mem0              67.13	51.15	72.93		55.51		66.88
       LangMem		62.23	47.92	71.12		23.43		58.10
       Zep			74.11	66.04	67.71		79.79		75.14
       OpenAI 		63.79	42.92	62.29		21.71		52.90
       Memobase  	    70.92	   46.88       77.17 		85.05 		75.78



## Minor/Languange/Style
typos:
"Reproducability statement" -> reproducibility p10

### overloaded styles
the paper mixes syntactical use of “” with overloaded semantic implications: you should only use it for examples but not for implementation 

 - implementation: “JSONB”  (2.1_ schema engine)
 - examples:  topic “Drink” and specific Element “Pure Milk” (same section) 


the paper again mixes syntactical use of “” with overloaded semantic implications, incresing confusion -- either you use “”for precise defs, or for approximation and intuitive metaphores

- precise definition:  [...] structured unit of knowledge within the system, also called a “Schema.”  [...]  “Assimilation" is the process of
- approximation/metaphore: This makes PISA’s schemas consistent with Piaget’s definition: they are executable “mini-programs”  [...] 
- examples: different breed of dog into the “dog” category

### mixed styles

the paper mixes styling --  choose one but mixing the two is not recommended 

	{\texttt RULER} vs RULER  
	{\texttt LOCOMO} vs LOCOMO
soundness: 2
strengths: - memory in LLM is an active topic 
- the paper is clearly written
summary: the paper proposes PISA, a task oriented memory system for LLM agents, which is an active research subject

the paper casts a shade of theoretically principled design (by pompously recalling Piaget & Cook -- already erroneously cited as Piaget, Cook *et al.*, as if there was a third unkown coauthor) onto a
quite practical  implementation of a memory structured around schemas, that can be assimilated/accomodated (adding values into existing ones) or created ex novo (for new concepts).

the paper shows relatively simple experiment on one public (LOCOMO) and one purposely  built benchmark.  in short, the paper is lightweight in contribution and evaluation, and  overall contribution seems quite thin



Jean Piaget, Margaret Cook, *et al.* The origins of intelligence in children, volume 8. International universities press New York, 1952
weaknesses: (detail of weaknesses expanded below in question section)

- the paper has lightweight contribution
- evaluation is lightweight and quite superficial 
- evaluation benchmark appears lower than what publicly reported

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 4
questions: ## Justification that the use of schemas is flexible and provides a significant improvement

The use of schemas is the central component of the system and of the paper.

- Q. Can you justify that the improved performance with the PISA system is because of the use of schemas instead of the retrieval module? In particular, how similar are the retrieval tools of the competitor methods?

The schema modelization seems simplistic. For formulating this issue objectively, can you explain how the mechanisms interact in those cases:

- Q. From example in Fig. 6. What about "The cat of my neighbor has 3 legs"? This may be true for this particular cat. How it should be handled ideally (the "cat" schema is updated? a new specific schema is created?)

- Q. From example in Fig. 1. What about "I like coffee with milk only when the music is jazz". How this is integrated?

- Q. Is it possible to update multiple schemas at once?


## Comparison with full context retrieval

Authors identify two challenges using AI agents: "risk of exceeding context window" and "irrelevant information may interfere with the model judgment". While I agree with the authors, the experiments against the full context size are not convincing on this aspect: comparing Tab. 2 and Tab. 3 on the AggQA financial dataset, without the retrieval module (fair comparison with full context), the results show that PISA (without the retrieval module) is not doing better than full context.

- Q. Can you confirm that in the full context experiment, no tool can be used

- Q. PISA is using both memory management and retrieval tools. Is it possible to compare in a fair way whether the proposed memory management is useful w.r.t. full context?

## Other questions

- Q. I don't understand clearly the initialization stage. Can you explain this phase in your experiment? What is the provided data at that stage?

- Q. why removing the adversarial category of locomo in Table 1?

- Q. what is the list of the memory pool buckets? In Fig. 1: user trait, user event, relationship; then in Fig. 3: user events, agent events. Is it fixed at the initialization stage?

- Q. How important is the conflict detection module in the experiment?
soundness: 2
strengths: - the concept of schemas is well explained and gives a structured way to represent the memory history, with some flexibility in creating and adapting memories
- consistent improvement against other memory retrieval methods on locomo (excluding the adversarial category)
- I checked the availability of both source code and data
summary: The authors of this paper propose a unified memory system for AI agents. The system is based on the concept of *schemas*, from the Piaget's schema theory. A schema encodes a structured knowledge, containing a meta (drink, music, ...), an element (coffee, jazz...), the value linked to the element (e.g. coffee details and preferences) and an experience identification.

The system itself proceeds in three steps: in a first step, the schemas are initialized based on existing data; then the schemas are updated over time continuously, with the possibility of assimilation (new values of an element), evolution (new element of a meta) or creation (new meta). At that stage, a conflict analysis is also performed. Finally, the system contains a retrieval module, that can query the memory pool with different tools (RAG, SQL, calculator).

The system is evaluated on LOCOMO and AggQA. AggQA is a new benchmark proposed by the authors, containing 134 questions from financial and medical domains (example of question: "What was the average closing price of Pinterest stock from April 1 to April 30, 2024?"). The evaluation is performed with gpt-5-mini against existing competitors (Mem0, LangMem, etc.) and direct answer by the model using full context. On LOCOMO, PISA shows better results compared to existing competitors, but is not better than full context. On AggQA, PISA outperforms both competitors and direct full context prompting.
weaknesses: - schema management seems simplistic for real cases (see questions below)
- the introduction of schemas does not seem to be the main component in the good performance
- the experiments do not clearly show the improvement compared to full context

---

confidence: 2
contribution: 2
initial_rating: 4
presentation: 3
questions: - **Q1**. How is the boundary between Meta and Element determined, and what prevents overly broad or excessively fragmented schema definitions?
- **Q2**. Following Q1, are the keywords used for Meta, Element, and Value guaranteed to be unique? If so, how does the system handle cases where a keyword that appears as a Meta category should also appear as an Element or Value in a different context?
- **Q3**. How does the framework address temporal dependencies—such as ordering and causality—during memory retrieval and decision-making?
- **Q4**. What mechanisms ensure unique identity tracking when multiple instances share the same Meta category (e.g., two different entities both categorized under “Dog”)?
soundness: 2
strengths: - **S1**. The authors propose a schema engine that supports a constructive and adaptive hybrid memory system, allowing more structured and flexible memory management.
- **S2**. The design of the AggQA task effectively demonstrates the task-oriented properties and practical advantages of the proposed PISA framework.
- **S3**. Including ablation studies and evolutionary threshold analyses provides valuable insights into the contributions and behavior of different model components.
summary: This paper introduces PISA, a unified memory system to improve an agent’s adaptability to diverse tasks and enhance task-oriented decision-making. Inspired by cognitive psychology, the authors propose a schema-based memory representation that enables agents to actively construct, refine, and retrieve relevant knowledge. The framework demonstrates improved performance in memory-dependent agent tasks, and the authors further design and evaluate the AggQA benchmark to assess task-oriented memory management. The results indicate that PISA offers advantages in organizing and utilizing memory for effective task execution.
weaknesses: - **W1**. The criteria and methodology for defining the Meta, Element, and Element-Value categories remain ambiguous.
- **W2**. The ablation study results show that initialization is critical, and the system depends heavily on it. However, there is insufficient explanation regarding how initialization is performed using prior knowledge and how the schema configuration may vary across different domains.
- **W3**. In the PISA framework, temporal relationships and identity tracking for similar entities (e.g., Dog1 vs. Dog2) are not adequately addressed, limiting clarity on how memory maintains continuity over time.
- **W4**. The adaptive processing modules introduce schema-level similarity scores for schema matching and element-level compatibility scores for element matching, yet the operational details of these computations are not fully described. More elaboration is needed as these components appear essential for memory management within the PISA framework.

---

confidence: 3
contribution: 1
initial_rating: 2
presentation: 3
questions: - Table 1 shows that providing full context to the model outperforms all memory systems on the LOCOMO benchmark. How do the authors reconcile this result with the paper's central premise that memory systems should address the limitations of full-context approaches (line 42: "excessive historical information [...] risks exceeding the model's context window; [...] amount of irrelevant information contained may interfere with the model's judgment and decision-making")?

- What accounts for the observation that both PISA and the full-context baseline achieve superior performance on AggQA's medium difficulty level compared to the easy difficulty level?
soundness: 2
strengths: **Novel memory architecture**: The authors introduce a psychologically-inspired approach to memory storage for AI agents that exploits hierarchical schema structures to create, access, and update memories more effectively.

**Comprehensive system description**: The paper provides detailed explanations of each architectural component and presents algorithms necessary for system functioning, with a commitment to make code publicly available.

**New evaluation benchmark**: The authors introduce AggQA dataset, to evaluate models on data analysis tasks across medical and finance domains with varying difficulty levels.
summary: This paper proposes a new memory system designed to enhance AI agents in long-term conversational and data analysis tasks. Drawing inspiration from Piaget's cognitive development theory, PISA introduces a three-module architecture: an Initialization Module that constructs task-oriented memory structures through schema creation, an Adaptation Module that implements a tri-modal mechanism (assimilation, accommodation via schema evolution, and accommodation via schema creation) for updating memories, and a Retrieval Module designed as a ReAct AI agent (Yao et al., 2023). The memory is organized hierarchically into schemas containing meta (category), element (specific instance), element-value (attributes), and experience ID (linking to original conversation). The proposed method is evaluated on the LOCOMO benchmark and a newly introduced AggQA dataset using an LLM-as-a-judge metric, demonstrating performance improvements over current state-of-the-art systems.
weaknesses: **Presentation clarity**: Section 2 suffers from verbose, nested explanations that would benefit from more formal mathematical notation. Concepts introduced in the "Notation and Symbols" table (line 220) are insufficiently explained in the main text and Figure 3, such as multiple elements belonging to the same schema.

**Unclear attribution of contributions**: For the Retrieval Module (Section 2.4 and Figure 4), the distinction between components taken from Yao et al. (2023) and those newly introduced or adapted for PISA is not clearly delineated.

**Ad-hoc design concerns**: Throughout the paper, extensive notation is introduced (Memory Pool, Buckets, Schema, Element, etc.) without clarifying whether these are created specifically for this work or inspired by existing literature. The categorization of queries in Section 2.4 appears constructed to target specific task types found in the evaluation datasets (e.g., Regional Fact Retrieval -> LOCOMO's single-hop, Multi-Fragment Reasoning -> LOCOMO's multi-hop and temporal, Aggregation -> LOCOMO's open domain and AggQA).

**Evaluation metric inconsistencies**: Original publications of compared models (Section 3.2) for which LOCOMO evaluations exist consistently report F1 and BLEU scores, while this paper relies solely on LLM-as-a-judge evaluation. Only Chhikara et al. (2025) reported LLM-as-a-Judge scores. Notably, relative LLM-as-a-judge scores between models show near-zero correlation with scores reported by Chhikara et al. (e.g., for open-domain: Chhikara reported Zep=77, A-mem=54; this paper reports Zep=39, A-mem=50; for multi-hop: Chhikara reported Zep=41, LangMem=48; this paper reports Zep=14, LangMem=32). This suggests the metric is unreliable or should be complemented by additional metrics such as F1 or BLEU.

**Questionable importance of core mechanism**: Figure 5 demonstrates that changing $\theta_{meta}$ from 0.85 to 0.95 causes substantial behavioral changes (shift from assimilation to schema creation in Figure 5b), yet evaluation scores remain relatively stable (Figure 5a). This suggests the schema/bucket organization may not be critical for task performance.

**Missing computational efficiency analysis:** The authors never report computational performance metrics (e.g., inference time, memory consumption, number of LLM calls, retrieval latency) of PISA compared to baseline methods.

**Incomplete experimental analysis**: The $\theta_{meta}$ hyperparameter sweep is only performed on the LOCOMO dataset. Ablation studies are incomplete: PISA without Initialization is only evaluated on AggQA's medical domain, and PISA without Retrieval only on AggQA's finance domain.

**Minor**:
- Code is not currently accessible from the provided link.
- LOCOMO's adversarial questions are introduced in the main text but not used for evaluation.