From Knowledge to Noise: CTIM -Rover and the Pitfalls of Episodic Memory in Software Engineering Agents (2025)

Abstract: We introduce CTIM-Rover, an AI agent for Software Engineering (SE) built on top of AutoCodeRover (Zhang et al., 2024) that extends agentic reasoning frameworks with an episodic memory, more specifically, a general and repository-level Cross-Task-Instance Memory (CTIM). While existing open-source SE agents mostly rely on ReAct (Yao et al., 2023b), Reflexion (Shinn et al., 2023), or Code-Act (Wang et al., 2024), all of these reasoning and planning frameworks inefficiently discard their long-term memory after a single task instance. As repository-level understanding is pivotal for identifying all locations requiring a patch for fixing a bug, we hypothesize that SE is particularly well positioned to benefit from CTIM. For this, we build on the Experiential Learning (EL) approach ExpeL (Zhao et al., 2024), proposing a Mixture-Of-Experts (MoEs) inspired approach to create both a general-purpose and repository-level CTIM . We find that CTIM-Rover does not outperform AutoCodeRover in any configuration and thus conclude that neither ExpeL nor DoT-Bank (Lingam et al., 2024) scale to real-world SE problems. Our analysis indicates noise introduced by distracting CTIM items or exemplar trajectories as the likely source of the performance degradation.

## introduction ai agents have recently proven themselves as a competitive way of scaling test-time compute, especially in se @xcite . a crucial yet underexplored component of ai agents is their memory, which allows them to dynamically adapt their behavior based on prior experiences. early approaches, such as react @xcite , rely on the agent's immediate trajectory or short-term memory for decision-making. reflexion @xcite extends this by introducing long-term 1 https://github.com/liqs-v2/ctim-rover memory in the form of self-reflections on past failed task attempts, enabling agents to improve their reasoning and planning on a single task instance through in-context learning (icl). while this yields performance gains on the current task instance, reflexion discards these self-reflections after task completion. this results in inefficient use of computational resources and loss of valuable cross-task-instance learning opportunities. @xcite address this limitation through experiential learning (el), which is learning from past experiences across task instances. their approach expel achieves promising results on hot-potqa @xcite , webshop @xcite , and alfworld @xcite . to better align with existing terminology, we name the memory consisting of knowledge extracted with el "ctim". our work investigates whether ctim generalizes to the more complex foot_0 domain of se. we choose se because we expect el to be particularly valuable for uncovering the structure of a repository, reducing the number of turns taken exploring the codebase. to adapt el to se we extend it to a moes inspired knowledge distillation (kd) approach that simultaneously captures high-level se best practices and repository-specific details (e.g., project structure). we experimentally evaluate this approach by augmenting autocoderover @xcite with ctim, which we name "ctim-rover", and comparing the results of ctim-rover with those of the autocoderover on a subset of swe-bench verified. we find that our adapted ctim does not generalize to se and instead degrades performance in all configurations compared to autocoderover. our detailed qualitative analysis identifies noisy ctim items as culprits and we propose the use of embedding-based retrieval meth- @xcite . ctim-rover first gathers new experiences on the train set of swe-bench verified which we introduce in section 3 (details in appendix a). then, it combines these experiences with existing experiences of autocoderover @xcite on swe-bench lite @xcite . next, it distills high-level and repository-level knowledge from these experiences. during evaluation, it recalls a past experience and conditions on the distilled knowledge. key departures from expel or autocoderover in blue: (a) we extend autocoderover with reflexion @xcite , allowing the agent to retry an instance up to three times while learning from its mistakes through self-reflection. (b) compared to expel, we also source experiences from past successful trajectories outside our system. (c) we introduce a novel domain-specific knowledge distillation (kd) phase (figure 2 ) that extracts repository-level insights (e.g., common bug patterns). ## dataset we use swe-bench verified @xcite without samples from the pylint, astropy and pydata/xarray repositories due to environment setup issues 3 as basis for our experiments. for details see section 6. for our experiments, we rely on swe-bench verified, opposed to swebench @xcite , as it guarantees that samples are theoretically solvable @xcite . for the collection of past successful trajectories (section 3.1) we use 401 samples from this benchmark and for the evaluation 45 samples. ## experiments to adapt el to se, we extend the ctim with a moe @xcite inspired repositorylevel ctim (section 4.1) and investigate icl with successful, task-similar exemplar trajectories (section 4.2). for distilling knowledge from trajectories, we use the reasoning model o1 (openai, 2024b) because we suspect that its capabilities are beneficial when identifying pivotal agent decisions in complex se agent trajectories (i.e., cause-effect relationships). we use gpt-4o (openai, 2024a) to power the agent during training trajectory collection and the final evaluations due to budget constraints. ## results we evaluate ctim-rover's performance across the configurations listed in ## conclusion in this work we extend expel @xcite , which showed promising performance on hot-potqa @xcite , webshop @xcite , and alfworld @xcite to the se domain. we introduce a repository-specific ctim and investigate its performance on a subset of swe-bench verified @xcite . our results show that this simple el implementation does not generalize to the se setting and that the findings reported by @xcite and @xcite are thus limited to simpler environments with shorter trajectories. an extension of a general ctim with repository-level ctim or exemplar-based icl does not suffice to amend this. our investigations reveal noisy ctim items as the likely culprit. we suggest removing the maximum size constraint from ctim and instead focus on embedding-based retrieval of highly relevant ctim items with respect to the problem statement. furthermore, we suspect that the use of the ctim should not be limited to the initial context, and instead a subset of relevant items should be provided at each turn @xcite .