Memory, Benchmark & Robots: A Benchmark for Solving Complex Tasks with Reinforcement Learning
Authors: Egor Cherepanov
Nikita Kachaev
Alexey Kovalev
Aleksandr Panov
Venue: ICLR 2026 Poster (2026)
Keywords: Memory
Benchmark
Robots
POMDP
RL
Primary area: applications to robotics, autonomy, planning
TL;DR: A benchmark of 32 memory tasks for tabletop robotic manipulation, a benchmark to test the memory of an RL agent and classification of memory tasks in RL by type of memory usage

Abstract: Memory is crucial for enabling agents to tackle complex tasks with temporal and spatial dependencies. While many reinforcement learning (RL) algorithms incorporate memory, the field lacks a universal benchmark to assess an agent's memory capabilities across diverse scenarios. This gap is particularly evident in tabletop robotic manipulation, where memory is essential for solving tasks with partial observability and ensuring robust performance, yet no standardized benchmarks exist. To address this, we introduce MIKASA (Memory-Intensive Skills Assessment Suite for Agents), a comprehensive benchmark for memory RL, with three key contributions: (1) we propose a comprehensive classification framework for memory-intensive RL tasks, (2) we collect MIKASA-Base -- a unified benchmark that enables systematic evaluation of memory-enhanced agents across diverse scenarios, and (3) we develop MIKASA-Robo -- a novel benchmark of 32 carefully designed memory-intensive tasks that assess memory capabilities in tabletop robotic manipulation. Our work introduces a unified framework to advance memory RL research, enabling more robust systems for real-world use.

Published as a conference paper at ICLR 2026
MEMORY, BENCHMARK & ROBOTS: A BENCHMARK FOR SOLVING COMPLEX TASKS WITH REINFORCEMENT LEARNING
Egor Cherepanoy Nikita Kachaev 1.3 Alexey K. Kovaley1.2, Aleksandr Panov AXXX, MIRIAI, SITMO University Al Talent Hub
cherepanov@axxx.ter
ABSTRACT
Memory crucial for enabling agents to tackle complex tasks with temporal and spatial dependencies. While many reinforcement learning (RL) algorithms incorporate memory, the field lacks a universal benchmark to assess an agent's s memory capabilities across diverse scenarios. This gap is particularly evident in tabletop robotic manipulation, where memory is essential for solving tasks with partial observability and ensuring robust performance, yet standardized benchmarks exist. To address this, we introduce MIKASA "Memory-Intensee Skills Assessment Suite for Agents), a comprehensive benchmark for memory RL, with three key contributions: (I) we propose comprehensive classification RL tasks, (2) we collect MIKASA-Bas a
framework for memory-intensse unified benchmark that enables systematic evaluation of memory enhannedd agents across diverse scenarios, and (3) we develop MIKASA-Robo
a novel benchmark of 32 carefully designed memory-intensive tasks that assess memory capabilities in tabletop robotic manipulation. Our work introduces a unified framework to advance memory RL research, enabling more robust systems for real-world use. MIKASA is available at https: / //tinyuriirr com/ /wwbbmcchooooss.....
INTRODUCTION
Space of all memory tasks
My metrics are better in my memory better!
Many real-world problems involve partial observability (Kaclbling et al.,
BEFORE
Can't evaluate whose memory mechanism is better
Memory testing concoptually difforent tasks
1998), where an agent lacks full access to the environment's state.
But beat you in O so my memory better...
These tasks often include sequential decision-making (Chen et al., 2021),
delayed or sparse rewards, and long-
WITH BENCH
term information retention (Lampinen
et al., 2021; Parisotto et al., 2020). One approach to tackling these chal- lenges is to equip the agent with
Memory lesting single set
memory, allowing it to utilize histor-
memory!
ical information (Meng et al., 2021;
Ni et al., 2021). While there are well-established benchmarks in Natu- ral Language Processing (An et al.,
Standard memory benchmark
Separation by type memory ublizztion
2023; Bai et al., 2023), the evalu- ation of memory in reinforcement learning (RL) remains fragmented. Existing benchmarks, such as POP- Gym (Morad al., 2023), DMLab- 30 (Hung et al., 2018) and MemoryGyym (Pleines et al., 2023), focus on specific aspects of memory utilization, as they are designed around particular problem domains.
Figure 1: Systematic classification of problems with mem- ory in RL reveals distinct memory utilization patterns and enables objective evaluation of memory mechanisms across different agents.
"ip install mi mikasa- robo- sui te

---

Published as a conference paper at ICLR 2026
Table 1: MIKASA-Rober A benchmark comprising 32 memory-intensive robotic manipulation tasks across 12 categories. Each task varies in difficulty and configuration modes. The table specifies episode timeout (T), the necessary information that the agent must memorize in order to succeed (Oracle Info), and task instructions (Prompt) for each environment. See Appendix K for details.
Memorry Task
Mode
Brief description of the tack
Orack Info
Promge
Memooy
ShellGame
Touch
Menorize the position of the ball ofher some time being covered by the cups and then interact with the the bull under
Push
90
Objectt
Pick
Intercept
Stow
Menorize the positions the rolling bull, estimate velocity through
Medium
these positions, and then zim the ball the target
90
Latrcal_veiocity
Spanial
Frod
Gute
Skow
Memoize the posiiions the rolling ball, cslmate velucity through these positions, and then cutch the bull with the prigper and lift it
Medium
90
initial_weionity
Sponia
Fust
RotateLenient
PoaNag
Pos
Menorize the initial position of the peg and rouse it given angle
901
v_anole_dirr
carqot_Angla
Spucial
RotateStriet
Pus
PosNeg
without shitting its center
Menar ine the initaa posiiins of the and notate it ta given angle
90
y_anolc_dirr
carqet_inglo
Spanial
TukettBlck.vv
Memorize the initul position of the cube, move to the target region,
180
xys_iniria:
Sponia
and then return to its initial position
RemembarCalor
31519
Mamoria the cuke of the cube and clasee smong other oduns
60
true ceueeclarr... indiiee
Object
SememberSSape
3X519
Mamsiia the slage of the cake and chusse smong oiler slayes
6l1
brue shape, inditws
Objesi
AndCColor
RememberrShpe-
53
3x23 31
Menorize the shaps and coke of the cube und clomse unong other
stages and coloes
60
Erue_ trae__happes_ice info
truo_cciors_iice
Object
BundiOOCattor
3Y5
Remenber the colurs the set of cabes slown simulumnssuly in the 120
Ciuuity
banch and truch them any order
SegtfCCooors
Remomber the oslom, of the sot of cthos shows soquemially and then 120
true do or. iesi
Capusty
schast them in any ander
ClainOOCalors
3X5
Remember the colurs the sut of cubes shows saquenially and then 120
Frwe_eslor__nnee Indiows
Sequonid
sclect them in the same wader
Total: 32 tabletup rebotic manipulation memory intensive tasks in 12 groups
In contrast to classical RL, where benchmarks like Atari (Bellemare et al., 2013) and Mu- JoCo (Todorov et al., 2012) serve as universal standards, memory-enhanced agents are typically evaluated on custom environments developed alongside their proposals Table 2. This fragmented evaluation landscape obscures important performance variations across different memory tasks. For instance, an agent might excel at maintaining object attributes over extended periods while struggling with sequential recall challenges. Such task-specific strengths and limitations often remain hidden due to narrow evaluation scopes, underscoring the need for a comprehensive benchmark that spans diverse memory-intensine scenarios.
The challenge of memory evaluation becomes particularly evident in robotics. While some robotic tasks naturally involve partial observability, e.g. navigation tasks (Ai et al., 2022; Yadav al., 2023), many studies artificially create partially observable scenarios from Markov Decision Processes (MDPs) (Kaelbling al., 1996) by introducing observation noise or masking parts of the state space (Kurniawati, 2022: Lauri et al., 2023; Meng et al., 2021; Spaan, 2012). However, these approaches do not fully capture the complexity of real-world robotic challenges (Lauri et al., 2023), where tasks may require the agent to recall past object configurations, manipulate occluded objects, or perform multi-step procedures that depend heavily on memory. Such tasks include, for example, situations where service robot needs to memorize occluded objects (e.g., a plate hidden under towel) or where a home robot needs to accurately wipe the door of a microwave oven several times. Without memory, the robot wouldn't 't detect the plate in the first case, and in the second, it would wipe the door endlessly, unsure whether it has cleaned the area or if it's time to stop.
In this paper, we aim to address these challenges with the following four contributions:
1. Memory Tasks Classification. We propose simple yet comprehensive framework that organizes memory-intensinn tasks into four key categories. This structure enables system- atic evaluation without added complexity (Figure 1), offering a clear guide for selecting environments that reflect core memory challenges in RL and robotics (Section 4).
2. Memory-RL, Benchmark. We introduce MIKASA-Base a Gymnasium- based (Towers et al., 2024) framework for evaluating memory memory-enhaneed RL agents (Section 5).
3. Robotic Manipulation Tasks. We introduce MIKASA-Ron
open-source benchmark (MIT license) comprising 32 robotic tasks that target specific memory
skills in
realistic settings (Section 6). We evaluate it using popular Online RL baselines (Section 6.2) as well as Visual-Language-Action (VLA) models (Section 6.4). Guidelines for customizing environments and configuring time horizons are provided in Appendix M.
4. Robotic Manipulation Datasets. We release datasets with expert quality trajectories for all
32 MIKASA Robo memory-intenn -intesvv tasks to support Offline RL research (see Appendix C), and conduct extensive evaluations using a range of Offline RL baselines (Section 6.3).
2

---

Published as a conference paper at ICLR 2026
Table 2: Key memory-intensive environments from the reviewed studies for evaluating agent memory. The Atari (Bellemare et al., 2013) environment with frame stacking is included memory-enhanced agents are tested solely in MDP.
2 RELATED WORKS
Multiple RL benchmarks are designed to assess agents' memory capabilities. DMLab-30 (Hung ct al., 2018) pro- vides 3D navigation and puzzle tasks, focusing on long-horizon exploration and spatial recall. PsychLab (Leibo et al., 2018) extends DMLab by in- corporating tasks that probe cognitive processes, including working memory. MiniGrid and MiniWorld (Chevalier- Boisvert et al., 2023) emphasize par- tial observability in lightweight 2D
illustrate that many
Benchmark first introduced in the same work Benchmark is open-soureed.
u 2004
Owe0 clse cal 2000
mano Bu 2a0
Sit meseaneen 2009
pat at 2000
AD oo 2001
mane sas 2800
DELAM th coou
200
Nort N00
aut
Mar will FaresMaak
2
and 3D environments, while Mini- Hack (Samvelyan et al., 2021) builds on NetHack (Küttler et al.. 2020), offering small roguelike scenarios that require both short- and long- term memory. BabyAI (Chevalier-
Sale
JND
Boisvert et al., 2019) combines nat- ural language instructions with grid- based tasks, requiring memory for multi-step command execution. POP-
Okel Narg
Gym (Morad et al., 2023) standardizes memory evaluation with tasks rang- ing from pattern- pattern-matching puzzles to complex sequential decision-making.
MMa:
Sexicm Maiz
Acdis Amroilann Baskerl
BSuite (Osband et al., 2020) offers a suite of carefully designed experiments that test core RI. capa- bilities, including memory, through controlled tasks on exploration, credit assignment, and scalability, Memory Gym (Pleines et al., 2023) offers suite of 2D grid environments with partial observability, designed to benehmark memory capabilities in decision- maaiinnnm-annngaaaaa agents, including endless versions of tasks for evaluating memory over extremely long time intervals. Memory Maze (Pasukonis et al., 2022) presents 3D maze navigation tasks that require memory to solve efficiently.
While these benchmarks offer valuable insights into memory mechanisms, they generally focus on abstract puzzles or navigation tasks. However, none of them fully encompass the broad range of memory utilization scenarios an agent may encounter, and the tasks themselves often differ fundamentally across benchmarks, making direct comparison of memory-cnhanced agents difficult. In the robotics domain, memory requirements become particularly challenging due to the physical nature of manipulation tasks. Unlike abstract environments, robotic manipulation involves complex physical interactions and multi-step procedures demanding both spatial and temporal memory. Existing
benchmarks, while useful for diagnostic purposes, struggle to capture these domain-specific challenges. The physical control and object interaction inherent in manipulation tasks introduce additional complexities not addressed by traditional memory evaluation frameworks.
Efforts have been made to classify memory-intensee environments by specific attributes. For example, Ni et al. (2023) divides them into memory/credit assignment based on temporal horizons. Yue et al. (2024) proposes memory dependency pairs model how past events influence current decisions, aiding imitation learning in partially observable tasks. Cherepanov et al. (2024a) defines agent memory types: long-term vs. short-term (based on context length), and declarative vs. procedural (based on environments and episodes), and formalizes memory-intensive environments. Leibo et al. (2018) instead adapts tasks from cognitive psychology and psychophysics to evaluate agents on human cognitive benchmarks. While these classifications highlight aspects of memory, they overlook physical dimensions in robotics. The link between physical interaction and memory remains underexplored, motivating a framework for spatio-temporal memory in real-world tasks.
Concurrent with our work Fang et al. (2025) also proposed MemoryBench, a benchmark for memory- intensive manipulation consisting of only three tasks designed to access only one type of memory, spatial memory. This benchmark is based on RLBench (James et al., 2020), which does not allow efficient parallelization of training.
3

---

Published as a conference paper at ICLR 2026
Slep:
Slep:
Lueft
fangett
forgets balt
Reward:: 0.079
Reword: 1.000
RememberCollrre-v
SouchGameTouchh-00
RotateL-miennPPss-v0
Figure Illustration of demonstrative memory-intensive tasks execution from the proposed MIKASA- Robo benchmark. The ThellGameTTuue -v task requires the agent to memorize the ball's location under mugs and touch the correct one. In RemembezColorg-vo, the agent must memorize a cube's color and later select the matching one. In Rotatelenienttossseee the agent must rotate peg while keeping track of its previous rotations.
3 BACKGROUND
3.1 PARTIALLY OBSERVABLE MARKOV DECISION PROCESS
Partially Observable Markov Decision Process (POMDP) (Kaelbling et al., 1996) extend MDP to account for partial observability, where an agent observes only noisy or incomplete information about the true environments state. POMDP defined by a tuple (S, A, R, 22 7), where: S is the action space;
is the set of states representing the complete environment configuration; is the transition function defining the probability of reaching state
T(s'|s, a): S x s' from state s after taking action a: R(s, a) S x A + R is the reward function specifying the immediate reward for taking action observations; (o)s,
[0. -
state 8; 12 is the observation space containing all possible S x A Ω + [0. is the observation function defining the probability of observing o after taking action a and reaching state 8; E [0, 1) the discount factor determining the importance of future rewards. The objective is to find policy π that maximizes the expected discounted cumulative reward: E, -5== 7 R(s de)]. where a, ~ T-1101:) depends on the history of observations rather than the true state. Relying partial observations makes POMDPs harder to solve than MDPs.
3.2 MEMORY-INTENSE ENVIRONMENTS
Memory-intensive intensivee environment is environment where agents must leverage past experiences to make decisions, often in problems with long-term dependencies or delayed rewards. More formally, following Cherepanov
al. (2024a), memory-intensivve task is POMDP where there exists a correlation horizon & >> 1, representing the minimum number of timesteps between an event critical for decision-makee making and when that information must be recalled. Popular memory-intensive environments in RL are listed in Table 2. One way to solving memory-intensive environments is to augment agents with memory mechanisms (see Appendix F).
3.3 ROBOTIC TABLETOP MANIPULATION
Robotic tabletop manipulation (Shridhar et al., 2022) involves robots manipulating objects on flat surfaces through actions like grasping. pushing, and picking. While crucial for real-world appli- cations (Levine et al., 2018), most existing simulators treat these tasks as MDPs without memory requirements, failing to capture the spatio-temporal dependencies present in real scenarios. This limitation hinders the development of -ennory-enhanced agents for practical applications.
CLASSIFICATION OF MEMORY-INTENSIV TASKS
The evaluation of memory capabilities in RI. faces two major challenges. First, as shown in Table 2, research studies use different sets of environments with minimal overlap, making diffieult to compare memory-ente memory-enhanced agents across studies. Second, even within individual studies, benchmarks may focus on testing similar memory aspects (e.g., remembering object locations) while neglecting others (e.g., reconstructing sequential events), leading to incomplete evaluation of agents' memory.
Different architectures may exhibit varying performance across memory tasks. For instance, an architecture optimized for long-term object property recall might struggle with sequential memory tasks, yet these limitations often remain undetected due to the narrow focus existing evaluation approaches.
4

---

Published as a conference paper at ICLR 2026
Agent Memory Tasks
MIKASA
Human Memory Tasks
Too simple to capture the full range of real world problems
Concise succinct
Too sophieticated to be formulated as RL problems
Figure 3: MIKASA bridges the gap between human-like memory complexity and RL agents require- ments. While agents tasks don't 't require the full spectrum of human memory capabilities, they can't t be reduced to simple spatio-temporal dependencies. MIKASA provides a balanced framework that captures essential memory aspects for agents tasks while maintaining practical simplicity.
To address these challenges, we propose a systematic approach to memory evaluation in RL. Draw- ing from established research in developmental psychology and cognitive science, where similar memory challenges have been extensively studied in humans, we develop a categorization framework consisting of four distinct memory task classes, detailed in Section 4.2.
4.1 MEMORY: FROM COGNITIVE SCIENCE TO RL
In developmental psychology and cognitive science, memory is classified into categories based on cognitive processes. Key concepts include object permanence (Piaget, 1952), which involves remem- bering the existence of objects out of sight, and categorical perception (Liberman et al., 1957), where objects grouped based on attributes like color or shape. Working memory (Baddeley, 1992) and memory span (Daneman & Carpenter, 1980) refer to the ability to hold and manipulate information over time, while causal reasoning (Kuhn, 2012) and transitive inference (Heckers al., 2004) involve understanding cause-and-effect relationships and deducing hidden relationships, respectively.
The RL field has attempted utilize these concepts in the design of specific memory- intensive environments Fortunato et al. (2020); Lampinen et al. (2021), but these have been limited at the task design level. Of particular interest, however, is how existing memory-intensive tasks can be categorized using these concepts to develop a benchmark memory capabilities of memory-cnhanced agents, and it is this problem that we address in this paper. Thus, aim to provide balanced framework that covers important aspects of memory for real-world applications while maintaining practical simplicity (see Figure 3).
which to test the greatest number of
4.2 TAXONOMY OF MEMORY TASKS
We introduce a comprehensive task classification framework for evaluating memory mechanisms in RL. Our framework categorizes inneoyeintensive tasks into four fundamental types, each targeting distinct aspects of memory capabilities:
Object Memory. Tasks that evaluate an agent's ability to maintain object-related information over time, particularly when objects become temporarily unobservable. These tasks align with the cognitive concept of object permanence, requiring agents to track object properties when occluded, maintain object state representations, and recognize encountered objects. Example: a robot remembers which fruit it put in the fridge.
2. Spatial Memory, Tasks focused on environmental awareness and navigation, where agents must remember object locations, maintain mental maps of environment layouts, and navigate based on previously observed spatial information. Example: the robot remembers the position of mug moved while cleaning and returns it to its place.
3. Sequential Memory. Tasks that test an agent's ability to process and utilize temporally ordered information, similar to human serial recall and working memory. These tasks require remembering action sequences, maintaining order-dependent information, and using past decisions to inform future actions. Example: a robot memorizes the order the ingredients it has added to soup.
4. Memory Capacity. Tasks that challenge an agent's ability to manage multiple pieces information simultancously, analogous to human memory span. These tasks evaluate information retention limits and multi-task information processing, Example: robot is able to memorize the positions of several different objects while cleaning a table.
This classification framework enables systematic evaluation of memory-enhanced RL agents across diverse scenarios. By providing structured approach memory task categorization, establish
5

---

Published as a conference paper at ICLR 2026
foundation for comprehensive benchmarking that spans the wide spectrum of memory requirements. In the following section, we present a carefully curated set of tasks based on this classification, forming the basis of our proposed MIKASA benchmark.
Table 3: Analysis of established robotics frameworks with ma- nipulation tasks, comparing their support for memory-intensive tasks. f excluding Franka Kitchen. * concurrent work with three memory tasks with only one type memory.
5 MIKASA-BASE
Motivation and Overview. Despite the importance of mem- ory in decision-making, the RI. community lacks standardized tools for benchmarking memory capabilities. Existing studies typically introduce bespoke environments tailored to their proposed algorithms, leading to fragmentation and limited comparability across works (see Table 2). Moreover, many popular memory benchmarks focus narrowly on specific memory types, overlooking the diversity of memory demands found in real-world applica- tions. To address this gap, we introduce MIKASA-Base, a unified benchmark that consol- idates widely used open-source memory-intensiee environments under common Gym-like API. Our goal is streamline reproducibility, support fair comparisons, and promote systematic evaluation of memory in RL.
Roboties Framework with Manipulation Tasks
Memory Tasks
Manipulation
Atomic
Low-Ievel
actions
MIKASA-Rebo
Memory MemeryBench Beach" (Fang et al., 2025)
ManiSkillS
al,, 2024)
ManiSkill HAB (Shukla el 2024)
X
FetchBench (Han
RobeCasa (Nasiriany et 2024)
X
CommasiummRoohoteee (de Tazcano al., 2024)
BEIAVVIO 1K (1.i al., 2024)
(Liu al., 2023)
ARNOLD (Gong et at., 2023)
X
Lovenavens (Zhang at, 2023)
iGihson 2.0 (Li al., 2022)
VIMA (Jiang
2022)
Issac Sim (Makoviycbuk et al. 2021)
panda-gym (GallonSdee et 2021)
Ravens (Zeng al., 2021)
Habitat 2.0 (Szot et 2021)
at.,
CausalWord (Ahmed al.,
RI Rench (James al., 2020)
rhoosuite (Zhu et al., 2020h)
dm control (Tunyasuvunakodl et 2020)
Franka Kitchea (Gupta et al., 2019)
SURREAL (Pan et al., 2018)
(Kolve 2017)
Benchmark Design Principles. MIKASA-Bas is designed around core principles that support rigorous and interpretable evaluation of memory in RL. To disentangle memory from unrelated challenges, we organize tasks into two tiers. The first tier consists of diagnostic vector- based environments that isolate specific memory mechanisms. The second tier includes complex image- based tasks that incorporate realistic perception challenges, thus more closely resembling real- world settings. This hierarchical structure enables researchers to validate memory capabilities incrementally from atomic reasoning to nigh-dimensional sensory input.
Task Classification and Selection. Building reviewed open-source memory benchmarks and categorized their tasks into four distinct types of memory usage. We selected diverse yet representative subset of environments to cover this taxonomy - -ranging from object permanence to sequential planning. All selected tasks are unified under a single, consistent API. Descriptions are provided in Appendix L., and overview of appears in Table 9. This consolidation supports architectural ablations, direct comparison of methods, and simplified evaluation pipelines. Implementation details can be found in Appendix E.
taxonomy from Section 4.2, systematically
tasks
MIKASA-Bast provides the first systematic and unified benchmark for evaluating memory in RL. It mitigates fragmentation by standardizing task access and evaluation, and its structured progression enables precise attribution of memory-related agent failures. By covering a broad spectrum of memory challenges within a common framework, MIKASA-Bass -ass offers a foundation for robust, reproducible research in memory-centric RI.
6 MIKASA-ROBO
The landscape of robotic manipulation frameworks reveals significant limitations in addressing memory-intensive tasks. While partial observability is well-studied in navigation, manipulation

---

Published as a conference paper at ICLR 2026
ShelGammTo.uch
RememberColen
Re
RemenberrCooo500
1.00
1400
1000
2 6.755
0.75
0.70
0.76
0.76
$0.00
2.00
0.50
30.25
0.26
0.25
0.25
0.00
Trainig Blozs 107
Training Stops
Traniig Stoza 100
2
0mmg
Inaving Stape 166
0m0
Inaning Stape 166
0.0g
tearing Stape 1e6
Figure 4: Performance of PPO-MLP trained in state mode, i.e., in MDP mode without the need for memory. These results suggest that the proposed tasks are inherently solvable with a success rate of 100%.
Figure 5: Online RL baselines with MLP and LSTM backbones trained in RGB+jjinng mode on the remmmbeeroolo--ee ment with dense rewards. Both architectures fail to solve medium and high complexity tasks.
environ-
scenarios are still predominantly evaluated under full observability, with limited focus on memory demands (see Table 3). Among frameworks that do consider memory, BEHAVIOR-IK (Li al., 2024) and iGibson 2.0 (Li et al., 2022) include highly complex, non-atomic tasks, which obscure the evaluation of specific memory mechanisms. VIMA (Jiang al., 2022) relies on high-level action abstractions, limiting temporal memory assessment. To address these gaps, we introduce MIKASA-Robo a benchmark specifically designed to evaluate diverse memory skills in robotic manipulation through well-isolated, fine-grained tasks.
Concurrently with our work, Fang et al. (2025) proposed MemoryBench, benchmark focused on spatial memory with three robotic tasks. In contrast, MIKASA-Rokd spans four memory categories and 32 tasks, enabling broader and more systematic evaluation of memory mechanisms in RL agents.
MIKASA-Robh Robo is benchmark designed for memory-intensive robotic tabletop manipulation tasks, simulating real-world challenges commonly encountered by robots. These tasks include locating occluded objects, recalling previous configurations, and executing complex sequences of actions over extended time horizons. By incorporating meaningful partial observability, this framework offers a systematic approach to test agent's memory mechanisms.
Building upon the robust foundation of ManiSkil13 framework (Tao et al., 2024), our benchmark leverages its efficient parallel GPU-based training capabilities to create and evaluate these tasks.
6.1 MIKASA-ROBO MANIFESTATION
In designing the tasks, we drew inspiration from the four memory types identified in our classifi- cation framework (Section 4.2). We developed 32 tasks across 12 categories of robotic tabletop manipulation, each targeting specific aspects of object memory, spatial memory, sequential memory, and memory capacity. These tasks feature varying levels of complexity, allowing for systematic evaluation of different memory mechanisms. For instance, some tasks test object permanence by requiring the agent to track occluded objects, while others challenge sequential memory by requiring the reproduction of strict order of actions. A summary of these tasks and their corresponding memory types is provided in Table 1, with detailed descriptions in Appendix K. Information on task customization, including adjustments of time horizons and environment parameters, can be found in Appendix M.
To illustrate the concept of our memory-intensive framework, we present Shel TouchhhamTTouhh--- vv, RememberColor
and Rot task, the agent observes red ball placed in one of three positions over the first 5 steps (t E 4). At f = 5, the ball and the three positions are covered by mugs. The agent must then determine the location of the ball by interacting with the correct mug. In the simplest mode (Touch), the agent only needs to touch the correct mug, whereas in other modes, it must either push or lift the mug. In the RememberColss or task, the agent observes a cube of a specific color for 5 steps (t E [0.4] 4)). After the cube disappears for 5 steps, 3. 5, or 9 (depending
tasks in Figure 2. In the
ShellGameTouchh
task mode) cubes of different colors appear at = 10. The agent's task is to identify and select the same cube it initially task, the agent must rotate randomly oriented peg by
saw. In the Rot at specified clockwise angle.
The --EAAA--Robooo benchmark offers multiple training modes: state (complete vector information including oracle data and Tool Center Point (TCP) pose). RGB (top-view and gripper-camme cameea images with TCP position), joi ints (joint states and TCP pose), oracle (lask-specific environment data
7

---

Published as a conference paper at ICLR 2026
for debugging), and prompt (static task instructions). While any mode combination is possible, RGB+joints serves as the standard memory testing configuration, with state mode reserved for MDP-MDsae tasks.
The MIKASA -oobo benchmark implements two types of reward functions: dense and sparse. The dense reward provides continuous feedback based the agent's progress towards the goal, while the sparse reward only signals task completion. While dense rewards facilitate faster learning in our experiments, sparse rewards better reflect real-world scenarios where intermediate feedback is often unavailable, making them crucial for evaluating practical applicability of memory-enhanced agents.
6.2 ONLINE RL BASELINES
nembe Shapes famantyrdrapae penembe Shepe3
Rort Demambertin woremberiitggeeettteeeeo
Ramentartittrn
For the experimental eval- uation, chose on-policy Proximal Policy Optimiza- tion (PPO, (Schulman et al., 2017)) with two underlying architectures: Multilayer Perceptron (MLP) and Long Short-Term Memory (LSTM, (Hochreiter & Schmidhuber, 1997)), as well popular in robotics off-policy Soft Actor-Critic (SAC, (Haarnoja et al., 2018)) and model-based Temporal Difference Learning for Model Predictive Control (TD- MPC2, (Hansen et al., 2024)).
na mar www.psaaaCCorrr33...
pamenberCaker
Object Memory
Intersegrsione
GralGamatio
Inkercaptrled
The Kamaehan
Neteroest
one Rommhouchh
Sequential Memory
Memory
Spatial Memory
Capacity
supprcaent
Sepprcoknn
Kotate@nriciappleg
SunchofClles?
Sundrofcsoos
RATE
The MLP variant serves as memory-less base- line, while LSTM rep-
Figure 6: Results of Offline RL baselines with memory (RATE, DT) and without memory (BC-MLP, CQL-MLP, DP) on all 32 MIKASA- Robo tasks. Training was performed in RGB mode with sparse rewards (success condition).
resents widely-adopted memory mechanism in RL,
known for effectiveness in solving POMDPs (Ni et al., 2021). This choice of architectures en- ables direct comparison between memory- and memory memory-enhancel enhanced agents while validating benchmark's 's ability to assess memory. We focus specifically on these fundamental architectures they align with our primary goal of benchmark validation rather than comprehensive algorithm comparison. To demonstrate that all proposed environments are solvable with 100% success rate (SR), we trained PPO-MLP agent using state mode, where it had full access system information. Results for select tasks are shown in Figure 4; full results are in Appendix G.
Training under the RGB+joing our tasks. Using the Remembe CColor-vCroorrrr task as an example, PPO-LSTM demonstrates superior performance compared to PPO-MLP when distinguishing between three colors (see Figure 5). However, both agents' success rates drop dramatically to near-zero as the task complexity increases to five or nine colors. Moreover, under sparse reward conditions, both architectures fail to solve even the three-color variant (see Appendix G, Figure 13). Additionally. our findings indicate that, while SAC and TD-MPC2 exhibit higher sample efficiency compared to PPO-MLP, when faced with more complex challenges, the lack of an explicit memory mechanism becomes critical shortcoming, resulting in low performance, which also emphasizes the inappropriateness of algorithms common in the robotics community for memory-intensiv tasks. These results validate benchmark's s effectiveness in evaluating agents' memory, showing clear performance degradation as memory demands increase.
mode with dense rewards reveals the memory-intensive nature of
8

---

Published as a conference paper at ICLR 2026
Table 4: Performance of VLA models on selected memory-intensive tasks from the MIKASA- Robo benchmark. Reported values denote average success rates over 100 evaluation episodes (mean + sem). Tasks include spatial reasoning (She 1 GameTouch, InterceptMedi um) and color-based memory retrieval (Rememberroloor3/5//
Model
SheliGameTouch
InterceptMedium
RemembezColor3
RemenberColor5
RemenberColor9
Octo-small
0.46 0.05
0.39 = 0.04
0 = 0.06
0.17 0.03
0.11 0.03
OpeaVLA (K=()
0.12 + 0,05
0,06 = 0,02
0.21 = 0.00
0.09 0.02
0.08 0.02
OpenVI.A (K
0.47
0.14
0.59 0.04
0.03
0.02
SpatiaVLLA (K -4)
0.23 1. 0.04
0.27 1 0.04
0.27 1. 0.05
0.17 0.03
0.11 0.03
no (K=1)
0.33
0.42
0.35
0.22 0.04
0.02
6.3 OFFLINE RL BASELINES
Since dense rewards are typically not available in the real world, is of particular interest train on sparse rewards represented as a binary flag of a successfully completed episode. Whereas models with online learning are extremely hard to handle in this setting, we also conducted experiments with five Offline RI. models: Decision Transformer (DT) (Chen et al., 2021)) and Recurrent Action Transformer with Memory (RATE) (Cherepanov al., 2024b)) based on the Transformer architecture, Standard Behavioral Cloning (BC) and Conservative Q-Learning (CQL) (Kumar et al., 2020)) with MLP backbones, as well as Diffusion Policy (DP) (Chi et al., 2023)) a recent and popular approach in robotic manipulation that leverages diffusion models for direct action prediction.
Experimental results with Offline RL models trained using two RGB camera views and sparse rewards are presented in Figure 6. As can be seen from Figure 6, none of the models - including those explicitly designed for sequence modeling - were able to successfully solve the majority MIKASA- Robo tasks, demonstrating the challenge posed by the benchmark. Training was performed using datasets consisting of 1000 successful trajectories per task, obtained by using PPO with oraele-level information about the environment (see details in Appendix C).
Notably, none of the evaluated models were able to solve tasks requiring high Memory Capacity or Sequential Memory, further underscoring their complexity. More detailed results for Offline RL algorithms are presented in Appendix, Table
6.4 VLA BASELINES
To investigate the capabilities of state-of-the-ar Visual-Language-Actton (VLA) models in memory- intensive robotic tasks, we selected four representative baselines: Octo (Team et al., 2024), Open- VLA (Kim et al., 2024), To (Black al., 2024), and SpatialVLA (Qu et al., 2025). Although none claims to implement sophisticated memory mechanisms, the experiments offer insights into existing memory capabilities in VLA models.
Octo is a transformer with diffusion heads pretrained on Open X-Embodiment (Collaboration et al., 2025); we fine-tuned only the readout heads, using the full pretrained context length of 10 and action chunk size (K=4). Open VLA uses Prismatic-7B backbone (Karamcheti et al., 2024), fine-tuned with LoRA adapters, chunking, and L1 loss (Kim et al., 2025). mo combines a pretrained VLM with lightweight wow-matching expert. SpatialVLA augments VLA with egocentric 3D position encodings and discretized action grids. We evaluate chunk sizes K=4 and K=8. All models were trained on 250 expert trajectories per task, using 128 128 RGB image pairs (base and wrist views) and end-effector control (see Appendix D).
Experimental results (Table 4) reveal notable trends. Octo (context size 10) outperforms random on simpler tasks, suggesting some innate memory capacity, but degrades with complexity. indicating limited scalability. Open VLA behaves differently across chunk sizes: with K 8. it exceeds ran- dom on tasks like RememberColor 3 and Shell SthellGame GaameTochh despite lacking step-wise history. However, performance drops on harder tasks. With K = 4. VLA, SpatialVLA, and mo fail across the board, showing random-like performance. These results suggest larger chunks can bypass memory by generating full trajectories from early cues, but this fails with smaller chunks, where initially correct actions often collapse into confusion. Thus, chunking offers limited compensation for lack of memory. We also conducted real-world experiments using the To.5 model (Black et al., 2025) (see Appendix N), where the empirical evidence further corroborates the central conclusion that modern VLA models lack the ability to retain rask-relevant information over long temporal horizons. The results reproduce the same pattern observed in simulation, with strong performance on fully observable tasks and a sharp degradation once the task introduces an occluded interval that requires persistent memory.

---

Published as a conference paper at ICLR 2026
The sharp decline harder tasks underscores the need for dedicated memory architectures and vali- dates the multi-difficulty hierarchy in MIKASA-Robo to prevent such "shortcuts." " Our experiments with Octo, Open VLA, To. and SpatialVLA LA highlight critical gap in current VLA models: without effective long-term memory, performance is brittle on tasks requiring strong memory. These findings reveal current limitations and reinforce the value of the MIKASA-Robo benchmark.
7 LIMITATIONS AND FUTURE WORK
Future work could explore more extensive adaptation of large VLA models within MIKASA to obtain a clearer picture of their memory capabilities. A complementary direction is to broaden the benchmark to encompass memory phenomena that fall outside the current focus spatio-tempore dependencies within single episodes. The present tasks do not capture higher-level processes such as meta-RL. These settings require problem formulations that extend beyond the POMDP structure instantiated in the benchmark, since they involve reasoning over distributions of tasks rather than isolated trajectories. would also be valuable to study how agents cope with spurious correlations, interference between stored representations, and the dynamics of memory overwriting or forgetting under limited memory capacity, all of which remain unexplored in the current framework. Finally, developing of additional evaluation metrics, instead of relying solely success rates returns, offers a promising path toward more precise and discriminative assessment of memory mechanisms.
CONCLUSION
We present MIKASA, a unified benchmark suite for evaluating memory in RL. Our work addresses key gaps in the field by introducing: (1) a taxonomy of memory types, spanning object, spatial, sequential, and capacity requirements; (2) MIKASA-Base, standardized collection of open-source memory tasks; (3) MIKASA-Robo,
suite of 32 robotic manipulation tasks designed to isolate and stress specific forms of memory; and (4) accompanying offline datasets to support reproducible large-scale evaluation. Experiments with online, offline, and VLA agents consistently show that existing methods struggle when memory requirements become substantial, which highlights the need for architectures with explicit and robust long-horizon memory mechanisms. We further validated our findings through real-world experiments on a physical robot platform. The results mirror the simulation hierarchy, with reliable performance in fully observable tasks, partial degradation under dynamic but non-occluded conditions, and failure when long-horizon occlusion is introduced. This consistency confirms that memory, rather than embodiment or perception noise, is the primary limitation. MIKASA is intended to guide and accelerate progress in memory-intensin RL for real-world applications. The MIKASA-Robo suite open-soure under the MIT license and can be conveniently installed via pip install mi kasa- robo-suite
ACKNOWLEDGEMENT
We thank Nuraddin Kerimov for providing the S0-101 robotic arm and for his extensive assistance with its setup and the execution of the real-world experiments. We also thank Andrei Spiridonov for his help with the robot and for valuable technical consultations during the course of this work.
REPRODUCIBILITY STATEMENT
We have taken several measures ensure the reproducibility of work. All benchmark tasks, including MIKASA- -ASe and MIKASA-Robo are publicly released under the MIT license with complete code and installation instructions (see Appendix E and Appendix A) (we will publish the datasets after the rebuttal period). Detailed environment descriptions and task customization guides are provided in Appendix K and Appendix M. For experiments, we report training and evaluation protocols in Appendix I, including random seeds, number of runs, and hardware setup. Offline datasets used in our experiments are released in processed form together with scripts for collection and preprocessing (see Appendix C). Architectural and algorithmic details for all online, offline, and VLA baselines are described in Section 6.2, Section 6.3, and Section 6.4. All additional implementation details and hyperparameters are provided in the Appendix. An anonymous repository containing the source code, dataset download links, and pretrained models will be submitted with the supplementary materials to facilitate full reproducibility of our results.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Ossama Ahmed, Frederik Träuble, Anirudh Goyal, Alexander Neitz, Yoshua Bengio, Bernhard Schölkopf, Manuel Wüthrich, and Stefan Bauer. Causalworld: A robotic manipulation benchmark for causal structure and transfer learning, arXiv preprint arXiv:2010.04296, 2020.
Bo Ai, Wei Gao, David Hsu, al. Deep visual navigation under partial observability. In 2022 International Conference on Robotics and Automation (ICRA), pp. 9439-9446. IEEE, 2022.
Chenxin An, Shansan Gong, Ming Zhong, Xingjian Zhao, Mukai Li, Jun Zhang, Lingpeng Kong, and Xipeng Qiu. L-eval: Instituting standardized evaluation for long context language models, arXiv preprint arXiv:2307.11088 2023.
Alan Baddeley. Working memory. Science, 255(5044):556-559 1992.
Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu. Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, et al. Longbench: A bilingual, multitask benchmark for long context understanding. arXiv preprint arXiv:2308. /4508, 2023.
Andrew Barto, Richard Sutton, and Charles W. Anderson. Neuronlike adaptive elements that can solve difficult learning control problems. IEEE Transactions Systems, Man, and Cybernetics, SMCC-135):834-888 -483344446.111 1983, doi: 10. 10911SMMC..... 1987.63377777
Marc G Bellemare, Yavar Naddaf, Joel Veness, and Michael Bowling. The arcade learning environ- ment: An evaluation platform for general agents. Journal of Artificial Intelligence Research, 47: 253-279, 2013.
Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Lucy Xiaoyang Shi, James Tanner, Quan Vuong, Anna Walling, Haohuan Wang, and Ury Zhilinsky, To A vision-language- action flow model for general robot control, 2024. URL https://arxiv. /ttp///axvvvv////////////////.va org. abs///////// /2410. 24164.
Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Robert Equi, Chelsea Finn, Niccolo Fusai, Manuel Y. Galliker, Dibya Ghosh, Lachy Groom, Karol Hausman, brian ichter, Szymon Jakubezak, Tim Jones, Liyiming Ke, Devin LeBlanc, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Allen Z. Ren, Lucy Xiaoyang Shi, Laura Smith, Jost Tobias Springenberg. Kyle Stachowicz, James Tan- ner, Quan Vuong, Homer Walke, Anna Walling, Haohuan Wang. Lili Yu, and Ury Zhilinsky. To.s: a vision- laan--lannaggaatttion model with open-world generalization. Song, and Hae-Won Park (eds.), Proceedings of The 9th Conference on Robot Learning, volume 305 of Proceedings of Machine Learning Research, pp. 17-40. PMLR, 27-30 Sep 2025. URL ht hittps:/ https://ppcceeiingee
Joseph Lim, Shuran
mlr pre.ssresss.... hiteess/v305// http///3l/bbbbba5eeee htt.cll55....ll.......m.m
Remi Cadene, Simon Alibert, Alexander Soare, Quentin Gallouedec, Adil Zouitine, Steven Palma, Pepijn Kooijmans, Michel Aractingi, Mustafa Shukor, Dana Aubakirova, Martino Russi, Francesco Capuano, Caroline Pascal, Jade Choghari, Jess Moss, and Thomas Wolf. Lerobot: State-of-the-art machine learning for real- world robotics in pytorch. https:://// lerobot, 2024.
setps://githhe /ttp///gtttbb////// www.buggiiggaacce/
Berk Calli, Aaron Walsman, Arjun Singh, Siddhartha Srinivasa, Pieter Abbeel, and Aaron M Dollar. Benchmarking in manipulation research: The ycb object and model set and benchmarking protocols. arXiv preprint arXiv: 11X.:1150220313333 2015.
Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Misha Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch. Decision transformer: Reinforcement learning via sequence modeling. Advances in newral information processing systems, 14:1544--50997, 2021.
Egor Cherepanov, Nikita Kachaev, Artem Zholus, Alexey K. Kovalev, and Aleksandr I. Panov. Unraveling the complexity of memory in rl agents: an approach for classification and evaluation, 2024a. URL https: / //axxiv orgg 12212..44 06531.
11

---

Published as a conference paper at ICLR 2026
Egor Cherepanov, Alexey Staroverov, Dmitry Yudin, Alexey K. Kovalev, and Aleksandr I. Panov. Recurrent action transformer with memory, arXiv preprint arXiv:2306.09459, 2024b. URL
https://arxiv: htrggabb/23000 09459.
Maxime Chevalier-Boisvert, Dzmitry Bahdanau, Salem Lahlou, Lucas Willems, Chitwan Saharia, Thien Huu Nguyen, and Yoshua Bengio. Babyai: A platform to study the sample efficiency of grounded language learning, 2019. URL https::/p//... //ap/vaarvvvva//aa www.wiowo.g..a..oaoooo.oooooooor /aas//b///a / 81 1. 08272.
Maxime Chevalier-Boisvert,, Bolun Dai, Mark Towers, Rodrigo de Lazcano, Lucas Willems, Salem Lahlou, Suman Pal. Pablo Samuel Castro, and Jordan Terry. Minigrid & miniworid: Modular & customizable reinforcement learning environments for goal-oriented tasks, 2023. URL https:
wtw.xii.orggabb/22302 .00.336663333.. 13831.
Cheng Chi, Zhenjia Xu, Siyuan Feng. Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, pp. 02783649241273668, 2023.
Junyoung Chung. Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv: 2014.
Embodiment Collaboration, Abby O' Abdul Rehman, Abhinav Gupta, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, Albert Tung, Alex Bewley, Alex Herzog, Alex Irpan, Alexander Khazatsky, Anant Rai, Anchit Gupta, Andrew Wang, Andrey Kolobov, Anikait Singh, Animesh Garg, Aniruddha Kembhavi, Annie Xie, Anthony Brohan, Antonin Raffin, Archit Sharma, Arefeh Yavary, Arhan Jain, Ashwin Balakrishna, Ayzaan Wahid, Ben Burgess-Limerics
Beomjoon Kim, Bernhard Schölkopf, Blake Wulfe. Brian Ichter, Cewu Lu, Charles Xu, Charlotte Le. Chelsea Finn, Chen Wang, Chenfeng Xu, Cheng Chi, Chenguang Huang, Christine Chan, Christopher Agia, Chuer Pan, Chuyuan Fu, Coline Devin, Danfei Xu, Daniel Morton, Danny Driess, Daphne Chen, Deepak Pathak, Dhruv Shah, Dieter Büchler, Dinesh Jayaraman, Dmitry Kalashnikov, Dorsa Sadigh, Edward Johns, Ethan Foster, Fangchen Liu, Federico Ceola, Fei Xia, Feiyu Zhao, Felipe Vieira Frujeri, Freek Stulp, Gaoyue Zhou, Gaurav S. Sukhatme, Gautam Salhotra, Ge Yan, Gilbert Feng, Giulio Schiavi, Glen Berseth, Gregory Kahn, Guangwen Yang, Guanzhi Wang. Hao Su, Hao-Shu Fang, Haochen Shi, Henghui Bao, Heni Ben Amor, Henrik Christensen, Hiroki Furuta, Homanga Bharadhwaj. Homer Walke, Hongjie Fang, Huy Ha, Igor Mordatch, Ilija Radosavovic, Isabel Leal, Jacky Liang, Jad Abou-Chakra, Jaehyung Kim, Jaimyn Drake, Jan Peters, Jan Schneider, Jasmine Hsu, Jay Vakil, Jeannette Bohg, Jeffrey Bingham, Jeffrey Wu, Jensen Gao, Jiaheng Hu, Jiajun Wu, Jialin Wu, Jiankai Sun, Jianlan Luo, Jiayuan Gu, Jie Tan, Jihoon Oh, Jimmy Wu, Jingpei Lu, Jingyun Yang, Jitendra Malik, João Silvério, Joey Hejna, Jonathan Booher, Jonathan Tompson, Jonathan Yang, Jordi Salvador, Joseph J. Lim, Junhyek Han, Kaiyuan Wang, Kanishka Rao, Karl Pertsch, Karol Hausman, Keegan Go. Keerthana Gopalakrishnan, Ken Goldberg, Kendra Byrne, Kenneth Oslund, Kento Kawaharazuka, Kevin Black, Kevin Lin, Kevin Zhang, Kiana Ehsani, Kiran Lekkala, Kirsty Ellis, Krishan Rana, Krishnan Srinivasan, Kuan Fang, Kunal Pratap Singh, Kuo-Hao Zeng, Kyle Hatch, Kyle Hsu, Laurent Itti. Lawrence Yunliang Chen, Lerrel Pinto, Li Fei-Fei, Liam Tan, Linxi "Jim" Fan, Lionel Ott, Lisa Lee, Luca Weihs, Magnum Chen, Marion Lepert, Marius Memmel, Masayoshi Tomizuka, Masha Itkina, Mateo Guaman Castro, Max Spero, Maximilian Du, Michael Ahn, Michael C. Yip, Mingtong Zhang, Mingyu Ding, Minho Heo, Mohan Kumar Srirama, Mohit Sharma, Moo Jin Kim, Muhammad Zubair Irshad, Naoaki Kanazawa, Nicklas Hansen, Nicolas Heess, Nikhil J Joshi, Niko Suenderhauf, Ning Liu, Norman Di Palo, Nur Muhammad Mahi Shafiullah, Oier Mees, Oliver Kroemer, Osbert Bastani, Pannag R Sanketi, Patrick "Tree" Miller, Patrick Yin. Paul Wohlhart, Peng Xu, Peter David Fagan, Peter Mitrano, Pierre Sermanet. Pieter Abbeel, Priya Sundaresan, Qiuyu Chen, Quan Vuong, Rafael Rafailov, Ran Tian, Ria Doshi, Roberto Martin- Martin, Rohan Baijal, Rosario Scalise, Rose Hendríx, Roy Lin, Runjia Qian, Ruohan Zhang. Russell Mendonca, Rutav Shah, Ryan Hoque, Ryan Julian, Samuel Bustamante, Sean Kirmani, Sergey Levine, Shan Lin, Sherry Moore, Shikhar Bahl, Shivin Dass, Shubham Sonawani, Shubham Tulsiani, Shuran Song, Sichun Xu, Siddhant Haldar, Siddharth Karamcheti, Simeon Adebola, Simon Guist, Soroush Nasiriany, Stefan Schaal, Stefan Welker, Stephen Tian, Subramanian Ramamoorthy, Sudeep Dasari, Suncel Belkhale, Sungjae Park, Suraj Nair, Suvir Mirchandani, Takayuki Osa, Tanmay Gupta, Tatsuya Harada, Tatsuya Matsushima, Ted Xiao, Thomas Kollar, Tianhe Yu, Tianli Ding, Todor Davchev, Tony Z. Zhao,
12

---

Published as a conference paper at ICLR 2026
Travis Armstrong, Trevor Darrell, Trinity Chung, Vidhi Jain, Vikash Kumar, Vincent Vanhoucke, Vitor Guizilini, Wei Zhan, Wenxuan Zhou, Wolfram Burgard, Xi Chen, Xiangyu Chen, Xiaolong Wang, Xinghao Zhu, Xinyang Geng, Xiyuan Liu, Xu Liangwei. Xuanlin Li, Yansong Pang, Yao Lu, Yecheng Jason Ma, Yejin Kim, Yevgen Chebotar, Yifan Zhou, Yifeng Zhu, Yilin Wu, Ying Xu, Yixuan Wang. Yonatan Bisk, Yongqiang Dou, Yoonyoung Cho, Youngwoon Lee, Yuchen Cui, Yue Cao, Yueh-Hua Wu, Yujin Tang, Yuke Zhu, Yunchu Zhang, Yunfan Jiang, Yunshuang Li, Yunzhu Li, Yusuke Iwasawa, Yutaka Matsuo, Zehan Ma, Zhuo Xu, Zichen Jeff Cui, Zichen Zhang, Zipeng Fu, and Zipeng Lin. Open embodiment: Robotic learning datasets and rt-x models, 2025. URL https: / arxiv. org/ abs / 2210. 08864.
Meredyth Daneman and Patricia A Carpenter. Individual differences in working memory and reading. Journal of verbal learning and verbal behavior, 19(4):450 466, 1980.
Rodrigo de Lazcano, Kallinteris Andreas, Jun Jet Tai, Seungjae Ryan Lee, and Jordan Terry. Gymnasium roboties, 2024. URL http: / thub. ccom/Farrama Foundatt on/
Gymnas um- Robotics...........
Ben Deverett, Ryan Faulkner, Meire Fortunato, Gregory Wayne, and Joel Z Leibo. Interval timing in deep reinforcement learning agents. Advances Neural Information Processing Systems, 32, 2019.
Kenji Doya. Temporal difference learning in continuous time and space. In Neural Informa- tion Processing Systems, 1995. URL htwps:///// //api. semanticschon Org/CorpusID: 1170136.
Kevin Esslinger, Robert Platt, and Christopher Amato. Deep transformer q-networks for partially observable reinforcement learning. arXiv preprint wrXiv:2206.01088 2022.
I Linxi Fan, Yuke Zhu, Jiren Zhu, Zihua Liu, Orien Zeng, Anchit Gupta, Joan Creus-Costa, Silvio Savarese, and Li Fei-Fei. Surreal: Open-source reinforcement learning framework and robot manipulation benchmark. In Aude Billard, Anca Dragan, Jan Peters, and Jun Morimoto (eds.), Proceedings of The 2nd Conference on Robot Learning, volume 87 of Proceedings of Machine Learning Research, pp. 767-782. PMLR, 29-31 Oct 2018. URL https: htteps:/prroceedings
mlr. hllz.pressv87/// fanl 8a html.
Haoquan Fang, Markus Grotz, Wilbert Pumacay, Yi Ru Wang, Dieter Fox, Ranjay Krishna, and Jiafei Duan. Sam2act: Integrating visual foundation model with a memory architecture for robotic manipulation. arXiv preprint arXiv:2501.18564 2025.
Meire Fortunato, Melissa Tan, Ryan Faulkner, Steven Hansen, Adrià Puigdomênech Badia, Gavin Buttimore, Charlie Deck, Joel Z Leibo, and Charles Blundell. Generalization of reinforcement learners with working and episodic memory, 2020. URL https:://aaatte http://:/aaxxvvoo//aaaae abs 11910. 13406.
Quentin Gallouédec, Nicolas Cazin, Emmanuel Dellandréa, and Liming Chen. panda-gym: gym: Open- Source Cona-Condd Environments for Robotic Learning. 4th Robot Learning Workshop: Self-Supervised and Lifelong Learning at NeurlPS, 2021.
Ran Gong, Jiangyong Huang, Yizhou Zhao, Haoran Geng, Xiaofeng Gao, Qingyang Wu, Wensi Ai, Ziheng Zhou, Demetri Terzopoulos, Song- Chun Zhu, al. Arnold: A benchmark for language- grounded task learning with continuous states in realistic 3d scenes. In Proceedings of the IEFF/CV International Conference on Computer Vision, pp. 10483-20495, 2023.
Anirudh Goyal, Abram Friesen, Andrea Banino, Theophane Weber, Nan Rosemary Ke, Adria Puig- domenech Badia, Arthur Guez, Mehdi Mirza, Peter C Humphreys, Ksenia Konyushova, et al. Retrieval- augmented reinforcement learning. International Conference on Machine Learning, pp. 7740-7765. PMLR, 2022a.
Anirudh Goyal, Abram L. Friesen, Andrea Banino, Theophane Weber, Nan Rosemary Ke, Adria Puig- domenech Badia, Arthur Guez, Mehdi Mirza, Peter C. Humphreys, Ksenia Konyushkova, Laurent Sifre, Michal Valko, Simon Osindero, Timothy Lillicrap, Nicolas Heess, and Charles Blundell. Retrieval-augmennt reinforcement learning, 2022b. URL https: / arriiv. org/ abs /2002. 08417.
13

---

Published as a conference paper at ICLR 2026
Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska- Barwiñska, Sergio Gómez, Edward Grefenstette, Tiago Ramalho, John Agapiou, Adrià Badia, Karl Hermann, Yori Zwols, Georg Ostrovski, Adam Cain, Helen King, Christopher Summerfield, Phil Blunsom, Koray Kavukcuoglu, and Demis Hassabis. Hybrid computing using a neural network with dynamic external memory. Nature, 538, 10 2016. doi: 10. 10.10388/ature20100
Jake Grigsby, Linxi Fan, and Yuke Zhu. Amago: Scalable in-context reinforcement learning for adaptive agents, 2024. htt.::tt////att
09971.
Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752. 2023.
Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state spaces. arXiv preprint arXiv:2111.00366, 2021.
Abhishek Gupta, Vikash Kumar, Corey Lynch, Sergey Levine, and Karol Hausman. Relay policy learning: Solving ong-horizon tasks via imitation and reinforcement learning. arXiv preprint 1rXiv:1910.11995 2019.
David Ha and Jürgen Schmidhuber. Recurrent world models facilitate policy evolution, 2018. URL https: aariiiiiivv./...aa/aaaa//aaaavvt org/ http///89.00990
Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference machine learning, pp. 1861-1870. Pmlr, 2018.
Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, and James Davidson. Learning latent dynamics for planning from pixels. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning. volume 97 of Proceedings of Machine Learning Research, pp. 2555 2565. PMLR, 09-15 Jun 2019. URL https: pttp///eooeeeccccccoo/pp mlr, wrwppppsssss.............eee
peress/v977 v https///////aaaeaeeeeeeeee 9a.. html.
Beining Han, Meenal Parakh, Derek Geng, Jack A Defay, Gan Luyang, and Jia Deng. Fetchbench: A simulation benchmark for robot fetching, arXiv preprint urXiv:2406.11798 2024.
Nicklas Hansen, Hao Su, and Xiaolong Wang. TD-MPC2: Scalable, robust world models for continuous control. In The Twelfth International Conference on Learning Representations, 2024. URL https::///// wwtp//eeeeeeeeeeepeeepppe net forum? id=Oxh5CstDDD
Matthew Hausknecht and Peter Stone. Deep recurrent q-learning for partially observable mdps, 2015.
Stephan Heckers, Martin Zalesak, Anthony P Weiss, Tali Ditman, and Debra Titone. Hippocampal activation during transitive inference in humans. Hippocampus, 14(2):153-162, 2004.
Felix Hill, Olivier Tieleman, Tamara von Glehn, Nathaniel Wong, Hamza Merzic, and Stephen Clark. Grounded language learning fast and slow, 2020. URL https:://arxai 01719.
abs/2009.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Comput., 9(8): 1735-1780, nov 1997. ISSN 0899-76677 doi: 10. 16.1nee/n..o. 1997.9.8.1733. URL https: //doi.org/ /t.rg//..11 11 62//neoo 1997.9 9. 8. 1735.
Jan Humplik, Alexandre Galashov, Leonard Hasenclever, Pedro Ortega, Yee Whye Teh, and Nicolas Heess. Meta reinforcement learning as task inference, 2019. URL
org/abs 1901. 0 6224.
Chia-Chun Hung, Timothy Lillicrap, Josh Abramson, Yan Wu, Mehdi Mirza, Federico Carnevale, Arun Ahuja, and Greg Wayne. Optimizing agent behavior over long time scales by transporting value, 2018. URL https: /ttpiixivvv.///////aaaa/aaaaav.a org/ abs/ 1810.11 06721.
Stephen James, Zicong Ma. David Rovick Arrojo, and Andrew J Davison. Rlbench: The robot learning benchmark & learning environment. IEEE Robotics and Automation Letters, 5(2): 3026, 2020.
14

---

Published as a conference paper at ICLR 2026
Yunfan Jiang, Agrim Gupta, Zichen Zhang, Guanzhi Wang, Yongqiang Dou, Yanjun Chen, Li Fei-Fei, Anima Anandkumar, Yuke Zhu, and Linxi Fan. Vima: General robot manipulation with multimodal prompts. arXiv preprint arXiv:2210.0300 2(3):6, 2022.
Leslie Pack Kaelbling, Michael L Littman, and Andrew W Moore. Reinforcement learning: A survey. Journal of artificial intelligence research, 4:237-285, 1996.
Leslie Pack Kaelbling, Michael L. Littman, and Anthony R. Cassandra. Planning and acting in partially observable stochastic domains. Artificial Intelligence, (01(1)99--134, 1998. ISSN 0004-3702. doi: httpa//cw....cceeeeeeeeee/aaeata sciencedirect
URL https: //www.
Yongxin Kang, Enmin Zhao, Yifan Zang, Lijuan Li, Kai Li, Pin Tao, and Junliang Xing. Sample efficient reinforcement learning using graph-based memory reconstruction. IEEE Transactions on Artificial Intelligence, 5(2):751-762, 2024. doi: 10. 1,.119991122223336666611
Steven Kapturowski, Georg Ostrovski, John Quan, Rémi Munos, and Will Dabney. Recurrent experi- ence replay in distributed reinforcement learning. In International Conference on Learning Repre- sentations, 2018. URL https: / /api. semanti wammantisccooar...ommmmmm Ctapg//orppsssii ID:93455798
Siddharth Karamcheti, Suraj Nair, Ashwin Balakrishna, Percy Liang, Thomas Kollar, and Dorsa Sadigh. Prismatic vlms: Investigating the design space of visually-conditione language models, 2024. URL http:::///riiie org/
/abg/aa/////aaao//a//////////aa/ abs 2402 07865.
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burch- fiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. Openvla: An open-source vision--anguage-cction model, 2024. URL https: pattps://arxivw /ttp//aax.v../aaaaaaa//// 09246.
Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-tuning onn models: Optimizing speed and success, 2025. URL htL.tttts///a/zv..ooro hbtp:///bb/////// 19645.
Rob Knight, Pepijn Kooijmans, Remi Cadene, Simon Alibert, Michel Aractingi. Dana Aubakirova, Adil Zouitine, Russi Martino, Steven Palma, Caroline Pascal, and Thomas Wolf. Standard open so- 100 & so-101 arms, 2024. URL Online.
com/ Stw.eeootttudio//eeeet...
Eric Kolve, Roozbeh Mottaghi, Winson Han, Eli VanderBilt, Luca Weihs, Alvaro Herrasti, Matt Deitke, Kiana Ehsani, Daniel Gordon, Yuke Zhu, al. Ai2-thor: An interactive 3d environment for visual ai. arXiv preprint 47Xiv:1712.05474. 2017.
Deanna Kuhn. The development of causal reasoning. Wiley Interdisciplinary Reviews: Cognitive Science, 3(3):327-335, 2012.
Aviral Kumar, Aurick Zhou, George Tucker, and Sergey Levine. Conservative q-learning for offline reinforcement learning. Advances in neural information processing systems, 33:1179-1199 2020.
Hanna Kurniawati. Partially observable markov decision processes and robotics. Annual Review of Control, Robotics, and Autonomous Systems, 5(1):253-277, 2022.
Heinrich Küttler, Nantas Nardelli, Alexander H. Miller, Roberta Raileanu, Marco Selvatici, Edward Grefenstette, and Tim Rocktäschel. The nethack learning environment, 2020. URL https:
wttp:iiv.rrg/ab/////aaaaa / 13760.
Andrew Lampinen, Stephanie Chan, Andrea Banino, and Felix Hill. Towards mental time travel: hierarchical memory for reinforcement learning agents. Advances in Neural Information Processing Systems, 34:2818) 182-28195. 2021.
Mikko Lauri, David Hsu, and Joni Pajarinen. Partially observable markov decision processes in robotics: A survey. IEEE Transactions Robotics, 39(1):21-40, February 2023. ISSN 1941- 0468. doi: 10.11099tt.20223200133 URL http: /xx doi meci.org/100 /ttg//1... 1109/TRO 2022. 3200138.
15

---

Published as a conference paper at ICLR 2026
Joel Z. Leibo, Cyprien de Masson d' Autume, Daniel Zoran, David Amos. Charles Beattie, Keith Anderson, Antonio García Castañeda, Manuel Sanchez, Simon Green, Audrumas Gruslys, Shane Legg, Demis Hassabis, and Matthew M. Botvinick. Psychlab: A psychology laboratory for deep reinforcement learning agents, 2018. URL wtw.tttpp///tm waa.ariivv..oo org/ abs 1801. 0811 161..
Sergey Levine, Peter Pastor, Alex Krizhevsky, Julian Ibarz, and Deirdre Quillen. Learning hand- eye coordination for robotic grasping with deep learning and large-scale data collection. The International journal of robotics research, 77(4-))441-444 2018.
Chengshu Li, Fei Xia, Roberto Martín-Martín, Michael Lingelbach, Sanjana Srivastava, Bokui Shen. Kent Elliott Vainio, Cem Gokmen, Gokul Dharan, Tanish Jain, Andrey Kurenkov, Karen Liu, Hyowon Gweon, Jiajun Wu, Li Fei-Fei, and Silvio Savarese. igibson 2.0: Object-centric simulation for robot learning of everyday household tasks. In Aleksandra Faust, David Hsu, and Gerhard Neumann (eds.), Proceedings of the 5th Conference Robot Learning, volume 164 of Proceedings of Machine Learning Research, pp. 455-455-4466 PMLR, 08-11 Nov 2022. URL https: https://ppoceedinng mlr
wwt.mmmr...p.s..e... press / vv 64/1122/2 html.
Chengshu Li, Ruohan Zhang, Josiah Wong, Cem Gokmen, Sanjana Srivastava, Roberto Martín- Martín, Chen Wang. Gabrael Levine, Wensi Ai, Benjamin Martinez, et al. Behavior- k: A human-centered, embodied ai benchmark with 1,000 everyday activities and realistic simulation. arXiv preprint arXiv:2403.0922 2024.
Alvin M Liberman, Katherine Safford Harris, Howard S Hoffman, and Belver C Griffith. The discrimination of speech sounds within and across phoneme boundaries. Journal of experimental psychology, 54(5):358, 1957.
Zichuan Lin, Tianqi Zhao, Guangwen Yang, and Lintao Zhang. Episodic memory deep q-networks. 2018. URL https: http:p:://axxxiooooto
arxia.xiiiv..o org/ abs abs//abs..... 1805.07605
Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning. Advances in Neural Information Processing Systems, 36:44776-44791, 2023.
Chris Lu, Yannick Schroecker, Albert Gu, Emilio Parisotto, Jakob Foerster, Satinder Singh, and Feryal Behbahani. Structured state space models for in-context reinforcement learning, 2023. URL
https::/aarei org/ /bss23333 03982.
Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, et al. Isaac gym: High performance gpu-based physics simulation for robot learning. arXiv preprint arXiv:2 10470, 2021.
Lingheng Meng, Rob Gorbet, and Dana Kulić. Memory-bar deep reinforcement learning for pomdps. In 2021 IEEE/RSJ international conference intelligent robots and systems (IROS), pp. 5619-5626. IEEE, 2021.
Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, and Pieter Abbeel. A simple neural attentive meta- learner, 2018. URL https: arxiv.. org/ abs 707 03141.
Steven Morad, Ryan Kortvelesy, Matteo Bettini, Stephan Liwicki, and Amanda Prorok. POPGym: Benchmarking partially observable reinforcement learning. In The Eleventh International Confer- ence on Learning Representations, 2023. URL https::/ https:://penreview.rrr net forum?id- chDrutUTsOK.
Soroush Nasiriany, Abhiram Maddukuri, Lance Zhang, Adeet Parikh, Aaron Lo, Abhishek Joshi, Ajay Mandlekar, and Yuke Zhu. Robocasa: Large-scale simulation of everyday tasks for generalist robots. arXiv preprint Xix:v:2406. 21v::2406002523, 02523, 2024.
Tianwei Ni, Benjamin Eysenbach, and Ruslan Salakhutdinov. Recurrent model-free can be a strong baseline for many pomdps. arXiv preprint arXiv:21 2021.
Tianwei Ni, Michel Ma, Benjamin Eysenbach, and Pierre-Luc Bacon. When do transformers shine in rl? decoupling memory from credit assignment, 2023. URL https: /wtxxv.iivv//////////a///a/aa org /bs/ 2307. 64.
16

---

Published as a conference paper at ICLR 2026
Junhyuk Oh, Valliappa Chockalingam, Satinder Singh, and Honglak Lee. Control of memory, active perception, and action in minecraft, 2016. URL https: /ta//aivvvv../// org/ abs /005 09128.
Ian Osband, Yotam Doron, Matteo Hessel, John Aslanides, Eren Sezener, Andre Saraiva, Katrina McKinney, Tor Lattimore, Csaba Szepesvári, Satinder Singh, Benjamin Van Roy, Richard Sutton, David Silver, and Hado van Hasselt. Behaviour suite for reinforcement learning. In International Conference on Learning Representations, 2020. URL https: /oop/rpppeeeeeeeeeeeeeeeee new./eee//ee... forum?? id-rygf-kSYwii.
Toshihiro Ota. Decision mamba: Reinforcement learning via sequence modeling with selective state spaces. arXiv preprint arXiv:2403.19923, 2024.
Emilio Parisotto and Ruslan Salakhutdinov. Neural map: Structured memory for deep reinforcement learning, 2017. URL htw.t: //arxiv wwwpeeiwwwiioooooooooeeeeeooeere org/ abwg//bb/// / 1702. 08360.
Emilio Parisotto, Francis Song, Jack Rae, Razvan Pascanu, Caglar Gulcehre, Siddhant Jayakumar, Max Jaderberg, Raphael Lopez Kaufman, Aidan Clark, Seb Noury, et al. Stabilizing transformers for reinforcement learning. In International conference machine learning, pp. 7487-7498. PMLR, 2020.
Jurgis Pasukonis, Timothy Lillicrap, and Danijar Hafner. Evaluating long-term memory in 3d mazes, 2022. URL httpp:://aattette http:/awwvvv..o.a.aaa/ www//ass.......o.oo.../........e 1233.1338..
John Piaget. The origins of intelligence in children. International University, 1952.
Marco Pleines, Matthias Pallasch, Frank Zimmer, and Mike Preuss. Memory gym: Partially observ- able challenges to memory-basee based agents in endless episodes. arXiv preprint arXiv:2309.17200 2023.
Delin Qu, Haoming Song, Qizhi Chen, Yuanqi Yao, Xinyi Ye, Yan Ding, Zhigang Wang. Jia Yuan Gu, Bin Zhao, Dong Wang, and Xuelong Li. Spatialvla: Exploring spatial representations for visual-language-action model, 2025. URL https: /eettps://rxivvv www.oorgg/aoooo haag///b////a /2011/5111.m 15830.
David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams. Learning representations by back- propagating errors. Nature, 323:533-553 1986. URL http:: //api www.stteencnnooooooeeee.
org/Corpus ID: 205001833 8334.
Mohammad Reza Samsami, Artem Zholus, Janarthanan Rajendran, and Sarath Chandar. Mastering memory tasks with world models, 2024. URL
/ttp:///////../.3.....// 04253.
Mikayel Samvelyan, Robert Kirk, Vitaly Kurin, Jack Parker-Holder, Minqi Jiang, Eric Hambro, Fabio Petroni, Heinrich Küttler, Edward Grefenstette, and Tim Rocktäschel. Minihack the planet: A sandbox for open-ended reinforcement learning research, 2021. URL https: httttpp://aaxx..oooog abs/2109 13202.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: 2017.
Mohit Shridhar, Lucas Manuelli, and Dieter Fox. Cliport: What and where pathways for robotic manipulation. In Conference on robot learning, pp. 894-906. PMLR, 2022.
Arth Shukla, Stone Tao, and Hao Su. Maniskill-hab: A benchmark for low-level manipulation in home rearrangement tasks. arXiv preprint 2rXiv:24/2.132/1 2024.
Aleksandrs Slivkins. Introduction to multi-armed bandits, 2024. URL. htl.httss://axiiiee abs/1 09990.4440 07272.
Jimmy T. H. Smith, Andrew Warrington, and Scott W. Linderman. Simplified state space layers for sequence modeling, 2023. URL https: : arxiv. org/ ab//abss/2/00 04933.
Artyom Sorokin, Nazar Buzun, Leonid Pugachev, and Mikhail Burtsev. Explain my surprise: Learning efficient long-term memory by predicting uncertain outcomes. Advances in Neural Information Processing Systems, 35:36875-36888, 2022.
17

---

Published as a conference paper at ICLR 2026
Matthijs T. J. Spaan. Partially observable Markov decision processes. In Marco Wiering and Martijn van Otterlo (eds.), Reinforcement Learning: State of the Art. pp. 387-414. Springer Verlag, 2012.
Andrew Szot, Alexander Clegg, Eric Undersander, Erik Wijmans, Yili Zhao, John Turner, Noah Maestre, Mustafa Mukadam, Devendra Singh Chaplot, Oleksandr Maksymets, et al. Habitat 2.0: Training home assistants rearrange their habitat. Advances in neural information processing systems, 34:251-266, 2021.
Stone Tao, Fanbo Xiang, Arth Shukla, Yuzhe Qin, Xander Hinrichsen, Xiaodi Yuan, Chen Bao, Xinsong Lin, Yulin Liu, Tse-kai Chan, et al. Maniskill3: Gpu parallelized robotics simulation and rendering for generalizable embodied ai. arXiv preprint arXiv:24 2024.
Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, Jianlan Luo, You Liang Tan, Lawrence Yunliang Chen, Pannag Sanketi, Quan Vuong, Ted Xiao, Dorsa Sadigh, Chelsea Finn, and Sergey Levine. Octo: An open-source generalist robot policy, 2024. URL https: wwwiiviivvvvv/////i///a///aaaaia org/ abss 2405 2213.
Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSI International Conference on Intelligent Robots and Systems, pp. 5036-5033, 2012. doi: 100///RS..2012..6888
Mark Towers, Ariel Kwiatkowski, Jordan Terry, John U Balis, Gianluca De Cola, Tristan Deleu, Manuel Goulão, Andreas Kallinteris, Markus Krimmel, Arjun KG, et al. Gymnasium: standard interface for reinforcement learning environments, arXiv preprint arXiv:2407. 170322 2024.
Saran Tunyasuvunakool, Alistair Muldal, Yotam Doron, Siqi Liu, Steven Bohez, Josh Merel, Tom Erez, Timothy Lillicrap, Nicolas Heess, and Yuval Tassa. him_contro:: Software and tasks for continuous control. Software Impacts, 6: 6:100022, 2020.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
Homer Rich Walke, Kevin Black, Tony Zhao, Quan Vuong, Chongyi Zheng, Philippe Hansen- Estruch, Andre Wang He, Vivek Myers, Moo Jin Kim, Max Du, al. Bridgedata v2: A dataset for robot learning at scale. In Conference on Robot Learning, pp. 1723-1736. PMLR, 2023.
Daan Wierstra, Alexander Förster, Jan Peters, and Jürgen Schmidhuber. Recurrent policy gradients. Logic Journal of the IGPL, 18:620-634, 10 2010. doi: 10. 10.10033/jgpal//zp044.
Karmesh Yadav, Jacob Krantz, Ram Ramrakhya, Santhosh Kumar Ramakrishnan, Jimmy Yang, Austin Wang, John Turner, Aaron Gokaslan, Vincent-Pierre Berges, Roozbeh Mootaghi, Oleksandr Maksymets, Angel X Chang, Manolis Savva, Alexander Clegg, Devendra Singh Chaplot, and Dhruv Batra. Habitat challenge 2023. https: htt.aaaaaaaattt.. org/ cha lew.leeee///223///////// 2023.
Renye Yan, Yaozhong Gan, You Wu, Junliang Xing, Ling Liangn. Yeshang Zhu, and Yimao Cai. Adamemento: Adaptive memory-assisted policy optimization for reinforcement learning, 2024.
URL https::/ wawaaaiivvvoo...aa..aaa..aa..aaa / abs/2410. 04498.
Tianhe Yu, Deirdre Quillen, Zhanpeng He, Ryan Julian, Karol Hausman, Chelsea Finn, and Sergey Levine. Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning. In Conference on robot learning, pp. 1094-1100. PMLR, 2020.
William Yue, Bo Liu, and Peter Stone. Learning memory mechanisms for decision making through demonstrations. arXiv preprint arXiv:2411.07954. 2024.
Tony Duan YuXuan Liu and Wesley Hsieh. Temporal convolutional policy networks, 2016. URL https://yyxuuu wu. f.m fi eesee////i weeees/teee http:/ppp.pp// .df.
Wojciech Zaremba and Ilya Sutskever. Reinforcement learning neural turing machines - revised, 2016. URL. URL.http:://aradvvoor///// 5ttp://wwww..or////////////../00 0052..
18

---

Published as a conference paper at ICLR 2026
Andy Zeng, Pete Florence, Jonathan Tompson, Stefan Welker, Jonathan Chien, Maria Attarian, Travis Armstrong, Ivan Krasin, Dan Duong, Vikas Sindhwani, et al. Transporter networks: Rearranging the visual world for robotic manipulation. In Conference on Robot Learning, pp. 726-747. PMLR, 2021.
Shengqiang Zhang, Philipp Wicke, Lütfi Kerem Şenel, Luis Figueredo, Abdeldjallil Naceri, Sami Haddadin, Barbara Plank, and Hinrich Schütze. Lohoravens: A long Long-horizon sanguage-conditione benchmark for robotic tabletop manipulation. arXiv preprint arXiv:2310.12220, 2023.
Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. Graph neural networks: review of methods and applications. Al open, 1:57-81, -1, 2020.
Deyao Zhu, Li Erran Li, and Mohamed Elhoseiny. Value memory graph: A graph-structured world model for offline reinforcement learning, 2023. URL https: /ttp///axvvvvvvae//ee//eeeeaaava wwwp:iiwwwg/a//////222 04384.
Guangxiang Zhu, Zichuan Lin, Guangwen Yang, and Chongjie Zhang. Episodic reinforcement
learning with associative memory. In International Conference on Learning Representations, 2020a. URL https:tpp//. https://aaaaae /tap:::///p..../ Segmmattieeccctlllaa..ooooooooop
Pengfei Zhu, Xin Li, Pascal Poupart, and Guanghui Miao. On improving deep reinforcement learning for pomdps, 2018.
07978.
Yuke Zhu, Josiah Wong, Ajay Mandlekar, Roberto Martin-Martín, Abhishek Joshi, Kevin Lin, Soroush Nasiriany, and Yifeng Zhu. robosuite: A modular simulation framework and benchmark for robot learning. In arXiv preprint wrXiv:2009112293 2020b.
19

---

Published as a conference paper at ICLR 2026
Table of Contents
A MIKASA-Robo Implementation Details
21
B MIKASA-Robo Tasks Customization
22
MIKASA-Robo Datasets Offfine and Imitation Learning
25
D MIKASA-Robo setup for VLA baselines
26
E MIKASA-Base Implementation Details
26
F Memory Mechanisms in RL
27
G Classic baselines performance the MIKASA-Robo benchmark
27
H Additional Offline RI. Results with Dense Rewards
28
Experiments Reproducing and Compute Resources
28
Additional Baselines: SSMs and Memory-Enhance Transformers
28
K MIKASA-Robo Detailed Tasks Description
31
ShellGame-vO
33
K.2
RememberColor-v0
34
K.3
RememberShape-v0
35
K.4
RememberShagpeAAuCCCooo:
36
K.5
Intercept-v0
37
IntercepiGrab-vO.
38
K.7
RotateLenient-v0
39
K.8
RotateStrict-vO..
40
K.9
TakeliBack-vO
41
K.10
SegOrColors-vO
................................
42
K.11
BunchOfColors-v0
43
K.12
ChainOfColors-vv
44
L MIKASA-Base Benchmark Tasks Description
45
L.1
Memory Cards
45
L.2
Numpad
................................
45
L.3 BSuite MemoryLength
46
L.4 Minigrid- Sinigrid-Memory
46
L.5 Ballet
46
L.6
Passive T-Maze
46
L.7
WZZoomm-Tw--Coll
46
L.8
Memory Maze
46
L.9 MemoryGym Mortar Mayhem
47
L.10
MemoryGym Mystery
47
L.11
POPGym environments
47
M MIKASA-Rohe Customization Guides
49
M.1
Intercept
50
M.2
Rotate
51
M.3
TakeltBack
51
M.4
RememberColor
52
M.5
RememberShape
52
M.6
RemenberShaapeAndCCooo
52
M.7
BunchOrColors
53
M.8
SeqOfColors
53
M.9
ChainOfColors....
53
Real-World Experiments
20

---

Published as a conference paper at ICLR 2026
A MIKASA-ROBO IMPLEMENTATION DETAILS
An example of running the environment from the MIKASA-Robs benchmark is shown in Code 1. For ease of debugging, also added various wrappers (found in mi htcccbbssutteuutllsssssoo on the video (Code 2). Thus, www.nnderteeIInoowaassssese ronment; Sewuggeewwddrrrprrrr.... () displays information about the full reward at the current step in the environment; DebugRewardWWappa
that display useful information about the episode displays the current step in the envi-
displays information about each component that generates the reward function at the current step, In addition, we also added task-specifrs wrappers each environment. For example, GemeemberroooorInooo.e.... of the cube in the RememberCon cooenbrrrrooooo oo or-v0 task, and Shel GameRenderCCup In nfoorapper () displays which mug the ball actually under in the Shel Gamee-G task.
() displays the target color
Code Getting started with MIKASA-Robs using the RememberCColor9-vo environment.
pip install
import mikasa robo, suite from mikasa_robo_aut wt.....apppresss import
robo_suite
+ StateonlyTemoorrodccccccpppp
from tqdm. import tqdm import torch
import gymnasium as gym
Create the environment via gym.make obo__oooce "oo_mode="rgp" for modes "RGB", "RCB+joint", "RGBtoracie" etc. abb__mode=sstaee for mode "state"
appisode_timoon - 90
env gym.make (" ("temeebsrccoooor-0oo" num_envs-512 obs @es__mode""rpp
render_mode="all1")
env Caateorll/eenoooTooitttttepppp (env)
always use this wrapper!
obs, env reset (seed=42)
print (obs bobsskeys|l ())
for
tęam (range (opisode_tinooutteetooooeeeeoeet Lnno
action torch.fron_nuppp: (env. wew.aaction_ssaee wttp__spcccssaplllllleee ())
obs, reward, terminated, truncated, info env. step (actien)
env.close()
Code 2: MIKASA-Robo wrappers system.
import mikasa_roka from cattsaa_rro_ssitee_ torch
http:eutteedatttttoooleeeooooooe htt.::/oo...tttt......eeeeerrree import env_info import gymnasium as gym from mani_skil1 wtt.ll.tilsssr.r.......... ws..pp.sssee import RecordRpisode
from IPython.di birchon.disppl disllay import Video
env gym.make ("RemenberColorg-vut
envs=512, rod"="rgb",
) sender_mode=""a
state state_ waapperwrpprrseeeee _ist,, episode_ timeout env info "Tememberrolorr-vo01)
for wrapper_class, wrapper_kwargs in state_wrapers_llst:: wrapper_class (env, wwrapper_kwaree env RecordEpisode (env, f", "v./vidoos", max_stops_per_vicer
+ episode_timeouts
obs, env. reset (seed=42) for i in range simeoude_timeout):
action torch. toruunirom_nuuppy (env. action wwtticcsppaeeeeteaeeeeeeeeeeeeee Lem.pp ())
obs, rewara, terminated, truncated, info env.step (acticn)
video (f"./"""vii htt..vvvvo//////ee mp enbed-True, width-640)
env.clow closel
21

---

Published as a conference paper at ICLR 2026
B MIKASA-ROBO TASKS CUSTOMIZATION
Beyond the official configurations, MIKASA-Robo designed to be fully customizable. Researchers can directly adjust environment parameters such as number of objects, episode length, cue duration, or delay intervals - to create variants tailored for debugging, ablation studies, or curriculum learning. This flexibility ensures that tasks can scale from minimal examples extremely challenging settings while preserving their memory-centrie nature.
We deliberately include highly challenging variants (e.g.. BunchOfColors, SeqOfColors. Cha i InOfColoo ors) to ensure that the benchmark continues to stress-test algorithms as memory capabilities advance. To avoid hidden shortcuts, we provide a set of fixed official configurations with multiple difficulty levels (e.g., RememberColor3 exposes its underlying parameters, making customization straightforward.
9). At the same time, each environment
For example, the RememberColor task can be customized as follows:
Code 3: Customized RememberColos enviromment.
from mani_skill utils wwg.istttiiiionoo import register_onn from salkasa_robo_suts _uroooossstteeeeee wwccmmbbrrcoooooooeoeeeeeoeeeeee import RonemborColorBasenn import gymnasius um as gym
@register_env weeemembeerollooDebbueee
max_spisode__seps-1100
class Remembe coo.orbbonrolloorrbouunnn SeeemmberrllorrBseeeeee
COLORS
= # 1-9 unique cubes
TIME TIME_OFFSET = 200 $ duration target cube is visible
GOAL THRESH
0.03
# success threshold
CURR_HALPSIZE 0.02
cube size
DELTA_TIME
100 8 delay before response
env = sym.make ("RemenberCaloorDeebuuuu
num een_enn== obs rgb_ render_moce sender_mode="al1" delta_cime-DEInk _TMM))
Similarly, the SegOfColors task can be configured with custom sequence length and timing:
Code 4: Customized SeqOfColors environment.
registratiinnnnnnnnnn import register_eny from iikasa_robo_sut seq_of_colorss import SeqDFColorsënv import gymnasium gym
from
@register env ("o max_ spisode_steps-1ee class
(SegOfcolorsEng
COLORS
8 1-9 unique cubes
GOAL_THRESH 0.03
CUBE HALFSIZE 0.02
SEQUENCE. LENGTH= 2 $ number of cubes in sequence
STEP_ _UR_ADDRTTOON = 15 # duration cube
empty delay between cubes
env = gym. make ("Sqoooooor@ebggg-0 num_envs=256,
The design of MIKASA-Roba emphasizes isolating the role of memory, rather than long-horizon credit assignment. For example, tasks like RememberColoz already become non-Markovian after a single occlusion, so even short horizons (e.g., 60 steps) suffice to reveal memory limitations. Still, researchers can easily scale horizon length and difficulty by tuning memory- related parameters.
Below we illustrate how to extend RememberColor into long- long-horizon setting, increasing both episode length and delay:
22

---

Published as a conference paper at ICLR 2026
Code 5: Long-horizon variant of RememberColor.
from mani_skill.utile registration import register_env from aatass_rrbb_suiiie rwwember_coloo import RenemberColorFaseenn import gymnasium as gym
@register_cnt ("#ememberrcoooorreebuieo
max__piiodb_sttp==1000 class RemenbeerCoooorDebhgEr (Renemeerrcloorbaseee...
COLORS
TIMR_OFFSET 50 # target cube duration
GOAI_THRKSH 0.03
CURK_HALPS IZE-
---
0.02
DELTA TIME 900
extended delay
env = ("RemmlercoloooDeebaggg num num_ sam__nns=256,
obs_mode="rgb", render_mode="ale delta_time=DELTA_TII
This flexibility applies to all tasks in the benchmark, making MIKASA-Robo suitable for controlled debugging, systematic ablations, or curriculum-basee studies of memory.
Embodiment flexibility in MIKASA-Robo. Memory requirements MIKASA-Robo are inde- pendent of the robot embodiment. The embodiment determines kinematics, actuator morphology, and low-level motion feasibility, but the memory load induced entirely by the task structure and the policy rather than the hardware platform. We default the Franka Panda
therefore depends because it is one of the most widely adopted manipulators in RL and robotics research, offering stable physics, high quality simulation assets, and broad community support. However, ManiSkill3 natively supports alternative embodiments, and all MIK MIKASA ASA-RoSRoo tasks can be instantiated with different arms without altering the underlying memory challenge (see Figure 7).
Extending atomic tasks with realistic object distributions. The objective MIKASA-RoN is to isolate the memory dimension of manipulation, and incorporating complex meshes, textures, or fragile grasping conditions introduces confounding factors that mask the memory dependencies we aim to evaluate. Primitive shapes yield clean, well-controleed environments in which performance differences can attributed memory mechanisms rather than physics artifacts or mesh-specific interactions. Although the benchmark is designed atomically to isolate memory from perception, control, and other competencies, each task class can be lifted to more realistic regimes. After an agent succeeds the atomic variants, the same task structures objects (for example, from YCB Dataset (Calli et al., 2015) or BridgeData V2 (Walke et al., 2023)), preserving the underlying memory requirements while introducing richer visual variability and more demanding manipulation dynamics. For example, Remembe rooler3- which evaluates recall of the color of one of three cubes, generalizes naturally RememberOb 3ectt 3ect3-v0, where the cubes are replaced with randomly sampled objects whose appearance and geometry necessitate substantially higher perceptual precision and control accuracy (see Figure 8).
be re-instantiated using complex
Clutter an optional difficulty dimension. We intentionally avoid cluttered scenes in the atomic task suite, since the goal of MIKASAARRbo is to isolate memory from other confounding factors such as occlusion handling, dense object-objed interactions, and contact-rich rearrangement. Clutter introduces additional challenges in perception, planning, and control that make it difficult to attribute performance differences to memory mechanisms alone. By keeping the scene minimal, the benchmark ensures that errors arise from failures in memory rather than from the incidental complexity of cluttered manipulation. At the same time, the framework does not restrict users from increasing task difficulty once policy succeeds the atomic variants. Clutter be added in controlled manner to produce more challenging instances, requiring stronger perception and robustness without altering the underlying memory structure of the original task (see Figure 9).
23

---

Published as a conference paper at ICLR 2026
Figure 7: Examples of the She 1 1 Toach Touch-vO task performed using different robot embodi- ments available in ManiSkill3, The scenes demonstrate that the benchmark is tied to particular manipulator: Franka Panda, Unitree arms, and other platforms can all execute the same task configu- ration. This illustrates that MIKASA -Robo supports interchangeable embodiments while preserving identical task logic and memory requirements.
Figure 8: Example rollouts for the Remembe r rOb bectt-v8 task. The left column shows the initial state which the robot observes single target object. The object is then removed, and the right column shows the subsequent scene populated with distractor objects, at which point the agent must recall and identify the object presented initially. The task preserves the same memory structure as GememberColors v-v but replaces colored cubes with diverse objects of varying geometry and appearance, thereby increasing perceptual and manipulation complexity while maintaining identical temporal memory requirements.
24

---

Published as a conference paper at ICLR 2026
Figure 9: Cluttered variants of the Remembe jecbber3rreeettt- task. The left column shows the initial observation containing single target object, while the right column shows the subsequent scene populated with clutter. Although clutter is not included in the atomic benchmark to avoid confounding perception and contact-rich interactions with memory, difficulty dimension once a policy reliably solves the base task. These examples illustrate how clutter increases visual complexity and occlusions.
be added optional
C MIKASA-ROBO DATASETS FOR OFFLINE RL AND IMITATION LEARNING
To train Offline RI. or Imitation Learning baselines on camera images (in "RGB" mode) with sparse rewards (success condition), we collected expert-quality datasets for each of the 32 MIKASA-Robo tasks. Datasets were collected using PPO-MLP agent trained to SR=100% in "state" mode (i.e., with full information about the task being solved) with sparse rewards (success condition). Thus, each dataset is represented by 1000 successful trajectories, where each trajectory consists of:
1. "rgb" (shape: (T, 128, 128, 6)) two RGB images (view from above and from the gripper)
2. "joints" (shape: (T, (5)55) Tool Center Point (TCP) position and rotation, and joint positions and velocities
3. "action" (shape: (T, 8)) action (8-dimensional vector)
4. "reward" (shape: (T, )) (dense) reward for each step
25

---

Published as a conference paper at ICLR 2026
Table 5: Tasks configurations for fine-tuning VLA models. The table lists the task ID, number of evaluation steps (T), and the associated language instruction
Task
T
Language instruction
Remenbe::Color3/5/99
60
Remember the color of the cube and then pick the matching one
90
Memorize the position the cup covering the ball, then pick that cup
Intercept um-vD
90
Track the hall's movement, estimate its velocity, then aim the ball at the target
5. "success" (shape: (T.)) (sparse) success flag for each step
6. "done" (shape: (T))) done flag for each step
These datasets are available for download from the project website. We have also published the weights of the PPO-MLP agent used to collect the dataset, well as scripts for collecting datasets of any size, to our repository.
D MIKASA-RORE SETUP FOR VLA BASELINES
For experiments involving Vision-Language-Actinn (VLA) models, we focused on a representative subset of spatial and object memory tasks from MIKASA-Robo For each task, generated dataset of 250 episodes using an oracle PPO policy with full access to the environment state. At every timestep, the policy recorded two synchronized RGB frames (one from the static "base" camera and one from the robot's wrist camera) along with the corresponding end-effector control actions _ee_eella Ataltaas _tt__psseeeee_e controller from (Tao et al., 2024)). Each task also paired with concise language instruction (see Table 5).
All VLA baselines were trained for 50000 iterations and evaluated independently each task. Com- plete training/evaluation seripts, language instruction templates, and detailed model hyperparameter settings are provided in the accompanying supplementary code.
E MIKASA-BASI IMPLEMENTATION DETAILS
An example of rumning environment from the MIKASA- Base benchmark is shown in Code 6. MIKASA-Base supports the standard Gymnasium API and is fully compatible with all its wrappers. This allows AsyncVectorEav. MIKASA-Base provides a predefined set of environments with different levels of difficulty. However, users can customize the environment parameters by passing specific arguments (see Code 6).
leverage various functionalities, including parallelization using
Code 6: Example code for running MemoryLength-v00 environment.
import mikasa base
import gymnasium gym
use pre-defined env
env_id = "Memoryleenghhaasy-oo" env_kwargs None
create env using custom parameters env id = "MemozyLength-vo" env_kwargs d"methry_length":: 10, "num_bits"; 1)
seed 123
env make tenv_id, env env_kwargs)
obs, env, reset (seed=seed)
for i range (11):
action env. www.actiio_spacce sample ()
next_obs, reward, cerminations, truncation ions, infos - env, step
4 action)
env.clow anv.close()
26

---

Published as a conference paper at ICLR 2026
F MEMORY MECHANISMS IN RL
In RI., memory mechanisms are techniques or models used to enable agents to retain and recall information from past interactions with the environment.
There are several approaches to incorporating memory into RL, including recurrent neural networks (RNNs) (Rumelhart et al., 1986; Hochreiter & Schmidhuber, 1997; Chung et al., 2014) which uses hidden states to store information from previous steps (Wierstra et al., 2010; Hausknecht & Stone, 2015), state-space models (SSMs) (Gu et al., 2021; Smith et al., 2023; Gu & Dao, 2023) which uses system state to store historical information (Hafner et al., 2019; Samsami et al., 2024), transform- ers (Vaswani et al., 2017) which uses attention mechanism to capture sequential dependencies inside the context window (Parisotto et al., 2020; Lampinen et al., 2021; Ni et al., 2023), graph neural networks (GNNs) (Zhou al., 2020) which uses graphs to store information (Zhu et al., 2023; Kang et al., 2024) etc. Popular agents with memory mechanisms are summarized in Table 2.
Temporal convolutions constitute an additional class of memory mechanisms. Here, information is stored implicitly through convolutional filters applied over the temporal dimension, enabling the agent integrate multi-step dependencies without explicit recurrent states (YuXuan Liu & Hsich, 2016; Mishra et al., 2018). Although they lack explicit recurrence or attention, their receptive fields can be designed to capture long-range temporal features. World models (Ha & Schmidhuber, 2018) implement memory by constructing internal predictive dynamics, effectively learning latent environment model that summarizes prior interactions. External memory mechanisms provide explicit storage that agents query. Read-only mechanisms such attention over fixed buffers (Oh et al., 2016; Lampinen et al., 2021: Goyal et al., 2022b; Cherepanov et al., 2024b) allow retrieval without modifying previously stored content, while read-write mechanisms (Graves et al., 2016; Zaremba & Sutskever, 2016; Parisotto & Salakhutdinov, 2017) enable dynamic creation, modification, and deletion of memory entries. These architectures support behaviors analogous to episodic recall or map-building,
Memory can also arise without dedicated architectural components. Some agents encode temporal information directly in their action patterns or latent policies, For instance, temporal intervals or event markers can be encoded implicitly through learned action sequences, allowing the agent to solve memory-intenive tasks despite lacking an explicit memory module. A more recent line of work explores autostigmergy (Deverett et al., 2019), in which memory externalized through persistent modifications to the environment or to an auxiliary representational medium. Unlike conventional external buffers, autostigmergic mechanisms allow the agent's 's own interactions to create state traces that influence future decisions, enabling distributed memory without explicit internal storage.
Together, these mechanisms span a wide spectrum, from implicit temporal encoding to explicitly structured memory architectures. They support tasks requiring intra-episode temporal reasoning as well as fast adaptation across tasks. Nonetheless, even when built upon similar foundational architectures, different works operationalize "memory" in distinct ways, leading heterogeneity in evaluation and interpretation.
G CLASSIC BASELINES PERFORMANCE ON THE MIKASA-ROBO BENCHMARK
In this section, we present a comprehensive evaluation of PPO-MLP and PPO-LSTM baselines on our MIKASA-Robo benchmark. Our experiments with PPO-MLP in state mode using dense rewards demonstrate perfect performance across all tasks, consistently achieving 100% success rate, as shown in Figure 10 and Figure 11. This remarkable performance serves as a crucial validation of our benchmark design: when an agent has access to complete state information and receives dense rewards, it can master these tasks completely. Therefore, any performance degradation in RGB+ SGB+joints mode observed with other algorithms or training configurations must stem from the algorithmic limitations or learning challenges rather than any inherent flaws in the task design. This empirical evidence confirms that our environments are well-calibrated and properly designed, establishing a solid foundation for evaluating memory- enhaoce- algorithmss All results are presented mean + standard error of the mean (SEM), where the mean is computed across three independent training runs, and each trained agent is evaluated on 16 different random seeds to ensure robust performance assessment.
27

---

Published as a conference paper at ICLR 2026
The performance evaluation of PPO-MLP and PPO-LSTM with dense rewards in the RGB+ joints mode is presented in Figure 12. This mode specifically tests the agents' memory capabilities, as requires remembering and utilizing historical information to solve the tasks. Our results demonstrate a clear distinction between memory-less and memory-enhance architectures, while also revealing the limitations of conventional memory mechanisms.
Consider the GememberColor-ve environment as an illustrative example. In its simplest config- uration with three cubes, the memory-less PPO-MLP achieves only 25% success rate. In contrast, PPO-LSTM, leveraging its memory mechanism, achieves perfect performance with 100% success rate. However, as task complexity increases five or nine cubes, even the LSTM's memory capabilities prove insufficient, with performance degrading significantly.
These results validate two key aspects of our benchmark: first, its effectiveness distinguishing between memory-less and even sophisticated memory mechanisms as task complexity increases. This demonstrates that MIKASA-Robo provides competitive yet meaningful evaluation framework for developing and testing advanced memory-enhanced agents.
architectures, and second, its ability to challenge
Our evaluation of PPO-MLP and PPO-LSTM baselines under sparse reward conditions in RGB+joints mode reveals the true challenge our benchmark tasks. shown in Figure 13, both architectures even the memory-enhance success rate across nearly all considered environments. This striking result underscores the extreme manipulation tasks when only terminal rewards are available, high-
LSTM - consistently fail to achieve any meaningful
difficulty of memory-intensiee lighting the substantial gap between current algorithms and the level of memory capabilities required for weal-world robotic applications.
H ADDITIONAL OFFLINE RL RESULTS WITH DENSE REWARDS
In the main paper, we focused on the sparse reward setting for Offline RI., which particularly challenging since online RL agents generally ineffective in this regime. To better contextualize the results, we also conducted supplementary experiments in the dense reward setting on a representative subset of tasks: ShellGame Touch, Int perceppteeda um, and RememberCColo33/5/5 We compared four representative algorithms: RATE, DT, BC, and CQL.
Table 6: Performance of Offline RL baselines under dense reward formulation.
Task
RATE
DT
BC
CQL
ShellGameTTouch-v0
0.97=0.02
0.50
0.38+10.03
0.02+10.01
IntercepttMedium-v0
0.39=00066
0.56:10.02
0.51+10.33
0.04:10.01
RememberColor3-v0
0.68=0.04
0.05=10.03
0.20=0004
0,00+0000
RememberColoz5-vo
0.11=0.04
0.05+10.33
0.13:0.02
0.01+0.01
RememmeerColoor9-v0
0.10=0.01
0.02++0.02
0.15++0.03
0.01+0.00
I EXPERIMENTS REPRODUCING AND COMPUTE RESOURCES
All baselines were trained and evaluated under reproducible standardized setup on single NVIDIA A100 GPU. For each task, we conducted three independent training runs. Within each run, evaluation was performed over 100 independent episodes with environment and agent random seeds ranging from 1 to 100. We first computed the mean success rate per run, and then report the overall performance as the mean + standard eтror (SEM) across the three run-level means.
J ADDITIONAL BASELINES: SSMS AND MEMORY-ENHANCEE ENHANCED TRANSFORMERS
To further strengthen our evaluation, we included two additional families of baselines in the offline RL setting: (1) DMamba (Ota, 2024), a recent state-space model designed for efficient long-sequence
28

---

Published as a conference paper at ICLR 2026
ShelltäameTouse
ShellGamoPPshh
ShellGamPPick
Interce Slow
1.00
1,00
1,00
1.00
eucosgg R3h
0.77
state
2 0.75
0.76
Sacoess Rab
0.75
0.50
R 80.50
R0.50 0..50
0.50
0.28
@ 3.25
0.26
0.25
state
state
0.00
0.00
0.00
0.00
Training Slips 167
Training 1
Training Steps 1e7
Triining Stipa 1o7
Inderoep
IntercepiFaas
IntercepiGratSllww
IntercepiGrabMeddi.
1.00
1.00
1.00
1.00
Sirses Rate
0.75
Suswer R
0.76
state
Succiss Rab
state
0,50
0.50
0.50
0.50
0.28
state
0.25
state
0.26
0.25
0.00
0.00
2
0.00
Training Steps 107
Training Sleps 107
Training Steps 167
Training Sieps 1o7
GrtercepdGrabfFael
RemlmbeerColrr
RemmemeerCoor
RememberrCotor
1.00
1.00
1.00
Sucssse Redo
0.75
Supcere 12
0.775
state
0.75
state
Succase Rel
0.75
state
0.50
0.50
0.50
0.50
0.26
state
0.25
0.26
0.25
0.00
2
0.00
2
4
0.00
0
0.00
4 Training Stops 1o5
Training Stops 107
Training Steps 1e5
Training Steps 1e6
Remember Shape3
Serremee
RememberSSapee
1.00
1.00
1.00
Suocsdg Rote
0.75
state
0.75
state
0.75
Supcess Rge
0.75
state
0.50
Sugpees
0.50
Succesg
0.50
0.50
0.25
0.25
0.25
0.25
state
0.00
0.00
0.00
0.00
4
Training Stops
Training Staps
Training Stops
Training Steps 1e6
RotateL anioniPosNag
1.00
1.00
1.00
1.00
Supcesa Rate
0.75
Sueceage Rate
0.75
Suerarg Rate
0.75
state
Success Rete
0.75
state
0.50
0.50
0.50
0.50
0.25
0.25
0.25
0.25
state
state
0.00
0.00
0.00
0.00
Training Sleps 1e6
Training Steps
1e5
Training Steps 1e7
Training Steps 1e7
RolaleStricPoo
RolateStticIPosNeg
TakelBack
1,00
1.00
1.00
0.75
0.75
$0.75
0.50
0.50
0.50
0.25
0.25
0.25
state
state
state
0.00
4 0.00
4
0.00
Traniig Steps 1e7
Traning Steps 107
Training Steps 1e8
Figure 10: Demonstration of PPO-MLP performance on MIKASA Robo benchmark when trained with oracle-level state information. In this learning mode, MDP problem formulation is considered, ie. memory is not required for successful problem solving. At the same time, the obtained results show that it is possible to solve these problems and obtain 100% Success Rate.
modeling, and (2) GTrXL (Parisotto et al., 2020), a gated recurrent transformer variant proposed specifically, adopted to the offline RL setting.
We tested these methods on a representative subset of tasks InterceptMediumg um, and Remembercolor3/5// and compared them against our primary baselines.
Overall, we find that while GTrXL shows some improvements over standard DT and BC base- lines, both it and DMamba still fail to match the performance of RATE model, especially on tasks with higher memory requirements (e.g., RememberColors, 9). These results confirm that
29

---

Published as a conference paper at ICLR 2026
SeqUiColors
SeqOfColorsb
SeqCrColors7
1,00
1.00
1.00
0.75
8 0.75
0.75
0.50
8 3..50
0.25
state
0.25
state
0.28
0.00
0.00
0,00
0.5
1.
Traning Steps 1e7
Traning 1e7
Triining Steps
BundhOrColors
BunchOrColrrs
BunchOrColors7
1.00
1.00
1.00
0.75
0.75
0.75
state
0.50
0.50
0.50
0.25
0.25
0.25
stato
state
Traniig Steps 1e7
Traning Steps 167
Traning Steps 167
ChainOfColore3
ChainOrCollos5
ChaloOOCollrs?
1.00
0.75
0.75
0.76
0.50
80.50
0.50
0.25
state
0.25
state
0.26
state
0.30
0.00
0.00
0.5
1,0
Training Stops te7
Training Steps 1e7
Training Stops 1e6
Demonstration of PPO-MLP performance on MIKASA-Robo benchmark when trained with oracle-level state information. Results are shown for memory capac- ity (SeqOfColors 5, 5, 7]-00, BunchOfColors [3, 7] -vo) and sequential memory (ChainOfColors(3.3 5, 7] -v)).
Figure 11:
RemmemberColor3-0
RemmemberDoloo--00
RemeemberrCooo0-000
TakeliBack
1.00
Pro-rup
LITTM
Roamm
0.75
SAC-ag
0.75
SAOREP
0,75
3AC-UUF
Susass pe
0.75
MLP
0.550
0.50
0.50
0.60
0.26
0.25
0
2
0.00g
0.00g
0.00
2
Training Steps 1o6
Training Steps 1es
Training Steps 1e6
Training Steps 1o7
InterceptSiow
InterceplMeddum-000
InterceptGrabMMediuu
RememberShapee
1.00
1.00
1.00
Suo0ese Roee
0.75
MLP
0.75
Suscess Raje
0.75
MLP
Supcass R36
0.75
MLP
0.50
0.50
0.25
0.25
0.25
0.25
0.00
0.00
0.00
2
0.00
Training Steps 1e7
Training Steps
Training Steps
Training Steps
ShalGamm Touchha
CectateLeriertfe
Rotatel onieniPosNeg
1,00
1.00
1.00
1.00
Bloowd Bate
0.75
MLP
LSTM
0.75
LSTM
MLP
Success Rate
0.75
0.50
0.50
WOML
3.500
0.50
LSTT
0.25
0.25
0.25
0.25
MLP
0.00
0.0g.3
0:00
0.00
Training
Traning Steps 1e5
Training Steps 167
Training Sleps 1e7
Figure 12: Performance evaluation of PPO-MLP and PPO-LSTM on the MIKASA-Robe benchmark using the "RGB+joints" training mode with dense reward function, where the agent only receives images from the camera (from above and from the gripper) and information about the state of the joints (position and velocity). The results demonstrate that numerous tasks pose significant challenges even for PPO-LSTM agents with memory, establishing these environments as effective benchmarks for evaluating advanced memory-enhanced architectures.
memory-centrie SSM and Transformer variants remain challenged by increasing sequence complexity, underscoring the importance of dedicated mechanisms for continual memory retention and rewriting.
30

---

Published as a conference paper at ICLR 2026
RememberColord
RememberColarrs
RamemberCCaoon
InterooptMedium
1.00
1,00
1,00
1.00
8ac0sng R3h
0.78
MLP
LSTT
2 0.75
MLP
LSTT
0.76
LSTM
MLF
Sacoerss Rab
0.75
LISTM
MLP
0.50
$0.50
0.550
0.50
0.28
@ 0.25
0.26
0.25
0.00
0.00
Training Slips 167
Training
Training Steps 167
Triining Stspa
Remember Shape3
RodaleLemien/PosNeg
ShellGame Touch
1.00
1.00
1.00
1.00
Bleseca Rale
0.75
LSTM
LSTM
0.75
LSTM
Supsiss 122
0.75
LSTT
0.50
0.50
0.50
0.50
0.28
0.25
0.26
0.25
0.00
Training Steps 167
Training Steps 1e5
Training Staps 167
Training Steps 1e6
Figure 13: Performance evaluation of PPO-MLP and PPO-LSTM on the MIKASA-Robo benchmark using the "RGB+joints" with sparse reward function training mode, where the agent only receives images from the camera (from above and from the gripper) and information about the state of the joints (position and velocity). This training mode with sparse reward function causes even more difficulty for the agent to learn, making this mode even more challenging for memory-enhanced agents.
Table 7: Offline RL performance of additional SSM/Transformer baselines (DMamba, GTrXL) compared with prior models.
Task
RATE
DT
BC
CQL.
DP
DMamba
GTrXI.
She 1ameeTouhh-v
0.92+00.01
0.53=20.07
0,28=0.01
0.16:00.04
0.18:00.02
0.21=0.02
0.80 10
intercepttMediummoo
0.09++0.03
0.56+0.01
0.31+0.14
0.03++0.01
0.24++0.1
0.14-0.07
0.64+0.04
RememberCColor3-v0
0.65+10.04
0.01:10.01
0.27=0.03
0.29::0.01
0.07:10.4
0.32=0003
0.39+10.06
RememberrColor5--t0
0.1310.03
0.07:10.05
0.1210002
0.15110.2
0.0111001
0.11:0.02
0.1910.04
RemmemberColoo99-v0
0.0910.02
0.0110.01
0.12=0.02
0.15:0.01
0.02:10.1
0.14=0.00
0.1610.01
K MIKASA-ROBO DETAILED TASKS DESCRIPTION
In this section, we provide comprehensive descriptions of the 32 memory-intensive tasks that comprise the MIKASA-Robh benchmark. Each task is designed to evaluate specific aspects of memory capabilities in robotic manipulation, ranging from object tracking and spatial memory to sequential decision-making For each task, detail its objective, memory requirements, observation space, reward structure, and success criteria. Additionally, we explain how task complexity increases across different variants and discuss the specific memory challenges they present. The following subsections describe each task category and its variants in detail.
Each of the proposed environment supports multiple observation modes:
State: Full state information including ball position
RGG++joint:: Two camera views (top-down and gripper) plus robot joint states
RGB:: Only visual information from two cameras
In the case of RotateLen i ent-v0 and RotateSt riveettriccte rittatttttctttv., the prompt information available at each step is additionally added to each observation.
31

---

Published as a conference paper at ICLR 2026
Table 8: Results for Offline RL baselines. The table shows comparison of transformer-based baselines (RATE, DT), behavior cloning (BC), classic Offline RL baselines (CQL), and Diffusion Policy (DP) all 32 tasks from the MIKASA- sem across the three runs, where each run is averaged over 100 episodes and sem is the standard error of the mean. Training gripper view) and using sparse rewards (success once condition). The results show that even models with memory (RATE, DT) are not able solve most of the benchmark problems, which makes it challenging and promising for further validation of the algorithm.
benchmark. Results are presented mean +
performed using only RGB observations (two cameras: top view and
# Environment
RATE
DT
BC
CQL
DP
1
ChellGGaeeTouch-eeh
0.92+0.01
0.53+0.07
0.28+0.01
0.1620.04
0.1820.02
2
ShcllGamePPsh-vO
0.78+0.06
0.62#0.14
0.27=0.01
0.25+0.01
0.22+0.03
3
ShellGamePick--vO
0.02+0.01
0.00=0.00
0.01
0.00=0.00
0.01
4
InterceptSlow-vO)
0.23+0.02
0.40=0.02
0.37+0.06
0.25+0.01
0.33+0.05
5
InterreptMedhun--00
0.32+0.02
0.56#0.01
0.31
0.03#0.01
0.68#0.02
6
InterceptFast-vó
0.30=0.04
0.3620.04
0.03=0.02
0.02=0.02
0.21
InterceptGrabSloo-v0
0.09=0.03
0.00=0.00
0.28=0.18
0.03=0.00
0.03+0.01
8
IntereppGrrabMccium--0
0.09=0.03
0.00=0.00
0.11=0.02
0.08=0.04
0.03=0.01
IntoroepiGrabFaatt-00
0.14+0.03
0.11#0.03
0.09=0.02
0.08#0.03
0.1820.02
10
RotateLcnmiemtPo--v00
0.11
0.01
0.15#0.03
0.16#0.02
0. 0..100
RotateLeniienPPooNee--W
0.29+0.03
0.05+0.02
0.22+0.01
0.12+0.02
0. 0.14+0.05
12
RotateStrictPos-vO
0.03+0.02
0.0520.04
0.01
0.03#0.01
0.06#0.02
RotateStriitPPsNegg-o
0.08+0.01
0.0520.03
0.04
0.0420.02
0.15+0.01
14
TakeltBack-vd
0.42+0.24
0.0820.04
0.33+0.10
0.0420.01
0.05#0.02
RememberColor3--x0
0.65+0.04
0.01
0.27+0.03
0.29#0.01
0.32+0.01
16
RememberCollo55-00
0.13#0.03
0.07=0.05
0.1200.01
0.15=0.02
0. 00#00.22
RememberColor9-v00
0.09=0.02
0.01
0.1240.02
0.15#0.01
0.17+0.01
18
RememberShape3--00
0.21
0.05=0.04
0.31
0.20=0.10
0.32+0.05
RememberShape5--vd
0.1720.04
0.0420.04
0.18#0.01
0.15#0.00
0.2140.04
20
RememberShape9--v0
0.05+0.00
0.05+0.02
0.1000.02
0.1420.01
0.11
21
0.14#0.02
0.04=0.02
0. 0.100..02
0.11
0. 0..420.02
22 Ceneembershuppchnmcooooosxx---
0.08=0.03
0.06#0.06
0.09#0.02
0.09=0.02
0. 16+0.01
23
RemeebbesShapcchndcoolossxxx
0.07+0.02
0.01
0.09#0.01
0.09=0.02
0. 10.003
24
BunchOrColons3-v0)
0.00=0.00
0.00=0.00
0.00=0.00
0.00=0.00
0.00=0.00
25
BunchOrCollorS-vv)
0.00=0.00
0.00=0.00
0.00=0.00
0.00=0.00
0.00+0.00
26 BunchOfColoos7-vv0
0.00=0.00
0.00=0.00
0.00=0.00
0.00=0.00
0.00=0.00
27
Seq
0.00=0.00
0.00=0.00
0.00=0.00
0.00=0.00
0.00=0.00
28
ScqOfColors5-v0
0.00=0.00
0.00=0.00
0.00=0.00
0.00=0.00
0.00=0.00
29
ScqOfColors7-v0
0.00=0.00
0.00=0.00
0.00=0.00
0.00=0.00
0.00=0.00
30
Cha
0.00=0.00
0.00=0.00
0.00=0.00
0.00=0.00
0. 0.00=0.00
31
ChainOfColonss-v00
0.00=0.00
0.00+0.00
0.0020.00
0.00=0.00
0.00=0.00
32
ChainOfColons7-v0
0.00=0.00
0.00=0.00
0.00=0.00
0.00=0.00
0.00=0.00
32

---

Published as a conference paper at ICLR 2026
Step: 0 Forget: Left
Step: 5
Step: 25 Target: Left
Target: Left
Reword: 0.079
Reword: 1.000
Figure 14:
She 1 Game Touch-vO: The robot observes ball in front of it. next, this ball is covered by mug and then the robot has to touch the mug with the ball underneath.
K.1 SHELLGAME-VO
The Shel Game v0 task (Figure 14) is inspired by a simplified version of the classic shell game, which tests person's 's ability to remember object locations when they become occluded. This task evaluates an agent's capacity for object permanence and spatial memory, crucial skills for real world robotic manipulation where objects frequently become temporarily hidden from view.
Environment Description The environment consists of three identical mugs placed on a table and red ball. The task proceeds three phases:
1. Observation Phase (steps 0-4): The ball is placed one of three positions, and the agent can observe its location.
2. Occlusion Phase (step 5): The ball and positions are covered by three identical mugs.
3. Action Phase (steps 6+): The agent must interact with the mug covering the ball's location. The type of target interaction depends on the selected mode: Touch, Push and P Pice ck.
Task Modes
The task includes three variants of increasing difficulty:
Touch:
The agent only needs to touch the correct mug
Push: The agent must push the correct mug to designated area
Pick: The agent must pick and lift the correct mug above specified height
Success Criteria Success is determined by:
Touch: Contact between the gripper and the correct mug
Push: Moving forward the correct mug to the target zone
Pick: Elevating the correct mug above 0.1m
Reward Structure
The environment provides both sparse and dense reward variants:
Sparse: Binary reward (1.0 for success, 0.0 otherwise)
Dense: Continuous reward based on:
Distance between gripper and target mug
Robot's 's motion smoothness (static reward based on joint velocities)
Task completion status (additional reward when the task solved)
33

---

Published as a conference paper at ICLR 2026
Step: Torget:
Stept Torget:
Step: 10 Torget:
Stap: 50 Target:
Figure 15: RememberColorg-eeee rColor9-vo. The robot observes a colored cube in front of it, then this cube disappears and an empty table is shown. Then 9 cubes appear on the table, and the agent must touch cube of the same color as the one it observed at the beginning of the episode.
K.2 REMEMBERCOLLOR-VO
The RememberColor task (Figure 15) tests agent's ability to remember and identify objects based on their visual properties. This capability is essential for real-world robotics applications where agents must recall and match object characteristies across time intervals.
Environment Description The environment presents a sequence of colored cubes on a table. The task proceeds in three phases:
1. Observation Phase (steps 0-4): A cube of specific color is displayed, and the agent must memorize its color.
2. Delay Phase (steps 5-9): The cube disappears, leaving an empty table.
3. Selection Phase (steps 10+): Multiple cubes of different colors appear (3, or depending on difficulty). and the agent must identify and interact with the cube matching the original color.
Task Modes
The task includes three complexity levels:
3 (casy): Choose from 3 different colors (red, lime, blue)
5 (Medium):
Choose from 5 different colors (red, lime, blue, yellow, magenta)
9 (Hard); Choose from different colors (red, lime, blue, yellow, magenta, cyan, maroon, olive, teal)
Success Criteria Success is determined by:
Correctly identifying and touching the cube that matches the color shown in the observation phase
Maintaining contact with the correct cube for at least 0.1 seconds
Reward Structure
The environment provides both sparse and dense reward variants:
Sparse: Binary reward (1.0 for success, 0.0 otherwise)
Dense: Continuous reward based on:
Distance between gripper and target cube
Robot's s motion smoothness (static reward based on joint velocities)
Additional reward for robot being static while touching the correct eube
Task completion status (additional reward when the task solved)
34

---

Published as a conference paper at ICLR 2026
Stap: Forget:
Stezs
Step: 10 Target: apherg
Shap: 47 Torget: aprere
aphere
Torget: sphere
0,000
Figure 16: Remember 0- The robot observes an object with specific shape in front of it, then the object disappears and empty table appears. Then objects different shapes appear on the table, and the agent must touch object of the same shape the one it observed at the beginning of the episode.
K.3 REMEMBERSHAPE--O
The Remember Shape-v0 task (Figure 16) evaluates an agent's ability to remember and identify objects based on their geometric properties. This capability crucial for robotic applications where shape recognition and recall are essential for successful manipulation.
Environment Description The environment presents sequence of geometric shapes on table. The task proceeds in three phases:
1. Observation Phase (steps 0-4): A shape (cube, sphere, cylinder, etc.) is displayed, and the agent must memorize its geometry.
2. Delay Phase (steps 5-9): The shape disappears, leaving an empty table.
3. Selection Phase (steps 10+): Multiple shapes appear (3, 5, or depending on difficulty),
and the agent must identify and interact with the shape matching the original geometry.
Task Modes The task includes three complexity levels:
3 (Easy): Choose from 3 different shapes (cube, sphere, cylinder)
5 (Medium):
Choose from 5 different shapes (cube, sphere, cylinder cross, torus)
9 (Hard): Choose from 9 different shapes (cube, sphere, cylinder cross, torus, star, pyramid, t-shape, crescent)
Success Criteria
Success is determined by:
Correctly identifying and touching the object with the same shape shown in the observation phase
Maintaining contact with the correct shape for at least 0.1 seconds
Reward Structure
The environment provides both sparse and dense reward variants:
Sparse: Binary reward (1.0 for success, 0.0 otherwise)
Dense: Continuous reward based on:
Distance between gripper and target object
Robot's ss motion smoothness (static reward based on joint velocities)
Additional reward for maintaining static position when touching correct object
Task completion status (additional reward when the task solved)
35

---

Published as a conference paper at ICLR 2026
Staps
Staps Torget:
Stap: Torget:
Slep: 30
Farget:
:
Target:
Reward:
An object of certain shape and color appears front of the agent. Then the object disappears and the agent sees an empty table. Then objects of different shapes and different colors appear the table and the agent has to touch what observed at the beginning of the episode.
Figure 17: Searembbe:ShappendCoollooo
K.4 REMEMBBRSHAPPANNDCOLORRWW
The wwwp:nbbeesaapeednnno...
task (Figure 17) evaluates an agent's ability to remember and identify objects based on multiple visual properties simultaneously. This task combines shape and color recognition, testing the agent's capacity to maintain and match multiple object features across time intervals.
Environment Description The environment presents sequence of colored geometric shapes table. The task proceeds in three phases:
1. Observation Phase (steps 0-4): An object with specific shape and color is displayed, and the agent must memorize both properties.
2. Delay Phase (steps 5-9): The object disappears, leaving an empty table.
3. Selection Phase (steps 10+): Multiple objects with different combinations of shapes and colors appear, and the agent must identify and interact with the object matching both the original shape and color.
Task Modes nations:
The task includes three complexity levels based on the number of shape-color combi-
3x2 (Easy): Choose from 6 objects (3 shapes x colors); shapes: cube, sphere, t-shape; colors: red, green
3x3 (Medium): Choose from 9 objects (3 shapes x 3 colors); shapes: cube, sphere, t-shape; colors: red, green, blue
5x3 (Hard): Choose from 15 objects shapes x colors); shapes: cube, sphere, t-shape, cross, torus; colors: red, green, blue
Success Criteria Success is determined by:
Correctly identifying and touching the object that matches both the shape and color shown in the observation phase
Maintaining contact with the correct object for at least 0.1 seconds
Reward Structure
The environment provides both sparse and dense reward variants:
Sparse: Binary reward (1.0 for success, 0.0 otherwise)
Dense: Continuous reward based on:
Distance between gripper and target object
Robot's 's motion smoothness (static reward based on joint velocities)
Additional reward for maintaining static position while touching correct object
Task completion status (additional reward when the task solved)
36

---

Published as a conference paper at ICLR 2026
Step:
Step: 27
Step: 41
Reward: 0.051
Reward: 1,000
Figure 18: InterceptMedium-von ball rolls on the table in front of the agent with a random initial velocity, and the agent's task is to intercept this ball and direct it at the target zone.
K.S INTERCEPT-VO
The Intercept vv task (Figure 19) evaluates agent's 's ability predict and intercept moving object based on its initial trajectory. This task tests the agent's capacity for motion prediction and spatial-temporal reasoning, which are essential skills for dynamic manipulation tasks in robotics.
Environment Description
The environment consists of red ball moving across a table and target zone. The task requires the agent
1. Observe the ball's initial position and velocity
2. Predict the ball's trajectory
3. Guide the ball to reach a designated target zone
Task Modes
The task includes three variants with increasing ball velocities:
Slow: Ball velocity range of 0.25-0.5 m/s
Medium: Ball velocity range of 0.5-0.75 m/s
Fast: Ball velocity range of 0.75-1.0 m/s
Success Criteria Success is determined by:
Guiding the ball to enter the target zone
The ball must come rest within the target area (radius 0.1m)
Reward Structure The environment provides both sparse and dense reward variants:
Sparse: Binary reward (1.0 for success, 0.0 otherwise)
Dense: Continuous reward based
Distance between gripper and ball
Distance between ball and target zone
Robot's motion smoothness (static reward based on joint velocities)
Task completion status (additional reward when the task solved)
37

---

Published as a conference paper at ICLR 2026
Step:
Step: 15
Step: 31
Reward: 0.018
Reward: 1.006
Figure 19: Interceptorabbeediun-o A ball rolls on the table in front of the agent with random initial velocity, and the agent's task is to intercept this ball with a gripper and lift it up.
K.6 INTERCHPTGRAB-VO
The InterceptGrab-ye task (Figure 19) extends the Intercept vv task by requiring the agent to not only predict the trajectory moving object but also grasp it while in motion. This task evaluates the agent's ability combine motion prediction with precise manipulation timing, simulating real-world scenarios where robots must catch or intercept moving objects.
Environment Description The environment consists of a red ball moving across a table. The task requires the agent to:
1. Observe the ball's initial position and velocity
2. Predict the ball's trajectory
3. Position the gripper intercept the ball's path
4. Time the grasping action correctly to catch the ball
5. Maintain stable grasp while bringing the ball to rest
Task Modes
The task includes three variants with increasing ball velocities:
Slow: Ball velocity range of 0.25-0.5 m/s
Medium um: Ball velocity range of 0.5-0.75 m/s
Fast: Ball velocity range of 0.75-1.0 m/s
Success Criteria Success is determined by:
Successfully grasping the moving ball
Maintaining stable grasp until the ball comes
The robot must be static with the ball firmly grasped
Reward Structure
The environment provides both sparse and dense reward variants:
Sparse: Binary reward (1.0 for success, 0.0 otherwise)
Dense: Continuous reward based on:
Distance between gripper and ball
Grasping reward
Robot's motion smoothness (static reward based on joint velocities)
Task completion status (additional reward when the task solved)
38

---

Published as a conference paper at ICLR 2026
Shap: Current: 0.0
Stap: deg
Step: deg Torget
Slep: 30
deg
Torget
0.0
40,22
Qurrank: 7723
Reward:
Figure 20: SotateLenient.os--e A randomly oriented peg is placed in front of the agent. The agent's task is to rotate this peg by a certain angle (the center of the peg can be shifted).
K.7 ROTATELENIENT-VO
The RotateLenient
task (Figure 20) evaluates an agent's ability to remember and exccute specific rotational movements. This task tests the agent's capacity to maintain and reproduce angular information, which is crucial for manipulation tasks requiring precise orientation control. This task the agent's ability to hold information in memory about how far peg has already rotated the current step relative to its initial position.
Environment Deseription The environment consists of blue-colored peg on table that must be rotated by specified angle. The task proceeds in one phase, but the static prompt information about the target angle is available to the agent at each timestep:
1. Action Phase: The agent rotate the peg to match the target angle
Task Modes The task includes two variants with different rotation requirements:
Pos: Rotate by a positive angle between 0 and n//2
PosNeg: Rotate by either positive or negative angle between -x/4 and T/4
Success Criteria Success is determined by:
Rotating the peg to within the angle threshold (+0.1 radians) of the target angle
Maintaining the final orientation in stable position
The robot must be static with the peg at the correct orientation
Reward Structure The environment provides both sparse and dense reward variants:
Sparse: Binary reward (1.0 for success, 0.0 otherwise)
Dense: Continuous reward based on:
Distance between gripper and peg
Angular distance to target rotation
Stability of the peg's orientation
Robot's 's motion smoothness (static reward based on joint velocities)
Task completion status (additional reward when the task solved)
39

---

Published as a conference paper at ICLR 2026
Step:
Step: 24
Step: 56 Target 89. 19.75 deg Current: deg Reward: 0.983
Target 89.75 deg
Target 89.75 deg Current: 38.87 deg
Current: 0.0 deg
Reward: 0.006
Figure 21: RotatestrictPos-cC: randomly oriented peg is placed in front of the agent. The agent's task is to rotate this peg by certain angle (it is not allowed to move the center of the peg)
K.8 ROTATESTRICT-VO
The Rot SotttStrict-ve task (Figure 21) extends the Rotatelenient--ve task with more stringent requirements for precise rotational control.
Environment Description The environment consists of blue-colored peg on table that must be rotated by a specified angle while maintaining its position. The task proceeds in one phase, but the static prompt information about the target angle is available the agent at each timestep:
1. Action Phase:
The agent must rotate the peg match the target angle while keeping centered
Task Modes
The task includes two variants with different rotation requirements:
Pos: Rotate by positive angle between 0 and x/2
PosNeg:
Rotate by either positive or negative angle between -x/4 and π 7/4
Success Criteria Success is determined by:
Rotating the peg within the angle threshold (+0.1 radians) of the target angle
Maintaining the peg's position within 5em of its initial XY coordinates
The robot must be static with the peg at the correct orientation
No significant deviation in other rotation axes
Reward Structure
The environment provides both sparse and dense reward variants:
Sparse: Binary reward (1.0 for success, 0.0 otherwise)
Dense: Continuous reward based on:
Distance between gripper and peg
Angular distance to target rotation
Position deviation from initial location
Stability of the peg's orientation
Robot's motion smoothness (static reward based on joint velocities)
Task completion status (additional reward when the task solved)
40

---

Published as a conference paper at ICLR 2026
Mtyp
Slep
Styp: 3V
Stept 36
Sher
Figure 22:
Take Backk v0: The agent observes a green cube in front of him. The agent's task the green cube the red target, and as it lights up violet, return the cube its original position (the agent does not observes the original position of the cube).
K.9 TAKEITBACK-VO
The TakeItBack-vó task (Figure 22) assesses the agent's ability to perform sequential tasks and memorize the starting position. This task tests the agent's capacity for sequential memory and spatial reasoning, requiring maintain information about past locations and achievements while executing a multi-step plan.
Environment Description The environment consists of green cube and two target regions (initial and goal) on a table. The task proceeds in two phases;
First Phase: The agent must move the cube from its initial position to a goal region
2. Second Phase:
After reaching the goal, goal region change it's color from red to magenta, and the agent must return the cube to its original position (marked by the initial region and invisible for the agent)
Success Criteria
Success is determined by:
First reaching the goal region with the cube
Then returning the cube to the initial region
Both goals must be achieved in sequence
Reward Structure
The environment provides both sparse and dense reward variants:
Sparse: Binary reward (1.0 for success, 0.0 otherwise)
Dense: Continuous reward based on:
Distance between gripper and cube
Distance to current target region
Progress through the task sequence
Stability of cube manipulation
Robot's motion smoothness (static reward based on joint velocities)
Task completion status (additional reward when the task solved)
41

---

Published as a conference paper at ICLR 2026
Terget
Seer
Steer MA
Tou
soarert
Figure 23: SogofColors7--oe In front of the agent, 7 cubes of different colors appear sequentially. After the last cube is shown, the agent observes empty table. Then cubes of different colors appear on the table and the agent has to touch the cubes that were shown at the beginning of the episode in any order.
K.10 S#QOFCOLORS--0
The SegOfColors- task (Figure 23) evaluates an agent's ability to remember and reproduce an unordered sequence of colors. This task tests memory capacity capabilities essential for robotic tasks that require following specific patterns or sequences.
Environment Description The environment presents sequence of colored cubes that must be reproduced in any order. The task proceeds in two phases:
Observation Phase (steps 0-(5 L)): A sequence of colored cubes is shown one at time, with each cube visible for 5 steps.
2. Delay Phase (steps (5N))(55 + 4)): All cubes disappear
3. Selection Phase (steps (5N + 5)+): A larger set of cubes appears, and the agent must identify and touch all previously shown cubes in any order
Task Modes
The task includes three complexity levels:
3 (Fasy):
Remember 3 colors demonstrated sequentially
Remember colors demonstrated sequentially 7 (Hard); Remember 7 colors demonstrated sequentially
5 (Medium):
Success Criteria Success is determined by:
Correctly identifying and touching all cubes from the observation phase
Order of selection doesn't 't matter
Each cube must be touched for at least 0.1 seconds
The demonstrated must be touched without any mistakes
Reward Structure
The environment provides both sparse and dense reward variants:
Sparse: Binary reward (1.0 for success, 0.0 otherwise)
Dense: Continuous reward based on:
Distance between gripper and next target cube
Number of correctly identified cubes
Static reward for stable contact
Robot's 's motion smoothness (static reward based on joint velocities)
Task completion status (additional reward when the task is solved)
42

---

Published as a conference paper at ICLR 2026
Terge
See 10
Seer
soacace
Figure 24: BunchofColors7-v00 cubes of different colors appear simultancously in front of the agent. After the agent observes empty table. Then, 9 cubes of different colors appear on the table and the agent has to touch the cubes that were shown at the beginning of the episode in any order.
K.11 BUNCHOFCOLORSS-VO
The BunchOfColors-y task (Figure 24) tests an agent's 's memory capacity by requiring it to remember multiple objects simultaneouslyy. This capability is crucial for tasks requiring parallel processing of multiple items.
Environment Description The environment presents multiple colored cubes simultaneously. The task proceeds in three phases:
Observation Phase (steps 0-4): Multiple colored cubes are displayed simultaneously
2. Delay Phase (steps 5-9): All cubes disappear
3. Selection Phase (steps 10+): A larger set of cubes appears, and the agent must identify and touch all previously shown cubes in any order
Task Modes
The task includes three complexity levels:
3 (Easy):
Remember 3 colors demonstrated simultaneously
Remember colors demonstrated simultaneously 7 (Hard); Remember 7 colors demonstrated simultaneously
5 (Medium):
Success Criteria Success is determined by:
Correctly identifying and touching all cubes from the observation phase
Order of selection doesn't 't matter
Each cube must be touched for at least 0.1 seconds
The demonstrated set must be touched without any mistakes
Reward Structure
The environment provides both sparse and dense reward variants:
Sparse: Binary reward (1.0 for success, 0.0 otherwise)
Dense: Continuous reward based on:
Distance between gripper and next target cube
Static reward for stable contact
Number of correctly touched cubes
Robot's motion smoothness (static reward based on joint velocities)
Task completion status (additional reward when the task solved)
43

---

Published as a conference paper at ICLR 2026
light sceo01
In front of the agent, 7 cubes of different colors appear sequentially. After the last cube is shown, the agent sees an empty table. Then 9 cubes of different colors appear on the table and the agent must unmistakably touch the cubes that were shown the beginning of the episode, in the same strict order.
Figure 25: ChainOfcolors?
K.12 CHAINOFCOLLORS-V
The Cha i wwwnnoooorrsssoo task (Figure 25) evaluates the agent's ability store and retrieve ordered information. This task simulates scenarios where the agent must track changing relationships between objects over time.
Environment Description The environment presents am ordered sequence (chain) of colored cubes that must be followed. The task proceeds multiple phases:
1. Observation Phase (steps 0-65N 1)): A sequence of N colored cubes is shown at time, with each cube visible for 5 steps.
2. Delay Phase (steps (5N))(5N + 4)): All cubes disappear
3. Selection Phase (steps (5NV 5)+): A larger set of cubes appears, and the agent must identify and touch all previously shown cubes in the exact order as demonstrated
Task Modes
The task includes three complexity levels:
3 (Easy):
Remember 3 colors demonstrated sequentially
5 (Medium):
Remember colors demonstrated sequentially
7 (Hard): Remember 7 colors demonstrated sequentially
Success Criteria
Success is determined by:
Correctly identifying and touching all cubes from the observation phase in the exact order
Each cube must be touched for at least 0.1 seconds
The demonstrated set must be touched without any mistakes
Reward Structure
The environment provides both sparse and dense reward variants:
Sparse: Binary reward (1.0 for success, 0.0 otherwise)
Dense: Continuous reward based
Distance between gripper and next target cube
Static reward for stable contact
Number of correctly touched cubes
Robot's 's motion smoothness (static reward based on joint velocities)
Task completion status (additional reward when the task is solved)
44

---

Published as a conference paper at ICLR 2026
Table 9: Classification of environments from the MIKASA-Base benchmark according the sug- gested memory- inttrrve tasks classification from the Section 4.2.
Emfronment
Memory Task
Brief description
Obserrativn Space
Space
Menury Cadds
Capuaity
Manosiac the poslions of revenled cach and cvurestly much pain whil minimising
vestur
discrdt
incorrect
Numpad
Sequentall
Memorise sequence
savigate rolling ball on 3xd and
discrete, continuous
following
order while aooldng musalles
BSuic Menury Length
Objas
Manotize iniial signal and
given sunker skep take
disende
Minigidd Memory
Object
Memoriz object surting zoom use information select
insgge
discrete
cortect path junction.
Balla
Seguntiid
Memuriae the sequemee nevements perfoenual by wach uniguely soloned and
imagee
diserde
Otfectt
staped cancer. thes stemmit and approach the dincer who executed the piom pattem. Memonze the orrpet color dispayed the wall dhring the intaal plase. After a brief distracoor phase. ideniiy and select the tugget color among the distractees by oupping ou the coenop-odinng goond jeal.
Paecive Viaut Match
Obgoct
imege
discrito
Passie----Me Sesse---Maaze
Obyect
Memonze the goal 's locatin upon minii otservation, navigate through the maze
vector
discrete
with limited sernery input, and select the comect poth at the junction.
ViZDDoom two coloes
Object
Memoria the color of the bitefly ippeariin pillar (green red) and collect hens of same colur survive the audd-iilial room.
Inage
discrete
Menury Maze
Spatia
Mamurias the localius of objects and the stucture wsing vivual class then
image
diserdte
renigeee officennlly find objects spectfic oolor and soom
MemoryCym Moctar Mayhem
sequeace
and then coneec order.
Insge
discrete
Seguzniial
MemusyGym Mystary
Capaaity
Manorisc invisille pach sarigaic withoun slepping
image
disende
Sponal
POPCym Repeat Flast
Object
Memorie the initial peesemed
step and recall corneddl ufter
vector
discrete
recelving sequence random values.
POPGym Repeai Presinus
Segunniial
Memusiz the value olaarea at cado step and rocall volue from it sleps cellier
vestur
disende
Object
when required
POPGym Autocncode
Sequemial
Memusize the sequeace of cards presente at the beginning and reproduce them in the sume oeder when required.
vector
discrete
POPGym Count Recall
Objas,
Menuria unique vallus uncmuntzred and count how man times specific value has appeared
vestor
disendie
Capocit
POPCyym vectorless Campole
Sequeneal
Memonse velecity dats over time and imegrat it inter the postion the poir for vector balance conzel.
continioss
POPCym vockenlows Pandulum Sequziiial
Manoria anguhe sekicity time and imngrtic to infer the passattn'' is posiioo control.
cuntimans
for succesful
POrCyym Mulliammed Bandit
Object, Capacity
Memonae reuard protehilimes of cifierent slor machines exploring them
discrete
ideatit the with the highest expected find posiion
POPGym Concentraion
Capaaity
pesiinsly
disende
puice.
POPOym Banteehhi
Sponia
Memonie cocedinates presious
their or feedrack
in internal ispresentation
avoid repeat shoes, strucgicalll target
ships for maximun rewards.
POPGym Min: Suceper
Syatial
Mamorizu rovcaled yri information and uue nameical infer tike while
vester
diserde
rvoiding mines,
POPGym Labylinh Explace
Syutil
Memorize previsusly visted cells and nasigate the mase efficieuly to discoves sew, mexgloord ueess and muximize rewards.
vector
discrete
POP Labyrith Escape
Spatial
Momoriz the more bysst while exploring and movignn efficinnly to find the cait and recerve a reward.
vestor
disendie
POPOym Hligher Lower
Object.
Memonze prevonucly revenled card minks and predier whether the west card will
vector
discrete
Soqueniaal
be higher or lower, updating the reference card after each prediction maximize wwards.
L MIKASA-BASE BENCHMARK TASKS DESCRIPTION
This section provides a detailed description of all environments included in the MIKASA- benchmark Section 5. Understanding the characteristics and challenges of these environments is crucial for evaluating RL algorithms. Each environment presents unique tasks, offering diverse scenarios test the memory abilities of RL agents.
L.1 MEMORY CARDS
The Memory Cards environment (Esslinger al., 2022) is memory game environment with randomly shuffled pairs of hidden cards. At each step, the agent sees one revealed card and must find its matching pair. correct guess removes both cards; otherwise, the card is hidden again, and a new one is revealed. The game continues until all pairs are removed.
L.2 NUMPAD
The Numpad environment (Humplik et al., 2019) consists of an N x N grid of tiles. The agent controls a ball that rolls between tiles. At the beginning of an episode, a random sequence of n neighboring tiles (excluding diagonals) is selected, and the agent must follow this sequence in the correct order. The environment is structured so that pressing the correct tile lights it up, while pressing an incorrect tile resets progress. A reward of +1 is given for the first press of each correct tile after a reset. The episode ends after a fixed number of steps. To succeed, the agent must memorize the sequence and navigate it correctly without mistakes. The ability to "jump" over tiles is not available.
45

---

Published as a conference paper at ICLR 2026
L.3 BSUITE MEMORYLENGTH
The MemoryLength environment (Osband et al., 2020) represents sequence of observations, where each step. the observation takes value of either +1 or -1. The environment is structured so that reward is given only at the final step if the agent correctly predicts the i-th value from the initial observation vector obs. The index of this i-th value is specified the last step observation vector in [bs[1]] To succeed, the agent must remember the sequence of observations and use this information to make an accurate prediction at the final step.
L.4 MINIGRID-MEMORE
Minigrid-Memont (Chevalier- theevvlier-Boisvertt et al., 2023) is two-dimensional grid-based environment that features T-shaped maze with small room at the beginning of the corridor, containing object. The agent starts at a random position within the corridor. Its task to reach the room, observe and memorize the object, then proceed the junction the maze's 's end and where an identical object is located. The reward function is defined as Rg = I - 0.9 x T for successful attempt: otherwise, the agent receives zero reward. The episode terminates when the agent makes a choice at the junction or exceeds time limit of steps.
towards the direction
L.5 BALLET
In the Ballet environment (Lampinen et al., 2021) tasks take place in an 11 11 tiled room, consisting of x central area surrounded by a one-tile-wide wall. Each tile is upsampled to 9 pixels, resulting in 99 x 99 pixel input image. The agent is initially placed at the center of the room, while dancers are randomly positioned one and color, selected from 15 possible shapes and 19 colors, ensuring uniqueness. These visual features serve only for identification and do not influence behavior. The agent itself is always represented as a white square. The agent receives egocentric visual observations, meaning its view is centered its own position, which has been shown to enhance generalization.
possible locations around it. Each dancer has a distinct shape
L.6 PASSIVE T-MAZE
The Passive T-Maze environment (Ni et al., 2023) consists of corridor leading to junction that connects two possible goal states. The agent starts designated position and can move in four directions: left, right, up, down. At the beginning of each episode, one of the two goal states is randomly assigned as the correct destination. The agent observes this goal location before starting its movement. The agent stays in place if it attempts to move into a wall. To succeed, the agent must navigate to the correct goal based on its initial observation. The optimal strategy involves moving through the corridor towards the junction and then selecting the correct path.
L.7 ViZDooM-TTo-CCLLO.
The WiZDeom-Two-Colon Two-Colors (Sorokin et al., 2022) is environment where agent is placed in room with constantly depleting health. The room contains red and green objects, one of which restores health (+1 reward), while the other reduces it (-1 reward). The beneficial color randomly assigned at the beginning of each episode and indicated by a column. The environment is structured so that the agent must memorize the column's 's color collect the correct items. Initially, the column remains visible, but in harder variant, it disappears after 45 steps. increasing the memory requirement. To succeed, the agent must maximize survival by collecting beneficial objects while avoiding harmful ones.
L.8 MEMORY MAZE
The Memory Maze environment Pasukonis et al. (2022) is procedurally generated 3D maze. Each episode, the agent spawns in a new maze with multiple colored objects placed fixed locations. The agent receives first-person view and a prompt indicating the color of the target object. It must navigate the maze, memorize object positions, and return to them efficiently. The agent receives a reward of for reaching the correct object and reward for incorrect objects.
46

---

Published as a conference paper at ICLR 2026
L.9 MEMORYGYM MORTAR MAYHEM
Mortar Mayhem (Pleines et al., 2023) is grid-based environment where the agent must memorize and execute sequence commands in the correct order. The environment consists of finite grid, where the agent initially observes a series of movement instructions and then attempts to reproduce them accurately. Commands include movements to adjacent tiles or remaining in place. The challenge lies in the agent's s ability to recall and execute growing sequence of instructions, with failure resulting in episode termination. A reward of +0.1 is given for each correctly executed command
L.10 MEMORYGYM MYSTERY PATH
Mystery Path (Pleines al., 2023) presents an invisible pathway that the agent must traverse without deviating. If the agent steps off the path, it is returned to the starting position, forcing it to remember the correct trajectory. The path is procedurally generated, meaning each episode introduces new configuration. Success in this environment requires the agent to accurately recall previous steps and missteps to avoid repeating errors. The agent rewarded +0.1 for progressing onto a previously unvisited tile
L.11 POPGYM ENVIRONMENTS
The following environments included from the POPGym benchmark (Morad et al., 2023), which designed to evaluate RL agents in partially observable settings. POPGym provides a diverse collection of lightweight vectorized environments with varying difficulty levels.
L.111 POPGYM AUTOENCODE
The environment consists of deck of cards that is shuffled and sequentially shown to the agent during the watch phase. While observing the cards, a watch indicator is active, but it disappears when the last card revealed. Afterward, the agent must reproduce the sequence of cards in the correct order. The environment is structured to evaluate the agent's ability to encode sequence of observations into an internal representation and later reconstruct the sequence one observation at time.
1.112 POPGYM CONCENTRATION
The environment represents classic memory game where shuffled deck of cards is placed face- down. The agent sequentially flips two cards and earns a reward the revealed cards form a matching pair. The environment is designed in such way that the agent must remember previously revealed cards to maximize its success rate.
L.113 POPGYM REPEAT FIRST
The environment presents the agent with an initial value from a set of four possible values, along with an indicator signaling that this the first value. In subsequent steps, the agent continues to receive random values from the same set but without the initial indicator. The structure requires the agent to retain the first received value memory and recall accurately receive reward.
L.11.4 POPGYM REPEAT PREVIOUS
The environment consists of a sequence of observations, where each observation can take one of four possible values at each timestep. The agent is tasked with recalling and outputting the value that appeared specified number steps in the past.
L.11.5 POPGYM STATELESS CARTPOLE
This is modified version of the traditional Cartpole environment (Barto et al., 1983) where angular and linear position information is removed from observations. Instead, the agent only receives velocity-baat based data and must infer positional states by integrating this information over time to successfully balance the pole.
47

---

Published as a conference paper at ICLR 2026
1.1166 POPGYM STATELESS PENDULUM
In this variation of the swing-up pendulum environment (Doya, 1995), angular position data is omitted from the agent's observations. The agent must infer the pendulum's position by processing velocity information and use this estimate to determine appropriate control actions.
L.11.7 POPGYM NOISY STATELESS CARTPOLE
This environment builds upon Stateless Cartpole by introducing Gaussian noise into the observations. The agent must still infer positional states from velocity information while filtering out the added noise maintain control of the pole.
L.1188 POPGYM NOISY STATELESS PENDULUM
This variation extends the Stateless Pendulum environment by incorporating Gaussian noise into the observations. The agent must manage this uncertainty while using velocity data to estimate the pendulum's position and swing it up effectively.
L.11.9 POPGYM MULTIARMED BANDIT
The Multiarmed Bandit environment is episodic formulation of the multiarmed bandit prob- lem (Slivkins, 2024), where a set of bandits is randomly initialized at the start of each episode. Unlike conventional multiarmed bandit tasks, where reward probabilities remain fixed across episodes, this structure resets them every time. The agent must dynamically adjust its exploration and exploitation strategies maximize long-term rewards.
L.11.10 POPGYM HIGHER LOWER
Inspired by the higher-lower card game, this environment presents the agent with a sequence cards. At each step, the agent must predict whether the next card will have higher lower rank than the current one. Upon making a guess, the next card is revealed and becomes the new reference. The agent can enhance its performance by employing card counting strategies to estimate the probability of future values.
1.11.11 POPGYM COUNT RECALL
At each timestep, the agent is presented with two values: next value and query value. The agent must determine and output how many times the query value has appeared so far. To succeed, the agent must maintain accurate count of past occurrences and retrieve the correct number upon request.
1.11.12 POPGYM BATTLESHIP
A partially observable variation of the game Battleship. where the agent does not have access to the full board. Instead, receives feedback on its previous shot, indicating whether it was a HIT or MISS, along with the shot's 's location. The agent carns rewards for hitting ships, receives no reward for missing, and incurs a penalty for targeting the same location more than once. The environment challenges the agent to construct internal representation of the board and update its strategy based on past observations.
L.11.13 POPGYM MINE SWEEPER
A partially observable version of the computer game Mine Sweeper, where the agent lacks direct visibility of the board. Observations include the coordinates of the most recently clicked tile and mined tile results in negative reward and ends the
the number of adjacent mines. Clicking game. To succeed, the agent must track previous selections and deduce mine locations based on the numerical clues, ensuring it avoids mines while uncovering safe tiles.
48

---

Published as a conference paper at ICLR 2026
1.11.14 POPGYM LABYRINTH EXPLORE
The environment consists of procedurally generated 2D maze in which the agent earns rewards for reaching new, unexplored tiles. Observations are limited to adjacent tiles, requiring the agent to infer the larger maze layout through exploration. A small penalty per timestep incentivizes efficient navigation and discovery strategies.
L.I1.15 POPGYM LABYRINTH ESCAPE
This variation of Labyrinth Explore challenges the agent to find an exit rather than merely exploring the maze. The agent retains the same restricted observation space, seeing only nearby tiles. Rewards are only given upon successfully reaching the exit, making it sparse reward environment where the agent must navigate strategically achieve its goal.
M MIKASA-ROBO CUSTOMIZATION GUIDES
Code 7: ShellGameTouch key difficulty knobs and a debug preset.
Shel [GameeTouch
from mani_ski u 1s, rercilsrrriiise
import regi aegister_ens from mikasa robo cecbo_suitte suite.shell game touch import ShellGameTouchhnn import gymnasium
gym
@register_ent (" She Game TouchDebug-vo", max max_episode_steps-1000 class
ShelllGameTouchEnv))
BALI_RADIUS = 0.02 8 radius of the ball MIN_DIST = 0.2 minimum distance between nearest cups
how long the ball is visible (no cups) GOAL_THRESH 0.08 threshold for the goal
TIME_OFFSEL
env = gym. make ( ("SheelGGaeToouuueeebrbr rgb", rencer rencer_mode="all mmdd=="al")
num_envs=226, obs_mode="
Code 8: ShellGamePush: key difficulty knobs and debug preset.
from mani mani_skill.
www.sskiiilllsleleeeeeooooeeiitt registration import register
from mikasa rubo_roobssteeeee muiteeshelll shell_game____eeeeeeeeeppaaaaaa_ push import ShellGamePushEnv import gymnasium as gym
@register_env
max_opisood__tcpp=11100
class BeeellGaamuuusDccugEtnn Cachhlaaeeeeee
BALI, BALI_RADIUS 0.02 radius of the ball MIN_DIST 0.2 # minimum distance between nearest cups
TIMR_OFFSE: = $ how long the ball is visible (no cups) GOAL THRESH 0.08 threshold for the goal
env = gym.make make (" ("BheelGaamPPuuiDeebuggu num_ oum_enn==25,, obs mode="
4 rgb", render_mode "all")
49

---

Published as a conference paper at ICLR 2026
Code 9: ShellGamePick: key difficulty knobs and debug preset.
ShellGamePick
from mani_skill, www.il.tiissiiiiee.a....a..iiilt utils, registration import register_ env from mikasa cas___roo___uuue www.ssuuueelllle......ee wwwel_gaaeeeeea___eeeeeeeeaaaaee import ShellGamePickEnv import gymnasium as gym
eregister_env ("hheeelammeeeeeeeeeeueee class ShellGameP v (Steesaaeeeaaaeeeeee.i......
www__pppdde_ssepp-1000
BAIII_RADIUS 0.02 8 radius of the ball
MIN_DIST 0.2 # minimum distance between nearest cups
FIMK_OFFSET" - # how long the ball is visible (no cups) GOAL
THRESH GOAL_THRESH 0.08 threshold for the goal
env = gym. make ("
num num num_envs=256, cbs mode="
+ rgb", render_mode="all1"
M.1 INTERCEPT
Code 10: Intercept: controlling projectile speed and target tolerance.
from sani_skill, .uillstils....l.....ll registration import register eny from mikasa_robo_sultee intercept import InterceptBaseEnv import gymnasium uu gym
@register_env Lnt.eeeteppeebbgggo. class InterceptteebugEny InteraaptBaaeeEnv)
max_episode_ssepss1100
VELOCITY VELLOCITYRRANG. _RANGE (0.0, 0.0)
(min_v, max_v] range for the ball
+ velccity randomization
BALL RADICS = 0.02 $ radius of the ball
RADIUS = 0.1
radius of the goal region
env gym. make (" "'IneerreptDebbggvoo" num_envs=256, obs_mode=" rggh"
+ render_mode="all
Code 11: InterceptGrab: grasp-based variant with velocity randomization.
from mani_skill.ur. rew.ll..tgtiittiiiinn import register_env from lacasa_robo_sute intereeetttrrrctraaaa.ee import InterocptGrabaascEnn import gymnasium gym
@register_emt _nv interccetttrabbeebeegeee max max_epi stt.epiiee_stesssi1ii0 class InterreeptGrabblebugin
VKI.CCITy_AANC (0.0, 0.0) # (min_v, max_v) range for the ball 4 velccity randomization
BALL RADIUS 0.02 # radius the ball
env = gym. make (" ""Inerrppttraabbugy"""" rgb", render_mode="all
num_enve=256, obs_mode="
50

---

Published as a conference paper at ICLR 2026
M.2 ROTATE
Code 12: Rotatel.enient one- vs. two-sided rotation with tolerance control.
from mani_sk utils registration import register_ent
from Lacasa_robb___e wttp:usuuteetttt_eeeeeee rowatoeeeentttteee import RotatelenientEav import gymnasium um gym
( senister_enviie (oootatelen ent max st____pisode_steppe_ee
class KotateLenieetteebggie (RotatelenientEnv) angie" or "pos_nee neg angle" defines
MODE = "pos, angle"
possible directions of rotation
PEG_HALP_MILE WADF_WIDTH 0.025 peg half width
PEG_HALF LENGTH = 0.12 $ pog half length
ANGLE_THRESHOLD = 0.1 (radians) defines the pormissible 4 deviation from the target angle
env ("ooaattaaaniiintteeo-ooo- num_envs-256, obs obs_mode-
4 rgb", rencer rencer_mode="all mode="all,, angle_thresho hww.e_tthehhho-AAiiiis (ttessoood-ANNNE__TREEE00)))))))
Code 13: RotateStrict: stricter alignment for fine-graieed control.
from mani_skill.utile registration import register_ from mikasa_robo_suitee rotate_strict import RotateStrictEnv import gymnasium gym
@register_enn
class Rotate@triccDDbuggEN (RotatcStrictEnv)
MODE "pos_angle" $ "pos_angie" "pos_neg defines
+ possible directions of rotation
PRG_HALF_WITT WIDTH 0.025 peg half width
- 0.12 $ peg half Jength
ANGL.__THRHSHOL - 0.1 (radians) defines the permissible
deviation from the target angle
env = gym.make ("Roaatetrriteeegggyuu num envs=256, obs mode="
4 render_mode-"all angle@threaalooonNii.TTEEBBBB
M.3 TAKEITBACK
Code 14: TakeltBBck: goal-region and object-size controls.
from skill utils. registration import register_ens env from mikasa robo wwwprrroioeee_eeeeeaeeeeerraeeee suite. it__bakk import TakeItBackEnv import gymnasium gym
@rogister_env ("TakaIBBacDDeebgggetrro class TakeitBackDebbugEn (TakeItBackEnv)
max_opiaodd_sstpp==100
GOAL_RADIUS: float 0. radius of the goal region cube size
FALFSIALF:: float 0.02
env = gym. make (" Backkeet nun_envs=256, obs rgb" render aande__odd=="a mode="all")
51

---

Published as a conference paper at ICLR 2026
M.4 REMEMBERCOLOR
Code 15: RememberColor: visibility window, occlusion delay, and tolerance.
from mani_ski uu 1s. registration import register_
from lassa__obo_suitee remember_cole import RememberCol or Baseennv import gymnasium as gyn
@register env ("RemeeberroooorrDebbuee class Remembbe RememberroolorBasennyy
max stsss_ss_sts=10000
COLORS TIME_OFFSE 200
1-9 unique cubes
how long target cubo is shown
THRESH 0.03
more difficult goal threshold
CURR_HALPSIZE
0.00
cubes size
DEL.A__TMM.
110
empty table duration (seconds)
env = gym. make (" ('Renemberoalorreebuggg
enn_enn== obs_mode="
rgb", rencer mode="all", deita time=DELTA TIME)
M.5 REMEMBERSHAPE
Code 16: RememberShape: number of shapes, scale, and fixed color option.
from mani_skill uuils, registration import register_emt env from Lacasa_rbb__see remember rememberrshhap import Remembberrhaapeassenn import gymnasium um as gym
eregister_env (*#emmmberrraaapeeeeuu
max max_opisode_steeee-ile
class remeenberrShapeeeebbpge (Remember (hammmmmeerhappeeseRenn
SHAPES
6
1-9 uni qigque shapes
FIMR_OFFSRT
200
how long target cube is shown
GOAL THRESH
0..03 more difficuit goal threshold
SHAPE SCALE
0.00
cubes size
COLOR [0, 0, 255, 255] & each object has the same color
DELTA_TIME
100
empty table duration (seconds)
env gym. make ( ("Remembsrrhapppebbbbng num_enve=256, obs_mode=" 4 rgb", render Lender_mode="ally" delta delta_ time-DELTA_ TIME)
M.6 REMEMIERSHAPPANDCOUUS
Code 17: RenemberShaapenndCCooo:: composite cue space and timing.
from mani_ utils wtw.::iwww............eo...eeee. registration import register env
from mi oetaaarrbbb_aiiieeeeeeeeeeetttaa Remeniberrshappencccoooorraes
import
import gymnasium as gym
@register _etveerr_vvveee (atraa/eeeeeeehhappppoooooooaaaa =1000)
max_episede_stepp
class RencmilesSShpppnnccoooorroggggg Bemnmeorrhaaappacooooooooeeeee
SHAFES
4 1 1-5 unique shapes with 3 colors (1-15
4 combinations)
TIMR_OFFSET
200
how long target cube is shown di difficult goal threshold
THRESH
0..33
SCALE
0..02
cubes size
COLOR 10, 0, 255, 255] & each object has the same color
DELTA_
100
empty table duration (seconds)
env gym. make ( TRemeelershaagaancccookkktteoeee 4 abs_mode- " "rg"", render_mode="aa delta__time-DB_AAAAA..
num envs=256,
52

---

Published as a conference paper at ICLR 2026
M.7 BUNCHOFCOLORS
Code 18: BunchOfColors: set size and presentation timing.
from mani_skill utils, registration import register_ eny from mikasa robo suite bunch of of_colors import BunchOfColorsEnv import gymnasium as gym
eregister_env ("Bunchhoccoolorsooobegegg- class fenormibbssshhppcaccoooooooogggn
max_opissod__stop==1000
(BunchofcolorsEnvv
COLORS
1-9 unique cubes
GOAL_THRESH 0.03
more difficult goal threshold
HALFSIZE. 0.02
cubes size
SEQUENCE LENGTH = 15
Number of cubes to show in sequence (1-9) $ Duration to show each cube
STEP DURATION
EMPTY DURATION
Duration of empty table
DELTA_TIME: 100
empty table duration (seconds)
make ""Banccofccooorsseeegggg-i num_envs=256, cbs_mode=" rgb", render_mode= delta_time-DBLTAATIIMg
M.8 SEQOFCOLORS
Code 19: SegOfColors: sequence length and tempo.
ittiillltiilillill from mikasa_robo_suict import gymnasium as gym
from mani_skill ut 1s registration import regi ster_env
import Sequfcolorsenv
@register_ent ("Secoofcoloossdeeeuuyyyl max fax__epssdd__ree class Cemcailloshhpeardddoooobbbbgegn (SegofcolorsEnv):
wewsedee_teess11000 steps=1000)
COLORS
1-9 unique queque cubes
GOAL_THRESS THRESH 0.03
more difficuit goal threshold cubes size
HARR_HALFSIEE 0.02
SEQURNCE_LENCE LENGTH = 15
Number of cubes show in sequence (1-9) Duration to show each cube Duration of empty table empty table duration (seconds)
STEP_DURATI DURATION
EMPTY DURATION
DELTA_TINE
100
gym. (ww.maaaeeeaaaaeea
num_envs=256, obs_mcde=" render_mode=" "aldde-="llll all", delta_time-DeUuA_TTMM
M.9 CHAINOFCOLORS
Code 20: ChainOfColors: chained sub-episodes with controlled pace.
from mani_skill uut 1s larii...rrgisssee registrttiiitti. import register_ent
from mikasa robo robo__uitee suite. seq.es_e_o ofco_oo__oolll colors import SegofCoLorsEnv import gymnasium gym
class (wseeopocolooosseeeeeeeeeee
@register_env (e
(SegofcolorsEnv)
COLORS
5
1-9 unique cubes
GOAL_THRESS 0.03
more difficult goal threshold cubes size
CUBE FALFSI2E 0.02
SEQUENCE, LENGTH
15
Number of cubes to show sequence (1-9)
STEP _UEATTON
15
$ Duration to show each cube
DERATIIN =
Duration of empty table
TIME 100
empty table duration (seconds)
anv gym.make (oeep
num num_envs-256, obs_mode="rgh
4 render_mede=""ll1, im
53

---

Published as a conference paper at ICLR 2026
Figure 26: Real-world experimental setup for memory-intensive tabletop manipulation. The platform uses the SO-101 arm (Knight et al., 2024) equipped with wrist-mounted RGB camera for egocentric observations and an external tripod-mounted RGB camera providing a fixed overhead view. The scene contains three colored cubic objects used in the Remembe rColor3 family of tasks. This configuration matches the sensing and embodiment used for fine-tuning and evaluation in our real- world experiments.
Table 10: Real-world performance of the fine-tuned mo.5 the three evaluation tasks using 30 episodes per task. For each color and each task, we report the fraction of episodes where the robot touched the correct cube (is_touched) and where it successfully picked (is_picked))
Task
Task 2
Task 3
Color
is touched
is picked
is touched
is picckeed
touched
is picked
Total
1.00
0.80
0.63
0.43
0.10
0.10
Red
1.00
0.90
0.70
0,40
0.10
0.10
Green
1.00
0.90
0.70
0.60
0.20
0.20
Blue
1.00
0.60
0.50
0.30
0.00
0.00
N
REAL-WORLD EXPERIMENTS
We conducted set of real-world experiments to evaluate whether modern VLA models can solve phys- ical instances of the atomic memory-intensinn tabletop manipulation tasks introduced in MIKASA- Robo benchmark. Although these tasks are visually simple, they impose explicit requirements temporal information retention that current VLA architectures do not model. Our objective to de- termine whether the failure modes observed in simulation persist under real-world sensing, actuation, and embodiment, and whether these failures can be attributed to memory limitations rather than to different confounding factors.
The experimental platform is shown in Figure 26. We used the SO-101 arm (Knight et al., 2024), the lerobot library (Cadene al., 2024), and pretrained To.s model (Black et al., 2025) that fine-tuner tune specifically for each of the three real-world tasks described below. The setup employs wrist-mounted RGB camera for egocentric observations and tripod-mounted RGB camera for fixed overhead view. The objects used in all experiments are the three colored cubes from the RememberCol oor3 family of tasks.
Real-world experiments introduce additional sources of variation that can obscure the contribution of memory, such as contact inaccuracies, lighting changes, and pose drift. To isolate memory as the limiting factor, we designed three-stage evaluation protocol. Task 1 verifies that the demonstration pipeline, training process, and embodiment are sufficient for simple MDP problems. Task 2 introduces dynamic scene changes without occlusion to test robustness to distribution shifts that do not require long- horizon memory. Task 3 instantiates the full occluded long of RememberColocm oorollorrooo 3-v0 to directly test memory.
long-horizon horizon dependency
54

---

Published as a conference paper at ICLR 2026
Figure 27: Task 1 (Sanity Check): real-world execution of the fully observable pick-and-place instruction. The robot observes all three cubes (red, green, blue) throughout the episode, receives the instruction to pick specified color, and performs a standard MDP pick action without any memory requirement.
For each task, we collected a balanced dataset of 300 expert teleoperated demonstrations, with 100 trajectories for each target color. We then fine-tuned the pretrained No.5 model for 100,000 gradient steps using lerobot. Across all tasks we obtained 900 expert trajectories, which we will release in the camera-ready version. Quantitative results are summarized in Table 10.
Task I: Sanity Check. This task verifies that the system solve fully observable pick-and-place instruction without any memory requirement. The robot observes three cubes on the table (red, green, blue) throughout the entire episode. Given the instruction "Pick the (color) cube", with (color) in (red, green, blue), ). the robot must pick the corresponding cube. Since both the instruction and all objects remain visible every timestep, the task is a standard MDPP and demands temporal memory. We collected 300 teleoperated demonstrations for fine-tuning (100 trajectories for each of the three target colors) and evaluated the fine-tuned To.5 across 30 episodes (10 per color). Since the correct cube is always visible, the task requires no temporal memory. Execution examples are shown in Figure 27. As reported in Table 10, the model reliably touches the correct cube in all episodes and achieves a pick success rate of 0.80. Per-color performance is consistent for red and green, with lower pick success rate for blue, but the critical observation is that the touch success rate is 100 percent across all colors. This indicates that the VLA consistently identifies the correct target cube, and that the residual errors in Task arise from grasp execution rather than from failures in interpreting the instruction selecting the correct object. These results confirm that the training pipeline and the fine-tuning procedure are sufficient for standard, fully observed manipulation behaviors.
Task 2: Dynamic Environment. This task introduces motion and dynamic scene changes without creating a genuine memory requirement. The robot receives the instruction "Remember the color of the cube and then pick the matching one". It first observes single cube of one color the table. After this observation, we throw the remaining two cubes of other colors onto the table. The robot must pick the cube it observed initially. Although the instruction implies memory, the horizon between the initial view and the final decision is extremely short. With action chunking the model effectively retains the initial observation inside single generated action sequence, so the task is solvable even without an explicit temporal memory mechanism. This task therefore controls for robustness to dynamic environmental changes while avoiding the occlusion and long horizon dependencies of KememberCooor' 3-v0.. Task 2 tests whether the model remains robust when the scene changes dynamically without creating long-horizon dependency. representative execution is shown in Figure 28. As summarized in Table 10, the model successfully touches the correct cube in 0.63 of episodes and completes the pick in 0.43. These results indicate that the model's 's performance degrades under dynamic scenes though actual memory is required. The model still succeeds substantially more often than in Task 3, which suggests that failures in Task 3 arise from missing memory rather than from difficulties with dynamic perception.
55

---

Published as a conference paper at ICLR 2026
Figure 28: Task 2 (Dynamic Environment): real-world execution under short-horizon scene changes without occlusion. The robot first observes a single target cube, after which the remaining cubes are thrown into the scene, introducing dynamic motion and distribution shift while maintaining full visibility all objects. The sequence illustrates the robot's approach and grasp using the wrist- mounted camera. Although the instruction suggests memory, the temporal gap is short enough that action chunking preserves the initial observation, which makes the task solvable without an explicit long-horizon memory mechanism.
Task 3: Real World RememberCClor3-vv. Task 3 instantiates the full memory requirement. The robot first observes the target cube for several timesteps. We then occlude the target cube using a tablet, place the two remaining cubes underneath the tablet, permute the positions of all three cubes under occlusion, and finally reveal the scene. The robot must recall the color observed before occlusion and identify the matching cube in the post-shuffle configuration. Figure 29 illustrates representative execution. As shown Table 10, the robot's 's performance collapses in this setting, with only 0.10 touch success and 0.10 0.10 pick success overall, with similar patterns across individual colors. The drop performance relative to Task 2 demonstrates that the model fails to retain required color information across the occluded interval. Since conditions other than the long-horizon memory dependency are comparable between Task and Task 3, the discrepancy isolates memory as the dominant failure mode.
The comparison across Tasks 1, 2, and 3 in Table 10 demonstrates systematic degradation as the memory requirement increases. The model performs reliably in standard MDP setting (Task 1). begins to degrade under purely dynamic variation (Task 2). and fails almost entirely when long- horizoorizon memory is required (Task 3). These findings align with our simulation results and provide strong evidence that current VLA models lack the temporal memory capabilities required for even simple real-world memory tasks. The results underscore the need for architectures with explicit and robust long-ho bong-horizon memory mechanisms in real-world robot learning.
Discussion and Conclusion.
Open-Source Datasets. To ensure full reproducibility and to support further research on memory- intensive real-world manipulation, publicly release all datasets used for fine-tuning and evaluation on Hugging Face. Each task contains 300 real-world trajectories in total, comprising 100 red, 100 green, and 100 blue cube episodes. The datasets are split differently across tasks solely for practical hosting and download convenience, while preserving the same color balance and total trajectory count.
56

---

Published as a conference paper at ICLR 2026
Figure 29: Task 3 (Real-world Remembe rCol p requirement with occlusion and object permutation. The robot first observes the target cube, after which the cube is covered with tablet and all three cubes are shuffled under occlusion. Once the tablet is removed, the robot must identify and pick the originally observed cube despite the occluded interval and the unknown permutation. The sequence shows the initial observation, the occlusion event, the post-shuffle scene, and the eventual grasp attempt from the wrist-mounted camera view.
execution of the full long-horizon memory
Table 11: Publicly released real-world datasets used in our experiments.
Datasct
URL
Tick
so101_rod_euhee 100, tisk
/wo101 sew__o_oo___ooeeeeeee green cube htops:/ http:///tttooaaaaaeeeeaaaaaaaa htt.://a.....eeaaa..../oaaaaaaa.
buap ing
100_
/ttggs////gggggg /hgggggggggg httpst
col01_Bsc_cubb_10k 100__a_______kkk
2uck
so101_egh.cube__m 3002
htspo://huggiggttet http:///ww.aaaaoooooooo////////s
Tack 3
so101__ghh 300 makks _eats_pat___od_
htupes /wwgg
woldl
htepst/ /hugg
cuse tas.k_part__e.._.
willl igh
mok3 part.
htepst //hugg ing woee no
mibe teaks of
so101_rgh_com cub_eee__bb_______. part__pprt____....._.... 6
htops: htt./og-nggaacccc nagccc es
soltll aolll__gh_cubbe 500 pasks__sp______ of
htupat //luugg Lon face.
so10) jotbi__gh_cube,__ (ask3 300_mac3__prt_t__c.. _art__________
btspst/ ingeee. co/
57


Reviews:
[{"review_id": "x7OR5cYySu", "reviewer": "Reviewer_iPbp", "summary": "This paper introduces MIKASA (Memory-Intensive Skills Assessment Suite for Agents), a comprehensive benchmark suite aimed at systematically evaluating memory capabilities in reinforcement learning (RL) agents. The authors address a major gap in the field — the lack of standardized benchmarks to assess memory across diverse settings, particularly in tabletop robotic manipulation, where partial observability and temporal dependencies are critical.", "questions": "1. How do you envision extending the benchmark to evaluate memory mechanisms in continual or lifelong learning settings, where robots must adapt to evolving task distributions or environments while not forgetting previous ones?\n\n2. Given that the benchmark’s tasks are both complex and memory-intensive, have you explored ways to reduce the computational cost? How do you balance the benchmark’s realism and difficulty with inclusivity for a wider research community?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The benchmark offers robust and comprehensive coverage, incorporating a diverse set of tasks that rigorously test models across a spectrum of memory requirements.\n\n2. This benchmark successfully strikes a critical balance, bridging the gap between current, often over-simplified evaluation methods and the complexity of real-world, human-level tasks. By incorporating challenges that mirror the cognitive demands of human problem-solving while maintaining experimental tractability, it establishes a new, more relevant standard for assessing model performance", "weaknesses": "1. The benchmark is specialized for tabletop robotic manipulation tasks. While this area is critical for memory in robotics, the benchmark does not cover other key areas of robotic memory, such as continual learning where the agent must adapt to changing task distributions or physical environments over extended, sequential periods.\n\n2. By design, the tasks are complex and memory-intensive, leading to high computational demands for training. This can limit the accessibility of the benchmark to researchers without substantial computational resources.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9PLLAWp9fW", "reviewer": "Reviewer_8hhi", "summary": "This paper presents a benchmark for testing the capabilities of an agent's memory in the context of robotic manipulation with reinforcement learning (RL).\nThey distinguish different types of memory tasks as 1) object memory, 2) spatial memory, 3) sequential memory, and 4) memory capacity, which seems intuitive and, to the best of my knowledge, hasn't been done before.\nThey propose a set of Gymnasium-based tasks for memory-enhanced RL and a set of 32 robotic tasks.\nThey also release a set of offline visual-based datasets for sparse-reward manipulation tasks, for offline RL.\nIt's built upon ManiSkill3, which offers fast GPU parallelisation.\nHowever, it is worth noting that while I do work on RL, I do not have experience with memory mechanisms in RL.", "questions": "See weaknesses", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The proposed benchmark addresses spatial and temporal memory -- an important and timely topic in RL. The authors evaluate a broad set of baselines, and since none achieve strong performance across all tasks, the benchmark appears to have an appropriate level of difficulty, leaving room for future progress. It is built on the ManiSkill3 simulation framework, enabling fast GPU parallelisation -- a valuable design choice for scalability. Overall, the paper is well written and easy to follow, with only minor issues to fix.", "weaknesses": "- The main text lacks details on the quality of the offline RL datasets. The authors should explicitly state that these are expert datasets, as this information is important for readers and enables potential use of imitation learning methods.\n- TD-MPC2 is cited as an arXiv preprint, but it has been published at ICLR 2024.\n- The bibliography has inconsistent capitalisation (e.g., “Td-mpc2” should be “TD-MPC2”). This can be fixed by correctly using braces {} in BibTeX.\n- It is unusual to use \"Subsection\". I would advise the authors to use \"Section\".\n- Formatting issues:\n    - Figure 1 needs more space around it.\n    - Table 1 requires additional white space below it.\n    - The text in Table 3 is too small; it should be made bigger.\n    - The text in Figure 6 is too small and difficult to read; it should be enlarged.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QoZNfmsErX", "reviewer": "Reviewer_YGtn", "summary": "This paper proposes a comprehensive classification framework for memory-intensive RL tasks, integrating multiple previous tasks into a unified benchmark (MIKASA-Base). The authors also develop a novel benchmark of 32 memory-intensive tasks for tabletop robotic manipulation (MIKASA-Robo). They evaluate several current algorithms on their benchmark, demonstrating a clear need for the development of more effective algorithms for memory-intensive tasks.\n\nOverall, I believe these contributions are of significant interest to the RL community and I strongly recommend acceptance. The comprehensive suite of tasks and thorough documentation make this a valuable resource for future research.", "questions": "**Questions for clarification:**\n\n1. You mention that \"The link between physical interaction and memory remains underexplored, motivating a framework for spatio-temporal memory in real-world tasks.\" What kinds of memory mechanisms do physical memory tasks require? What specific memory capabilities are not considered in the current framework? A more detailed discussion of this would strengthen the motivation.\n2. What are your plans for continued support of the benchmark? Are you planning on adding new environments or algorithms? A section on future work would be welcome.\n3. In Section 4.2, what does each memory task correspond to with regard to cognitive processes? \n\n---\n\n**Minor comments:**\n\nFigure 6 is difficult to digest. I’m not sure what the legend items for Sequential Memory, Memory Capacity, Spatial Memory, and Object Memory correspond to on the plot since the background shading of the different tasks doesn’t match the colors in the legend. Additionally, Sequential Memory and Memory Capacity have different legend icons than Spatial Memory and Object Memory.\n\n---\n\n**Suggestions for improvement (not factored into score):**\n\n1. The authors could elaborate more on the connection between cognitive processes and agentic memory requirements, maybe through a more systematic categorization. This would strengthen the theoretical foundation of the benchmark and help researchers understand what types of memory mechanisms are needed for different task categories.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "The paper is well-motivated, addressing an important gap in existing RL benchmarks. The comprehensive suite of 32 tasks provides comprehensive coverage of different types of memory requirements in physical robotic tasks.\n\nThe categorization of previous tasks and integration into a single framework is an important contribution that will facilitate further research.\n\nThe authors evaluate both online and offline algorithms and motivate the need for both in memory-intensive tasks.\n\nThe authors convincingly demonstrate that the tasks are challenging for current algorithms, showing the benchmark's value for driving future research.\n\nThe benchmark codebase is well-documented and the authors provide an easy startup notebook.", "weaknesses": "This is not the first benchmark for physical memory-intensive tasks. However, the authors acknowledge this limitation and clearly state that their benchmark captures more types of memory tasks than previous work.\n\nThe connection to cognitive processes and memory mechanisms could be more fully developed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Awc2o84fuL", "reviewer": "Reviewer_KGih", "summary": "The paper “Memory, Benchmark & Robots: A Benchmark for Solving Complex Tasks with Reinforcement Learning” introduces MIKASA (Memory-Intensive Skills Assessment Suite for Agents), a unified benchmark framework designed to systematically evaluate memory mechanisms in reinforcement learning (RL). Recognizing the absence of a standard evaluation suite for memory-based RL, especially in robotic manipulation, the authors propose a taxonomy of four memory types: object memory, spatial memory, sequential memory, and memory capacity. Building upon this taxonomy, they release two complementary components: MIKASA-Base, a consolidated collection of existing memory-intensive RL environments unified under a common API, and MIKASA-Robo, a new suite of 32 tabletop robotic manipulation tasks that require various forms of temporal and spatial memory. Experimental evaluations with online RL, offline RL, and VLAs demonstrate that current models, such as PPO-LSTM, Decision Transformer, and Octo, struggle to generalize across tasks with strong memory dependencies. The results highlight that while RL agents can perform well under full observability, their performance deteriorates sharply when memory retention is required, validating MIKASA as a rigorous benchmark for developing and assessing memory-centric RL algorithms.", "questions": "1.How well do the tabletop tasks in MIKASA-Robo reflect real-world robotic manipulation challenges? Since most tasks are simplified color or position memory exercises, can performance on these synthetic environments meaningfully predict how well an agent will perform in real, noisy physical settings?\n\n2.The paper proposes four memory categories (object, spatial, sequential, capacity). How are the metrics and evaluation protocols designed to isolate these different memory mechanisms? Could overlapping factors (e.g., attention or perception) interfere with measuring pure memory ability?\n\n3.MIKASA currently uses RGB and joint states for perception. Would incorporating other sensory modalities such as depth, tactile feedback, or language instructions change the evaluation of memory skills? \n\n4.Has the author considered scenarios where occlusion exists from the outset (e.g., retrieving a bottle of cola from a closed refrigerator)? Can the task be accomplished solely with RGB+joint data in such cases? Additionally, can this benchmark support extensions like dual-arm coordination for humanoid robots and other heterogeneous embodied systems? Could the authors provide a unified interface to facilitate such expansions?\n\n5.Have the authors considered adding diagnostic metrics or analyses beyond success rate, such as error decomposition or memory recall tracking, to better reveal the failure causes across different memory types?\n\n6.The evaluation of large-scale vision-language-action models like Octo and OpenVLA was limited due to computational constraints. How might more thorough fine-tuning or longer training horizons change the conclusions about these models’ memory abilities? Does the benchmark currently support scalable training for such large architectures (provide scalable dataset or distributed training)?\n\nI reserve the possibility of raising my score if the authors can adequately address all the concerns raised during the rebuttal phase.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper introduces a benchmark that systematically studies memory in reinforcement learning for tabletop manipulation, extending ManiSkill3 with new task variants emphasizing temporal dependency and spatial recall. The motivation is relevant and well-grounded, as memory remains a key bottleneck for long-horizon robotic reasoning. The paper provides clear organization, a well-documented codebase, and thorough evaluations across multiple RL and imitation learning baselines. By categorizing tasks based on distinct memory demands, the benchmark offers a structured framework for analyzing policy retention and forgetting behaviors. The authors also contribute useful diagnostic discussions and visualization tools for understanding agent performance over time. Overall, the work is timely and thoughtfully executed, offering a meaningful step toward standardized evaluation of memory mechanisms in robotic learning, even though its scope remains largely limited to simplified simulation settings.", "weaknesses": "There are some weaknesses arise:\n\n1.This benchmark foucs on solving memory issuse encuntered in the table top manipulation scenarios. The benchmark is build upon maniskill3, where there is only a franka panda arm as the embodiment. And the manipulation task distribution is limited at pick-place blocks with various colors. However, in the real-world, more complex spatial resoning and operations are required. The lack of more diverse, real-world tasks and embodiments (such as retrieving a can of cola from a closed refrigerator, or a humanoid robot sequentially using both arms to assemble components collaboratively) undermines the overall contribution of this benchmark. Therefore, this benchmark can be considered merely an incremental supplement to the previous Maniskill3.\n\n2.This benchmark does not conider the sim2real gaps that encounter in the real-world deployment, even though the authors claimed that their unified framework that can enable more robust systems for real-world use. Lack of real-world performance as evidence undermines the overall contribution of this benchmark. The RL memory problems that authors analyzed occur only in simulation or also in real-world? If memory problems (such as occuled objects, spatial reasoning) also encounter in the real-world, current evidence reported in the paper is not enough to support the claim about real-world application. Although I acknowledge simplfied simulation task setup is useful for controlled testing, these settings may not capture the sensory noise, continuous dynamics, and unstructured complexity found in real robot environments.\n\n3.As this benchmark is an incremental work of maniskill3, current task distribution is limited in pick-place various blocks. However using such tasks for spatial/object evaluation is already covered in previous benchmarks, such as Ravens, LoHoRavens. Moreover, benchmark like LIBERO involve both object/spatial reasoning tasks and also include long-horizon tasks that require long-temporal dependencies, and the operated objects are more close to the real-world setting (e.g., coffee machine).  \n\n4.Although the benchmark divides tasks by memory type, the evaluation mainly reports success rates. It lacks diagnostic tools to analyze why an agent fails—whether due to perception errors, memory decay, or decision confusion. This makes it difficult to understand failure mechanisms for each memory type.\n\n5.A minor point as the paper mentions that the evaluation of models like Octo and OpenVLA was restricted due to computational constraints, and fine-tuning was minimal. This means the reported performance may not reflect their full capability, reducing the fairness and depth of comparison across architectures. More extensive large-model adaptation is needed for a complete evaluation.\n\nIn summary, while the benchmark extends ManiSkill3 with 32 tasks, its contribution to memory-intensive RL remains limited because it does not address the sim-to-real gap and relies solely on success rates without richer diagnostic metrics. Most tasks center on pick-and-place cubes. Therefore, it is unclear how this benchmark provides substantial improvement over existing RL manipulation benchmarks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)