A Large Recurrent Action Model: xLSTM enables Fast Inference for Robotics Tasks
Authors: Thomas Schmied
Thomas Adler
Vihang Prakash Patil
Maximilian Beck
Korbinian Pöppel
Johannes Brandstetter
Günter Klambauer
Razvan Pascanu
Sepp Hochreiter
Venue: ICML 2025 poster (2025)
Keywords: reinforcement learning
rnn
xlstm
mamba
multi-task
robotics
Primary area: reinforcement_learning->batchoffline

Abstract: In recent years, there has been a trend in the field of Reinforcement Learning (RL) towards large action models trained offline on large-scale datasets via sequence modeling. Existing models are primarily based on the Transformer architecture, which results in powerful agents. However, due to slow inference times, Transformer-based approaches are impractical for real-time applications, such as robotics. Recently, modern recurrent architectures, such as xLSTM and Mamba, have been proposed that exhibit parallelization benefits during training similar to the Transformer architecture while offering fast inference. In this work, we study the aptitude of these modern recurrent architectures for large action models. Consequently, we propose a Large Recurrent Action Model (LRAM) with an xLSTM at its core that comes with linear-time inference complexity and natural sequence length extrapolation abilities. Experiments on 432 tasks from 6 domains show that LRAM compares favorably to Transformers in terms of performance and speed.

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
Thomas Schmied Thomas Adler! Vihang Patil Johannes Brandstetter 12 Günter Klambauer 12
Maximilian Beck
Korbinian Pöppel 12
Razvan Pascanu" 34 Sepp Hochreiter 12
Abstract
and sequence models trained via behavior cloning (Chen et al., 2021; Janner et al., 2021). Such approaches, in which agents are trained on large-scale offline datasets with causal sequence modeling objectives, have been driven by the pro- liferation of Transformer-based architectures and gave rise to what we refer to as Large Action Models (LAMs)) to highlight their similarity large language models (LLMs) (Radford et al., 20.8). LAM approaches can also be used in multi-task settings to develop generalist agents such as Gato (Reed et al., 2022).
In recent years, there has been trend in the field of Reinforcement Learning (RL) towards large action models trained offline on large-scale datasets via sequence modeling. Existing mod- els are primarily based on the Transformer archi- tecture, which results in powerful agents. How- ever, due to slow inference times, Transformer- based approaches are impractical for real-time applications, such robotics. Recently, mod- ern recurrent architectures, such XLSTM and Mamba, have been proposed that exhibit paral- lelization benefits during training similar to the Transformer architecture while offering fast infer- ence. In this work, we study the aptitude of these modern recurrent architectures for large action models. Consequently, we propose Large Re- current Action Model (LRAM) with an xLSTM at its core that comes with linear-time inference complexity and natural sequence length extrapo- lation abilities. Experiments on 432 tasks from 6 domains show that LRAM compares favorably to Transformers in terms of performance and speed.
Existing LAMs are primarily based on the Transformer (Vaswani et al., 2017) architecture. Because of their powerful predictive performance, robotics has become an emergent application area for large models (Brohan et al., 2023b;a; Octo Model Team et al., 2024; Gu et al., 2023; Wang et al., 2023), and a number of large multi-tast task datasets were collected (Jia et al., 2024; Embodiment Collaboration et al., 2024; Jiang et al., 2023; Mandlekar et al., 2023). This development bears the potential to produce robotics agents that learn master complex tasks in wide range of environments and even different embodiments. For exam- ple, recently it has been demonstrated, albeit in restricted settings, that sequence models trained on multi-episodic con- texts can perform in-context learning (ICL) (Laskin et al., 2020; Lee et al., 2023). One potential application of ICL can be to learn new related tasks in robotics without the need for re-training or fine-tuning.
1. Introduction
Reinforcement Learning (RL) has been responsible for im- pressive success stories such as paae-ig (Silver et al., 2016; Vinyals et al., 2019; Berner et al., 2019; Patil et al., 2022), plasma control for fusion (Degrave et al., 2022), or navigation of stratospheric balloons (Bellemare et al., 2020). While these successes were based on classical RL approaches, in which agents have been trained online with RL objectives,, recently there has been a trend towards offline RL settings (Levine et al., 2020; Schweighofer et al., 2022)
One of the key reasons for the success of Transformer-basee models is their ability to scale to large datasets through their efficient parallelization during training. However, despite numerous success stories in RL, language modeling (Brown et al., 2020) or computer vision (Dosovitskiy et al., 2021; He et al., 2022), a persistent drawback of Transformer-baaee architectures is their high inference cost in terms of both speed and memory (Kim et al., 2023). Consequently, de- ploying Transformer-bas scenarios, such as on devices with limited hardware capac- ity and/or rell-time constraints, e.g., robots or smartphones, is prohibitive because of the required fast inference times (Firoozi et al., 2023; Hu et al., 2023), A basic principle of control theory that the controller sample rate should be in the order of magnitude of the sample rate of the sensors
models in resource-constraing
'ELLIS Unit, LIT AI Lab, Institute for Machine Learning. JKU Linz, Austria FNXAI GmbH, Linz, Austria 2 GGoogle DeepMind "Mila Québec Al Institute.
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
432 Tusks from G Domains
Large Recurrent Action Model
Figure I. Illustration of our Large Recurrent Action Model (LRAM) with an xLSTM (Beck et al., 2024) at its core.
(Franklin et al., 1998, Ch. 11). To illustrate this, for typ- ical robots such as drones or industrial robot arms, rates of 100Hz-1000Hz are required to keep the system stable (Salzmann et al., 2023; El-Hussieny, 2024; Hu et al., 2023; Chignoli et al., 2021). This implies inference times of less than 10ms. At 1000Hz, 15 15-second movement of the agent corresponds to sequence of 15K steps (El-Hussieny, 2024), resulting in long context lengths even without ICI. While there exists range of techniques to make large models faster, such as quantization (Frantar et al., 2023), distillation (Hinton al., 2015), or pruning (LeCun et al., 1989), the quadratic-time complexity of self attention still remains.
the modern recurrent architectures considered in this work can process this large-scale training set as efficiently as the Transformer, while being faster at inference. Experiments across 4 model sizes with our multi-task models indicate that LRAM compares favorably to Transformers in terms of both performance and speed. In addition, we study the effect of modern recurrent architectures fine-tuning per- formance and in-context learning abilities, and find that they exhibit strong performance in both dimensions.
The main purpose of this paper is to test the hypothesis that modern recurrent model architectures are better suited for building LAMs than Transformers. Hereby, we make the following contributions.
Recently, modern recurrent architectures have been pro- posed, which exhibit similar parallelization properties dur- ing training as the Transformer architecture while offering linear-time inference complexity. These modern recurrent architectures include xLSTM (Beck et al., 2024) and state- space models (SSMs), such as Mamba (Gu Dao, 2023; Dao & Gu, 2024) and Griffin/Hawk (De et al., 2024), and have challenged the dominance of the Transformer in lan- guage modeling but also in other domains such as computer vision (Alkin et al., 2024; Zhu et al., 2024), and biomedicine (Schmidinger et al., 2024). More importantly, their linear- time inference makes them suitable for deployment in sce- narios with limited compute, large context sizes, and real- time requirements, such as robotics.
We propose a Large Recurrent Action Model (LRAM) with an xLSTM at its core that enables efficient infer- ence.
We assess the aptitude of modern recurrent architec- tures as backbones for large-action models with re- spect to their efficiency at inference time and overall performance multi-task, fine-tuning, and in-context learning settings.
To foster further research on large action models, we release our data preparation pipeline and our datasets.!
In this work, assess the aptitude of modern recurrent architectures, such as XLSTM and Mamba, as large action models. To this end, we introduce Large Recurrent Action Model (LRAM) with an xLSTM at its core (see Figure 1). We train our agents on 432 tasks from 6 domains using a supervised learning setting similar to that of the Decision Transformer (Chen et al., 2021, DT). We use data collected during online-RL training of single-task specialist agents and compile these trajectories alongside other expert demon- strations into large-scale multi-domain dataset comprising 894M transitions. Due to their parallelization properties,
2. Related work
Sequence Models in RL. LSTM (Hochreiter & Schmidhu- ber, 1997) is the dominant backbone architecture for par- tially observable online RL problems and has been behind achievements such as mastering Starcraft II (Vinyals et al., 2019), Dota 2 (Berner et al., 2019), and Atari (Espeholt et al., 2018: Kapturowski et al., 2019). After the success of the Transformer NLP (Devlin et al., 2019; Radford et al., 2019; Brown al., 2020), computer vision (Dosovitskiy
GittHub: https: / thub. www.huccccc...cccooo.ooo..eee... /ml anljjee 1-jjkk/RRa
2

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
al., 2021; He al., 2022; Radford et al., 2021; Fürst et al., 2022) and speech recognition (Radford et al., 2022; Baevski et al., 2020), the architecture has found its way into RL. Chen et al. (2021) proposed the Decision Transformer (DT), a GPT--ttyle model (Radford et al., 2018), that learns to pre- dict actions from offline trajectories via behavior cloning. Trajectory Transformer (Janner et al., 2021) predicts actions along with states and rewards, which allows for dynamics modeling. Other follow-up up works build on the DT (Zheng et al., 2022; Wang et al., 2022; Shang et al., 2022; Meng et al., 2021; Siebenborn et al., 2022; Schmied et al., 2024a) or replace the Transformer with Mamba (Ota, 2024; Dai et al., 2024). Furthermore, sequence models trained to pre- dict the next action were found to exhibit ICL if conditioned on previous trajectories (Laskin et al., 2022; Lee et al., 2022: Kirsch et al., 2023), albeit in limited scenarios.
tion (Katharopoulos et al., 2020). Mamba-2 (Dao & Gu, 2024) highlights the connection between SSMs with input- dependent state and input matrices and (Gated) Linear atten- tion variants. Most recently, the xLSTM (Beck et al., 2024) was proposed as an improvement over the classic LSTM (Hochreiter & Schmidhuber, 1997) that combines gating, linear recurrences, and recurrent weights into a single ar- chitecture for language modeling. First, xLSTM leverages exponential gating with stabilization to RNNs for stronger emphasis on important inputs. Second, xLSTM is composed of two variants, the mLSTM variant with an emphasis on memory that proves important in language modeling, and the sLSTM variant that keeps the non-diagonalized recur- rent matrix to enable state-tracking (Merrill et al., 2024). State tracking is important in logic tasks and cannot be mod- eled fundamentally by linearized recurrent or state-space models like Mamba, Griffin, or Transformers.
Large Action Models (LAMs). LAMs, such as the De- cision Transformer, are well-suited for multi-task settings. Lee et al. (2022) found that a multi-game DT can learn to play 46 Atari games. Reed et al. (2022) introduced a gener- alist agent trained on over 600 tasks from different domains, ranging from Atari to manipulation of robot arm. Jiang et al. (2022) Transformer for robot manipulation based on multi modti- prompts, that allow to steer the model to perform new tasks. Recently, Raad et al. (2024) introduced an agent instructable via language to play a variety of com- mercial video games. Since then, robotics has become an emergent area for developing LAMs (Brohan et al., 2023b;a; Octo Model Team et al., 2024; Gu al., 2023; Wang et al., 2023; Kim et al., 2024), also due to the availability of large- scale datasets (Jia et al., 2024; Embodiment Collaboration et al., 2024; Jiang et al., 2023; Mandlekar et al., 2023).
3. Large Recurrent Action Models
3.1. Background
Reinforcement Learning. We assume the standard RL for- mulation via a Markov Decision Process (MIP) represented by tuple of (S, P, R), where S and A denote state and action spaces, respectively. At every timestep f, the agent observes state 8 E S, predicts action a, A, and receives scalar reward r;- The reward is determined by the reward function R(r; | 8t; 0z). P((s++1 [8,, az) defines the transi- tion dynamics and constitutes a probability distribution over next states 8t+1 when executing action ag in state 8,- The goal of RL to learn a policy x <aa 8t) that predicts an action 0e in state 8, that maximizes rt-
Next-generation Sequence Modeling Architectures. Lin- ear recurrent models, such as state-space models (SSM, Gu et al., 2021; 2022b; Smith et al., 2023; Orvieto et al., 2023) have challenged the dominance of the Transformer (Vaswani et al., 2017) architecture on long-range tasks (Tay et al., 2020). The key insight of those linear RNNs was to diagonalize the recurrent state matrix and enforce sta- ble training via an exponential parameterization (Gu et al., 2022a; Orvieto et al., 2023). Since then, there have been efforts to include features such as gating from RNNs (EI- man, 1990; Jordan, 1990; Hochreiter & Schmidhuber, 1997: Cho et al., 2014). Non-linear gates are believed have higher expressivity, but are harder to train. Griffin (De et al., 2024) mixes gated linear recurrences with local attention to achieve more training data efficiency than Llama-2 (Tou- vron et al., 2023) and better sequence extrapolation. Mamba (Gu& & Dao, 2023) introduces a selection mechanism sim- ilar to gating into SSMs, which makes its state and input matrix time-dependent. This is similar the gating mech- anism of RNNs but also bears resemblance to approaches like fast weights (Schmidhuber, 1992) and Linear Atten-
Decision Transformer (Chen et al., 2021) casts the RL problem setting as next action prediction task via causal sequence modeling. At training time, DT aims to learn a policy Ty that maps future rewards to actions, which is often referred to as upside-down RL (Schmidhuber, 2019). At in- ference time, the DT is conditioned via a target return to emit high-reward actions. Consequently, we assume access to dataset D = { containing N trajectories т, consisting of quadruplets T, (81, fū, a a1 Tl.-.., 8T, Po, @r, rT) of state st, return-to-go (RTG) Rr = Et= Tv, action at, and reward r,. Here, T refers to the length of the trajectory. The DT अe is trained to predict the ground-truth action a, conditioned sub-trajectories from the dataset:
âe ~ no(ār 1st-c, Ru- C,,
(I)
R.)
8t-1,
where C < T' is the size of the context window. In fact, Equation 1 describes the setting of the multi-game DT (Lee et al., 2022), which also includes rewards in the sequence representation.
3

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
Table I. Dataset statistics for all 432 training tasks.
Dataset
Tasks
Trajectories
Mean Trj. Length
Total Transitions
Repetitions
Atari
41
136K
2733
205M
1.03
Composuite
240
480K
500
240M
0.87
DMControl
11
110K
1000
110M
1.92
Meta-World
45
450K
200
90M
2.34
Mimicgen
83
83K
300
25M
8.5x
Proegen
12
2185K
144
224M
0.94x
Total
432
3.4M
894M
3.2. Large Recurrent Action Models (LRAMs)
the last timestep. This enables fast inference with linear- time complexity along the sequence length. In addition, the recurrent-style inference is well-suited for online fine-tuning via RL objectives, similar to LSTM- --asedd policies in online RL. To speed up inference, we leverage custom kernels for the xLSTM backbone (see Appendix 21).
Our LRAM has modern recurrent architecture at its core (see Figure 1), which comes with parallel training and a recurrent inference mode. We instantiate LRAM with three different variants, two different xLSTM configurations, and Mamba. We use training protocol similar to that of Lee et al. (2022) and Reed al. (2022) with important differences that aim speed up inference across backbones.
Our unified discrete action representation enables consistent training of our agents via the cross- cross-entropy loss as training objective across all tasks and domains, similar to Reed et al. (2022). We use separate reward scales per domain and target returns per task. Furthermore, we do not make use of timestep encodings as used by Chen al. (2021), which are detrimental when episode lengths vary. We provide additional implementation details in Appendix C.
Multi-modal sequence representation. To encode input from different environments with varying state and action spaces, we use separate encoders per modality that are shared across tasks and domains. For encoding images, we use CNN similar to Espeholt al. (2018), whereas for low-dimensional inputs we use fully connected network. We refrain from patchifying images and tokenizing continu- ous states to avoid unnecessarily long sequences. Similarly, we use linear layers to encode rewards and RTGs. We omit actions in our sequence formulation, as we found that this can be detrimental to performance, in particular for con- tinuous control tasks with smoothly changing actions (see Section 4.3). Consequently, our trajectories have the form 7i (81, R1, ri,...... ,.. Rr,ry and we train our policy Tp to predict the ground-truth action or
4. Experiments
We study the aptitude of modern recurrent architectures as LAMS on 432 tasks from domains: Atari (Bellemare et al., 2013), Composuite (Mendez al., 2022). DMControl (Tassa et al., 2018), Meta-World (Yu et al., 2020b), Mim- iegen (Mandlekar et al., 2023), and Proegen (Cobbe et al., 2020b). To this end, we compile large-scale dataset con- taining 894 million transitions (see Section 4.1). Across all experiments, we compare four backbone variants: xLSTM [7:1], XLSTM [1:0] (Beck et al., 2024), Mamba (Gu & Dao, 2023), and the GPT-2 style Transformer employed in the DT (Chen et al., 2021). Following (Beck al., 2024), we use the bracket notation for xLSTM, which indicates the ratio of MLSTM to sLSTM blocks. For example, XLSTM [1:0] contains only mLSTM blocks.
âg ~ Tāp |st-c, R
81-1, R1 Re).
Shared action head. Action spaces in RL typically vary across environments. For example, in the environments we consider, there are 18 discrete actions and maximum of 8 continuous dimensions for continuous control environments. Therefore, we employ discretization continuous action dimensions into 256 uniformly-spaced bins, similar to Reed al. (2022) and Brohan et al. (2023b). Unlike prior work, we leverage shared action head predict all discrete tions or continuous action dimensions jointly. We found that this setup significantly reduces inference time compared to using autoregressive action prediction of continuous actions.
In Section 4.2, we conduct a scaling comparison for four model sizes ranging from 16M to 206M parameters that shows that modern recurrent architectures achieve perfor- mance comparable or favorable to the Transformer baseline across different model sizes. In Section 4.3, we study the im- pact of the recurrent backbones on fine-tuning performance, ICL abilities, and further analyze our trained recurrent back- bones. Finally, in Section 4.4, we empirically examine the differences at inference time in terms of latency and through-
Recurrent inference mode, At inference time, we leverage the recurrent backbone and maintain the hidden states of

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
XLSTM [7:1]
XLSTM [1:0]
Mamba
DT
0.80
Validation Perplexity
46
Normalized Score
0.75
0.70
0.65
0.55
43
0.50
0.45
0.25
0.50
0.75
1.00
1.25
1.50 1.75
2.00
2,25
0.25 0.50 0.75 1.00 1.25 1.50 1.75
2.00 2,25
Number of Parameters
Number of Parameters
(a) Sequence prediction
(b) Environment interaction
Figure Scaling comparison. We compare xLSTM, Mamba, DT four model sizes: 16M, 48M, 110M, and 206M parameters. show the (a) validation perplexity on the hold-out datasets, and (b) normalized scores obtained from evaluating in the training task environments, averaged over all domains.
put between xLSTM and Transformer-based agents, which indicate advantages for the recurrent backbone.
4.2. Scaling comparison
To conduct our main comparisons, we train our four back- bone variants on the full training task mixture of 432 tasks. For each architecture backbone, we report performance scores for four model sizes: 16M, 48M, 108M. and 206M parameters. We train all models for 200K updates with batch size of 128 and a context length of 50 timesteps. All domains are represented with approximately equal pro- portion, resulting in 33K updates per domain. Additional implementation details and hyperparameters for every back- bone variant and model size are available in Appendix C.
4.1. Datasets & Environments
Datasets. We compile a large-scale dataset comprising 432 tasks from six domains. We leverage datasets from prior works if available, and generate our own data other- wise, For Atari, we extract 5M transitions per task from the DON-Replay dataset released by Agarwal et al. (2020). For Composuite, we leverage the datasets released by (Hussing et al., 2023). For Meta-We World, we use 2M transitions per task released by (Schmied et al., 2024a). For DMControl, we generate 10M transitions per task using spec-ipe RL agents. For Mimicgen, we use the datasets for the 21 tasks released by (Mandlekar et al., 2023) and generate trajecto- ries for the remaining 62 tasks. Finally, for Procgen, we tract 20M transitions from the datasets released by (Schmied et al., 2024b). Our final dataset contains 3.4M trajectories and in total 894M transitions (see Table 2). We reserve an additional 37 tasks from the same domains for zero-shot evaluation. To foster future research, we release our data- preparation pipeline and generated data. We provide the rationales for our specific dataset selection in Appendix B.
Sequence prediction performance. In Figure 2a, report the validation set perplexity for all backbones and model sizes averaged over the individual scores from all domains. To achieve this, we maintain a hold-out set of trajectories for each training task (2.5%) and compute the perplexities after every 50K steps (see Figure 12 for training perplexities). Both recurrent backbones outperform the Transformer base- line considerably, especially as the model sizes increase.
Evaluation performance. During training, evaluate our agents after every 50K step in all 432 training envi- ronments. In Figure 2b, we report the resulting normal- ized performances averaged across all six domains. The recurrent backbones outperform the Transformer one across model sizes. While xLSTM and Mamba perform similarly at smaller scales, xLSTM tends to outperform Mamba at larger scales (206M). This is important advantage of XLSTM, as LRAM agents can strongly benefit from more data and consequently larger models. Note that Mamba has a significantly higher number of parameters than competi- tors. For the zero-shot evaluation performances on the 37 hold-out tasks, we refer to Figure in Appendix D.2.
Environments. Atari and Procgen come with image obser- vations and discrete actions. In contrast, the remaining four domains exhibit state-based observations and continuous actions. Consequently, our experiments involve a mixture of state and action spaces as well as varying episode lengths (see Table 2). Periodically evaluating the trained agents all 432 tasks sequentially is time-consuming, and we, therefore, distributed the evaluation across GPUs and par- allel processes (see Appendix C). Additional details our datasets and environments are available in Appendix B.
5

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
XLSTM [7:1]
XLSTM [1:0]
Mamba
DT
1.0
Normalized aenaaliedd
0.6
0.4
0.0
mata-worid
dmcontrol
atar
procgen
mimicgen
composuite
Figure Normalized scores per domain model size 206M. For Meta-World DMControl, Mimiegen, Composuite, Proegen, we report aata--ormalized scores, for Atari report normaloor scores.
Performance per domain. In Figure 3, we report the nor- malized scores for the 206M models attained on all six domains. For Merl-World, DMControl, Mimicgen, Com- posuite, and Proegen, we use data-normalized scores, as suggested by (Levine et al., 2020). For Atari, we report scores. We observe that xLSTM outper- forms competitors on three of the six domains, while they perform similarly the remaining domains.
episodic context, which results large context length (see Appendix D.4 for details the experiment setup). In partic- ular, we adopt the Algorithm Distillation (AD, Laskin et al., 2022) framework and exchange the Transformer backbone architecture with modern recurrent architectures. In Figure 4, we report the ICL performance on the 20 hold-out tasks (see Figure 16 for training tasks). We find that xLSTM [7:1] attains the highest overall scores both the 80 training and 20 hold-out tasks, which we attribute the state-tracking abilities (Merrill et al., 2024) of sLSTM blocks.
4.3. Analyses Ablations
Fine-tuning. assess the effect of the recurrent backbones on fine-tuning performance, we fine-tune our models 37 held-out environments from all 6 domains. We evaluate the fine-tuning performance of the xLSTM architecture for the 16M pretrained models and compare it against an xLSTM trained from scratch. The pretrained LRAM outperforms the randomly initialized xLSTM model in most domains (see Figure 15). This suggests that fine-tund ten-ttnnng performance affected negatively by switching the backbone.
Embedding space analysis. In Figure 5, we analyze the representations learned by our model. We sample 32 sub- trajectories from every task, extract the sequence represen- tation at the last layer, cluster them using UMAP (McInnes et al., 2018), and color every point by its domain (see Ap- pendix F for more details). We find that tasks from the same domain cluster together. Furthermore, xLSTM ex- hibits a more refined domain separation compared to DT, which may further contribute to the better downstream per- formance. See Appendix F for a more detailed discussion on the embedding space analysis and a comparison to Mamba.
XLSTM [7:1]
XLSTM (1:0)
Mamba
AD
atari
composuite
dmcantrol
mimicgen
mc50
procgen
80
10
30
Step
Figure 4. In-context Learning with modem recurrent architec- 20 hold-out tasks for Dark-Room 10 10.
1907
(a) DT
(b) XLSTM
In-context Learning. Next, study the ICL abilities of our recurrent backbones on the Dark-Room environment considered in prior work on in-context RL (Laskin et al., 2022; Lee et al., 2023; Schmied et al., 2024b). To study ICL isolation, we train models from scratch with multi-
Figure 5. Embedding space comparison. UMAP clustering hidden states for all tasks for 16M colored by domain. XLSTM exhibits better domain separation DT.
6

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
121
XLSTM
Mamba
0.000
3.027
Latwey Libeey/tle se RN40
2,044
C.004
Latereg (thre
3,003
(itee
0.000
0.000
5.003
200
400 3602 2000 6420
3203025600
200
000
1803
2002 6950
25600
32
420 500
5005 6400
2200023800
Cor
Cantiet Langth
(a) Latency, B
(b) Latency, B 16
(c) Memory, B
Figure 6. Lateney comparison A100. We report latency for varying context lengths (in timesteps) with batch sizes (a) B = 1 and (b) = 16. (c), we show the memory consumption in GPU memory with B We compare DT to xLSTM and Mamba with the same number of layer blocks and parameters on Atari F Freeway. Missing bars for DT indicate out-of-memory (OOM).
Removing Actions & Effect of Context Length. We found that removing actions from the context results in better per- formance across backbones. While context lengths beyond 1 hurt performance on Meta-World and DMControl, and when training with actions, the reverse is true when training without actions (see Figures 23, 24, 26). This is in contrast to recent works, which did not benefit from longer contexts (Oeto Model Team al., 2024). While removing actions improves performance Meta- SettaWorlddDMConnrro not affect performance discrete control environments. For Meta-Worldie World/DMMomtroll, we observed that the models become overly confident, which is problematic poor ini- tial actions are produced. This is because many robotics environments exhibit smoothly changing actions, and by observing previous actions, the agent can learn shortcuts. A similar issue has been observed by Wen et al. (2020) and termed the copycal problem. Removing actions from the input prevents the agent from using shortcuts and, therefore, alleviates the copycat problem. Importantly, the evalua- tion performance improves across domains as the sequence length increases, which indicates that the history helps to predict the next action (e.g., by observing mistakes made in the past, see Figures 25, 27).
we maintain the same ratios for consistency (see Appendix C.3). While mLSTM is parallelizable, sLSTM enables state- tracking (Merrill et al., 2024). To better understand the effect of the ratio, we conduct ablation studies both on the 432 tasks and on Dark-Room (see Appendix E.3), similar to Beck et al. (2024). We find that other ratios, such as [3:1], can be effective, and highlight the importance of plac- ing sl SLMSM at lower-level layers (Figure 31). However, the effectiveness of sLSTM layers is dependent the task at hand. Complex tasks with long horizons or partial ob- servability, as are common in real-world applications, may benefit from the state-tracking abilities provided by sLSTM.
does
We present additional ablations on the effect of reducing the number of layers in xLSTM and disabling Dropout on DT in Appendix E.5 and E.4, respectively.
4.4. Inference Time Comparison
Finally. we empirically examine the difference between recurrent and Transformer-based agents inference time. Similar to De et al. (2024), we report both latency and throughput. We focus our analysis on latency, as it is the more important dimension for real-time applications.
Return-conditioning vs. Behavior Cloning. Across our experiments, we utilized a sequence representation that in- cludes return-togo tokens, as commonly used in DTss (Chen et al., 2021; Lee et al., 2022). However, many recent works focus on behavior cloning without return conditioning (Reed al., 2022: Brohan et al., 2023a). Therefore, we study the effect of excluding the RTG/reward tokens from the se-
Setup. We conduct all inference time tests on A100s with 40GB of RAM using 206M models. For the Transformer, we use KV-caching and FlashAttention (Dao, 2023) as sup- ported by PyTorch (Paszke al., 2019). For xLSTM, use recurrent-style inference using custom kernels to accel- erate computations (see Figure 21 for the impact of kernel acceleration). For Mamba, we make use of the kernels in- troduced by Gu Dao (2023). For DT and xLSTM, we use torch. comp 1e, but not for Mamba because we found the kernels to be incompatible with compilation. The Trans- former with KV-caching has a linear time complexity per step and quadratic in the sequence length. In contrast, the XLSTM and Mamba have constant time complexity per
quence at the 206M parameter scale, to validate that our findings transfer to the behavior cloning setting. Indeed, we find that the same trends hold (see Figures 28 and 29).
MLLST--to-LSSTM Ratio. Throughout experiments, we compare two xLSTM variants: xLSTM [7:1] and xLSTM [1:0]. These ratios were proposed by Beck al. (2024) and
7

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
step and are linear in the sequence length. Therefore, we expect speed-ups especially for longer sequences and larger batch sizes, as observed by De et al. (2024). To ensure fair comparison, we compare all backbones with the same number of layer blocks and increase the hidden size of xl.- STM and Mamba to match the number of parameters of DT (see Appendix E.5 for evaluation performance of these models). We provide further details on our inference time tests in Appendix D.5.
DT
XLSTM,
XLSTM, H16
Mamba
20000
15000
Thimaghpue
10000
5000
We conduct all inference time tests on the en- vironment that exhibited the longest average episode lengths in our experiments, the Atari game Freeway. Every episode in Freeway lasts for 8192 steps, which is equiv- alent to 24576 tokens (s/ttg/1). We evaluate all models for episodes and preserve the KV-cache/hidden state across episode boundaries. The reported latencies and throughputs are averaged across all evaluation episodes, except for the first episode, which we discard to exclude compilation times and prefilling. We opted for measuring the inference times during environment interaction, i.c., including simulator latency, rather than mere token generation.
Environment.
3
16
32
64
96
128
Batch size
Figure 7. Throughput comparison on A100 for varying batch sizes with = 1600 timesteps on the Atari Freeway environ- ment. We compare DT, xLSTM with 4 and 16 heads, and Mamba. Missing bars for DT indicate OOM.
with. For XLSTM, we report numbers for two variants with 4 and 16 heads, respectively. We found that decreasing the head dimension (more heads, same total hidden dim) is important for xLSTM to enable high throughput. This is because a higher head dimension incurs more FLOPS (see Figure 22 in Appendix D.5.4 for an ablation on the impact of the head dimension). As expected, we find that both Mamba and XLSTM attain considerably higher throughputs than the DT. These benefits increase with larger batch sizes. While the DT with quadratic complexity in the sequence length goes OOM for batch sizes above 64, the recurrent backbones with linear complexity can easily handle larger batch sizes. This throughput advantage may be particularly relevant for online fine-tuning of agents in many parallel environments.
Latency. Similar De al. (2024), we measure latency by the average time (in seconds) taken to perform single inference step with a fixed batch size B (lower is better). In Figure, 6, we report the latencies for varying context lengths, C' E [50, 25600] and two batch sizes B E {1, 16). Note that C' is in time steps, and every time step contains tokens (state, reward-to-ge to-go, reward). Hence, the effective sequence length for the largest C' is 76800. As expected, we find that the recurrent backbones attain lower inference latencies than the Transformer one, especially for longer sequences and with larger batch size. For B = 1, we find that Mamba is slower than the Transformer and XI,SMM which we believe is because of the incompatibility with torch. comp i 1 e. Note that we expect the gap to XLSTM to be closed with compatible kernels. As the sequence length increases, DT
5. Conclusion
In this work, we study the aptitude of modern recurrent archi- tectures as alternatives to Transformers for building LAMs. We found that our LRAM with an XLSTM or Mamba at its core compares favorably to the Transformer in terms of evaluation performance across model scales ranging from 16M to 206M parameters (see Section 4.2). Moreover, we demonstrated that LRAM exhibits higher inference speeds, especially at large context sizes (see Section 4.4). Thus, the empirical evidence suggests that recurrent backbones can be attractive alternatives for LAMs. Notably, the linear-time inference complexity of XLSTM and Mamba may enable applications that require long context lengths (e.g., ICL) and facilitate the application of large-scale agents for real-time applications, such as robotics.
runs out of memory due to the increasing size of the KV cache (see Figure 6c). In contrast, the inference speeds for Mamba/xI.STT LST are independent of the context length and therefore, enable significantly longer context lengths. This property is particularly interesting for in-context RL, which requires keeping multiple episodes the context (Laskin et al., 2022). Nevertheless, our experiments highlight that the materialization of the complexity advantage depends on the device, model size, batch size, and the context length, which is similar to findings by De et al. (2024).
Throughput. Throughput is measured by the total number of inference steps performed per second for a model with a fixed context length. In Figure 7, we report the throughputs for varying batch sizes, B E [1, 128) for a fixed context length of C 1600. Here, the batch size can be interpreted as the number of parallel environments the agent interacts
Modern recurrent architectures and Transformers come with different advantages and disadvantages. xLSTM and Mamba, on the one hand, exhibit a fundamental complex- ity advantage over Transformers. Their linear complexity
8

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
ensures that the computational requirements increase more slowly with the sequence length, which enables more effi- cient inference and is particularly relevant for edge applica- tions. While we conduct our inference time comparisons on a high-end data center GPU, applications on edge devices may have to deal with less powerful accelerators. Impor- tantly, we found that LAMs strongly benefit from longer sequences (see Section 4.3). Their ability to efficiently handle long sequences can be beneficial for applications in real-world environments, which often exhibit long-term dependencies. Similarly, longer context can be relevant for ICL applications, which benefit from keeping multi- ple episodes (such as demonstrations or previous trials) in the context. Transformers, on the other hand, are ef- fective for applications that require exact recall of tokens (such as particular locations in a grid, signs in an image) in sequence, which can be important for decision-make making (Ni et al., 2024). Finally, XLSTM in particular enables state-tracking via sLSTM blocks, which Transformers and Mamba cannot perform (Merrill et al., 2024). State tracking can be important for logic tasks or dealing with partial ob- servability and may be a useful tool for practitioners. Given these differences, different backbones should be considered depending on the task at hand.
unknown scenarios. We therefore strongly discourage users from blindly following the predictions made by real-world LAMs without appropriate precautions regarding safety and robustness. It is essential to ensure the responsible deploy- ment of such future technologies, and we helieve that more research on the robustness of LAMs is necessary.
Acknowledgements
We acknowledge EuroHPC Joint Undertaking for awarding us access to Karolina at ITT4Innovations, Czech Republic, MeluXina at LuxProvide, Luxembourg, and Leonardo at CINECA, Italy. The ELLIS Unit Linz, the LIT AI Lab, the Institute for Machine Learning, are supported by the Federal State Upper Austria. We thank the projects FWF AIRI FG 9- N (10.55776/FG9), A14Grren (FFG- 899943), Stars4Waters FWF Bilateral Artificial Intelligence (10.55776/C0E12), We thank NXAI GmbH, Audi AG, Silicon Austria Labs (SAL), Merck Healthcare KGaA, GLS (Univ. Waterloo), TÜV Holding GmbH, Software Competence Center Hagen- berg GmbH, dSPACE GmbH, TRUMPF SE + Co. KG.
References
Limitations & Future Work. The primary target appli- cation of LAMS is robotics. While the majority of our experiments involve robotic simulations, we do not yet pro- vide experiments for real robots. We do, however, believe that our findings translate real-world scenarios and aim to provide further evidence in future work. Moreover, our fine- tuning experiments are limited to offline RL. We envision that agent pre-trained on large-scale datasets can be suc- cessfully fine-tuned via online RL to explore new strategies that do not appear in the training data. Modern recurrent ar- chitectures offer both parallel and recurrent training modes, which might be the key to success for such applications. While we provide evidence for improved ICL abilities of LRAM, we only consider a grid-world setting. We aim to further investigate the ICL abilities of LRAM in more complex enviromments.
Agarwal, R., Schuurmans, D., and Norouzi, M. An opti- mistic perspective on offline reinforcement learning. In International Conference Machine Learning, pp. 104- 114. PMLR, 2020.
Agarwal, R., Schwarzer, M., Castro, P. S., Courville, A. C., and Bellemare, M. Deep reinforcement learning the edge of the statistical precipice. Advances in neural in- formation processing systems, 34:29304 29320, 2021.
Alkin, B., Beck, M., Pöppel, K., Hochreiter, S., and Brand- stetter, J. Vision-Istm: xlstm as generie vision backbone. CoRR, abs/2406.04300 2024. doi: 10.48550/ARXIV. 2406.04303. URL https: /tw.i/iiio//// http:/ii...gg//////////o 0001144/.0000..55555/ 48550/ a 1:xiv.24055 04303.
Baevski, A., Zhou, Y., Mohamed, A., and Auli, M. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in Neural Information Process- ing Systems, 33:12449- 2020.
Impact Statement
While we conduct all our experiments in simulated envi- ronments, the primary target application of our method is robotics. We believe that our work can positively impact ap- plications in the near future that require efficient inference, on-device processing, or have real-time constraints. How- ever, robotics applications in the real world are not without risks. In particular, in areas where humans are involved, such as factory settings, special care is required. LAMs are trained via next-action prediction similar to LLMs. Con- sequently, LAMs may also suffer from hallucinations in
Beck, M., Pöppel, K., Spanring, M., Auer, A., Prudnikova, O., Kopp, M., Klambauer, G., Brandstetter, J., and Hochreiter, S. xlstm: Extended long short-term memory. CoRR, 0ala/2005.05511 2024. doi: 10.48550/ARXIV. 2405.04517. URL https: /ttp://ii.ogg////////// /1005////..///////// arXiv. 24.iv.2405.00 04517.
Bellemare, M. G., Naddaf, Y., Veness, J., and Bowling. M. The Arcade learning environment: An evaluation plat-

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
form for general agents. Journal of Artificial Intelligence Research, 47:253-279, 2013.
Chignoli, M., Kim, D., Stanger-Jones, E., and Kim, S. The mit humanoid robot: Design, motion planning, and con- trol for acrobatic behaviors. In 2020 IEEE-RAS 20th Inter- national Conference on Humanoid Robots (Humanoids), pp. 1-8. IERE, 2021.
Bellemare, M. G., Candido, S., Castro, P. S., Gong, J., Machado, M. C., Moitra, S., Ponda, S. S., and Wang, Z. Autonomous navigation of stratospheric balloons using reinforcement learning. Nature, 588(783)) 2020.
Cho, K., van Merrienboer, B., Gülşehre, Ç., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Moschitti, A., Pang, B., and Daelemans, W. (eds.), Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, pp. 1724-1734. ACL, 2014. doi: 10.3115/ V1/D14-1179. URL https: /doi/dii.o/// wtw.//..org/.... 3115/
Berner, C., Brockman, G., Chan, B., Cheung, V., Dkebiak, P., Dennison, C., Farhi, D., Fischer, Q.. Hashme, S., Hesse, C., et al. Dota with large scale deep reinforce- ment learning. arXiv preprint arXiv: 19/2.06680 2019.
Brohan, A., Brown, N., Carbajal. J., Chebotar, Y. Chen, X., Choromanski, K., Ding, T., Driess, D., Dubey, A., Finn, C., et al. Rt-2: Vision-lagguage-action models transfer web knowledge to robotic control. arXiv preprinl arXiv:2307.15818 2023a.
Brohan, A., Brown, N., Carbajal, J., Chebotar, Y., Dabis, J., Finn, C., Gopalakrishnan, K., Hausman, K., Herzog, A., Hsu, J., Ibarz, J., Ichter, B., Irpan, A., Jackson, T., Jes- month, S., Joshi, J., Julian, R., Kalashnikov, D., Kuang, Y., Leal, I., Lee, K., Levine, S.. Lu, Y., Malla, U., Manju- nath, D., Mordatch, I., Nachum, O., Parada, C.,, Peralta, J., Perez, E., Pertsch, K., Quiambao, J., Rao, K., Ryoo, M. S., Salazar, G., Sanketi, P. R., Sayed, K., Singh, J., Sontakke, S., Stone, A., Tan, C., Tran, H. T., Vanhoucke, V., Vega, S., Vuong. Q., Xia, F., Xiao, T., Xu, P., Xu, S., Yu, T., and Zitkovich, B. RT-1: robotics transformer for real-world control at scale. In Bekris, K. E.,, Hauser, K., Herbert, L., and Yu, (eds.), Robotics: Science and Systems XIX. Daegu, Republic of Korea, July 10-14, 2023, 2023b. doi: 10. 10.15507/SSS 202007/SSS22223..2.... 2003 025. URL https: //doi. wwt.:////.............//......./ /w0.55677/...5
Cobbe, K., Hesse, C., Hilton, J., and Schulman, J. Lever- aging procedural generation to benchmark reinforcement learning. In International conference on machine learn- ing, pp. 2048-2056. PMLR, 2020a.
Cobbe, K., Hesse, C., Hilton, J., and Schulman, J. Lever- aging procedural generation to benchmark reinforcement learning. In Proceedings of the 37th International Con- ference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Ma- chine Learning Research, pp. 2048-2056. PMLR, 2020b. URL http: http:://pprceeee http://poeeeeeeneeeeeeeeee mlr www.s/pesssvv.e /v119/ cobbe20a. htmml..
Dai, Y., Ma, O.,, Zhang, L.,, Liang, X., Hu, S., Wang, M., Ji, S., Huang. J., and Shen, L. Is mamba compatible with trajectory optimization in offline reinforcement learning? arXiv preprint arXiv:2405.1 1204,, 2024.
XIX. 025.
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.,
Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G.,
Askell, A., Agarwal, S., Herbert-Vossse A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D. Language models are few-shot learners. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 1877-1901. Curran Associates, Inc., 2020. URL https: http:///ccceeeeeengeeeee neurips. oc ccc/ppppeefflllesspper///////// 4.5Tccdabfaaa96671111182l.....d pdf.
Dao, T. Flashattention-2: Faster attention with bet- ter parallelism and work partitioning. arXiv preprint arXiv:2307.08691, 2023.
Dao, T. and Gu, A. Transformers are ssms: Generalized models and efficient algorithms through structured state space duality. arXiv preprint arXiv:2405.2 2024.
De, S., Smith, S. L., Fernando, A., Botev, A., Cristian- Muraru, G., Gu, A., Haroun, R., Berrada, L., Chen, Y., Srinivasan, S.,, et al. Griffin: Mixing gated linear recur- rences with local attention for efficient language models. arXiv preprint arXiv: 19427, 2024.
Chen, I.,, Lu, K., Rajeswaran, A., Lee, K., Grover, A., Laskin, M., Abbeel, P., Srinivas, A., and Mordatch, I. De- cision transformer: Reinforcement learning via sequence modeling. Advances in neural information processing systems, 34:1 5084-15097, 2021.
Degrave, J., Felici, F., Buchli, J., Neunert, M., Tracey, B., Carpanese, F., Ewalds, T., Hafner, R., Abdolmaleki.. A., de Las Casas, D., et al. Magnetic control of tokamak plasmas through deep reinforcement learning. Nature, 602(7897):4144419 2022.
10

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
Devlin, J., Chang, M., Lee, K., and Toutanova, K. BERT: pre-training of deep bidirectional transformers for lan- guage understanding. In Burstein, J., Doran, C., and Solorio, T. (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolo- gies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume I (Long and Short Papers), pp. 4171- 4186. Association for Computational Linguistics, 2019. doi: 10. 10.186/3vv//11 1/1 1423.
Tomizuka, M., Itkina, M., Castro, M. G., Spero, M., Du, M., Ahn, M., Yip, M. C.,, Zhang, M., Ding, M., Heo, M., Srirama, M. K., Sharma, M., Kim, M. J., Kanazawa, M., Hansen, N., Heess, N., Joshi, N. J., Suenderhauf, N., Liu, N., Palo, N. D., Shafiullah, N., Mees, O., Kroemer, O., Bastani, O., Sanketi, P. R., Miller, P., Yin, P., Wohlhart, P., Xu, P., Fagan, P., Mitrano, P., Sermanet, P., Abbeel, P., Sundaresan, P., Chen, Q., Vuong, Q. Rafailov, R., Tian, R., Doshi, R., Martin,,artin, R., Baijal, R., Scalise, R., Hendrix, R., Lin, R. Qian, R., Zhang, R., Mendonca, R., Shah, R., Hoque, R., Julian, R., Bustamante, S., Kirmani, S., Levine, S., Lin, S., Moore, S., Bahl, S., Dass, S., Son- awani, S., Song, S., Xu, S., Haldar, S., Karamcheti, S., Adebola, S., Guist, S., Nasiriany, S., Schaal, S., Welker, S., Tian, S., Ramamoorthy, S., Dasari, S., Belkhale, S.. Park, S., Nair, S., Mirchandani, S., Osa, T., Gupta, T., Harada, T., Matsushima, T., Xiao, T., Kollar, T., Yu, T., Ding, T., Davchev, T., Zhao, T. Z.,, Armstrong, T., Darrell, T., Chung, T., Jain, V., Vanhoucke, V., Zhan, W., Zhou, W., Burgard, W., Chen, X., Wang, X., Zhu, X., Geng, X., Liu, X., Liangwei, X., Li, X., Lu, Y., Ma, Y., Kim, Y., Chebotar, Y., Zhou, Y., Zhu, Y., Wu, Y., Xu, Y., Wang. Y., Bisk, Y., Cho, Y., Lee, Y., Cui, Y., Cao, Y., Wu, Y., Tang, Y.. Zhu, Y., Zhang, Y., Jiang, Y., Li, Y., Li, Y., Iwasawa, Y., Matsuo, Y., Ma, Z., Xu, Z., Cui, Z., Zhang, Z., Fu, Z., and Lin, Z. Open x x-embodiment: Robotic learning datasets and rt-x models, 2024.
Dosovitskiy, A., Beyer, L.,, Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. An image is worth 16x16 words: Transformers for image recognition at scale. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.nee 2021.
El-Hussieny, H. Real-time deep learning- model pre- dictive control of 3-dof biped robot leg. Scientific Re- ports, 14(1):16243, 2024.
Elman, J. L. Finding structure in time. Cogn. Sci., 14(2): 179-211, 1990. doi: 10. 10.1207/1151117700000000222 URL
ht s1 s1551 670160779ogg110 10
1207/
Embodiment Collaboration, O'Neill, A., Rehman, A., Mad- dukuri, A., Gupta, A., Padalkar, A., Lee, A., Pooley, A., Gupta, A., Mandlekar, A., Jain, A., Tung, A., Bewley, A., Herzog, A., Irpan, A., Khazatsky, A., Rai, A., Gupta, A., Wang, A., Singh, A., Garg, A., Kembhavi, A., Xie, A., Brohan, A., Raffin, A., Sharma, A., Yavary, A., Jain, A., Balakrishna, A., Wahid, A., Burgess-Limerick, B., Kim, B., Schölkopf, B., Wulfe, B., Ichter, B., Lu, C., Xu, C.,, Le, C., Finn, C., Wang, C., Xu, C., Chi, C., Huang, C., Chan, C., Agia, C., Pan, C., Fu, C., Devin, C., Xu, D., Morton, D., Driess, D., Chen, D., Pathak, D., Shah, D., Büchler, D., Jayaraman, D., Kalashnikov, D., Sadigh, D., Johns, E., Foster, E., Liu, F. Ceola, F., Xia, F., Zhao, F., Stulp, F., Zhou, G., Sukhatme, G. S., Salhotra, G., Yan, G., Feng, G., Schiavi, G.,, Berseth, G., Kahn, G., Wang, Su, H., Fang, H., Shi, H., Bao, H., Amor, H. B., Chris- tensen, H. I., Furuta, H., Walke, H., Fang, H., Ha, H., Mordatch, I., Radosavovic, I., Leal, I., Liang, J., Abou- Chakra, J., Kim, J., Drake, J., Peters, J., Schneider, J., Hsu, J., Bohg, J., Bingham, J., Wu, J., Gao, J., Hu, J., Wu, J. Wu. J., Tan, J.. Oh, J., Wu. J., Lu, J., Yang. J., Sal- vador, J., Lim, J. J., Han, J.. Wang, K., Rao, K. Pertsch, K., Hausman, K., Go. K., Gopalakrishnam, K., Goldberg, K., Byrne, K., Kawaharazuka, K., Black, K., Lin, K., Zhang, K., Ehsani, K., Lekkala, K., Ellis, K., Rana, K., Fang, K., Singh, K., Zeng, K., Hatch, K., Hsu, K., Itti, L., Chen, L. Y. Pinto, L.. Fei-Fei, L., Tan, L... Fan, L., Ott, L., Lee, L.,, Weihs, L.., Chen, M., Lepert, M., Memmel, M.,
Espeholt, L., Soyer, H., Munos, R., Simonyan, K., Mnih, V., Ward, T., Doron, Y., Firoiu, V., Harley, T., Dunning, I., et al. Impala: Scalable distributed deep-rl with im- portance weighted actor-learner architectures. In Interna- tional conference on machine learning, pp. 1407-1416. PMLR, 2018
Firoozi, R., Tucker, J., Tian, S., Majumdar, A., Sun, J., Liu, W., Zhu, Y., Song, S., Kapoor, A., Hausman, K., et al. Foundation models in robotics: Applications, challenges, and the future. The International Journal of Robotics Research, pp. 027836492412815008 2023.
Franklin, G. Powell, J. D., Workman, M. L., et al. Digital control of dynamic systems, volume 3. Addison- -eesley Menlo Park, 1998.
Frantar, E., Ashkboos, S., Hoefler, T., and Alistarh, D. OPTQ: accurate quantization for generative pre- trained transformers. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali. Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL https:/ httpp:://ppereviiit http///eeeeeeeeeee////ete net/ forum? id-tcbBPnfwxS.
Fürst, A., Rumetshofer, E., Lehner, J., Tran, V., Tang, F., Ramsauer, H., Kreil, D., Kopp, M., Klambauer, G., Bitto-

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
Nemling, A., and Hochreiter, S. Cloob: Modern hopfield networks with infoloob outperform clip, 2022.
He, K., Chen, X., Xie, S., Li, Y., Dollár, P., and Girshick, R. B. Masked autoencoders are scalable vision learners. In IEEE/CVF Conference Computer Vision and Pat- tern Recognition, CVPR 2022. New Orleans, LA, USA, June 18-24, 2022, pp. 15979-15988. IEFE, 2022. doi: 10.1 N0.1109CCVPR522882222.222
Gu, A. and Dao, T. Mamba:
Linear-time sequence modeling with selective state spaces. CoRR, abs/aals//aaaaeea 12.00752, 2023. doi: 10.48550ARXXV.2312007777 moi..org/10..88335 2tt.:44488550001112122....0..000 00752.
URL https://
Hessel, M., Modayil, J., van Hasselt, H.,, Schaul, T., Ostro- vski, G., Dabney, W., Horgan, D., Piot, B., Azar, M. G., and Silver, D. Rainbow: Combining improvements in deep reinforcement learning. ArXiv, 2017.
Gu, A., Johnson, I., Goel, K., Saab, K., Dao, T., Rudra, A., and Ré, C. Combining recurrent, convolutional, models with linear state space layers. In Ranzato, M., Beygelzimer, A., Dauphin, Y. N., Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurlPS 2021, December 6-14, 2021, virtual, pp. 572-585, 2021. URL
and continuous-timg
Hinton, G. E., Vinyals, O., and Dean, J. Distilling the knowl- edge in neural network. CoRR, URL http: www.aawwww..a..aaaaao..ooooooooe aaacg//aa////e abs// attg//bb/11533025511
2015, 02531.
Hochreiter, S. and Schmidhuber, J. Long short-term memory. Neural Comput., 98): 1997.
neurips ec wttpp//peee/////e/e/eeee/e/e/e/e (144600288b09755c005eececcc html.
Abstract.
Hu, Y., Xie, Q. Jain, V., Francis, J., Patrikar, J., Keetha, N., Kim, S., Xie, Y., Zhang, T., Zhao, Z., et al. Toward general purpose robots via foundation models: A survey and meta-analysis. arXiv preprint arXiv:2312.08782. 2023.
Gu, A., Goel, K., Gupta, A., and Ré, C. On the parameteri- zation and initialization of diagonal state space models. Advances in Neural Information Processing Systems, 35: 2022a.
Hussing, M., Mendez, J. A., Singrodia, A., Kent, C., and Eaton, Robotic manipulation datasets for offline compositional reinforcement learning. arXiv preprint arXiv:2307.0709) 2023.
Gu, A., Goel, K., and Ré, C. Efficiently modeling long sequences with structured state spaces. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenRe- view.net, 2022b. URL https: opwnreview..w.eee///e/ee /ettt forum?
Janner, M., Li, Q., and Levine, S. Offline reinforcement learning as one big sequence modeling problem. Ad- vances neural information processing systems, 34: 1273-1286, 2021.
Gu, J., Kirmani, S., Wohlhart, P., Lu, Y., Arenas, M. G., Rao, K., Yu, W., Fu, C., Gopalakrishnan, K., Xu, 7 Sundaresan, P., Xu, P., Su, H., Hausman, K., Finn, C., Vuong, Q., and Xiao, T. Rt-trajectory: Robotic task generalization via hindsight trajectory sketches, 2023.
Jia, X., Blessing, D., Jiang, X., Reuss, M., Donat, A., Li- outikov, R., and Neumann, G. Towards diverse behav- iors:
A benchmark for imitation learning with human demonstrations. In The Twelfth International Confer- ence Learning Representations, 2024. URL https: / //openreview.ct net wwwenneeeootuure forum?id-----o 6pP YRXKPpw.
Gu, X., Wang, Y.J.,, and Chen, Humanoid-gym: Rein- forcement learning for humanoid robot with zero-shot sim2real transfer, 2024.
Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. Soft actor-critic: Off-policy maximum entropy deep reinforce- ment learning with stochastic actor. In Dy, J. G. and Krause, A. (eds.), Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stock- holmsmässan, Stockholm, Sweden, July 10-15, 2018 vol- ume 80 of Proceedings of Machine Learning Research, pp. 1856-1865. PMLR, 2018
Jiang, Y., Gupta, A., Zhang, Z., Wang, G., Dou, Y., Chen, Y., Fei-Fei, I.., Anandkumar, A., Zhu, Y., and Fan, Vima: General robot manipulation with multimodal prompts. arXiv preprint arXiv: 2022.
Jiang, Y., Gupta, A., Zhang, Z., Wang, G., Dou, Y., Chen, Y., Fei-Fei, L., Anandkumar, A., Zhu, Y., and Fan, L. Vima: General robot manipulation with multimodal prompts, 2023.
Hafner, D., Lillicrap. T., Fischer, I., Villegas, R., Ha, D., Lee, H., and Davidson, J. Learning latent dynamics for planning from pixels. In International conference on machine learning, pp. 2555 25655. PMLR, 2019.
Jordan, M. Attractor dynamics and parallelism in a con- nectionist sequential machine, pp. 112-127. IEEE Press, 1990. ISBN 081 8620153.
12

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
Kapturowski, S., Ostrovski, G., Dabney, W., Quan, J., and Munos, R. Recurrent experience replay in dis- tributed reinforcement learning. In International Confer- ence on Learning Representations, 2019. URL https: //openreview eww. net formmoormmpmm d= yyy
Loshchilov, I. and Hutter, F. Decoupled weight decay reg- ularization. In International Conference Learning Representations, 2018.
Mandlekar, A., Nasiriany, S., Wen, B., Akinola, I., Narang, Y., Fan, L.., Zhu, Y., and Fox, D. Mimicgen: A data gen- eration system for scalable robot learning using human demonstrations, 2023.
Katharopoulos, A.,
Vyas, A., Pappas, N., and Fleuret, Transformers are rnns: Fast autoregressive transformers with linear attention. In International conference on ma- chine learning, pp. 5156-5165 PMLR, 2020.
McInnes, L., Healy, J., and Melville, J. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:18022 03426, 2018.
Kim, M. J., Pertsch, K., Karamcheti, S., Xiao, T., Balakr- ishna, A., Nair, S., Rafailov, R., Foster, E., Lam, G., San- keti, P., et al. Openvla: An open-source vision-language- action model. arXiv preprint arXiv:2406.092. 2024.
Mendez, J. A., Hussing, M., Gummadi, M., and Eaton, E. Composuite: A compositional reinforcement learn- ing benchmark. In Chandar, S., Pascanu, R., and Pre- cup, D. (eds.), Conference on Lifelong Learning Agents, CoLLAs 2022, 22-24 August 2022. McGill University, Montréal, Québec, Canada, volume 199 of Proceedings of Machine Learning Research, pp. 982-1003, PMLR, 2022. URLhttps: htthttts://ppoeeeeeeeeee proceed ngs ml www.mmrr.pprs.... presss/ v199 1999/mende2222 html.
Kim, S., Hooper, C., Wattanawong. T., Kang, M., Yan, R., Genc, H., Dinh, G., Huang, Q., Keutzer, K., Mahoney, M. W., et al. Full stack optimization of transformer infer- ence: a survey. arXiv preprint wrXv:23322.1017,, 2023.
Kirsch, L.. Harrison, J., Freeman, C., Sohl-Dickstein, J., and Schmidhuber, J. Towards general- general-purpose in-context learning agents. In NeurIPS 2023 Workshop General- ization in Planning, 2023.
Meng, L., Wen, M., Yang. Y., Le, C., Li, X., Zhang, W., Wen, Y., Zhang, H., Wang. J., and Xu, B. Offline pre- trained multi-agent decision transformer. One big se- quence model conquers all starcraftii tasks. arXiv preprint arXiv:2112.02845. 2021.
Laskin, M., Lee, K., Stooke, A., Pinto, L., Abbeel, P., and Srinivas, A. Reinforcement learning with augmented data. ArXiv, 14990, 2020.
Merrill, W., Petty, J., and Sabharwal, A. The illusion of state in state-space models. CoRR, abs/2404.088 19, 2024. doi: 10.45550ARXXV22444.000555 doi doi.orye org/org/100 000444855500/111 2404. 08819.
Laskin, M., Wang, L., Oh, J.. Parisotto, E., Spencer, S., Steigerwald, R., Strouse, D., Hansen, S., Filos, A., Brooks, E., et al. In-context reinforcement learning with algorithm distillation. arXiv preprint arXiv:2210.14211 2022.
URL https: //
Micikevicius, P., Narang, S., Alben, J., Diamos, G., Elsen, E., Garcia, D., Ginsburg, B., Houston, M., Kuchaiev, O., Venkatesh, G., et al. Mixed precision training. arXiv preprint 170.0377,0 2017.
LeCun, Y., Denker, J. S., and Solla, S. A. Opti- mal brain damage. In Touretzky, D. S. (cd.), Ad- vances in Neural Information Processing Systems 2, [NIPS Conference, Denver, Colorado, USA, Novem- ber 27-30, 1989], pp. 598-605. Morgan Kaufmann, 1989. URL http: http://pppprrpppp..p. 150-optima--bbainne wtt.eerss..ips..cppppp.e.ee
Mnih, V. Kavukcuoglu, K., Silver, D.,, Rusu, A. A., Ve- ness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wier- stra, D., Legg. S., and Hassabis, D. Human-level control through deep reinforcement learning. Nature, 518(7540): 529-533, 2015. doi: (0.1038/nature144 14236.
Lee, J. N., Xie, A., Pacchiano, A., Chandak, Y., Finn, C., Nachum, O., and Brunskill, F. Supervised pretraining can leam in-context reinforcement learning, arXiv preprint arXiv:2306. 14892, 2023.
Ni, T., Ma, M., Eysenbach, B., and Bacon, P.-L. When do transformers shine in rl? decoupling memory from credit assignment. Advances in Neural Information Processing Systems, 36, 2024.
Lee, K.-H., Nachum, O., Yang, M., Lee, L., Freeman, D., Xu, W., Guadarrama, S., Fischer, I., Jang, E., Michalewski, H., et al. Multi-game decision transformers. arXiv preprint arXiv:2205 12220051524. 15241, 2022.
Octo Model Team, Ghosh, D., Walke, H., Pertsch, K., Black, K., Mees, O., Dasari, S., Hejna, J., Kreiman, T., Xu, C., Luo, J., Tan, Y. L., Sanketi, P., Vuong, Q., Xiao, T., Sadigh, D... Finn, C., and Levine, S. Octo: An open- source generalist robot policy, 2024.
Levine, S., Kumar, A., Tucker, G., and Fu, J. Offline rein- forcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint arXiv:2005.0 0/643,, 2020.
13

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
Orvieto, A., Smith, S. L.,, Gu, A., Fernando, A., Gülçehre, Ç., Pascanu, R., and De, S. Resurrecting recurrent neural networks for long sequences. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), International Conference on Ma- chine Learning, ICML 2023, 23-29 July 2023, Hon- olulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pp. 26670-26698, PMLR,
scale weak supervision. arXiv preprint arXiv:2212.043555 2022.
Raparthy, S. C., Hambro, E., Kirk, R., Henaff, M., and
Raileanu, R. Generalization to new sequential decision making tasks with in-context learning, 2023.
Reed, S. E., Zolna, K., Parisotto, E., Colmenarejo, S. G., Novikov, A., Barth-Maron, G., Gimenez, M., Sulsky. Y., Kay, J., Springenberg. J. Fecles, T., Bruce, J., Razavi, A., Edwards, A., Heess, N., Chen, Y., Hadsell, R., Vinyals, O., Bordbar, M., and de Freitas, N. A generalist agent. CoRR, abs/2205.06175 2022. doi: 10.48550/rxXiv22205. 06175.
2023. URL.ttps: http:////cceedeeeeeeee ngs m.l..press/ htt////vveeeo2aa..m.
Ota, T. Decision mamba: Reinforcement learning via sequence modeling with selective state spaces. arXiv preprint arXiv:2403. 1992: 2024.
Salzmann, T., Kaufmann, E., Arrizabalaga, J., Pavone, M., Scaramuzza, D., and Ryll, M. Real-time neural mpe: Deep learning model predictive control for quadrotors and agile robotic platforms. IEEE Robotics and Automation Letters, 8(4):2397- 2023.
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et Pytorch: An imperative style, nigh-performance deep learning library. Advances in neural information processing systems, 32, 2019.
Schmidhuber, Learning to control fast-weight memo- ries: An alternative to dynamic recurrent networks. Neu- rul Comput., 4(1):131-139) 1992. doi: 10. 10.1 162/NEOO 1992.4.1.131. URL httss:: /tt./oioooooooo//ooooo wtwp:oot.....o.oooooooo.. 1162/ neco 19 1992.99999. ....9........................... 131.
Patil, V., Hofmarcher, M., Dinu, M., Dorfer, M., Blies, A., and Hochre- iter, S. Align-rudder: Learning from few demonstrations by reward redistribution. In Chaudhuri, K., Jegelka, S., Song, L., Szepesvári, C., Niu, G., and Sabato, S. (eds.), International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, vol- ume 162 of Proceedings of Machine Learning Research, pp. 17531-17572. PMLR, 2022.
M., Brandstetter, J., Aajona-Medinng
Schmidhuber, J. Reinforcement learning upside down: Don't predict rewards-just map them to actions. arXiv preprint 1912v 2019.
Schmidinger, N., Schneckenreiter, L., Seidl, P., Schimunek, J., Luukkonen, S.,, Hoedt, P.-.., Brandstetter, J., Mayr, A., Hochreiter, S., and Klambauer, G. Bio-xlstm: Genera- tive modeling, representation and in-context learning of biological and chemical sequences. Under reveiw, 2024.
Raad, M. A., Ahuja, A., Barros, C., Besse, F., Bolt, A., Bolton, A., Brownfield, B., Buttimore, G., Cant, M., Chakera, S., et al. Sealing instructable agents across many simulated worlds. arXiv preprint arXiv:2404.10119, 2024.
Schmidt, D. and Schmied, T. Fast and data-efficient training of rainbow: experimental study on atari. arXiv preprint arXiv:2111.1000 2021.
Radford, A., Narasimhan, K., Salimans, T., Sutskever, I., et al. Improving language understanding by generative pre-training. 2018.
Schmied, T., Hofmarcher, M., Paischer, F., Pascanu, R., and Hochreiter, S. Learning modulate pre-trained models in rl. Advances in Neural Information Processing Systems, 36, 2024a.
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, I. Learning transferable visual models from natural language supervision. In Meila, M. and Zhang, T. (eds.), Proceedings of the 38th International Conference on Machine Learning, ICMI. 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pp. 8748- 8763. PMLR. 2021.
Schmied, T., Paischer, F., Patil, V., Hofmarcher, M., Pas- canu, R., and Hochreiter, S. Retrieval augmented de- cision transformer: External memory for in-context rl. arXiv preprint arXiv:2410.07077 2024b.
Schulman, J., Wolski, F,, Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms. ArXiv, 2018.
Schwarzer, M., Ceron, J. S. O., Courville, A., Bellemare, M. G., Agarwal, R., and Castro, S. Bigger, better, faster: Human-level atari with human-level efficiency.
Radford, A., Kim, J. W., Xu, T., Brockman, G., McLeavey, C., and Sutskever, I. Robust speech recognition via large-
14

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
In International Conference on Machine Learning, pp. 30365-30380. PMLR, 2023.
Todorov, E., Erez, T., and Tassa, Y. Mujoco: A physies engine for model-based control. In 2012 IEEE/RSI Inter- national Conference on Intelligent Robots and Systems, pp. 5026550260000 5033. IEEE, 2012b.
Schweighofer, K., Dinu, M.-.,, Radler, A., Hofmarcher, M., Patil, V. P., Bitto-Nemling, A., Eghbal-zadeh. H., and Hochreiter, S. A dataset perspective on offline rein- forcement learning. In Conference Lifelong Learning Agents, pp. 470-517. PMLR, 2022.
Touvron, H., Martin, L.,, Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., Bikel, D., Blecher, L., Canton-Ferree C., Chen, M., Cucurull, G., Esiobu, D., Fernandes, J., Fu, J., Fu, W., Fuller, B., Gao, C., Goswami, V., Goyal, N., Hartshorn, A., Hosseini, S., Hou, R., Inan, H., Kardas, M., Kerkez, V., Khabsa, M., Kloumann, I., Korenev, A.. Koura, P. S., Lachaux, M., Lavril, T., Lee, J., Liskovich, D., Lu, Y., Mao, Y., Martinet, X., Mihaylov, T., Mishra, P., Molybog, I. Nie, Y., Poulton, A., Reizenstein, J., Rungta, R., Saladi, K.. Schelten, A., Silva, R., Smith, E. M., Sub- ramanian, R., Tan, X. E., Tang, B.. Taylor, R., Williams, A., Kuan, J. X., Xu, P., Yan, Z., Zarov, I., Zhang, Y., Fan, A., Kambadur, M., Narang, S., Rodriguez, A., Stojnic, R., Edunov, S., and Scialom, T. Llama 2: Open founda- tion and fine-tuned chat models. CoRR, abs/2307.09288. 2023, doi: 10. 10.48550/AAXXXI2337..0000
Shang, J.. Kahatapitiya, K., Li, X., and Ryoo, M. Star- former: Transformer with Sate-action-rewar represen- tations for visual reinforcement learning. In European Conference on Computer Vision, pp. 462-479. Springer, 2022.
Siehenborn, M., Belousov, B., Huang, J., and Peters, J. How crucial is transformer in decision transformer? arXiv preprint ar arXiv:2211.1460
2022.
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., Dieleman, S., Grewe, D., Nham, J., Kalchbrenner, N., Sutskever, I., Lillicrap, T. P., Leach, M., Kavukcuoglu, K., Graepel, T., and Has- sabis, D. Mastering the game of Go with deep neural networks and tree search. Nature, :29(7587):4844488 2016. doi: 10. 16910
URL https: 10. 48540 attpv.riivvi.// 2307 09288.
Vaswani, A., Shazeer, N.,, Parmar, N., Uszkoreit, Jones, L.., Gomez, A. N., Kaiser, I., and Polosukhin, I. Atten- tion is all you need. Advances in neural information processing systems, 30, 2017.
Smith, J. T. H.. Warrington, A., and Linderman, S. W. Sim- plified state space layers for sequence modeling. In The Eleventh International Conference on Learning Repre- sentations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL net/ wowumoruumimom id=ii 8HW3Axqks.
Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M.,
Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., Oh, J., Horgan, D.. Kroiss, M., Danihelka, I., Huang, A., Sifre, L., Cai, T., Agapiou, J. P., Jaderberg, M., Vezhnevets, A. S., Leblond, R., Pohlen, T., Dalibard, V., Budden, D., Sulsky, Y., Molloy, J., Paine, T. L.., Güllçehre, Ç., Wang, Z., Pfaff, T., Wu, Y., Ring, R., Yogatama, D., Wünsch, D., McKinney, K., Smith, O., Schaul, T., Lillicrap. T. P., Kavukcuoglu, K., Hassabis. D., Apps, C., and Silver, D. Grandmaster level in starcraft II using multi-agent reinforcement learning. Nat., 575(7782)3350- 354, 2019. doi: 10.1038/s41586-1 110--019-1777 1724-z.
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1):1929-199 1958, 2014.
Tassa, Y. Doron, Y., Muldal, A., Erez, T. Li, Y., de Las Casas, D., Budden, D., Abdolmaleki, A., Merel, J., Lefrancq, A., Lillicrap, T. P., and Riedmiller, M. A. Deepmind control suite. CORR, abs/180) /////8001.0690, 2018.
Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L., and Anandkumar, A. Voyager: An open- ended embodied agent with large language models, 2023.
Tay, Y., Dehghani, M., Abnar, S., Shen, Y., Bahri, D., Pham, P., Rao, J., Yang, L., Ruder, S., and Metzler, D. Long range arena: A benchmark for efficient transformers. arXiv preprint arXiv:2011.04000 2020.
Wang, K., Zhao, H., Luo, X., Ren, K., Zhang, W., and Li, D. Bootstrapped transformer for offline reinforeement learning. arXiv preprint arXiv:2206. 200.:22266.88699 2022.
Wen, C., Lin, J., Darrell, T., Jayaraman, D., and Gao, Y. Fighting copycat agents in behavioral cloning from ob- servation histories. Advances in Neural Information Pro- cessing Systems, 33:2564-2575. 2020.
Todorov, E., Frez, T., and Tassa, Y. MuloCo: A physics engine for model-based control. In 2012 IEEE/RSJ Inter- national Conference on Intelligent Robots and Systems, pp. 5026-5033, October 2012a. doi: 10.1109/IROS.000 2012. 6386109.
Wolezyk, M., Zajkac, M., Pascanu, R., Kuciński, L., and Milos, P. Continual world: A robotic benchmark for
15

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
continual reinforcement learning. Advances in Neural
Information Processing Systems, 34:28496-28510 2021.
Yu, T., Kumar, S., Gupta, A., Levine, S., Hausman, K., and Finn, C. Gradient surgery for multi-task learning. Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Information Process- ing Systems 33: Annual Conference on Neural Informa- tion Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020a.
Yu, T., Quillen, D., He, Z., Julian, R., Hausman, K., Finn, C., and Levine, S. Meta-world: A benchmark and evalua- tion for multi-task and meta reinforcement learning. In Conference on robot learning, pp. 1094-1100. PMLR, 2020b.
Zheng, Q.. Zhang, A., and Grover, A. Online decision transformer. In Chaudhuri, K., Jegelka, S., Song, I.., Szepesvári, C., Niu, G., and Sabato, S. (eds.), Interna- tional Conference on Machine Learning. ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pp. 27042-27059, PMLR, 2022.
Zhu, G., Lin, Z., Yang, G., and Zhang, C. Episodic reinforce- ment learning with associative memory. In International Conference on Learning Representations, 2020.
Zhu, L., Liao, B., Zhang, Q., Wang, X., Liu, W. and Wang, X. Vision mamba: Efficient visual represen- tation learning with bidirectional state space model. CoRR, abs/2401.0944
2024. doi: 10.48550/ARXIV.
2401.09417. URL https:
48550/
arXiv. arXiv.2401 2401.09417.
16

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
Appendix
A. Reproducibility Statement
We make the code base used for our experiments publicly available and release the datasets we generated. Both are available at: https: htt///gtttb/////// commml-jjse jkku /LAM. We describe the environments we use for our experiments and provide dataset statistics in Appendix B. Furthermore, in Appendix C, provide implementation details for all methods and a list of hyperparameters used for our experiments. In Appendix D, we present additional figures that accompany our results in the main text (e.g.. all model sizes). Finally, in Appendices E and F, we provide further details on the conducted ablation studies and the embedding space analysis, respectively.
B. Environments & Datasets
B.1. General
We compile a large-scale dataset comprising 432 tasks from six domains, 3.4M trajectories, and 894M transitions in total (see Table 2). A key motivation behind dataset compilation is the scarcity of suitable datasets that span many simulated tasks. To address this and to enable robust comparison of different sequence model architectures, we aimed to assemble a collection of datasets that span as many tasks possible. In particular, we focused on trajectories in simulated environments rather than real-world trajectories (Embodiment Collaboration et al., 2024), to enable faster iteration cycles. To facilitate usability for future works, we consider standard benchmarks that are widely adopted by the community (e.g.. Atari, Meta-Wool World..
We release our data pipeline and generated dataset, and hope that they can serve as solid basis for future research on multi-task agents. To enable fast and targeted data-loading, every trajectory is stored in separate hdf5 file. We trade off some data-loading speed for disk space efficiency by compressing trajectories that contain image-based observations.
B.2. Atari
The Arcade Learning Environment (ALE) (Bellemare et al., 2013) is the standard benchmark for evaluating RL agents and consists of 57 Atari games. Input observations in Atari are RGB images, but as is standard practice, we gray-scale and crop frames (SS = 1 x 64 x 64). There are 18 discrete actions across all 57 Atari games ([.A) = 18), but individual games may use only a subset of these actions. Furthermore, we adopt the standard Atari recipe as used in prior works, including a frame skip of 4, maximum number of no-ops of 30, resetting on life loss, and reward clipping to [-1, 1] (Mnih et al., 2015; Hessel et al., 2017).
Tasks. Similar Lee et al. (2022), we assign 41 games the training set and additional tasks to the hold-out set. The 41 training tasks include:
amidar, assault, asterix, atlantis, bank-heist, battle-zone, beam-rider, boxing, breakout, carnival, centipede, chopper-commen command, prazy-climber demon-attack, mouble-dunke enduro, fishing-derby, freeway, frostbite, gopher, gravitar, hero, ice-hockey, jamesbond, kangaroo, krull, kung-fu-master name-this-game, phoenix, pooyan, obert, riverraid, road-runner, robotank, seaquest, time Lime--plot, zaxxon
video-pinball, wizard-of wor, yars rers-revenge,
The 5 hold-out tasks include: alien, pong, ms ms-pacman, www.s-ppaaaaeeeeeeeeeeeeeaeeeeae star-gunner
Dataset. For Atari, we leverage the DQN-Replay dataset released by Agarwal et al. (2020). The dataset contains the trajectories seen over the entire training of the DQN agent (50M frames). We extract a subset of the last 5M transitions for every task, amounting to 205M transitions total for the 41 training tasks. The number of episodes, the episode lengths, and total achieved rewards vary across tasks, as shown in Table 2.
B.3. Meta-World
The Meta- World benchmark (Yu et al., 2020a) consists of 50 manipulation tasks using a Sawyer robotic arm, ranging from opening or closing windows to pressing buttons. Meta- Meta-World is based on the MuloCo physics engine (Todorov et al., 2012a). Observations in Merl-World are 39-dimensional continuous vectors (S| x 64 x 39), and actions represented by 6
17

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
Table Atari Dataset Statistics.
Task
# of Trajectories
Mean Length
Mean Return
amidar
1813
2753
145
pooyan
2773
1800
176
frostbite
5218
766
18
video-pinbal1
1023
3902
266
wizard-of-wo-
3059
1314
chopper-command
5452
738
breakout
3780
1300
paoenix
3307
1509
asterix
5250
951
enduro
571
8720
636
kung-fu-master
1775
2812
131
bero
3022
1345
168
assault
3782
1170
77
demon-attack
1649
2431
116
qbert
3939
1138
155
jamesbend
2841
1758
11
bank-heist
4146
1204
62
ap-n-down
3246
1538
99
centipede
6879
582
81
boxing
4796
1041
63
battle-zone
1933
2134
15
name-this-game
988
5049
389
zaxxon
2561
1950
12
beam-rider
1232
3248
time-pilot
3886
1029
ice-hockey
1465
3407
riverraid
2645
1512
143
krull
3032
1319
528
gopher
1817
2338
185
freeway
2438
2048
seaquest
2807
1779
150
double-dunk
1774
2815
zoad-runner
3308
1217
135
atlantii
186
26349
1394
gravitar
6187
646
ya:s-revenge
4094
1036
96
crazy-climbe=
1105
3954
572
kangareo
1787
2792
50
fishing-dezth
2737
1825
0
caznival
21131
194
37
zobotank
747
6652
56
Average
3321
2734
153
continuous dimensions (IA) al. (2021) and Schmied al. (2024a), we limit the episode lengths 200 interactions.
18) in range [-1, 1]. All tasks share a common action and state space. Following Wolczyk
Tasks. We follow Yu al. (2020a) and split the 50 BeradWorld tasks into 45 training tasks (MT45) and evaluation tasks (MT5).
The 45 training tasks are:
reach, push, pick-place, door-open, drawer-open, drawer-close, buttonn-rress-toooovn pag-insert-siie, window-open, window-close, door-close, reach-wall, pick-place-wall. push-wall, button-press, peg-unplug-side, disassemble, hammer, plate-slide, ilee obt plate-alide-bbaok-sid,, handle
button-gress-toodvv--alll
button-press-vall,
plate-slide-back,
handeeeeppp handle-pull-sidee -
stick-push, stick-pull, basketball 1,soccer, faucet-open, faucet-close, coffee-push,
coffee-pull, coffee-button, sweep, sweep sweep-into, push-back, -eab-balkk.eeer--ul,,,dd dial-turr
assembly, shelf -hacce
The 5 evaluation tasks are: bin-picking, box-close, door-lock, door-unlock, hentr
18

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
ITWA
(b) Panda
(c) Jaco
(d) Gen3
Figure 8. Illustration of the four supported robot arms in Composuite (Mendez et al., 2022).
Dataset. For Meta- Meta-World, the datasets released by (Schmied et al., 2024a), which contain 2M transitions per task and consequently 90M transitions in total for the training set. All episodes last for 200 environment interaction steps, and consequently, there are 1OK episodes for every task. For detailed dataset statistics per task, we refer to their publication.
B.4. DMControl
The DMControl benchmark (Tassa al., 2018) consists of 30 different robotic tasks. Unlike Meta-World the benchmark contains robots with different morphologies instead of single common Sawyer arm. Due to the different robot morphologies, the state and action spaces vary across tasks (3 & |S] < 24, - |A| S 6), with all actions in the range [-1,
Tasks. We not use all tasks contained in the DMControl benchmark, but select 16 the 30 tasks that have been used in prior works (Hafner al., 2019; Schmied al., 2024a:b), which we refer as DMCIL and DMC5, respectively.
The 11 training tasks
finger-turn easy, fish-upright hopper-stand, point massmaasssssss wa walker-stand, walker-run, ball.in.cup-catch, pettpole-swinguus cheetah-run singer-sping reacher-easy
The 5 evaluation tasks are:
cartpole-balaned finger-turn hard,
reacher
walker-walk
Dataset. For DMControl, generate 10M transitions per task by training task-specific SAC (Haarnoja et al., 2018) agents, using the same setup Schmied et al. (2024a). Episodes in all DMControl tasks last for 1000 environment steps, and per time-step maximum reward of +1 can be achieved, which results in maximum reward of 1000 per episode. Consequently, our training set contains 10K episodes per task, amounting to 10K episodes and 110M transitions in total across all tasks. We list the dataset statistics for 11 tasks Table 3.
Table DMControl Data statistics.
Task
# of Trajectories
Mean Length
Mean Return
point.mass.easy
10K
1K
851
cheetah.
10K
1K
385
walker.run
10K
1K
230
ball.in.cup.catch
10K
1K
969
hopper.stand
10K
1K
460
walker.stand
10K
1K
939
finger.turn.easy
10K
1K
954
reacher.easy
10K
1K
938
cartpole.swingup
10K
1K
817
fish.upright
10K
1K
815
finger.spin
10K
1K
966
Average
19628
152
8.2
19

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
B.5. Composuite
The Composuite benchmark (Mendez al., 2022) roboties benchmark for grasping and object manipulation. The benchmark is implemented on top of robotsuite e (Zhu et al., 2020), which in turn leverages the MuJoCo simulator under the hood (Todorov et al., 2012b). Composuite contains mix of 4 simulated robot arms: IIWA, Jaco, Gen3, and Panda (see Figure 8). All arms share common state and action space containing 93 continuous state dimensions and 8 continuous action dimensions, respectively (IS) = 93, A 8).
Tasks. CompoSuite designed as a compositional multi-task benchmark for RL, in which a particular robot manipulates a particular object given an objective, while avoiding obstacles. Overall, there are robot arms, objects, obstacles, and task objectives. This results in 256 possible robot/cheec//jjecciieebbstt combinations. For experiments, we assign 240 tasks to the training and use the remaining 16 tasks hold-out set (Panda and object Wall) combinations. For list of all 256 tasks, we refer Mendez et al. (2022).
Dataset. For Composuite, we leverage the datasets released by Hussing et al. (2023). For every task, we select 2000 episodes, which last on average for 500 steps. This amounts to IM transitions per task, and 240M transitions across all 240 training tasks. For dataset statistics, we refer to Hussing al. (2023).
B.6. Mimicgen
Similar to Composuite, Mimicgen (Mandlekar et al., 2023) is based on robosuit te and the MuJoCo simulator. Mimicgen designed for automatically synthesizing large-scale datasets from only handful human demonstrations. Observations in Mimicgen can be represented as images (from multiple cameras) or oow-dimensional continuous states. For our experiments, opt for the low-dimensional state representation simplify learning. Therefore, observations and actions are represented by 37-dimensional and 7-dimensional continuous vectors, respectively ([S) 37, |A| 7). Similar to Composuite, Mimicgen supports 4 different robot arms: Panda, IIWA, Sawyer, and UR5e (see Figure 9).
(a) IIWA
(b) Panda
Sawyer
(d) URSe
Figure 9. Illustration of the four supported robot arms in Mimicgen (Mandlekar et al., 2023) solving the stack-three task.
Tasks. Mimicgen consists of 24 diverse tasks, including stacking blocks, reassembling objects, and even long-horizon tasks like coffee preparation. These 24 tasks can be performed with the four supported robot arms, amounting to 96 tasks in total.
Dataset. Mandlekar et al. (2023) released datasets for the 24 tasks using the default robot arm P Panda. To increase the dataset diversity, we additionally generated data for the remaining robot arms. However, not all data generation runs produce successful trajectories, and we discard the ones with too few successful trajectories. Our final dataset for Mimicgen contains 83 training and evaluation tasks. For each task, collect 1000 successful demonstrations (we do not include unsuccessful trajectories). Episode lengths vary across tasks, ranging from 260 to 850 environment steps.
B.7. Procgen
The Procgen benchmark consists of 16 pooeduralll-ggeeeraae video games (Cobbe et al., 2020a). Observations in Procgen RGB images of dimension 3 64 x 64. However, for training efficiency, we apply gray-scaling image observations ((S) 1 x 64 x 64). All 16 environments share common action space of 15 discrete actions (I.A) 16). Procgen is designed to test the generalization abilities of RI. agents. Consequently, procedural generation is employed to randomize background and colors, while retaining the game dynamics.
Tasks. Following prior works (Raparthy et al., 2023; Schmied et al., 2024b), we assign 12 and tasks to the training and hold-out sets, respectively. The 12 training tasks are:
20

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
bigfish, bossfight, caveflyer, chaser, coinrun, dodgeball, fruitbos heist, leaper, maze, miner, starrpspill
The 4 hold-out tasks are: el imber, ninja, plunder, jumper
We leverage the datasets released by (Schmied et al., 2024b), which contain 20M transitions per task. The datasets were generated by recording all transitions observed by training RL agents for 25M steps, followed by uniform subsampling to 20M transitions. Consequently, the dataset contains mixed quality trajectories ranging from random (beginning of training) expert (end of training). We list the dataset statisties for all 16 tasks Table 4.
Dataset,
Table Proegen Data statistics.
Task
of Trajectories
Mean Length
Mean Return
bigfish
82835
230
6,251
bossfight
112459
141
1.946
caveflyer
151694
105
7.745
chaser
93612
212
3,248
coinrun
261117
51
9.473
dodgeball
144364
137
2.884
fruitbot
73653
270
16.094
heist
101361
196
8.405
leaper
296084
67
4,446
maze
482245
41
9,432
miner
288818
68
11.8
starpilet
96468
206
17.3
Average
182059
144
8.3
C. Experimental & Implementation Details
C.1. Training & Evaluation
In our experiments, we compare two variants of xLSTM, Mamba and DT. For our main experiments in Section 4.2, we train all models for 200K updates, and evaluate after every 50K update steps. We report the mean and 95% confidence intervals over three seeds in our experiments, as suggested by Agarwal et al. (2021). For every evaluation task, we take the average of 3 evaluation seeds.
We train our agents with a batch size of 128 and gradient accumulation across the domains, such that every domain is represented with the same proportion. This is to compare Consequently, the effective batch size is 768. We use learning rate of 1e-1, 4000 linear warm-up steps followed by cosine decay to le-6, and train using the AdamW optimizer (Loshchilov & Hutter, 2018). In addition, we employ gradient clipping of 0.25, weight decay of 0.01 for all models. We do not employ Dropout, as is standard practice in DT's, as we found that negatively affects performance (see Section 4.3). We use separate reward scales of 200, 100, and 20 for Meta- Meta-World, DMControl, and Atari, respectively. Furthermore, for all domains, we set the target return to the maximum return achieved for a particular task in the training datasets. This is particularly useful for domains where the maximum returns differ heavily across tasks (e.g., Atari). We list hyperparameters in Table
We want highlight that opt to represent every domain with approximately equal proportion in every update step. This is, because we aim to study how the different backbones perform across domains, rather than optimizing performance on specific domains. However, to better understand the impact of the data ratios on multitask capabilities, we believe it would be interesting to study other data ratios in future work. Varying the data ratios would, for example, allow studying potential interferences between the 432 tasks.
C.2. Context Lengths
By default, we train all models with a context length C = 50 timesteps. For every timestep. there are three tokens (s/rt/r), and consequently, the effective context length is 150. We found that performance improves for longer context lengths (see Section 1), but limit our experiments C 50 reduce the computational cost.
21

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
Table 5. Hyperparameters for LRAM.
Parameter
Value
Gradient steps
200K
Evaluation frequency
50K
Evaluation episodes
Optimizer
AdamW
Batch size
128
Gradient accumulation
6
Lr schedule
Linear warm-up + Cosine
Warm-up steps
4000
Learning rate
le-4 - le-6
Weight decay
0.01
Gradient clipping
0.25
Dropout
0.2
Context len (timesteps)
50
Reward scale
per-domain per-task
Target return
C.3. Model Architectures
We train models across 4 model sizes: 16M, 48M, 110M, and 206M. We follow Lee al. (2022) in selecting the number of layers and hidden dimensions. For xLSTM and Mamba, we use twice the number of layers blocks to match the number of parameters of the Transformer (Beck et al., 2024; Gu et al., 2024) (see Table 6) For our XLSTM [7:1] variant, which contains sLSTM blocks, we strive to maintain the same ratio as proposed by Beck et al. (2024). Not all our model sizes are divisible by 8, and only the 16M and 110M models exhibit the exact 7:1 ratio of mLSTM to sLSTM blocks. For consistency, however, we maintain the same notation as (Beck et al., 2024). We place sLSTM blocks at positions [1]. [1, 31, [1, 3]. and [1, 5] for the 16M, 48M, 1 10M, 206M, respectively.
Across backbones, we use linear layers encode continuous states, reward retumssto-go, similar to Chen al. (2021). The maximal state dimension across continuous control environments is 204 in our experiments. To use shared linear embedding layer for continuous states, we pad states that have a lower number of dimensions to 204 dimensions using zeros. To encode image inputs on visual domains, we use the IMPALA-CNN proposed by Espeholt et al. (2018) and adopted by previous works on Proegen (Cobbe et al., 2020a) and Atari (Schmidt & Schmied, 2021: Schwarzer et al., 2023). Consequently, we do not make use of discretization of continuous states or patchification images. This design choice significantly reduces the sequence length to only three tokens per time-step (see Appendix C.2) and consequently results in faster inference.
For continuous actions, we make use of discretization and discretize of every action dimension into 256 uniformly-spaced bins, similar to Reed et al. (2022) and Brohan et al. (2023b). We experimented with lower/higher numbers of bins, but did not observe a benefit beyond 256 bins. Consequently, this resolution is sufficient for the environments we consider. We use a shared action head to predict the action bins of all continuous dimensions jointly. The maximum number of continuous action dimensions is our experiments, and consequently, the number of discrete action classes is 2048. In addition, there are 18 discrete actions originating from Atari and Procgen. Therefore, our action head learns to predict the correct action among the 2066 discrete classes. While different environments may have different action dimensions, the model predicts all action dimensions jointly. At inference time, the number of action dimensions of the current environment is known, and we extract the respective dimensions from the joint predictions. We opt for the shared action head representation, as this further speeds up inference and does not require autoregressive action prediction.
For the Transformer baseline, we use global positional embeddings similar to Chen et al. (2021). For the recurrent backbones, we do not make use of positional encodings.
22

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
Table Model Sizes.
Model
Layers
Hidden Dim
Heads
Parameters
Transformer
4
512
8
16M
Transformer
6
768
12
48M
Transformer
8
1024
16
110M
Transformer
10
1280
20
206M
Mamba
8
512
16M
Mamba
12
768
48M
Mamba
16
1024
110M
Mamba
20
1280
206M
XLSTM
8
512
4
16M
XLSTM
12
768
4
48M
xLSTM
16
1024
4
110M
xLSTM
20
1280
206M
C.4. Hardware & Training Times
We train all our models on server equipped with 4 A100 GPUs. We use distributed data parallel to distribute the workload, supported in PyTorch (Paszke et al., 2019). Training times range from 5 hours for the smallest DT model to 30 hours for the largest Mamba model. Throughout all our experiments, supported in Py Torch to speed up training time.
mixed precision training (Micikevicius et al., 2017) as
We evaluate our models after every 50K steps. However, periodically evaluating the trained agents on all 432 tasks sequentially time-consuming. Therefore, we perform parallel evaluation with 4 processes at time. For multi-GPU setups, we distribute the evaluation workload among the available GPUs. For example, with 4 available GPUs and evaluation processes per GPU, 16 environments are evaluated simultaneously. Consequently, the total evaluation time for all 432 tasks ranges from 18 minutes for the smallest DT model roughly 2 hours for the largest Mamba model.
23

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
D. Additional Results
D.I. Training Tasks
In Figures 10 and 11, we report the normalized scores obtained per domain and the average learning curves across tasks for all four model sizes.
xLSTM [7:1]
xLSTM [1:0]
Mamba
DT
Nomaiited Spore
0.8
0.4
0.2
mete world
dmcontrol
stan
proggen
mimicgen
composurte
(a) 16M
1.0
Normelized Scote
0.8
0.0
moco-word
proggen
minicgen
composute
(b) 48M
1.0
Hormalted Score
meto-word
dmccontrll
stari
mimicgen
composute
(c) 110M
1.2
0.0
0.4
0.0
melavereta
prorgen
mimiggen
componate
(d) 206M
Figure 10. Normalized scores per-domain all four model sizes: 16M, 48M, 110M, and 206M For Meta-World DMControl, Mimicgen, Composuite, and Proegen report data-normale normalzzmdd scores, for Atari report human-normalizz scores.
In Figure 12, report the training perplexity the 432 training tasks over 200K updates. Here, observe that the training perplexity behaves similarly to the validation perplexity. This is expected, as our models see most transitions only single time (see Table 2 for the number of repetitions per domain).
Furthermore, we report the scaling curves with an additional model size of 408M parameters Figure 13. Due to the high computational cost of the 408M models, we were currently only able to conduct a single run for this size. However, we aim to provide further empirical evidence for these model sizes in future work.
24

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
XLSTM [7:1]
XLSTM [1:0]
Mamba
DT
0.7
Normalzed Score
6.5
Normaized Score
0.4
0.3
0.
6.0
Stap
(a) 16M
(b) 48M
0.0
0.8
Normalized Score
0.6
Normalized Score
0.
0.4
0.2
02
0.0
(c) 110M
(d) 206M
Figure 11. Learning eurves for all four model sizes, 16M, 48M, 110M, and 206M, the training tasks.
D.2. Hold-out Tasks
In Figure 14, we show the zero-shot evaluation performance on the hold-out tasks 14. We want to highlight that the performance declines for all methods and model sizes compared performance on training tasks. This is because hold-out tasks exhibit severe shifts in state-spaces, action-spaces, and reward functions.
D.3. Fine-Tuning
In Figure 15, we present the fine-tuning evaluation performance on the held-out tasks. We compare xLSTMs trained from scratch against xLSTMs initialized with the pre-trained weights. We do observe consistent improvement of the pre-trained models over the models trained from scratch. While we train on a substantial number of environments, the total amount of data used is still only a fraction of that employed in training other large-scale models, such as LLMs. Consequently, we do not observe comparable few-shot generalization. However, we anticipate that few-shot generalization capabilities will emerge as we increase both data volume and model size.
D.4. In-context Learning
We assess the ICL. abilities of modern recurrent architectures on the Dark-Room environment considered in prior works on in-context RL (Laskin et al., 2022; Lee et al., 2023; Schmied et al., 2024b). In Room, the agent is located in a dark room. The task is to navigate to an invisible goal location in that dark room. The state is partially observable, as the agent only observes its own x-y position on the grid ([S) 2). The action space consists of 5 discrete actions: move up, move down, move left, move right, stay (I.Al 5). Upon reaching the goal location, the agent receives reward of +1 for every step in the episode it resides in the goal location. Consequently, the agent first has to explore the room to find the goal. Once the goal location is found (as indicated by the positive reward), the agent can exploit this knowledge. Given multi-episodic context, the agent should be able to exploit information contained in the previous trials (e.g., exploiting one path vs. avoiding another).
In our experiments, the Bork is 10 10 grid and episodes last for 100 steps, starting in the top left corner the
25

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
XLSTM [7:1]
XLSTM [1:0]
Mamba
DT
Training Perplexity
46
45
44
43
42
0.75
1.75
2,25
Number of Parameters
(a) Training Perplexity
Figure 12. Scaling comparison. We compare xLSTM, Mamba, DT in four model sizes: 16M, 48M, 110M, and 206M parameters. We show the training perplexity on the training dataset to evaluate the sequence prediction performance.
XLSTM [7:1]
XLSTM [1:0]
Mamba
DT
0.80
Validation Perplexity
0.75
Noraalized Score
0.70
a5
0.65
0.60
0.55
0.50
0.45
42
Number of Parameters
Number of Parameters
(a) Sequence prediction
(b) Environment interaction
Figure Scaling comparison with additional 408M parameter models. show (a) validation perplexity on the hold-out datasets, and (b) normalized scores obtained from evaluating in the training task environments, averaged over all 6 domains.
grid. We adopt the experiment setup Schmied al. (2024b) and leverage their datasets. We train 16M parameter agents datasets from 80 randomly selected goal locations in the grid. The datasets contain 100K transitions per task and are obtained by training task-specific PPO (Schulman et al., 2018 8) agents. Then, we evaluate the in-context abilities of our agents on 20 hold-out goal locations. During evaluation, the agent is given 40 episodes to interact with the environment, which we refer to as ICL-trials. Furthermore, we adopt the AD (Laskin et al., 2022) framework for training our agents with a multi- epitiie context. We use the same sequence representation as used in our main experiments, consisting of states, returns-to-go (target return set to during evaluation), and rewards. Note that this differs from sequence representation used by Laskin et al. (2022). We the context length for all agents to the equivalent of two episodes, which amounts to 200 timesteps in total.
In Figure 16, we report the ICL performance over the 40 ICI. trials on (a) 80 training locations and (b) 20 hold-out locations for the different backbones considered in this work. We observe that the recurrent backbones attain considerably higher scores than the Transformer backbone. Furthermore, we find that xLSTM [7:1] attains the highest overall scores, which we attribute to the state- state-trackinn tracking abilities (Merrill et al., 2024) sLSTM blocks. We aim to explore the ICL abilities of modern recurrent backbones more in future work.
26

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
XLSTM [7:1]
XLSTM [1:0]
Mamba
DT
Normalized Score
0.10
0.08
0.04
0.25 0.50 0.75
1.00
1.25
1.50
1.75 2.00 2.25
Number Parameters
Figure 14. Scaling comparison. Zero-shot performance hold-out tasks four model sizes, 16M, 48M, 110M, and 206M. Note that performance declines for all methods and model sizes compared to performance on training tasks. This is because hold-out tasks exhibit severe shifts in state-spaces, action-spaces, and reward functions.
XLSTM-Scratch [1:0]
XLSTM [1:0]
XLSTM [7:1]
0.8
Normalize maallized Score
0.0
procgen
mimicgen
composuite
Figure 15. Fine-tune performance hold-out tasks. We compare the performance of pretrained xLSTM against xLSTM trained from scratch, both with million parameters. select the top trajectories from our held-out tasks based on performance and use this subset fine-tune the models. We perform 25K update steps during fine-tuning and show the normalized scores, averaged across held-out tasks from each domain.
D.5. Inference Time Comparisons
We empirically examine the difference in inference speed between of our models. Similar to De et al. (2024), we report both latency and throughput. For real-time applications, latency is the more important dimension, and therefore, we focus our analysis on latency.
D.5.1. LATENCY
In Figures 17 and 18, we report the latencies for DT and xLSTM with the number of layer blocks as DT, and twice the number of layer blocks as DT, respectively. We conduct comparison for two different batch sizes and across varying sequence lengths.
D.5.2. THROUGHPUT
In Figures 19 and 20, we similarly report the attained throughput for DT and xLSTM with the same number of layer blocks as DT, and twice the number of layer blocks as DT, respectively. We conduct our comparison for two fixed context lengths and varying batch sizes.
27

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
(LSTM[7:1)
XLSTM [1:0]
Mamba
AD
80
80
70
70
Reward
Reward
40
30
30
20
20
10
15
20 Step
25
30
3b
40
0
5
10
20
25
30
35
40
Step
(a) 80 training tasks
(b) 20 hold-out tasks
Figure 16. In-context Learning on Dark-Room 10 x 10.
DT
xLSTM
0.0230
0.008
Locency tire s200)
0.007
ro0ne In
0.0020
9.044
0.0015
0.03
0.02
0.003
50
200
400
930
1600
3200 6400 12000 25600
50
700
400
890
1600
3700
6400 17800 75600
Langth (bimettpe))
Lenigh (timestapo)
(a)
(b)
B = 16
Figure Latency. report latency with (a) batch size (b) batch size DT xLSTM with 206M parameters. XLSTM, we use the same number of layer blocks DT and higher hidden dimension match parameters.
D.5.3. XLSTM: KERNEL COMPARISONS
We leverage custom kernels for xLSTM to conduct our inference-speed comparisons. In particular, we compare variants: recurrent-style inference with and without kernel acceleration, and chunkwise inference with and without kernel acceleration. In our experiments, every timestep contains 3 individual tokens. Consequently, regular recurrent-style inference requires iterating over the token sequence of length 3 in loop, given the hidden state of the previous timestep. This requires 3 forward passes. In contrast, the chunkwise implementation operates chunks of timesteps given hidden state. Consequently. this only requires single forward pass. In Figure 21, we illustrate the impact of kernel acceleration. We find that our chunkwise kernels result in considerably lower latencies. Interestingly, we find that for B = 1, our chunkwise implementation without kernel acceleration is faster than the recurrent-style inference with kernel acceleration. However, as the batch size increases, this trend reverses. This highlights the importance of kernel acceleration for efficient inference.
D.5.4. XLSTM: IMPACT HEAD DIMENSION
In experiments, found that choosing the appropriate head dimension is critical enable high throughput for x.STM. Therefore, we conduct an inference ablation with xL.TTM 206M in which we vary the number of heads between 4 and 32, while keeping the total hidden dimension constant, resulting in different head dimensions. We find that throughput increases considerably when increasing the number of heads (see Figure 22). For heads, and therefore the highest head dimension, the total throughput saturates at batch size 96. In contrast, when increasing the number of heads to 32 (i.c., decreasing the head dimension), the total throughput continues to increase. This is because higher head dimension incurs more FLOPS.
28

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
DT
XLSTM
0.006
(20g)
0.0025
0.20: 19899)
0.007
0.055
0,0230
0.005
2.0020
Lacency
0.03
0.0055
8.02
0.001
50
200
400
8:0
1600
3200
6400 12800 25600
200
400
800 1600
3200
6400
12800 25600
(a)
(b) B
Figure Latency. We report latency with batch size and (h) batch size of 16 for DT and xLSTM with 206M parameters. For XLSTM, we use twice the number of layer blocks and the same hidden dimension as the Transformer.
XLSTM
10000
Throug stocs 5201
10000
Through stace (0c)
8000
0000
8000
4000
4000
2000
15
1.2
15
32
56
122
Batchh size
Botch stze
(a) C = 800
(b) C' 1600
Figure 19. Throughput. We report throughput with (a) context size 800, and (b) context size of 1600 timesteps for DT and xLSTM with 206M parameters. For XLSTM, the same number of layer blocks DT and higher hidden dimension to match parameters.
E. Ablations
E.1. Removing action condition
E.I.I. DT ON META-WORLL
We found that removing actions from the context results in better performance across backbones. In Figure 23, report the learning curves over 200K updates for DT with varying context lengths on Meta- Meta-World, both with and without actions in the context. While context lengths beyond 1 hurt performance when training with actions, the reverse is true when training without actions. This is in contrast to recent works, which did not benefit from longer contexts (Octo Model Team et al., 2024). However, while removing actions improves performance on Meta-World, it does not affect performance on discrete control. On Meta-World, we observed that the models become overly confident (high action logits), which problematic if poor initial actions are produced. We assume this is because in robotics, actions change smoothly, and by observing previous actions, the agent leams shortcuts. A similar issue has been identified by Wen et al. (2020) and termed the copycat problem, because the agent is incentivized to copy previous actions. Our solution is to remove actions from the input sequence. This prevents the agent from learning shortcuts and alleviates the copycat problem.
1.2. DT ON ALL 432 TASKS.
To further investigate the effect of removing actions from the context, we repeat this ablation on the full 432 tasks and domains the 206M model scale. Figure 24, we report the learning curves for DT with varying sequence lengths
29

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
DT
XLSTM
12000
10000
Throughtutt (tdb) (to0l (880)
Trrosphout (tobal /o0)
0000
6000
4800
2000
16
126
15
32
96
128
(a) C 800
(b) C 1600
Figure 20. Throughput. We report throughput with (a) context size of 800, and (b) context size of 1600 timesteps for DT and xLSTM with 206M parameters. For XI.TTM, we use twice the number of layer blacks and the same hidden dimension as the Transformer.
recurrent w/o kernels
recurrent w/ kernels
chunkwise w/o kernels
chunkwise w/ kernels
0.005
0.0175
Latency (250)
Latancy tame sta8)
0.004
0.0125
0.003
0.0075
0.002
6.0090
0.0025
1500 3200 6600 12800 25600 Length (Uimes)eeps)
Length (limestops)
(a) batch size
(b) batch size
16
Figure 21. Impact of kernel acceleration. We report latency with (a) batch size of and (b) batch size of 32 for DT and XLSTM with 206M parameters, For XLSTM, use the same number of layer blocks as DT and higher hidden dimension to match parameters.
trained (a) with and (b) without actions in the agent's 's context. Similar to the single-domain study on Meta- Woold with smaller models, we find that providing a longer context does not improve performance, resulting in a normalized score of around 0.3 across domains. In contrast, without action in the context, we observe a consistent improvement in the evaluation performance as the sequence length increases. In fact, the normalized score increases from around 0.3 with C' 0.7 with C 50. For computational reasons, we only report one seed per sequence length in this experiment, but believe that the overall trends are clear.
To better understand on which domains the longer context benefits or hurts our agents, we also present the normalized score domain in Figure 25, Without actions in the context, we find that longer context consistently benefits the performance across domains. With actions in the context, we observe that on Meta-Worlds World and DMControl, the performance deteriorates for > 1. In contrast, on the discrete control domains Atari and Procgen, but also on the continuous control domain Composuite, performance tends to improve with C 1. This suggests that the copycat problem particularly present on Meta-World and DMControl. However, note that the final performances on Atari, Procgen, and Mimicgen are considerably worse when actions present in the context compared when they not.
To further investigate this, we compute the MSE between subsequent actions in the training dataset (similar to Wen et al. (2020)) for the continuous control domains and report them in Table 7. Indeed, we find that Meta- World and DMControl exhibit significantly lower MSEs between subsequent actions than Composuite. While Mimicgen also exhibits a low MSE between consecutive actions, all backbones perform poorly on this challenging benchmark. Consequently, we conclude that removing actions from the agent's context particularly effective for domains where actions change smoothly.
30

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
HB
416
H32
Throughou stops 500)
25000
20000
1.0000
Figure Throughput comparison for xLSTM 206M varying numbers heads but fixed total hidden size. By default, we used heads for our experiments. Increasing the number of heads results in higher throughput.
10
1400
1400
1200
1000
1000
Reward
800
Reward
800
600
600
400
400
200
0.00
0.25
0.30
0.75
1.00
1.25
1.30
1.75
2.00
0.00
0.25
0.30
0.75
1.00
1.25 Step
1.30
1.75 2.00
Step
w/ actions
(b) w/o actions
Figure 23. Ablation removing the action condition for varying context lengths C. Performance of DT (a) with, and (b) without action condition on Meta-Worts World. With action in the context, harms performance due to overconfidence in action predictions. Without actions the context, the performance of DT improves with increasing C.
This result highlights the fact that large action models strongly benefit from increased context length, even the simulated environments we consider in this work. Furthermore, we believe that this effect can be even bigger in complex real-world environments that require longer-term interactions.
E.1. XLSTM ON ALL 432 TASKS.
To validate that modern recurrent backbones also benefit from training with longer sequence lengths, we repeat the same ablation as presented in Appendix 1.1 using xLSTM [1:0]. We report the learning curves, validation perplexities, and evaluation performance across all 432 tasks for varying context lengths in Figure 26. Note that the validation perplexity curves in Figure 26a, start step 50K for readability. Again, observe considerable improvements in the validation perplexities and the normalized scores (0.4 for C' = to 0.8 for C = 50) as the context length increases.
In addition, we provide the normalized scores per domain for xLSTM with varying sequence lengths in Figure 27, Across domains, we observe increasing performance with increasing
E.2. Return-conditioning vs. Behavior Cloning
Across experiments presented in the main text, except for the ICL experiments, we utilized a sequence representation that includes return-to-go tokens (RTG) as commonly used in the DT literature (Chen et al., 2021; 1; Lee et al., 2022). At inference time, the RTG allows condition the model on high target return to produce high-quality actions. This particularly
31

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
2
10
25
50
0.7
0.3
0.6
Normalieed Score
Normalized Score
0.5
0.2
0.4
0.3
0.3
0.2
0.0
0.0
0.25
1.00
2.00
0.25
1.00
2.00
Step
Step
(a) w/ actions
(b) w/o actions
Figure 24. Ablation on removing the action condition for varying context lengths C. Performance of DT (a) with, and (b) without action condition on all 432 tasks. Without actions the context, the performance DT improves with increasing C.
Table 7. Average MSF (+ standard deviation) between subsequent actions in robotics datasets.
Meta-World
DMControl
Composuite
Mimicgen
Avg. MSE
0.08
0.2
2.1 10.3
0.01510.007
useful when the datasets contain a mixture of optimal and suboptimal trajectories. However, many recent works focus on behavior cloning without return conditioning (Brohan et al., 2023b;a; Octo Model Team et al., 2024).
To better understand whether our findings transfer to the behavior cloning setting, we conduct an ablation study in which we exclude the RTG tokens and the reward tokens from the sequence representation. This means that the sequence consists of state and reward tokens, state-tokens only. In Figures 28 and 28, report the (a) validation perplexities and (b) evaluation performance on the 432 task for the four considered backbones when removing RTG or RTG and reward, respectively. We retain the same training settings and datasets as reported in Appendix C (200K updates, evaluation after every 50K steps). We observe similar learning dynamics as for the 206M models that include RTGG/reward tokens in the sequence representation (see Figure 2 and Figure 11). Consequently, we conclude that the same performance trends hold for training the considered backbones with and without RTG/reward condition. Note that the final performances are lower compared to the models that include the RTG condition, and that can be conditioned on a high return inference time.
E.3. Effect of MLSTM-to-sLSTY ratio.
Throughout our experiments, we compare two xLSTM variants: xLSTM [7:1] and xLSTM [1:0]. The bracket notation was introduced by (Beck al., 2024) and denotes the ratio of mLSTM to sLSTM blocks. For example, xLSTM [7:1] contains I sLSTM block for every 7 mLSTM blocks. As described in Appendix C, we aim maintain the same ratio proposed by Beck et al. (2024). While mLSTM blocks are fully parallelizable, sLSTM blocks are not. However, sLSTM preserves the non-diagonalized recurrent matrix to enable state-rracking (Merrill et al., 2024). As such, sLSTM can be attractive for tasks that require state-t (see Figure 4 in Beck et al. (2024)).
We first conduct an ablation study on the effect of the mLSTM-to-sLST ratio on the evaluation performance across all 432 tasks. For this experiment, we use the 16M parameter model that contains 8 xLSTM blocks in total. Consequently, we compare the following ratios [1:0] (only mLSTM), [0:1] (only sLSTM), [1:1], [1:3], [7:1]. In addition, we investigate the placement of sLSTMs across all 8 blocks. To indicate the placement, we use @ followed by the layer index (starting 0). For example, [3:1] @ 1,3 indicates that the second and fourth layers are sLSTMs. In Figure 30, we report the validation perplexities and evaluation performance for different ratios and layer placements across the 432 tasks. For computational reasons, we conduct this experiment with only seed per ratio. We find that at the 16M parameter scale, xLSTM [1:0] on average outperforms the variants that leverage sLSTM blocks. This indicates that these domains do not strongly benefit from the state tracking abilities of SLSTM.
32

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
2
10
25
50
1.0
0.6
Normalzed
0.0
meta-word
dimcontrol
etari
procgen
minicgen
composuite
actions
1.0
Normalized Score
0.8
0.6
0.4
0.0
meta-world
dmcontral
atari
procgen
mimicgen
composuite
(b) w/o actions
Figure 25. Ablation on removing the action condition for varying context lengths C. We show the normalized score per domain for all context lengths (a) with and (b) without actions.
Next, conduct the same analysis on Dark-Room 10 10 ICL environment as used Appendix D.4. Unlike most the 432 tasks used in our main experiments, Dark-Room exhibits a partially observable observation space and sparse rewards. Consequently, Dark-Room more likely to require state tracking abilities. In fact, we already observed better performance for xLSTM [7:1] than for xLSTM [1:0] in Appendix 16. In Figure 31, report the ICL curves for the 80 train tasks and 20 hold-out tasks. We observe that XLSTMM variants that contain SL. blocks at lower-level positions, such as [7:1] @ and [3:1] @ 1,3 outperform xLSTM [1:0]. In contrast, xLSTM variants that contain sLSTM blocks at deeper-level positions, such as [0:1] and 3:1 @ 5,7, perform poorly. This is similar to findings by Beck et al. (2024) who also place sLSTM layers at lower-level positions.
We conclude that sLSTM layers can be important building blocks for tasks that require state-tracking, such as Most of the 432 tasks consider in the main experiments of this work contain fully observable observation spaces and may not require Kate-tracking However, believe that more complex tasks with longer horizons partial observability, as is common in real-worlde applications, could greatly benefit from the state-tracking abilities provided by sLSTM blocks. As such, equipping an agent with the ability to perform state-tracking by including sLSTM blocks may be a valuable option for practitioners. This is distinguishing factor of XLSTTM from Mamba, which does not exhibit state-tracking.
E.4. Effect of Dropout in DT
DTs use by default a Dropout (Srivastava et al., 2014) rate of 0.1. However, during our experiments, we found that Dropout has detrimental effects on the evaluation performance, particularly on continuous control domains like Composuite. In Figure 32, we show the validation perplexities and evaluation performance for a DT trained with and without Dropout. Consequently, we remove Dropout from our DT variant.
E.5. Effect of reducing number of layers xLSTM
In prior works, xLSTM and Mamba use twice the number of layers blocks as the Transformer baseline, while maintaining the same hidden dimension (Gu & Dao, 2023; Beck et al., 2024). For our inference-time comparisons, we therefore reduce
33

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
2
10
25
50
0.8
56
Normalized Score
Normalized Score
0.6
52
48
0.2
44
0.6
0.8
1.0
1.2
1.4
1.5
1.8
2.0
0.00
0.25
0.50
0.75
1.00
1.25
1.50
1.75
2.00
Step
Step
(a) Sequence Prediction Performance
(b) Evaluation Performance
Figure 26. Ablation on effect varying the context length for xLSTM. report (a) validation perplexity and (b) evaluation performance across the 432 training tasks for xLSTM [1:0]. Without actions in the context, the performance of DT improves with increasing C.
2
10
25
50
1.2
Nomallzzed Score
1.0
0.8
0.6
0.2
meta-world
dmcontrol
atari
procgen
mimicgen
composuiite
w/o actions
Figure 27. Ablation on the effect of varying the context length C for xLSTM. We show the normalized scores per domain for all context lengths.
the number of layer blocks in XLSTM by half. To ensure a fair comparison, we consequently adjust the hidden size of XLSTM to match the number parameters of the Transformer baseline. this section, we investigate the effect of these modifications of the xLSTM architecture on the model performance.
In Figure 33, report the validation perplexities and evaluation performance for the regular xL.STM with twice the number of layer blocks as DT, and an xLSTM with half the number of blocks. Reducing the number of layer blocks results in slight decrease in performance on both metrics. However, XLSTM still outperforms the Transformer baseline (see Figure 2).
F. Embedding Space Analysis
In Figure 5, we analyze the representations learned by our models using UMAP (Melnnes et al., 2018). Here, we explain the clustering procedure more detail. For every task, we sample 32 sub-trajectories containing 50 timesteps (150 tokens) and encode them using our sequence models. Then, extract the hidden states at the last layer of our model and aggregate them via mean pooling. We cluster all vectors using the default hyperparameters of UMAP into two-dimensional space. Finally, we color the resulting points by their domain.
The purpose of this analysis to examine how the models organize their representations of different environments. In general, tasks within the same domain tend to share similar input characteristics, such as visual inputs (e.g., image frames), possible actions to perform, and reward structures. Therefore, they are more likely to be "grouped" together in the embedding
34

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
XLSTM [7:1]
XLSTM [1:0]
Mamba
DT
0.6
Norrmllized Score
Nonmalized Score
0.5
45
0.3
0.2
44
0.1
0.6
0.8
1.0
1.2
1.4
1.5
1.8
2.0
0.00
0.25
0.50
0.75
1.00
1.25
1.50
1.75
2.00
Step
Step
(a) Sequence Prediction Performance
(b) Evaluation Performance
Figure 28, Ablation on of omitting RTG condition. We report the learning curves for (a) validation perplexity and (b)
evaluation performance across the 432 training tasks for 206M parameter models. We observe similar performance trends as when including the RTG in sequence.
XLSTM [7:1]
XLSTM [1:0]
Mamba
DT
0.7
Normalized Score
0.6
Normalized Score
0.5
0.4
0.3
0.2
0.1
0.0
-0.1
0.6
0.8
1.0
1.4
1.5
1.8
2.0
0.80
0.25
0.50
0.75
1.00
1.25 Step
1.50
1.75
2.00
Step
(a) Sequence Prediction Performance
(b) Evaluation Performance
Figure 29, Ablation the effect of omitting the RTG condition and the reward condition. We report the learning for (a) validation perplexity and (b) evaluation performance across the 432 training tasks for 206M parameter models. We observe similar performance trends as when including the RTG in the sequence.
space. For example, when embeddings Atari games are closer to each other than to Procgen games, it indicates that Atari games share more similar underlying dynamics or input structures compared to Procgen. We indeed find that tasks from the same domain cluster together. more refined and better-separated embedding space may result in better final performance, potentially because it facilitates task identification inference time. This may, however, be specific the mixture of training tasks at hand. Therefore, we believe that studying the learned embedding spaces of multi-task agents in wide range of environments is interesting for future work.
Analogous to Figure 5 for DT and xLSTM, we show the UMAP clustering for Mamba 16M in Figure 34. In comparison to DT, Mamba exhibits slightly stronger grouping of the embedding space.
G. Raw Scores
In this section, we report the raw scores for all 432 training tasks for the 206M parameter scale. See Tables 8, 9, 10, 11, 12 for Procgen, Atari, Meta- Meta-World, DMControl, and Mimicgen, respectively. The raw scores for Composuite are available in Tables 13, 14, 15, and 16.
35

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
@
1,3,5,7
@ 1,3 7:1 1
0:1
3:1 @
5,7
7:1 @
Normalized Score
19
Normalized Score
0.5
0.4
88
0.3
07
46
0.0
-0.1
0.6
0.8
1.0
1.2
1.4
1.6
1.8
2.0
0.00
0.25
0.50
0.75
1000
1.25
1.30
2,00
Step
Step
(a) Sequence Prediction Performance
(b) Evaluation Performance
Figure 30. Ablation on the effect of the MLSTTM-to-LLSTM ratio. We report the learning curves for (a) validation perplexity and (b) evaluation performance across the 432 training tasks for 206M parameter models with varying ratios.
1.3.
@ 5,7
0:1
3:1 @ 1,3
7:1
70
70
60
Reward
Reward
40
80
30
20
Step
Step
(a) 80 training tasks
(b) 20 hold-out tasks
Figure 31. In-context Learning on Dark-Room 10 x 10 for varying mLSTM-to-sLSTM ratios.
w/o dropout
w/ dropout
51
50
Validation Perplext plexity
0.70
Normalized Score
0.65
88
0.60
47
0.50
45
0.40
0.25
0.50 0.75
1.00
1.25
1.50
1.75
2.00
0.25
0.50
2.00 0.75
1.00
1.25
1.50
1.75
Number of Parameters
Number of Parameters
(a) Sequence Prediction Performance
(b) Evaluation Performance
Figure Ablation on the effect of dropout on DT performance. We show the (a) validation perplexity (b) evaluation performance on the training tasks. DT performance drops considerably if training with dropout.
36

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
half
regular
Validittion Perplexitty
46
0.75
Nornalized Score
45
0.85
64
0.60
0.50
0.25 0.30 0.75
1.00
1.25
1.50
1.75
2.00
0.25
0.50
0.75
1.00 1.25
1.50 1775 2,00
Number Parameters
Number of Parameters
(a) Sequence Prediction Performance
(h) Evaluation Performance
Figure 33, Ablation on the effect of reducing the number of layer blocks xLSTM. show the (a) validation perplexity and (b)
evaluation performance the training tasks for the layer regular and layer-matched xLSTM models. Reducing the number of layer blocks in xLSTM results in slight performance decrease.
atari
composuite
dmcontrol
mimicgen
mt50
procgen
UKAP festure 2
UH4P feature
UM6P feture
UMAP festure
UNAP
(a) DT
(b) Mamba
(c) xLSTM
Figure UMAP clustering of hidden states for 432 tasks produced by (a) DT, (b) Mamba, and (c) xLSTM with 16M parameters, colored by domain. We again depict the embedding spaces for DT and xLSTM from Figure 5 for better readability.
Table 8. Raw Scores for Procgen.
Task
DT
Mamba
xLSTM [1:0]
xLSTM [7:1]
bigfish
2.53
2.0
4.6
5.13
bossfight
6.73
4.1
9.27
2.0
caveflyer
6.67
6.3
6.67
4.87
chaser
3.41
3.91
4.92
4.2
coinrun
10.0
9.0
10.0
10.0
dodgeball
2.8
3.4
4.27
3.87
fruithet
13.33
19.8
19.73
19.27
heist
7.33
7.0
6.67
6.67
leaper
5.33
4.0
8.67
5.33
maze
8.67
10.0
7.33
7.33
miner
8.07
11.0
9.0
8.27
starpilot
24.93
10.1
21.8
28.2
Avg. Reward
8.32
7.55
8.73
8.76
37

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
Table
Raw Scores for Atari.
Task
DT
Mamba
xLSTM [1:0]
xLSTM [7:1]
Amidar
82.27
30.8
71.07
26.73
Assault
438.2
224.7
410.2
494.13
Asterix
573.33
540.0
763.33
583.33
Atlantis
42573.33
97240.0
83760.0
76973.33
BankHeist
2.67
9.0
0.0
8.67
BattleZone
2000.0
2400.0
2600.0
1733.33
BeamRider
126.13
61.6
176.0
243.47
Boxing
80.8
77.7
83.8
84.93
Breakout
68.13
136.6
92.93
93.73
Carnival
618.67
424.0
697.33
484.0
Centipede
1802.13
1238.2
2416.73
1806.6
ChopperCommand
813.33
800.0
813.33
766.67
CrazyClimber
96853.33
65960.0
106606.67
79873.33
DemonAttack
100.0
65.0
181.33
130.67
DoubleDunk
-2.53
-3.0
-2.93
-3.87
Enduro
34.53
65.5
98.73
48.53
FishingDerby
-72.47
-68.2
-72.07
-71.0
Freeway
29.0
29.8
30.0
28.6
Frostbite
774.67
1248.0
1162.67
1049.33
Gopher
314.67
34.0
132.0
12.0
Gravitar
116.67
175.0
176.67
136.67
Hero
14004.67
11381.0
14688.67
16522.0
IceHockey
.4.8
-6.3
-7.6
-5.93
Jamesbond
490.0
540.0
603.33
510.0
Kangaroo
1426.67
2880.0
2620.0
2653.33
Krull
8880.67
10090.0
8918.0
9569.33
KungFuMaster
8866.67
12700.0
8120.0
11233.33
NameThi sGame
7976.67
7967.0
7789.33
7232.0
Phoenix
592.0
1600.0
1807.33
1052.67
Pooyan
283.33
87.5
371.67
406.67
Obert
4306.67
1700.0
805.0
2613.33
Riverraid
2888.67
6923.0
6688.0
7446.67
RoadRunner
1320.0
350.0
1340.0
213.33
Robotank
18.67
13.2
23.07
25.13
Seaquest
182.67
396.0
448.0
209.33
TimePilot
2533.33
3520.0
3200.0
2966.67
UpNDown
10598.0
12043.0
15340.67
12815.33
VideoPinball
1669.07
0.0
220.4
140.6
WizardOfWor
113.33
160.0
160.0
206.67
YarsRevenge
14356.27
14499.0
16815.0
21403.67
7axxon
0.0
0.0
20.0
0.0
Avg. Reward
5556.81
6281.27
6705.61
6383.35
38

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
Table 10. Raw Scores for Meta-
Task
DT
Mamba
x [1:0]
x [7:1]
reach
1860.69 12.51
5.79
1859.17 12.62
1864.37 6.57
push
1588.19 207.0
1605.03 107.81
1493.31 238.01
1759.33
pick-place
137.85 99.18
161.74 153.95
389.81 37.36
296.21
dooz-open
1552.95 6.51
1562.39
1
1570.16 14.83
drawer-open
1735.13 21.76
1714.4
1740.48
1747.33
dramer-close
3.06
1858.05 0
2.38.
1859.33
batton-press-topdon
1322.3 3.12
1326.55 19.93
1
1322.83
peg-Lnsert-side
1557.59 98.52
1607.59
1640.43
1574.75 90.34
window-open
1594.16 34.13
1568.55 14.38
1576.22
1578.18 70.3
window-close
1474.26 + 16.88
1443.94 18.99
18.78
1452.21 26.56
dooz-close
1538.02 14.64
1544.31
1546.0
1541.64
reach-wall
1837.64
1845.12 3.06
1833.76
1771.17 94.47
pick-place-wall
219.67
2
206.88 184.28
385,57 151.52
pusa-wall
1689.67 12.74
1701.7
1599.63 189.06
1487.69 195.8
bot-on-sssss
1512.08 9.54
1488.1 38.83
1541.77
1527.3 10.16
bulLor-pres--lggdon--ell
1314.49 62.73
1295.2
1321.26 17.59
1328.74 24.16
button-press-well
1359,83 = 173.51
1547.14 13.84
1326.57 + 109.09
1267 8.78
pog-unplug-side
1415.68 162.54
1517.49 25.27
1393.98 173.0
1422.64 192.05
disassemble
44.54
1441.18 29.15
1220.27 441.51
1072.31 374.95
hammez
1446.68 169.03
1683.04
1669.54
1642.34 72.23
plále-slide
1673.66 1.72
1676.83
1682.41 5.02
1677 5.46
plate-slide-side
1719.4 + 7.85
1694.35 46.29
1686.38 61.27
1690.72 12.97
llate-slide-back
1790.96 6.39
1787.65 5.99
1797.78 1.17
1797.17 &: 0.43
plate-s1ide--backsiidd
1773.26 9.72
1763.24
1785.11 7.42
1788 61 + 6.67
handle-press
1734.75 220.82
1829.07 29.91
1881.23 + 15.62
1881.92 # 10.56
handle-pull
1590, 74 35.98
1627.4 34.18
1616.62 52.0
1627 6 21.86
handle-press-side
1852.25 + 7.0
1857.4 10.13
1847 6
1857 36 + 5.57
handle-pull-side
1651.05 3.48
1601.3 22.56
1655.75
1651.77 &: 7.53
stick-push
1595.45 6.88
1585.22
1595.35 3.29
1595.21 + 0.88
stick-pull
1377.41 108.31
1401.91 32.79
+
1442.68 43.23
basketball
1529.79 79 + 11.41
1528.22 18:23
1543.02
1542.8 + 17.81
soccer
649.69 + 160.32
929.06 64.35
792.21 + 139.63
732.44 290.49
faucet-open
1676.95 + 121.6
1703.83 41.97
1727.05 45.155
1744.83 = 15,93
faucet-close
1772.91 = 9.23
13
1778.25 3.96
1775.25 0.79
coffee-push
340.21 276.9
232.01 + 225.2
61.35 51.79
41.79 40.9
coffee-pull
1346.29 101 .93
1261,39 195.18
1409.68 34.66
1293.92 + 129.94
coffee-button
1595, 91..94422 + 16.57
+ 1592.77
1593.15 49.98
1562.92 36.79
sweep
1485,79 79 # 12.17
1452.38 13.74
1508.58 14.96
1471.73 = 29.08
sweep-into
1796.25 = 7.64
1472.64 455.9
1804.27 2.38
1786.27 = 14.64
pick-out-oI-hole
1437.38 181.15
1499.35 35.73
1529 8.09
1415.99 # 176.44
assembly
1229.39 + 16.96
1216.34 22.21
1236.68 21.77
1227.88 + 7.67
shelf-place
1446.07 + 30.41
1448.75 39.73
1485.4 + 12.31
1463 a 9.04
push-back
1226.32 172.59
1022.98 = 158.35
1011.25 396.65
1027.48 + 303.73
lever-pull
1604,74 + 3.32
1634.06 6.08
1639.31 10.11
1626.09 + 23.72
dial-Lurn
16 # 22.94
1667.37 + 41.45
35.16 1713.38
1686.59 55.09
Avg. Reward
1486.05
1486.18
1455.15
1464.16
39

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
Table 1I. Raw Scores DMControl.
Task
DT
Mamba
xLSTM [1:0]
xLSTM [7:1]
finger-turn-easy
121.27 104.6
396.4 + 122.47
449.8 186.65
640.13 82.48
fish-upright
181.14 70.82
154.59 + 34.64
277.23 # 105.37
241.73 # 257.01
hopper-stand
296.15 141,83
304.78 32.65
413.95 # 35.83
392.34 = 152.75
point.mass-easy
342.26 # 37.42
720.11 42.95
734.95 + 114.17
823.74 57.3
walker-stand
911.72 # 38.16
785.21 23.53
947.31 22.13
864.14 + 181.56
walker-run
0.8. 155.91
274.83
201.344
145.01 31.71
ball.in.cu--catch
970.9
977.33 0.5
975.93 + 0.42
cartpole-swingup
688.5 + 42.6
762.4 63.93
800.14 13.64
591.08 86.49
chestah-run
81.21 # 96.85
482.39 17.23
358.52 127.92
389.04
finger-spin
209.27 # 20.57
430.8 # 61.66
673.47 # 94.37
626.93 + 29.21
reacher-easy
45.4 5.21
180.7 + 133.64
78.73 + 20.59
58.0 + 13.91
Avg. Reward
364.52
496.65
505.06
522.55
40

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
Table 12. Raw Scores for Mimicgen.
Tisk
DT
Mamba
(I-TM
st.TTM [7:1]
Fands.Cotreee repezattoootooooomm
000000
013:0.12
Panda,Doff reee repretttet I tongh1
0.00000
0.0000
Fanda_too foffuee
04
0.:0000
02202
0071
Vanda,So S1
02:
0.
112 11200.2
a
Fanda Coffee
0.12
00:000
0.:00
Fanda KssnarCiaanup..
1,0
0.5
0.14
0.47 +
0.31
Fands_Fit:chenn20
0.23
0.0000
1.0
0.0
Fanda_Kitchen.22
0.0000
0.0:000
0,0
0.0
Fandb->h:pCJeenup..0
0.340.12
0.220.2
0.12
0.0000
0.2
0.7 0.12
0.:000
0.0
0.11
Savyer_Fickkleew.po
0.1000
0.:000
1.0
0.0
00:00
Fanda.
Panda.Square.D2
0.1
0.0000
0.07 + 0.12
0.7 0.12
Fanda_Stackthrea.DD
0.0
0.0000
0,17 = 0.12
0.0
Fandb_Stackthres.b)
0.0000
0.00000
0.07
0.0
Fanda_Stack_D0
04740112
Pands_5tack_J1
04
00000
0.27 0.12
Pamri_.hhrwading.00
0.2
022000
0.27 0.12
0.2
Vanda, thc.aaiingg
0.35
0.0
0,17 = 0.12
0
threePie
00:000
0.0
Fanda
0.0000
0.0
I1MM.Cottee.D0
00:000
0.0000
0.0
0.0
Sawver.2sEie..00
0.3
0.00000
0.12 0.112
0.2022
URSc.Coffes.D
0.3
0.2000
0.331
0.4202
2INA.Cotteo_11
Sawger_Poffee.31
0.1
0.000
0.07 0.13
0.0
URSe_coffeejj!
013+0012
00=00
0.2
033
L.NA,Coffwe.j
00
0.0
1.0
0.0
USSe Coffee
0.1:114
0.3
0
2IMA
0.0:00
0.0
0.7
09
0.33 =
0.23
USSe HannerCleanuu DO
0.0
091
1.0
2
21M C.W...aaneerCicaa
0.:00
0.0:000
0.0
0.22
0.7
04
0.8 0.2
0.
TTMA_Kis
00+00
00+00
0.0
0.0
UKS sitcclennoo
0.1
08
1.0
0.0
UnSe Kitshen.Di
0.0
0
TTKA Nugh! eamugi
00=00
0.0
1 www.Rugeleenngge
0.
0.:00
0.0
0.0
USSe Ruccleanco, 21
0.7
0.00000
0.13 0
0.13
IIMA Muthoserbly DO
0.:00
0.0000
0.0
0.0
Bavyer.Rurhaseeeb.y.
URSe Syseeettesssebly...
0.
0.:00
0.000.0
0.12
(TMA,PickPless Sawger_PickPlaee.p pD0
00
00+00
0.0
0.0
00
0.00000
1.0
00
UnSe MnseePickPlace.gg
0.10000
0.0
0.100
TTKA Square
0.=00
022022
04
0.33
= 0.12
0.33 0.23
USSe Seuare. PD
0.33
0.2
0.27 = 0.12
0.:00
0.000.0
0.0
0.00000
Saxyer.Square..
URbee URbe.,guareee.
00
0.:00
0.0
0.0000
TTWA, att.tthhhheeee
00
00+00
0.0
0.0
Sawywe www.hachheeee.e......m .00
00
0.
0.0
00
TTAA, StMkStaccchheee.1
00+00
Gawyer Sitessttteeee Stecttheeettt
0.0000
00:00
0.0
0.7 0.12
UsSi stackihree..r
0.1
0.
0.0
Stack
00:00
00:00
0.0
0.00000
Saxycr.Staec.DD
0.2000
0.2
URbe_Stace_Diu
04402
0.3
0.11
0.12
00+60
00+00
0.0
0.0
Sowyer.Staex.D1
02402
0.0000
0.2
02720.12
URbbe_ttacs_11
0.11
(TNA,hhreedi
00-00
0.0
0.1
0.00000
0.17
= 0.12
0.3
uxte whw.aalaagddooo p0
0.31
011
11.4
042
IIRA Throzding
0.0:0.0
0.0
0.0000
Bawycr.Throading..t
(Rbe.Threeddnng..1
0.7
0.:00
0.0
0.0
0.0000
00+00
0.0
0.0
0.0000
0.0000
0.0
0.00000
TIA TWAA,hreedt lecekseerblyy...
00+00
Gawyer 2heeett
0.0
0.0000
0.0
0.0
ustil shessariccasseebbll pl
0.0
0.1:000
0.0
0.1
PhssePlccoassebbll
0.:0000
0.0
0.0000
0.0000
URbe.ThreoP wttrooiecsshserbbly.e
0.0000
0.0000
0.0
0.0

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
Table 13. Raw Scores for Composuite, Partl.
Task
DT
Mamba
xISTM [1:0]
XLSTM [7:1]
BIsA.Box.onee_Pin
Pos..Mone.PiccPecee PickPlace
402.74
414.73 10.49
424.35 12,95
421.33
11We_Box.Hone.ruah
388.61 35.63
2.03
424.4
4
WWW__Pox_None_hhoo
370.3 à 80.53
417.61 1.44
417.78 0.96
416.41 1.87
IIMA Box None Trashean
329.27 + 113.43
424.39 1.04
429.54
426.07 + 3.98
367.68 + 81.93
421.6=4.11
4.3.0
429.29 1.97
299.69 77.03
337,81
88.42
4
318.19 9 50.76
IIMA wtw.Goaaaaalllheeee....s
48.292
9.82
40..1
402.31 3.08
1 1
376.45 83.64
4
429.15
425.64 3.88
IIMA Bow.BBxx.Col I
389.21 2
417.89
413.82
414.08 3.83
106.51
403.59
373.61 40.95
397.45
11w.BBox_.cjjocttoorr..ses
129.42 67.73
CbweetDooz Trasheer
325.45 72.77
391.58 16.76
393.52
2.2.76
420.61 2.99
421.61 1.06
420.21 3:5
1.2.76
410.19 1.62
411.5 3.13
1IMA Box CbjectNel1 Stelf
400.86
408.22
3.99
396.64 10.55
1
414.43 +
41.471
417.11 1.69
414.46 0.8
1IMA_ 11XA_Dumbbe: Numbbe::l.NoneePicccii
286.95
2.993
421.32 2.32222000
421.94 1 1.48
11MA www.bebl.bbeeeeeeeee..
90.96
4.0023 6.13
393.34 36.66
310.45 73.45
344.81553.
380.51 5.34
350.8 + 52.16
1IXA Dumbbee11
None
386.09 40.69
0.77
414.03
3.336
1.16
415.64
410.7
413.51
316.49
367.45
336.67
371.92
Dumbbell Gos_Wa111laaaaaaaa Shelf
395.63
3
3.6.75
372.77 4.25
379.45 58.51
55.11
412.22
406.03 5.03
1IWA_ wtt.:a/bll...ggpcccooooooroooo..
358.13 26.76
364.62 40.18
393.83 2.05
347.28 39.81
1IMA Dumbbeel1.09/yec:D0oor0.00
400.9
383.81 8.44
0.7
364.06 35.78
11 hto____mmbee111100000000000....
369.75 14.29
325.7 * 30.94
350.7 21.76
335.84 40.36
11XA_
393.05 + 3.92
358.77 36.88
397.23 1.73
389.54 9.14
403.51 1 12.08
404.28 1.23
401.15 10.64
330.77 30.29
296.98 68.18
334.1 : 22.28
307.4 33.85
11 11AA Dumbbel1 0htp11100@ee000aaa Shelf
S
353.9 29.5
374.39 6.58
358.29 33.75
358.76 76 18.87
394.48 4.39
361.99 39.17
398.06 0.59
383.43 4
0.59
424.4
424.59
425
TTWA Pl.ate.Noooo.Puhh
424.25
419.86
418.42
1IVB.PlateeNone.Bteelt
408.07 0.95
6.7402
396.55 10.03
394.93 10.81
1IX..Flaare..onneerrahhheee
419,62 1.81
420.24
420.37
419.42 2.61
424.69 2.67
423.93 2: 1.77
421.83
420.13 8.21
3.55
397.97 13.41
390.46 14.79 14.79
388.89 3.01
11
404.92
396.09 + 4.6
393.01 * 5.77
401.81 + 8.93
1
420.68 2.82
420.29 1.48
421.31 + 1.93
408.48
403.23
397.51 1.65
401.53 1.76
4.44
395.97 16.84
389.33 7.78
385.77 + 1.21
IIıa Plate PihaaPPaateeObhpcctoooottttill
3
373.43
369.41
374.16 13.75
400.27
4.0.74
399.28
400.23
3.1.3
416.76
409.31
Plate
3.994
6.53
40..51
405.27 1.34
393.23
376.64 12.49
386.41 8.65
382.81 6.78
410.85 1.07
408.87 3.99
408.98 0.82
409.35 2.6
94..13
6.93
428.62
426.38 3.26
186.22
422,49 8.01
427.73
426.12 2,3
416.65 6.66
419.89 11.03
418.34
415.11 0.89
424.38 2.77
421.62
426.9
425.99 99 + 1.81
430.17 3.37
427.76 0.48
427.91 0.76
426.47 4: 1.62
IIMA How.oHoox
3.9.3
373.0
390.09 9.46
394.35 14.43
1IN.HHollokbbv..ooavvlalllhhhhhh
2.4.55
379.05 64.32
423.51
419.69
0.773
425.27
424.8
3.33
276.87 109.64
369.45
301.41 112.33
|
1 |
326.56 109.6
352.22 53.97
390.78 6.35
324,09 55.59
IIMA
339.03 43.755
370.75 + 8.36
362.72 30.31
353.98 38.19
8.7
370.39 35.98
387.21 14.61
387,9
21.95
364.95 27.07
355.61 76.66
356.01 8.3
369.47 24.62
IIXA [ot1loooottooCCeeeecctel1111111 Push
4.000
414.47 8.08
5.5
408.53 8.05
1.N.HHolloooboox.0eeeetttt......
2.
400.31 1.28
403.69 2.06
401.27 1.97
IIWa_Hol
415.82
416.68 0.14
392.79 44.13
417.34 0.77

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
Table 14. Raw Scores for Composuite, Part 2.
Task
Mamba
xLSTM
XLSTM (7:1]
Anco_AxxNoons Wowoo_icckPlsse PickPlace
401.38 3.88
40041
5.35
396.54
daco Box None
399.84
397.79
392.77
397.31
Jacu Bex Shelf
383.53
384.65
385.85
386.34
Jaco_Box Too..hhhe.r
Trashcan
374.88 43.66
398.46 2.69
397.66 4,99
398.21
Jaco_Rox_Gov [
a
4
392.3 a
389.93
3.83
Jaco Rox Goa 1al11 Push
317.78 67.67
343.43 7.49
20.65 351.67
336.02 8.59
Jacs Bex Sheil
374.62 20.35
387,0 1.42
387.73 2.11
384,74 1.19
374.07 + 30.72
393.81 0.68
395,49 1.23
392.53 3,46
396.05 1.12
391.81 4.67
388.37 1.26
383.39 9.07
Jaco Rox Objectiloor Push
364.64 + 38.39
383.07 * 5.73
366.91
33.04
387,51 2.93
Jaco_Box.0b oec._boo.00jeeectooo..tttet.
Jaco Bex Bttp//cxx.bbbbbe.. ObjeetDoor__Beeff
373.8 + 2.81
379.75 4 1.45
375.38 6.27
6.27
376.86 1.37
3 1.28
353.97 52.06
389.38 2.0
389.81 2.89
Jaco baco_sox.0bjecckkelllPcccc.....e
394.31 2.66
385.33
388.54 7.62
387.82 2.26
Ja
387.4
384.75 4.29
383.61 + 7.58
383.32 7.73
Jaco Bex www.eoebeeeeeeleeeeeee.e
3
361.28
367.38 2.04
369.22 2.79
aaco.Box..bbjetttalllrtaaat
6.8573
1.13
385,34 0.74
380.01 5.08
319.87 1.83
334.2
376.46 9.19
334.95 68.5
5..41
369.49 4.36
300.81 61.26
15.44
361.77 6:21
362.88 8.22
JacoJUmmbell...nnee.raaaa..
369.52
13.38
387,28 1.88
377.27 9.7
40..12
306.26 32.85
349.04 18.3
348.42 37.3
107.91 29.9
136.11 9.04
245.71 30.15
188.19 58.09
Jaco
Dww.beee1111.GG.11.111111 Shelr
300.97 114.65
368.99 0.5
363.58 9.74
346.57 27.41
32181 : 87.58
317.94 23.15
376.09 2.22
378.49 4.52
382.35 1.62
5.17
349.1 32.92
7.66
382,32 1.08
353.42 2.
353.85 a 6.83
338.66 + 19.03
Jaco BacooDuumbee1110aae1111 Shelr Shelf
312.14
3 30,97 47..32
331.5 + 37.18
371.06 8.48
375.34 4.07
373.78 4 6.05
370.06 8.94
279.55 111.58
314.05
360.29 15.75
360.38 + 12.02
Jaco_Dumbbel henaJJumbeell11cd:ooco///111...1
381.11 3.7
351,38 1.82
349.16
352.64 11.94
Jaco Dumbbell 020 Waaeeetaaeeeaeaaeaaaaaaa_ Sheir
354.95 1.59
316.33
342.43 7.94
332.97 15.33
367.01 8.38
354.32 22.23
365.47 7.45
363.25 3.18
Jaco
0.77
389.99
384.38 =
380.69
195.18
9.12
Jaco Plate None Shal
381
356.49
380.99
Vaco.Plate.KoneeTraaii
0.76
390.62 0.57
391.2
390.3 1.83
379.45 24.14
378.13 6.34
377.33 11.32
376. 4.31
Jaca laco_Platte_Gaalk::11..
293.6 4: 38.38
319.4 4: 24.13
320.49 24.25
320.5 31.85
Jaco Plate GoalWa11 Shelf
358.04 22.32
369.8 15.11
367.73
12.97
362.35 3.32
Naco_Plate PaccPPaate..aaNe111111
383.53 1.33
387.55 1.56
389.51 + 2.03
388.57 1.98
390.4 1.3
381.92 15.09
376.2
380.34 9.73
Maca_Platee_0bjjcctooo....t
372.01 +: 4.07
366.41 # 16.51
359.43 #: 10.46
355.71 3.99
Jaco Plate ObjectDoor Shelf
366.15 6.61
357.96 8.35
368.82 * 4.35
362.39 7.11
Macs_Plate Jaco.Plate_Qbbectbuooo [llatee@@beecDDDoeeeaaaaate Trashean
382.66 0.58
3.3.
384.0
383.57 1.1
390.73 1.55
378.98 6.95
376.76 8.54
373.98 5.41
378.3 4.449
372.47=10.13
364.42
360.69 3.82
Jaco Plate ObjectWal1 Shelf Shelf
364.2
364.64 3.00
368.33 1.95
360.73 6.42
Jaco Plate PacooPPaaeee0beeec11111111112321
Trashhnn
3.761
375.68 1.54
2.76
373.86 4.91
4.0023
386.75=22..35
396.5
398.48 3.76
392.65
4.1.5
7.7
396.63 0.38
Jasa_Hol1
377.5
382.06
4.82
394.85 1.28
39228
393.68
392.87 1.71
395.2 1.44
378.92 9.441
379.34 7.17
349.5 34.56
337.43
34 11.76
340.9
Mac__Holllccoooxx.eee11111111111
357.89 19.58
349.29 4 10.1
344.53 & 6.27
333.97 A 12.22
Jace
385.01 1.04
3.7
386.58 2 0.37
384,52 0.05
335.16 76.71
387.66 8.98
375.68 +: 4.01
344.62 44.5
41..64
386.82 11.07
38.24
385.73 7.74
371.32 0.65
362.29 13.12
366.7244.12
360.22 + 15.51
Jaco Hac.:ooooooooooooooeoooooooooaaa
Trashcan
358.07 46.79
385.01 1.12
383.6
385. 2 0.42
393.5 2.63
377.85
378.61 8816
375.96 16.9625.555
114444477 391.74
382.69 12.26
387.67 99552
379.01 6.44
Baca_Hollawhox.ccoo htt.:ollloooooocccccooe1..11....
371.33
367.26 11.73
365.73 7.59
356.39 4 16.14
Jaco Hollowbox Cb Trashcan
382.6 1.63
385.72
382.62 1.19
382.01 4.22

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
Table Raw Scores for Composuite, Part
Task
DT
Mamba
XLSTM
xLSTM [7:1]
Kinova3_Bo www..ooo......eee.......
432,49 3.69
432.11 7.68
432.28 3.45
431.06 2.67
Kinova3 Rox Nono. Push
398.81 44.71
416.96 17.33
428.52 + 1.83
416.41 18.69
Ki Noooa3_Rox.None_Shee
411.22 3.9
+13.65
415.58 4.21
411.67 3.98
378.21 81.97
426.67
431.01 0.89
427.82 1.12
Kinova3 Bcx. www.:GollWa111221 PickPlace
347.29 145.33
430.92
431.3
408.26 2 40.64
GoalWall
325.78
390.05
382.78
388.29 6.07
Manova33Boxx.GoalWalll.....
357.79 96.22
395.77 28.11
418.95
417.37 1.02
373.8 = 80.27
424.09
428.12
427.05 0.87
Kinova3 Bex ObjectDeez PickPlace
1.7
4.443
424.25 + 2.86
424.5 2.45
Kinova3_Bcx.com
395.44 30.77
414.0 5.47
406.02 0.61
410.58 8.15
37.98
2.
408.55
381.75
392.17 40.87
415.87 2:48
419.24
416.46
405.45 21.25
387.27 50.08
425.83 2.68
423,06 3.66
Ki [
419.98 a: 2.8
414.6 1.04
412.82 4: 1.07
415.16 7.28
K [oooa33_cxx((0oocctt1111
399.47 + 4.56
399.51 1.29
402.37 2.66
402.42 1.48
416.15
412.41
31.99
36.15
380.36 55.46
418.88
419.3
416.89 2.86
K kinova3 Dumbbel1 Puzh
394,84 25.64
396.29 13.63
367,03 53.29
390.74 22.17
290.98 + 123.89
394.73 4.82
386.09 19.99
397,38 2.93
358 26 43.32
377.36 53.06
413.01 6.02
414.39 1.97
Kinova3 Dumbbell 0oa.hal1oaaaa1111111111111111111 20ckaaae1lcckPeess 2ickPlace
408.52 = 19.13
392.63 23.38
404.51 4.31
412.68 11.05
294.63
321.72
310.79 67.84
384.01
383.06 15.17
395.02 0.83
377.15 28.52
377.28 51.33
370.59 31.83
413.63 2.06
378.76 27.34
Kinova3 Dwmbbel1
(btabbeel.d@jjetDoerre.. pic.eeaaeeopccceeeeeeeeeeee PickPlace
415.58 5.38
404.89 11.83
405.77 7.4
410.95 8.75
359.17 15.53
265.44 62.94
367.39 23.91
311.57 45.56
363.99
409.92
407.09
0.7.
407.57 2.85
Kinova
404.63 1 16.95
409.29 4.6
406.14 2.11
411.69 6.71
Ki Tooova3__umbbbl11..ggpssstt
311.79 94.94
285.81 62.32
342.04 22,98
244.56 16.32
K1 Srwaa33uumbell__0bjeece 1_Shel
378.68 + 3.03
378.63 0.91
376.92 0.76
361.79 25.06
Kinova3 wtt.s__0mmbelll..00eeeell...
400.98 4.19
401,96
395.81 3.51
Linova3.Plate.Noo.
424.09
4.2.36
424.82
425.02 2.92
Kinova3 Plata
412.25 19.8
422.75
417.63 6.13
416.41 4.33
Minova3.Plate.Monee.hhe
409.96 + 0.2
409.11
410.28 0.65
409.52 1.61
422.54 2.13
1.15
421.73 1.36
422.97 0.74
Kinova3 Plate GoalWall
PickPlace
427.74 0.81
4.1.23
416.44
416.35 15.86
40146
385.01 15.39
371.6
386.87 12.31
<inovaa....tte.oo..Wlll...e
410.49
0.11
409.63 0.665
407.67 3.33
421.05 0.88
4.44
422.63 0.811
423.21 = 1.16
1inova3_Plate
423.26 a 0.3
407.55
406.43 2.07
414.11 7.32
258.58 18.57
278.08 34.02
300.72 90.5
257.79 48.13
Ki 1inoca3377sac220000tooorr11h1111
415.34 + 1.08
0.35
416.09 +.31
414.34 1.85
420.16 2.07
A13.68
408.0
411.83 4.11
Ki [mnova3__laato__@aasssal111.....
400.11 16.39
4
406.48 5.73
403.65 6.23
391 (09 3.65
391.99 6.62
386.25 16.53
391.7 + 5.14
413.36 16::::110
413.44
413.82 4
415.
Kinova3 Kinovss3BeellooooxxoeeeeeFF2
424.86
433.78
430.43
Kinova3 lowloa Fush
361.99 40.33
369.17 +
396.28 28.04
380.94 28.74
417.73 + 13.43
417.46
423.26
424.02 2.62
Kinoves_.cc1oowboooooeeeee
424.65 + 1.15
409.34 12.4
425.0
416.0 15.33
Kinova3 Hcllowbox GoalWel1 PickPlace
386.68 49.29
425.24
421.85 8.69
420.32
Kinova3 MinovasBBclloooooccceeeeel......
403.57
383.09
384.13
385.7
39.501
423.93
417.05 13.43
406.37 27.44
3,661
405.09 22.54
389.36 36 32.05
k
344.01 63.38
364.3 13.82
387.53 20.66
324.36 55.48
390.98 46.38
416.05 8.96
405.41 5.34
606.764. 16.92
12.39
390.42
357.94
405.87 4.17
411.24
414.22
408.73
424.57 0.92
408.98 6.4
417.83 5.67
419.63 9,2
K
249.37 176.18
319.13 111.09
324 394.39444 76.09
335.61 74.98
394.7 9.3
328.52 61.08
357. 89 37.75
362. 16 40.05
Kinova htt.::ooo11ooooooo0booee.00
354.65 48.89
353.43 78.59
407.99 1.96
408.29 4.94

---

A Large Recurrent Action Model: xLSTM Enables Fast Inference for Robotics Tasks
Table 16. Raw Scores for Composuite, Part 4.
Task
Dy
Mamba
xLSTM [1:0]
x1STM [7:1]
409.21 5.27
408 66 7.81
=09.83
405.46 3.84
Panda. Box None Push
402.52 + 2.55
373.74
49.95
400.35 2.33
399.37
Panda. Box Mone_Shell
383.69 + 4.34
381.42 3.66
3.666
383.55 5.74
386.01 + 1.29
Landa.Box.ooee.raashee.
5.64
2.77
407.95 1.92
406. 17
Panda. Landa.Box.Coo.
389.57
Panda_Rox_ccoa1 all1.eeess
272.61 79.58
2
263.72 45.71
281.71 31.21
Bentk.Box.Gos11:111hheee1
384.43 1: 1.66
389.06 3.69
388.59
383.94 2.0
Panda_Box__cos11ka111Traaaaa
400.68 +: 4.51
400.18 6.03
403.24 5.665
392.28 a: 16.82
365.3 + 5.97
359.63 0.79
359.27 10.88
Panda, Box Pax Sox_00bjectDDoo:Puaat
363.07 3.13
352.85 13.71
340.37
340.5 4.97
Banda.Box.0bbeectcoo...553
346.29
$45.84490
341 11,05
Box Boxx.00eeettDeee.Traasccco
361.19
3.2277
356.66
337,69
342.62
310.15 24.64
2.7
342.12 31,28
Sanda_Dimbbell.Moneee.
2
341.64 42.57
359.06 42.88
263.35 154.81
264.01 = 101.29
362.15 0.87
319.71 +: 33.9
297.54 67.67
Panda_humbbe _Mona_Trasccnn
+
329.06 * 43.08
373.77 * 16.73
327 93 68.
Panda, Dumbbell 00mbbeee1..Gooaaa11PPP....
310.61 42.65
268.34 147.91
329.02 62.28
360.39
43.29
282.01
270.81
GoaiNall
319.5
347.34 20.01
2.6
318.6
Panda. Dumbbell Www.aaaalaaaaaaaaaaaaaaaea
5.27
360.98 9.73
379.05 2.52
337.19 40.73
Panda_Dumbbeel. wtt.mm/el...c.occttoooooooo.....
344.54 5.77
346.57 0.33
3.0
338.46 10.42
Panda. Banda.Dumbbbel..0
289.31 2 11.14
308.25 9.24
309.4 5.02
304.1 8.06
Panda_Dumbbe htt_mmbbbellcooppttttoorggggge
323.26 3.52
279.85 + 18.84
313.19 17.79
323.49 + 0.27
334.05
337.49 0.68
341.0 3.14
333.06
Panda_Plaatc.oooepPPicckleel.
384.37
404.77
1.3
398.41
Pands_Plaaee_ooo_Phshh
397.95 1.05
398.1 4.91
397.22
397 64
Panda Plate
Panda_Plate
None Shelf
352.29 37.8
372.12 13.92
370.46
367.5 6.03
392.99 +: 1.41
393.63 4 2,91
394.05 3.74
393.71 1.27
Plate SttteeeGGGGMeallcccPhh..e
3.99
398
4.551
1.9.0
399.02 4.553
Panda Plate Gaatseeooollll1111 Push
387.68 0.49
377.79 11.92
355.01 13.001
350.1 22.72
380.05
367.67
339.46 40.63
391.1
3.9.44
2.49
393.96 2.68
350.33 + 18.2
348.67 8.14
329.35
336.64 16.61
346.4 9.33
337,36 17.06
3.6922
323,51 2.24
290.68 11.21
321.54 17.89
326.04 4: 18.76
305.25 20.96
348.09 3.63
349.43 4.05
351.8 0.25
349.29 1.91
Panda Bwwlooboloooooooo None PickPlace
410.32 6.76
412.25 10.25=3..0
408.01 1.93
405
Banda_HollowboxxNoons.
1.07
402.46
SSeif
5.119
380.86 10.45
369.22 14.85
369.57 4.84
Panda lowbox Trashcan
399.09 2.01
400).52 5.27
401 5.277
392.82 7.37
406.02 10.18
403.47 0.97
405 9.39
407.16 + 3.77
Push
259.87 75.12
293.02
117.06
341.55 23.29
281.79
42.98
Panda_Ho wwwpll/oooooooaaooooaoa #
387.38 3.45
369.01 6.14
365.26 6.74
316.46 81.46
377.54 44.77
395.3 4.85
396.82 4.117
401.54
334.94
57.68
257.48 13.16
230.54
Panda Hollowbox otw.ebDoooobbeetteoooooooooooo Shelf Shelf
343,92 10.22
202. 17 1 4.87
328.01 1: 42.52
285.35
64.92
338.02 36.48
363.04 2.59
360.88 2.45
363.04 1.29
45


Reviews:
[{"review_id": "C7dy0vbu59", "reviewer": "Reviewer_8NAr", "summary": "The authors introduce a Large Recurrent Action Model (LRAM), which replaces traditional Transformer architectures with xLSTMs to address real-time robotic applications. They demonstrate that the proposed model achieves significant speed improvements without compromising performance. Experimental validation across 432 tasks from six domains confirms LRAM's superiority over Transformers regarding inference speed while maintaining competitive predictive performance.\n\n## Update after rebuttal\nThe authors have partially addressed my questions. However, without any experimental results (e.g., via an anonymous link), the claims cannot be verified by the reviewers. Therefore, I maintain my score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your helpful feedback on our work and your positive assessment. We are glad that you consider it a clear demonstration, robust experimental validation and valuable dataset pipeline. We address your open points in the following. \n\n**Policy formulation:** We agree with the reviewer that the differences between our policy formulation that omits actions (Equation 2) and the formulation in Equation 1 can be highlighted more clearly. Many thanks for the suggestion of using color coding. We now introduced color coding and believe it enhances readability. \n\n**Data ratios:**\n\n* A key motivation behind our dataset compilation was the scarcity of suitable existing datasets that span a large number of simulated tasks. To address this, our primary target was to assemble a collection of datasets that span as many tasks as possible to enable a robust comparison of sequence model architectures. To facilitate usability for future works, we consider standard benchmarks (e.g., Meta-World, Atari, Procgen) that are widely adopted by the community. Therefore, we hope that our data pipeline can serve as a solid basis for future research on multi-task agents.\n* Note that during pre-training, every domain is represented with approximately equal proportion in every update step (see Section 4.2 and Appendix C). Because the dataset sizes vary across domains (due to different numbers of tasks), this results in different numbers of total repetitions per dataset (Table 1). We opt for equal proportions, because we aim to study how the different backbones perform across domains, rather than optimizing performance on specific domains.\n* We agree with the reviewer that understanding the impact of the data ratios on multitask capabilities would be insightful. Varying the data ratios would, for example, allow studying potential interferences between the 432 tasks. While we believe that a thorough investigation on data ratios is out-of-scope for this work, because of the excessive cost or pre-training with varying ratios, we believe that studying data ratios in large action models represents an interesting direction for future work. We hope that our data pipeline and datasets may facilitate such future studies. \n\nWe updated our manuscript to include a short discussion about the rationales behind our data ratios.\n\n**Embedding plots for Mamba:** Thank you for the suggestion. We added the embedding plots for Mamba to Appendix F in our manuscript (we cannot update the PDF). Mamba exhibits a more refined embedding space than DT, but slightly less refined than xLSTM. However, while a more refined embedding may help with task identification at inference time, it is unclear how strongly it benefits final performance. \n\n**Inference speed of Mamba:** We agree with the reviewer that the inference speeds for Mamba were missing from our original manuscript and are important to provide the entire picture. Initially, we focused our inference time tests on DT and xLSTM, as xLSTM tends to be slightly faster than Mamba in prior work (see Figure 9 in [1]). Therefore, we now repeated the inference time analyses conducted in Section 4.4 with Mamba and provide the detailed results in our updated manuscript. As expected, we find that Mamba exhibits the same linear scaling properties as xLSTM. Consequently, Mamba exhibits advantages over DT in terms of speed and memory for longer sequence lengths. In our experiments, Mamba runs slightly slower compared to xLSTM (0.002 sec/step vs. 0.0048 for B=16 in Figure 6b), but requires slightly less GPU RAM. We hypothesize that this is because the Mamba kernels are not compatible with `torch.compile`, which may result in a slowdown. With compatible kernels, that gap might be closed. Moreover, Mamba achieves throughputs similar to xLSTM. When using larger batch sizes with xLSTM, we found that decreasing the head dimension (more heads, same total hidden dim) is important for enabling higher throughput, and added an ablation on this to the Appendix. This is because a higher head dimension incurs more FLOPS. Generally, these results reflect findings in existing works [1,2].\n\nOur manuscript improved considerably by addressing and incorporating your comments: thank you. In particular, believe that the additional results for Mamba strengthen the paper. \n\n[1] “xLSTM: Extended Long Short-Term Memory”, NeurIPS, 2024\\\n[2] “xLSTM 7B: A Recurrent LLM for Fast and Efficient Inference”, Arxiv, 2024", "claims_and_evidence": "The authors' claims are supported by substantial experimental evidence, demonstrating that xLSTM generally performs at least as well as the Decision Transformer (DT) and often surpasses it. However, the experimental results also show that xLSTM frequently achieves performance similar to the state-space model (SSM) \"Mamba.\" While the authors clearly illustrate xLSTM's speed advantages over DT, the paper lacks explicit evidence comparing inference speed against state-space models, which somewhat limits the strength of claims regarding real-time applicability. \nLastly, the authors acknowledge that the primary target application of LAMs is robotics and that they only tested in simulations; they believe that their findings translate to real-world scenarios. Although there is no direct evidence of this translation in the current work, the authors address this limitation in their discussion.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria, including benchmark datasets, appear appropriate for the targeted problem. Evaluations of performance scores, inference speed, and the impact of fine-tuning are reasonable and thorough.", "theoretical_claims": "No major theoretical claims requiring validation are presented, as the study primarily emphasizes empirical model performance.", "experimental_designs_or_analyses": "The experiments and analyses are well-structured, effectively demonstrating the benefits of the xLSTM architecture in inference efficiency and multitask performance. A significant contribution is the thorough investigation into the beneficial effects of xLSTM block ratios, confirming that sLSTMs are advantageous for state-tracking over longer horizons. This aligns with findings in related fields (e.g., time series forecasting, (P-LSTM, Kong et al., ArXiv 2024), (xLSTM-Mixer, Kraus et al., 2024)), broadening the potential impact of this work. However, since the authors highlight dataset compilation as a contribution, evaluating how different data source distributions affect model performance would further enhance the paper's insights into offline RL action model pretraining.", "relation_to_broader_scientific_literature": "The authors build on existing large action models by addressing the inherent limitations of Transformer-based approaches in real-time robotics. By integrating and extending recurrent networks particularly through xLSTM architectures the paper advances prior work that has traditionally relied on Transformers. Moreover, by discussing state-space models (SSMs), the work connects to a broader scientific trend of improving inference speed and scalability in sequence modeling.", "essential_references_not_discussed": "None that I am aware of.", "other_strengths_and_weaknesses": "S1. Clear demonstration of xLSTM’s advantages for potential real-time robotic inference.\n\nS2. Robust experimental validation across diverse robotic tasks.\n\nS3. Valuable dataset compilation and pipeline.\n\nW1. Although omitting actions from your policy formulation (Equation 2) is deliberate, its current form may confuse readers comparing it directly to Equation 1. Clarifying the differences, possibly through color-coding or multi-line formatting, would enhance readability.\n\nW2. As mentioned, omitting Mamba’s performance from the experimental comparisons limits the impact of the work, as readers would benefit from a direct performance comparison between DT, xLSTMs, and Mamba.\n\nW3. The model configuration and their particual area of application is not quite clear to me, as the results in figure 3 are not properly discussed. Why does [1:0] work so much better on procgen than [7:1].", "other_comments_or_suggestions": "I am really leaning towards a full accept, however in its current form the experiments do not provide the entire picture, as the performance of SSMs are missing.", "questions_for_authors": "Q1: The dataset compilation is highlighted as a notable contribution. Could you explain the rationale behind the specific data ratios presented in section 4.1? Understanding the impact of different ratios on multitask capabilities would be insightful.\n\nQ2: Can you provide embedding plots for Mamba alongside xLSTM and DT in Figure 5?\n\nQ3: Could you include latency results for Mamba in Figure 6 to clarify the comparative performance advantages of xLSTMs versus other state-space models (Mamba)?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8HeLJCdLMJ", "reviewer": "Reviewer_2HrF", "summary": "The authors propose changing the Decision Transformer (DT) backbone from a transformer to the recently-proposed xLSTM. They perform large-scale experimentation and compare to other DT backbones.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your helpful feedback. We are glad that you consider our experiments well-founded and the paper well written.\n\n**Embedding space:**\n\n* Regarding your question on the embedding space, we want to clarify that Figure 5 is constructed using the aggregated hidden states (averaged across the sequence) from the final layer of the respective agents and visualized via UMAP (see Appendix F for details). \n* The purpose of this visualization is to examine how the models organize their representations of different environments. In general, tasks within the same domain tend to share similar input characteristics - such as visual inputs (e.g., image frames) possible actions to perform, and reward structures - and are therefore more likely to be “grouped” together in the embedding space. Consequently, when embeddings of Atari games are closer to each other than to Procgen games, it indicates that Atari games share more similar underlying dynamics or inputs structures compared to Procgen. For reference, we now also include the embedding space plots for Mamba in our updated manuscript, as suggested by Reviewer 8NAr14.\n* We observe that xLSTM exhibits a slightly more refined and better-separated embedding space, which may be a reason for its better final performance. In contrast, DT produces embeddings with less clear separation between domains. This suggests that for the environments we consider in this work, it may be beneficial for the model to learn more “separate” representations, potentially because it facilitates task identification at inference time. However, we fully agree with the reviewer that generalization across domains is generally desirable if environments share structural similarities. Therefore, we believe that studying the learned embedding spaces of multi-task agents in environments that share more structure across domains is interesting for future work.\n\nWe hope to have clarified your remaining question. Thank you again for your positive assessment of our work.", "claims_and_evidence": "The authors contribute:\n- A Large Recurrent Action Model (LRAM) using an xLSTM with favorable performance\n- Comparisons against other recurrent backbones\n- They release code and datasets\n\nIn my opinion, they do what they claim. I will note that the claims are fairly weak.", "methods_and_evaluation_criteria": "The authors evaluate their method by examining:\n- Scaling capacity compared to other popular LRAMs\n- Returns across many popular benchmarks (Atari, MuJoCo, etc)\n- The embedding space\n- Latency and resource usage compared to a decision transformer\n\nThese experiments make sense, given they are comparing DT backbones.", "theoretical_claims": "The claims made are empirical and cannot be proven.", "experimental_designs_or_analyses": "The experiments seem well-founded and span a wide range of tasks and model baselines.", "relation_to_broader_scientific_literature": "Recent prior work on DT has focused on changing backbones with Mambas, State Space Models, etc. It is only natural to consider the xLSTM as well.", "essential_references_not_discussed": "None that I am aware of", "other_strengths_and_weaknesses": "The paper is well written and easy to follow. The authors are the first to use an xLSTM within a decision transformer framework, and provide a thorough study of their architecture with large number of experiments.\n\nHowever, it is a bit disappointing that there is not much novel here besides changing the DT backbone with another well-known sequence model. As such, there appear to be relatively incremental increases in performance.", "other_comments_or_suggestions": "There isn't much new here, but the authors provide more than sufficient empirical validation of their method.", "questions_for_authors": "The meaning of the embedding space (fig 5) is not clear to me. What does it mean if two Atari game embeddings are closer than an Atari and ProcGen embedding? Wouldn't the latter case suggest better generalization (generalizing across platforms, rather than within platforms). Could you comment on this?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1BCjEMQ1NS", "reviewer": "Reviewer_qHZg", "summary": "This paper investigates various architectures, including Transformers, Mamba, and xLSTM, for reinforcement learning. Building on the Decision Transformer framework, it systematically compares these architectures across 432 tasks spanning six datasets. The empirical results highlight xLSTM’s advantages in both performance and inference speed, demonstrating its effectiveness as a scalable alternative.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your helpful feedback and positive assessment of our work. \n\n**Imitation Learning:** We agree with the reviewer that studying modern recurrent architectures in settings other than the DT setting is important. In this work, our goal is to better understand whether modern recurrent backbones can be alternatives for large action models (LAMs) and focus our analyses on the DT setup. However, we want to highlight that we already provide experiments in an imitation learning / behavior cloning (BC) setting at the 206M scale, as suggested by the reviewer (Section 4.3 and Appendix E.2). Notably, performance trends across backbones mirror the results in the DT setting (Figures 27, 28), which indicates that our findings generalize beyond the DT framework. Moreover, we agree with the reviewer that studying modern recurrent architectures in RL is interesting. While we believe that such experiments are out-of-scope for this work, we listed online RL with modern recurrent architectures in our future work section (Lines 431-435). \n\n**Differences between Backbones:** While we discuss when and why to use which backbone (Section 5), we agree with the reviewer that an extended discussion on the reasons for performance differences between backbones is useful: \n\n* We observed that xLSTM outperforms Transformers/Mamba in terms of sequence prediction and env performance (Figure 2). One benefit of xLSTM is that it enables state tracking [1] via sLSTM blocks, which Transformers/Mamba cannot. This property can be useful for partially observable environments and may be a reason for the enhanced ICL performance (Figure 4). Another reason may be the enhanced domain-separation in the embedding space (Figure 5), which could facilitate task identification at inference time. Moreover, our results reflect performance improvements in language settings [2].\n* Nevertheless, the differences may depend on the environment. For example, Transformers may have advantages for tasks where exact recall is required. For such tasks, self-attention is typically superior (Figure 5 in [2]) and can be important for decision-making tasks [3]. Therefore, the choice for the right backbone may depend on the task at hand.\n\n**Long Context:**\n\n* Generally, performance improves when training with longer sequences both for DT and xLSTM (Section 4.3). For DT, the avg. normalized score increases from 0.3 with C=1 to 0.7 with C=50 (Figure 23). Similarly, for xLSTM it increases from 0.4 to 0.8 (Figure 25). Furthermore, domains with longer episode lengths, like DMC or Atari, benefit more from increasing context (Figures 24, 26). This is because the history helps to predict the next action (e.g., by observing past mistakes). This highlights that LAMs can strongly benefit from increased context length, even on the simulated environments we consider.\n* We agree with the reviewer that for our environments, increasing the context to several thousand timesteps may not result in better performance. However, we believe that the ability to handle longer sequences can be beneficial for complex real-world applications that exhibit longer-term dependencies. Similarly, longer context can be beneficial for ICL applications, which benefits from keeping multiple episodes in the context, as indicated in Figure 4.\n\nWe added a short discussion on long sequences in real-world tasks to our manuscript.\n\n**Experiment Setup:** Yes, we train a single model on datasets comprising 432 tasks. The environments contain both discrete/continuous action inputs and image/vector-based state representations. To handle both discrete/continuous actions, we make use of discretization. Every action dimension is discretized into 256 uniformly spaced bins. The shared action head is used to predict the action bins of all continuous dimensions jointly, which allows handling envs with different action spaces. At inference time, the number of dimensions of the current environment is known, and we extract the respective dimensions from the joint predictions. We describe this procedure in detail in Appendix B.3.\n\n**Inference Speed:** The reviewer is correct that the benefits of modern recurrent architectures become more important at longer sequence lengths. Importantly, the inference speed and memory consumption of xLSTM/Mamba does not change with increasing sequence lengths, which is particularly apparent in memory consumption (Figure 6c). It is true that in our experiments there are some inference time speed-ups at shorter context lengths, which may result from the different kernel implementations. Note that the sequence lengths are measured in timesteps with each timestep containing 3 tokens (largest C is 76K tokens).\n\nWe revised our manuscript, and hope to have clarified your questions. \n\n[1] The Illusion of State in State-Space Models, ICML 2024 \\\n[2] xLSTM: Extended Long Short-Term Memory, NeurIPS, 2024 \\\n[3] When Do Transformers Shine in RL? Decoupling Memory from Credit Assignment, NeurIPS, 2023", "claims_and_evidence": "The claims in the paper are supported by thorough experiments.", "methods_and_evaluation_criteria": "The proposed method and evaluation make practical contributions to the community.", "theoretical_claims": "There is no theoretical proof to check in this paper.", "experimental_designs_or_analyses": "The paper conducts comprehensive experiments and ablation analyses.", "relation_to_broader_scientific_literature": "The paper is based on the Decision Transformer [1] and further explores recent architectures Mamba and xLSTM to improve the performance and inference speed.\n\n[1] Chen, Lili, et al. \"Decision transformer: Reinforcement learning via sequence modeling.\" *Advances in neural information processing systems, 2*021.", "essential_references_not_discussed": "The references are enough to understand the paper.", "other_strengths_and_weaknesses": "Strengths:\n\n- The paper is well-written and easy to follow.\n- The paper provides extensive experiments and ablation studies.\n- The paper compares the latent representations between Transformer and xLSTM, which is informative.\n\nWeaknesses:\n\n- The paper is based on Decision Transformer which has not been used much these days. It would be better to extend the architectures to other reinforcement learning or imitation learning settings.\n- The paper mainly compares Transformer, Mamba, and xLSTM from empirical evaluations, but does not discuss the reason. It would be valuable to provide more discussions regarding why xLSTM can outperform Mamba and xLSTM.\n- The paper compares the inference time using different context lengths from 50 to 24576. However, there is no indication that the model will perform better with a very long context length. Additionally, most robot learning tasks do not contain such long trajectories. It would be beneficial to discuss scenarios where they offer a significant advantage.", "other_comments_or_suggestions": "The paper introduces Transformer, Mamba, and xLSTM in the Introduction and Related Work parts. Since these architectures are the main focus, maybe it makes sense to explain their differences using a single paragraph.", "questions_for_authors": "- I am a bit confused regarding the experiment setting. The paper uses 6 different datasets. Do the authors train one model over all the tasks? Additionally, in Section 3.2 “Shared action head”, can the authors explain more about action discretization?\n- From my understanding, Mamba and xLSTM will only inference faster when there are more than several thousand tokens (I am not absolutely sure). However, in Figure 6, the xLSTM performs still faster with a small context length.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8dI0ve96ka", "reviewer": "Reviewer_xo6G", "summary": "This paper tackles the recurrent approach to the large action model, where current robotic policies are de facto implemented as transformers. The paper systematically analyzes xLSTM under various conditions and across different environments, demonstrating the superior performance of xLSTM compared to transformer-based policies. Due to the recurrent characteristics of xLSTM, the Recurrent Large Action Model (RLAM) can efficiently scale up to large contexts without incurring the quadratic complexity associated with increasing tokens.\n\n## update after rebuttal\n\nAfter carefully reading the rebuttal, I have decided to keep my original score.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive feedback. We appreciate your positive assessment and are very glad that you believe that our work may mark a potential turning point for real-world agent design.\n\n**Real-World Experiments:** We agree that experiments in real-world robotics settings would be valuable. While we believe that such experiment are out-of-scope for the current work, evaluating modern recurrent architectures in real-world robotics settings is an exciting direction for future research. Nevertheless, we believe our findings will transfer to real-world scenarios and that the advantages of recurrent architectures may be even more pronounced in real-world settings. For example, we conduct our inference time comparison on a high-end data-center GPU with 40GB of RAM and the Transformer runs OOM for larger context sizes. In contrast, applications on edge devices in real-world scenarios, may have to deal with less powerful accelerators, making the use of modern recurrent architectures attractive. Moreover, their ability to handle long sequences without increasing computational cost/requirements can be particularly beneficial for complex real-world applications, which often exhibit longer-term dependencies. \n\n**Memory-intensive Environments:** We again agree that exploring the advantages of modern recurrent architectures in long-term memory retention more would be interesting. While we cannot provide a detailed study within the time-frame of the rebuttal, we believe that this direction is interesting for future work. Note that the experiments on Dark-Room, which exhibits sparse rewards and a partially-observable observation space, go in a related direction (Figure 4 and 16). There we find that recurrent backbones compare favorably to Transformers (especially xLSTM [7:1], which enables state-tracking). In the meantime, we refer to [1] for a comparison of vanilla LSTM to the Transformer in memory intensive RL environments. Furthermore, we refer to the Figure 5 in [2], for a comparison outside the field of RL of xLSTM and Transformer on associative recall tasks. \n\n**Discretization for Continuous Actions:** The reviewer is correct, that we make use of discretization of continuous actions similar to prior works. The 432 task we consider exhibit both discrete/continuous action inputs and image/vector-based state representations. The use of discretization in our large action models is motivated by the need to handle both discrete and continuous action spaces. To this end, every action dimension is discretized into 256 uniformly spaced bins. The shared action head is used to predict the action bins of all continuous action dimensions jointly, which allows handling envs with different action spaces. Furthermore, because of the shared action head, we do not (have to) rely on autoregressive prediction of continuous action dimensions, which further speeds up inference for all backbones. At inference time, the number of dimensions of the current environment is known, and we extract the respective dimensions from the joint predictions. We describe this procedure in detail in Appendix B.3. \n\n**Alternative Designs:** In this work, our goal is to better understand whether modern recurrent backbones can be alternatives for large action models (LAMs) and focus our analyses on the DT setup as correctly identified by the reviewer. DTs rely on return-conditioning, but existing large-scale robotic models instead rely on behavior cloning (e.g., [2]). Therefore, we validate that our findings also transfer to a behavior cloning setting at the 206M scale (Section 4.3 and Appendix E.2). Exploring alternative architectures beyond the DT and BC settings is an exciting direction for future work. We believe that exploring MoEs [4,5] for multi-task settings, which may require specialization, can be a promising approach. Moreover, we would like to point the reviewer to an interesting concurrent work that studies the design space of imitation learning policies [6]. \n\nOnce again, thank you for your helpful comments and positive assessment of our work. \n\n[1] “When Do Transformers Shine in RL? Decoupling Memory from Credit Assignment”, NeurIPS, 2023\\\n[2] “xLSTM: Extended Long Short-Term Memory”, NeurIPS, 2024\\\n[3] “RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control”, ArXiv, 2023\\\n[4] “Mixture of Experts in a Mixture of RL settings”, RLC, 2024\\\n[5] “Mixtures of Experts Unlock Parameter Scaling for Deep RL”, ICML, 2024\\\n[6] “X-IL: Exploring the Design Space of Imitation Learning Policies”, ArXiv, 2025", "claims_and_evidence": "The claims of this work are clear and well-supported. The xLSTM-based RLAM efficiently exploits large contexts, with sufficient ablation studies on each xLSTM block (mLSTM and sLSTM). The authors also conducted latency and throughput studies, comparing xLSTM to transformer-based architectures, demonstrating that the xLSTM-based architecture is more lightweight and efficient to deploy compared to transformers.", "methods_and_evaluation_criteria": "The proposed method focuses on utilizing xLSTM with an existing RL model backbone, evaluating it with various RL benchmarks in offline RL with large transitions. Although the paper does not propose a new architecture, the comprehensive analysis of xLSTM-based RLAM is sufficient to identify trends in efficiency and performance for recurrent-based models compared to transformers through experiments on various benchmarks.", "theoretical_claims": "I checked the theoretical claims, especially Equation 2, which is based on the Decision Transformer framework. I also reviewed the reasoning behind excluding action tokens, which is well-justified.", "experimental_designs_or_analyses": "I reviewed the experimental design, and the paper provides sufficient information. However, in Section 4.4 (Inference Time Comparison), the model setup with custom kernels was not fully clear. Therefore, I have complaint that I did not entirely understand how the custom kernels were designed and adapted to accelerate inference speed, where this is not common knowledge for general readers.", "relation_to_broader_scientific_literature": "The contribution of this paper is not in introducing a novel architecture but in demonstrating the feasibility of recurrent models for large action models. With comprehensive analysis and ablation studies, the paper provides an important milestone for recurrent-based robotic action models.", "essential_references_not_discussed": "Overall, the paper sufficiently discusses related references.", "other_strengths_and_weaknesses": "Strengths\n1. The recurrent-based action model is especially important in online, real-world settings, where context length varies dynamically, and parallelization is difficult. I think this paper would marks a potential turning point for real-world agent design.\n2. The paper thoroughly analyzes different types of recurrent-based large action models, including xLSTM and Mamba, conducting ablation studies to assess performance and efficiency.\nWeaknesses\n1. While the paper argues for the importance of recurrence in real-world settings, all experiments are conducted in offline RL. To fully validate RLAM, real-world experiments would be highly beneficial.\n2. The paper extensively analyze architectural efficiency and performance, but it does not fully explore the advantages of recurrence in long-term memory retention. More experiments in memory-intensive environments could highlight how recurrent backbones differ from transformers in terms of state persistence and recall.", "other_comments_or_suggestions": "none", "questions_for_authors": "1. The paper adopts 256-bin discretization for continuous actions, following [Reed et al., 2022] and [Brohan et al., 2023b]. Many robotic models (e.g., OpenVLA [Kim et al., 2024]) use this discretization, especially for autoregressive architectures; however, some (e.g., ACT [Zhao et al., 2023]) output continuous actions directly. Are there specific reasons for modeling with discretization rather than direct continuous actions, or was this decision influenced by the Decision Transformer backbone?\n2. The paper replaces Decision Transformer’s transformer backbone with xLSTM. Are there alternative architectures beyond Decision Transformer that might better suit recurrent models? Do the authors have insights into what could be an even better design for recurrent large-action models?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper presents an expansive comparison of xLSTM backbone against transformers for real-time robotics applications, showing that xLSTMs lead to a significant efficiency boost. However, it seems the performance is similar to Mamba, so it is not clear what the significance of the contributions of this paper are.

There are no theoretical contributions, so the work stands only on the empirical results. While there is emphasis on real-time robotics, there are no robotics experiments. The authors point out that this is still useful for general multi-task agents, but then their own emphasis could have been in that direction instead. 

I recommend the authors modify the messaging and add some results for the multi-task setting for the camera-ready to make the paper stronger.
Decision: Accept (poster)