Title: BEAST: Efficient Tokenization of B-Splines Encoded Action Sequences for Imitation Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: Action Tokenizer

Imitation Learning

Vision-Language-Action Models
Abstract: We present the B-spline Encoded Action Sequence Tokenizer
(BEAST), a novel action tokenizer that encodes action sequences into compact discrete or continuous tokens using B-splines. In contrast to existing action tokenizers based on vector quantization or byte pair encoding,  BEAST requires no separate tokenizer training and consistently produces tokens of uniform length, enabling fast action sequence generation via parallel decoding. Leveraging our B-spline formulation, BEAST inherently ensures generating smooth trajectories without discontinuities between adjacent segments. We extensively evaluate BEAST by integrating it with three distinct model architectures: a Variational Autoencoder (VAE) with continuous tokens, a decoder-only Transformer with discrete tokens, and Florence-2, a pretrained Vision-Language Model with an encoder-decoder architecture, demonstrating BEAST's compatibility and scalability with large pretrained models.  We evaluate BEAST across three established benchmarks consisting of 166 simulated tasks and on three distinct robot settings with a total of 8 real-world tasks. Experimental results demonstrate that BEAST (i) significantly reduces both training and inference computational costs, and (ii) consistently generates smooth, high-frequency control signals suitable for continuous control tasks while (iii) reliably achieves competitive task success rates compared to state-of-the-art methods.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces BEAST (B-spline Encoded Action Sequence Tokenizer), a novel method for representing continuous robot action sequences as compact, fixed-length discrete or continuous tokens. A core aspect of BEAST is its use of B-splines, which intrinsically ensures smooth, continuous trajectories without requiring separate tokenizer training.\n\nThe key contributions of this work are:\n*   **Novel B-spline-based Tokenization:** BEAST encodes continuous action sequences using B-splines, providing a compact and efficient representation.\n*   **Versatility Across Architectures:** The paper demonstrates BEAST's compatibility by integrating it with various model architectures, including VAEs (BEAST-ACT), decoder-only Transformers (BEAST-D), and Vision-Language Models like Florence-2 (BEAST-F), supporting both continuous and discrete token objectives.\n*   **Enhanced Efficiency:** BEAST significantly reduces computational costs during both training and inference. Its fixed-length token sequences enable faster action generation through parallel decoding, requiring significantly fewer tokens compared to binning-based methods.\n*   **Guaranteed Smoothness:** By leveraging clamped B-splines and enforcing initial conditions, BEAST ensures mathematically guaranteed smooth transitions between consecutive action chunks, preventing jerky movements.\n*   **Strong Performance:** Extensive evaluations on 166 simulated tasks (CALVIN, LIBERO, ALOHA) and 8 real-world robotic tasks across three different setups demonstrate that BEAST achieves competitive or superior task success rates compared to state-of-the-art methods, often without relying on large-scale pretraining.\n*   **Ablation Studies:** The paper includes thorough ablation studies analyzing the impact of design choices such as discrete versus continuous tokens, the number of basis functions, and model size, providing insights into BEAST's performance characteristics.\n\nA noted limitation is BEAST's sensitivity to the chosen number of B-spline basis functions, which can impact performance.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths:\n\n*   **Originality:** The core idea of using B-splines for action sequence tokenization, specifically the direct encoding without separate training and the inherent smoothness guarantee, appears novel in the context of imitation learning and VLA models. This differentiates it from common binning, VQ-VAE, or compression-based methods.\n*   **Quality - Technical Soundness:** The mathematical formulation of B-splines and their application to action sequences (Equation 4, clamped B-splines for smooth transitions) is well-grounded. The claim of mathematically guaranteed smoothness is a strong technical advantage. The efficiency gains from fixed-length tokens and parallel decoding are clearly articulated and supported by empirical results.\n*   **Quality - Empirical Performance:** The paper presents extensive evaluations across numerous simulated benchmarks (CALVIN, LIBERO, ALOHA, 166 tasks) and real-world robot setups (8 tasks across 3 different robots). Consistently achieving competitive or state-of-the-art results, often with smaller models and without large-scale robot-specific pretraining, demonstrates the method's effectiveness. The ablation studies provide useful insights into design choices.\n*   **Clarity:** The paper is generally well-written and easy to follow. The introduction clearly outlines the problem and BEAST's proposed solution. Figure 1 and 2 effectively illustrate the core concepts of B-splines and the encoding pipeline. The experimental setup and results are presented clearly with appropriate tables and figures.\n*   **Significance:** BEAST offers a practical solution to several challenges in robot imitation learning, including action sequence discretization, ensuring smooth trajectories, and improving inference efficiency. Its compatibility with various architectures (VAE, Transformer, VLM) makes it a versatile tool. The efficiency gains (2.14x to 101.4x faster inference, faster training convergence) are highly significant for real-world robotic applications and research. The reduced token count is also a notable contribution to efficiency.\n\n### Weaknesses:\n\n*   **Quality - Sensitivity to Basis Function Selection (Actionability):** While the paper acknowledges the sensitivity to the number of B-spline basis functions (\\`N\\`) in the limitations, the explanation for why \\`N=20\\` performs significantly worse than \\`N=15\\` for a 20-step chunk in Table 4 is somewhat vague (\"reduce compression, which can also negatively influence the performance\"). This is a critical hyperparameter, and the guidance \"5-10 bases works well for one-second robot trajectories\" might not generalize well to different action chunk lengths or robot dynamics. A more principled or robust way to select \\`N\\`, or a deeper understanding of its impact beyond a simple trade-off, is missing. \n*   **Clarity - Notation for B-spline Parameters:** There's a slight potential for confusion between the B-spline degree \\`P\\` and the number of control points \\`N\\` (e.g., \"Remark 1\" stating \\`T=N\\` for \\`P=0\\` vs. \\`N <= T\\` for \\`P\\`-th degree, and how \\`N\\` is precisely chosen for \\`P > 0\\` degrees given a \\`T\\`-length action sequence). While explained, a concise clarification on the typical choices of \\`P\\` and \\`N\\` in relation to the action sequence length \\`T\\` in practice would improve understanding.\n\nQUESTIONS:\nHere are some questions and suggestions for the authors, focusing on actionable points for a productive rebuttal:\n\n1.  **Robustness and Generalization of Basis Function Selection (Line 337-339):**\n    *   **Question:** The limitations section acknowledges sensitivity to the number of B-spline basis functions (`N`), stating that \"The optimal count depends on the smoothness and sampling frequency of the trajectory; our experiments indicate that using 5-10 bases works well for one-second robot trajectories.\" However, Table 4 shows a significant drop in average sequence length (performance) from `N=15` (4.14) to `N=20` (2.71) for a 20-step action chunk (Line 322). This suggests that `N=20` might be suboptimal or that higher `N` values *can* negatively influence performance, which contradicts the intuition that more basis functions provide more representational power. Could the authors elaborate on the trade-offs beyond \"compressing vs. representational power\"? Is there a specific threshold or relationship between `N` and `T` (action sequence length) beyond which performance degrades significantly, or a methodology for finding the optimal `N` given varying trajectory characteristics?\n    *   **Suggestion:** Expand on the reasons for the performance drop at `N=20` in Table 4. Discuss more deeply the practical considerations and a more systematic approach (e.g., a small search, heuristic, or adaptive method) for selecting `N` across different tasks and action sequence lengths, especially given the sensitivity mentioned.\n    *   **Criteria for Score Change:** Providing a more detailed analysis of the `N` selection, perhaps with a clear methodology for practitioners to determine `N` or a deeper insight into the observed performance drop, would strengthen the paper's claims about its practical applicability. If the authors can show a more robust way to handle this sensitivity, the score would increase.\n\n2.  **Comparison with Other Trajectory Parameterizations / Implicit Representations:**\n    *   **Question:** The paper primarily compares BEAST against \"binning-based tokenization\" (Section 5.1) and other VLA/Diffusion policies that might use simpler action chunking or direct prediction. While BEAST's B-spline approach offers advantages like smoothness and fixed-length tokens, there are other established methods for continuous trajectory parameterization in robotics (e.g., Gaussian Process Motion Priors, Dynamic Movement Primitives, or even learned implicit representations for trajectories). Could the authors discuss how BEAST compares conceptually and/or empirically (if relevant prior work exists) against these alternative trajectory representations, especially concerning their ability to ensure smoothness, achieve compression, and integrate with deep learning architectures?\n    *   **Suggestion:** Add a brief discussion in the Related Work or Limitations section acknowledging other continuous trajectory representations. Even a qualitative discussion of their pros and cons relative to BEAST's B-spline approach would add significant value by contextualizing BEAST within a broader spectrum of trajectory encoding methods.\n    *   **Criteria for Score Change:** Acknowledging and briefly discussing other continuous trajectory parameterization methods would demonstrate a more comprehensive understanding of the landscape and position BEAST more clearly within it, leading to a higher score.\n\n3.  **Implications of \"Without Pretraining\" (Line 247-248):**\n    *   **Question:** The paper frequently highlights that BEAST-F achieves strong results \"without relying on additional pretraining\" (Line 247) or \"without robot data pretraining\" (Line 248). While this is a significant advantage, Florence-2 itself is a large pre-trained Vision-Language Model. Could the authors clarify the distinction more explicitly? Is the claim specifically that BEAST-F *doesn't require additional robot-specific pretraining* (beyond what Florence-2 already has for general vision-language tasks), or that it performs well *from scratch* (which seems less likely given Florence-2's nature)? A nuanced explanation would prevent potential misinterpretations regarding the extent of \"pretraining\" involved.\n    *   **Suggestion:** Rephrase or elaborate on the \"without pretraining\" claim to clearly state that it refers to *robot-specific* or *second-stage* pretraining on large robot datasets, distinguishing it from the foundational pretraining of Florence-2. This would make the contribution clearer and more precise.\n    *   **Criteria for Score Change:** A precise clarification of what \"without pretraining\" entails in the context of BEAST-F, acknowledging the base VLM's pretraining, would increase the clarity and accuracy of the paper's claims, leading to a higher score.\n\nLIMITATIONS:\nPlease check the weakness section.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 1, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces BEAST (B-spline Encoded Action Sequence Tokenizer), a novel method for tokenizing continuous action sequences in imitation learning. Instead of relying on existing tokenization methods like vector quantization or binning, BEAST employs B-spline basis functions to compress continuous action trajectories into fixed-length, smooth representations. These can be used as discrete tokens (after quantization) or continuous control points, enabling compatibility with both autoregressive and parallel decoding architectures.\n\nKey contributions include:\n\n1. A tokenizer that does not require separate training and produces fixed-length, smooth action sequences.\n2. Compatibility with multiple model types (e.g., transformer, CVAE, Florence-2).\n3. Strong empirical performance across simulation benchmarks (CALVIN, LIBERO) and real-world robotic tasks.\n4. Substantial improvements in training/inference efficiency via parallel decoding and action compression.\n\nSTRENGTHS AND WEAKNESSES:\n# Quality\n\n**Strengths**: novel and practical method that improves both performance and efficiency; extensive experimental validation, including simulation and real-world benchmarks; strong ablations showing the value of B-spline representation and design choices; impressive improvements in inference speed (2-100x over baselines); smooth trajectory generation is demonstrably better than binning or quantized chunking.\n\n**Weaknesses**: some ablation results (e.g., choice of basis functions) suggest non-trivial sensitivity, but there's limited discussion on tuning strategies; paper could better detail failure cases or conditions where BEAST might underperform\n\n# Clarity\n\n**Strengths**: the methodology is well-explained, especially the B-spline encoding and its advantages; figures (e.g., tokenization pipeline, spline smoothness) are informative; experimental setup and benchmarks are clearly described.\n\n**Weaknesses**: the distinction between discrete and continuous token modes in some sections could be more explicit; descriptions of integration into large models like Florence-2 may assume familiarity with these architectures.\n\n\n# Significance\n\n**Strengths**: addresses a core bottleneck in modern VLAs-tokenization of continuous actions; broadly applicable to robotics, VLMs, and imitation learning; effective across multiple benchmarks and robots without large-scale pretraining; could lower the barrier to applying VLAs in real-world robotics.\n\n# Originality\n\n**Strengths**: B-spline-based tokenization is novel in this domain; clever exploitation of B-spline properties (local support, smooth transitions, fixed-length encoding); avoids training a separate tokenizer: conceptually elegant and practically useful.\n\n**Weaknesses**: some concepts (e.g., B-spline smoothing, ridge regression) are not new, though their application here is novel.\n\nQUESTIONS:\n1. Section 5.5 shows that BEAST is sensitive to the number of basis functions. Can you provide heuristics or general rules for selecting this parameter across different robot platforms and frequencies?\n\n2. Are there scenarios where BEAST fails to outperform traditional tokenizers, such as in highly discontinuous or impulsive control settings?\n\n3. You highlight reduced token counts. Can you elaborate on any observed tradeoffs between compression and policy expressivity beyond MSE?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-14"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes an action sequence modeling approach that is better suited for representing continuous actions and is more efficient in capturing temporal dynamics.\n\nSTRENGTHS AND WEAKNESSES:\n- Strengths：\n    - The idea is both interesting and intuitive. It adopts a more explainable approach to action modeling and demonstrates how it improves the smoothness of transitions between action chunks. The method proves to be effective and efficient in the context of robotic manipulation.\n-  Weaknesses:\n   - The main weakness lies in the experimental design used to demonstrate how the proposed method works and its advantages. I believe this is partly due to the experimental setup and partly due to the writing. Please refer to the questions for more details.\n\nQUESTIONS:\nWhen I read the introduction, I expect your work to address a concrete problem in VLA—specifically, the limitations of existing methods that use autoregressive models to predict discrete actions. However, when I get to Section 4.3 and the experiments, I no longer understand what problem your method is trying to solve. At one point, it is combined with ACT (which already predicts continuous actions rather than discrete ones), and at another point, it is compared with methods like Pi0, which use DIT to output continuous actions—even though the architectures are fundamentally different. I think you are trying to show your work is goot at lots of aspects, but such experiments design is weak to prove that.\n\nThat said, I think your idea is valuable. What I expect to see is whether your method can be plugged into existing frameworks—for example, replacing the action modeling strategy in Pi0 or OpenVLA—and bring consistent improvements. Instead, the current evaluation focuses on combining your method with DaViT and Florence to form Beast-VLA, but these vision-language models are rarely used in prior VLA work, which makes the comparison less convincing.\n\nThe following are detailed questions that may overlap with the overall points mentioned above:\n\n- Why do you choose DaViT and Florence to build your VLA model, instead of using more commonly adopted backbones such as DINO or SigLIP?\n\n- Why do you combine with ACT or replace ACT’s action modeling, given that ACT is unrelated to discrete action modeling?\n\n- Why not integrate your method into PI0 or OpenVLA to demonstrate that your action modeling approach can improve existing VLA models? While PI0 uses DiT to predict continuous actions and is not based on discrete action modeling, it still serves as a strong and relevant baseline to show your action modeling strategy can even outperform DiT's action modeling.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-22"}