APPLE: Toward General Active Perception via Reinforcement Learning
Authors: Tim Schneider
Cristiana de Farias
Roberto Calandra
Liming Chen
Jan Peters
Venue: ICLR 2026 Poster (2026)
Keywords: Active Perception
Reinforcement Learning
Tactile Sensing
Transformers
Primary area: reinforcement learning
TL;DR: APPLE is a reinforcement learning framework for general active perception which we evaluate on a range of different tasks, focussed on tactile perception

Abstract: Active perception is a fundamental skill that enables us humans to deal with uncertainty in our inherently partially observable environment. For senses such as touch, where the information is sparse and local, active perception becomes crucial. In recent years, active perception has emerged as an important research domain in robotics. However, current methods are often bound to specific tasks or make strong assumptions, which limit their generality. To address this gap, this work introduces APPLE (Active Perception Policy Learning) – a novel framework that leverages reinforcement learning (RL) to address a range of different active perception problems. APPLE jointly trains a transformer-based perception module and decision-making policy with a unified optimization objective, learning how to actively gather information. By design, APPLE is not limited to a specific task and can, in principle, be applied to a wide range of active perception problems. We evaluate two variants of APPLE across different tasks, including tactile exploration problems from the Tactile MNIST benchmark. Experiments demonstrate the efficacy of APPLE, achieving high accuracies on both regression and classification tasks. These findings underscore the potential of APPLE as a versatile and general framework for advancing active perception in robotics.

Project page: https://timschneider42.github.io/apple

Published as a conference paper at ICLR 2026
APPLE: TOWARD GENERAL ACTIVE PERCEPTION VIA REINFORCEMENT LEARNING
Tim Schneider Cristiana Farias', Roberto Calandra', Liming Chen', and Jan Peters*
Department of Computer Science, TU Darmstadt, Germany.
LIRIS, CNRS UMR5205, École Centrale de Lyon, France. LASR Lab & CеTI, TU Dresden, Germany.
DDFII Hessian.Al, RIG, and Centre for Cognitive Science, TU Darmstadt, Germany. Corresponding author: romort -eantt-eeaiinnng de
ABSTRACT
Active perception is a fundamental skill that enables us humans to deal with uncertainty in our inherently partially observable environment. For senses such as touch, where the information is sparse and local, active perception becomes crucial. In recent years, active perception has emerged as an important research domain in robotics. However, current methods are often bound to specific tasks or make strong assumptions, which limit their generality. To address this gap, this work introduces APPLE LE (Active Perception Policy Learning) novel framework that leverages reinforcement learning (RL) to address a range of different active perception problems. APPLE jointly trains transformer-basee perception module and decision- maaiinnnnmminn policy with unified optimization objective, learning how to actively gather information. By design, APPLE is not limited to specific task and can, in principle, be applied to a wide range of active perception problems. We evaluate variants of APPLE across different tasks, including tactile exploration problems from the Tactile MNIST benchmark. Experiments demonstrate the efficacy of APPLE, achieving high accuracies both regression and classification tasks. These findings underscore the potential of APP LE as a versatile and general framework for advancing active perception in robotics.
Project page: hhtgs///umsehneeieee/Rgguuuooobp
INTRODUCTION
Imagine searching for a set of tools inside a clut- tered toolbox. You do not know where tool located or how it is positioned. Rather than waiting passively for the information to reveal it- self, most people would place their hands inside the box and begin exploring. They would probe and adjust their motions based on the feedback received. This process illustrates the concept of active perception: the deliberate selection of ac- tions to acquire information in the face of uneer- tainty Bajesy (1988). Active perception does not aim to exhaustively explore every aspect of the world; it focuses on finding efficient strategies that reduce uncertainty about specific properties of the environment. Equipping robots with this capability is a key step toward enabling them to
Thesta HARST
APPLE
Action
Prediction
Our method Active Perception Policy Learning (APPLE) aims to infer properties, such as object classes, of its environment based limited per-step information. To do so. jointly optimizes an action policy for information gathering and a pre- diction model for inference. Both the action policy and prediction models use shared tramsformer based backbonc process input sequences. Shown at the top four benchmark tasks
Figure
evaluate APPLE. act autonomously in unstructured environments, where information is sparse, noisy, and incomplete.

---

Published as a conference paper at ICLR 2026
Particularly relevant to active perception is the sense of touch. While vision has been widely explored in this context, especially for tasks such as object search and next-best- best-view planning Yang et al. (2019); Xiong al. (2025a); Tan et al. (2020), tactile sensing poses distinct challenges. Unlike vision, which can provide wider information coverage from a single observation, touch is inherently local, with each contact providing only small glimpse of the environment [Prescott et al. (2011). This locality makes tactile sensing natural fit for active perception, since purposeful interaction often the only practical way to gather meaningful information. Several works have investigated active tactile perception, e.g., for shape estimation Björkman et al. (2013): [Smith et al. (202T), texture (naniitionnBBhmmmmm
recognition BBoehm et al. (2024), and grasping De Farias al. (2021). However, these approaches are typically algorithmically tailored to a task-specifie objective, such as maximizing force closure for grasping or reconstructing shape, often using greedy information- gain heuristics. Moreover, methods such as Björkman et al. (2013): Dragiev et al.] (2013);[] De Farias et al. (202T) simplify the problem by assuming that objects remain stationary during exploration, an assumption that overlooks the dynamic and contact-rich nature of tactile exploration. While effective in narrow settings, such methods remain fundamentally tied to predefined objectives and assumptions. In contrast, alternative options, such as reinforcement learning (RL), can provide more general framework for acquiring more dynamic active perception policies. RL enables agents to learn sequential decision-makke strategies directly from interaction, guided by broader task-level goals rather than hand-crafted optimization criteria.
Some works have explored RI. for learning active perception strategies. For tactile sensing, various RL algorithms have been employed, including REINFORCE Fleer et al. (2020), PPO Xu et al. (2022); ;
Shahidzadeh et al. (2024), and DQN Smith et al. (202I). Notably, Fleer et al. (2020) introduced the Haptic Attention Model (HAM), which adapts the recurrent model of visual attention proposed by Mnihh et al.] 22144) to the tactile domain. HAM demonstrated that even simple policy gradient methods such as REINFORCE can learn exploration strategies by jointly optimizing perception and action, enabling haptic object classification. However, on-policy approaches, such as REINFORCE and PPO, can be sample-inefficient limiting their scalability. Moreover, the generality of active perception methods has not been widely studied: most existing approaches are tied to specific tasks and objectives, lacking a unified formulation. Ideally, the formulation of such methods should be agnostic to the underlying task, enabling transfer across modalities and problem settings.
Thus, in this work, we ask the following: can we design a principled RL-based algorithm for discovering active perception policies by relying only loss during training? And more broadly, can such an approach be general enough to extend across a diverse set of active perception problems, ranging from classification regression, without requiring task-specific exploration heuristics? To investigate these questions, we frame active perception within the setting of partially observable Markov decision processes (POMDPs), where an agent must act under uncertainty to actively reduce ambiguity about a target property. Note that in this work, we purely aim to evaluate the agent's ability to actively perceive. Accordingly, we focus on tasks where the agent's 's primary objective is to learn property of the environment (e.g., the class or pose of an object) and leave the evaluation of problems that involve active perception for another downstream task to future work. Building on this formulation, we introduce APPLE (Active Perception Policy Learning), framework that combines reinforcement learning with supervised learning, requiring only differentiable loss function and POMDP environment. APP LE jointly trains a decision-makee policy and a perception module on top of a shared transformer backbone, allowing it accommodate diverse sensor inputs without task-specific modifications. We present two variants of APPLE, extending SAC Haarnoja et al. (208)) and Cross Bhatt et al. (2019). and evaluate them across five benchmarks that include classification, volume estimation, and localization tasks (see Fig. D. Our experiments show that our RL framework can be effective for active perception in several tasks and serves as a step towards a more general framework. Our main contributions are:
ground-truth label and differentiable
A unified formulation for active perception that motivates using combination of policy gradient methods and supervised learning to solve interactive supervised learning problems.
A framework for active perception that jointly trains a reinforcement learning policy and perception module on a shared transformer backbone. This formulation enables adaptability across different tasks by making minimal assumptions about the nature of the underlying POMDP.
Comprehensive empirical evaluation of two method variants, based SAC and CrossQ, on classification, volume estimation, and localization tasks, demonstrating that APPLE can discover active exploration policies without task-specific heuristics.
2

---

Published as a conference paper at ICLR 2026
2 RELATED WORK
Active perception has been studied for both vision and touch (Bajesy 19888; Bajesy et al. (2018); Bohg
iet al. 2017): Taylor et al. (2021) and in the context of Applications range from object localization
and tracking Yang et al. (2019); Gounis al. (024)); Tallamraju et al. (2019); Mateus al. (2022),
to scene description T lan et al. (2020), object property identification Boehm et al. (2024) shape
estimation Yi et al. 2016); Björkman et al. (201)), UAV navigation Bartolomei et al. (2021) J and
robotic manipulation De Farias et al. (2021): Xiong et al. (2025a); [Daagiev et al. (2013). Many of these works have been formulated using reinforcement learning or non-parametric methods such as Bayesian optimization, with few imitation- wewno-lllaannnaaanaeeae
approaches emerging more recently Yang ct al. (0044); Chuang et al. (2025): Liu et al. (2025); Dai et al. (2022); Xiong et al. (2025b). However, these methods usually tailored to specific tasks, environments, and objectives, and often assume the agent does not influence the environment through its actions. To our knowledge, there exists no active perception method that has been shown work wide range of tasks, objectives, and environments.
Active Perception for Tactile Sensing: Tactile sensing allows robots to infer object geometry, texture, and materials through physical contact, complementing or substituting visual sensing, especially in occluded scenarios. Early tactile sensing systems primarily used simple binary contacts Yousef
itt al. (2011), whereas recent approaches employ vision-based tactile sensors Yuan et al. (0117);
Lambeta al. (2020): (loyd & Lepora (0024). These sensors provide high-resolution data useful in complex tasks, including shape reconstruction, texture recognition, and advanced manipulation, such as autonomous page turning Zheng et al. (2022), object reorientation Yin al.
(2023): Qi et al. (2023). scraping Chebotar et al. (2014), and handling deformable objects [Bauer et al. (2025) Active tactile perception involves deliberately selecting contact locations and trajectories to optimize information gain during interaction. Previous approaches leverage Gaussian Processes Yi et al. (2016); B Björkman cct al. (2013) and Bayesian optimization Dragiev et al. (2013): De Farias et al. (2021); Bochm et al. (2024) to efficiently reconstruct shapes, discriminate textures, or identilyy grasp points. However, these approaches typically rely only on sparse contact points, assume the object is stationary, and often require additional sensing modalities such as vision Björkman et al. (2013). Moreover, most works are tailored to specific tasks. Gaussian process-based methods, such as et al. (2016); ::6:;Bija Björkman jet al. (2013); Dragiev et al. (2013), focus on shape reconstruction, while Boehhm et al. (2024) performs texture recognition using vision vision-based tactile. For comprehensive surveys on tactile manipulation, we refer readers to Li al. (2020); Yousef et al. (2011).
Reinforcement Learning in Active Perception: In active perception, previous works have explored RL-based approaches. In the visual domain, Yang al. (2019) performs active object search and Tan et al. (2020) generates semantic scene annotations, both use ININORC to train camera-contre policies that improve perception. Related camera-com controll methods include Cheng et al.) (2018). who visual input, Dass et al. 20225 who recover a known context variable from visual observations using PPO without memory. Hu et al.
apply RL to manipulation task using
(2025) focuses real-world, rision-based active perception and proposes RL training recipe that uses privileged sensing and demonstrations to learn deployable viewpoint-selcction policies. Active perception for 3DD scene understanding has also been studied. Dayaraman & Grauman (018))
learn LSTTM-hased policies for actively completing panoramic scenes, while Lv et al. (2023) uses a differentiable simulator select informative viewpoints with attention to sim-to-real transfer. Other works jointly model motor and sensor policies under partial observability, as in Shang & Ryoo (2023).
or integrate point-cloud conditioning and distillation for mobile manipulation Uppal al. (2024).
In the tactile domain, RL-based methods have addressed object shape reconstruction. Particularly, PPO et al. (2022) and DDQN Smith et al. (202I) have been applied to build object shape estimates by selecting informative contact points, with the former assuming a binary tactile sensor in a 2D environment and the latter requiring both vision and touch. By assuming that the object is static, these approaches avoid uncertainty in object pose and maintain explicit shape reconstructions that serve as the policy state. Complementing these, Rajeswar et al. (0022) propose curiosity-driven haptic exploration based on mismatches between visual predictions and tactile observations. Related to our work is the Recurrent Models of Visual Attention (RAM) Mnih et al.] (2014), which uses REINFORCE to select sequential image glimpses for MNIST classification via an LSTM policy. Although developed for classification, the idea naturally extends to regression. Building on RAM, Fleer al. (2020) introduce the Haptic Attention Model (HAM), which learns control policy
3

---

Published as a conference paper at ICLR 2026
Trandforma mer
Figure 2: Active perception process in APPLE. framework. this task, the agent's 's goal is to classify the digit using touch alone. At each step, it receives a tactile reading and state information (e.g., sensor position). A Vision Transformer encodes the tactile input, which is concatenated with state data and processed as sequence over time by transformer. At every step, the model outputs label prediction yı. evaluated against the ground truth via loss function l, and action at that controls the sensor's next movement.
for a taxel-based tactile sensor to classify four static objects, though training requires millions of interactions. Niemann et al. (2024) extends this with binary "done" action, again evaluating tactile classification.
Finally, our classification experiments fit within the broader family of internally rewarded RL (IRRL) methods formalized by Li et al.] (2023), which use internal discriminators to produce rewards defined mutual information between trajectories and labels. While our rewards also depend the agent's own prediction model, our framework is more general, allowing any differentiable loss rather than targeting mutual information specifically.
3 ACTIVE PERCEPTION POLICY LEARNING
Our objective in this work is to develop an active perception method that, unlike prior approaches, is tied a particular task or environment. Our guiding principle here is similar RL. That is, just as RL requires only reward function, we want to specify a perception objective and let the agent learn an appropriate perception policy on its own, without imposing strong task sask-spe assumptions. On a high level, we frame active perception minimize loss E(jr, y/.) between its current prediction ÿt and the ground-truth label yr. However, unlike in classical supervised learning, we assume that the agent is not simply presented with static data point as input, but rather with an interactive environment that it can actively gather data from. E.g.. the agent could be presented with an object and has to decide actively how to examine to extract the information it needs. This perspective defines active perception fundamentally as a sequential recision-making problem embedded within supervised learning problem. An example of this process is given in Fig. 2 Here, the agent is faced with a classification task, where it must identify a digit from touch alone. Hence, in every step, the agent chooses where move the sensor while also making a prediction about the class label. The agent's 's objective is to minimize the loss function throughout this process (illustrated in the bottom row of the Fig. Thus, it must optimize its actions to be as informative as possible.
supervised learning problem. The agent's goal is to
In the remainder of this section, we formally define our problem and derive the Active Perception Policy Learning (APPLE) framework. We propose two variants of APPLE, based on SAC Haarnoja et al. 2018) and CrossQ Bhatt et al. (2019).
3.1 PROBLEM STATEMENT
Formally, we define the problem of active perception a special case of a Partially Observable Markov Decision Process (POMDP). Here, the environment is governed by unknown dynamics p( (h: [h, āz), where h, is the hidden environment state and ã, is the action taken at time t. The agent then makes observations through the distribution, ph), where o, is the observation. In the active perception scenario, the agent's objective is to learn a particular property of the environment, e.g.. the class or pose of an object. We assume that the ground truth value y of this property at time t is part of the hidden state h, and thus not directly accessible to the agent. Hence, the hidden state decomposes into hy (14,ūe). where h2 is the remainder of the hidden state without the ground
4

---

Published as a conference paper at ICLR 2026
truth property value. Additionally, the agent's action space contains not only control actions a, but also a current estimate Y/s of the desired environment property. In other words, the action space decomposes into â (a,, yr), mcaning that the agent predicts the desired environment property at every step. As is typical for RI., the action ae is a control signal, e.g., a desired finger movement, which is communicated to the agent's motor controllers.
The overall reward function i consists of two parts: a differentiable prediction loss l and an RL reward That is, F(he; Wr: at, Ut) r(h,; a+) 10-(( -((),. Here, the prediction loss, E(ve: yr). could, e.g, be cross-entropy loss in the case of classification task or the Euclidean distance in the case of a pose estimation task. The RL reward, r(he, at), does not have to be differentiable or known to the agent. In this work, we only use it to regularize the agent's 's actions az. In the following, to simplify the notation, we denote p h. y, a. = o) p h. o, a). ग <Ty (yyo)) = I== (3e oox)) and p h, y, a) pl p(hoo ho, vo) IT-o p(ox) he) x (a, ooo)) p(her [ ax) pl Un hz) where h h0:00 y = Vo..0 and so on.
The objective is now to find policy π Taael 00:) for which the expected discounted return is maximized. That is, given the discount factor 7 E [0, 1).
max (T) == E p (I....... x
3.2 OPTIMIZING THE ACTIVE PERCEPTION OBJECTIVE
Let l (We: 0o:) == Eu ( states, we can rewrite Eq. ①1 as
((. m.).
Since the agent's predictions ye do not influence future
oo Σ a 1=
J(x)= B P (h.....))
(2)
In this work, assume that the policy is neural network, parameterized by parameters 0 e RRM, which allows us to compute gradient of Eq. (2) and optimize the problem with gradient descent algorithms, Computing the gradient of J J(TT) now yields
00 (TTO = (o.
In (a o) "F (e 08
(o( Σn' P (2) m0x)
(3)
policy gradient
prediction loss gradient
Intermediate steps can be found in Appendix G
As can be seen in Eq. 3 the gradient of the objective
function J (TT)) decomposes into policy gradient and a negative supervised prediction loss gradient.
3.3 DERIVING APPLE-SAC AND APPLE-CROSSE
We use RL-based techniques to estimate the policy gradient in Eq. (3). Here we propose two variations of APPLE based on two actor-critic methods: SAC Haarnoja al. (2018) and Cross Bhatt et al.) 2019). SAC is an off off-policy RL algorithm that jointly learns a policy and two Q-networks. The Q-networks are optimized to minimize the Bellman residual for the policy, and the policy is optimized to maximize the smaller of the two values predicted by the Q-networks. To stabilize the training of the Q-networks, SAC deploys target networks, which slowly track the actual Q-networks. Cross@ SQ similar to SAC but drops the target networks and instead uses BatchRenorm Toffe (207)) layers in the Q-network to stabilize their training. To use either SAC and Cross in the active perception setting, we adjust three components:
1. The active perception setting is partially observed. Hence, instead of state 8t, the policy and the Q-networks receive a trajectory of past observations Oze-
2. During the training of the Q-networks, SAC and CrossQ sample transitions and rewards from the replay buffer to compute the Bellman residual. The presence of the prediction loss énp requires us to dynamically recompute the total reward when evaluating the Bellman residual, yielding
Laritie ED 1 (Q0(oaz az) (r2- Lup (Ue, o0x) + 7/ L6 (05(0oz-- az-1))) 10
5

---

Published as a conference paper at ICLR 2026
3. During the policy update, the policy gradient is augmented by the prediction loss gradient.
In the following, we denote those APPLE variants as APPLE-SAC and APPLE-CrosSQ.
3.4 INPUT PROCESSING
We assume that the sequence of past observations con- sists of both images and scalar data. tactile-based active perception, image observations typically correspond to high- dimensional tactile inputs (represented as images), while the scalar data encodes the position of the sensor. As illustrated in Fig. 2 the agent receives this exact combination of tac- tile images and sensor positions. To efficiently process this data in the policy and Q-networks, we use an architecture similar the Video- Video-Vision-
Cire- leSquare
TactileM- NIST
(ViViT) Amab al. (2021) architecture. First, Vision Transformer (ViT) Doso- vitskiy al. (202T) is used to generate embeddings for each tactile image. These embeddings are then concatenated with the scalar inputs and processed by a transformer to generate an embedding ma for each time step. We empirically found that sharing these embeddings across Q-networks Qo(=s at); action policy ae(a; 0u:r) and prediction policy (ye oa:) yields better results than training individual representations for each of these components.
TactileMNIST- Volume
Toolbox
Figure 3: Active perception benchmarks on which we evaluate our method. Tac- tileMNIST, TactileMNISTT Toolbox are tactile perception tasks from the Tactile MNIST Benchmark Suite [Schneider al. (025)) where the agent must decide how to gather in- formation with its tactile sensor. Cire- leSquare and TNNSI are classifi- cation tasks and TactileMNNISTVolumee is regression task, where the agent must determine an object's 's volume. Toolbox is pose estimation task, where the agent must determine the 2D pose of the object. All tasks require the agent gather in- formation actively and are not accurately solvable via random exploration.
and
4 EXPERIMENTS
With APPLE, our goal is to answer two questions: (1) can we design a general and principled RL-based algorithm that successfully discovers aetive-perceplee a task label and a differentiable loss during training? and (2) can such an approach extend across diverse active-pert perieeeeeceetion problems, including both classification to regression without the need to over-design task-specific exploration heuristics? To answer these questions, we focus mainly on the tactile domain, which is particularly suited for active perception due to the local and sparse nature of touch. Thus, we run experiments that span different observation spaces (such low-dimensional image arrays and higher-dimenssonal tactile images) and varied downstream prediction goals (including classification and regression). Here, our four evaluated active perception tasks are: CircleSquare, TactileMNIST, TactileMNISTTollue
policies using only
and Toolbox introduced by /Schneider et al. (2025) (see Fig. (3). We also compare our approach against the MHSB tactile shape classification task from Fleer et al. (2020). In each task, the agent must actively gather information and jointly learn both policy and prediction model. All experiments described in this section are run with 5 random seeds per method, with all models trained from scratch for each seed. In addition to our core configurations, APPLE-CrOSSQ and APPLE-SAC, we also evaluate the following baselines:
(i) APPLE-RND: a random policy baseline sharing the same configuration as APPLE-SAC, but not optimizing an action policy. Instead, actions are sampled uniformly at random throughout training. Importantly, while the policy remains random, the perception module is still trained, enabling the model to learn how to interpret tactile observations even without control over the spatial allocation of its haptic glances.
(ii) HAM: the Haptic Attention Model introduced by Fleer et al. (020). HAM employs a recurrent neural network (LSTM) integrate tactile observations over time and jointly learns to classify objects while optimizing its exploratory actions through REINFORCE.
For evaluation the classification tasks, report two complementary metrics. The average class prediction accuracy considers the agent's predictions at all steps of an episode and computes the average accuracy across them. The final class prediction accuracy only uses the prediction from the

---

Published as a conference paper at ICLR 2026
MHHS
CircleSquare
TactileNNIST
Tactil:MNISTTVolmm
1
Avg. (cm²
Avg. NCC.
0.8
1.2
0.6
0.4
Fin. (cm²)
1.4
Fin. aec.
0.8
1.2
0.6
0.4
0.5
0.1
0.2
0
2
0
2
1
APPLESSCC
APPSS-CrOsSQ
APPLE-RND
HAM
Steps
-10
Figure Average and final prediction accuracies for our methods APPLE-SAC and APPLE-CrOSSQ. HAM Fleerr et al. (2020), and APPLE-RND across various tasks. MHSB refers to the tactile classification task used in Fleer et al. (2020). All methods were trained with 5 seeds. Shaded areas represent one standard deviation. Metries computed evaluation tasks with unseen objects, except for CireleSquare and the MHSB classification task, which have only two four, respectively.
final step of each episode, thus showing the model's decision after completing its exploratory sequence. For regression tasks, we report average error and final error. The hyperparameters of all methods are tuned with HEBO Nowen-Rivers et al. (2022). For all methods except set of hyperparameters, tuned on the TactileMNIST task, for all experiments involving visual tactile inputs and a different set, tuned on the CircleSquare task, for other tasks. For APPLE-Crosse we use the same hyperparameters, tuned on the TactileMNIST task, for all experiments.
we use one
4.1 THE MHSB CLASSIFICATION TASK
To enable more direct comparison between our method and HAM, we use the benchmark task provided in Ln/Feer al.] (2020). The dataset for this benchmark is generated in Gazebo and consists of data from a haptic classification task where a set of four blocks from the Modular Haptic Stimulus Board (MHSB) were arranged in the simulated environment. Each block represents a distinct local shape feature, and data was collected using simulated Myrmex tactile sensor array that produces 16x 16 pressure images upon contact with the block surface. Here, each data point d E 'D consists of the tuple d (z, sp, p), where [-1, 1] is the location of the sensor, o [- 7 the angle and p the corresponding normalized pressure (image) array. The agent's 's goal is to identify the correct object, so each time step it selects continuous movement actions de (æ,, sp). The closest associated datapoint de 'D is then selected and the agent makes a prediction. Each episode allows the agent to perform 10 touches. For extra details, please refer to Fleer al. 2020).
In this setting, we evaluate APPLE-RND, APPLE-SAC. and CrosssC with the vision encoder removed to ensure comparability with HAM. The first column of Fig shows the comparison between the different methods on this task. While HAM is generally able to solve this task (see Appendix C.5 for longer run), it requires a large amount of samples
learn an effective policy. After 1M training steps, HAM achieves only final accuracy of 68%, while all of our approaches, including the random baseline, approach 100% after around 250K steps. Additionally, the good performance of the APPLE--ND baseline highlights limitation of this task: due to the discretization of position and angles into 41 bins each, the observation space contains only total of 1,681 distinct values per class. Agents can quickly learn memorize all of these values and then solve this task perfectly with just a few random touches. This insight raises questions about HAM's capabilities in learning active perception policies, HAM was only evaluated on this task in Fleer et al. (2020).
4.2 THE CIRCLESQUARE TASK
The CircleSquare task has the goal of evaluating active perception a low-dimensional space. Here, the agent is presented with 28x28 grayscale image containing either white circle square placed randomly in the field (Fig. B left)..t). Its goal is to identify the correct object class, but only observe 5x5 glimpse time and must explore the image over time. Each episode
7

---

Published as a conference paper at ICLR 2026
allows the agent to take up 16 steps. A color gradient offers directional guidance, but the agent starts without information about the object's location. selects a continuous movement action a, e [-1,1] 1 per step to reposition its glimpse, encouraging learned search strategies over random behavior. Since the agent's glance in this task is small, we again do not use our vision encoder but treat the inputs a flat representation of the input image. In this setting, in addition to the HAM baseline, we also compare our APPLE SAC and APPLE CrossQ to the random baseline agent, APPLE -RN.. As shown in Fig. 4 APP PEPLLSSAAA SAC and APPLE-Crosse learn to complete the task with similar final prediction accuracies of 97% and 96%, respectively. For a visualization of policy learned by APPLE-Crosshe refer to Fig. 8 APPLE achieves 68% accuracy, highlighting the need for active perception the CireleSquare task. Despite extensive tuning the learning rate and parameter with HEBO and training for 10M steps, we could not find configuration for which HAM reached a prediction accuracy beyond random guessing. For full implementation details, including an ablation with longer training times, where we compare HAM with a PPO baseline, see Appendix B.1
25-dimensional vector for better comparability to HAM, which also processes
4.3 TACTILEMNIST CLASSIFICATION
0.8
Label Probability
While the CircleSquare environment already presents non-trivial active perception problem for more generic agents, it remains relatively simple: the input space contains just 25 pixels, there are only two object classes, with a color gradient providing search direction. In contrast, (visual-) tactile perception add more complexity. First, the input is high-dimensional image requiring encoding into latent space. Second, real-world classification tasks often involve many classes with diverse instances. E.g., robot sorting waste into plastics, glass, and metal must handle objects of various shapes and textures that belong to the same
0.6
0.4
0.2
0.8
Accurscy
0.6
class. Finally, tactile exploration often lacks directional cues, as retrieving an object from a cluttered bag requires systematic search strategies. To investigate this, we evaluate our methods on the TacSileMNIST classification. Here, an agent uses GelSight Mini sensor to explore a randomly placed and oriented 3D MNIST digit without prior location knowledge. The goal is to classify the digit within a fixed time budget (see Fig. 3 top-right for a visualization). In this setting, and compare them with APPLE RND. As this task requires vision encoder, direct comparison with HAM is not possible.
AWLLSOWW
5
15
Glances
Figure 5: Exploration efficiency of final policies on the TNST task. Shown are the predicted prob- ability of the correct label (top) and accuracy (bottom) after N glances.
evaluate APPLE-SAC and APPLE-Cross:
As shown in Fig. 4 both APPLE-SSAC and APPLE-CrOSSQ reach similar high final prediction accuracies of 87% and 89% on the evaluation task. APPLE-RND RND, however, eventually stagnates at an accuracy of around 74%, highlighting the importance of action selection on this task. The average class prediction accuracy, as shown in Fig. 4 (first column), presents similar trend, with APPLE-SAC achieving 80% and APPLE-Cross@ 81%. An additional insight into the agent's s performance is given by Fig. 5 which shows how the accuracy and correct label probability of the trained agents over the course of an episode, averaged over multiple episodes. This figure shows that the active agents gather information much quicker and are much more certain about the class label than the random agent. For more details this task. see Appendix B.2
4.4 TACTILEMNIST VOLUME ESTIMATION
In the Tactile task, the agent again uses GelSight Mini sensor to explore randomly placed and oriented 3D MNIST digit without prior location knowledge. Unlike the prior classification- based tasks, the objective here is to estimate the digit's 's volume within a fixed time budget, which makes this task a regression task. A visualization of the TactileMNISTVolume task can be seen in Fig. on the bottom left. To succeed, the agent must simultaneously localize the digit on the workspace and gather sufficient shape information cues to enable accurate volume estimation. For more details about this task, refer Appendix B.3
8

---

Published as a conference paper at ICLR 2026
In this setting, we evaluate APPLE-SAC APPLE-CrOSSO. and APPLE-RND, analyzing their ability to perform regression tasks. In the last column of Fig. 4 we show the comparison results for this task. The final average prediction error throughout the task reaches 1.28 cm for APPLE-RND. APPLE-CrOSSQ and APPLE-SAC, reach an average prediction error of 1.10 cm2 and 1.16 cm², respectively. For the final enror APPLE-SAC, with 0.99 cm² outperforms both APPLE-RND with an error of 1. 1.07 cm and APPLE- Crosso with 1.05 cm² The higher variances and performance fluctuations in Fig. indicate that this task is more challenging than the other tasks. In part, these results might be explained by the fact that neither method was tuned on regression task. Additionally, estimating the volume of an unknown object requires much more complete information about its shape than classifying it, making both the perception and policy optimization more challenging.
4.5 TOOLBOX
Average Error
Final Error
In the Toolbox task, the agent has to find a wrench on platform by touch alone. Like Tac- tileMNiITVollue, Toolbox is a regression task, but here, the agent has to predict the 2D posi- tion and ID orientation of the wrench in the workspace. As such, the task consists of two problems: the agent must find the wrench in the workspace and determine its pose. Crucially, as can be seen in Fig. 3 bottom-right, most parts of the wrench are ambiguous location when touched, and the information of multiple touches has to be combined make an accurate predic- tion. For example, when touching the handle, the agent may extract information about the lat- eral position of the wrench, but does not yet know where it is currently touching the handle longitudinally, and whether the open end is left or right. Hence, to disambiguate the wrench pose, strong exploration strategy must include finding one of its ends. Similar to the previous two tasks, the agent moves a GelSight Mini sensor freely in 2D space, constrained by the platform boundaries. Since the platform for this task is larger than that for the other tasks, we allow 64 steps for exploration before the episode is terminated.
Linear [cm]
2
Angular [dag]
60
40
20
0
5
10
10
APPLE-SAC
Steps
-109
AFFLEDAND
Figure 6: Average and final prediction accuracics for our methods APPLE-SAC and LE-Corsss.. as well as the baseline APPLE-RND on the Toolbox task. Each method was trained on 5 seeds for 10M steps.
The results in Fig. 6 show that APPLE-CrossO reaches final accuracy of 1.9cm and 13 on aver- age, while APP LE SAC and the random baseline APPLE-RNU stagnate at much lower accuracies. Throughout the training, APPLE-CrossQ learned a sensible exploration strategy, comprised of finding the handle of the wrench and then sliding along it to disambiguate its orientation (see Fig. [2). The low performance of the APPLE- RND again highlights the importance active perception for this task. It is important to note that we again did not tune any of these methods on this task and instead relied on hyperparameters optimized for MaNSI. While tuning APPLE SAC on this task can lead to stronger performance, these results indicate that APPLE-CrosSQ might be more robust w.r.t the choice of byperparameters. However, more experiments are needed to answer this question definitively. For training and evaluation details, and hyperparameter settings, see Appendix B.4
DISCUSSION
The results in Section 4 show that APPLE successfully learns exploration strategies across diverse active perception tasks. We evaluated APPLE on the CircleSquare classification task and three simulated tactile benchmarks: TactileMNIST (digit classification), TactileMNIISTVolmm (volume estimation), and Toolbox (pose estimation). In all three tactile benchmarks, the agent must learn an image encoder and refine its exploration policy jointly. The poor performance of the APPLE-RND baseline across our tasks confirms the necessity of structured exploration and confirms that methods learned policies that go beyond random exploration. Notably, APPLE-Cres high performance when switching tasks without hyperparameter tuning, highlighting its robustness.
retained

---

Published as a conference paper at ICLR 2026
We compare method to HAM et al. (2020), which failed to learn an effective strategy even on the Circle-Sques Square toy task, defaulting to predicting the mean class despite extensive tuning using HEBO. This points to fundamental limitation of HAM in this setting, namely that HAM relies on on-policy RI., which discards samples after a single update, limiting sample efficiency. Further exper- iments using a PPO-based variant of our method (see Fig. [1] underline our hypothesis that on-policy RL is not suited for active perception. In contrast, we employ off-policy methods (APPLE SAC and APPLE-Cros
enabling sample reuse- a critical factor in active perception, where supervised learning benefits from multiple passes over the same data. Both APP SAC and APPLE-CrosS perform comparably on environments they have been tuned on, but APPLE-Crosse has proven more robust to new environments without hyperparameter tuning and offers a clear computational advantage. By avoiding target network updates, it requires roughly half the transformer forward passes during training, leading to 53% reduction in training time on average, without sacrificing performance. The strong performance of APPLE, especially of APPLE-Crosse without retuning, demonstrates its potential
robust, general framework for active perception.
Particularly, in Fig. S and Fig. 12 we show the learned behaviors for our policies in the CircleSquare and Toolbox environments, respectively. In both environments, we can observe that agents have learned interesting exploration behaviors that ultimately help in solving the task. In CircleSquare, APPLE first moves rapidly along the background gradient, which reliably leads toward the object's location. Once it reaches the circle or square, the agent stabilizes its position and keeps the glimpse above the object for the remainder of the episode. More interestingly, in the Toolbox environment, learns to both locate the wrench and disambiguate its orientation. At the beginning of the episode, the agent learns circular search pattern that efficiently sweeps the workspace. Then, when first encountering the wrench handle, it transitions into sliding motion along the handle's 's length. While these patterns are intuitive from a human perspective, for the TactileMNIST and MactileMNIISTollune the strategies are less clear. In those, the agent typically moves toward the center of the platform early in the episode, as most objects extend into the central region. After initial contact, it often follows local edges strokes of the digit. but such behaviors vary considerably across digits and sceds. In many cases, the final contacts do not follow a recognizable pattern, suggesting that, in this case, the agent relies more on aggregating many local cues. Overall, these emergent behaviors highlight that APPLE can learn different active perception policies, thus adapting its exploration behavior to the demands of each environment with minimal changes to the setup, suggesting a promising step toward more general active perception frameworks.
6 CONCLUSION
We introduced APPLE, a framework combining reinforcement learning and transformer-based mod- els for active tactile perception. We evaluated it on five benchmarks where APPLE consistently outperforms its baselines, underscoring its efficiency in learning exploration policies. Notably. our method requires no hand-crafted heuristics and learns exploration policies in a principled way by minimizing the prediction loss function. The current state-of-the-ar method for tactile classification HAM - cannot solve any of these tasks beyond the MHSB task, which was originally developed for. On the Circle-Square task, HAM resorted to always predicting a 50/50 probability for both labels, even after long training, while the other three tasks require an image encoder, which HAM lacks. APPLE, on the other hand, achieves high performance across all tasks, with the exception of APPLE- SAC on the Toolbox task, where was not tuned on. These results demonstrate APPLE s versatility across diverse tasks, including both classification and regression. Future work will focus on improving the sample efficiency of APPLE, extending to more realistic applications such as in-hand pose estimation and texture recognition, and exploring multi-modal integration with vision and touch. Another poten- tial future research avenue will be deploying APP LE on a real robotic system. While, in principle, APPLE could be applied real-world tasks as-is, its sample efficiency poses a practical challenge. In other works, sample efficiency issues of RL methods have been addressed through large-scale domain randomization and sim-to-real transfer [Bohlinger et al./ (2024); Akkaya et al. (2019): Handa et al. (203)). However, their soft gel makes vision-based tactile sensors particularly hard to simulate. Nevertheless, using sim-to-real transfer jointly with accurate soft-body tactile simulation Nguyen et al. (2024) and transferrable representations Yang et al. (0024a) or realistic rendering Chen et al. (2022) are promising avenues towards the application of APPLE to real-world tasks.
transfer[Bohlinger
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported by the German Federal Ministry of Education and Research (BMBF) and the French Research Agency, I' l'Agence Nationale de Recherche (ANR), through the project Aristoule (Grant no.: ANR-21-FAII -0009-000,,--)). through the EU's Horizon Europe project ARISE (Grant no.: 101 135959), and through the BMBF's projects Robotics Institute Germany (RIG) (Grant no.: 16ME1001), DEMETER (Grant no.: 01DR25003), and Genius Robot (Grant no.: 011S24083).
Furthermore, this work is also supported by the German Research Foundation (DFG, Deutsche Forschungsgemeeinccaatt) as part of Germany's 's Excellence Strategy EXC 2050/1 Project ID 390696704 Cluster of Excellence Centre for Tactile Internet with Human-in-the-tt Technische Universität Dresden by the BMBF, and by the German Academic Exchange Service (DAAD) in project 57616814 14 (chool of Embedded and Composite Al), The computations were conducted on the IAS Compute Cluster.
(CeTTT) of
REPRODUCIBILITY STATEMENT
We took special care make this work reproducible. The link to the source code can be found on the project page http:://inccccedddee.egguub.bbop
and further implementation details are provided the supplementary material. For transparency, all hyperparameters implementation details, including GPU specifications and memory considerations, are provided in Appendix F
listed in Appendix E and
LARGE LANGUAGE MODEL USAGE
A large language model (LL.)) was used solely for language editing polishing phrasing, enhancing readability, and correcting minor typographical errors.
REFERENCES
Ilge Akkaya, Marcin Andrychowicz. Maciek Chociej, Mateusz Litwin, Bob McGrew, Arthur Petron, Alex Paino, Matthias Plappert, Glenn Powell, Raphael Ribas, et al. Solving rubik's cube with robot hand. arXiv preprint arXiv:1910.0711 2019.
Anurag Arnab, Mostafa Dehghani, Georg Heigold, Chen Sun, Mario Lučic, and Cordelia Schmid. Vivit: A video vision transformer In Proceedings of the IEEE/CVF international conference on computer vision, pp. 6836 6846. 2021.
R. Bajesy. Active perception. Proceedings of the IEEE, 76(8):966-1005, 1988. doi: 10.1109/5.596.
Ruzena Bajesy, Yiannis Aloimonos, and John K Tsotsos. Revisiting active perception. Autonomous Robots, 47 2018.
Luca Bartolomei, Lucas Teixeira, and Margarita Chli. Semantic-aware active perception for uavs using deep reinforcement learning. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3101-3108, 2021. doi: 10.1109/RROS1 168 2021 .9635993.
Dominik Bauer, Zhenjia Xu, and Shuran Song. Doughnet: A visual predictive model for topological manipulation of deformable objects In European Conference on Computer Vision, pp. 128108 Springer, 2025.
Aditya Bhatt, Daniel Palenicek, Boris Belousov, Max Argus, Artemij Amiranashvili, Thomas Brox, and Jan Peters. CrossQ: Batch Normalization in Deep Reinforcement Learning for Greater Sample Efficiency and Simplicity In The Twelfth International Conference on Learning Representations, 2019.
Marten Björkman, Yasemin Bekiroglu, Virgile Högman, and Danica Kragic. Enhancing visual perception of shape through tactile glances) In 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3180-3186, IEEE, 2013.
11

---

Published as a conference paper at ICLR 2026
A. Boehm, T. Schneider, B. Belousov, A. Kshirsagar, L. Lin, K. Doerschner, K. Drewing, C.A. Rothkopf, and J. Peters. What Matters for Active Texture Recognition With Vision Booon- Tactile Sensors In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2024.
Jeannette Bohg, Karol Hausman, Bharath Sankaran, Oliver Brock, Danica Kragic, Stefan Schaal, and Gaurav S Sukhatme. Interactive perception: Leveraging action in perception and perception in action. IEEE Transactions Robotics, 33(6):1733-129) 2017.
Nico Bohlinger, Grzegorz Czechmanowski, Maciej Krupka, Piotr Kicki, Krzysztof Walas, Jan Peters, and Davide Tateo. One policy run them all: an end-to-end learning approach to multi- embodiment locomotion. Conference on Robot Learning (CoRL), 2024.
James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman- and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018. URL
http:/ /ttp://gtthbbb/t commjaxx-mmjaa
Yuri Burda, Harrison Edwards, Amos Storkey, and Oleg Klimov. Exploration by random network distillation. In Seventh International Conference on Learning Representations, pp. 1-17, 2019.
Yevgen Chebotar, Oliver Kroemer, and Jan Peters. Learning robot tactile sensing for object ma- nipulation. 2014 IEEE/SS International Conference on Intelligent Robots and Systems, pp. 1368-3375. IEEE, 2014.
Weihang Chen, Yuan Xu, Zhenyang Chen, Peiyu Zeng, Renjun Dang, Rui Chen, and Jing Xu. Bidirectional sim-to-real transfer for gelsight tactile sensors with cyclegan. IEEE Robotics and Automation Letters, 7(3):6187-6194, 2022.
Ricson Cheng, Arpit Agarwal, and Katerina Fragkiadaki. Reinforcement learning of active vision for manipulating objects under occlusions. In Conference on robot learning, pp. 42244222444 431. PMLR, 2018.
Ian Chuang, Andrew Lee, Dechen Gao, M-Mahdi Naddaf-Sh, and Iman Soltani. Active vision might be all you need: Exploring active vision in bimanual robotic manipulation. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pp. 7952 7959.7959. IEEE, 2025.
Alexander Cowen-Rivers Wenlong Lyu, Rasul Tutunov, Zhi Wang, Antoine Grosnit, Ryan Rhys Griffiths, Alexandre Max Maraval, Hao Jianye, Jun Wang, Jan Peters, al. Hebo: Pushing the
limits of sample-efficient hyper-parameter optimisation Journal of Artificial Intelligence Research, 74:1269-1349 2022.
Xu-Yang Dai, Qing-Hao Meng, Sheng Jin, and Yin-Bo Liu. Camera view planning based on generative adversarial imitation learning in indoor active exploration. Applied soft computing, 129: 109621, 2022.
Shivin Dass, Jiaheng Hu, Ben Abbatematteo, Peter Stone, and Roberto Martín-Martin Learning to look: Seeking information for decision making via policy factorization. In Conference on Robot Learning, pp. 4425-4445 PMLR, 2025.
Cristiana De Farias, Naresh Marturi, Rustam Stolkin, and Yasemin Bekiroglu. Simultaneous tactile exploration and grasp refinement for unknown objects IEEE Robotics and Automation Letters, (2):3349-3356, 2021.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021.
Stanimir Dragiev, Mare Toussaint, and Michael Gienger. Uncertainty aware grasping and tactile exploration, In 2013 IEEE International conference on robotics and automation, pp. 113-119. IEEE, 2013.
12

---

Published as a conference paper at ICLR 2026
Sascha Fleer, Alexandra Moringen, Roberta L Klatzky, and Helge Ritter. Learning efficient haptic shape exploration with rigid tactile sensor array PloS one, 15(1)=e0226880, 2020.
Konstantinos Gounis, Nikolaos Passalis, and Anastasios Tefas. Uav active perception and motion control for improving navigation using low-cost sensors. arXiv preprint wwXiv:2407.15122 2024.
Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor In International conference
machine learning. pp. 1861-1870. PMLR, 2018.
Ankur Handa, Arthur Allshire, Viktor Makoviychuk, Aleksei Petrenko, Ritvik Singh, Jingzhou Liu, Denys Makoviichuk. Karl Van Wyk, Alexander Zhurkevich, Balakumar Sundaralingam, al. Dextreme: Transfer of agile in-hand manipulation from simulation to reality. In 2023 IEEE International Conference Robotics and Automation (ICRA), pp. 5977-5984, IEEE, 2023.
Jonathan Heek, Anselm Levskaya, Avital Oliver, Marvin Ritter, Bertrand Rondepierre, Andreas Steiner, and Mare van Zee. Flax: A neural network library and ecosystem for JAX, 2024. URL
htttp://githbe
flax
Edward S Hu, Jie Wang, Xingfang Yuan, Fiona Luo, Muyao Li, Gaspard Lambrechts, Olch Rybkin, and Dinesh Jayaraman. Real-World Reinforcement Learning of Active Perception Behaviors Advances in Neural Information Processing Systems, 2025.
Sergey loffe. Batch renormalization: Towards reducing minibatch dependence in saich-normalized models Advances in neural information processing systems, 30, 2017.
Dinesh Jayaraman and Kristen Grauman. Learning to look around: Intelligently exploring unseen environments for unknown tasks. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 1238-1247, 2018.
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. Master's thesis, Toronto, ON, Canada, 2009.
Mike Lambeta, Po-Wei Chou, Stephen Tian, Brian Yang, Benjamin Maloon, Victoria Rose Most, Dave Stroud, Raymond Santos, Ahmad Byagowi, Gregg Kammerer, et al. Digit: A novel design for a low-cost compact high-resolution tactile sensor with application to in-hand manipulation IEEE Robotics and Automation Letters, 5(3)38838 3883-3845, 2020.
Hojoon Lee, Dongyoon Hwang, Donghu Kim, Hyunseung Kim, Jun Jet Tai, Kaushik Subramanian, Peter R Wurman, Jaegul Choo, Peter Stone, and Takuma Seno. Simba: Simplicity bias for scaling up parameters in deep reinforcement learning. In 13th International Conference Learning Representations, ICLR 2025, pp. 50050-5000 50082. International Conference on Learning Representations, ICLR, 2025.
Mengdi Li, Xufeng Zhao, Jae Hee Lee, Comelius Weber, and Stefan Wermter. Internally rewarded reinforcement learning. In International Conference on Machine Learning, pp. 20556-20574. PMLR, 2023.
Qiang Li, Oliver Kroemer, Zhe Su, Filipe Fernandes Veiga, Mohsen Kaboli, and Helge Joachim Ritter. A review of tactile information: Perception and action through touch IEEE Transactions Robotics, 36(6):161911644 2020.
Shaopeng Liu, Chao Huang, and Hailong Huang. Behavior cloning- based active scene recognition via generated expert data with revision and prediction for domestic robots. IEEE Transactions on Robotics, 2025.
John Lloyd and Nathan F Lepora. fose-and-shean based tactile servoing The International Journal of Robotics Research, 43(7): 2024.
Jun Lv, Yunhai Feng, Cheng Zhang, Shuang Zhao, Lin Shao, and Cewu Lu. Sam-rl: Sensing- aware model-based based reinforcement learning via differentiable physics- based simulation and rendering. In Robotics science and systems, 2023.
13

---

Published as a conference paper at ICLR 2026
Matheus G Mateus, Ricardo B Grando, and Paulo LJ Drews. Active perception applied to unmanned aerial vehicles through deep reinforcement learning. In 2022 Latin American Robotics Sympo- sium (LARS), 2022 Brazilian Symposium on Robotics (SBR), and 2022 Workshop Robotics in Education (WRE), pp. 1-6. IEEE, 2022.
Volodymyr Mnih, Nicolas Heess, Alex Graves, et al. Recurrent models of visual attention Advances in neural information processing systems, 27, 2014.
Michal Nauman, Mateusz Ostaszewski, Krzysztof Jankowski, Piotr Miłos, and Marek Cygan. Bigger Regularized, Optimistic: scaling for compute and sample-efficient continuous control In Advances in neural information processing systems, 2024.
Duc Huy Nguyen, Guillaume Duret, Tim Schneider, Alap Kshirsagar, Boris Belousov, and Jan Peters. Tacex: Gelsight tactile simulation in isaac sim-combining boft-body and visuotactile simulators. In CoRL Workshop Learning Robot Fine and Dexterous Manipulation: Perception and Control, 2024.
Christopher Niemann, David Leins, Luca Lach, and Robert Haschke. Learning when to stop: Efficient active tactile perception with deep reinforcement learning. Conference on Intelligent Robots and Systems (IROS), pp. 685-692. IEEE, 2024.
2024 IEEE/RS] International
Deepak Pathak, Pulkit Agrawal, Alexei A Efros, and Trevor Darrell. Curiosity-driven exploration by self-supervised prediction. In International conference on machine learning, pp. 2778-2787. PMLR, 2017.
Tony Prescott, Mathew E. Diamond, and Alan M. Wing. Active touch sensing Phil. Trans. R. Soc. B, (11661551):29992999999 Nov 2011. ISSN 1471-2970. doi: 10.1098/ssb.201100165
Haozhi Qi, Brent Yi, Sudharshan Suresh, Mike Lambeta, Yi Ma, Roberto Calandra, and Jitendra Malik. General in hand object rotation with vision and touch In Conference on Robot Learning. pp. 2549 2564. PMLR, 2023.
Sai Rajeswar, Cyril Ibrahim, Nitin Surya, Florian Golemo, David Vazquez, Aaron Courville, and Pedro O Pinheiro. Haptics-based curiosity for sparse-reward tasks. In Conference on Robot Learning, pp. 395-405. PMLR, 2022.
Tim Schneider, Guillaume Duret, Cristiana de Farias, Roberto Calandra, Liming Chen, and Jan Peters. Tactile mnist: Benchmarking active tactile perception. arXiv preprint arXiv:2506.066: 2025.
Amir-Hossein Shahidzadeh, Seong Jong Yoo, Pavan Mantripragada, Chahat Deep Singh, Comelia Fermüller, and Yiannis Aloimonos. Actexplore: Active tactile exploration on unknown objects In 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 3411-3418, IEEE, 2024.
Jinghuan Shang and Michael S Ryoo. Active vision reinforcement learning under limited visual observability. Advances Neural Information Processing Systems, 36:10316-1093 10338, 2023.
Zilin Si and Wenzhen Yuan. Taxim: An example-based simulation model for gelsight tactile sensors IEEE Robotics and Automation Letters, 7():2361-223) 2022.
Edward Smith, David Meger, Luis Pineda, Roberto Calandra, Jitendra Malik, Adriana Romero Sori- ano, and Michal Drozdzal. Active 3D Shape Reconstruction from Vision and Touch Advances in Neural Information Processing Systems, 34:16064-16078 2021.
Rahul Tallamraju, Eric Price, Roman Ludwig, Kamalakar Karlapalem, Heinrich H Bülthoff, Michael Black, and Aamir Ahmad. Active perception based formation control for multiple acrial vehicles. IEEE Robotics and Automation Letters, 4(4):4491-444 2019.
Sinan Tan, Huaping Liu, Di Guo, Xinyu Zhang, and Fuchun Sun. Towards embodied scene description. arXiv preprint 2004 14638, 2020.
Annalisa Taylor, Thomas A Berrueta, and Todd D Murphey. Active learning in robotics: A review of control principles. Mechatronics, 77:102576, 2021.
14

---

Published as a conference paper at ICLR 2026
Shagun Uppal, Ananye Agarwal, Haoyu Xiong. Kenneth Shaw, and Deepak Pathak. Spin: Simul- taneous perception interaction and navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18133-188 18142, 2024.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma. Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger. Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of-the- art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Pro- cessing: System Demonstrations, pp. 38 45, Online, October 2020. Association for Computational Linguistics. URL //wp::://wwww./ ac wwwebwb....... www.w//.ttt.oooooooooooooooooo/e 2020 emnlp demos 6.
Haoyu Xiong, Xiaomeng Xu, Jimmy Wu, Yifan Hou, Jeannette Bohg, and Shuran Song. Vision in action: Learning active perception from human demonstrations. In Conference on Robot Learning. pp. 5450-5463. PMLR, 2025a.
Haoyu Xiong, Xiaomeng Xu, Jimmy Wu, Yifan Hou, Jeannette Bobg, and Shuran Song. Vision in action: Learning active perception from human demonstrations. In Joseph Lim, Shuran Song, and Hae-Won Park (eds.). Proceedings of The 9th Conference on Robot Learning, volume 305 of Proceedings of Machine Learning Research, pp. 5450-5400555 54600 PMLR, 27-30 Sep 2025b. URL
httpps://procedd:
mlr wtw.mmrrrr.rrsse..mmmmmm.mmmm v05/ httmomm222...m html
Jingxi Xu. Shuran Song, and Matei Ciocarlie. Tandem: Learning joint exploration and decision making with tactile sensors, IEEE Robotics and Automation Letters, 703)11393 2022.
Jianwei Yang, Zhile Ren, Mingze Xu, Xinlei Chen, David Crandall, Devi Parikh, and Dhruy Batra. Embodied amodal recognition: Learning to move to perceive objects. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 2040-2050, 2019. doi: 10.1 N0.1100/CCV2220900022...
Max Yang, Chenghua Lu, Alex Church, Yijiong Lin, Chris Ford, Haoran Li, Efi Psomopoulou, David AW Barton, and Nathan F Lepora. Anyrotate: Gravity-invariant in-hand object rotation with sim-to-real touch. Proceedings of Machine Learning Research, 270:4727-4744 2024a.
Ning Yang, Fei Lu, Guohui Tian, and Jun Liu. Long-term active object detection for service robots: Using generative adversarial imitation learning with contextualized memory graph. IEEE Transactions on Industrial Electronics, 2024b.
Zhengkun Yi, Roberto Calandra, Filipe Veiga, Herke van Hoof, Tucker Hermans, Yilei Zhang, and Jan Peters. Active tactile object exploration with gaussian processes In 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4925-4930. IEEE, 2016
Zhao-Heng Yin, Binghao Huang, Yuzhe Qin, Qifeng Chen, and Xiaolong Wang. Rotating without seeing: Towards in-hand dexterity through touch, Robotics: Science and Systems, 2023.
Hanna Yousef, Mehdi Boukallel, and Kaspar Althoefer. Tactile sensing for dexterous in-hand
manipulation in robotics-A review] Sensors and Actuators A: Physical, 167(2):11-117,, 2011. ISSN 0924-4247. doi: https://doi.org/ 10.1000.11.. sww.://......011l..... 1..00.0,,. Solid-State Sensors, Actuators and Microsystems Workshop.
Wenzhen Yuan, Siyuan Dong, and Edward H Adelson. Gelsight: High-resolution robot tactile sensors for estimating geometry and force Sensors, 17(12):2762, 2017.
Yi Zheng, Filipe Fernandes Veiga, Jan Peters, and Veronica J Santos. Autonomous learning of page flipping movements via tactile feedback IEEE Transactions on Robotics, (166):233-2244)) 2022.
15

---

Published as a conference paper at ICLR 2026
APPENDIX
A
HAM AS A GENERAL ACTIVE PERCEPTION METHOD
HAM in its original version is an active classification method. Though is in principle able to use different loss functions than cross-entropy loss, its reward is defined as 0-1 reward, yielding 1 for a correct classification and 0 for an incorrect classification. For a regression task, such a reward does make much sense, one would have hand-define thresholds classify predictions as correct or incorrect. However, we found that on the task it was developed for - the MHSB tactile classification task using the negative prediction loss directly as a reward yields similar results to the original implementation. With this modification, HAM fits in the APPLE framework, and would, in principle, also be a candidate algorithm for solving the objective in Eq. (T). However, as we show throughout our experiments, HAM'S on on-policy nature makes it impossible to compete with our off- off-policy approaches APPLE- SAC and APPLE -rosssQ.
Nevertheless, to allow for a fair comparison, we used this modified version of HAM throughout our experiments. The reasons for this are (a) that otherwise, the APPLE methods and HAM would be evaluated on different rewards, making them less comparable, and (b) that found in reproduction study the work Fleer et al. (2020), that HAM is, surprisingly, more stable when using the negative prediction loss reward.
B ENVIRONMENT DETAILS
Here, we detail each of the tasks that were evaluated, with the exception of the MHSB classification task, for which details can be found in Fleer et al. (2020). Thus. present the details for the CireleSquare 2D classification task, the Tactile MNIST tactile classification and volume estimation tasks, and the Toolbox pose estimation task. These tasks part of the TactileMNIST Benchmark Suite, and extra details can be found in Schneider et al. (0055)
B. CIRCLESQUARE TASK
In each CircleSquare episode, the agent receives a 28x28 grayscale image containing either a circle or square. It can only observe a 5x5 pixel region (glimpse), with the initial location randomized. A color gradient in the background helps guide exploration, but the agent has no access to the object's position. See Fig. 7 and Fig. for an illustration of the CircleSquare task.
Actions 1] are mapped to pixel motion as a 5.6 px (20% of the image width), allowing smooth movement across the image. We use bilinear interpolation compute the glimpse values even at non-integer positions. The agent's prediction e R22 is interpreted as logits for circle vs.
square:
e
e
Peircle (1::)
Pequare
evl
+ E
ev 22
Cross-entropy loss is used for training:
((vivv.) v.) =-
Σ
8(vi c) log (Pc(
CE sqquieeeessuuurr)
We apply a regularizing reward penalty on the magnitude of each action: r(he, az) = 100 lael2
Due to the small input size, do not use vision encoder; instead, the input is directly flattened into a 25-dimensional vector. This design choice allows a fairer comparison to HAM et al. 2222)). which also operates on flat image data.
B.2 TACTILE MNIST CLASSIFICATION
In the Tactile MNIST classification task (see 9h. the agent is presented with 3D model of high- resolution MNIST digit, placed randomly on I2x 12em plate. Each digit is up to 10cm in width and height. The agent uses single simulated GelSight Mini sensor Yuan al. (2017) explore the plate
16

---

Published as a conference paper at ICLR 2026
(a)
(b)
(d)
Figure Episode starting conditions the CircleSquare task. The agent's glimpse object (circle a or square a are placed in random locations on the field. Besides the color gradient, the agent receives no information about the object's location on the field.
(a)
(b)
(c)
(d)
Figure 8: Visualization of learned APPLE-Cros policy in the CircleSquare task. The agent starts at random location and uses the color gradient to locate the object. It can only observe 5 5 pixel patch. (b) The agent follows the gradient, gradually gathering information. Without full certainty, predicts 50/50 probability between classes along the way. Colored boxes show past glances, with color indicating prediction confidence. (c) The agent reaches the object at the corner. (d) Upon confident identification, the agent classifies the object as square (bright green box) and maintains this prediction in later steps.
Figure 9: The simulated Tactile MNIST classification benchmark Schneider et al, 22025). which we use for evaluating our method. The objective of the Tactile MNIST task is to ideatify the numeric value of the presented digit by touch only. In every step, the agent decides how to move the finger and predicts the class label. The haptic glance computed via the Taxim Si & Yuan (2022 tactile simulator.
17

---

Published as a conference paper at ICLR 2026
The simulated Tactile MNIST- Volume Schneider al. (2025) task, which we use evaluating our method. The objective this task is to estimate single continuous value representing the volume of a 3D MNIST digit by touch alone. In every step. the agent decides how to move the finger and predicts the volume of the digit. The haptic glance is computed via the Taxim Si & Yuan (022 tactile simulator, and in red, we show the current volume estimation in single run.
Figure 10:
surface. The sensor outputs a 32x32 pixel image rendered from depth map by Taxim Si Yuan (2022). Additionally, the agent receives the 3D position of the sensor as input.
Each episode begins with randomly selected digit, randomly placed and oriented. The agent has 16 steps to explore and classify the object. At each step, it chooses movement action é 1]", which corresponds to a translation of up to 1.5cm per axis. The sensor is automatically positioned to maintain 2mm indentation into the 4mm-thick gel. To simulate the object shifting around when being manipulated by the agent, we apply Gaussian random noise the position and orientation of the object throughout the episode.
The classification output is 10-dimensional logit vector. A standard cross-entropy loss is used:
10 (ū vc) - Σ 8 (vl log (pe (B+)), Pe (Br)
el!
10 ev!
The reward is used only for regularizing motion: r(h,, 04) 10 10-31|
We train APPLE-SAC SAC,
and APPLE-RND from scratch (no pre-trained encoders), using 5 random seeds for 5M steps. Hyperparameters are tuned via the HEBO Bayesian optimizer. HAM HAM Fleer et al. (2020) is not evaluated here as it lacks an image encoder. Evaluation is done using digits not seen during training.
B.3 TACTILE MNIST-VOLUME
In the Tactile ANIST-Volume task (see Fig. T0p. each episode begins with the 3D model of high- resolution MNIST digit being placed randomly on 12x 12cm plate. The agent explores the scene using simulated GelSight Mini sensor, identical that used in the classification variant of the MNIST task. The sensor provides 32x32 tactile image, and the agent also received its 3D sensor position as input. Actions a, [-1, 1] again correspond
maximum motion of 1.5cm per axis. The sensor maintains a fixed 2mm indentation into the gel on each step. Unlike Tactile MNIST, this a regression task, and the target in each episode is the volume of the digit, normalized zero mean and unit variance across all digits.
Each agent APPLE-SAC, APPLE-CrOsSO, and APPLE-RND is trained from scratch using 5 random seeds for 5M steps. We reuse the hyperparameters optimized on Tactile MNIST with- out modification evaluate robustness across tasks. Evaluation is performed on held-out shape configurations, which are not during training.
B.4 TOOLBOX TASK
In the Toolbox task (see 11). the agent has to estimate the pose of 24cm long wrench that is placed uniformly random 2D position and orientation on a 30x30cm plate. Similar to the previous two tasks, the agent explores the scene using simulated GelSight Mini sensor, providing 32x32 tactile image, while also receiving the 3D sensor position as input. Actions a, [[11,]] again correspond to maximum motion of 1.5cm per axis, and the sensor maintains fixed 2mm indentation into the gel on each touch. To simulate the object shifting around when being manipulated by the agent, we apply Gaussian random noise the position and orientation of the object throughout the episode.
18

---

Published as a conference paper at ICLR 2026
Figure The simulated Toolbox task, which we use for evaluating our method. The objective of Toolbox task is to determine the 2D pose (i.e., 2D position and orientation angle) of the object relative to the platform's center. In every step, the agent decides how to move the finger and predicts the 2D pose. The haptic glance is tactile simulator.
computed via the Taxim Si & Yuan 2022
(a)
(b)
(c)
(d)
Figure 12: Exploration strategy learned by our APPLE-Crose agent. In the beginning (a), both sensor and wrench start uniformly random places
platform. The agent guesses a central position the wrench (illustrated by the transparent wrench) to minimize error in the absence of any further information. To find the object efficiently, the agent has learned a circular search pattern and therefore quickly locates the object (b). However, the information it currently has is not enough, as the orientation of the wrench is not clear just by touching the handle. Thus, randomly guesses the wrong orientation, with the open pointing left instead of right. To gather information, moves along handle (c) until finds open jaw and immediately corrects the angle of its pose estimation.
The Toolbox task poses two challenges: finding the object and determining its exact position and orientation. Once the object is found, determining orientation is still not trivial, as many touch locations only provide ambiguous data. Hence, as shown in Fig. [2 even after the object is found, an exploration strategy for determining its pose must be executed.
Each agent APPLE-SAC, AFPLE-Crosst and APPLE - is trained from scratch using random seeds for 10M steps. We reuse the hyperparameters optimized on TactileMNIST without modification to evaluate robustness across tasks.
B.5 CIFAR-10 CLASSIFICATION TASK
Although omitted in the main paper due to space constraints, we also evaluated our approach on the CIFARIO task introduced by Schneider al. (2025). Similar CircleSquare, the agent moves small 5x5 pixel glimpse across an image in order to classify it. In contrast to CircleSquare, the agent is presented with 32x32 pixel RGB images from the CIFAR- 10 dataset Krizhevsky et al. (2009). which it must classify into 10 classes. See Fig. B for illustration of this task.
Actions a, 1 are mapped to pixel motion as a, 6.4 px (20% of the image width), allowing smooth movement across the image. We use bilinear interpolation compute the glimpse values even at non-integer positions. The agent's prediction ye e IR 0 is interpreted as logits for the classes:
e
(e():)
Σ1 eV
Cross-entropy loss is used for training:
10
Elvi v:) = Σ 80 log (pe(v)).
We apply a regularizing reward penalty the magnitude of each action: r(h4, 10-3 |
19

---

Published as a conference paper at ICLR 2026
(
(b)
(c)
(d)
Figure Episode starting conditions the CIFARIO task. agent's glimpse is placed in random location on the field. It must move the glimpse around in order to learn the class of the displayed image.
Due to the small input size, we do not use a vision encoder; instead, the input directly flattened into 25-dimensional vector.
The experimental results for CIFARIO are presented in Appendix C.6
C FURTHER EXPERIMENTS
In this section, we present additional experimental results that could not be included the main body due to space constraints.
C.I EVALUATING PPO AND HAM CIRCLESQUARE
Figure 14 shows an additional experiment on the CircleSquare task (chnniderr et (2025) where we compare HAM to two PPO-based variants, one using HAM's LSTM model (APPLE- PPOPPPPPPSSSSPPPPP LSTM) and one using our transformer model (APPLE-PPO), Note that, unlike in the experiment shown in Fig. 4 where stopped training after 250K environment steps. here, for 10M steps. Despite the longer training time, HAM fails to achieve a final prediction accuracy that is better than random guessing. The PPO variant using HAM's model (APPLE-PPO-LSTT achieves a final prediction accuracy of 72% after 10M steps, while the PPO variant with our model (APPLE-PPO) achieves around 79%. In addition to the difference in final performance of APPLE-PPO- beginning than APPLE POPPLE-PPO-TT LSTM. All hyperparameters for this experiment
let the training
and APPLE-PP being 7%, APPLE-P2 PPP improves much quicker in the
again tuned using HEBO Rivers et al. 2022 and each method was trained from scratch on seeds.
These results, in conjunction with the results shown in Section 4.2 indicate that HAM's issues in solving the CircleSquare task may have two reasons: First, HAM'S LSTM model seems to be ill- suited for learning this task, as all other compared methods (APPLE-SAC, APPLE-CroSSO and APPLE-PPC) each performed worse with HAM's LSTM model than with our transformer model. Second, the fact that PPO and HAM are both on-policy algorithms likely has a negative impact on their sample efficiency, as samples collected from the environment cannot be reused later on. Off-policy algorithms, such as APPIE APPLE-SAC SAC and APPLE -CosssC on the other hand, store samples over course of the entire training and revisit them many times. making optimal use of the information gathered during training. We see this effect clearly in Fig. [4 where RPPLE-Crosse outperforms APPLE-PPE by orders of magnitude in sample-efficienge despite both algorithms using the same model for their policy and class predictions.
C.2 ABLATION STUDY ON CIRCLESQUARE
To gain an understanding of the effect of the different components of APPLE, we conduct an ablation study on our CircleSquare experiment.
First, we replace the transformer model with LSTM model. The results shown in Fig. 15 show that, although our model is still learning to solve the task, convergence is much slower compared to the transformer variant of our method, despite extensive tuning of the LSTM baselines with Cowen- Rivers et al. (2022). These findings are in line with the results from Appendix [C.]] in which the transformer-hase architecture also outperformed the LSTM architecture when used with
20

---

Published as a conference paper at ICLR 2026
Avorago Accuracy
A
Final Accuracy
0.8
0.8
0.6
0.6
0.4
0.4
10
2
10
Steps
-10"
Steps
-10"
APPLE-PPO-LSTM
APPLE-PPO
HAM
APPLE-CrossQ
Figure 14: Experiments on the CircleSquare task, comparing HAM with two PPO-based variants: APPLE-PE and APPLE- -LLEE-PPPLSTTE The difference between APPLE-PPO and APPLE-PPO-ISS is that APPLE-PE transformer model, while PPLLE-PPOLLSTE 250K steps shown for reference. The plot shows the average prediction accuracies, while the right plot shows the final prediction accuracies. Training is terminated after 10M environment steps.
HAM's LSTM model. APPLE-CrossE
with
Averago Accuracy
Final Accuracy
0.8
0.6
0.6
0.4
0.4
Steps
-10"
Steps
-10
APPLE-SAC APPLE-CrOsSQ-
APPLE-SAC-LSTM APPLE-CIOSQ-LSII
Figure 15: Ablation the CireleSquare environment, comparing
APPLE variants with modified L.STM model place of APPLE'' transformer model. The left plot shows the
versions, in which average prediction accuracies, while the right plot shows the final prediction accuracies. Training terminated after IM enviromment steps.
PPO variant of APPLE. Hence, we conclude that the choice of model can have a significant influence on APPLE's overall performance.
Second, we investigate the importance of utilizing the target label and the loss function é during training. Hence, in this experiment, just f(h,, Uy at 31) directly an RL-reward and make y become part of the action space. Instead of relying on supervised learning for optimizing π < ona). the agent must now rely on its policy gradient and essentially find the optimal 31 via trial and error. We call these variants APPLE SAC-PURE RL and APPLE CrossQ PURE-RA respectively.
As shown in Fig. 16 although the final accuracy APPLE SAC PURE briefly peaks to 80%, the absence of target labels and the loss function makes the training of these agents very inefficient and unstable. Since CircleSquare is fairly simple task, this experiment highlights how challenging it is for RL agents to discover and learn correlation between their observations and the reward they receive this setting no structure imposed.
Finally, we compare our approach to variant of itself, which, instead of learning an exploration policy, uses a static grid-search strategy to gather information. This strategy moves the glimpse in grid pattern across the image, always starting at the closest corner. While this strategy produces better exploration than a random policy. it stagnates at a final accuracy of 73%. The reason for this poor performance is that the time budget of the CircleSquare task is specifically chosen to be too low for exhaustive exploration of the entire environment. Instead, the agent must learn to follow the gradient to efficiently locate and classify the target object.
21

---

Published as a conference paper at ICLR 2026
Averago Accuracy
Final Accuracy
Steps
-10
Steps
-100
SAC APPLE-CnssQ
APPLE SAC PURE RI.- APPLE CIA.EE.CCS...OO PURE RL
Ablation the CircleSquare environment, comparing versions, which we treat the prediction loss a regular RL reward, using neither differentiability nor target lahel. Instead, label prediction becomes part it through its regular policy gradient. The left plot shows the average prediction accuracies, while the right plot shows the final prediction accuracies. Training is terminated after 1M environment steps.
Figure 16:
APPLE variants with modified
regular action space
agent has optimize
Average Accuracy
Final Accuracy
Steps
-10
Steps
-10
SAPLE-SAC
APPLE-CrussQ)
APPLE-GRIE
APPLE variants with modified version using a heuristic grid-scarch policy. The grid-scarch policy searches through the image in grid pattern, always starting from closest corner and moving vertically first. Aside from the agent having no control over its actions, everything else is kept the same. The left plot shows the average prediction accuracies, while the right plot shows the final prediction accuracies. Training is terminated after 1M enviromment steps.
Figure 17: Ablation the CireleSquare environment, comparing
22

---

Published as a conference paper at ICLR 2026
Tinal Accuracy
0.8
0.6
0.4
0
5
5-10-2
0.1
0.15
0.2
0.25
Steps
-10
APPLE-SAC (sparse) APPLE-CrossQ (sparse)
APPLE-SAC
APPLE-CrossQ
Figure 18: Experiment on sparse version of CireleSquare, which only the agent's 'eet's prediction the last time step counts. The original learning curve the non-sparse version of CircleSquare is also displayed for reference. Training is terminated after IM environment steps.
C.3 HOW WELL DOES APPLE DEAL WITH SPARSE REWARDS?
In the real world, labels might not always be available for every time step. For example, if tracking system used generate ground-truti data for a tactile pose estimation task, there may be some time steps in the data where the tracking system loses sight of the object. To support this case in our formalism, we can store boolean variable wy {0, 1). indicating whether the object was tracked
at time step (w, 1) or not (w; 0) alongside the perception targets Ve- With tracking-aware loss function {((). wx) == w; MSF 3t), we can now make sure that only those steps get considered in the prediction loss in which the tracking
successful.
To briefly evaluate the feasibility of using APPLE with sparse rewards, we conducted a small CireleSquare environment. In this experiment, using the above loss function and setting w; = 0 for all time steps except the last one, where we set wy = Essentially, the agent receives its perception reward only for the prediction it makes in the final time step. As shown in Fig. [8] we observe that both APPLE variants continue to learn to solve the task, although our Cross variant converges slightly more slowly.
experiment
made the reward sparse by
C.4 CAN THE PERCEPTION LOSs HELP IN THE PRESENCE ANOTHER DOWNSTREAM TASK?
Our long-term objective with this line of work is tackle tasks in which active perception is not the primary objective, but rather tasks which the agent must use active perception to fulfill another objective. One such example is finding and retrieving specific tool from toolbox. In this task, the main objective is not to find the tool, but still has to be found in order to be retrieved. Although we deem these types of tasks be outside of the scope of this work, conducted small experiment to pre-validate the feasibility of using APPLE in this context.
Specifically, created variant of the CircleSquare task, in which the agent must stay close squares and far from circles. We call this variant GirceSsqurelHideenndn and implement it by using the following base reward instead of the regular CireleSquare base reward:
r(h,, a.) 10 |a.11
[
Ipt agent
agent
Pt
object ||
if square
111p
Pt
object
if circle
Hence, to solve this task optimally, the agent must first identify the class the object and then either remain close move far away. Here, active perception is only an intermediate step towards solving this task, as knowing the label yields reward only indirectly. In Fig. 19 we compare two variants of APPLE-CrosSO agent this task: regular APPLE-CrosSQ agent, which utilizes label and loss-function information, and a variant LEPLE--rrosS--M--PRRS which just maximizes the base return.
Our results show that the regular agent, which utilizes the label and the loss function, significantly outperforms the agent relying pure RI. This result suggests that incorporating supervised
23

---

Published as a conference paper at ICLR 2026
Cumnllative (Base) Returns
0
0
0.2
0.4
0.6
0.8
Steps
-109
APPLE-CrossQ APPLLE-CROSQ-NO-PREEE
Figure 19: Experiment on the CirdesSquareiideendSeee variant of CireleSquare. this variant, the agent must stay close to squares and avoid circles. We test two variants: FPLE-Crosss? our regular approach, and APPLE-Cross$ sa-s----222 -NRDRRDD which does not make of the target label and the loss function. APPLE-SKA SAC variants were included neither them learned any meaningful behavior. Training terminated after 1M environment steps.
Averggo Accuracy
Final Accuracy
0.8
0.8
0.6
0.6
0.4
0.4
2
3
Steps
-10
Steps
-10
HAM APPLE APPLE-CrosI
Figure 20: Additional experiments on the MHSB Classification task Fleer et al. 2020), where we let HAM run for longer. -Foos-Cssss's with 1M step is shown for reference. The left plot shows the average prediction accuracies, while the right plot shows the final prediction accuracies. HAM eventually converges within steps, reproducing the results of Fleer el al. 2020). Note that the experiment here differs slightly from Fleer et al. 2020), as we use the classification loss as a reward signal for the RI. agent instead of a binary reward. We chose to do this modification to allow for fair comparison to our methods.
good policy
learning problem
inductive bias to guide agents toward discovering relevant information be beneficial for solving downstream tasks. However, more experiments on more complex environments needed to investigate whether adding such active perception bias also brings advantages in broader applications.
C.5 LONGER EVALUATION OF HAM OF THE MHSB TASK
To validate our implementation of HAM, we conduct longer experiment on the MHSB task, on which it was originally evaluated. The results of this experiment can be seen in Fig. 20 where we see that HAM eventually converges to good performance. Since these results are similar to those of Fleer et al. 2020), we conclude that our implementation is correct.
C.6 CIFARIO CLASSIHICATION TASK
In Fig. 21 we present the results of running our two APPLE variants, as well as a random baseline, on the CIFAR1O 10 task, described in Appendix B.5 Although the task is similar, CIFAR10 is significantly more challenging than CircleSquare, as the agent now has to deal with much more diverse data and ten labels instead of two.
24

---

Published as a conference paper at ICLR 2026
0.8
0.8
Averago Accuracy
Final Accuracy
0.7
0.7
1.5
2.5
0.5
1
1.5
2.5
Steps
-10"
Steps
-10
APPLE-SAC
APPLE-CrossQ
APPLE-RND
Figure 21: Average and final prediction accuracies for our methods APPLE- 8AC and APPLE -CrossO, and APPLE-RND the CIFAR1O task. All methods were trained with 5 seeds. Shaded areas represent one standard deviation. Metrics computed the split with unseen images.
As shown in Fig. 21 despite not being tuned for this task, APPLE- SAC still performs well, achieving a final accuracy of 76%. MPLE-CCossQ converges slightly slower and reaches a slightly lower final accuracy of 73%. Our random baseline reaches final accuracy of 67% after 2.5M steps.
D LIMITATIONS
While APPLE demonstrates strong performance across various active perception tasks, it also has certain limitations. One of the primary drawbacks its reliance on large amounts training data, requiring up to 5M steps for the tactile perception tasks. This high data requirement arises from the combination of a transformer- this approach enhances the generality of APPLE, allowing it to adapt to different tasks without hyperparameter tuning, comes at the cost sample efficiency. A promising avenue to solve this issue is leveraging pre-trained transformer models, which could improve sample efficiency by providing useful feature representations. Furthermore, recent advancements in sample-efficient reinforcement learning Nauman al. (022)); [ee et al. (2025) offer potential alternatives for improving the practicality of APPLE in real-world applications. Another limitation and future direction is explore a more diverse and practical set of tasks. Applications such as object pose estimation, shape reconstruction, material property inference remain unexplored and could pose additional challenges our methodology. Moreover, our current experiments use a single tactile sensor, but in principle, the APPLE model architecture supports multi-fingered robotic hands and multi-modal perception (e.g., combining vision and touch). However, the practical scalability of APPLE to those applications remains open question, the increased action and observation space complexity may introduce additional challenges training efficiency, policy learning stability, and computational demands. Future work will explore these extensions by evaluating APPLE LE multi-fing multi-fingered robotic systems and integrating complementary sensing modalities to enhance active perception capabilities.
architecture and RL-based policy optimization. While
E HYPERPARAMETER OPTIMIZATION
For fair comparison between our method and the baselines, we have performed extensive hyperpa- rameter searches with the HEBO Cowen-River et (2022) Bayesian optimizer.
Procedure. We select the CircleSquare and TactileMNIST classification environments as repre- sentative environments on which to tune hyperparameters. Specifically, CircleSquare is used as the representative environment without vision encoder, and TactileMNIST is used as the representative environment with a vision encoder. We evaluate each candidate configuration by training with a single seed (250K steps CircleSquare and 2.5M steps MNISi, except for HAM and PPC. which are trained for IM steps on CircleSquare) and measure the episode return averaged across the entire training run. Averaging rewards over time, rather than using final performance alone, ensures
25

---

Published as a conference paper at ICLR 2026
Table 1: Hyperparameters determined by the HEBO Nowen-Rivers al. 2022 Bayesian optimizer for APPLE-SAC and APFLE-CrossQ. The no vision-encoder configuration was trained on the CireleSquare environment, while the vision-encoder configuration was trained on the TactileMNIST environment. Hyperpa- rameters with Rel. are relative to the total number of steps throughout the training.
APPLE-SAC
APPLE-CrOsSQ
Hyperparameter
vis.-enc.
vis.-enc.
vis.-enc.
vis.-enc.
Optimizer type
ADAMW
ADAMW 5- 5-10-4
ADAMW
ADAMW 3 10
Learning-rate (actor)
5 10-5
1- 1.10-5
Learning-rate (critic)
5 10
5-10-3
1- 10
6- 1-10
LR-schedule (both)
none
cosine-decay
none
none
Rel. LR cosine warm-up (both)
N/A
0.15
N/A
N/A
Initial UTD
0.75
0.25
5.0
0.25
Final UTD
4.0
1.5
0.5
3.5
Rel. UTD warm-up
0.9
0.4
0.3
0.45
that sample efficiency is taken into account: two hyperparameter sets achieving the same final return may differ greatly in how quickly they reach that level of performance.
Search space. Because Bayesian optimization scales poorly with dimensionality, we restrict the search to hyperparameters we found most impactful for performance. All methods are tuned for learning rate, learning-rate schedule (none, cosine decay, linear). schedule parameters (c.g., warm- up steps), and optimizer choice (ADAM, ADAMW, SGD), For off-policy APPLE methods, we additionally tune the update-to-data (UTD) schedule: initial and final UTD ratios and the number of warm-up steps.
Findings. Despite an extensive search, we were unable to identify hyperparameters yielding com- petitive performance for HAM on CircleSquare. In contrast, the search provided valuable insights for APPLE- SAC and Crooss. Although both achieved comparable performance on their tuning tasks, APPLE- SEPPL--Crosse Cross demonstrated substantially greater robustness when transferred to unseen environments (Section 4 Interestingly, applying APPLE-CrossO? vision-encoder hy- perparameters to CircleSquare produced no measurable degradation in performance. To simplify evaluation, therefore adopt the vision-encoder configuration for all environments in subsequent experiments.
The final hyperparameter settings selected by HEBO are summarized in Table
F IMPLEMENTATION DETAILS
The implementations of APPLE, APPLE-PPO, and HAM are built on JAX Bradbury al. (2018) with the Flax framework Heek et al. (2024), and use Hugging Face transformers Wolf et al. (2020). For performance, the training loop is fully JIT-compiled, and environment interactions are handled via host
throughput at the expense of some implementation flexibility.
Replay buffer design. A common bottleneck in deep learning arises from the transfer of data between VRAM (GPU) and RAM (CPU). To minimize this overhead, we keep the replay buffer in VRAM, so that host-device communication is limited to stepping the environment and logging. The drawback is reduced capacity for environments with vision inputs, since VRAM is smaller than RAM. On Nvidia RTX A5000 GPUs (24GB VRAM), storing downscaled 32 32px visual inputs allows for roughly 3M transitions before memory is exhausted. Consequently, for vision-encoder configurations (i.e., the Tactile MNIST classification and volume estimation tasks as well as the Toolbox localization task), set the replay-bufis buffer size to 3M, whereas for non-vision-encode configurations, use replay-buffer size equal to the total number of environment steps.
Hardware setup. All experiments
Nvidia RTX A5000 RTX 3090 GPUs with the hyperparameters from [ For vision-encoder configurations, we dedicate one GPU per run. A single run of 5M training steps takes about 40-50 hours, depending the algorithm.
26

---

Published as a conference paper at ICLR 2026
Parallelization. Non-vision-encodee configurations demand less VRAM, enabling multiple runs to share a GPU. On a single RTX A5000/3090, we can accommodate up to 28 parallel runs, depending on the environment and algorithm. As wall-clock runtime then depends heavily the number of concurrent runs, reporting averages is not meaningful. Nevertheless, we observe that HAM and PPO typically about four times faster than APPLE.
G DETAILED DERIVATION OF EQ.
Due space constraints, omitted intermediate steps in the derivation of Eq. in the main paper and instead note them here:
oa Σr
(r( - (ie, - (or)
J (TT)
E 00
00
pa o...
1=0
0
ʃ pa
h. y. o.a
Σ (1(he, as) Lao (ic onr) 00)) d(h)dd(hh,) d(b.).,,a)
00
=
-f
20
h y. o. a
Σ "' (r(he,as) - Lx, (ūr (ut))
0
pa
h h,y, a,
(r(h), - ET (Ur: (vt: 00zt) dl d(h, y,
00
0
=
0
In no (a Σ Bi v1)
Σn r 00
lT (uer Onx))
po (n......a))
00
pa (5.00)
t=
policy gradient
prediction loss gradient
H CONNECTION TO CURIOSITY-BASED INTRINSIC REWARD METHODS
Since formulation in Eq. 2 somewhat resembles the augmented rewards commonly used in curiosity- based intrinsic reward RL methods, such as RND Burda et al. (2019) or Pathak et al. (2017). one might wonder what the connection between APP LE and those methods is. Fundamentally. APPLE LE solves a different problem than curiosity- based intrinsic reward methods. Intrinsic reward methods typically focus enhancing policy learning in MDP setting by providing intrinsic exploration bonuses for under-explored areas of the state space. Hence, they are concerned with letting the agent collect rich experiences to facilitate efficient policy learning. Crucially, the MDP the agent faces during training is, because of the augmentation with the intrinsic reward bonus, different from the one it faces during evaluation.
APPLE. on the other hand, is concerned with learning hidden properties of the environment within an episode. Although our formulation may resemble the augmented rewards of intrinsic motivation methods, it operates on fundamentally different level of exploration. In our case, the extraction of information the objective posed by the task and not a surrogate used to learn better policies.
On a high level, one could say that intrinsic reward methods utilize exploration to learn the optimal policy parameters, while APPLE uses exploration to learn about the hidden state the POMDP operates in. Hence, the exploration procedure of intrinsic reward methods occurs across episodes, while the exploration procedure APPLE takes place within each episode. These two concepts are fully orthogonal, and one could indeed combine ICM or RND with APPLE in attempt to speed up its learning progress.
27


Reviews:
[{"review_id": "9NYf1whK9N", "reviewer": "Reviewer_bEC3", "summary": "This paper introduces APPLE, a reinforcement learning framework designed for active perception tasks. The core formulation treats active perception as a POMDP and jointly optimizes a transformer-based policy for action selection and a perception module for task prediction, using a reward signal that incorporates task-specific differentiable losses. The method is evaluated on multiple simulated benchmarks covering classification, regression, and pose estimation tasks. Results indicate that APPLE outperforms both a random baseline and a prior method (HAM), demonstrating its ability to learn effective exploration strategies without relying on task-specific heuristics.", "questions": "The \"RL reward\" is mentioned but not specified in detail for each task. How was this term defined across the different benchmarks?\nThe transformer backbone introduces considerable complexity. Was a comparison performed with a simpler recurrent model to ensure that performance gains stem from the proposed framework rather than the representational capacity of the transformer?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The principal strength lies in the consistent performance across multiple tasks without requiring task-specific modifications, demonstrating good innovation.\nExtensive experiments across five benchmarks offer convincing evidence of the framework's capability and robustness.\nThe comparison with HAM effectively underscores the benefits of off-policy methods in improving sample efficiency.", "weaknesses": "The comparison is largely limited to a random baseline and HAM. Evaluation against other non-RL active perception methods would strengthen the validity of the claims.\nThe paper lacks thorough ablation analysis. For instance, the importance of the transformer architecture remains unclear.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3IsCDzl9Zq", "reviewer": "Reviewer_WrTJ", "summary": "This paper proposes a general RL framework for task-agnostic exploration using intrinsic rewards derived from a jointly optimized prediction model. The authors instantiate an active perception setting with tactile sensing for 3D-MNIST digit classification and evaluate their methods (both SAC and CrossQ variants) on a shared Transformer architecture. Experiments indicate the approach is effective and the architectural choices are reasonable.", "questions": "- Relation to IRRL: How does your RL-based active perception framework compare to the IRRL framework (Ref. [2]) in terms of objective, training signal?\n\n\n- Question on the backbone choice: the tasks in this submission are with relatively small exploration spaces, is the Transformer architecture superior to an LSTM-based alternative, and why?\n\n\n- Real-world deployment:\n(1) Is it advisable to include warm-up strategies (e.g., demonstrations, heuristic exploration priors) for the policy, or analogous prior/knowledge injection into the prediction module?\n(2) In practical deployments, access to ground-truth labels during exploration is limited or delayed. How do you handle sparse/late/partial labels, and what would the deployment workflow look like?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- A general formulation with a clear, well-motivated architecture. Clear, principled formulation of active perception as minimizing a supervised loss inside an interactive POMDP, with a clean gradient decomposition (policy gradient minus prediction-loss gradient). This is a neat unification that explains the joint training signal succinctly.\n- The paper is well written with clear presentation.\n- Robotic scenarios are relevant and valuable for tactile-enabled active perception with a broad set of tactile tasks (classification, regression, localization).", "weaknesses": "- The claim of “a novel framework that leverages RL to address a range of different active perception problems” could be more specific. Which component is novel—the unified formulation (e.g., Eq. 3), the Transformer with shared components, or something else? Clarifying this and discussing generalizability e.g. with other modalities beyond tactile sensing would help.\n\n- Additional discussion/experiments comparing sparse (more realistic when labels arrive late in real-world settings) vs. dense rewards would strengthen claims about generality.\n\n- Including related baselines or discussion would be helpful, e.g., RL exploration with haptics as reward [1] and earlier related work [2].\n\n- More discussion of real-world deployment (learning time, sim-to-real if any transfer is intended) would be valuable, since real-world “exploration” can be costly or impractical.\n\n- Minor: clarifying the definition and role of $\\tilde{r}$ in Eq. 1 would aid interpretability.\n\n\n\nReferences\n\n[1] Rajeswar, Sai, Cyril Ibrahim, Nitin Surya, Florian Golemo, David Vazquez, Aaron Courville, and Pedro O. Pinheiro. \"Haptics-based curiosity for sparse-reward tasks.\" In Conference on Robot Learning, pp. 395-405. PMLR, 2022.\n\n[2] Li, Mengdi, Xufeng Zhao, Jae Hee Lee, Cornelius Weber, and Stefan Wermter. \"Internally rewarded reinforcement learning.\" In International Conference on Machine Learning, pp. 20556-20574. PMLR, 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "riVYV0Zc7f", "reviewer": "Reviewer_LSx3", "summary": "The authors propose to solve active perception tasks, where the agent  interacts with the environment to collect features for prediction. The work is focused on tactile manipulation tasks where the robot must feel objects to determine their class. They use a transformer based architecture and optimize a reward function that minimizes prediction error. They show results on simulated benchmarks where the agent explores via touch sensor or limited field of view to determine an object of interest. They outperform a random policy and an LSTM policy.", "questions": "Aside from my concerns above:\n\nWhat happens if the prediction loss is not differentiable? Would RL still work on this objective?\n\nAs mentioned in the concerns, it would have been nice to have tasks where active perception is part of completing the task, but not the end goal. And comparing against a baseline that just gets task reward.\n\nAre there videos of the learned behaviors? Is there anything interesting there, like the agent learns some efficient exploration strategy?\n\nThe experimental section gets a bit repetitive. I wish there were more qualitative analysis and visualizations instead of just comparing final accuracy scores. For example, what behaviors are learned, visualizations of the exploration behavior of the policies, etc.\n\n---\n\nOverall, I think this paper needs some rewriting and tweaks on the experiments, and I look forward to the changes. I like the overall direction, as the intersection of tactile sensing and active perception is under-explored, and is likely to be very important as humanoids/dexterous manipulation  become popular.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The proposed objective and architecture is sensible for solving active perception tasks in the tactile sensing domain, although I have some issues with the framing of it as a general framework.\n\nActive perception for tactile sensing is a niche area with lots of potential for growth, so this paper is very timely. \n\nThe paper is well written and provides good background on tactile sensing and active perception.", "weaknesses": "## Claim of generality of method is misleading\nThe authors claim that this active perception method is not tied to any particular task, and is \"general\". This is misleading and needs to be revised.\n\nThe method proposes an additional reward term, which is a differentiable prediction term. First, requiring the term to be differentiable reduces the amounts of tasks this method is applicable to. Next, adding an additional prediction term introduces tuning complexity and the danger of reward misspecification. This objective would not work in tasks where it is hard to specify the state (no access to pose or  state). \n\nFinally, perhaps most importantly, the authors only evaluate in tasks where the task success is defined as correctly perceiving the state. But in most realistic tasks, perceiving the state is only an intermediate step towards solving the task. And the degree to which we need to perceive the state to solve the task is often quite low.\n\nThere is an approach that bypasses all of these limitations.  Maximizing the return with RL is the most general approach. If the task demands active perception in the optimal policy, then RL will discover it to maximize reward. Of course, this can be intractable in many cases, so we need additional mechanisms, like the reward term proposed, to improve learning. But this reduces generality.\n\nI would like the authors to reframe their claims here, perhaps saying this is a good way of solving a particular class of active perception tasks where the goal is to discover a state, without overstating the generality.\n\n## Lack of baselines\nI think there could be more baselines than just a random policy and a LSTM policy. For example, what about a hardcoded raster-scanning policy? Or some policy that uses a heuristic for exploration / information gain, etc. These simple approaches would be considered first for a roboticist. \n\n\n## Framing of background work\nI have an issue with the background section. It has a few sentences that I disagree with.\n>In the context of active perception, a few works have explored RL as an option.\n> However, these methods are usually tailored to specific tasks, environments, and objectives, and often assume the agent does not influence the environment through its actions. To our knowledge, there exists no active perception method that has been shown to work on a wide range of tasks, objectives, and environments.\"\n\nThese sentences give the feeling that active perception is under-explored, and current methods are unsatisfactory in generality and performance. But this is not true.\n\nA quick search on the web for most recent papers on active perception, e.g.  \"CoRL 2025 active perception\" or \"Neurips 2025 active perception\" shows multiple papers that use RL, VLA, reasoning to do active perception on a variety of tasks on real world robots.  \n\nWhile at the time of writing the authors may not have known these works, it's clear the broader robot learning / RL / VLA community has recognized the importance of active perception, and have made progress there. Another missing line of work is the recent wave of learning active perception policies via imitation learning and teleoperation platforms with head eye cameras. Finally, the active inference literature proposes an objective that is very general and does handle both active perception and decision making jointly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WHXflGpmhG", "reviewer": "Reviewer_yWQL", "summary": "The APPLE paper introduces a unified framework for active perception that jointly trains a sensing policy and a prediction model within the learning process. Instead of treating perception and control as separate modules, the method combines them through a single objective that encourages both accurate predictions and efficient sensing. The approach is implemented using standard off-policy reinforcement learning methods and evaluated on simulated visual and tactile tasks. The results show consistent improvements over random and simple baseline policies.", "questions": "1. The paper emphasizes multi-sensor and task-agnostic capabilities, yet all experiments are single-sensor and per-task. Could you clarify what evidence supports those claims?\n\n2. Can the framework handle sensor noise or real-world uncertainty (e.g., drift, latency, or stochastic contacts)?\n\n3. How does APPLE differ algorithmically from curiosity-based intrinsic reward methods such as ICM or RND, beyond replacing intrinsic prediction error with supervised task loss?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "**Strengths**\n\n**1. Clear Motivation and formulation:** Presents a simple, mathematically sound loss-augmented return that ties prediction and control under a single objective. Theoretical formulation is also explained clearly and is easy to follow.\n\n**2. Good Writing:** The paper is clear, concise, and well-organized. Its explanations of tasks and evaluation approach are logically structured, facilitating straightforward comprehension of the methodology and results.\n\n**3. Implementation clarity:** Detailed reporting of prediction rewards, loss and hyperparameters are provided, and it seems easy to reproduce.", "weaknesses": "**Weaknesses**\n\n**1. Overstated Claims:** The paper repeatedly suggests that APPLE is sensor-agnostic and can integrate data from “diverse sensor inputs without task-specific modifications” (Line 88). However, every reported experiment uses only a single simulated sensor type: the tactile tasks (TactileMNIST, Volume, Toolbox) rely on the GelSight Mini, while the visual task (CircleSquare) uses a synthetic 5 × 5 pixel grid rather than an actual camera. For justification, the authors must provide more experiments on diverse settings.\n\n**2. Novelty:** The paper’s conceptual foundation is not particularly novel, as it builds upon long-established principles in active perception and intrinsic exploration. A broad range of prior work spanning attention-based models, curiosity-driven reinforcement learning, and active sensing  has already linked exploratory behavior to prediction-based or feedback-driven learning signals. These approaches share a common theme: agents act to maximize informational or predictive value, whether expressed through task accuracy, novelty, or uncertainty reduction. APPLE remains situated within this same conceptual framework, differing mainly in its operationalization, substituting intrinsic prediction error with a supervised task loss embedded in the reward. While the formulation is elegant and well-executed, it represents a refinement of existing paradigms rather than a substantive theoretical advancement.\n\n**3. Experimental Evaluations:** The experimental evaluation remains simulation-bound and lacks any real-world validation, making the conclusions about generality largely speculative. All tactile experiments rely on simulated tactile images rather than data from a physical sensor, so issues such as contact noise, latency, or sensor drift are never addressed. The supposedly “general” Toolbox benchmark is limited to a single rigid object (a wrench) and one fixed workspace, offering no evidence of cross-object or cross-material generalization, which is an essential aspect for active perception in robotics. Moreover, there are no multi-object or multi-tool variations to demonstrate robustness to geometry or frictional diversity, and no tests in cluttered environments. This narrow scope undercuts the paper’s broader claims of task and sensor generality.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)