Taccel: Scaling Up Vision-based Tactile Robotics via High-performance GPU Simulation
Authors: Yuyang Li
Wenxin Du
Chang Yu
Puhao Li
Zihang Zhao
Tengyu Liu
Chenfanfu Jiang
Yixin Zhu
Siyuan Huang
Venue: NeurIPS 2025 spotlight ()
Keywords: tactile sensing
simulation
robotics
Primary area: applications
TL;DR: We propose Taccel, a high-performance GPU-based physical simulator for robots with vision-based tactile sensor.

Abstract: Tactile sensing is crucial for achieving human-level robotic capabilities in manipulation tasks. As a promising solution, Vision-based Tactile Sensors (VBTSs) offer high spatial resolution and cost-effectiveness, but present unique challenges in robotics for their complex physical characteristics and visual signal processing requirements. The lack of efficient and accurate simulation tools for VBTSs has significantly limited the scale and scope of tactile robotics research. We present Taccel, a high-performance simulation platform that integrates Incremental Potential Contact (IPC) and Affine Body Dynamics (ABD) to model robots, tactile sensors, and objects with both accuracy and unprecedented speed, achieving a total of 915 FPS with 4096 parallel environments. Unlike previous simulators that operate at sub-real-time speeds with limited parallelization, Taccel provides precise physics simulation and realistic tactile signals while supporting flexible robot-sensor configurations through user-friendly APIs. Through extensive validation in object recognition, robotic grasping, and articulated object manipulation, we demonstrate precise simulation and successful sim-to-real transfer. These capabilities position Taccel as a powerful tool for scaling up tactile robotics research and development, potentially transforming how robots interact with and understand their physical environment.

Taccel: Scaling Up Vision-based Tactile Robotics via High-performance GPU Simulation
Yuyang 133,5
Wenxin Du 2+
Chang Yu 2+
y 9.li@stu.pku.. edu. en setsunainn@gmail com
Shangyul@g.ucla.e
Puhao Li3,4
Zihang Zhao 1,4,3
Tengyu Liu 3,4
1iph238mail tainnguua.... edu.en
zhaozihang@stu.pp www.ggsstu.uuuueeee www.uu.kuuueuuuee.. en
liutengyu@higai...
Chenfanfu Jiang 2,03 of jiangeanttthh ucia.
Yixin Zhu 1,4,00,8
Siyuan Huang 3,4,2
yixin. zhu@pku hetpskueedd.coo edu. on huangeiyuan suc.yuunnuccaccdu
Eqqual contributor Corresponding author
Peking University 2 University of California, Los Angeles Beijing Institute for General Al
State Key of General AI Beijing Key Laboratory Behavior Mental Health, Peking University 6 Embodied Intelligence Lab, PKU-Wuhan Institute for Artificial Intelligence
attps://tace
simulator github. io
Abstract
Tactile sensing is crucial for achieving human-level robotic capabilities in ma- nipulation tasks [54]. As a promising solution, Vision-based tactile sensors (VBTSs) [61, 37] offer high spatial resolution and cost-effectiveness, but present unique challenges in robotics for their complex physical characteristics and vi- sual signal processing requirements. The lack of efficient and accurate simulation tools for VBTSs has significantly limited the scale and scope of tactile robotics re- search [51, 8J. We present Taccel, high-performance simulation platform that integrates Incremental Potential Contact (IPC) and Affine Body Dynamics (ABD) model robots, tactile sensors, and objects with both accuracy and unprece- dented speed, achieving total of 915 FPS with 4096 parallel environments. Un- like previous simulators that operate at sub-real-ting lelization, Taccel provides precise physics simulation and realistic tactile sig- nals while supporting flexible robot-sensor configurations through user-friendly APIs. Through extensive validation in object recognition, robotic grasping, and articulated object manipulation, we demonstrate precise simulation and success- ful sim-to-real transfer. These capabilities position Taccel as powerful tool for scaling up tactile robotics research and development, potentially transforming how robots interact with and understand their physical environment.
speeds with limited paral-
Introduction
The ability to physically interact with the environment through touch is fundamental to robotic manipulation [3. 13]. While vision provides global scene understanding, tactile sensing captures crucial local contact information [57] essential for precise manipulation. Among various tactile sensing technologies [64, 25, 38, 26], vision-based tactile sensors (VBTSs) such as GelSight [61] and 9DTaac [37] have emerged as a central focus in tactile research. Their ability to provide high- resolution tactile feedback through cepttrrd deformation patterns of elastic gel pads, com- bined with cost-effectiveeness, has driven significant advances in robotics [66, 35, 47, 8, 65].
The primary challenge in scaling up VBTS-equipped robot simulation lies in accurately modeling the hyperelastic soft gel pad and its contact [66, 12]. Current approaches follow two main directions: rigid-body approximations [51, 56] and soft-body simulations [47, 8, 12, 21, 30, 66]. While rigid- body methods efficiently support basic tasks like pick-and-place [1, 51]. they cannot capture the fine-grained contact and elastomer deformations essential for complex manipulation tasks [66, 12]
39th Conference on Ncural Information Processing Systems (NcurIPS 2025).

---

and detailed force distribution analysis [40, 47]. Soft-body simulations offer higher fidelity but face significant computational challenges that limit their practical application in large-scale experiments.
An ideal VBTS simulator must simultaneously achieve:
Precise modeling of robots, sensors, and ob- jects with physically valid solutions, particularly main- taining inversion-free and intersection- crrssecttoo-freeeeee states dur- ing complex contact interactions; generation of real- istic tactile signals across multiple resolutions, from high-resolution RGB patterns and depth maps to low- resolution marker movements.
Precision:
Scalability: Capability for large-scale parallelization for extensive simulation data generation.
Flexibility: Support for diverse robotic platforms and sensor configurations, from parallel grippers to multi- finger hands with varying sensor arrangements.
Mabjong Tile
Stomlatedd
Figure 1: Taccel demonstration of tactile robotics simulation. An Allegro Hand with four VBTSs performing pre- cision grasp on a mahjong tile. The de- formation map precisely captures the tile's surface geometries.
As detailed in Tab. 1, existing solutions often compromise on precision, scalability, or flexibility. They typically pro- duce suboptimal physics, operate slower than real-time with limited parallel environments, or focus on specific sensor setups simple grippers. These limitations significantly impede the broader application of tactile robotics.
To address these challenges, present Taccel, high- high-performance simulation platform for scal- ing up robots with VBTS-integrand vides dedicated components for simulating robots (Sec. 4), tactile sensors (Sec. 4.2), and tactile signal generation (Sec. 4.3). Comprehensive evaluations (Sec. 5) demonstrate its characteristics:
Built on wate-of-the-aat simulation techniques, Taccel pro-
Taccel leverages advanced solid material simulation techniques (IPC [32] and ABD [31]) for physical accuracy. IPC guarantees inversion- and intersection-free contact solu- tions, while the integration of ABD allows for efficient and precise simulation.
Precision:
Scalability: With an efficient ABD-IPC implementation with NVIDIA Warp [41]. Taccel achieves unprecedented parallelization. On single H100 GPU, it reaches over 900 FPS in to- tal (4096 environments, 18x x wallclock time) for a peg- ingeinnertion task with dual sensors and 12.67 FPS (256 environments, 0.25x wallclock time) in dexterous grasping task with full-hand tactile sensing.
Flexibility: Taccel provides user-friendly APIs for seamless integration of diverse robotic plat- forms and sensor configurations. Users can easily load and configure robots through Unified Robot Description Format (URDF) with auxiliary configurations, supporting applications from simple grippers to complex manipulation tasks, like the mahjong tile sensing task in Fig. 1.
Taccel's 's utility is validated through three fundamental tactile-informed robotic tasks (Sec. 6). In object classification, models trained solely on Taccel's 's synthetic tactile signals demonstrate strong generalization to real-world data without adaptation. In grasping experiments across four robotic hand designs, we showcase the platform's 's versatility in handling diverse robot configurations and
Table 1: Comprehensive comparison of FEM-based VBTS simulators. Soft Mat.: modeling of deformable materials (FEM: Finite Element Methods). Stiff Mat.: modeling of stiff materials (Rigid: rigid body, ABD: Affine Body Dynamics, MPM: Material Point Methods, PBD: Position-basee Dynamics). Contact: collision handling method (Virtual: approximated contact, Penalty: penalty-based, IPC: Incremental Potential Contact). RGB Signal: RGB tactile pattern generation method (Look-up: look-up tables, DNN: Deep Neural Network, X: not supported). Robot: range of supported robotic systems. The last two columns report parallel simulation capabilities: maximum parallel environments and simulation speed relative real-time in peg insertion test, with dual sensors in low/high resolutions, measured on an NVIDIA H100 80G GPU, Details are in Fig. 4.
Simulator
Soft Mat.
Stiff Mat.
Contact
RGB Signal
Robot
# Env f
Sim Speed
Taxim [46]
Rigid
Virtual Penalty
Look-up
Sensor
Diffaattiee [47]
FEM
MPM/PBD
DNN
Gripper
SAPIEN-IPC [8]
FEM
ABD
IPC
Gripper
256 /4
0.81 x.033x
Taccel (Ours)
FEM
ABD
IPC
DNN
Any
4096 / 64
18.30 0.25%

---

tactile signal types. In articulated object manipulation tasks, we demonstrate Taccel's 's physical fidelity through close correspondence between simulated and real-world robot behavior.
Our key contributions include: (i) development of a high-performance simulation platform combin- ing precise physics, realistic tactile signal generation, and massive parallelization; (ii) user-friendly APIS enabling flexible robot-sensor integration and high-fidelity tactile signal synthesis; (iii) com- prehensive evaluation of the platform's 's precision and scalability; and (iv) extensive experimental validation across diverse tactile robotic tasks. By enabling large-scale, high-fidelity simulation of VBTS eqTSpppd robots, Taccel aims accelerate future research tactile robotics.
2 Related Work
Early VBTS simulators focused on normal deformation scenarios, approximating hyperelastic be- havior through geometric computations and surface modifications [17, ., 51, 56, 1, 1,4. While efficient in generating high-resolution tactile signals through physics-based basses rendering or look-up tables, these approaches inadequately capture elastomer dynamics during complex manipulation tasks, especially those involving tangential forces and continuous interactions [66].
Recent approaches have achieved higher physical fidelity by incorporating advanced solid material simulation techniques, Methods using Material Point Methods (MPM) [49, 22, 23] and Finite Ele- ment Methods (FEM) [32, 31] better model elastomer properties through time-integrated deforma- tion computations [10, 8, 47, 12J. Notable improvements include the adoption of IPC [32] by several simulators [12, 8]. providing robust contact handling with guaranteed inversion- and intersection- free solutions. Tab. compares key features of representative approaches.
Taccel builds these advances combining IPC and ABD in unified platform, achieving both physical accuracy and computational efficiency while supporting diverse robot configurations and enabling large-scale parallel simulation for robot learning applications.
A more comprehensive review on related works in Sec. B concerning the VBTSs and tactile- informed robotic tasks.
3 Unified IPC Simulation in Taccel
This section presents the unified IPC simulation framework Taccel, detailing its mathematical foundations. For complete derivations, refer readers Sec. and the original works [31, 32, 9].
3.1 Problem Formulation and Soft Body Dynamics
We consider ng tetrahedralized soft bodies discretized into N, vertices with positions 31, 2..... .... NN,, in Cartesian space. The system state is represented by the stacked position vec- tor x = [o] V2 ".. J" ER'N,,, Following Lagrangian mechanics, we express the system's La- grangian as x) x) (x),, where T'x, x) = } 1=xx MX represents kinetic energy with mass matrix M 3R3,, x3N. The potential energy V(x) comprises two terms: elastic energy
(x) utilizing the Neo-Hookean constitutive model for hyperelastic materials (characterized by Young's modulus & and Poisson's 's ratio v), and external forces (xx(
3.2 Frictional Contact
The Euler Lagrange equation 2 (xx x) tal Potential (IP) energy minimization problem [32] under the backward Euler integration scheme:
26 (x, x) 0 equivalent the following Incremen- ex
(x) } (x-xx - MMxx) - - Atx*) (x)(
a+1 arg min Fxp(x),,
where time is discretized into steps (4n nAl: ne N) with fixed step step size Al > 0, and x(Ix).
To ensure intersection-free trajectories, IPC augments the objective with log barrier functions b( dg (x)), which diverge to +00 as the distance (x) between contact primitive pair & approaches
3

---

zero. Additionally, IPC introduces an approximate frictional potential energy D& (x,xx"") that cap- tures frictional forces through its gradient. The full simulation thus minimizes the IPC energy:
Exp((x)= Exp(x) At B(x)+ + At D(,
with (xx) R Eken (x)( x(x,x") Eker (x x"), where 0 controls contact stiffness.
3.3
ABD and Unified Simulation
For a affine bodies, we introduce reduced coordinate space y 122 with embedding map d: R12 - R that projects reduced coordinates to full-space vertices o(y) [31], where Na denotes the total vertex count of affine bodies' surface meshes. Each affine body uses 12 Degree of Freedom (DoF): three for translation (IR3) and nine for affine deformation 3xx33).
1 T(y. 5) 5 MX ( ( (Jy) -5 (J" 5yyMM'
1
1
1
(3)
σy e R 3N, 12n, is the Jacobian, M is the full-space matrix, and M = JJ the reduced-space mass matrix. The potential energy V(y) includes an As-Rigid-As-Possinn (ARAP) term d (y) 4x (o(x)) with large K₂ limit deformation, plus external forces (x)
where J = 06
Combining with Eq. (2) yields the unified IPC energy [9] for the full system state {y:x} e 122ng + +3N.
Expcly; Exp(x) + Eyp At Atd A o((")) ((y")
Ey 5 - Aly") -1"
The next timestep's configuration follows from minimizing the following barrier-augmented IP: , x= arg min (y;
(5)
yx
3.4 Kinematic Constraints
We express kinematic constraints as S*x s and Sty sY where S E R x3N, 8 R3N, for soft bodies, and S E p x12m, sd E 12 for affine bodies. To enforce the constraints, we augment Eice using the Augmented Lagrangian method with Lagrangian multipliers X E A e R
(y: Eipc x)+ x -- -"))
(6)
4 Robot and VBTS Simulation in Taccel
Building upon the unified ABD-IPC, Taccel implements robot and VBTS simulation through modular design that leverages the complementary strengths of affine- and soft-body dynamics.
4.1 Robot and Sensor Simulation
Robot Modeling For robot with D-DoF, L links, and N vision-based tactile sensors (VBTSs), Taccel constructs its kinematic model from a URDF specification, loading visual and collision meshes as affine bodies (Sec. 3.3). Given the robot's global transformation T, and joint configuration E Q, forward kinematics yields the transformation of each link j in the world frame, 1, (T)
Tactile Sensors Modeling Each VBTS simulated by tetrahedral mesh as a soft volumetric body (Sec. 3) for its gel pad, attached to its corresponding robot link. For the i-th sensor's gel pad G, attached to link I with local transformation T, , we denote its outer surface S, 2g. The surface comprises a reflective-coated region G. and a sensor-attacher region à G. Contact interactions during simulation cause gel pad deformation, transforming the coated surface + to a+ & and marker positions to P, These deformed quantities serve as the foundation for generating multiple types of tactile signals, each suited for different robotic applications.

---

PyRullet
SAPIEN
Tarcel (Ours) wio ABD
Taccel (Ours)
Wrong callisson
Wzong collision
29K modes (bolt and nut)
6k nodes (bolt and nut)
(a) bolt-andd-uut test
DiffTactile
Taccel (Ours)
SAPTEN-IPC
Taccel (Ours)
(b) soft block pressing test
Isaac Sim Implementation
Taccel (Ours)
16
Insceurate physies rigid
physies body sim
covs
cues
(c) an articulated object manipulation task
(d) multi-envirrnmeent paraliellization test (low-res high-res)
Figure 2: Comprehensive evaluation of physies simulation capabilities across VBTS simulators. (a) Bolt-
nut assembly involving contact between non-convex objects, where Taccel achieves stable simulation; (h)
Soft block pressing test with soft-soft contacts, where Taccel maintains cenetration-fres interactions; (c)
Tactile-informed articulated object manipulation, where Taccel replicates real-world interactions with 1%
physical error; (d) Parallel environment test using peg insertion task, showing Taccel's 's scalability.
Robot Actions For scene initialization, we compute affine states through forward kinematics for robot links and explicit state specification for stiff objects. Gel pad node positions are transformed to the world frame, with all states written x, y, and velocities x, ý initialized zero. Robot actions are implemented through kinematic constraints. From joint space targets, we compute affine state targets for links and node position targets gel pad attached surfaces Đ-G, assembled as s* s*. Selection matrices S* SY apply these constraints, with remaining states solved via time stepping.
4.2 Tactile Signal Simulation in Taccel
While many works directly generate tactile signals from the object geometry and frictional forces during rigid body simulation for efficiency [51, 56, 1]. this approach can lead to inauthentic signals in dynamic scenarios. Instead, Taccel supports accurate simulation of high-resolution soft bodies to fully capture the fine-grained contact and deformation patterns.
RGB Images and Depth Maps High-resolution tactile signals, including RGB images and depth maps, are essential when fine-gra the-grained details like object texture and local geometries are required. The signal generation process proceeds two stages. First, we extract the depth and normal maps d(u.e); ())))) for pixel coordinates (u, u) from the deformed coated surface 2+ Next, following the method of Si et al. [47], we apply Deep Neural Network (DNN) to generate RGB tactile signals from the depth information. Specifically, for each pixel coordinate (u, v). a pixel-to-pixel DNN parameterized by 0 maps the inputs to the pixel colors relative to reference image (RGB signal of undeformation gel pad): fo (7(,, v): P((u,))) - (u.r). which are added to reference image to obtain the final RGB image. 7(,;) :) provides the 2D positional encoding of the pixel coordinate. The model is trained on patches from 200 real tactile images and corresponding depth map annotations.
Markers Within GGG. m, markers
positioned at locations G: G.P: (pk R
1,.., my J each defined by barycentric coordinates in its triangle: Pt: = - 0, (i.k)
(i.k)
where E 0n =, [0, Low-resolution tactile signals primarily track marker positions and their movements by computing their new positions throughout the simulation (at time step t): ((i) (t) and projecting them on the tactile image. The marker flows, representing local deformation patterns, then computed as: AP;
Ed ,,,,,, (t.b)) (1),
(pt
Pk (i) (Ap
Tactile Signals The depth map and marker positions can transformed into dense or sparse 3D point cloud in the world frame using robor kinematics and sensor configurations. These 3D tactile signals provide crucial spatial information for robotic manipulation tasks. We demonstrate their effectiveness through the simulation of Tac-Man framework [66] in Sec. 6.3.

---

We further demonstrate Taccel's capability to scale up synthetic data generation of tactile signals via robotic grasping simulations and explore object recognition model learning (Secs. 6.1 and 6.2).
4.3 API Designs in Taccel
Taccel provides intuitive Python APIs designed to make tactile robotics simulation accessible to researchers while maintaining high performance through NVIDIA Warp [41]. The APIs allows for seamless loading of robots from URDF files with automatic parsing, sensor configurations from auxiliary files, and objects from mesh files. Users can efficiently reset simulation states or apply kinematic targets to control the robots in familiar formats (NumPy arrays, PyTorch tensors). Further, Taccel supports parallel simulation of multiple environments similar Isaac Gym [43].
To foster community development, we will release Taccel codebase and documentations, while maintaining active collaboration with researchers to incorporate feedback, add features, and expand capabilities, ensuring its evolution as comprehensive tool for tactile robotics research.
5 Performance Evaluation of Taccel
We evaluate Taccel through comprehensive benchmarks on its precision and efficiency. Our analy- sis demonstrates that the combination of ABD and IPC provides significant advantages over existing approaches (Sec. 5,1), generates high-quality tactile signals (Sec. 5.2), ensures precise frictions and deformation solving for the gel pad (Sec. 5.3), and enables efficient scaling (Sec. 5.4).
5.1 Overall Comparisons
As illustrated in Fig. 2, Taccel achieves superior precision and efficiency through its unified ABD and IPC framework, demonstrated across four challenging scenarios.
First, the bolt screwing task (Fig. 2a) demonstrates Taccel's 's ability handle complex physical interactions between highly non-convex objects, where conventional simulators including PyBullet and SAPIEN [55] fail to handle. This capability stems from our ABD formulation, which provides an efficient approach to simulating dense rigid-soft body interactions. Alternative approaches either model stiff objects as soft bodies, introducing excessive DoFs and computational overhead (Fig. 2a, ours w/o ABD), or rely on RBD [14], which requires expensive nonlinear Continuous Collision Detection (CCD) calculations to avoid intersection, significantly degrading performance.
Next, physical realism in Taccel is demonstrated through the soft block pressing scenario (Fig. 2b), where collision-fre and intersection frre guarantees produce notably more realis- tic deformations compared to penalty-based approaches. This precision extends to practical robotics applications, shown in the Tac-Man microwave manipulation task (Fig. 2c). Here, Taccel's 's accurate contact force solving enables faithful reproduction of gel pad-object handle interactions, closely matching real-world execution patterns (detailed analysis in Sec. 6.3).
Finally, the computational efficiency of Taccel emerges from our optimized implementation of the ABD and IPC algorithms, enabling unprecedented scaling capabilities. In peg insertion task, Taccel achieves parallel simulation of over 4096 environments on a single GPU with 80GB VRAM represetnnnn 64-fold improvement over SAPIEN-IPC [8]. This opens possibilities for large-scale robotics simulation and learning; see also the comprehensive analysis in Sec. 5.4.
5.2 Tactile Signal Simulation
We evaluate the fidelity of tactile signals generated by Taccel with real-world samples. We press calibrated GelSight-type sensor perpendicullarly on 18 objects from a standard tactile shape testing dataset [16] on a real-world setup (Fig. 3(a)) and in Taccel; see Sec. D. for setup details.
The qualitative and quantitative comparisons shown Fig. 3(a) demonstrate Taccel's 's ability to produce highly realistic tactile patterns, achieving an average SSIM of 0.93 across all test objects. Minor variations between simulated and real signals primarily stem from manufacturing tolerances in the 3D-prin 3D-printed objects and challenges in precise camera calibration. Despite these practical limi- tations, the results establish Taccel's 's capability to generate high-fidelity tactile signals suitable for VBTSs, with simulated patterns closely matching experimental measurements.

---

[bJ
0.95
fixed
20-00 Zmn
grasp & puil at 2mmn/é
Avg Li err:
fixed
0.028mm
02
04
12
0.4mm
1, d 0.5m
Bar 0.mm
Figure 3; Evaluations the simulation precision. (a) Data collection setup, examples of the real and simu- lated tactile patterns, and sim-real SSIM distribution (violin plot) in the tactile signal ovaluation. (b) Data collection setup and the shear deformation magnitude trajectories in the frictions simulation evaluation.
+ Lowvers Prg
High Pag. *6 Deat Hand Did converge
Turrel (Cung)
SAMIEN-PC
Real wolleleek (Ar
Low-res High-
10
10
F75
10
Number of environments
Peg Insertion Test Dexterous Hand Test
Figure Parallel simulation performance analysis across environment scaling, (a) FPS achieved by Taccel (FP64) and SAPIEN-IPC (FP32) on an NVIDIA H100 80G GPU. (b) Task visualizations. Sec. C. reports the VRAM occupancy in the low-resolution peg-insertion test.
5.3 Precision on Frictions and Shear Deformation
We further investigate Taccel's precision on solving frictions and shear deformation. Fig. 3(b) shows the evaluation setup: A VBTS--ripppr grasps a fixed bar with various forces (causes tactile depth d) then pulls back at mm/s, recording gel deformation represented by the marker deforma- tions. We calibrate the object's friction coefficient with one record and use two others with various ds compare the errors between simulated and real-world trajectories. Small sim-real errors (avg 28 um) tested on two bars with various frictions highlights Taccel's 's fidelity on gel deformation and frictions in both static and slipping cases.
5.4 Multi-cnviromment Simulation
Efficient and stable parallel simulation of multiple environments is crucial for scaling up synthetic data collection across diverse downstream tasks. To evaluate Taccel's 's capabilities in this regard, we designed three test cases of increasing complexity: (i) dual-sensor (139 nodes per gel pad) peg-insertion task adapted from SAPIEN-IPC [8], (ii) the peg-insertion task with higher solution (1.5k nodes per gel pad), and (iii) a grasping task using a customized five-fingered dexterous hand equipped with 17 gel pads wrapped around the hand links (5k nodes). These tasks involve continuous contact and gel pad deformation, shedding light on how the simulator would perform on various robotic tasks. Sec. D.2 provide more details on these tasks.
With a single NVIDIA HI00 80G GPU, benchmarked Taccel against SAPIEN-IPC. The results in Fig. 4 showcase Taccel's superior performance: in the low-resolution peg-insertion task, Taccel achieves 915 FPS (18.30x wallclock time) while managing over 4096 paral- lel environments abled by our efficient parallelization implementation. The high-resolution test further demonstrates Taccel's 's advantages, maintaining both stability and precision while consistently outperforming the baseline. Even the complex dexterous hand scenario, Taccel efficiently handles full-hand tactile sensing, achieving 12.67 FPS across 256 environments.
16-fold improvement over the baseline using the same GPU memory, en-

---

(a)
b synthetic
View
shank_tap
ReaNet-18
LegSotMax
onchor
Frome View
Figure S: Object classification experiment setups. (a) mechanical parts selected, each with diverse and fine-grained geometries, rendered from the top and front. (b) Synthetic and real-world data collection setup. (c) A small ResNet is learned to perform object classification.
We observed that SAPIEN-IPC'S use of FP32 precision leads to convergence issues when solving contact forces in Eq. (2), particularly in the logarithmic barrier energy term calculations, as indi- cated by red outlines in Fig. 4. We further investigate this gap in the low-resolution peg-insertion benchmark. We set the maximum Newton iterations to and optimization residue tolerance set to 0.01m/s, iee the IPC solver performs conjugate gradient descent step until (i) the 50-step limit is exceeded, or (ii) the optimization residue is below 0.01m/s. We report the key quantities including the Newton iterations used, its optimization residue, total time spent for the simulation, and total FPS in Tab. 2. Although each step takes longer for FP64, the convergence is much faster due be the better precision, and thus the better simulation speed.
Noteworthy, while Taccel's 's FP64 precision is ideal for HPC GPUs (which have a high 1:2 FP32:FP64 FLOPS ratio), it remains highly performant on more accessible consumer cards like RTX 3090 and 4090; see Sec. C.2 for more details.
Table Simulation Performance Comparison. Results are grouped by the number of parallel environments. The number of Newton iterations (# Newton Its.) and optimization residue (Optim. Residue) are reported as with standard deviation (in parentheses), calculated across the 200-step simulation.
Simulator
Envs
# Newton Its.
Optim. Residue / (m/s) Time
Total FPS
Taccel (FP64)
1.59 (=1.13)
7.64 10-5 (+6.55 x 10-5)
69.78
2,87
SAPIEN- (FP32)
24.09 (+:00.5)
0.001 (#0.01)
82.41
2.43
Taccel (FP64)
16
2,83 (+: 2.57)
1.87 10 (t:2.09 10-
141.92
22.55
SAPIEN-IPE (FP32)
16
50.0 (+000)
0.11 (#0.44)
291.46
10.98
Taccel (FP64)
64
3.45 (#3326)
2.0 10 x
196.71
65.07
SAPIEN-IPC (HP32)
64
50.0 (=00.)
0.20 (=0.44)
668.95
19.13
6
Applications in Tactile Roboties
We demonstrate Taccel's 's capabilities across three tactile-informed robotic tasks: training object classification models with synthetic data (Sec. 6.1), generating a large-scale dataset through parallel grasp simulations (Sec. 6.2), and manipulating articulated objects (Sec. 6.3). These applications showcase how Taccel enables precise robotic simulation and scalable synthetic data generation.
6.1 Learning Object Classification Models
To demonstrate Taccel's 's ability generate synthetic training data that generalizes to real-world scenarios, we developed a tactile-base object classification system. As shown in Fig. 5, our ap- proach trains DNN to classify objects using high- high-resolution tactile signals.
Table 3: Object classification and sim-to-real performance. Performance metrics include number tactile sensors (N), Degree of Freedom (DoF), total sensing area (S.A.), average sensor-object contact area (C.A.) with percentage of total sensing area in parentheses. and classification accuracy (Acc.).
(a) Sim-to-real results on mechanical part recognition.
(b) Results on various robotic hand configurations.
Data
Dol'
N
/.A./mm²
C.A./ cm?
Acc.
Data
Dof
S.A. / cm
C.A. / cmu
Acc.
Mech.
2
32.0
4.77 (14.92%)
86.50%
Gripper
32.0
5.05 (15.78%)
44.56%
Moch. (Real)
2
2
32.0
5.69 (17.78%)
70.94%
Rabotiq 3F
27.0
4.98 (18.43%)
44.61%
Allegro
23.0
7.67 (33,34%)
54.30%
F-TAC [65]
15
17
59.7
4.00 (6.700%)
42.54%
8

---

(b) simulated 3D tactile signal Figure 6: Examples of the synthesized grasps and the simulated tactile signals.
simulated 2D tactile signals
Following Yang et al. [58], we selected 10 mechanical parts with distinct fine-grained geometries (illustrated in Fig. top). We collect training dataset 4K tactile depth maps of tactile sensor pressing on them within Taccel, each with object pose and tactile depth randomly sampled. We also collect real-world test set that consists of 160 depth maps for testing. Sec. D.3 illustrates more details for the data collection protocol. We trained a ResNet-18 model [20] for 10-category object classification using the simulated depth images. To enhance robustness, we augmented the depth maps through random affine transformations, morphological operations (erosion and dilation). and Gaussian filtering. The model is then directly evaluated them on the real-world samples, with each sample tested 4 times with random shear deformation. As shown in Tab. 3, our model achieved 86.50% accuracy on the synthetic test set and 70.94% on real-world samples without any domain adaptation. This modest sim-to-real gap demonstrates Taccel's 's capability to generate precise tac- tile signals that enable data-efficient training of transferable tactile perception models.
6.2 Robotic Grasping with Tactile Sensors
We investigate robotic grasping across different hand configurations with varying tactile sensor ar- rangements. We first extend the DFC algorithm [39] for generating contact-oriented grasping poses for four robotic hands, then simulate their tactile responses within Taccel, as illustrated in Fig. 6. We generate grasps on 10 diverse objects from ContactDB [4], YCB [7], and adversarial object [42] datasets. For each object, we generated grasps using 4 different robotic hands, producing ~ 14k total grasps. These were simulated in Taccel to generate tactile signals, with key metrics summarized in Tab. 3. Fig. Al provides an additional visualizations for the grasps across 4 robotic hands. Sec. D.4 explains the details of the modification to DFC.
Simulating robotic grasps with tactile sensors yields synthetic data extending the existing robotic grasping datasets with tactile perception capabilities, serving foundation for various robotic tasks. To demonstrate its utility, we implemented the object classification task described in Sec. 6.1, adapting it to use the deformed coat's point cloud as input and PointNet as the feature extractor.
The classification performances in Tab. 3 reveals an interesting trade-off: while robots differ in sensor count and sensing area, higher dexterity (Allegro Hand) enables better object contact de- spite fewer sensors. This enhanced contact leads to superior classification accuracy, highlighting the balance between sensor count and dexterity in tactile hand design. These findings demonstrate Taccel's 's value in validating robotic hand designs before physical fabrication.
6.3 Articulated Object Manipulation
We consider articulated object manipulation, challenging task where tactile perception provides critical feedback on hand-object contact, informing object articulation and guiding robot actions [26, 66). Sec. D.5 detailedly explain the algorithm.
Tac-Man's effectiveness relics heavily on gel pad deformation for motion adaptation and tactile feedback. While the original implementation by Zhao al. used rigid body simulation with gripper compliance approximations, couldn't authentically replicate gel pad deformation, contact dynam- ics, and tactile feedback, resulting in significant sim-to-real gaps during large-scale verification. Taccel overcomes these limitations through accurate simulation sensor-object contact and gel pad deformation. We simulate Tac-Man three types of articulated objects: drawers with pris- matic joints, cabinets with revolute joints, and bolt-nut pairs with helical joints. Fig. 7(a) shows the manipulation sequences and corresponding tactile signals.
To evaluate sim-real correspondence, we compared our simulation against real-world Tac-Man ex- ecution using microwave (revolute joint) and drawer (prismatic joint), shown in Fig. 7(b-c). For

---

comprehensive comparison, we also tested Tac-Man's official implementation these objects. We use the ceccution-recovery switch count" as our key metric, which tallies how often the agent switch between the execution and recovery state when executing the Tac-Man algorithm (see Zhao al. [66] and Sec. D.5 for details). As demonstrated Fig. 7, Taccel faithfully reproduces real-world execution patterns, with execution-recovery switch counts averaging 68.75 and 0.0 for revolute and prismatic settings, Isaac Sim successfully simulated the manipulation, its dynamics gap resulted in substantially higher switch counts. These results highlight Taccel's capability to authentically replicate physical inter- actions in manipulation scenarios.
reriiivelly--mrrabbll.l. matching real-world observations. While
markers
marker flow (8x length)
7 Summary
Drawer (Prismatic)
Cabinet (Revolute)
Bolt and Nut (Helical)
We present Taccel, a flexible and high-performance simulator for VBTS-integrated robots. Its user- friendly APIs enables precise and ef- ficient simulation of complex tactile robotic tasks with realistie tactile sig- nals. Taccel excels in capturing in-
recovery
recovery
(a) Tac-Man simulation three types of articulated objects
Real world
Isaae Sim Implementation
Ours
tricate deformation and contact dy- namics of soft gel pads with un- precedented stability, while support- ing thousands of parallelized environ- ments. These capabilities also posi- tion Taccel as a powerful tool for hand-sensor validating before fabri- cation, potentially reducing develop- ment time and costs.
recovery
exccution
microwave oven
switch 68
switch 500
switch 68.75 (=2.38)
Limitations Despite the high per- formance of Taccel, its computa- tional demands of large-scale simula- tion remain challenging, with jor bottleneck being the PCG-based linear system solving. Possible reme- dies include carefully relaxing con- vergence tolerances, simplifying sim- ulation protocols, and using larger timesteps, enabled by IPC's uncon- ditionally stable solver. Besides, the Neo-Hookean model itself may still be insufficient in capturing all nec- essary physical characteristics of gel pad. Further, the data-driven tac- tile signal generation method is still trained from limited data and thus may not generalize to all contact conditions, These limiations are to be addressed in future studies.
exemition
drawer
switch
switch 18.25 (11.49)
switch 0.0 (=0)
(b) comparisons among real-world execution, Isaac Sim simulation, and Taccel simulation
Figure 7: Tac-Man manipulation simulation. (a) Demonstra- tion of Tac-Man's exccution-recoveet cycles on three articu- lated objects. Three-way comparison among real-world ccution, Isaac Sim implementation from Zhao et al. [66], and Taccel. The execution-recovery switch counts (#swwtch) demon- strate Taccel's 's accuracy in replicating real-world behavior (1.10% error), compared Isaac Sim's higher counts.
Broader Impact This paper focuses on fundamental research, with limited direct societal impact.
Acknowledgments and Disclosure of F Funding
This work is supported in part by the National Science and Technology Innovation 2030 Major Pro- gram (2025ZD0219402), the National Natural Science Foundation of China (62376009). the PKU- BingJi Joint Laboratory for Artificial Intelligence, and the National Comprehensive Experimental Base for Governance Intelligent Society, Wuhan East Lake High-Tech Development Zone. We thank Lei Yan (LeapZenith AI Research) for his support in mechanical engineering, Zhenghao Qi (PKU, THU), Sirui Xie (PKU) for their assistance setting up experiments, and Ms. Hailu Yang (PKU) for her help in procuring materials.
10

---

References
[1] Iretiayo Akinola, Jie Xu, Jan Carius, Dieter Fox, and Yashraj Narang. Tacsl: A library for Visuotactile sensor simulation and learning. IEEE Transactions Robotics, 2025. 1,3,5
[2] Maria Bauza, Antonia Bronars, and Alberto Rodriguez. Tac2pose: Tactile object pose esti- mation from the first touch. International Journal of Robotics Research (IJRR), 42(13):1185- 1209, 2023. A3
[3] Aude Billard and Danica Kragic. Trends and challenges in robot manipulation. Science, 364(64466)caat84444 14, 2019, A3
[4] Samarth Brahmbhatt, Cusuh Ham, Charles C Kemp, and James Hays. Contactdb: Analyzing and predicting grasp contact via thermal imaging. In Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
[5] Antonia Bronars, Sangwoon Kim, Parag Patre. and Alberto Rodriguez. Texterity: Tactile ex- trinsic dexterity. In IEEE International Conference on Robotics and Automation (ICRA), 2024.
A3
[6] Roberto Calandra, Andrew Owens, Dinesh Jayaraman, Justin Lin, Wenzhen Yuan, Jitendra Malik, Edward H Adelson, and Sergey Levine. More than feeling: Learning to grasp and regrasp using vision and touch. IEEE Robotics and Automation Letters (RA-L., 3(4):3300- 3307, 2018. A3
[7] Berk Calli, Arjun Singh, Aaron Walsman, Siddhartha Srinivasa, Pieter Abbeel, and Aaron M Dollar. The ycb object and model set: Towards common benchmarks for manipulation re- search. In JEEE International Conference on Robotics and Automation (ICRA), 2015. 9
[8] Weihang Chen, Jing Xu, Fanbo Xiang, Xiaodi Yuan, Hao Su, and Rui Chen. General-purposs sim2real protocol for learning contact-rich manipulation with marker-based visuotactile sen- sors. IEEE Transactions on Robotics (T-RO), 40: 10:1509-1526, 2024. 1, 2, 3, 6, 7, A3, A4
[9] Yunuo Chen, Minchen Li, Lei Lan, Hao Su, Yin Yang, and Chenfanfu Jiang. A unified newton barrier method for multibody dynamics. ACM Transactions on Graphics (TOG), -144):1-14, 2022. 3, 4, A2
[10] Zixi Chen, Shixin Zhang. Shan Luo, Fuchun Sun, and Bin Fang. Tacchi: A pluggable and low computational cost elastomer deformation simulator for optical tactile sensors. IEEE Robotics and Automation Letters (RA-L), 8(3): 1246, 2023.
[11] Siyuan Dong, Wenzhen Yuan, and Edward H Adelson. Improved gelsight tactile sensor for measuring geometry and slip. In IEFE/RS International Conference Intelligent Robots and Systems (IROS), 2017. A3
[12] Wenxin Du Wenqiang Xu, Jieji Ren, Zhenjun Yu, and Cewu Lu. Tacipe: Intersection- inversion-free fem-based elastomer simulation for optical tactile sensors. IEEE Robotics and Automation Letters (RA-L), 2024. 1,
[13] Mark Edmonds, Feng Gao, Hangxin Liu, Xu Xie, Siyuan Qi, Brandon Rothrock, Yixin Zhu, Ying Nian Wu, Hongjing Lu, and Song-Chun Zhu. A tale of two explanations: Enhancing human trust by explaining robot behavior. Science Robotics, x(37):eayy4663, 2019. 1
[14] Zachary Ferguson, Minchen Li, Teseo Schneider, Francisca Gil-Ureta, Timothy Langlois, Chenfanfu Jiang, Denis Zorin, Danny M Kaufman, and Daniele Panozzo. Intersection-frre rigid body dynamics. ACM Transactions on Graphics (TOG), 40(4), 2021.
[15] Carlo Ferrari, John F Canny, et al. Planning optimal grasps. In IEEE International Conference on Robotics and Automation (ICRA), 1992. A3
[16] Daniel Fernandes Gomes, Paolo Paoletti, and Shan Luo. Generation of gelsight tactile images for sim2real learning. IEFE Roboties and Automation Letters (RA-I.), 6(2):4177-418 2021.
6, A4
11

---

[17] Daniel Fernandes Gomes, Achu Wilson, and Shan Luo. Gelsight simulation for sim2real learn- ing. In ICRA ViTae Workshop, 2019.
[18] Phillip L Gould and Yuan Feng. Introduction to linear elasticity, volume 2. Springer, 1994. A3
[19] Xiaofeng Guo, Hung-Jui Huang, and Wenzhen Yuan. Estimating properties of solid particles inside container using touch sensing. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2023. A3
[20] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for im- age recognition. In Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR), pages 770-778, 2016.
[21] Francois R Hogan, Jose Ballester, Siyuan Dong, and Alberto Rodriguez. Tactile dexterity: Manipulation primitives with tactile feedback. IEEE International Conference on Robotics and Automation (ICRA), 2020. 1, A3
[22] Yuanming Hu, Yu Fang, Ziheng Ge, Ziyin Qu, Yixin Zhu, Andre Pradhana, and Chenfanfu Jiang. A moving least squares material point method with displacement discontinuity and two-way rigid body coupling. ACM Transactions on Graphics (TOG), 37(4):1-14, 2018.
[23] Yuanming Hu, Jiancheng Liu, Andrew Spielberg, Joshua B Tenenbaum, William T Freeman, Jiajun Wu, Daniela Rus, and Wojeiech Matusik. Chainqueen: A real-time differentiable physi- cal simulator for soft robotics. In IEEE International Conference on Robotics and Automation (ICRA), 2019.
[24] Hung-Jui Huang, Xiaofeng Guo, and Wenzhen Yuan. Understanding dynamic tactile sensing for liquid property estimation. Robotics: Science and Systems (RSS), 2022. A3
[25] Bassam Jalgha, Daniel Asmar, Elie Shammas, and Imad Elhajj. Hierarchical fall avoidance strategy for small-scale humanoid robots. In 2012 IEEE International Conference on Robotics and Biominetics (ROBIO), 2012.
[26] Carlos A Jara, Jorge Pomares, Francisco A Candelas, and Fernando Torres. Control frame- work for dexterous manipulation using dynamic visual servoing and tactile sensors' feedback. Sensors, 14()):1877-1844 2014. 1,9
[27] Yousef Javanmardi, Huw Colin-York, Nicolas Szita, Marco Fritzsche, and Emad Moeendar- bary. Quantifying cell-generated forces: Poisson's ratio matters. Communications physics, 4(1):237, 2021. A3
[28] Chunpeng Jiang, Wenqiang Xu, Yutong Li, Zhenjun Yu, Longehun Wang, Xiaotong Hu, Zhengyi Xie, Qingkun Liu, Bin Yang, Xiaolin Wang, et al. Capturing forceful interaction with deformable objects using a deep learning-powere stretchable tactile array. Nature Com- munications, 15(1):9513 2024. A3
[29] Roland Johansson and J Randall Flanagan. Coding and use of tactile signals from the finger- tips in object manipulation tasks. Nature Reviews Neuroscience, 10(5):345-359 2009. A3
[30] Sangwoon Kim, Antonia Bronars, Parag Patre, and Alberto Rodriguez. Texterity-tacteo ex- trinsic dexterity: Simultaneous tactile estimation and control for extrinsic dexterity. In IEEE International Conference on Robotics and Automation (ICRA), 2024.
[31] Lei Lan, Danny M Kaufman, Minchen Li, Chenfanfu Jiang, and Yin Yang. Affine body dy- namies: fast, stable and intersection-fies simulation of stiff materials. ACM Transactions on Graphics (TOG), 41(4):1-14, 2022. 2, 3, 4, A2
[32] Minchen Li, Zachary Ferguson, Teseo Schneider, Timothy R Langlois, Denis Zorin, Daniele Panozzo, Chenfanfu Jiang, and Danny M Kaufman. Incremental potential contact: intersection- and inversion-free. large-deformation dynamics. ACM Transactions on Graphics (TOG), 39(4):49, 2020. 2, 3, A.
12

---

[33] Mingxuan Li, Lunwei Zhang, Yen Hang Zhou, Tiemin Li, and Yao Jiang. Easycalib: Simple and low-cost in-situ calibration for force reconstruction with rision-based tactile sensors. IEEE Robotics and Automation Letters, 2024. A3
[34] Rui Li, Robert Platt, Wenzhen Yuan, Andreas Ten Pas, Nathan Roseup, Mandayam A Srini- vasan, and Edward Adelson. Localization and manipulation of small parts using gelsight tac- tile sensing. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2014. A3
[35] Wanlin Li, Zihang Zhao, Leiyao Cui, Weiyi Zhang, Hangxin Liu, Li-An Li, and Yixin Zhu. Minitac: An ultra-compact mm vision-based tactile sensor for enhanced palpation in robot-assisted minimally invasive surgery. IEEE Robotics and Automation Letters (RA-L), 9(12) 177, 2024. 1, A3
[36] Yuyang Li, Bo Liu, Yiran Geng, Puhao Li, Yaodong Yang, Yixin Zhu, Tengyu Liu, and Siyuan Huang. Grasp multiple objects with one hand. IEEE Robotics and Automation Letters (RA-L), (15):422-40344 2024. A3
[37] Changyi Lin, Han Zhang, Jikai Xu, Lei Wu, and Huazhe Xu. 9dtact: A compact vision-based tactile sensor for accurate 3d shape reconstruction and generalizable 6d force estimation. IEEE Robotics and Automation Letters (RA-L), 2023. 1, A3
[38] Hongbin Liu, Juan Greco, Xiaojing Song, Joao Bimbo, Lakmal Seneviratne, and Kaspar Al- thoefer. Tactile image based contact shape recognition using neural network. 2012 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), 2012.
[39] Tengyu Liu, Zeyu Liu, Ziyuan Jiao, Yixin Zhu, and Song-Chun Zhu. Synthesizing diverse and physically stable grasps with arbitrary hand structures using differentiable force closure estimator. IEEE Robotics and Automation Letters (RA-L), 7(1 7(1):470-477, 2021. 9, A3, A5
[40] John Lloyd and Nathan F Lepora. Pose-and-shear-baaeee based tactile servoing. International Journal of Robotics Research (IJRR), 43(7):1024-1055, 2024. 2, A3
[41] Miles Macklin. Warp: A high-performance python framework for gpu simulation and graph- ics. httpp:://gt Conference (GTC). 2,
http:/ciia//i////oo//oomm// March 2022. NVIDIA GPU Technology
[42] Jeffrey Mahler, Jacky Liang, Sherdil Niyaz, Michael Laskey, Richard Doan, Xinyu Liu, Juan Aparicio Ojea, and Ken Goldberg. Dex-net 2.0: Deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics. In Robotics: Science and Systems (RSS), 2017.9
[43] Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, et al. Isaac gym: High performance gpu-based physics simulation for robot learning. arXiv preprint arXiv:2108. 10470, 2021. 6
[44] Juntian Qu, Baijin Mao, Zhenkun Li, Yining Xu, Kunyu Zhou, Xiangyu Cao, Qigao Fan, Minyi Xu, Bin Liang, Houde Liu, et al. Recent progress in advanced tactile sensing technologies for soft grippers. Advanced Functional Materials, 33(41):2306244) 2023. A3
[45] Yu She, Shaoxiong Wang, Siyuan Dong, Neha Sunil, Alberto Rodriguez, and Edward Adelson. Cable manipulation with tactive-reactive gripper. International Journal of Robotics Research (IJRR), 13) 2021. A3
[46] Zilin Si and Wenzhen Yuan. Taxim: An example-based simulation model for gelsight tactile
sensors. IEEE Robotics and Automation Letters (RA-L). 7()):3611-2388, 2022. 2.
[47] Zilin Si, Gu Zhang, Qingwei Ben, Branden Romero, Zhou Xian, Chao Liu, and Chuang Gan. Difftactile: A physics-based differentiable tactile simulator for contact-rich robotic manipula- tion. In Proceedings of International Conference on Learning Representations (ICLR), 2024.
3.
13

---

[48] Bruno Siciliano and Oussama Khatib, Robotics and the handbook. In Springer Handbook of Roboties, pages 1-6, Springer, 2016. A3
[49] Deborah Sulsky, Zhen Chen, and Howard L Schreyer. A particle method for history-deppendet materials. Computer methods in applied mechanics and engineering, 118(1-2):79--16, 1994.
3
[50] Sudharshan Suresh, Haozhi Qi. Tingfan Wu, Taosha Fan, Luis Pineda, Mike Lambeta, Jitendra Malik, Mrinal Kalakrishnan, Roberto Calandra, Michael Kaess, al. Neuralfeels with neural fields: Visuotactile perception for in-hand manipulation. Science Robotics, 9(6))ead106288 2024. A3
[51] Shaoxiong Wang. Mike Lambeta, Wei Chou, and Roberto Calandra. Tacto: A fast, flexible, and open-source simulator for high-resolution vision- tactile sensors. IEEE Robotics and Automation Letters (RA-L). 7(2):393039937, 2022. 3,
[52] Zhi Wang, Yuchen Mo, Shengmiao Jin, and Wenzhen Yuan. Doorbot: Closed-loop task plan- ning and manipulation for door opening in the wild with haptic feedback. In IEEE International Conference on Robotics and Automation (ICRA), 2025, A3
[53] Benjamin Ward-Ch Murd-Cherrier, Nicholas Pestell, Luke Cramphorn, Benjamin Winstone, Maria Elena Giannaccini, Jonathan Rossiter, and Nathan F Lepora. The tactip family: Soft optical tactile sensors with 3d- ad-printed biomimetic morphologies. Soft Robotics, 5(2):216-227, 2018. A3
[54] Goran Westling and Roland S Johansson. Factors influencing the force control during precision grip. Experimental Brain Research, 53:277-284, 1984. 1, A3
[55] Fanbo Xiang, Yuzhe Qin, Kaichun Mo, Yikuan Xia, Hao Zhu, Fangchen Liu, Minghua Liu, Hanxiao Jiang, Yifu Yuan, He Wang, et al. Sapien: A simulated part-based interactive envi- ronment. In Proceedings of Conference Computer Vision and Pattern Recognition (CVPR), 2020.
[56] Jie Xu, Sangwoon Kim, Tao Chen, Alberto Rodriguez Garcia, Pulkit Agrawal, Wojciech Ma- tusik, and Shinjiro Sueda. Efficient tactile simulation with differentiability for robotic manip- ulation. In Conference on Robol Learning (CoRL), 2023. 1, 3,5
[57] Wenqiang Xu, Zhenjun Yu, Han Xue, Ruolin Ye, Siqiong Yao, and Cewu Lu. Visual-tactile sensing for in-hand object reconstruction. In Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 1, A3
[58) Sanghoo Yang, Won Dong Kim, Hyunkyu Park, Seojung Min, Hyonyoung Han, and Jung Kim. In-hand object classification and pose estimation with sim-to-real tactile transfer for robotic manipulation. IEEE Robotics and Automation Letters (RA-L), (11):659-666 2023. 9,
A3
[59] Kunpeng Yao and Aude Billard. Exploiting kinematic redundancy for robotic grasping of multiple objects. IEEE Transactions on Robotics (T-RO), 3993) 2023, A3
[60] Zhenjun Yu, Wenqiang Xu, Pengfei Xie, Yutong Li, and Cewu Lu. Dynamic reconstruction of hand-object interaction with distributed force-aware contact representation. arXiv preprint arXiv:2411.09572 2024. A3
[61] Wenzhen Yuan, Siyuan Dong, and Edward H Adelson. Gelsight: High-resolution robot tactile sensors for estimating geometry and force. Sensors, 17(12)2772,, 2017. 1, A3
[62] Wenzhen Yuan et al. Tactile measurement with gelsight sensor. PhD thesis, Massachusetts Institute of Technology, 2014, A3
[63] Wenzhen Yuan, Rui Li, Mandayam A Srinivasan, and Edward Adelson. Measurement of shear and slip with gelsight tactile sensor. In IEEE International Conference on Robotics and Automation (ICRA), pages 304-311. IEEE, 2015. A3
[64] Kappassov Zhanat, Juan-Antonio Corrales, and Perdereau Veronique. Tactile sensing dex- terous robot hands-review. Robotics & Autonomous Systems, 74:195-220, 2015. 1
14

---

[65] Zihang Zhao, Wanlin Li, Yuyang Li, Tengyu Liu, Boren Li, Meng Wang, Kai Du, Hangxin Liu, Yixin Zhu, Qining Wang, al. Embedding high-resolution touch across robotic hands enables adaptive human-like grasping. Nature Machine Intelligence, pages 1-12, 2025. 1,8
[66] Zihang Zhao, Yuyang Li, Wanlin Li, Zhenghao Qi. Lecheng Ruan, Yixin Zhu, and Kaspar Althoefer. Tac-Man: Tactile-informan informed prior-free manipulation of articulated objects. JEEE Transactions Robotics (T-RO), 41:538-557, 2024. 1, 3, 5, 9, 10, A3, A4, A6
[67] Yixin Zhu, Tao Gao, Lifeng Fan, Siyuan Huang, Mark Edmonds, Hangxin Liu, Feng Gao, Chi Zhang, Siyuan Qi Ying Nian Wu, et al. Dark, beyond deep: A paradigm shift to cognitive ai with humanlike common sense. Engineering, 6(3):310-345 2020. A3
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction clearly state the claims made.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Section Sec. 7 includes a "Limitations" paragraph.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification asymptotic approximations only holding locally). The au- thors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the ap- proach. For example, facial recognition algorithm may perform poorly when image resolution is low images taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to ad- dress problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers dis- cover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Re- viewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
16

---

Justification:
The derivation of the unified IPC-ABD algorithm is included in Sec. 3.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should clearly stated or referenced in the statement of any theo- rems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be comple- mented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: The paper discloses the required information for the simulation system and the experiments. The codebase is ready and will be released upon acceptance for better reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, other means that are appropriate the research performed.
While NeurIPS does not require releasing code, the conference does require all sub- missions to provide some reasonable avenue for reproducibility, which may depend on the nature the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results way produce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which thors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
17

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The codebase and dataset are ready and will be released upon acceptance for better reproducibility.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please the NeurlPS code and data submission guidelines (https: minis..co
public/ gwwdee/ie/eeeg S CoddeSudeeSSe ss sinn Poncy) for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: /nips. cc/ cc/ppbllcccccccccc c/ i des Codeeubm ssi sisooooolii icyy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Most of the key details are provided within the text. Full details will be pro- vided within the code.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should presented in the core the paper to level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropri- ate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The simulation speed test in Sec. 5 (see Fig. 4) includes the error bars. We also report the standard deviation for the Tac-Man simulation in Fig. 7.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
18

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should prefer- ably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The paper includes necessary information on the computer resources.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: /tetts://neurppii httppublic/tthccsttttttt CaGuidee nes?
Answer: [Yes]
Justification: The research conducted in the paper conforms with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative so- cietal impacts of the work performed?
Answer: [Yes]
Justification: The broader impact is discussed in Sec. 7.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
19

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact spe- cific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitiga- tion strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper does now contain data or models with a high rist for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards allow for controlled use of the model, for example by re- quiring that users adhere to usage guidelines restrictions to access the model or implementing safery filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The authors of the assets are cited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
20

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paper thoode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors encouraged reach out the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documenta- tion provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, lim- itations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsoureing experiments and research with human subjects, does the pa- per include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not contain crowdsourcing experiments research with hu- man subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contri- bution of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, cura- tion, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not contain research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsoureing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equiva- lent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
21

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLMs are only used for editing the manuscript. Thus the declaration is not needed.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer our LLM policy (https: wtt////uiiggg meterrips..o cc/conferenneee////////ceooc 2025/LIN) for what should or should not be described.
22

---

A Unified ABD-IPC in Taccel
In Taccel, robot links are efficiently modeled as affine bodies to capture their primarily rigid mo- tion, while VBTSs are simulated soft bodies to accurately represent their deformation mechanics. This natural division allows Taccel to balance computational efficiency with physical accuracy while maintaining consistent contact handling through IPC.
We provide a more detailed derivation of the unified ABD-IPC algorithm underlying Taccel, as previously briefed in Sec. 3.
A.1 Soft Body Dynamics
Recall that we consider n, tetrahedralized soft bodies with N, vertices: # 222,.... &N,- The stacked position vector x [a] 22 ... #x,, 1 RRS, represents the system state, and thus the system's m Lagrangian is (xx, x) = (x, x) V (x) with the kinetic energy T(x, x) 1 Mă, given the mass matrix M 3 x3N, The potential energy (xx) composed of the elastic energy ((x) utilizing the Neo-Hookean constitutive model for hyperelastic materials (characterized by Young's modulus and Poisson's ratio v), and the external forces Fxx((), The elastic energy is defined as $(x)
Sa (xx)dx, where (x) denotes elastic energy density over the volume region Ω of all objects in rest configuration.
A.2 Time Stepping
Substituting L(x,*) into the Euler LagranLe equation 21 (x, x) d 2L (x, x) 0 yields the gov- eming dynamics:
dV (x).
MX
(A1)
We temporally discretize Eq. (AI) using backward Euler:
a++
M(x*+1 xn) dV
2+1
(xx+1),
(A2)
At
At
dx
where time is discretized into steps {4n nA:: ne NJ with step size Al > 0, and x" =x(x). Under this discretization, Eq. (AI) can formulated as:
(Eir(x")) =0.
(A3)
If we define the incremental potential energy of the constrained system
Ex((x) = 2 x A1x"" x" -Atx")
(A4)
then the general simulation problem in a conservative system be reformulated the minimiza-
tion problem:
#+ arg min Fxp(x).
(A5)
A.3 Frictional Contact
We employ IPC [32] to handle contact interactions. The method operates on surface contact pairs B, comprising point-triangle and edge-edge pairs from the surface meshes soft and affine objects. For each contact pair & B with distance dy > 0, IPC defines two key energy terms, First, barrier energy that prevents interpenctration.
de à (d)- log( ( (
a)
bdd (x))
(A6)
where d O is the distance threshold for contact force activation and T(.) is the indicator function.
A1

---

Second, an approximated friction potential energy:
Da(x, x²) - Eu [o
(A7)
where x" represents the configuration at the previous timestep t, ΛE is the magnitude of the lagged normal contact force, and E R2 denotes the tangential relative displacement in the local contact frame.
The friction transition function fo(Ø) - Se (1 A. uses:
1-
122 EAR + c. ye (0, Atc.): 1,
(1(
(A8)
Atc..
where Eu > 0 serves as velocity threshold distinguishing between static and dynamic friction regimes. These contact and friction terms augment our incremental potential energy:
Eipc(x) (xx(x) At Dxx
(A9)
with B(x)= - EkeB (x)(dx(x)), D(x,x") LKER xx),,")), where x 0 controls contact stiffness.
A.4 ABD and Unified Simulation
For n. affine bodies, introduce reduced coordinate space y R 12n with embedding map o: 122== - 3 that projects reduced coordinates to full-space vertices Φ(y) [31], where N, denotes the total vertex count of affine bodies' surface meshes. Each affine body uses 12 DoF: three for translation (R$) and nine for affine deformation (R3x3).
T M
2
1 (
(A10)
1 y (J = - y
(J)) -
e 12 x 12m. the Jacobian, M is the full-space mass matrix, and M = J MJ where = 86 Ły the reduced-space mass matrix. The potential energy Vyy) includes an As-Rigid-Ass Ps-Possille (ARAP) term 4 (y) 45x (d(x)) with high stiffness K, to limit deformation, plus external forces Fy)
Combining with Eq. (A9), we obtain the unified affine-deformable coupled IPC energy [9] for the full system state xy: R 12ma +3N.;
Exp x) = Exp(x) BAA)
(A11)
+ ,4x
where Eyp(y) is defined as:
Eyp(y 2 (y y - (y Aty")
(A12)
+yFV(y).
The next timestep's configuration follows from minimizing this barrier- Gurrie--augmenedd incremental po- tential:
n+1 n+1 arg min xy;
(A13)
y:x
A.5 Kinematic Constraints
The kinematic constraints expressed as
Sxx=s"SSy==0
(A14)
where 8x R2 x3N. RNN for soft bodies, and SY TR=xx12n, 1212 for affine bodies. The constraints are applied by selecting the constrained DoFs of the state vectors and specifying the
A2

---

constraint values with. To enforce these constraints, we employ the Augmented Lagrangian method by augmenting Eipc to:
EAL (y; x) (y;
(A15)
+
-
where X "eR" and X e eR are Lagrangian multipliers.
Optimizing (y: yields the solution the constrained system:
n x n+ arg min Eypc(y;x),
(A16)
Vix
s.t. - and y
(A17)
A.6 Guides on Setting the Materials Parameters for Gel Pads
We provide guidance properly choosing the parameters of the gel pads, which plays vital role in vision-based tactile sensing. Emperically, the pad's Poisson's ratio y is typically in [0.3, 0.45], within which the response variation is subtle. The Young's modulus typically falls in [0.01, 10 and the response remains almost constant within the same order magnitude. For simulations without measuring the parameters of the material, and carefully adjust them by inspecting the simulation scenes.
practice is to start within the typical range
While the simulation is not overly sensitive to these parameters, we still recommend measuring these parameters using standard techniques. For example, the sliding experiments for friction coef- ficients [62] or tensile tests for elastic moduli [18, 27, 33].
B Additional Related Work
B.1 Robot Tactile Sensors
Tactile sensing plays fundamental role in precise manipulation, established by neuroscientific studies [54, 29, 28, 3]. This understanding has driven the development of artificial tactile sensing systems for robots [44]. Among these, VBTSs have gained prominence by offering high-resolution sensing with cost-effectiveness and operational simplicity [61, 53, 37, 35]. While these sensors have advanced robotic manipulation [45, 40, 66], their development remains constrained by the reliance on physical hardware experimentation. Taccel addresses this limitation by providing a compre- hensive simulation platform accelerate research and development in tactile robotics.
B.2 Tactile-Informed Robotic Tasks
Tactile sensing enhances robotic capabilities across three fundamental domains through precise con- tact interaction measurements:
Perception Tactile feedback enables sophisticated object understanding through contact-based sensing. Applications include shear and slip detection [63, 11], object classification and pose estima- tion [34, 58, 50, 2], material property inference [19, 24], and interaction reconstruction [50, 60, 57]. These perceptual capabilities form the foundation for advanced manipulation algorithms.
Grasping Stable grasping requires precise control of contact forces to balance external loads [15, 48). Tactile sensing provides direct force-torque feedback essential for diverse grasping strate- gies [39, 36, 59]. This tactile information complements vision-based approaches by enabling fine- grained contact monitoring and in-hand adjustments [6, 5].
Manipulation Tactile feedback enables complex manipulation beyond basic pick-and-place op- erations. Applications include precision tasks like peg insertion [8], object pivoting [21], and artic- ulated object manipulation [67, 3]. Systems such as Tac-Man [66] and DoorBot [52] demonstrate how tactile sensing guides contact geometry understanding and articulation control. This sensing modality is particularly crucial for high-frequency object tracking during dexterous manipulation.
We validate Taccel's capabilities through three representative applications: (i) multi-platform
robotic grasping with both rigid and soft objects, (ii) object classification using purely synthetic
A3

---

training data with strong real-world transfer, and (iii) articulated object manipulation including draw- ers, cabinets, and bolt-nut assembly tasks, extending the Tac-Man framework [66].
C
Additional Tests on Simulation Speed
C.1
VRAM Usage
Tab. Al report the VRAM occupancy of the low-resolution peg-insertion test (Sec. 5.4). The GPU memory (VRAM) scales linearly with the number of environments, enabling efficient scaling until memory saturation.
Table A1: VRAM Usage of Taccel in the low-resolution peg-insertion test.
Envs
16
256
1024
4096
VRAM/ GiB
4.3
4.3
7.2
16.7
54.0
C.2 Simulation Speed on Various GPUs
Tab. A2 reports the simulation speed of the low-resolution peg-insertion test on various GPUs (Sec. 5.4). While HPC GPUs with high FP32:FP64 ratio (1:2) is ideal for optimal performance, desktop GPUs like the 3090 (FP32:FP64 ratio 1:64) also deliver scalable performance, achieving real-time performance (50FPS) with around 100 environments.
Table A2: Simulation speed of Taccel in the low-resolution peg-insertion task on various GPUs.
GPU
H100
RTX 4090
RTX 3090
RTX 3090
RTX 3090
# Envs 256
256
256
128
64
Total FPS
185.52
129.12
121.38
74.81
43.40
D Experiments Details
D.1 Real-world Data Collection for Tactile Signal Evaluation
For real-world data collection, our experimental setup consisted of a calibrated GelSight-type sensor, mounted on vertical rack for precise movement control, as shown Fig. 3(a). A white mount fixed in the center of the platform to hold the test objects.
We 3D-print the 18 objects from a standard tactile shape testing dataset [16] 0.2mm layer height. For each object, we first fix it on the mount, slide the sensor along the rack to press its gel pad on the object, and put 500 g weight on the sensor to ensure an appropriate amount of gel deformation. We record the RGB tactile patterns five times and compute their mean image to partially remove the noise. We then replicated this pressing sequence Taccel using a high-resolution soft body gel pad (maximal cell volume Vmax x 10 m²) to ensure signal fidelity. Sec. 5.2 compares the similarity between the real-world and simulated tactile signals.
D.2 Multi-environment Simulation Test
Our multi-environment simulation test involves three scripted tasks. First, we implemented a peg- insertion task adapted from SAPIEN-IPC [8], where two gel pads (139 nodes and 317 cells each) fol- low scripted trajectory. The trajectory involves 200 simulation steps, where the sensors squeeze the peg and manipulate around the hole. Next, we scaled this task higher resolution (1,533 nodes and 5,360 cells each), which results in a larger system to solve. Finally, to demonstrate Taccel 's potential for advanced robotics research, we created scripted grasping task with a customized five- fingered dexterous hand equipped with 17 gel pads covering the entire hand, totaling 5,157 nodes and 14,311 cells. The trajectory also involves 200 simulation steps, with the hand approaches, grasps, lifts, maneuvers, and releases stiff cylinder.
A4

---

(b) simulated 2D tactile signals
(c) simulated tactile signal Figure Al: Examples of the synthesized grasps and the simulated tactile signals in and 3D. (a) Diverse grasps are synthesized and simulated for different objects and robatic hands. (b) An example 2D tactile signals generated for grasp with the Panda Hand. (c) An example of the 3D tactile signals generated for grasp with the Allegro Hand with Digit sensors.
(a) synthesized grasps for four robotic hands
D.3 Data Collection and Training Details for Learning Object Classification
In the object classification sim-to-real experiment, we collect tactile signals of grasping 10 3D- printed mechanical parts in both simulation and the real world. We also train object classification model with the simulated data and evaluate it on the real data.
For each object, we simulated 200 grasp trials using paraliel gripper with randomized grasp- ing poses. The gripper closes towards the object until the depth deformation exceeds thresh- old randomly sampled from Ta ([.5, 1.5mmm, and with each grasp yields 2 depth map sam- ples after grasping. To ensure the granularity of the depth map, high-resolution gel pad models max V > 10 1 approximately 4,000 training samples. We split the samples into training set (85%) and a valida- tion set (15%). The former is used to train the classification model, supervised the NLL Loss, using the Adam optimizer with a learning rate le-4 for 100 epochs. During training, we apply an exponential LR scheduler.
m. The depth maps d(u,o) extracted from these simulated tactile signals yielded
For real-world validation, we collected tactile signals using a RobotiQ-2F85 parallel gripper equipped with GelSight-type sensors. The test objects were 3D printed at 0.2mm layer height to maintain high geometric fidelity. We gathered 8 grasps (16 depth maps) per object.
D.4 Dexterous Grasping
Our key modification to the DFC algorithm promotes perpendicular contact between gel pads and object surfaces, optimizing for downstream tasks that rely on tactile perception.
Following Liu el al. [39], we synthesize grasping poses in the robot's joint space E Q relative to the object frame by minimizing modified Gibbs energy:
E(O, g, T) = EDFC + Acontact Eccntact (O, T)-
(A (A18)
The force-closure term EDPC maintains its original formulation [39], with added constraints to ensure gel pad penetration depth e (typically 0.5 mm). We introduce a new contact term Econtact that aligns the normals of gel pad contacts e; 2 G, with their corresponding object surface normals
o = arg min ||o- [lo-c:::
Frontact O0, T) = -
(A 19)
where -.)- represents the surface normals of the gel pad and object surfaces.
Fig. A1 shows the examples of the synthesized grasps and the simulated tactile signals both 2D and 3D.
A5

---

D.5 Articulated Object Manipulation with Tac-Man
In the Tac-Man framework [66], the system alternates between execution and recovery phases. Dur- ing execution, the system performs coarse manipulation actions (e.g., pulling backward) to gradu- ally move the articulated object part. When the actual motion deviates from intended trajectories due to articulation constraints, the gel pad deforms, creating contact deviation reflected in marker flow magnitudes. Once these flows exceed threshold 60, the system enters recovery mode restore stable contact by reducing deviation, before resuming execution. This execution-recovery cycle typically requires tens of iterations to finish manipulation.
For our sim-real comparison, we manually created URDF models for the microwave oven and the drawer match real-world geometries and kinematies, maintaining identical initial grasping poses across simulation and physical setups. Following Tac-Man's implementation, we set gy 0.4 mm and a 0.6.
A6


Reviews:
[{"review_id": "TrSJ1FfzRW", "reviewer": "Reviewer_ePNg", "summary": "This paper presents Taccel, a high-performance simulation platform for vision-based tactile sensors (VBTSs) integrated with robotic systems. The key technical contribution is the combination of Incremental Potential Contact (IPC) and Affine Body Dynamics (ABD) to achieve both physical accuracy and computational efficiency. The platform achieves an 18-fold speedup over real-time simulation across thousands of parallel environments on a single GPU. The authors validate their system through three applications: object classification with sim-to-real transfer, large-scale robotic grasping dataset generation, and articulated object manipulation. The paper demonstrates that Taccel can generate realistic tactile signals while maintaining stable, penetration-free physics simulation at scale.", "questions": "* How does Taccel compare to the recently presented TacEx system (Nguyen et al., CoRL 2024 Workshop), which also combines soft-body simulation with visuotactile rendering in Isaac Sim? Could you provide a detailed comparison in terms of accuracy, performance, and features?\n* How difficult would it be to extend Taccel to support other VBTS designs beyond GelSight-type sensors? What modifications would be required for sensors with different optical properties or gel geometries?\n* The current validation focuses on relatively slow, controlled movements. How does the simulation accuracy degrade under dynamic conditions with rapid contact formation/breaking or high-velocity impacts?\n* What is the relationship between memory usage and the number of parallel environments? Is there a practical upper limit beyond GPU memory constraints?\n* How sensitive are the results to the choice of material parameters (Young's modulus, Poisson's ratio)? Do you provide guidance for parameter identification from real sensor data?\n* Are there plans to support multi-GPU execution for even larger-scale simulations? What would be the main challenges in implementing this?\n* For online robot learning applications, what is the minimum hardware configuration needed to achieve real-time performance with a reasonable number of environments (e.g., 100-1000)?", "limitations": "The authors acknowledge computational demands as a limitation, identifying PCG-based linear system solving as a bottleneck. However, several other limitations deserve mention:\n\n- The fidelity of tactile signal generation depends on the quality of the trained DNN model, which may not generalize to all surface textures or contact conditions\n\n- The Neo-Hookean material model may not accurately capture all aspects of real gel pad behavior, particularly under large deformations\n\n- The current implementation appears limited to single-GPU execution, which may constrain very large-scale simulations", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths: \n+ The unified IPC-ABD formulation seems mathematically sound and addresses key limitations of existing simulators. The use of IPC guarantees intersection-free trajectories while ABD efficiently handles rigid components, avoiding the computational overhead of treating everything as soft bodies.\n+ The ability to simulate 4096+ parallel environments with tactile sensors is a significant achievement. The performance benchmarks showing 915 FPS for low-resolution tactile simulation represent a major improvement over existing solutions like SAPIEN-IPC.\n+ The paper provides thorough validation across multiple dimensions - physics accuracy (bolt-nut assembly, soft block pressing), tactile signal fidelity (SSIM of 0.93), and real-world task performance (Tac-Man manipulation with 1.1% error compared to real execution).\n+ The sim-to-real transfer results (70.94% accuracy on real mechanical parts after training only on synthetic data) demonstrate the practical value of the simulator for developing tactile perception systems.\n+ The provision of Python APIs and support for standard formats (URDF) makes the tool accessible to the robotics community.\n\nWeaknesses:\n- Limited tactile sensor models: While the paper focuses on GelSight-type sensors, it's unclear how easily the framework extends to other VBTS designs (e.g., different gel materials, alternative optical configurations). The RGB signal generation relies on a DNN trained on only 200 real images, which may limit generalization.\n- Despite the impressive speedup, the system still requires high-end GPUs (H100) for optimal performance. The scalability analysis doesn't adequately address performance on more accessible hardware or the memory-computation tradeoffs.\nIncomplete comparisons: The paper lacks detailed comparisons with some recent simulators. Most notably, there's no comparison with TacEx (Nguyen et al., 2024, TacEx: GelSight Tactile Simulation in Isaac Sim -- Combining Soft-Body and Visuotactile Simulators), which also combines soft-body and visuotactile simulation in Isaac Sim. This is a significant omission given the similar goals.\n-The paper doesn't thoroughly discuss when and why the simulation might fail to match real-world behavior. The sim-to-real gap in object classification (86.5% -> 70.94%) suggests room for improvement.\n- The validation focuses primarily on static/quasi-static scenarios. Dynamic manipulation tasks with rapid contact changes or multi-contact scenarios are underexplored.", "was_revised": true, "final_justification": "I appreciate the authors' efforts in preparing an extensive rebuttal. Taccel is an impressive work in tactile parallel simulation, advancing the state-of-the-art on IPC simulation, with comprehensive evaluations in simulation and validation on real-world tactile sensors. The rebuttal largely confirms my assessment and I will up my score.", "rebuttal": "We thank the reviewers and the chairs for their time and effort in providing highly valuable feedback. We appreciate the reviewers for recognizing our formulation sound (ePNg, TkE2), experiments thorough (ePNg, VcE1, TkE2, rfK1), performance significantly improved (ePNg, VcE1, rfK1), and the paper well-written (VcE1, TkE2). Here we address your concerns and answer your questions below.\n\n## Sensor Compatibility (W1, Q2)\n\n**Taccel is highly extensible and supports a wide range of VBTS** with various:\n\n- **gel materials** with adjustable simulation parameters like Young’s Modulus\n- **gel geometry**, including cuboid, curve-shaped (DIGIT), dome-shaped (DIGIT360, PPTac)\n- **mechanism and optical setup**: Our data-driven tactile image generation handles **different mechanisms** (eg., 9DTact) and **optical setups** via the learnable tactile image model\n\nWe have validated the integration and physical simulation of four sensors in Taccel: our custom GelSight and 9DTact sensors, commercial sensors like GelSight Mini, DIGIT, and DIGIT360.\n\nWe further validate the data-driven tactile image generation for a 9DTact sensor (different mechanism compared to GelSights). The calibration and validation protocols follows Sec. 5.2, and the simulated tactile images achieves SSIM = 0.97, confirming the high fidelity.\n\n## Generalization of the DNN (W1, L1)\n\nThis is a very interesting point. The DNN model learns a local, pixel-to-pixel (rather than patch-based) mapping from the pixel coordinate and surface normal to the color change (Sec. 4.2). **This is a generalizable formulation across object geometries** (Fig. 3a), analogous to the widely-used GelSight depth model [1]. Based on this, the actual data for training includes `n_img x h x w` data points (32M, for our 400 x 400 images), practically sufficient for a small MLP (<100K parameters).\n\n**We agree its limitation of generalizing to all textures or extreme contacts** (eg., not producing shadows casted by the gel when largely deformed) and will clarify this in revision.\n\n## Comparisons with other Simulators (W2, Q1)\n\nWe agree a comparison to recent simulators like TacEx is needed.\n\n- **Physics Accuracy:** TacEx simulates soft bodies with GIPC, asynchronously sending their nodal positions back to PhysX for rigid body simulation. This lacks a global energy minimization guarantee, which may introduce potential coupling errors and simulation instabilities at soft-rigid contact. Taccel **integrates IPC and ABD in a unified augmented Lagrangian formulation**, with soft and rigid bodies strongly coupled by contact forces, providing **theoretically better accuracy in contact**.\n- **Sensor Signal Accuracy:** TacEx uses Taxim that converts object geometry to tactile signals via look-up tables. Taccel uses the deformed elastomer to compute marker flow (via camera projection) and contact depth map (via ray-tracing), generating tactile images with a learned model. This allows Taccel to **capture finer and richer elastomer deformation modes**, particularly shear deformations, yielding **more accurate tactile signals**.\n- **Efficiency:** GIPC uses a quasi-Newton method to accelerate Newton iterations, which is a potential improvement for Taccel. Currently, Taccel focuses on **optimizing overhead for massive parallelization environments on GPU**.\n- **Features:** TacEx is a useful plug-in for Isaac Lab, while Taccel is a comprehensive simulation platform for robots, objects, and sensors. It also offers additional tools and APIs for sensor integration and simulation.\n\nThe code of TacEx is not yet released upon access, making a more detailed comparison impossible. We opt to a quantitative comparison with TacFlex[3], another recently published (T-RO) FEM-based simulator.\n\n**Experiment:** We conduct sim-real twisting experiments to test elastomer deformation. In the real world, a GelSight Mini in controlled by a robot arm, pressed perpendicularly onto a fixed sphere (radius $r$ = 10mm) to achieve a contact depth $d \\in$ [6,12]mm, then twisted about the z-axis for an angle $\\theta$. Marker flows are collected as $\\theta$ reaches 2.5, 5, 7.5, and 10 degrees. The scene is simulated in Taccel and TacFlex, both using a 2k-node elastomer. Despite the multi-engine support, only the Flex solver was available in their codebase currently, so we used it for our comparison.\n\n**Results:** We report the pair-wise error of marker flow magnitudes (length difference) and directions (angular difference). Taccel demonstrates **lower errors in marker flow simulation**, which we attribute to its **superior accuracy of the IPC solver in handling friction and elastomer deformation**.\n\n| Simulator | Flow Magnitude Error / mm ↓ | Flow Direction Error / deg  ↓ |\n| --------- | --------------------------- | ----------------------------- |\n| Taccel    | 0.0540 (±0.0019)            | 0.2877 (±0.0349)              |\n| TacFlex   | 0.0968 (±0.0198)            | 0.3981 (±0.1565)              |\n\n## Hardware Requirement and Usage (W2, Q4, Q7)\n\nWe additionaly benchmark the peg-insertion task on various GPUs (timestep $\\Delta t$ = 1/50s). Taccel leverages FP64 for improved precision and faster Newton convergence (see reponse to reviewer VcE1's Q4), making HPC GPUs (FP32:FP64 FLOPS ratio = 1:2) ideal. **More accessible cards like the 3090**, despite their lower FP32:FP64 ratio (1:64), **also deliver scalable performance**, achieving real-time performance (50FPS) with around 100 environments.\n\n| GPU (FP64:FP32) | H100 (1:2) | RTX 4090 (1:64) | RTX 3090 (1:64) | RTX 3090 (1:64) | RTX 3090 (1:64) |\n| --------------- | ---------- | --------------- | --------------- | --------------- | --------------- |\n| # Envs          | 256        | 256             | 256             | 128             | 64              |\n| Total FPS       | 185.52     | 129.12          | 121.38          | 74.81           | 43.40           |\n\nThe GPU memory (VRAM) **scales linearly with the number of environments**, enabling efficient scaling until memory saturation. For the peg insertion task on an H100 GPU:\n\n| # envs     | 1    | 16   | 256  | 1024 | 4096 |\n| ---------- | ---- | ---- | ---- | ---- | ---- |\n| VRAM / GiB | 4.3  | 4.3  | 7.2  | 16.7 | 54.0 |\n\n**Currently the bottleneck for scalability is still on the GPU memory**, which can extend further via Multi‑GPU support (Q6). We aim to further enhance Taccel's scalability in the future.\n\n## Accuracy Validations on More Dynamics Tasks (W3, Q3)\n\nTaccel's foundation in IPC and ABD allows it to robustly handle highly dynamic events involving high-speed contact. We have validated it through qualitative tasks like throw-and-catch, where the results are visually realistic at a cost of more Newton iterations. However, quantitative sim-real comparisons is much harder due to the stochasticity of dynamic events. For a qualitative reference, the original IPC paper's demos [2] illustrate its capability to simulate rapid, high-impact contacts with high physical fidelity.\n\n## Potential Failure in Sim-Real Mismatch (W2)\n\nWe do observe sim-real mismatch that requires further improvement, despite the moderate sim-real gap revealed by our experiments. Primary sources include:\n\n- Errors in **camera calibration** (distortions, noises, refraction through the gel [3]), which can be observed in Fig. 5(b)\n- LEDs and the **sensor degradation** through usage (light dimming, elastomer wear, and environmental light leakage)\n- Use of **imprecise physical parameters** (eg., Young’s Modulus) and suboptimal solver parameters (eg., optimization tolerance) in simulation, despite its robustness\n- Slight sim-real **geometrical mismatch** of the sensor and the objects due to **discretization error**\n\nWe will add a discussion on this for completeness.\n\n## Affects and Calibration of Materials Parameters (Q5)\n\nPractically, the sensor elastomer's Poisson’s ratio $\\nu$ is typically in [0.3, 0.45], within which the response variation is subtle. The Young’s modulus $E$ typically falls in [0.01, 10] MPa, and the response remains almost constant within the same order of magnitude. **The simulation is not overly sensitive to these parameters.**\n\n**Standard techniques exist for precise calibration**, such as sliding experiments for friction coefficients [4] or tensile tests for elastic moduli [5,6,7]. For simulations without calibrations, our practice is to start within the typical range and carefully adjust them by inspecting the simulation scenes. We will provide guidance and references for these methods in revision.\n\n## Multi-GPU Support (Q6, L3)\n\n**We plan to support multi-GPU execution** in future work. Our parallel-environment design allows for isolated environments to be distributedly solved. **Currently we foresee no major technical barriers to this.**\n\n## Additional Limitations (L2, L3)\n\n**We acknowledge the limitations and will incorporate them in the revision.**\n\n> [1] Wang, Shaoxiong, et al. \"Gelsight wedge: Measuring high-resolution 3d contact geometry with a compact robot finger.\" *2021 IEEE International Conference on Robotics and Automation (ICRA)*. IEEE, 2021.\n>\n> [2] Li, Minchen, et al. \"Incremental potential contact: intersection-and inversion-free, large-deformation dynamics.\" *ACM Trans. Graph.* 39.4 (2020): 49.\n>\n> [3] Zhang, Chaofan, et al. \"TacFlex: Multi-Mode Tactile Imprints Simulation for Visuotactile Sensors with Coating Patterns.\" *IEEE Transactions on Robotics* (2025).\n>\n> [4] Yuan, Wenzhen. *Tactile measurement with a gelsight sensor*. Diss. Massachusetts Institute of Technology, 2014.\n>\n> [5] Gould, Phillip L., and Yuan Feng. *Introduction to linear elasticity*. Vol. 2. New York: Springer, 1994.\n>\n> [6] Javanmardi, Yousef, et al. \"Quantifying cell-generated forces: Poisson’s ratio matters.\" *Communications physics* 4.1 (2021): 237.\n>\n> [7] Li, Mingxuan, et al. \"EasyCalib: Simple and low-cost in-situ calibration for force reconstruction with vision-based tactile sensors.\" *IEEE Robotics and Automation Letters* (2024)."}, {"review_id": "nbQe49tFuA", "reviewer": "Reviewer_VcE1", "summary": "This paper introduces Taccel, a parallel simulation platform to simulate vision-based tactile sensors (VBTS) for robotics. Taccel combines Incremental Potential Contact (IPC) as a contact model, Affine Body Dynamics (ABD) for stiff materials, and Finite Element Method (FEM) for soft-body simulation of hyper-elastic soft gels used in VBTSs. Taccel enables accurate and high-speed simulation. The experiments demonstrate that Taccel can be used for applications such as generating synthetic data to train a model for object classification, simulating robotic grasping using multi-fingered dexterous hands with integrated VBTSs, and articulated object manipulation using tactile perception feedback.", "questions": "L61-63: I believe these numbers are for total effective FPS, instead of per-environment FPS? If this is the case, then I don’t think it’s accurate to describe 18x “real-time” performance in the abstract L10), because the single-environment FPS is still sub 1 FPS (Figure 4). \n\nL309: What are execution-recovery switch counts? This metric is not explained in the subsection.\n\nFigure 2(a): It looks like only two rigid planes are interacting with the bolt (while the Franka gripper is only used for visualization). Does the task fail if the finger collision shapes are smaller (similar to what is used in PyBullet and Sapien)? What collision simplifications are made by Taccel, such as for the articulated rigid body tasks in Figure 7? \n\nSAPIEN-IPC is also implemented using NVIDIA Warp. Is Taccel based on SAPIEN-IPC (which appears fully open sourced)? What are the major differences between Taccel’s implementation of IPC and SAPIEN-IPC that enable faster simulation (Figure 2 (d))? L251 mentions that SAPIEN-IPC’s use of FP32 leads to convergence issues, while Taccel uses FP64 (Figure 4). I would expect use of FP64 to lead to slower simulation (and larger memory requirements). Does FP64 enable fewer solver iterations or larger timesteps?\n\n---\n[minor]\n- References should not in the abstract?\n- L37-38: “contactsd” spelling?\n- L60: “warp” is lowercase?", "limitations": "Yes.", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "[S1] The paper is generally well-written, and the proposed Taccel simulator and accompanying experiments are comprehensive. The authors motivate the use of IPC+ABD based on the limitations of prior work simulating VBTSs alternative simulation techniques. The performance gains of parallel simulation enabled by Taccel over single-environment simulators are impressive (16x). \n\n[W1] To the best of my knowledge, while the Taccel simulator is available on Github, most of the actual (IPC) simulation code is closed source (only compiled binaries are provided). Given that the authors appear committed to developing Taccel with the community (L187-189), I would encourage the authors to consider fully open sourcing Taccel. My main concern here is that IPC is notoriously slow, and understanding the optimizations made for IPC that achieves the performance reported in Taccel, would greatly benefit the community. Furthermore, it seems that Taccel uses SAPIEN-IPC? (an IPC implementation based on NVIDIA Warp that is fully open sourced.)\n\n[W2] It is difficult to directly compare the quality of Taccel’s simulation of VBTSs to existing VBTS simulators such as SAPIEN-IPC, Taxim, or DiffTactile, given the experimental results in Section 5 or 6. I think an experiment that compares directly pixel-wise tactile marker MSE between VBTS simulators would be helpful. While there is a quantitative result reporting SSIM between the simulated and real-world RGB tactile image, this robustness may be due to the depth -> RGB neural network, rather than quality improvements in tactile sensor simulation. \n\n- - -\n\nI am willing to raise my score if [W1] and [W2] are resolved.", "was_revised": true, "final_justification": "The authors prepared an extensive rebuttal that resolved most of my questions and concerns. While [W1] is not fully addressed (full code release is still in preparation), I will still increase my score (-> 6) in a show of good faith. Taccel is an impressive work in tactile parallel simulation, advancing the state-of-the-art on IPC simulation, with comprehensive evaluations in simulation and validation on real-world tactile sensors.", "rebuttal": "We thank the reviewers and the chairs for their time and effort in providing highly valuable feedback. We appreciate the reviewers for recognizing our formulation sound (ePNg, TkE2), experiments thorough (ePNg, VcE1, TkE2, rfK1), performance significantly improved (ePNg, VcE1, rfK1), and the paper well-written (VcE1, TkE2). Here we address your concerns and answer your questions below.\n\n## Code Release of Taccel (W1)\n\nWe agree that fully open-sourcing the code is beneficial for the community, and **do plan to open-source Taccel in the near future**. Before that, we are still working on optimizing and testing the core implementations to improve their robustness and accuracy. We are **also working on adapting the infrastructure** to `warp.sim` and NVIDIA Newton framework for broader compatibility and a richer ecosystem.\n\n## Comparisons on Tactile Markers (W2)\n\n**Experiment:** We conduct sim-real twisting experiments to test elastomer deformation, and compare with TacFlex[3], a recently published (T-RO) FEM-based simulator. In the real world, a GelSight Mini in controlled by a robot arm, pressed perpendicularly onto a fixed sphere (radius $r$ = 10mm) to achieve a contact depth $d \\in$ [6,12]mm, then twisted about the z-axis for an angle $\\theta$. Marker flows are collected as $\\theta$ reaches 2.5, 5, 7.5, and 10 degrees. The scene is simulated in Taccel and the public codebase of TacFlex, both using a 2k-node elastomer.\n\n**Metric:** However, while pixel-wise comparison (eg., SSIM, MSE) is suitable for comparing tactile image quality (Fig. 3(a)), they are less suitable for evaluating tactile images with markers, as the they are overly sensitive to minor positional shifts of these high-contrast marker pixels. A small, physically insignificant displacement can lead to a disproportionately large error, masking the true geometric accuracy of the simulation. Therefore, we adopt a more direct and conventional evaluation by comparing the marker flow, which directly measures the simulated deformation accuracy. This methodology is the established standard in the vast majority of prior works [1-3].\n\n**Results:** We report the pair-wise error of marker flow magnitudes (length difference) and directions (angular difference) below. **Taccel demonstrates lower errors in marker flow simulation**, which we attribute to its **superior accuracy of the IPC solver in handling friction and elastomer deformation**.\n\n| Simulator | Flow Magnitude Error / mm ↓ | Flow Direction Error / deg  ↓ |\n| --------- | --------------------------- | ----------------------------- |\n| Taccel    | 0.0540 (±0.0019)            | 0.2877 (±0.0349)              |\n| TacFlex   | 0.0968 (±0.0198)            | 0.3981(±0.1565)               |\n\n## Simulation FPS Computation (Q1)\n\nYou are correct. The reported FPS represents the total effective FPS across all parallel environments (ie., total simulation steps per second). **We will revise the abstract and main text to state this with greater clarity.** Thank you for pointing this out.\n\n## The Execution-Recovery Switch Count (Q2)\n\nWe apologize for omitting the explanation from the main text; **it was in the appendix (Sec. C.5) and will be moved to the main text to make it self-contained.**\n\nThe \"switch count\" is a key performance metric for the Tac-Man algorithm [4], which manipulates articulated objects using tactile feedback. The algorithm frequently switches between an execution stage (moving the part following a guessed direction) and a recovery stage (readjusting the grasp based on tactile signals to ensure stable contact) based on tactile deformation thresholds. The \"switch\" between the two stages occurs when deformation exceeds a threshold (execution → recovery) or the large deformation is compensated (recovery → execution). **The \"switch counts\" is the number of such switches during the manipulation.**\n\nThe simulation of Tac-Man requires precisely solving the friction and gel pad deformation during a long sequence. The close match in the \"switch counts\" between our simulation and the real world indicates that Taccel accurately replicates the complex, long-horizon contact dynamics of the task.\n\n\n## Collision Settings (Q3)\n\nIn Figure 2(a), **the gray “rigid planes” are actually the soft gel pads** of the two VBTS sensors attached on the fingers (the sensor shells are not visualized for clarity). The gripper grasps the bolt with the pads and screws it into the nut, during which **all the elements, including the robot links, gel pads, and objects, are involved in the physical simulation** (collision detection, contact solving, time stepping, etc). \n\nBased on ABD, Taccel **supports efficient simulation using either the collision meshes** (simplified shapes, convex decomposition, or convex hull) **or the high-resolution, non-convex visual mesh on GPU**. While we directly use these detailed meshes in our experiments (as rendered), we recommend using simplified collision meshes for greater efficiency in practice.\n\n## Relation with SAPIEN-IPC (W1, Q4)\n\n**Taccel is an independent development and is not based on SAPIEN-IPC [1].** Although both use IPC and NVIDIA Warp, we attribute Taccel's faster speed to two key factors:\n\n**Implementation Optimizations:** We developed a more compact data management and optimization solving scheme, which allows more parallel environments on a single GPU and thus the faster speeds. Environment isolation in collision detection and step size filtering are introduced for each environment to evolve independently with its own optimization step size. Environments meeting their individual stopping criteria (eg., Newton relative error) are frozen to free resources. A failure detection stage further isolates unstable environments, enhancing overall robustness.\n\n**FP64 Precision:** You are correct that FP64 arithmetic is slower per operation than FP32 (as used in SAPIEN-IPC). However, IPC benefits from FP64’s higher precision, enabling more accurate search directions and line search filtering, which significantly accelerates convergence by shortening Newton iterations. This reduction in iterations more than compensates for the higher cost per iteration, resulting in a faster and more stable simulation overall. See the next question for a quantitative inspection.\n\n## Computing Data Type of FP64 (Q4)\n\n**FP64 does cause slower simulation speed, but the higher precision enables much fewer solver iterations for larger timesteps.** To demonstrate this, we run the peg-insertion benchmark (Sec. 5.4) in Taccel and SAPIEN-IPC with 1 / 16 / 64 environments, with the newton iteration limit set to 50 and optimization residue tolerance set to 0.01m/s. During each iteration, the IPC solver performs conjugate gradient descent until (i) the 50-step limit is exceeded, or (ii) the optimization residue is below 0.01m/s. The table below reports the maximum Newton iteration required for each step, the optimization residue, total time and total FPS. **Although each iteration takes longer for FP64, the convergence is much faster and thus the better simulation speed.**\n\n| Simulator         | # Envs | # Newton Iterations (max 50) ↓ | Optim. Residue / (m/s) ↓ | Time / s ↓  | Total FPS ↑ |\n| ----------------- | ------ | ------------------------------ | ------------------------ | ----------- | ----------- |\n| **Taccel (FP64)** | **1**  | **1.585 (±1.128)**             | **7.643e-5 (±6.550e-5)** | **69.78**   | **2.866**   |\n| SAPIEN-IPC (FP32) | 1      | 24.085 (±10.354)               | 0.001 (±0.006)           | 82.411      | 2.427       |\n| **Taccel (FP64)** | **16** | **2.83 (± 2.565)**             | **1.866e-4 (±2.089e-4)** | **141.919** | **22.548**  |\n| SAPIEN-IPC (FP32) | 16     | 50.0 (±0.0)                    | 0.113 (±0.440)           | 291.462     | 10.979      |\n| **Taccel (FP64)** | **64** | **3.445 (±3.261)**             | **2.043e-4 (±2.200e-4)** | **196.712** | **65.069**  |\n| SAPIEN-IPC (FP32) | 64     | 50.0 (±0.0)                    | 0.196 (±0.442)           | 668.950     | 19.134      |\n\n> Note that we have increased the Newton iteration limit from 20 (used in Sec. 5.4) to 50 to inspect the convergence behavior under FP32. The results are not comparable.\n\n## Editorial Issues (Q5)\n\n**Thanks for pointing them out! We will carefully revise the typos and editorial issues.**\n\n\n> [1] Chen, Weihang, et al. \"General-purpose sim2real protocol for learning contact-rich manipulation with marker-based visuotactile sensors.\" *IEEE Transactions on Robotics* 40 (2024): 1509-1526.\n>\n> [2] Zhang, Chaofan, et al. \"TacFlex: Multi-Mode Tactile Imprints Simulation for Visuotactile Sensors with Coating Patterns.\" *IEEE Transactions on Robotics* (2025).\n>\n> [3] Du, Wenxin, et al. \"Tacipc: Intersection-and inversion-free FEM-based elastomer simulation for optical tactile sensors.\" *IEEE Robotics and Automation Letters* 9.3 (2024): 2559-2566.\n>\n> [4] Zhao, Zihang, et al. \"Tac-man: Tactile-informed prior-free manipulation of articulated objects.\" *IEEE Transactions on Robotics* (2024)."}, {"review_id": "8tGS4euRXd", "reviewer": "Reviewer_TkE2", "summary": "This paper introduces Taccel, a high-performance simulation platform tailored for vision-based tactile sensors (VBTSs) in robotics. VBTSs offer rich tactile feedback through gel pad deformations, but simulating their behavior remains challenging due to the complex mechanics of soft gel pads and the high computational cost. The proposed method addresses these challenges by integrating Incremental Potential Contact (IPC) and Affine Body Dynamics (ABD), achieving high-fidelity, collision-free simulations at >18 wall-clock speed.\n\nPowered by efficient implementation with NVIDIA Warp, Taccel supports large-scale parallelization. Its modular design also enables flexible sensor-robot configurations via user-friendly Python APIs, and it generates high-resolution RGB and depth-based tactile signals. The paper tests Taccel on diverse tasks, including object classification, grasping, and articulated manipulation, to demonstrate its strong capability for sim-to-real transfer, which outperforms prior simulators like TacMan in both precision and scalability.", "questions": "* As discussed in the \"Strengths and Weaknesses\" section, can Taccel handle different object materials or objects with heterogeneous materials? If so, what is an efficient way to do that? If not, what are the challenges?\n* Part of the main sim-to-real challenges for VBTS come from (1) sensor degradation over time and (2) variance among different sensor configurations and among individual sensors of the same configuration. Would that be mitigated by using Taccel simulation? If so, how would Taccel efficiently simulate different sensor configurations and/or sensor degradation over time to reduce the sim-to-real gap?\n * Open-ended question for discussion: one weakness of VBTS is force estimation, as real sensors only use embedded cameras to capture the gel deformation. However, now given the simulation environment and the known full state of the sensor and objects, is it possible to derive force/torque measurement from the simulation, which then helps to train a NN to learn the mapping of tactile images to force estimation?  (Context: one of current methods for force estimation using VBTS is to collect a bunch of data pressing objects on the sensor with different forces, then train a NN. With Taccel, would it be easier to gather ground truth data with more variation, thus making the task easier?)", "limitations": "yes", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "**Strengths**\n* The paper proposes an IPC-ABD combined simulation to simulate vision-based tactile sensors (VBTS), which is novel, original, and significant. It improves the simulation accuracy compared to rigid body simulations like Tacto and includes the robot in the loop compared to FEM based simulator like DiffTactile. Authors have provided detailed mathematical derivations and justifications for when to use which simulation method.\n* The paper provides a thorough description of the simulation pipeline, covering all important aspects of VBTS, i.e. RGB images, normal maps, marker flow, and 3D point cloud information.\n* The experiment section provides comprehensive examples to demonstrate Taccel's capability of sim-to-real transfer and applications in robotic manipulation tasks through three main tasks: training object classification models, generating synthetic data with scalable parallelization, and manipulating articulated objects. They demonstrate the significance of VBTS and efficient learning with simulations like Taccel. \n* The paper and supplementary material are overall well-written and easy to follow. The video provides a good dissection and demonstrates the simulation scenes effectively. \n\n\n**Weakness**\n* The paper focuses on sensor and robot modeling, but doesn't elaborate on the object modeling. It mentions that Taccel can handle both rigid and soft objects, but could it handle object of different materials? would the simulation speed be affected? More testing examples and analysis would be helpful.\n* Clarification question: in \"object classification\" experiment, since the tactile data only provide local information, how does the model learn to differentiate different objects VS different parts of the same model? (Fig. 5 (a) shows nine mechanical parts, but the caption says ten. Is any of them considered as two objects?) Also, those parts seem to fabricated using the same material thus only differ in geometry. How would Taccel handle objects of different materials, e.g. wooden plate VS metal plate?\n* The paper mentions \"the simulation time scales linearly with the number of mesh vertices\". Does it mean simulating sensor contact with objects with more complicated geometries would take a longer time? How does it generalize if we want to simulate the gel pad with higher resolution, emulating a higher-resolution embedded camera? How does it generalize to daily objects? i.e. Taccel achieves 18x faster than wall clock time with \"bolt and nut\" task; how does it change with more complex but still common object manipulation tasks?", "was_revised": true, "final_justification": "Thanks for the detailed rebuttal and thoughtful insights provided in the open discussion. The rebuttal addresses most of my concerns and I would like to raise my score to \"Strong Accept\".", "rebuttal": "We thank the reviewers and the chairs for their time and effort in providing highly valuable feedback. We appreciate the reviewers for recognizing our formulation sound (ePNg, TkE2), experiments thorough (ePNg, VcE1, TkE2, rfK1), performance significantly improved (ePNg, VcE1, rfK1), and the paper well-written (VcE1, TkE2). Here we address your concerns and answer your questions below.\n\n## Object Modeling, Material Compatibility, and Their Overhead (W1, Q1)\n\n**Taccel supports a range of object materials** by implementing widely used elastic constitutive models like Neo-Hookean and StVK. It **also accommodates heterogeneous materials** by allowing different parameters (e.g., Young's modulus) to be assigned to each tetrahedral element of an object's mesh. **Simulation speed theoretically remains unaffected**, since the computation follows the same procedure with no extra computational overhead introduced. In practice, different material properties can lead to different physical outcomes (eg., how an object deforms when grasped), which may cause minor variations in the total number of solver iterations needed for the simulation to converge.\n\n## Simulation Speed (W3)\n\nThank you for raising this vital point. **FEM-based simulators like Taccel requires more time to simulate tactile sensor meshes or object meshes with more vertices**, as validated by the Sec. 5.4.\n\nOur high-resolution benchmarks were **intended to stress-test Taccel's performance and stability**. For many practical applications, **a more efficient approach is to simulate only low-resolution marker flows** [1,2], which requires far fewer vertices. When high-resolution tactile images are needed in the loop, relying solely on dense physical simulation is not an efficient approach. **A promising solution is to directly fuse simulated deformation with object surface** [1], which allows for high-resolution tactile image rendering with low-resolution physics simulation. We plan to further explore this in future work.\n\n## Classification Experiment (W2)\n\nIn our object classification experiment, the objects are in similar shape (around 50-100mm in length) and mainly differs on surface geometries (diameter, pitch, thread depth). **The sensor's contact area (40×40mm) is large enough to capture these fine geometric features that distinguish the 10 objects.** We further randomize the object's pose during data collection, ensuring the model is exposed to different parts of each object (eg., a bolt's head vs. its threads) so **it can identify the same object from its different regions**.\n\nWe use 10 individual objects for this experiment. We apologize for the missing object in Figure 5(a) and will correct it in the revision.\n\n## Classifying Objects of Different Materials (W2)\n\nThis is an excellent point. **Theoretically, Taccel can simulate the contact with different materials with various surface textures** (eg., wood grain vs. smooth metal), as long as fine-grained meshes are available.\n\nWhile beyond our main focus in this paper, to practically incorporate these materials inside VBTS simulator could be expensive as it requires fine-grained meshes for some materials (eg., meshes that includes the cracks of the wooden plate). **Intuitive solution may include adjusting the tactile signal with the texture normal maps**, which are easily accessible online. We believe this is a promising direction for future study.\n\n## Sim-to-Real with Sensor Degradation and Variances (Q2)\n\nSensor configuration variances (production variance, elastomer elasticity, assembly error, etc) **could be remedied in Taccel by adding perturbations** to the sensor model as an augmentation during simulation.\n\nHandling sensor degradation (eg., dimming LEDs) is more challenging and is a common issue for all VBTS simulators. Currently, this should be **addressed via recalibration the simulation model** with real world samples.\n\n## Open Discussion on Learning Force Estimation (Q3)\n\nThank you for this thought-provoking question. **VBTS simulators like Taccel do serve as a practical tool for learning estimation of forces.** We provides the APIs to obtain the forces within each FEM element, enabling the automatic generation of large, diverse datasets that pair simulated gel deformations with exact force/torque labels. Data collection protocol and applications have been explored in simpler settings [3-5]. We believe several aspects are important for learning an estimation model:\n\n1. **Systematic Data Generation:** The training data must cover diverse object geometries, contact types (indentation, shear), material properties, and dynamic trajectories.\n2. **Physics-Inductive Models**: Using architectures like Graph Neural Networks (GNNs) over the FEM mesh or Physics-Informed Neural Networks (PINNs) can improve model accuracy and interpretability [6, 7].\n\nCurrently a few challenges remains:\n\n1. **Speed and accuracy trade-offs:** FEM-based simulators like Taccel still have a large space for improvement in speed. Faster speed indicates better scalability and more agile iteration of synthetic data.\n2. **Real-world fine-tuning:** Precise calibration or fine-tuning with real-world data (eg., [3]) is still necessary for better performance. Despite the accuracy of the existing simulators, the sim-to-real gaps inevitably exists. To this end, noises and sensor degradation, as mentioned in Q2, should be systematically modeled and remedied. Such studies were rich in vision (eg., [8]) but still lacking for VBTS simulators.\n\n>[1] Chen, Weihang, et al. \"General-purpose sim2real protocol for learning contact-rich manipulation with marker-based visuotactile sensors.\" *IEEE Transactions on Robotics* 40 (2024): 1509-1526.\n>\n>[2] Si, Zilin, et al. \"Difftactile: A physics-based differentiable tactile simulator for contact-rich robotic manipulation.\" *arXiv preprint arXiv:2403.08716* (2024).\n>\n>[3] Helmut, Erik, et al. \"Learning force distribution estimation for the gelsight mini optical tactile sensor based on finite element analysis.\" *arXiv preprint arXiv:2411.03315* (2024).\n>\n>[4] Higuera, Carolina, et al. \"Sparsh: Self-supervised touch representations for vision-based tactile sensing.\" *Conference on Robot Learning*. PMLR, 2025.\n>\n>[5] Yuan, Wenzhen. *Tactile measurement with a gelsight sensor*. Diss. Massachusetts Institute of Technology, 2014.\n>\n>[6] Shi, Haochen, et al. \"Robocraft: Learning to see, simulate, and shape elasto-plastic objects in 3d with graph networks.\" *The International Journal of Robotics Research* 43.4 (2024): 533-549.\n>\n>[7] Karniadakis, George Em, et al. \"Physics-informed machine learning.\" *Nature Reviews Physics* 3.6 (2021): 422-440.\n>\n>[8] Zhang, Xiaoshuai, et al. \"Close the optical sensing domain gap by physics-grounded active stereo sensor simulation.\" *IEEE transactions on robotics* 39.3 (2023): 2429-2447."}, {"review_id": "TaDqgs9a3Y", "reviewer": "Reviewer_rfK1", "summary": "This paper presents a simulation framework for vision-based tactile sensors based on Incremental Potential Contact (IPC) and Affine Body Dynamics (ABD) in which IPC guarantees inversion- and intersection-free contact solutions and ABD is for efficient and precise simulation of stiff materials. This implementation is based on NVIDIA Warp and can achieve high FPS through parallelization. It supports a wide range of robotic tasks and environment setups and its performance is validated by many robotic tasks using tactile signals.", "questions": "The presentation is clear and I don't have any questions.", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 1, "strengths_and_weaknesses": "Strengths: This work presented in this paper is very complete, including very detailed technical and implication details, sufficient experiment across many scenarios, and benchmarks compared with prior works. The proposed tactile simulation framework is more stable and can handle more complex physical interactions with smaller sim2real gap. It also achieves higher computational efficiency enabling more simulation environments running in parallel.\nWeaknesses: This work is valuable but more like an incremental project. The combination of IPC and ABD has been implemented in other simulators for the same purpose and the tactile RGB image generation has also been explored previously. It is clear that this work outperforms the previous works but the technical novelty is marginal.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewers and the chairs for their time and effort in providing highly valuable feedback. We appreciate the reviewers for recognizing our formulation sound (ePNg, TkE2), experiments thorough (ePNg, VcE1, TkE2, rfK1), performance significantly improved (ePNg, VcE1, rfK1), and the paper well-written (VcE1, TkE2). Here we address your concerns and answer your questions below.\n\n## Technical Novelty and Contributions\n\nThank you for your feedback. While we build upon foundational techniques like IPC and ABD, **Taccel's novelty and primary contribution lie in the synergistic integration and engineering advancements** that enables improved scalability for massive parallelization and the flexibility to integrate various robots and sensors.  We also introduce a few new features to better support robotic manipulation:\n\n- **A solver pipeline redesigned for multi‑environment execution**, enabling numerous environments to run efficiently in parallel and making large‑scale robot learning feasible\n- **Complete functions for VBTS simulation**, including tools for tactile sensor fabrication, tactile signal generation in various formats, etc.\n- **Improved infrastructure**, which is also compatible with NVIDIA `warp.sim`, which allows Taccel to leverage additional functions from `warp`, including the OpenGL-based and ray-tracing-based renderer, disentangled simulation and rendering.\n\nIn essence, while the components may be known, Taccel represents a significant engineering achievement that pushes the boundary from few-instance, slow simulation to large-scale, high-performance experimentation."}]

Author rebuttal:

Meta-review: The paper presents Taccel, a high-performance GPU-based simulation platform for vision-based tactile sensors. By integrating IPC and ABD, the system achieves stable, scalable simulation of contact-rich interactions, supporting thousands of environments in parallel. The authors validate the framework across multiple tasks, including object classification, grasping, and articulated object manipulation, with promising sim-to-real results.

The reviews are overall positive (two Strong Accepts, one Accept, one Borderline Accept). The rebuttal addresses key concerns, including sensor generality, simulation under dynamic contacts, and comparisons with recent work such as TacEx. While some components build on existing techniques, the engineering effort and system-level contribution are solid. One reviewer raises concerns about novelty, but others emphasize the practical value and thorough validation.

On balance, this is a well-executed systems paper that will be of interest to the tactile robotics and simulation communities. The AC recommends acceptance.
Decision: Accept (spotlight)