robbyant/lingbot-touch-small

LingBot-Touch: A Foundation Model for Heterogeneous Tactile Representation Learning

LingBot-Touch is a JEPA-based tactile representation model that bridges heterogeneous tactile signals and dexterous robotic manipulation. It uses modality-specific tokenizers and a shared Transformer to learn representations of contact structure and temporal changes through prediction in latent space.

The model covers three sensing modalities: magnetic, piezoresistive, and visuotactile.

• Magnetic: three-axis tactile measurements.
• Piezoresistive: scalar tactile measurements placed in the Z channel, with X and Y set to zero.
• Visuotactile: decoded scalar deformation or intensity measurements placed in the Z channel, with X and Y set to zero. Raw RGB images are not model inputs.

Model Bundles

Model	Encoder Weights	Inference Configuration
LingBot-Touch-Small	encoder.pt	config.json

Each bundle contains a matched config.json and encoder.pt. The configuration describes the inference architecture and modality routes; the weight file contains only the frozen target encoder's parameter tensors. Keep the two files together.

This model repository distributes weights and their necessary inference configurations only. It does not include inference source code, sensor adapters, sample datasets, predictors, downstream task heads, optimizer states, training configurations or training logs. The compatible LingBot-Touch inference implementation is distributed separately.

Input and Output

The encoder consumes normalized tactile signals with shape [B,T,5,M,3], canonical taxel coordinates, and a sensing-modality selection. The fixed finger order is thumb, index, middle, ring and pinky. Measurement X/Y are normalized to [-1,1] and Z to [0,1]. Sensor geometry, handedness, point ordering and fixed measurement ranges must be adapted correctly before inference.

The output consists of token-level latent representations and the corresponding temporal, finger and spatial addresses. These features can be used for downstream tactile perception and robotic manipulation; attention-based querying is recommended for task-specific readout.

License

The model bundles are distributed under the Apache License, Version 2.0.