VoMP: Predicting Volumetric Mechanical Property Fields

## Abstract

Abstract Physical simulation relies on spatially-varying mechanical properties, often laboriously hand-crafted. VoMP is a feed-forward method trained to predict Young’s modulus( E E ), Poisson’s ratio( ν \nu ), and density( ρ \rho ) throughout the volume of 3D objects, in any representation that can be rendered and voxelized. VoMP aggregates per-voxel multi-view features and passes them to our trained Geometry Transformer to predict per-voxel material latent codes. These latents reside on a manifold of physically plausible materials, which we learn from a real-world dataset, guaranteeing the validity of decoded per-voxel materials.
To obtain object-level training data, we propose an annotation pipeline combining knowledge from segmented 3D datasets, material databases, and a vision-language model, along with a new benchmark. Experiments show that VoMP estimates accurate volumetric properties, far outperforming prior art in accuracy and speed.

## 1 Introduction

Accurate physics simulation is a critical part of modern design and engineering, for example, in workflows like creating Digital Twins (virtual replicas of real systems) Grieves2017, Real-2-Sim (generating digital simulation from the real world) omniverse, and Sim-2-Real (transfering policies trained in simulation to real-world deployment) rudin2021learning. However, setting up reliable simulations remains labor-intensive,
partially due to the necessity to provide accurate mechanical properties throughout the volume of every object, namely the spatially-varying
Young’s Modulus($E$), Poisson’s ratio($\nu$), and density($\rho$). Common 3D capture methods kerbl3Dgaussians and
3D repositories objaverse rarely contain such annotations, forcing artists and engineers to guess or copy-paste coarse material presets in a subjective, time-consuming process. We focus on automatic prediction of these parameters, addressing important limitations of prior art.

We propose VoMP, *the first feed-forward model trained to estimate simulation-ready mechanical property fields $(E,\nu,\rho)$ within the volume of 3D objects across representations*. Rather than specializing on inputs like Gaussian Splats shuai2025pugszeroshotphysicalunderstanding; xie2024physgaussian, our method works for any geometry that can be voxelized and rendered from turnaround views, including meshes, Gaussian Splats, NERFs and SDFs([Fig.˜1](https://arxiv.org/html/2510.22975v1#S0.F1)). Unlike virtually all prior works, VoMP is fully feed-forward, requiring
no per-object optimization of feature fields zhai2024physicalpropertyunderstandinglanguageembedded; shuai2025pugszeroshotphysicalunderstanding or run-time aggregation of Vision-Language Model (VLM)  lin2025phys4dgenphysicscompliant4dgeneration or Video Model lin2025omniphysgs supervision.
Uniquely among others, VoMP outputs true mechanical properties (a.k.a. material parameters), like those measured in the real world.
Many existing pipelines target fast, approximate simulators, resulting in simulator-specific parameters physdreamer; huang2024dreamphysicslearningphysicsbased3d that may not transfer reliably across frameworks ([Fig.˜2](https://arxiv.org/html/2510.22975v1#S1.F2)), whereas our result is directly compatible with any accurate simulator. Finally, unlike prior art, our method is designed to assign materials throughout the object volume, which is critical for simulation fidelity.

To enable learning physically valid mechanical properties, we first train a latent space on a database of real-world values $(E,\nu,\rho)$ using a variational auto-encoder MatVAE (§[3](https://arxiv.org/html/2510.22975v1#S3)).
To predict mechanical property fields for 3D objects, our method first voxelizes the input geometry and aggregates multi-view image features
across the voxels (§[4.1](https://arxiv.org/html/2510.22975v1#S4.SS1)). This process accepts many representations(^1^11We describe available methods for meshes, SDFs, and NeRFs, and present a method for Splats in §[6.1](https://arxiv.org/html/2510.22975v1#S6.SS1).) and is fast, unlike optimization used in concurrent work le2025pixiefastgeneralizablesupervised. We pass the voxel features through the Geometry Transformer (§[4.2](https://arxiv.org/html/2510.22975v1#S4.SS2)), trained to output per-voxel material latents. The MatVAE latent space decouples learning material assignments for objects from learning what materials are valid, ensuring that the final volumetric properties $(E,\nu,\rho)$ decoded by MatVAE are physically valid, even in the case of interpolation. To create material property fields for training, we propose a pipeline (§[5](https://arxiv.org/html/2510.22975v1#S5)) combining the knowledge from part-segmented 3D assets, material databases, visual textures, and a VLM. Our experiments(§[6](https://arxiv.org/html/2510.22975v1#S6)) show
that VoMP estimates simulation-ready spatially-varying mechanical properties across a range of object classes and representations, resulting in realistic elastodynamic simulations.
We evaluate our method on an existing mass prediction benchmark and contribute a new material estimation benchmark (§[6.3](https://arxiv.org/html/2510.22975v1#S6.SS3)), consistently outperforming prior art shuai2025pugszeroshotphysicalunderstanding; lin2025phys4dgenphysicscompliant4dgeneration; zhai2024physicalpropertyunderstandinglanguageembedded. In summary, our contributions are:

- •
The first (to our knowledge) method to estimate object mechanical material property fields that (1) is a trained feed-forward model with minimal preprocessing, (2) generalizes across 3D representations, (3) predicts physically valid properties that can be used with an accurate simulator, and (4) predicts mechanical properties *within the volume* of objects (§[4](https://arxiv.org/html/2510.22975v1#S4)).
- •
The first (to our knowledge) mechanical properties latent space (§[3](https://arxiv.org/html/2510.22975v1#S3)).
- •
An automatic data annotation pipeline and a new benchmark for volumetric physics materials (§[5](https://arxiv.org/html/2510.22975v1#S5)).
- •
Thorough evaluation through high-fidelity simulations and quantitative metrics on existing and new benchmarks, significantly outperforming the prior art (§[6](https://arxiv.org/html/2510.22975v1#S6)).

Figure: Figure 2: Simulator differences when dropping a solid sphere with $(E,\nu,\rho)=(10^{4}Pa,0.3,10^{3}\operatorname{kg/m}^{3})$ with XPBD xpbd and MPM mpm vs. more accurate FEM.
Refer to caption: x2.png

## 2 Related Work

### 2.1 Background

All algorithms for continuum-based simulation of solids and liquids require material models as input. The material, or constitutive, model is the function that determines the force response of a class of materials (e.g., rubbers, snow, water) to internal strains and strain rates. To produce the correct constitutive behavior for a given material, the model requires an accurate set of corresponding material parameters for every point in the simulated volume. For locally isotropic material models, Young’s modulus ($E$, in the 1D linear regime, the proportionality constant between stress and strain), Poisson’s ratio ($\nu$, the negative ratio of transverse to axial strain under uniaxial loading) and density ($\rho$, unit mass per volume) are ubiquitous. Given an accurate and valid triplet ($E$, $\nu$, $\rho$) along with a reasonable material model, a consistent numerical simulation can produce accurate predictions of an object’s behavior under load. Measured, real-world parameters are portable to any consistent simulation algorithm (we use high-resolution Finite Element Methods). Further, they are portable across any material model that relies on density, Young’s modulus and Poisson’s ratio, or derived quantities, such as shear or bulk modulus (e.g., Neo-Hookean, St. Venant–Kirchhoff, As-Rigid-As-Possible, Co-Rotated Elastic, Mooney–Rivlin, and Ogden models).
On the other hand, many physics simulation algorithms are not implemented or applied in a consistent fashion, favoring speed over accuracy xpbd; mpm. In these cases, material parameters must be modified to avoid inaccurate behavior ([Fig.˜2](https://arxiv.org/html/2510.22975v1#S1.F2)).

### 2.2 Inferring Mechanical Properties of Static Objects

Our goal is to predict volumetric mechanical properties given only shape and appearance, a challenging inverse problem,
which research suggests humans learn good intuition about
10.1117/12.429489; FLEMING201462; 10.1167/13.8.9; sharan2009material. However, progress in learning-based approaches has been hampered by limited data.
Existing datasets are small gao2022objectfolder20multisensoryobject; downs2022googlescannedobjectshighquality; chen2025matpredictdatasetbenchmarklearning,
contain noisy labels lin2018learning,
use simulator-specific parameters mishra2024latticemldatadrivenapplicationpredicting; Xie__2025; belikov2015material,
provide only coarse annotations ahmed20253dcompat200; slim20233dcompat; 10.1007/978-3-031-20074-8_7 or are biased towards rigid or man-made objects cao2025physx-3d. Worse, data collection is difficult, relying on rigorous physical experiments ASTM_D638_2022; ASTM_E8_E8M_2024; pai2000robotics, and even then lacking spatial material fields loveday2004tensile due to digitization and annotation challenges.

As a result, works that infer physical properties from appearance often leverage knowledge from large pre-trained models.
NeRF2Physics zhai2024physicalpropertyunderstandinglanguageembedded and PUGS shuai2025pugszeroshotphysicalunderstanding optimize language-embedded feature fields for a NeRF mildenhall2020nerf or 3D Gaussians kerbl3Dgaussians, respectively, to predict coarse stiffness categories and density, but require per-object optimization and are limited in their ability to predict values inside objects due to the lack of meaningful features inside NeRFs or splats. Many approaches distill signals from a Video Generation Model and optimize physics parameters by backpropagating through fast, approximate physics simulators, resulting in a slow optimization process, yielding
materials deviating from real-world values and overfit to a specific simulation setup physdreamer; huang2024dreamphysicslearningphysicsbased3d; Liu_2025_CVPR; cleach2023differentiablephysicssimulationdynamicsaugmented; liu2024physics3d; lin2025omniphysgs (§[2.1](https://arxiv.org/html/2510.22975v1#S2.SS1)). Many methods are also tailored
to a specific 3D representation or real-time simulation implementations, such as Splats  xie2024physgaussian or explicit Material Point Methods mpm; le2025pixiefastgeneralizablesupervised, or work with coarse material categories  fischer2024sama; hsu2024autovfxphysicallyrealisticvideo; lin2025phys4dgenphysicscompliant4dgeneration; Xia_2025_CVPR that must be manually mapped to simulation parameters. Instead, we aim to augment
objects across 3D representations with fine-grained spatially-varying mechanical properties that are physically accurate and compatible across accurate simulators.
Like our method, many techniques leverage vision-language (VLM) models. PhysGen liu2024physgen and PhysGen3D Chen_2025_CVPR use a VLM to infer mass, elasticity, and friction for segmented parts of a single image. Phys4DGen lin2025phys4dgenphysicscompliant4dgeneration uses a VLM to annotate parts of a 3D model with coarse material labels, which are then mapped to physical parameters, a baseline used in our evaluation. Most works above rely on aggregation of large model outputs for every input shape, which can be brittle and time-consuming at run-time, and can only leverage external segmentation. Instead, our method uses a VLM paired with other data sources to annotate a *training dataset* for a feed-forward model leveraging 3D data to annotate and learn internal material composition.

Like our method, SOPHY cao2025sophylearninggeneratesimulationready, PhysX-3D cao2025physx-3d, PhysSplat zhao2024efficient_aka_automated; zhao2024automated (a.k.a. SimAnything) and the concurrent Pixie le2025pixiefastgeneralizablesupervised leverage pretrained models and 3D data to annotate a *training* dataset with physical materials. PhysSplat trains a network to predict spatially-varying simulator-specific material offset weights for MPM by using outputs from video distillation liu2024physics3d, not focusing on material accuracy.
SOPHY and PhysX-3D are 3D generative models, designed to
generate new shapes augmented with physical attributes, and cannot augment existing assets, which is our goal. Still, we detail similar aspects of these works.
Like these works, our method uses a VLM to annotate 3D objects with Young’s Modulus, Poisson’s ratio, and density, but
we do not rely on the human-in-the-loop and instead leverage multiple data sources, not just VLM knowledge, to ensure more accurate physical properties.
As a baseline, SOPHY does implement a material decoder, but it has not been made available, and only considers object surface, while we aim to estimate volumetric properties.
Like our method, PhysX-3D adopts the structural latent space of TRELLIS, but trains a joint generative model over these and learned shape-aware physical properties latents in order to generate physics-augmented shapes from scratch. In contrast,
we treat material prediction as deterministic inference for simplicity, and further adjust the TRELLIS pipeline to facilitate
accurate material prediction inside the object. Pixie le2025pixiefastgeneralizablesupervised, a concurrent work and the only other feed-forward approach, is trained on semantically-segmented objects and uses points from filtering NeRF densities. Thus, Pixie is trained on segments biased toward the surface, as we show in Fig. [14](https://arxiv.org/html/2510.22975v1#A2.F14), while we demonstrate being able to estimate volumetric properties with internal structures. Furthermore, unlike Pixie, we specifically focus on estimating physically plausible material properties, such as those measured in the real world.

Figure: Figure 3: VoMP Overview. For any input geometry, we aggregate multi-view DINOv2 features across its volumetric voxelization (§[4.1](https://arxiv.org/html/2510.22975v1#S4.SS1)). A trained GeometryTransformer (§[4.2](https://arxiv.org/html/2510.22975v1#S4.SS2)) predicts per-voxel material latents, decoded by MatVAE (§[3](https://arxiv.org/html/2510.22975v1#S3)) into mechanical properties ($E$, $\nu$, $\rho$).
Refer to caption: x3.png

## 3 Mechanical Properties Latent Space

To learn a latent space of valid Young’s modulus, Poisson’s ratio, and density triplets ($E$, $\nu$, $\rho$) ([§˜2.1](https://arxiv.org/html/2510.22975v1#S2.SS1)), we propose MatVAE,
a variational autoencoder (VAE) trained on a dataset of real-world values $\{m_{i}:=(E_{i},\;\nu_{i},\;\rho_{i})\}$ ([§˜5.1](https://arxiv.org/html/2510.22975v1#S5.SS1)). The model’s objective is to map these triplets $m$ into a 2-dimensional latent space, $z\in\mathbb{R}^{2}$, from which they can be accurately reconstructed. While this offers only minor compression ($\mathbb{R}^{3}\to\mathbb{R}^{2}$), this latent 2D space of material
properties is now easy to visualize, sample, and interpolate within, and
results in consistent distances between material triplets with
disparate units ([Fig.˜7](https://arxiv.org/html/2510.22975v1#S6.F7),[§˜6.4](https://arxiv.org/html/2510.22975v1#S6.SS4)). MatVAE acts like a continuous tokenizer that allows us to always ensure VoMP output properties that fall inside the range of some materials.

We build on VAE kingma2022autoencodingvariationalbayes,
with the reconstruction component of the loss defined as
mean-squared error between the input $(E_{i},\nu_{i},\rho_{i})$
and reconstructed material values $(\hat{E}_{i},\hat{\nu}_{i},\hat{\rho}_{i})$:

$$ $\mathcal{L}_{\text{Recon}}=\frac{1}{N}\sum_{i=1}^{N}\bigl\|((E_{i},\nu_{i},\rho_{i})^{N})^{\mathsf{T}}-((\hat{E}_{i},\hat{\nu}_{i},\hat{\rho}_{i})^{N})^{\mathsf{T}}\bigr\|_{2}^{\,2},$ (1) $$

where ${\mathsf{T}}$ denotes transpose and ${N}$ per-property normalization, where $E$ and $\rho$ are first log-transformed ($\log_{10}(E)$, $\log_{10}(\rho)$), then normalized to $[0,1]$, while $\nu$ is directly normalized to $[0,1]$. We find other normalization schemes without $\log$-transform or standard $z$-score normalization induce a heavy-tailed feature distribution, which is poorly conditioned for learning (§[C](https://arxiv.org/html/2510.22975v1#A3)).

We make several modifications over standard VAE. First, to capture a complex posterior beyond a simple Gaussian, the encoder’s output is transformed by a (radial) Normalizing Flow pmlr-v37-rezende15, giving us a more flexible variational distribution $q_{\phi}(z|m)$ since we observe heavy-tailed distribution for Young’s Modulus and Density while Poisson’s Ratio concentrates near the boundaries after normalization. Second, we decompose the KL-divergence term of the ELBO following chen2018isolating. This allows us to directly penalize the total correlation $\text{TC}(z)=\text{KL}(\bar{q}_{\phi}(z)||\prod_{j}\bar{q}_{\phi}(z_{j}))$ where $\bar{q}_{\phi}(z)$ is the aggregated posterior, $z_{j}$ is the $j\in\{1,2\}$-th coordinate of the latent vector $z$. Penalizing TC allowed us to reduce the high dependence between latent coordinates which caused MatVAE to encode density in both dimensions. Third, we observe imbalanced reconstruction, i.e. the latent space collapses to one property, giving us low reconstruction errors for one property and high reconstruction error for others (§[C](https://arxiv.org/html/2510.22975v1#A3)).
Thus, to ensure the 2 latent dimensions are actively utilized, we introduce a capacity constraint ($\delta\times z_{\text{dim}}$) based on higgins2017beta, resulting in the following final objective:

$$ $\displaystyle\mathcal{L}_{\text{MatVAE}}$ $\displaystyle=\underbrace{\mathcal{L}_{\text{Recon}}}_{\text{MSE}}+\overbrace{\underbrace{\gamma\cdot\text{MI}(z)}_{\text{Mutual Information}}+\underbrace{\beta\cdot\text{TC}(z)}_{\text{Total Correlation}}}^{\text{Latent Space Regularization}}+\alpha\cdot\sum_{j=1}^{d}\underbrace{\max\!\big(\delta,\ \text{KL}(q_{\phi}(z_{j})\,\|\,p(z_{j}))\big)}_{\text{Dimension-wise KL}},$ (2) $$

where we set $\gamma,\beta,\alpha=(1.0,2.0,1.0)$, with a free nats constraint $\delta=0.1$. See §[F.1](https://arxiv.org/html/2510.22975v1#A6.SS1) for more details.

## 4 Predicting Mechanical Property Fields

To predict volumetric mechanical properties across 3D representation, VoMP first aggregates volumetric features for the input geometry (§[4.1](https://arxiv.org/html/2510.22975v1#S4.SS1)), which are then processed by a trained feed-forward transformer model (§[4.2](https://arxiv.org/html/2510.22975v1#S4.SS2)) that learns in the latent space of MatVAE (§[3](https://arxiv.org/html/2510.22975v1#S3)). See [Fig.˜3](https://arxiv.org/html/2510.22975v1#S2.F3).

### 4.1 Aggregating Features

Our method accepts any 3D representation that can be voxelized and rendered from multiple views. Following recent works wang2023autorecon; Dutt_2024_CVPR; xiang2025structured3dlatentsscalable, we compute rich DINOv2 oquab2024dinov2learningrobustvisual image features across 3D views and lift them to 3D by projecting each voxel center into every view using the camera parameters to retrieve the corresponding image features. The retreived image features are then averaged to obtain a feature for every voxel.
A critical difference with these prior works is that we also voxelize and process the interior of the objects and not just their surface, which allows us to learn and predict material properties *inside* the objects (See §[6.1](https://arxiv.org/html/2510.22975v1#S6.SS1) for voxelization schemes and see §[F.3](https://arxiv.org/html/2510.22975v1#A6.SS3) for details on voxelization for training).
Let’s denote all active voxel center positions in a 3D grid of size $N^{3}$ as $\{\mathbf{p}_{i}\}_{i=1}^{L}$ where $L$ denotes the number of voxels, $\mathbf{p}_{i}\in\mathbb{R}^{3}$ denotes the voxel center, and $\Pi_{j}:\mathbb{R}^{3}\to[-1,1]^{2}$ the camera projection for view $j\in J$ where $J$ is the set of rendered views. Let the DINOv2 patch-token map be $T_{j}\in\mathbb{R}^{1024\times n\times n}$ which is bilinearly sampled to get a feature map $\mathcal{F}_{j}:[-1,1]^{2}\to\mathbb{R}^{1024}$. Then for each voxel $i\in\{1,2,\ldots,L\}$, we obtain a feature $\mathbf{f}_{i}$:

$$ $\mathbf{f}_{i}=\operatorname{Average}(\mathcal{C}_{i}=\left\{\mathcal{F}_{j}\!\big(\Pi_{j}(\mathbf{p}_{i})\big)\;\big|\;j\in J\right\})\in\mathbb{R}^{1024}$ (3) $$

This propagates multi-view information to the voxels in the interior of the object, encoding useful information that our model learns to process to predict internal material composition.

### 4.2 Geometry Transformer

The main component of VoMP is a Transformer $\mathbf{F}$
that maps voxelized image features to our trained material latent representation. The backbone of our model follows TRELLIS xiang2025structured3dlatentsscalable encoder/decoder, and the backbone layers of our model are initialized with TRELLIS weights. The encoder processes a variable-length set of active voxels, represented by their positions and features $\mathbf{X}=\{(\mathbf{p}_{i},\mathbf{f}_{i})\}_{i=1}^{L}$. To make this data suitable for a Transformer, we first serialize the voxel features into a sequence and then inject spatial awareness by adding sinusoidal positional encodings derived from each voxel’s 3D coordinates. Similar to TRELLIS and state-of-the-art 3D Transformers, we adopt a 3D shifted window attention mechanism liu2021swin; yang2025swin3d.
Contrary to TRELLIS xiang2025structured3dlatentsscalable, to handle assets of various sizes, we define a maximum sequence length of $L_{N}$. For assets with fewer voxels $L\leq L_{N}$, we use the complete set. However, for larger assets where $L>L_{N}$, we use a stochastic sampling strategy, selecting a random subset of $L_{N}$ voxels at the start of each training epoch. This dynamic resampling ensures the model is exposed to different parts of the asset over epochs and have a larger number of "effective" max voxels.

For each training asset, we first define $\mathcal{S}$ as the set of voxel indices to be processed in the current iteration. The corresponding sequence of image features $\mathbf{X}_{\mathcal{S}}$ obtained from voxel features ([§˜4.1](https://arxiv.org/html/2510.22975v1#S4.SS1)), is passed to $\mathbf{F}$. The resulting latent representation is then fed into the frozen decoder of pre-trained MatVAE to predict material properties. The MatVAE is run $L$ times i.e. once per voxel, which gives us material triplets $(E,\nu,\rho)$ for each voxel. We train this transformer with the mean squared error between the predicted materials and the ground truth materials, averaged over all voxels in the set $\mathcal{S}$,

$$ $\mathcal{L}_{\mathbf{F}}=\frac{1}{|\mathcal{S}|}\sum_{i\in\mathcal{S}}\|\mu_{\theta}(\mathbf{F}(\mathbf{X}_{\mathcal{S}})_{i})-((E_{i},\nu_{i},\rho_{i})^{N})^{\mathsf{T}}\|_{2}^{2},$ (4) $$

where $\mu_{\theta}(\cdot)$ denotes the output of the frozen MatVAE decoder, $((E_{i},\nu_{i},\rho_{i})^{N})^{\mathsf{T}}$ is the ground truth material vector for voxel $i$, and $\mathbf{F}(\mathbf{X}_{\mathcal{S}})_{i}$ is the latent representation for voxel $i$.

To transfer voxel materials back to the original representation (i.e. splat means, tets for FEM simulation, quadrature points for simulation, etc.), we use nearest neighhbour interpolation as outlined in §[G.1](https://arxiv.org/html/2510.22975v1#A7.SS1). The per-voxel latents are passed into the decoder model of MatVAE ([§˜3](https://arxiv.org/html/2510.22975v1#S3)), which yields per-voxel material triplets, as shown in [Fig.˜3](https://arxiv.org/html/2510.22975v1#S2.F3).

## 5 Training Data Generation

### 5.1 Material Triplets Dataset (MTD)

To train MatVAE ([§˜3](https://arxiv.org/html/2510.22975v1#S3)), we collect Material Triplet Dataset(MTD), containing 100,562
triplets $(E,\nu,\rho)$ for real-world materials. We first collect a dataset of measured material properties
from multiple online databases matweb; wikipedia_density; wikipedia_poissons_ratio; wikipedia_youngs_modulus; engineeringtoolbox_materials; cambridge_materials_databook, containing values obtained experimentally, typically
with valid *ranges* for all three properties for all materials. We sample numeric triplets from
each material, with the number of samples proportional to the range size. Finally, we filter out duplicates resulting
from overlapping ranges for some materials.

### 5.2 Geometry with Volumetric Materials (GVM) Dataset

Figure: Figure 4: Training Data annotation leverages accurate 3D data labels together with a VLM.
Refer to caption: x4.png

To train Geometry Transformer([§˜4](https://arxiv.org/html/2510.22975v1#S4)), we develop an automatic annotation pipeline to overcome the
limited availability of detailed volumetric material datasets (§[2.2](https://arxiv.org/html/2510.22975v1#S2.SS2)). Like
prior works lin2025phys4dgenphysicscompliant4dgeneration; cao2025sophylearninggeneratesimulationready; le2025pixiefastgeneralizablesupervised, we leverage a pre-trained VLM, but overcome
its limitations by introducing *additional sources of knowledge* present in our 3D dataset and the MTD([§˜5.1](https://arxiv.org/html/2510.22975v1#S5.SS1)).
We collect high-quality 3D meshes from nvidia_omniverse_commercial_assets; nvidia_omniverse_residential_assets; nvidia_omniverse_simready; nvidia_omniverse_vegetation_assets, containing 1624 part-segmentated 3D models, with a total of 8089 parts, and treat each part as having isotropic material. Each part contains an English material name and its own realistic PBR texture, which can be used
as additional cues to the VLM. For each part in each object, we pass the following information to the VLM: rendering of the full object, detail rendering of the part’s visual material mapped onto a sphere (showing visual aspects that tend to correlate with material composition), the material names, and the ranges of three closest real-world materials in the MTD ([§˜5.1](https://arxiv.org/html/2510.22975v1#S5.SS1)) based on the material names (See Fig.[4](https://arxiv.org/html/2510.22975v1#S5.F4), detailed prompt in Fig. [22](https://arxiv.org/html/2510.22975v1#A5.F22)). The vision-language model then outputs material triplets for each part, and we map to all volumetric voxels within it, resulting in a total of 37M voxels annotated with ($E$, $\nu$, $\rho$). By guiding VLM with real-world material values and extra clues, we avoid inaccuracies and implausible material values. See additional details in §[6.1](https://arxiv.org/html/2510.22975v1#S6.SS1), §[E](https://arxiv.org/html/2510.22975v1#A5).

## 6 Experiments and Results

We evaluate VoMP end-to-end, showing diverse realistic simulations in §[6.2](https://arxiv.org/html/2510.22975v1#S6.SS2). Quantitative results are presented in §[6.3](https://arxiv.org/html/2510.22975v1#S6.SS3), with MatVAE evaluated separately in §[6.4](https://arxiv.org/html/2510.22975v1#S6.SS4). See video and §[A](https://arxiv.org/html/2510.22975v1#A1) for many additional results, §[B](https://arxiv.org/html/2510.22975v1#A2) for extra comparisons with concurrent work and §[C](https://arxiv.org/html/2510.22975v1#A3) for ablations.

### 6.1 Implementation Details

Voxelization:
For voxelizing 3D Gaussian splats kerbl3Dgaussians, we present a new voxelizer, that works in three phases:
(1) 3D Gaussians are voxelized over a 3D grid as solid ellipsoids defined by the 99th percentile iso-surface, (2) this set of voxels is rendered from several dozen viewpoints sampled over a sphere to form depth maps, (3) these depthmaps are used to carve away empty space around the exterior of the object, but leaving unseen *interior* voxels to form a solid approximation of the object. We then sample this solid at jittered sample points on a regular grid. We employ octrees as acceleration structures and GPU implementations for both phases. Our test objects can be voxelized in 31 ms (see  [§˜6.3](https://arxiv.org/html/2510.22975v1#S6.SS3)).
To voxelize meshes and SDFs we use standard methods (see §[F.3](https://arxiv.org/html/2510.22975v1#A6.SS3)).

Data and Training: For material annotation, we experimentally choose Qwen 2.5 VL-72B VLM bai2023qwenvlversatilevisionlanguagemodel; bai2025qwen25vltechnicalreport. We partition our MTD and GVM datasets (§[5](https://arxiv.org/html/2510.22975v1#S5)) into 80-10-10 train, validation, and test sets. See §[E](https://arxiv.org/html/2510.22975v1#A5) for data details. For rendering we use Omniverse nvidia_omniverse_replicator and Blender blender, and for DINOv2 we use an optimized implementation nvidia_nv_dinov2_2025. During training and testing, we set the maximum number of non-empty voxels per object $L_{N}=32,768$ (sampled stochastically, [§˜4.2](https://arxiv.org/html/2510.22975v1#S4.SS2)), and sparse data structures for efficiency. See §[F](https://arxiv.org/html/2510.22975v1#A6) for more details. All experiments were performed on a machine with four 80GB A100 GPUs, where training took about $12$ hours for MatVAE and 5 days for the Transformer.

Simulation: We used FEM simulator for meshes and sparse Simplicits 10.1145/3658184; KaolinLibrary for our large-scale simulations combining splats and meshes. Details in §[G](https://arxiv.org/html/2510.22975v1#A7).

Figure: Figure 5: Simulation-ready physics materials of VoMP enable realistic simulations for meshes and splats.
Refer to caption: x6.png

### 6.2 End-to-End Qualitative Evaluation

We qualitatively evaluate VoMP by using it to annotate volumetric mechanical fields for several meshes and 3D Gaussian Splats, and running
physics simulation with these exact spatially varying ($E$, $\nu$, $\rho$) values, resulting in realistic simulations without any hand-tweaks ([Fig.˜5](https://arxiv.org/html/2510.22975v1#S6.F5), [Fig.˜8](https://arxiv.org/html/2510.22975v1#A1.F8), : 0:36).
We also show that our approach can work across more representations, including meshes, 3D Gaussian Splats, SDF, and NeRFs ( [Fig.˜8](https://arxiv.org/html/2510.22975v1#A1.F8), with additional results in §[A.2](https://arxiv.org/html/2510.22975v1#A1.SS2).

### 6.3 Quantitative Evaluation

Datasets and Metrics: The 10% hold-out test set of GVM (§[5](https://arxiv.org/html/2510.22975v1#S5)) consists of 166 high-quality 3D objects with per-voxel mechanical properties for a total of 4.9 million point annotations, significantly larger than previous works, e.g. 31 points across 11 objects  zhai2024physicalpropertyunderstandinglanguageembedded.
We contribute this as *a new benchmark* and use it for evaluation against baselines.
We measure standard metrics, Average Log Displacement Error (ALDE), Average Displacement Error (ADE), Average Log Relative Error (ALRE), and Average Relative Error (ARE) for each mechanical property, further detailed in §[D.1](https://arxiv.org/html/2510.22975v1#A4.SS1). We provide additional intuition for interpreting these errors through targeted simulations in  §[D.4](https://arxiv.org/html/2510.22975v1#A4.SS4).

Figure: Table 1: Wall-clock comparisons and breakdown.

Baselines:
We compare against prior art NeRF2Physics zhai2024physicalpropertyunderstandinglanguageembedded and PUGS shuai2025pugszeroshotphysicalunderstanding, where we look up
material properties at the voxel locations (with proper scaling) using
their optimized representations. Note that these techniques do not output Poisson’s ratio. Phys4DGen  lin2025phys4dgenphysicscompliant4dgeneration is an important
baseline, aggregating VLM prediction directly,
but does not provide code. We used our best effort
to replicate their method and used prompts provided by the authors,
designating this implementation Phys4DGen^⋆. More baseline details in §[F.5](https://arxiv.org/html/2510.22975v1#A6.SS5). We also include early comparisons against
concurrent (and as yet unpublished) Pixie le2025pixiefastgeneralizablesupervised, with additional explorations in §[B](https://arxiv.org/html/2510.22975v1#A2).

Estimating Mechanical Properties:
Quantitative evaluation of material estimates ($E$, $\nu$, $\rho$) of our
method against prior art on our new detailed benchmark shows a *dramatic quality boost across all properties and metrics* ([Fig.˜6](https://arxiv.org/html/2510.22975v1#S6.F6)). According to our explorations (§[D.4](https://arxiv.org/html/2510.22975v1#A4.SS4)), ALRE under $0.05$ for $E$ and ARE under $0.15$ for other properties result in similar simulations, suggesting that our materials will lead to more faithful simulations than competitors when using an accurate simulator.
Qualitatively ([Fig.˜6](https://arxiv.org/html/2510.22975v1#S6.F6)), we observe that this performance difference may be due to baselines occasionally mislabeling segments (e.g. by Phys4DGen), due to noisy estimates (e.g. NeRF2Physics and PUGS), and less accurate values in the objects’ interior due to the baselines’ design.

Run-Time: To show approximate speed difference, we report average material estimate speeds across 100 runs on objects with an average of 53.9K Gaussians for our method and the baselines in [§˜6.3](https://arxiv.org/html/2510.22975v1#S6.SS3). To ensure fair compute between CPU and GPU heavy methods, we ran this experiment on a machine with only one A100 GPU and 64 CPUs. While we do not provide timing breakdown of the other methods, this result suggests a speed
up of 5-100x achieved by our method, which is not surprising given that it is the only feed-forward model among previous work. Concurrent Pixie, which is also feed-forward, involves a heavier pre-processing step, including per-object optimization, affecting its end-to-end time.
In the timing breakdown of our method, rendering and pre-processing take the most time, and could be further optimized.

Figure: (a) Qualitative Comparison: We show that qualitiative VoMP tends to provide less noisy volumetric? values compared to the baselines. We show the color coded fields and slice planes through the fields.
Refer to caption: x8.png

Mass Estimation: Following NeRF2Physics zhai2024physicalpropertyunderstandinglanguageembedded and PUGS shuai2025pugszeroshotphysicalunderstanding, we also evaluate our dataset on the ABO-500 Collins_2022_CVPR object mass estimation
benchmark, following the evaluation protocol of PUGS. We run our model to estimate density $\rho$ for upto 32678 voxels per object, then average these values and multiply by the known object volume to obtain mass. While this is only an imperfect proxy for measuring the accuracy
of volumetric density $\rho$, it is a benchmark used by prior works, and we include it for completeness. We achieve
better or on-par performance across most metrics ([Fig.˜6](https://arxiv.org/html/2510.22975v1#S6.F6)), with qualitative results in  §[A.3](https://arxiv.org/html/2510.22975v1#A1.SS3).

Validity: To gauge how well different methods are at predicting realistic materials, such as those measured in the real world, which is our goal, we leverage our MTD dataset of real materials. First, we run all methods on GVM test set objects, and for each test voxel compute relative errors to the nearest possible material range from MTD (error is 0 for estimates within an existing material range). These errors are averaged across all the voxels and reported in  [Fig.˜6](https://arxiv.org/html/2510.22975v1#S6.F6). We observe that our method, on average, outputs much more realistic materials, as it was explicitly designed to do so.

Figure: (a) MatVAE shows excellent reconstruction errors on the MTD test set across all metrics.
Refer to caption: x9.png

### 6.4 Reconstructing and Generating Materials with MatVAE

Given no prior works exploring a latent space of material triplets ($E$, $\nu$, $\rho$), we evaluate MatVAE on the MTD test
set (§[6.1](https://arxiv.org/html/2510.22975v1#S6.SS1)), achieving low reconstruction errors in  [Fig.˜7](https://arxiv.org/html/2510.22975v1#S6.F7) (See [§˜D.1](https://arxiv.org/html/2510.22975v1#A4.SS1), [§˜D.4](https://arxiv.org/html/2510.22975v1#A4.SS4)) for metrics). Further, in [Fig.˜7](https://arxiv.org/html/2510.22975v1#S6.F7) we show the desirable properties of this learned latent space. In (a), samples throughout the 2D latent space map to real-world material ranges in MTD. In (b), we show that ($E$, $\nu$, $\rho$)values of real materials encoded to the latent space vary smoothly. Further,
the latent space ensures valid interpolation points between materials (c), facilitating valid assignment from predicted voxel materials back to the original geometry. We include detailed ablations of MatVAE design (§[C](https://arxiv.org/html/2510.22975v1#A3)), and additional latent space explorations (§[A.4](https://arxiv.org/html/2510.22975v1#A1.SS4)).

## 7 Discussion

We introduce a representation-agnostic method that maps any 3D asset (mesh, SDF, Gaussian splat, or voxel grid) to a volumetric field of physically valid mechanical properties $(E,\nu,\rho)$. We show that our method significantly outperforms prior art in accuracy and speed, lowering the barrier for integrating accurate physics into digital workflows across 3D representations, with potential impact across digital twins, robotics, and beyond.

While we show important advances over existing works, our method is not without limitations, which we hope will open exciting avenues of future research. Due to fixed-grid voxelization, our output resolution is limited, causing oversmoothing in highly heterogeneous regions, and may result in approximation errors when transferring results to more detailed input geometry.
During annotation, we assume part-level materials are isotropic, which is not a true assumption for some common materials like wood.
Further, future work could extend our method to predict additional properties like yield strength, shear modulus and thermal expansion,
or to adapt true material properties output by our method to simulator-specific scales required for faster algorithms or implementations. We hope to support future directions in this area by releasing our material estimation benchmark, and trained models.

## References

- Adelson (2001)
Edward H. Adelson.
On seeing stuff: the perception of materials by humans and machines.
In Bernice E. Rogowitz and Thrasyvoulos N. Pappas (eds.), *Human Vision and Electronic Imaging VI*, volume 4299, pp. 1 – 12. International Society for Optics and Photonics, SPIE, 2001.
doi: 10.1117/12.429489.
URL [https://doi.org/10.1117/12.429489](https://doi.org/10.1117/12.429489).
- Ahmed et al. (2025)
Mahmoud Ahmed, Xiang Li, Arpit Prajapati, and Mohamed Elhoseiny.
3dcompat200: Language-grounded compositional understanding of parts and materials of 3d shapes.
*arXiv preprint arXiv:2501.06785*, 2025.
- Ashby & Cebon (1993)
Michael F Ashby and David Cebon.
Materials selection in mechanical design.
*Le Journal de Physique IV*, 3(C7):C7–1, 1993.
- ASTM Committee D20 (2022)
ASTM Committee D20.
Standard test method for tensile properties of plastics.
Astm standard d638, ASTM International, West Conshohocken, PA, 2022.
Approved June 30, 2022.
- ASTM Committee E28 (2024)
ASTM Committee E28.
Standard test methods for tension testing of metallic materials.
Astm standard e8/e8m, ASTM International, West Conshohocken, PA, 2024.
ANSI approved.
- ASTM International (2015)
ASTM International.
Standard Test Method for Rubber Property—Durometer Hardness.
ASTM Standard D2240-15, 2015.
URL [https://doi.org/10.1520/D2240-15](https://doi.org/10.1520/D2240-15).
doi:10.1520/D2240-15.
- Bai et al. (2023)
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou.
Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond, 2023.
URL [https://arxiv.org/abs/2308.12966](https://arxiv.org/abs/2308.12966).
- Bai et al. (2025)
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin.
Qwen2.5-vl technical report, 2025.
URL [https://arxiv.org/abs/2502.13923](https://arxiv.org/abs/2502.13923).
- Belikov et al. (2015)
VV Belikov, NP Vabishchevich, PN Vabishchevich, UV Katishkov, and NA Mosunova.
Material property database.
*Mathematical Models and Computer Simulations*, 7:95–102, 2015.
- Bender & contributors (2015)
Jan Bender and contributors.
Positionbaseddynamics: Physically‐based simulation library.
[https://github.com/InteractiveComputerGraphics/PositionBasedDynamics](https://github.com/InteractiveComputerGraphics/PositionBasedDynamics), 2015.
Commit retrieved 3 Aug 2025.
- Bhat et al. (2002)
Kiran S. Bhat, Steven M. Seitz, Jovan Popović, and Pradeep K. Khosla.
Computing the physical parameters of rigid-body motion from video.
In Anders Heyden, Gunnar Sparr, Mads Nielsen, and Peter Johansen (eds.), *Computer Vision — ECCV 2002*, pp. 551–565, Berlin, Heidelberg, 2002. Springer Berlin Heidelberg.
ISBN 978-3-540-47969-7.
- Blender Online Community (2021)
Blender Online Community.
*Blender - a 3D modelling and rendering package*.
Blender Foundation, Blender Institute, Amsterdam, 2021.
URL [http://www.blender.org](http://www.blender.org).
- Bray & Curtis (1957)
J. Roger Bray and J. T. Curtis.
An ordination of the upland forest communities of southern wisconsin.
*Ecological Monographs*, 27(4):325–349, 1957.
doi: https://doi.org/10.2307/1942268.
URL [https://esajournals.onlinelibrary.wiley.com/doi/abs/10.2307/1942268](https://esajournals.onlinelibrary.wiley.com/doi/abs/10.2307/1942268).
- Brubaker et al. (2009)
Marcus A. Brubaker, Leonid Sigal, and David J. Fleet.
Estimating contact dynamics.
In *2009 IEEE 12th International Conference on Computer Vision*, pp. 2389–2396, 2009.
doi: 10.1109/ICCV.2009.5459407.
- Brussee (2025)
Arthur Brussee.
Brush: 3d reconstruction for all.
[https://github.com/ArthurBrussee/brush](https://github.com/ArthurBrussee/brush), 2025.
GitHub repository.
- Cao & Kalogerakis (2025)
Junyi Cao and Evangelos Kalogerakis.
Sophy: Learning to generate simulation-ready objects with physical materials, 2025.
URL [https://arxiv.org/abs/2504.12684](https://arxiv.org/abs/2504.12684).
- Cao et al. (2025)
Ziang Cao, Zhaoxi Chen, Liang Pan, and Ziwei Liu.
Physx-3d: Physical-grounded 3d asset generation.
*arXiv preprint arXiv:2507.12465*, 2025.
- Chen et al. (2025a)
Boyuan Chen, Hanxiao Jiang, Shaowei Liu, Saurabh Gupta, Yunzhu Li, Hao Zhao, and Shenlong Wang.
Physgen3d: Crafting a miniature interactive world from a single image.
In *Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR)*, pp. 6178–6189, June 2025a.
- Chen et al. (2025b)
Chuhao Chen, Zhiyang Dou, Chen Wang, Yiming Huang, Anjun Chen, Qiao Feng, Jiatao Gu, and Lingjie Liu.
Vid2sim: Generalizable, video-based reconstruction of appearance, geometry and physics for mesh-free simulation.
*IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, 2025b.
- Chen et al. (2018)
Ricky T. Q. Chen, Xuechen Li, Roger B Grosse, and David K Duvenaud.
Isolating sources of disentanglement in variational autoencoders.
In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (eds.), *Advances in Neural Information Processing Systems*, volume 31. Curran Associates, Inc., 2018.
URL [https://proceedings.neurips.cc/paper_files/paper/2018/file/1ee3dfcd8a0645a25a35977997223d22-Paper.pdf](https://proceedings.neurips.cc/paper_files/paper/2018/file/1ee3dfcd8a0645a25a35977997223d22-Paper.pdf).
- Chen et al. (2024)
Yunuo Chen, Tianyi Xie, Zeshun Zong, Xuan Li, Feng Gao, Yin Yang, Ying Nian Wu, and Chenfanfu Jiang.
Atlas3d: Physically constrained self-supporting text-to-3d for simulation and fabrication, 2024.
URL [https://arxiv.org/abs/2405.18515](https://arxiv.org/abs/2405.18515).
- Chen et al. (2025c)
Yuzhen Chen, Hojun Son, and Arpan Kusari.
Matpredict: a dataset and benchmark for learning material properties of diverse indoor objects, 2025c.
URL [https://arxiv.org/abs/2505.13201](https://arxiv.org/abs/2505.13201).
- Cheng et al. (2024)
An-Chieh Cheng, Hongxu Yin, Yang Fu, Qiushan Guo, Ruihan Yang, Jan Kautz, Xiaolong Wang, and Sifei Liu.
Spatialrgpt: Grounded spatial reasoning in vision-language models.
In *NeurIPS*, 2024.
- Cleac’h et al. (2023)
Simon Le Cleac’h, Hong-Xing Yu, Michelle Guo, Taylor A. Howell, Ruohan Gao, Jiajun Wu, Zachary Manchester, and Mac Schwager.
Differentiable physics simulation of dynamics-augmented neural objects, 2023.
URL [https://arxiv.org/abs/2210.09420](https://arxiv.org/abs/2210.09420).
- Collins et al. (2022)
Jasmine Collins, Shubham Goel, Kenan Deng, Achleshwar Luthra, Leon Xu, Erhan Gundogdu, Xi Zhang, Tomas F. Yago Vicente, Thomas Dideriksen, Himanshu Arora, Matthieu Guillaumin, and Jitendra Malik.
Abo: Dataset and benchmarks for real-world 3d object understanding.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, pp. 21126–21136, June 2022.
- Davis et al. (2015)
Abe Davis, Katherine L Bouman, Justin G Chen, Michael Rubinstein, Fredo Durand, and William T Freeman.
Visual vibrometry: Estimating material properties from small motion in video.
In *Proceedings of the ieee conference on computer vision and pattern recognition*, pp. 5335–5343, 2015.
- Deitke et al. (2023)
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsanit, Aniruddha Kembhavi, and Ali Farhadi.
Objaverse: A Universe of Annotated 3D Objects .
In *2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, pp. 13142–13153, Los Alamitos, CA, USA, Jun 2023. IEEE Computer Society.
doi: 10.1109/CVPR52729.2023.01263.
URL [https://doi.ieeecomputersociety.org/10.1109/CVPR52729.2023.01263](https://doi.ieeecomputersociety.org/10.1109/CVPR52729.2023.01263).
- Department of Engineering, University of Cambridge (2011)
Department of Engineering, University of Cambridge.
*Materials Data Book*.
University of Cambridge, 2011.
URL [https://teaching.eng.cam.ac.uk/sites/teaching.eng.cam.ac.uk/files/Documents/Databooks/MATERIALS%20DATABOOK%20(2011)%20version%20for%20Moodle.pdf](https://teaching.eng.cam.ac.uk/sites/teaching.eng.cam.ac.uk/files/Documents/Databooks/MATERIALS%20DATABOOK%20(2011)%20version%20for%20Moodle.pdf).
Data extracted from the Cambridge Engineering Selector (CES EduPack), courtesy of Granta Design Ltd. For educational use.
- Downs et al. (2022)
Laura Downs, Anthony Francis, Nate Koenig, Brandon Kinman, Ryan Hickman, Krista Reymann, Thomas B. McHugh, and Vincent Vanhoucke.
Google scanned objects: A high-quality dataset of 3d scanned household items, 2022.
URL [https://arxiv.org/abs/2204.11918](https://arxiv.org/abs/2204.11918).
- Dutt et al. (2024)
Niladri Shekhar Dutt, Sanjeev Muralikrishnan, and Niloy J. Mitra.
Diffusion 3d features (diff3f): Decorating untextured shapes with distilled semantic features.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, pp. 4494–4504, June 2024.
- Feng et al. (2024)
Yutao Feng, Yintong Shang, Xuan Li, Tianjia Shao, Chenfanfu Jiang, and Yin Yang.
Pie-nerf: Physics-based interactive elastodynamics with nerf, 2024.
URL [https://arxiv.org/abs/2311.13099](https://arxiv.org/abs/2311.13099).
- Fischer et al. (2024)
Michael Fischer, Iliyan Georgiev, Thibault Groueix, Vladimir G Kim, Tobias Ritschel, and Valentin Deschaintre.
Sama: Material-aware 3d selection and segmentation.
*arXiv preprint arXiv:2411.19322*, 2024.
- Fleming (2014)
Roland W. Fleming.
Visual perception of materials and their properties.
*Vision Research*, 94:62–75, 2014.
ISSN 0042-6989.
doi: https://doi.org/10.1016/j.visres.2013.11.004.
URL [https://www.sciencedirect.com/science/article/pii/S0042698913002782](https://www.sciencedirect.com/science/article/pii/S0042698913002782).
- Fleming et al. (2013)
Roland W. Fleming, Christiane Wiebel, and Karl Gegenfurtner.
Perceptual qualities and material classes.
*Journal of Vision*, 13(8):9–9, 07 2013.
ISSN 1534-7362.
doi: 10.1167/13.8.9.
URL [https://doi.org/10.1167/13.8.9](https://doi.org/10.1167/13.8.9).
- frankaemika (2025)
frankaemika.
franka_description: Official models of franka robotics gmbh robots.
[https://github.com/frankaemika/franka_description](https://github.com/frankaemika/franka_description), 2025.
GitHub repository, accessed June 2025.
- (36)
Clement Fuji Tsang, Maria Shugrina, Jean Francois Lafleche, Or Perel, Charles Loop, Towaki Takikawa, Vismay Modi, Alexander Zook, Jiehan Wang, Wenzheng Chen, Tianchang Shen, Jun Gao, Krishna Murthy Jatavallabhula, Edward Smith, Artem Rozantsev, Sanja Fidler, Gavriel State, Jason Gorski, Tommy Xiang, Jianing Li, Michael Li, and Rev Lebaredian.
Kaolin: A pytorch library for accelerating 3d deep learning research.
URL [https://github.com/NVIDIAGameWorks/kaolin](https://github.com/NVIDIAGameWorks/kaolin).
- Gao et al. (2022)
Ruohan Gao, Zilin Si, Yen-Yu Chang, Samuel Clarke, Jeannette Bohg, Li Fei-Fei, Wenzhen Yuan, and Jiajun Wu.
Objectfolder 2.0: A multisensory object dataset for sim2real transfer, 2022.
URL [https://arxiv.org/abs/2204.02389](https://arxiv.org/abs/2204.02389).
- Grieves & Vickers (2017)
Michael Grieves and John Vickers.
*Digital Twin: Mitigating Unpredictable, Undesirable Emergent Behavior in Complex Systems*, pp. 85–113.
Springer International Publishing, Cham, 2017.
ISBN 978-3-319-38756-7.
doi: 10.1007/978-3-319-38756-7_4.
URL [https://doi.org/10.1007/978-3-319-38756-7_4](https://doi.org/10.1007/978-3-319-38756-7_4).
- Guo et al. (2024)
Minghao Guo, Bohan Wang, Pingchuan Ma, Tianyuan Zhang, Crystal Elaine Owens, Chuang Gan, Joshua B. Tenenbaum, Kaiming He, and Wojciech Matusik.
Physically compatible 3d object modeling from a single image.
In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (eds.), *Advances in Neural Information Processing Systems*, volume 37, pp. 119260–119282. Curran Associates, Inc., 2024.
URL [https://proceedings.neurips.cc/paper_files/paper/2024/file/d7af02c8a8e26608199c087f50a21d37-Paper-Conference.pdf](https://proceedings.neurips.cc/paper_files/paper/2024/file/d7af02c8a8e26608199c087f50a21d37-Paper-Conference.pdf).
- Higgins et al. (2017)
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner.
beta-vae: Learning basic visual concepts with a constrained variational framework.
In *International conference on learning representations*, 2017.
- Hsu et al. (2024)
Hao-Yu Hsu, Zhi-Hao Lin, Albert Zhai, Hongchi Xia, and Shenlong Wang.
Autovfx: Physically realistic video editing from natural language instructions, 2024.
URL [https://arxiv.org/abs/2411.02394](https://arxiv.org/abs/2411.02394).
- Hu & contributors (2018)
Yuanming Hu and contributors.
taichi_mpm: High‐performance mls‐mpm solver.
[https://github.com/yuanming-hu/taichi_mpm](https://github.com/yuanming-hu/taichi_mpm), 2018.
Commit retrieved 3 Aug 2025.
- Huang et al. (2024a)
Kemeng Huang, Floyd M. Chitalu, Huancheng Lin, and Taku Komura.
Gipc: Fast and stable gauss-newton optimization of ipc barrier energy.
*ACM Trans. Graph.*, 43(2), mar 2024a.
ISSN 0730-0301.
doi: 10.1145/3643028.
- Huang et al. (2025)
Kemeng Huang, Xinyu Lu, Huancheng Lin, Taku Komura, and Minchen Li.
Stiffgipc: Advancing gpu ipc for stiff affine-deformable simulation.
*ACM Trans. Graph.*, 44(3), May 2025.
ISSN 0730-0301.
doi: 10.1145/3735126.
- Huang et al. (2024b)
Tianyu Huang, Haoze Zhang, Yihan Zeng, Zhilu Zhang, Hui Li, Wangmeng Zuo, and Rynson W. H. Lau.
Dreamphysics: Learning physics-based 3d dynamics with video diffusion priors, 2024b.
URL [https://arxiv.org/abs/2406.01476](https://arxiv.org/abs/2406.01476).
- Kerbl et al. (2023)
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis.
3d gaussian splatting for real-time radiance field rendering.
*ACM Transactions on Graphics*, 42(4), July 2023.
URL [https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/).
- Kingma & Welling (2022)
Diederik P Kingma and Max Welling.
Auto-encoding variational bayes, 2022.
URL [https://arxiv.org/abs/1312.6114](https://arxiv.org/abs/1312.6114).
- Lang et al. (2003)
Jochen Lang, Dinesh K Pai, and Hans-Peter Seidel.
Scanning large-scale articulated deformations.
In *Graphics Interface*, pp. 265–272, 2003.
- Le et al. (2025)
Long Le, Ryan Lucas, Chen Wang, Chuhao Chen, Dinesh Jayaraman, Eric Eaton, and Lingjie Liu.
Pixie: Fast and generalizable supervised learning of 3d physics from pixels, 2025.
URL [https://arxiv.org/abs/2508.17437](https://arxiv.org/abs/2508.17437).
- Li et al. (2025)
Jinxi Li, Ziyang Song, Siyuan Zhou, and Bo Yang.
Freegave: 3d physics learning from dynamic videos by gaussian velocity.
*CVPR*, 2025.
- Li et al. (2020a)
Minchen Li, Zachary Ferguson, Teseo Schneider, Timothy Langlois, Denis Zorin, Daniele Panozzo, Chenfanfu Jiang, and Danny M. Kaufman.
Incremental potential contact: intersection-and inversion-free, large-deformation dynamics.
*ACM Trans. Graph.*, 39(4), August 2020a.
ISSN 0730-0301.
doi: 10.1145/3386569.3392425.
URL [https://doi.org/10.1145/3386569.3392425](https://doi.org/10.1145/3386569.3392425).
- Li et al. (2023)
Xuan Li, Yi-Ling Qiao, Peter Yichen Chen, Krishna Murthy Jatavallabhula, Ming Lin, Chenfanfu Jiang, and Chuang Gan.
Pac-nerf: Physics augmented continuum neural radiance fields for geometry-agnostic system identification, 2023.
URL [https://arxiv.org/abs/2303.05512](https://arxiv.org/abs/2303.05512).
- Li et al. (2022)
Yuchen Li, Ujjwal Upadhyay, Habib Slim, Ahmed Abdelreheem, Arpit Prajapati, Suhail Pothigara, Peter Wonka, and Mohamed Elhoseiny.
3d compat: Composition of materials on parts of 3d things.
In Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, and Tal Hassner (eds.), *Computer Vision – ECCV 2022*, pp. 110–127, Cham, 2022. Springer Nature Switzerland.
ISBN 978-3-031-20074-8.
- Li et al. (2020b)
Yunzhu Li, Toru Lin, Kexin Yi, Daniel Bear, Daniel Yamins, Jiajun Wu, Joshua Tenenbaum, and Antonio Torralba.
Visual grounding of learned physical models.
In Hal Daumé III and Aarti Singh (eds.), *Proceedings of the 37th International Conference on Machine Learning*, volume 119 of *Proceedings of Machine Learning Research*, pp. 5927–5936. PMLR, 13–18 Jul 2020b.
URL [https://proceedings.mlr.press/v119/li20j.html](https://proceedings.mlr.press/v119/li20j.html).
- Lin et al. (2018)
Hubert Lin, Melinos Averkiou, Evangelos Kalogerakis, Balazs Kovacs, Siddhant Ranade, Vladimir Kim, Siddhartha Chaudhuri, and Kavita Bala.
Learning material-aware local descriptors for 3d shapes.
In *2018 International Conference on 3D Vision (3DV)*, pp. 150–159. IEEE, 2018.
- Lin et al. (2024a)
Ji Lin, Hongxu Yin, Wei Ping, Yao Lu, Pavlo Molchanov, Andrew Tao, Huizi Mao, Jan Kautz, Mohammad Shoeybi, and Song Han.
Vila: On pre-training for visual language models, 2024a.
URL [https://arxiv.org/abs/2312.07533](https://arxiv.org/abs/2312.07533).
- Lin et al. (2024b)
Jiajing Lin, Zhenzhong Wang, Yongjie Hou, Yuzhou Tang, and Min Jiang.
Phy124: Fast physics-driven 4d content generation from a single image, 2024b.
URL [https://arxiv.org/abs/2409.07179](https://arxiv.org/abs/2409.07179).
- Lin et al. (2025a)
Jiajing Lin, Zhenzhong Wang, Dejun Xu, Shu Jiang, YunPeng Gong, and Min Jiang.
Phys4dgen: Physics-compliant 4d generation with multi-material composition perception, 2025a.
URL [https://arxiv.org/abs/2411.16800](https://arxiv.org/abs/2411.16800).
- Lin et al. (2025b)
Yuchen Lin, Chenguo Lin, Jianjin Xu, and Yadong MU.
OmniphysGS: 3d constitutive gaussians for general physics-based dynamics generation.
In *The Thirteenth International Conference on Learning Representations*, 2025b.
URL [https://openreview.net/forum?id=9HZtP6I5lv](https://openreview.net/forum?id=9HZtP6I5lv).
- Liu et al. (2024a)
Fangfu Liu, Hanyang Wang, Shunyu Yao, Shengjun Zhang, Jie Zhou, and Yueqi Duan.
Physics3d: Learning physical properties of 3d gaussians via video diffusion.
*arXiv preprint arXiv:2406.04338*, 2024a.
- Liu et al. (2024b)
Shaowei Liu, Zhongzheng Ren, Saurabh Gupta, and Shenlong Wang.
Physgen: Rigid-body physics-grounded image-to-video generation.
In *European Conference on Computer Vision*, pp. 360–378. Springer, 2024b.
- Liu et al. (2021)
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo.
Swin transformer: Hierarchical vision transformer using shifted windows.
In *Proceedings of the IEEE/CVF international conference on computer vision*, pp. 10012–10022, 2021.
- Liu et al. (2025)
Zhuoman Liu, Weicai Ye, Yan Luximon, Pengfei Wan, and Di Zhang.
Unleashing the potential of multi-modal foundation models and video diffusion for 4d dynamic physical scene simulation.
In *Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR)*, pp. 11016–11025, June 2025.
- Lloyd & Pai (2001)
John E Lloyd and Dinesh K Pai.
Robotic mapping of friction and roughness for reality-based modeling.
In *Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No. 01CH37164)*, volume 2, pp. 1884–1890. IEEE, 2001.
- Loveday et al. (2004)
Malcolm S Loveday, Tom Gray, and Johannes Aegerter.
Tensile testing of metallic materials: A review.
*Final report of the TENSTAND project of work package*, 1, 2004.
- Macklin (2022)
Miles Macklin.
Warp: A high-performance python framework for gpu simulation and graphics.
[https://github.com/nvidia/warp](https://github.com/nvidia/warp), March 2022.
NVIDIA GPU Technology Conference (GTC).
- Macklin et al. (2016)
Miles Macklin, Matthias Müller, and Nuttapong Chentanez.
Xpbd: position-based simulation of compliant constrained dynamics.
In *Proceedings of the 9th International Conference on Motion in Games*, MIG ’16, pp. 49–54, New York, NY, USA, 2016. Association for Computing Machinery.
ISBN 9781450345927.
doi: 10.1145/2994258.2994272.
URL [https://doi.org/10.1145/2994258.2994272](https://doi.org/10.1145/2994258.2994272).
- MatWeb, LLC (2025)
MatWeb, LLC.
Matweb: Online materials information resource.
[https://www.matweb.com/](https://www.matweb.com/), 2025.
Accessed: 2025-06-25.
- Mezghanni et al. (2022)
Mariem Mezghanni, Théo Bodrito, Malika Boulkenafed, and Maks Ovsjanikov.
Physical simulation layer for accurate 3d modeling.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, pp. 13514–13523, June 2022.
- Mildenhall et al. (2020)
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng.
Nerf: Representing scenes as neural radiance fields for view synthesis.
In *ECCV*, 2020.
- Mishra (2024)
Akshansh Mishra.
Latticeml: A data-driven application for predicting the effective young modulus of high temperature graph based architected materials, 2024.
URL [https://arxiv.org/abs/2404.09470](https://arxiv.org/abs/2404.09470).
- Modi et al. (2024)
Vismay Modi, Nicholas Sharp, Or Perel, Shinjiro Sueda, and David I. W. Levin.
Simplicits: Mesh-free, geometry-agnostic elastic simulation.
*ACM Trans. Graph.*, 43(4), July 2024.
ISSN 0730-0301.
doi: 10.1145/3658184.
URL [https://doi.org/10.1145/3658184](https://doi.org/10.1145/3658184).
- Mottaghi et al. (2016)
Roozbeh Mottaghi, Hessam Bagherinezhad, Mohammad Rastegari, and Ali Farhadi.
Newtonian scene understanding: Unfolding the dynamics of objects in static images.
In *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, June 2016.
- Ni et al. (2024)
Junfeng Ni, Yixin Chen, Bohan Jing, Nan Jiang, Bin Wang, Bo Dai, Puhao Li, Yixin Zhu, Song-Chun Zhu, and Siyuan Huang.
Phyrecon: Physically plausible neural scene reconstruction, 2024.
URL [https://arxiv.org/abs/2404.16666](https://arxiv.org/abs/2404.16666).
- NVIDIA (2019)
NVIDIA.
Nvidia unveils omniverse - open, interactive 3d design collaboration platform for multi-tool workflows, 2019.
URL [https://blogs.nvidia.com/blog/omniverse-collaboration-platform/](https://blogs.nvidia.com/blog/omniverse-collaboration-platform/).
NVIDIA Blog.
- NVIDIA (2025)
NVIDIA.
Nv-dinov2.
NVIDIA AI Foundation Models (Model card), 2025.
- NVIDIA Corporation (2025a)
NVIDIA Corporation.
Commercial assets pack.
[https://docs.omniverse.nvidia.com/usd/latest/usd_content_samples/downloadable_packs.html](https://docs.omniverse.nvidia.com/usd/latest/usd_content_samples/downloadable_packs.html), 2025a.
URL [https://docs.omniverse.nvidia.com/usd/latest/usd_content_samples/downloadable_packs.html](https://docs.omniverse.nvidia.com/usd/latest/usd_content_samples/downloadable_packs.html).
Accessed: 2025-06-13.
- NVIDIA Corporation (2025b)
NVIDIA Corporation.
Nvidia omniverse replicator.
Developer Documentation, 2025b.
URL [https://docs.omniverse.nvidia.com/extensions/latest/ext_replicator.html](https://docs.omniverse.nvidia.com/extensions/latest/ext_replicator.html).
- NVIDIA Corporation (2025c)
NVIDIA Corporation.
Residential assets pack.
[https://docs.omniverse.nvidia.com/usd/latest/usd_content_samples/downloadable_packs.html](https://docs.omniverse.nvidia.com/usd/latest/usd_content_samples/downloadable_packs.html), 2025c.
URL [https://docs.omniverse.nvidia.com/usd/latest/usd_content_samples/downloadable_packs.html](https://docs.omniverse.nvidia.com/usd/latest/usd_content_samples/downloadable_packs.html).
Accessed: 2025-06-13.
- NVIDIA Corporation (2025d)
NVIDIA Corporation.
Vegetation assets pack.
[https://docs.omniverse.nvidia.com/usd/latest/usd_content_samples/downloadable_packs.html](https://docs.omniverse.nvidia.com/usd/latest/usd_content_samples/downloadable_packs.html), 2025d.
URL [https://docs.omniverse.nvidia.com/usd/latest/usd_content_samples/downloadable_packs.html](https://docs.omniverse.nvidia.com/usd/latest/usd_content_samples/downloadable_packs.html).
Accessed: 2025‑06‑13.
- NVIDIA Developer (2025)
NVIDIA Developer.
Simready assets.
[https://developer.nvidia.com/omniverse/simready-assets](https://developer.nvidia.com/omniverse/simready-assets), 2025.
URL [https://developer.nvidia.com/omniverse/simready-assets](https://developer.nvidia.com/omniverse/simready-assets).
Accessed: 2025-06-13.
- OpenAI & et al. (2024)
OpenAI and Josh Achiam et al.
Gpt-4 technical report, 2024.
URL [https://arxiv.org/abs/2303.08774](https://arxiv.org/abs/2303.08774).
- Oquab et al. (2024)
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mahmoud Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Hervé Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski.
Dinov2: Learning robust visual features without supervision, 2024.
URL [https://arxiv.org/abs/2304.07193](https://arxiv.org/abs/2304.07193).
- Pai (2000)
Dinesh K Pai.
Robotics in reality-based modeling.
In *Robotics Research: the Ninth International Symposium*, pp. 353–358. Springer, 2000.
- Pai et al. (2001)
Dinesh K. Pai, Kees van den Doel, Doug L. James, Jochen Lang, John E. Lloyd, Joshua L. Richmond, and Som H. Yau.
Scanning physical interaction behavior of 3d objects.
In *Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques*, SIGGRAPH ’01, pp. 87–96, New York, NY, USA, 2001. Association for Computing Machinery.
ISBN 158113374X.
doi: 10.1145/383259.383268.
URL [https://doi.org/10.1145/383259.383268](https://doi.org/10.1145/383259.383268).
- Pai et al. (2008)
Dinesh K Pai, Jochen Lang, John Lloyd, and Robert J Woodham.
Acme, a telerobotic active measurement facility.
In *Experimental Robotics VI*, pp. 391–400. Springer, 2008.
- Pinto et al. (2016)
Lerrel Pinto, Dhiraj Gandhi, Yuanfeng Han, Yong-Lae Park, and Abhinav Gupta.
The curious robot: Learning visual representations via physical interactions, 2016.
URL [https://arxiv.org/abs/1604.01360](https://arxiv.org/abs/1604.01360).
- PlayCanvas contributors (2025)
PlayCanvas contributors.
SuperSplat: 3d gaussian splat editor.
[https://github.com/playcanvas/supersplat](https://github.com/playcanvas/supersplat), 2025.
GitHub repository.
- Radford et al. (2021)
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever.
Learning transferable visual models from natural language supervision, 2021.
URL [https://arxiv.org/abs/2103.00020](https://arxiv.org/abs/2103.00020).
- Rezende & Mohamed (2015)
Danilo Rezende and Shakir Mohamed.
Variational inference with normalizing flows.
In Francis Bach and David Blei (eds.), *Proceedings of the 32nd International Conference on Machine Learning*, volume 37 of *Proceedings of Machine Learning Research*, pp. 1530–1538, Lille, France, 07–09 Jul 2015. PMLR.
URL [https://proceedings.mlr.press/v37/rezende15.html](https://proceedings.mlr.press/v37/rezende15.html).
- Rudin et al. (2021)
Nikita Rudin, David Hoeller, Philipp Reist, and Marco Hutter.
Learning to walk in minutes using massively parallel deep reinforcement learning.
In *5th Annual Conference on Robot Learning*, 2021.
URL [https://openreview.net/forum?id=wK2fDDJ5VcF](https://openreview.net/forum?id=wK2fDDJ5VcF).
- Schäffer et al. (2024)
Katalin Schäffer, Yasemin Ozkan-Aydin, and Margaret M. Coad.
Soft wrist exosuit actuated by fabric pneumatic artificial muscles.
*IEEE Transactions on Medical Robotics and Bionics*, 6(2):718–732, May 2024.
ISSN 2576-3202.
doi: 10.1109/tmrb.2024.3385795.
URL [http://dx.doi.org/10.1109/TMRB.2024.3385795](http://dx.doi.org/10.1109/TMRB.2024.3385795).
- Sharan et al. (2009)
Lavanya Sharan, Ruth Rosenholtz, and Edward Adelson.
Material perception: What can you see in a brief glance?
*Journal of Vision*, 9(8):784–784, 2009.
- Sharp et al. (2019)
Nicholas Sharp et al.
Polyscope, 2019.
www.polyscope.run.
- Shi et al. (2023)
Haochen Shi, Huazhe Xu, Samuel Clarke, Yunzhu Li, and Jiajun Wu.
Robocook: Long-horizon elasto-plastic object manipulation with diverse tools.
*arXiv preprint arXiv:2306.14447*, 2023.
- Shuai et al. (2025)
Yinghao Shuai, Ran Yu, Yuantao Chen, Zijian Jiang, Xiaowei Song, Nan Wang, Jv Zheng, Jianzhu Ma, Meng Yang, Zhicheng Wang, Wenbo Ding, and Hao Zhao.
Pugs: Zero-shot physical understanding with gaussian splatting, 2025.
URL [https://arxiv.org/abs/2502.12231](https://arxiv.org/abs/2502.12231).
- Slim et al. (2023)
Habib Slim, Xiang Li, Yuchen Li, Mahmoud Ahmed, Mohamed Ayman, Ujjwal Upadhyay, Ahmed Abdelreheem, Arpit Prajapati, Suhail Pothigara, Peter Wonka, et al.
3dcompat++: An improved large-scale 3d vision dataset for compositional recognition.
*arXiv preprint arXiv:2310.18511*, 2023.
- Standley et al. (2017)
Trevor Standley, Ozan Sener, Dawn Chen, and Silvio Savarese.
image2mass: Estimating the mass of an object from its image.
In Sergey Levine, Vincent Vanhoucke, and Ken Goldberg (eds.), *Proceedings of the 1st Annual Conference on Robot Learning*, volume 78 of *Proceedings of Machine Learning Research*, pp. 324–333. PMLR, 13–15 Nov 2017.
URL [https://proceedings.mlr.press/v78/standley17a.html](https://proceedings.mlr.press/v78/standley17a.html).
- Sulsky et al. (1994)
D. Sulsky, Z. Chen, and H.L. Schreyer.
A particle method for history-dependent materials.
*Computer Methods in Applied Mechanics and Engineering*, 118(1):179–196, 1994.
ISSN 0045-7825.
doi: https://doi.org/10.1016/0045-7825(94)90112-0.
URL [https://www.sciencedirect.com/science/article/pii/0045782594901120](https://www.sciencedirect.com/science/article/pii/0045782594901120).
- Tancik et al. (2023)
Matthew Tancik, Ethan Weber, Evonne Ng, Ruilong Li, Brent Yi, Justin Kerr, Terrance Wang, Alexander Kristoffersen, Jake Austin, Kamyar Salahi, Abhik Ahuja, David McAllister, and Angjoo Kanazawa.
Nerfstudio: A modular framework for neural radiance field development.
In *ACM SIGGRAPH 2023 Conference Proceedings*, SIGGRAPH ’23, 2023.
- The Engineering Toolbox (2024)
The Engineering Toolbox.
Engineering materials – properties.
[https://www.engineeringtoolbox.com/engineering-materials-properties-d_1225.html](https://www.engineeringtoolbox.com/engineering-materials-properties-d_1225.html), 2024.
Accessed: 2025-06-25.
- Wang et al. (2023)
Yuang Wang, Xingyi He, Sida Peng, Haotong Lin, Hujun Bao, and Xiaowei Zhou.
Autorecon: Automated 3d object discovery and reconstruction.
In *CVPR*, 2023.
- Wikipedia contributors (2024a)
Wikipedia contributors.
Density.
[https://en.wikipedia.org/wiki/Density](https://en.wikipedia.org/wiki/Density), 2024a.
Accessed: 2025-06-25.
- Wikipedia contributors (2024b)
Wikipedia contributors.
Poisson’s ratio.
[https://en.wikipedia.org/wiki/Poisson%27s_ratio](https://en.wikipedia.org/wiki/Poisson%27s_ratio), 2024b.
Accessed: 2025-06-25.
- Wikipedia contributors (2024c)
Wikipedia contributors.
Young’s modulus.
[https://en.wikipedia.org/wiki/Young%27s_modulus](https://en.wikipedia.org/wiki/Young%27s_modulus), 2024c.
Accessed: 2025-06-25.
- Wu et al. (2015)
Jiajun Wu, Ilker Yildirim, Joseph J Lim, Bill Freeman, and Josh Tenenbaum.
Galileo: Perceiving physical object properties by integrating a physics engine with deep learning.
In C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett (eds.), *Advances in Neural Information Processing Systems*, volume 28. Curran Associates, Inc., 2015.
URL [https://proceedings.neurips.cc/paper_files/paper/2015/file/d09bf41544a3365a46c9077ebb5e35c3-Paper.pdf](https://proceedings.neurips.cc/paper_files/paper/2015/file/d09bf41544a3365a46c9077ebb5e35c3-Paper.pdf).
- Wu et al. (2016)
Jiajun Wu, Joseph J Lim, Hongyi Zhang, Joshua B Tenenbaum, and William T Freeman.
Physics 101: Learning physical object properties from unlabeled videos.
In *BMVC*, volume 2, pp.  7, 2016.
- Wu et al. (2017)
Jiajun Wu, Erika Lu, Pushmeet Kohli, Bill Freeman, and Josh Tenenbaum.
Learning to see physics via visual de-animation.
In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), *Advances in Neural Information Processing Systems*, volume 30. Curran Associates, Inc., 2017.
URL [https://proceedings.neurips.cc/paper_files/paper/2017/file/4c56ff4ce4aaf9573aa5dff913df997a-Paper.pdf](https://proceedings.neurips.cc/paper_files/paper/2017/file/4c56ff4ce4aaf9573aa5dff913df997a-Paper.pdf).
- Xia et al. (2024)
Hongchi Xia, Zhi-Hao Lin, Wei-Chiu Ma, and Shenlong Wang.
Video2game: Real-time interactive realistic and browser-compatible environment from a single video.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, pp. 4578–4588, June 2024.
- Xia et al. (2025)
Hongchi Xia, Entong Su, Marius Memmel, Arhan Jain, Raymond Yu, Numfor Mbiziwo-Tiapo, Ali Farhadi, Abhishek Gupta, Shenlong Wang, and Wei-Chiu Ma.
Drawer: Digital reconstruction and articulation with environment realism.
In *Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR)*, pp. 21771–21782, June 2025.
- Xiang et al. (2025)
Jianfeng Xiang, Zelong Lv, Sicheng Xu, Yu Deng, Ruicheng Wang, Bowen Zhang, Dong Chen, Xin Tong, and Jiaolong Yang.
Structured 3d latents for scalable and versatile 3d generation, 2025.
URL [https://arxiv.org/abs/2412.01506](https://arxiv.org/abs/2412.01506).
- Xie et al. (2025)
Han Xie, Ru Jia, Yonglin Xia, Lei Li, Yue Hu, Jiaxuan Xu, Yufei Sheng, Yuanyuan Wang, and Hua Bao.
An ab initio dataset of size-dependent effective thermal conductivity for advanced technology transistors.
*arXiv preprint arXiv:2501.15736*, 2025.
- Xie et al. (2024)
Tianyi Xie, Zeshun Zong, Yuxing Qiu, Xuan Li, Yutao Feng, Yin Yang, and Chenfanfu Jiang.
Physgaussian: Physics-integrated 3d gaussians for generative dynamics.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pp. 4389–4398, 2024.
- Xu et al. (2019)
Zhenjia Xu, Jiajun Wu, Andy Zeng, Joshua B. Tenenbaum, and Shuran Song.
Densephysnet: Learning dense physical object representations via multi-step dynamic interactions, 2019.
URL [https://arxiv.org/abs/1906.03853](https://arxiv.org/abs/1906.03853).
- Xue et al. (2023)
Haotian Xue, Antonio Torralba, Joshua B. Tenenbaum, Daniel LK Yamins, Yunzhu Li, and Hsiao-Yu Tung.
3d-intphys: Towards more generalized 3d-grounded visual intuitive physics under challenging scenes, 2023.
URL [https://arxiv.org/abs/2304.11470](https://arxiv.org/abs/2304.11470).
- Yang et al. (2024)
Yandan Yang, Baoxiong Jia, Peiyuan Zhi, and Siyuan Huang.
Physcene: Physically interactable 3d scene synthesis for embodied ai.
In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*, pp. 16262–16272, June 2024.
- Yang et al. (2025)
Yu-Qi Yang, Yu-Xiao Guo, Jian-Yu Xiong, Yang Liu, Hao Pan, Peng-Shuai Wang, Xin Tong, and Baining Guo.
Swin3d: A pretrained transformer backbone for 3d indoor scene understanding.
*Computational Visual Media*, 11(1):83–101, 2025.
- Yao & Hauser (2023)
Shaoxiong Yao and Kris Hauser.
Estimating tactile models of heterogeneous deformable objects in real time.
In *2023 IEEE International Conference on Robotics and Automation (ICRA)*, pp. 12583–12589, 2023.
doi: 10.1109/ICRA48891.2023.10160731.
- Ye et al. (2024)
Vickie Ye, Ruilong Li, Justin Kerr, Matias Turkulainen, Brent Yi, Zhuoyang Pan, Otto Seiskari, Jianbo Ye, Jeffrey Hu, Matthew Tancik, and Angjoo Kanazawa.
gsplat: An open-source library for gaussian splatting, 2024.
URL [https://arxiv.org/abs/2409.06765](https://arxiv.org/abs/2409.06765).
- Yildirim et al. (2016)
Ilker Yildirim, Jiajun Wu, Yilun Du, and Joshua B. Tenenbaum.
Interpreting dynamic scenes by a physics engine and bottom-up visual cues.
*arXiv preprint*, 1605., 2016.
arXiv:1605.02470.
- Yu et al. (2024)
Samson Yu, Kelvin Lin, Anxing Xiao, Jiafei Duan, and Harold Soh.
Octopi: Object property reasoning with large tactile-language models, 2024.
URL [https://arxiv.org/abs/2405.02794](https://arxiv.org/abs/2405.02794).
- Yuksekgonul et al. (2025)
Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Pan Lu, Zhi Huang, Carlos Guestrin, and James Zou.
Optimizing generative ai by backpropagating language model feedback.
*Nature*, 639:609–616, 2025.
- Zhai et al. (2024)
Albert J. Zhai, Yuan Shen, Emily Y. Chen, Gloria X. Wang, Xinlei Wang, Sheng Wang, Kaiyu Guan, and Shenlong Wang.
Physical property understanding from language-embedded feature fields, 2024.
URL [https://arxiv.org/abs/2404.04242](https://arxiv.org/abs/2404.04242).
- Zhang et al. (2024)
Kaifeng Zhang, Baoyu Li, Kris Hauser, and Yunzhu Li.
Adaptigraph: Material-adaptive graph-based neural dynamics for robotic manipulation.
*arXiv preprint arXiv:2407.07889*, 2024.
- Zhang et al. (2025)
Tianyuan Zhang, Hong-Xing Yu, Rundi Wu, Brandon Y. Feng, Changxi Zheng, Noah Snavely, Jiajun Wu, and William T. Freeman.
Physdreamer: Physics-based interaction with 3d objects via video generation.
In Aleš Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Torsten Sattler, and Gül Varol (eds.), *Computer Vision – ECCV 2024*, pp. 388–406, Cham, 2025. Springer Nature Switzerland.
ISBN 978-3-031-72627-9.
- Zhao et al. (2024a)
Haoyu Zhao, Hao Wang, Xingyue Zhao, Hao Fei, Hongqiu Wang, Chengjiang Long, and Hua Zou.
Efficient physics simulation for 3d scenes via mllm-guided gaussian splatting.
*arXiv preprint arXiv:2411.12789*, 2024a.
- Zhao et al. (2024b)
Haoyu Zhao, Hao Wang, Xingyue Zhao, Hongqiu Wang, Zhiyu Wu, Chengjiang Long, and Hua Zou.
Automated 3d physical simulation of open-world scene with gaussian splatting.
*arXiv e-prints*, pp. arXiv–2411, 2024b.

## Supplementary Contents

## Appendix A Additional Results

### A.1 End-to-end Examples with Simulation

We demonstrate the process that enables creating simulation-ready, realistic assets from Gaussian Splats and meshes in [Fig.˜8](https://arxiv.org/html/2510.22975v1#A1.F8), demonstrating convincing simulations without any fine-tuning. For example, in [Fig.˜8](https://arxiv.org/html/2510.22975v1#A1.F8), we first capture a video of an object, and then train a 3D Gaussian Splatting model. Then, we pass it to VoMP which estimates the mechanical properties in a couple of seconds. We then use these properties in a simulator to produce a realistic (see : 0:36), greatly reducing the barrier toward constructing realistic interactive digital worlds directly
from our physical reality.

### A.2 More Mechanical Property Prediction Results

We show qualitative results for inferring mechanical property fields in [Fig.˜9](https://arxiv.org/html/2510.22975v1#A1.F9) and [10](https://arxiv.org/html/2510.22975v1#A1.F10).
We notice from [Fig.˜9](https://arxiv.org/html/2510.22975v1#A1.F9) row 1, column 2 that our model can pick up small details like the stem of the orange at the top of the object, which is given a different Young’s modulus, though it only spans a few voxels (see : 1:38). We notice from [Fig.˜9](https://arxiv.org/html/2510.22975v1#A1.F9) row 2, column 2 that our model finds that the space isnide the pot should be made up of properties that fall in the range of dirt, even though the inside of the pot was not observed through external renders (see : 1:50). We notice from [Fig.˜9](https://arxiv.org/html/2510.22975v1#A1.F9) row 3, column 2 that our model can tolerate some noise in assets such as the Gaussian splat of a bowl with fruits segmented from a larger Gaussian splat (see : 2:01). We notice from [Fig.˜10](https://arxiv.org/html/2510.22975v1#A1.F10) row 1, column 1 that our model can accurately predict thing segments and thin boundaries, like for the seat of the chair (see : 2:10). We notice from [Fig.˜9](https://arxiv.org/html/2510.22975v1#A1.F9) row 3, column 2 that our models can handle complex assets and complex materials like trees and accurately handle fine details, such as understanding where all the leaves lie and giving them different material properties than wood (see : 2:16). We notice from [Fig.˜9](https://arxiv.org/html/2510.22975v1#A1.F9) row 4, column 1 that our models can handle complex volumetric materials, such as annotating the properties of wood under the flowers (see : 2:28). We notice from [Fig.˜9](https://arxiv.org/html/2510.22975v1#A1.F9) row 4, column 2 that our models can handle thin materials and identify the vein of the leaves (see : 2:35). For completeness, we also include our metrics normalized by total test set voxels in  [Tb.˜2](https://arxiv.org/html/2510.22975v1#A1.T2) and observe the
same performance boost compared to prior art as with the per-object normalization as presented in the main paper [Fig.˜6](https://arxiv.org/html/2510.22975v1#S6.F6).

### A.3 Mass Estimation Example Results

We show qualitative results from our model on mass estimation on the ABO-500 Collins_2022_CVPR dataset in [Fig.˜11](https://arxiv.org/html/2510.22975v1#A1.F11).

Figure: (a) Inference across representations: We show that VoMP can operate on any geometry that can be voxelized and rendered, including meshes, SDF, NeRFs, and Gaussian Splats (: 1:30).
Refer to caption: x19.png

Figure: Figure 9: Inferred Mechanical Property Fields. We show additional mechnical property fields and slice planes through mechanical property fields estimated by VoMP (: 1:40).
Refer to caption: assets/fields/1.png

Figure: Figure 10: Inferred Mechanical Property Fields. We show additional mechnical property fields and slice planes through mechanical property fields estimated by VoMP (: 2:10).
Refer to caption: assets/fields/11.png

**Table 2: Voxel Mechanical Property Estimation. Errors for predicting mechanical properties from 3D objects averaged across all voxels in the test set.**
| Method | Young’s Modulus Pa ($E$) | Poisson’s Ratio ($\nu$) | Density $\frac{kg}{m^{3}}$ ($\rho$) |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
|  | ALDE ($\downarrow$) | ALRE ($\downarrow$) | ADE ($\downarrow$) | ARE ($\downarrow$) | ADE ($\downarrow$) | ARE ($\downarrow$) |
| NeRF2Physics zhai2024physicalpropertyunderstandinglanguageembedded | 2.5719 ($\pm$1.15) | 0.4122 ($\pm$0.08) | - | - | 1354.9458 ($\pm$1315.71) | 1.1496 ($\pm$0.67) |
| PUGS shuai2025pugszeroshotphysicalunderstanding | 3.8619 ($\pm$2.01) | 0.4512 ($\pm$0.11) | - | - | 3641.0715 ($\pm$3320.78) | 4.0413 ($\pm$4.16) |
| Phys4DGen^⋆ lin2025phys4dgenphysicscompliant4dgeneration | 5.2977 ($\pm$3.36) | 0.4825 ($\pm$0.14) | 0.0394 ($\pm$0.05) | 0.1425 ($\pm$0.21) | 1285.9489 ($\pm$1981.11) | 1.0445 ($\pm$2.53) |
| Ours | 0.3765 ($\pm$0.39) | 0.0421 ($\pm$0.05) | 0.0250 ($\pm$0.01) | 0.0837 ($\pm$0.03) | 113.3807 ($\pm$301.90) | 0.0908 ($\pm$0.14) |

Figure: Figure 11: Mass Estimation. We show qualitative results of estimating mass from the ABO-500 Collins_2022_CVPR dataset.
Refer to caption: assets/abo/1.png

### A.4 Additional MatVAE Results

#### A.4.1 Distribution learned by MatVAE

We measure standard metrics used for measuring the difference in the distribution learned by MatVAE and the distribution of the test set in Tb. [3](https://arxiv.org/html/2510.22975v1#A1.T3), observing small errors which suggest that MatVAE learned a good approximation of the true material distribution.

**Table 3: Distribution learned by MatVAE compared to the distribution of MTD test set.**
| Young’s Modulus ($E$) | Poisson’s Ratio ($\nu$) | Density ($\rho$) |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| W1 ($\downarrow$) | W2 ($\downarrow$) | $D_{\text{KL}}$ ($\downarrow$) | W1 ($\downarrow$) | W2 ($\downarrow$) | $D_{\text{KL}}$ ($\downarrow$) | W1 ($\downarrow$) | W2 ($\downarrow$) | $D_{\text{KL}}$ ($\downarrow$) |
| 0.0405 | 0.0798 | 0.1379 | 0.0317 | 0.0437 | 0.0342 | 0.0132 | 0.0172 | 0.0260 |

#### A.4.2 Moving Across MatVAE Latent Space

We show an example of moving across the latent space in [Fig.˜13](https://arxiv.org/html/2510.22975v1#A1.F13). For each setting, we take a point in our latent space and move across both of its dimensions to obtain multiple smoothly varying material properties. We apply these properties to a bunny and simulate dropping it to the ground with a FEM simulator and these various material properties. The color plots show the average displacements of the mesh from its rest state across simulation steps (e.g. [Fig.˜13](https://arxiv.org/html/2510.22975v1#A1.F13)), demonstrating that even the actual physical behavior correlates with the dimensions of the latent space. Thus, we find that MatVAE learns a rich, meaningful latent space with smooth interpolation and ensures generating physically valid material triplets.

#### A.4.3 Interpolation with MatVAE

We show additional examples of interpolating in the MatVAE latent space ([§˜6.4](https://arxiv.org/html/2510.22975v1#S6.SS4)) in [Fig.˜12](https://arxiv.org/html/2510.22975v1#A1.F12).

Figure: Figure 12: Interpolating in MatVAE latent space: an additional example of interpolation, complementary to [Fig.˜7](https://arxiv.org/html/2510.22975v1#S6.F7).

Figure: (a) Corner Materials for our experiments on moving across the latent space ([Fig.˜13](https://arxiv.org/html/2510.22975v1#A1.F13), [13](https://arxiv.org/html/2510.22975v1#A1.F13) and [13](https://arxiv.org/html/2510.22975v1#A1.F13)).

## Appendix B Comparison with Concurrent work

Although Pixie le2025pixiefastgeneralizablesupervised is concurrent with us, we still compare aspects of our approach with Pixie. We discuss differences with Pixie in [§˜2.2](https://arxiv.org/html/2510.22975v1#S2.SS2).

##### Data Annotation Process.

We compare our data annotation process with the annotation process of Pixie le2025pixiefastgeneralizablesupervised in  [Fig.˜14](https://arxiv.org/html/2510.22975v1#A2.F14). Our method performs annotation from meshes while Pixie le2025pixiefastgeneralizablesupervised gets points from training a NeRF, which often produces noisy points, and the segmentation is performed based on CLIP features, which often produces noisy segmentation for difficult objects.

Figure: Figure 14: Data Annotation. We compare data annotation process of VoMP and Pixie.
Refer to caption: x42.png

##### Validity of Materials for Data Annotation.

Although we do not have access to the Pixie le2025pixiefastgeneralizablesupervised dataset, Pixie uses in-context physics examples, which include material names and ranges of mechanical property triplets in the annotation process. We analyze these in-context physics examples and compare them with real material ranges from MTD ([§˜5.1](https://arxiv.org/html/2510.22975v1#S5.SS1)) in [Tb.˜4](https://arxiv.org/html/2510.22975v1#A2.T4). We find that some of these in-context properties might create pleasing simulations with a particular simulator but can fall outside the range of real materials.

Table: Table 4: Comparison of Mapped Material Properties between Pixie’s in-context physics examples le2025pixiefastgeneralizablesupervised and the datasets of known material properties.

##### Comparison of Outputs.

Although Pixie le2025pixiefastgeneralizablesupervised is concurrent with us, we still compare the outputs of Pixie with VoMP on one object in [Fig.˜15](https://arxiv.org/html/2510.22975v1#A2.F15).

Figure: Figure 15: Comparing Estimations. We compare property estimates between VoMP and Pixie for a mesh.
Refer to caption: x43.png

## Appendix C Ablations

We provide an in-depth analysis motivating our MatVAE and Geometry Transformer training scheme by ablating each component in [Tb.˜5](https://arxiv.org/html/2510.22975v1#A3.T5) and [6](https://arxiv.org/html/2510.22975v1#A3.T6). Our ablations require changing the hyperparameters for fair comparisons; thus, for each ablation, we tune our hyperparameters within an identical compute budget.

##### MatVAE vs Vanilla VAE.

Technically, MatVAE ([§˜3](https://arxiv.org/html/2510.22975v1#S3)) is built on top of the vanilla VAE kingma2022autoencodingvariationalbayes and we can use a vanilla VAE kingma2022autoencodingvariationalbayes in its place. In [Tb.˜5](https://arxiv.org/html/2510.22975v1#A3.T5), we show material property reconstruction and distributional metrics between a vanilla VAE and our MatVAE. MatVAE outperforms the vanilla VAE in almost all metrics. We find that Vanilla VAE collapses to the Young’s Modulus property, giving us a low reconstruction error for Young’s Modulus but significantly higher errors for other properties.

##### Image Features.

For aggregating image features ([§˜4.1](https://arxiv.org/html/2510.22975v1#S4.SS1)), we experiment with using DINOv2 oquab2024dinov2learningrobustvisual, CLIP radford2021learningtransferablevisualmodels, and RGB colors by average pooling in the voxel. The results are shown in [Tb.˜6](https://arxiv.org/html/2510.22975v1#A3.T6). Our model had many layers in the Geometry Transformer ([§˜4.2](https://arxiv.org/html/2510.22975v1#S4.SS2) and [F.2](https://arxiv.org/html/2510.22975v1#A6.SS2)) initialized from a generation model xiang2025structured3dlatentsscalable. This set of ablations was trained starting from random weights, due to the absence of generation weights for these settings. We find that using DINOv2 oquab2024dinov2learningrobustvisual and CLIP radford2021learningtransferablevisualmodels without initializing the weights from TRELLIS xiang2025structured3dlatentsscalable performs slightly worse, whereas simply using RGB colors performs significantly worse.

##### MatVAE.

While MatVAE acts as a continuous tokenizer, it is possible to have the Geometry Transformer directly predict a $\mathbb{R}^{3}$ vector i.e. directly predict the material triplets. We find this produces significantly worse results for Young’s Modulus estimation and Poisson’s Ratio estimation.

##### Normalization Scheme.

We experiment with different normalization schemes ([§˜3](https://arxiv.org/html/2510.22975v1#S3)) like $Z$-score, and not using $\log$-space trasnform for either Young’s Modulus or Density. All of these normalization schemes lead to a significant degradation in performance. Most notably, removing the logarithmic scaling for Young’s Modulus (w/o $\log(E)$) or using a simple $Z$-score severely harms prediction accuracy.

##### Loss.

Our Geometry Transformer ([§˜4.2](https://arxiv.org/html/2510.22975v1#S4.SS2)) is trained with $\ell_{2}$ loss for reconstruction ([Equation˜4](https://arxiv.org/html/2510.22975v1#S4.E4)). We test the effect of replacing this with an $\ell_{1}$ loss. This change results in a substantial drop in performance across all metrics, with errors increasing by a factor of 2-3x for most properties. This indicates that the squared error penalty of the $\ell_{2}$ loss is more effective for this material property regression task.

**Table 5: MatVAE Ablation. We ablate MatVAE by comparing it against a vanilla VAE and present reconstruction and distirbutional metrics.**
| Model | Young’s Modulus ($E$) | Poisson’s Ratio ($\nu$) | Density ($\rho$) |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | W1 ($\downarrow$) | W2 ($\downarrow$) | $D_{\text{KL}}$ ($\downarrow$) | W1 ($\downarrow$) | W2 ($\downarrow$) | $D_{\text{KL}}$ ($\downarrow$) | W1 ($\downarrow$) | W2 ($\downarrow$) | $D_{\text{KL}}$ ($\downarrow$) |
| Vanilla VAE kingma2022autoencodingvariationalbayes | 0.0653 | 0.0868 | 0.0547 | 0.0849 | 0.1057 | 0.0689 | 0.0547 | 0.0744 | 0.0175 |
| MatVAE | 0.0405 | 0.0798 | 0.1379 | 0.0317 | 0.0437 | 0.0342 | 0.0132 | 0.0172 | 0.0260 |
|  | (-0.025) | (-0.007) | (+0.083) | (-0.053) | (-0.062) | (-0.035) | (-0.042) | (-0.057) | (+0.009) |
| w/o NF | 0.0339 | 0.0447 | 0.0441 | 0.0417 | 0.0504 | 0.0848 | 0.0599 | 0.0819 | 0.0529 |
| w/o TC penalty | 0.0633 | 0.0855 | 0.0500 | 0.0844 | 0.1052 | 0.0672 | 0.0525 | 0.0715 | 0.0109 |
| w/o free nats | 0.0749 | 0.1168 | 0.1311 | 0.2014 | 0.2064 | 0.6376 | 0.0421 | 0.0507 | 0.0223 |
|  | $\log(E)$ ($\downarrow$) | $\nu$ ($\downarrow$) | $\rho$ ($\downarrow$) | $\log(E/\rho)$ ($\downarrow$) | $\log(G)$ ($\downarrow$) | $\log(K)$ ($\downarrow$) | L.S. ($\downarrow$) | E.A. ($\downarrow$) | Bray–Curtis ($\downarrow$) |
| Vanilla VAE kingma2022autoencodingvariationalbayes | 0.0512 | 15366.8750 | 0.8306 | 0.0542 | 0.0544 | 0.0447 | 0.2384 | 2.1893 | 0.4690 |
| MatVAE | 0.0034 | 0.0426 | 0.0330 | 0.0054 | 0.0036 | 0.0036 | 0.0131 | 0.4439 | 0.0411 |
|  | (-0.048) | (-15366.8) | (-0.798) | (-0.049) | (-0.051) | (-0.041) | (-0.225) | (-1.745) | (-0.428) |
| w/o NF | 0.0020 | nan | 0.0160 | 0.0033 | 0.0021 | 0.0021 | 0.0086 | 0.1729 | 0.0234 |
| w/o TC penalty | 0.0499 | 15567.0986 | 0.8298 | 0.0537 | 0.0530 | 0.0437 | 0.2382 | 2.1514 | 0.4562 |
| w/o free nats | 0.0036 | 16332.7829 | 0.0276 | 0.0053 | 0.0037 | 0.0041 | 0.0116 | 0.2966 | 0.0436 |

**Table 6: Ablations. We ablate VoMP with choice of image features, using MatVAE, normalization scheme, and choice of a loss function. We report the voxel-level mechanical property difference errors.**
| Method | Young’s Modulus Pa ($E$) | Poisson’s Ratio ($\nu$) | Density $\frac{kg}{m^{3}}$ ($\rho$) |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
|  | ALDE ($\downarrow$) | ALRE ($\downarrow$) | ADE ($\downarrow$) | ARE ($\downarrow$) | ADE ($\downarrow$) | ARE ($\downarrow$) |
| Image Features (Geometry Transformer start from random weights, [§˜4.1](https://arxiv.org/html/2510.22975v1#S4.SS1)) |  |  |  |  |  |  |
| w/ DINOv2 oquab2024dinov2learningrobustvisual | 0.2888 ($\pm$0.41) | 0.0536 ($\pm$0.06) | 0.0259 ($\pm$0.02) | 0.0803 ($\pm$0.08) | 373.5183 ($\pm$675.90) | 0.3126 ($\pm$0.79) |
| w/ CLIP radford2021learningtransferablevisualmodels | 0.2695 ($\pm$0.42) | 0.0508 ($\pm$0.06) | 0.0250 ($\pm$0.02) | 0.0771 ($\pm$0.07) | 383.5844 ($\pm$766.41) | 0.3110 ($\pm$0.85) |
| w/ RGB colors | 1.2176 ($\pm$0.88) | 0.6593 ($\pm$0.49) | 0.1379 ($\pm$0.06) | 1.1642 ($\pm$0.78) | 3678.4451 ($\pm$8421.17) | 4.7430 ($\pm$2.85) |
| MatVAE ([§˜3](https://arxiv.org/html/2510.22975v1#S3)) |  |  |  |  |  |  |
| w/o MatVAE | 1.1284 ($\pm$0.52) | 0.1289 ($\pm$0.08) | 0.0480 ($\pm$0.02) | 0.1638 ($\pm$0.08) | 917.5879 ($\pm$428.50) | 0.8637 ($\pm$0.61) |
| Normalization Scheme ([§˜3](https://arxiv.org/html/2510.22975v1#S3)) |  |  |  |  |  |  |
| w/ $Z$-score | 0.8838 ($\pm$0.61) | 0.0996 ($\pm$0.08) | 0.0814 ($\pm$0.04) | 0.2938 ($\pm$0.20) | 5269.2900 ($\pm$946.03) | 6.4656 ($\pm$4.00) |
| w/o $\log(\rho)$ | 0.6654 ($\pm$0.54) | 0.0748 ($\pm$0.07) | 0.0542 ($\pm$0.03) | 0.1806 ($\pm$0.11) | 549.9512 ($\pm$513.57) | 0.5976 ($\pm$0.39) |
| w/o $\log(E)$ | 0.9033 ($\pm$0.60) | 0.1024 ($\pm$0.09) | 0.1182 ($\pm$0.05) | 0.4189 ($\pm$0.25) | 4051.9121 ($\pm$838.98) | 5.0428 ($\pm$3.22) |
| Loss |  |  |  |  |  |  |
| w/ $\ell_{1}$ | 0.8947 ($\pm$0.62) | 0.1038 ($\pm$0.09) | 0.0468 ($\pm$0.04) | 0.1666 ($\pm$0.16) | 568.7543 ($\pm$734.10) | 0.6337 ($\pm$0.88) |
| Ours | 0.3765 ($\pm$0.39) | 0.0421 ($\pm$0.05) | 0.0250 ($\pm$0.01) | 0.0837 ($\pm$0.03) | 113.3807 ($\pm$301.90) | 0.0908 ($\pm$0.14) |

## Appendix D Metrics

We present an explanation of the metrics we use, and experiments on interpreting these metrics.

### D.1 Metrics for Mass and Field Estimation

To evaluate the accuracy of predicted scalar quantities such as object mass, as well as continuous scalar fields like density or stiffness, we use several commonly adopted metrics. Let $y$ denote a ground-truth scalar value or voxel-wise field (e.g., density), and $\hat{y}$ its predicted counterpart.

##### Absolute Difference Error (ADE).

The average absolute error between predicted and ground-truth values:

$$ $\mathrm{ADE}=\frac{1}{N}\sum_{i=1}^{N}|y_{i}-\hat{y}_{i}|.$ (5) $$

This metric is scale-sensitive and reports the error in physical units (e.g., $\mathrm{kg/m^{3}}$ for density, $\mathrm{kg}$ for mass).

##### Absolute Log Difference Error (ALDE).

The average absolute error in logarithmic space:

$$ $\mathrm{ALDE}=\frac{1}{N}\sum_{i=1}^{N}|\log y_{i}-\log\hat{y}_{i}|.$ (6) $$

This metric captures multiplicative error and is particularly useful for quantities that vary over several orders of magnitude.

##### Average Relative Error (ARE).

The mean relative deviation between predictions and ground truth:

$$ $\mathrm{ARE}=\frac{1}{N}\sum_{i=1}^{N}\left|\frac{y_{i}-\hat{y}_{i}}{y_{i}}\right|.$ (7) $$

This dimensionless metric penalizes over- and under-estimates proportionally, making it appropriate for comparing across varying scales.

##### Minimum Ratio Error (MnRE).

A symmetric and bounded measure of relative accuracy:

$$ $\mathrm{MnRE}=\frac{1}{N}\sum_{i=1}^{N}\min\left(\frac{y_{i}}{\hat{y}_{i}},\frac{\hat{y}_{i}}{y_{i}}\right).$ (8) $$

This metric ranges from 0 to $1$ and is maximized when predictions are perfectly accurate. As suggested in prior work mass_estimation_visual, MnRE avoids bias toward systematic over- or under-estimation and reduces sensitivity to outliers, making it particularly effective for evaluating physical quantity predictions across heterogeneous samples.

### D.2 Metrics to Measure Differences in Mechanical Properties

We use multiple commonly used metrics for measuring differences between mechanical properties.

##### Relative Error in log ⁡ ( E ) \log(E) .

Relative error between predicted and true values of the logarithm of Young’s modulus $E$ reported in units of Pa. This captures relative error in material stiffness across several orders of magnitude.

##### Relative Error in ν \nu .

Relative Error in linear space for Poisson’s ratio $\nu$, a dimensionless measure of lateral contraction under uniaxial loading.

##### Relative Error in ρ \rho .

Relative Error between predicted and true values of material density $\rho$, reported in units of $\mathrm{kg/m^{3}}$.

##### Relative Error in log ⁡ ( E / ρ ) \log(E/\rho) .

Relative Error in the logarithm of specific modulus, where $E$ is Young’s modulus and $\rho$ is density. Reflects relative deviation in stiffness-to-weight efficiency.

##### Relative Error in log ⁡ ( G ) \log(G) .

Relative Error in the logarithm of shear modulus $G=\frac{E}{2(1+\nu)}$, representing resistance to shear deformation.

##### Relative Error in log ⁡ ( K ) \log(K) .

Relative Error in the logarithm of bulk modulus $K=\frac{E}{3(1-2\nu)}$, characterizing resistance to uniform volumetric compression.

##### Lightweight Stiffness Ashby Index ( P = E 1 / 2 / ρ P=E^{1/2}/\rho ).

The Relative Error in $\log(P)$, where $P=E^{1/2}/\rho$, reflecting relative error in predicting material efficiency for maximizing stiffness per unit weight ashby1993materials.

##### Energy Absorption Ashby Index ( P = E 1 / 3 / ρ P=E^{1/3}/\rho ).

The Relative Error in $\log(P)$, where $P=E^{1/3}/\rho$, quantifying relative deviation in predicted energy absorption efficiency ashby1993materials.

##### Bray–Curtis dissimilarity.

Bray–Curtis dissimilarity https://doi.org/10.2307/1942268 between predicted and ground-truth property vectors $\mathbf{x}$ and $\mathbf{y}$:

$$ $\mathrm{BC}(\mathbf{x},\mathbf{y})=\frac{\sum_{i}|x_{i}-y_{i}|}{\sum_{i}(x_{i}+y_{i})}.$ (9) $$

A normalized, dimensionless measure in $[0,1]$ capturing overall distributional divergence across multiple material properties.

### D.3 Metrics to Measure Differences in Distributions

We use multiple commonly-used metrics for measuring differences between distributions.

##### Wasserstein–1 Distance ( W 1 W_{1} ).

For probability measures $\mu,\nu$ on a space $\mathcal{X}$,

$$ $W_{1}(\mu,\nu)=\inf_{\gamma\in\Pi(\mu,\nu)}\int_{\mathcal{X}\times\mathcal{X}}\|x-y\|\,d\gamma(x,y),$ (10) $$

where $\Pi(\mu,\nu)$ is the set of all couplings of $\mu$ and $\nu$. $W_{1}$ equals the minimum average “work” to move mass from $\mu$ to $\nu$.

##### Wasserstein–2 Distance ( W 2 W_{2} ).

For probability measures $\mu,\nu$ on a space $\mathcal{X}$,

$$ $W_{2}(\mu,\nu)=\Bigl(\inf_{\gamma\in\Pi(\mu,\nu)}\int_{\mathcal{X}\times\mathcal{X}}\|x-y\|^{2}\,d\gamma(x,y)\Bigr)^{1/2},$ (11) $$

where $\Pi(\mu,\nu)$ is the set of all couplings of $\mu$ and $\nu$. The root‐mean‐square transport cost between $\mu$ and $\nu$.

##### Kullback–Leibler Divergence ( D KL D_{\mathrm{KL}} ).

For densities $p,q$ on $\mathcal{X}$,

$$ $D_{\mathrm{KL}}(p\|q)=\int_{\mathcal{X}}p(x)\,\log\frac{p(x)}{q(x)}\,dx.$ (12) $$

$D_{\mathrm{KL}}$ measures the expected extra log‐likelihood of data drawn from $p$ when it is coded using $q$ instead of $p$.

Figure: Figure 16: Simulations to Interpret Errors. We demonstrate simulations performed to show the relation between relative error and simulation error.
Refer to caption: assets/compress_two_cubes/frame_000000.png

### D.4 Interpreting Errors for Material Property Estimation

We experimentally demonstrate an interpretation of how relative changes in material properties affect simulations of the finite element method (FEM) solver. We do so by simulating the deformation of unit cubes under many different material properties and scenarios with an FEM solver.

For each baseline triplet $(E_{0},\nu_{0},\rho_{0})$ representing Young’s modulus, Poisson’s ratio, and density, we introduce variations following the scaling laws: density variations follow linear scaling $\rho_{\text{new}}=\rho_{0}(1+\Delta)$, Poisson’s ratio variations use the same linear relationship $\nu_{\text{new}}=\nu_{0}(1+\Delta)$, while Young’s modulus variations use exponential scaling $E_{\text{new}}=E_{0}e^{\Delta}$ to accommodate the wide range of stiffness values. We then apply every such unique material triplet to a unit cube and perform a simulation under some external forces.

During each of these simulations, we measure the final volume and potential energy of the cube after the Newton iterations have converged.

##### Measuring Volume.

For a body undergoing deformation, the deformation gradient $\mathbf{F}=\nabla\mathbf{u}+\mathbb{I}$ maps material points from the reference to the current configuration, where $\mathbf{u}$ represents the displacement field and $\mathbb{I}$ is the $3\times 3$ identity tensor. The local volume change is quantified by the Jacobian $J=\det(\mathbf{F})$, which represents the ratio of deformed to reference volume at each material point. The total deformed volume is: $V_{\text{def}}=\int_{\Omega_{0}}J\,dV$, where $\Omega_{0}$ denotes the reference configuration. The relative volume change, defined as $\Delta V/V=(V_{\text{def}}-V_{0})/V_{0}$, provides a dimensionless measure of volumetric deformation.

##### Measuring Potential Energy.

We compute the total potential energy by combining elastic strain energy and kinetic-potential contributions. We use corotated linear elasticity, where we calculate the deformation gradient $\mathbf{F}=\nabla\mathbf{u}+\mathbb{I}$ and symmetric strain tensor $\mathbf{S}=\frac{1}{2}(\mathbf{F}+\mathbf{F}^{T})-\mathbb{I}$ to obtain the energy density $W=\mu\,\text{tr}(\mathbf{S}^{2})+\frac{\lambda}{2}(\text{tr}(\mathbf{S}))^{2}$, with Lamé parameters $\mu$ and $\lambda$ derived from the Young’s modulus and Poisson’s ratio. We use three distinct contributions in the kinetic-potential term: an inertial component $E_{\text{inertia}}=\int_{\Omega}\frac{\rho}{2\Delta t^{2}}|\mathbf{u}^{n+1}-\mathbf{u}^{n}|^{2}\,dV$ that captures displacement changes between iterations in our quasi-static solver, a gravitational potential $E_{\text{gravity}}=-\int_{\Omega}\rho\,\mathbf{u}\cdot\mathbf{g}\,dV$ accounting for body forces, and an external work term $E_{\text{ext}}=-\int_{\Omega}\mathbf{u}\cdot\mathbf{f}_{\text{ext}}\,dV$ representing the applied loads. We thus compute the total potential energy as $E_{\text{total}}=\int_{\Omega}W\,dV+E_{\text{inertia}}+E_{\text{gravity}}+E_{\text{ext}}$, evaluated at the converged displacement field.

We perform the simulations in the following scenarios,

##### Gripping force by robots.

We simulate a 140 N compressive force, which is common in robotic gripping applications, for example, the Franka Emika franka_description “Hand” end effector applies a maximum of 70 N per finger with a maximum clamping force of 140 N. We demonstrate the results from 486 simulations in this setting, all of which were run to convergence in [Fig.˜17](https://arxiv.org/html/2510.22975v1#A4.F17).

##### Impact Force on Dropping Objects.

We simulate a 120 N impact force that simulates package drop scenarios, calculated from the impact dynamics of a 1 kg package dropped from 0.6 m height with a 5 cm deformation distance. We demonstrate the results from 486 simulations in this setting, all of which were run to convergence in [Fig.˜18](https://arxiv.org/html/2510.22975v1#A4.F18).

##### Tensile Testing Machines.

We simulate a 330 N force corresponding to standard tensile testing conditions employed in bench-top universal testing machines ASTM_D638_2022; ASTM_E8_E8M_2024. We demonstrate the results from 486 simulations in this setting, all of which were run to convergence in [Fig.˜19](https://arxiv.org/html/2510.22975v1#A4.F19).

##### Tension.

We simulate a 200 N force, which represents typical pretension in tendon-driven robotic systems, where continuum arms and wearable assistive devices maintain structural stiffness through cable tensions Sch_ffer_2024. We demonstrate the results from 486 simulations in this setting all of which were run to convergence in [Fig.˜20](https://arxiv.org/html/2510.22975v1#A4.F20).

Figure: Figure 17: Gripping Force by Robots. We demonstrate the relation between relative errors in materials and relative change in P.E. (top) and volume (bottom). We then show the confidence bounds in light shaded regions.
Refer to caption: x44.png

Figure: Figure 18: Impact Force on Dropping Objects. We demonstrate the relation between relative errors in materials and relative change in P.E. (top) and volume (bottom). We show the confidence bounds in light shaded regions.
Refer to caption: x50.png

Figure: Figure 19: Tensile Testing Machine. We demonstrate the relation between relative errors in materials and relative change in P.E. (top) and volume (bottom). We show the confidence bounds in light shaded regions.
Refer to caption: x56.png

Figure: Figure 20: Tension. We demonstrate the relation between relative errors in materials and relative change in P.E. (top) and volume (bottom). We show the confidence bounds in light shaded regions.
Refer to caption: x62.png

## Appendix E Dataset Details

We present additional details about our datasets for training MatVAE ([§˜3](https://arxiv.org/html/2510.22975v1#S3)) and Geometry Transformer ([§˜4.2](https://arxiv.org/html/2510.22975v1#S4.SS2)).

### E.1 Annotation with Vision-Language Model

To create our training dataset ([§˜5](https://arxiv.org/html/2510.22975v1#S5)) for the Geometry Transformer, we use a Vision-Language Model (VLM) coupled with multiple other data sources like 3D assets, component-wise part segmentations, material databases ([§˜5.1](https://arxiv.org/html/2510.22975v1#S5.SS1)), visual textures, and material names to annotate our dataset. We run the VLM on every segment of every object individually. We experiment with Qwen2.5-VL 7B, Qwen2.5-VL 32B, Qwen2.5-VL 72B bai2023qwenvlversatilevisionlanguagemodel; bai2025qwen25vltechnicalreport, VL-Rethinker wang2023autorecon, SpatialRGPT cheng2024spatialrgpt, and Cosmos Nemotron lin2024vilapretrainingvisuallanguage. We experimentally choose Qwen2.5-VL 72B for data annotation. We show the system prompts and user prompts we use in [Fig.˜21](https://arxiv.org/html/2510.22975v1#A5.F21), [22](https://arxiv.org/html/2510.22975v1#A5.F22), [23](https://arxiv.org/html/2510.22975v1#A5.F23) and [24](https://arxiv.org/html/2510.22975v1#A5.F24). We find the best performing system prompts with TextGrad yuksekgonul2025optimizing. We show a response from the model for one of the segments from an object in our dataset in [Fig.˜25](https://arxiv.org/html/2510.22975v1#A5.F25) and [26](https://arxiv.org/html/2510.22975v1#A5.F26).

Figure: Figure 21: System Prompt. The System Prompt we use for every segment of every object.

Figure: Figure 22: User Prompt I. The User Prompt we use for every segment of every object.

Figure: Figure 23: User Prompt II. The User Prompt we use for every segment of every object.

Figure: Figure 24: User Prompt III. The User Prompt we use for every segment of every object.

Figure: Figure 25: Example Response I. We demonstrate an example response for a segment from one of the objects from our dataset. The given object has one part.
Refer to caption: assets/aluminumpallet.png

Figure: Figure 26: Example Response II. We demonstrate an example response for a segment from one of the objects from our dataset. The given object has two parts, and we show the response for the "rubber cap" part.
Refer to caption: assets/aluminumstepstand.png

We construct a tiny dataset consisting of complex objects that are manually annotated and compare these properties with the outputs from the VLM in [Tb.˜7](https://arxiv.org/html/2510.22975v1#A5.T7). We observe that the VLM, given significant additional information, performs very well in annotating complex objects. We report the commonly used metrics we list in [§˜D.2](https://arxiv.org/html/2510.22975v1#A4.SS2) for measuring differences in mechanical properties.

**Table 7: VLM Annotation Errors. Errors for the VLM annotation for mechanical property annotation.**
| $\log(E)$ ($\downarrow$) | $\nu$ ($\downarrow$) | $\rho$ ($\downarrow$) | $\log(E/\rho)$ ($\downarrow$) | $\log(G)$ ($\downarrow$) | $\log(K)$ ($\downarrow$) | L.S. ($\downarrow$) | E.A. ($\downarrow$) | Bray–Curtis ($\downarrow$) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 0.0295 | 0.0426 | 0.1348 | 0.1961 | 0.0303 | 0.0330 | 0.2022 | 0.2162 | 0.2342 |

### E.2 Dataset Statistics

Our dataset comprises a diverse collection of 1,692 objects, sourced from four datasets: simready nvidia_omniverse_simready, residential nvidia_omniverse_residential_assets, vegetation nvidia_omniverse_vegetation_assets, and commercial nvidia_omniverse_commercial_assets. As shown in [Tb.˜8](https://arxiv.org/html/2510.22975v1#A5.T8), the majority of objects belong to the simready and residential categories, with vegetation and commercial objects providing additional variety. Each object is decomposed into multiple segments, with a total of 8,128 segments across the dataset. Most parts are labeled with English material names, and for few parts that do not have these material names, we infer these from the PBR texture names that were applied to these parts. To characterize the physical realism and diversity of the dataset, we analyze the distribution of key material properties for all segments in [Tb.˜9](https://arxiv.org/html/2510.22975v1#A5.T9). The wide range of material properties highlights the heterogeneity of the dataset, which is essential for robust learning and evaluation of material-aware models. We summarize the most frequent material categories (e.g., metal, plastic, wood, cardboard) and object classes (e.g., residential, shelf, container), along with their respective counts and proportions in [Tb.˜10](https://arxiv.org/html/2510.22975v1#A5.T10).

We report the statistics of our Material Triplet dataset in [Tb.˜11](https://arxiv.org/html/2510.22975v1#A5.T11). To train MatVAE, we use the "Filtered Dataset".

**Table 8: Dataset Statistics. Number of objects, total segments, total points, average segments per object (std. dev.), and average points per object (std. dev.) for each dataset.**
| Dataset | Total Objects | Segments (%) | Voxels (%) | Avg. Segments/Object | Avg. Voxels/Object |
| --- | --- | --- | --- | --- | --- |
| commercial | 82 | 650 (8.0) | 1,812,064 (4.9) | 7.93 ($\pm$7.19) | 22,098 ($\pm$22,774) |
| residential | 449 | 4225 (52.2) | 9,109,380 (24.4) | 9.41 ($\pm$21.82) | 20,288 ($\pm$21,714) |
| simready | 1029 | 2544 (31.5) | 24,148,660 (64.7) | 2.47 ($\pm$1.33) | 23,468 ($\pm$25,032) |
| vegetation | 104 | 670 (8.3) | 2,267,848 (6.1) | 6.44 ($\pm$4.53) | 21,806 ($\pm$19,428) |
| train | 1333 | 6477 (80.1) | 28,709,190 (76.9) | 4.86 ($\pm$12.69) | 21,537 ($\pm$23,431) |
| validation | 165 | 552 (6.8) | 3,719,996 (10.0) | 3.35 ($\pm$3.19) | 22,545 ($\pm$23,095) |
| test | 166 | 1060 (13.1) | 4,908,766 (13.1) | 6.39 ($\pm$11.33) | 29,571 ($\pm$25,987) |
| Total | 1664 | 8089 (100.0) | 37,337,952 (100.0) | 4.86 ($\pm$11.97) | 22,439 ($\pm$23,786) |

**Table 9: Material property statistics for all segments in the dataset. We report the minimum, maximum, mean, median, standard deviation, and outlier count (% of values outside $\pm 3\sigma$) for Young’s modulus, Poisson’s ratio, and Density.**
| Property | Min | Max | Mean | Median | Std Dev | Outliers (%) |
| --- | --- | --- | --- | --- | --- | --- |
| Density ($\mathrm{kg}/\mathrm{m}^{3}$) | $5.0\times 10^{1}$ | $1.93\times 10^{4}$ | $2.28\times 10^{3}$ | $1.20\times 10^{3}$ | $2.44\times 10^{3}$ | 25 (0.3) |
| Young’s Modulus (Pa) | $1.0\times 10^{5}$ | $2.8\times 10^{11}$ | $4.19\times 10^{10}$ | $1.0\times 10^{10}$ | $6.53\times 10^{10}$ | 165 (2.0) |
| Poisson’s Ratio | $1.6\times 10^{-1}$ | $4.9\times 10^{-1}$ | $3.36\times 10^{-1}$ | $3.5\times 10^{-1}$ | $4.36\times 10^{-2}$ | 88 (1.1) |

**Table 10: Most frequent high-level material categories and object classes. We report the top high-level material categories (aggregated and deduplicated) and the most common object classes in the dataset, with their respective counts and percentages.**
| Mat. Category | Count (%) | Object Class | Count (%) |
| --- | --- | --- | --- |
| Metal | 1434 (17.7) | residential | 477 (28.2) |
| Plastic | 553 (6.8) | shelf | 250 (14.8) |
| Wood | 707 (8.7) | container | 193 (11.4) |
| Cardboard | 315 (3.9) | cardboard box | 106 (6.3) |
| Leather | 140 (1.7) | vegetation | 105 (6.2) |
| Chrome | 171 (2.1) | crate | 101 (6.0) |
| Glass | 85 (1.0) | commercial | 82 (4.8) |
| Fabric | 60 (0.7) | pallet | 67 (4.0) |
| Rubber | 55 (0.7) | barricade tape | 59 (3.5) |
| Stone | 40 (0.5) | inclined plane | 22 (1.3) |

**Table 11: Dataset Details. We present statistics of our dataset for training MatVAE ([§˜3](https://arxiv.org/html/2510.22975v1#S3)).**
| Dataset | Size |
| --- | --- |
| Material Ranges | 249 |
| Extracted Materials | 105456 |
| Filtered Materials | 101517 |

Figure: Figure 27: Young’s Modulus Pa ($E$). Histogram of Young’s Modulus in our Geometry with Volumetric Materials Dataset ([§˜5](https://arxiv.org/html/2510.22975v1#S5)).

Figure: Figure 28: Poisson’s Ratio ($\nu$). Histogram of Poisson’s Ratio in our Geometry with Volumetric Materials Dataset ([§˜5](https://arxiv.org/html/2510.22975v1#S5)).

Figure: Figure 29: Density $\frac{kg}{m^{3}}$. Histogram of Density in our Geometry with Volumetric Materials Dataset ([§˜5](https://arxiv.org/html/2510.22975v1#S5)).

## Appendix F Additional Implementation Details

We present additional implementation details.

### F.1 Design of MatVAE

We explain the motivation behind the design of MatVAE.

##### Normalizing Flow.

Material triplets remain statistically non‑Gaussian even after normalization (heavy‑tailed/multi‑modal $\log_{10}E,\log_{10}\rho$; boundary‑concentrated $\nu\in[0,0.5)$), so a diagonal‑Gaussian $q_{\phi}(z\mid m)$ tends to mode‑average and miscalibrate tails. We therefore parameterize the posterior with a bijective normalizing flow pmlr-v37-rezende15 $f_{\psi}$ where $\psi$ is the parameter of the flow network f, and $\Psi$ is the space of all parameters $\psi$. This $f_{\psi}$ applied to a Gaussian base $q_{0}$: sample $u\sim q_{0}(u\mid m)=\mathcal{N}(u;\mu_{\phi}(m),\mathrm{diag}\,\sigma_{\phi}^{2}(m))$, set $z=f_{\psi}(u)$, and compute the density by change of variables

$$ $\displaystyle\log q_{\phi}(z\mid m)$ $\displaystyle=\underbrace{\log q_{0}\bigl(f_{\psi}^{-1}(z)\mid m\bigr)}_{\text{base density}}+\underbrace{\log\bigl|\det J_{f_{\psi}^{-1}}(z)\bigr|}_{\text{log-Jacobian}}$ (13) $\displaystyle=\underbrace{\log q_{0}(u\mid m)}_{\text{base density}}-\underbrace{\log\bigl|\det J_{f_{\psi}}(u)\bigr|}_{\text{log-Jacobian}}\Big|_{u=f_{\psi}^{-1}(z)}.$ $$

with a standard normal prior $p(z)=\mathcal{N}(0,I)$ and decoder likelihood $p_{\theta}(m\mid z)$.

This keeps the ELBO form unchanged while strictly enlarging the variational family (the identity map recovers the Gaussian case), allowing $q_{\phi}$ to match the true posterior and avoid mode‑averaging on $(E,\nu,\rho)$. We instantiate $f_{\psi}$ as a single radial flow,

$$ $\begin{split}f_{\psi}(u)&=u+\underbrace{\beta\,h(u)}_{\text{radial scale}}\overbrace{(u-z_{0})}^{\text{displacement}},\\ h(u)&=\frac{1}{\bigl(\alpha+\lVert u-z_{0}\rVert_{2}\bigr)},\end{split}$ (14) $$

whose log-determinant has the closed form (substitute $h^{\prime}=-h^{2}$ in the closed form from pmlr-v37-rezende15),

$$ $\begin{split}\log\det J_{f_{\psi}}(u)&=\underbrace{(D-1)\,\log\bigl(1+\beta h(u)\bigr)}_{\text{angular}}\\ &\quad+\underbrace{\log\bigl(1+\beta h(u)-\beta\,h(u)^{2}\,r(u)\bigr)}_{\text{radial}},\\ r(u)&=\lVert u-z_{0}\rVert_{2},\end{split}$ (15) $$

where $D$ is the dimensionality of the latent space and $z_{0}$ is a *trainable* $D$-dimensional vector (one per flow layer) representing the centre of the deformation.

Radial flows are invertible iff $\alpha>0$ and $\beta>-\alpha$, we satisfy these by
$\alpha=\mathrm{softplus}(\tilde{\alpha}),\;\beta=-\alpha+\mathrm{softplus}(\tilde{\beta})$ pmlr-v37-rezende15 with unconstrained trainable parameters $\tilde{\alpha},\tilde{\beta}\in\mathbb{R}$. We show our implementation in [Algorithm˜1](https://arxiv.org/html/2510.22975v1#alg1).

##### Penalizing TC.

We observed high dependence between latent coordinates in the aggregated posterior $\bar{q}_{\phi}(z)$ (both dimensions tended to encode $\rho$). Thus, we decompose the KL-divergence term of the ELBO following chen2018isolating. For MatVAE, this allows us to directly penalize the total correlation $\text{TC}(z)=\text{KL}(\bar{q}_{\phi}(z)||\prod_{j}\bar{q}_{\phi}(z_{j}))$ where $\bar{q}_{\phi}(z)$ is the aggregated posterior, $z_{j}$ is the $j\in\{1,2\}$-th coordinate of the latent vector $z$. This allowed us to reduce the high dependence between latent coordinates, causing both dimensions to encode density. During training, we follow chen2018isolating and approximate the aggregated posterior $\bar{q}_{\phi}(z)=\mathbb{E}_{m\sim p_{data}}[q_{\phi}(z\mid m)]$ using samples from a mini-batch where $p_{data}$ is the empirical data distribution.

Figure: Algorithm 1 MatVAE posterior update with a radial normalizing flow.

##### Preventing Posterior Collapse.

For a fixed $m$, define per-dimension marginals $q_{\phi}(z_{j}\mid m)$ and

$$ $\mathrm{KL}_{j}(z\mid m)=\mathrm{KL}\left(q_{\phi}(z_{j}\mid m)\,\|\,p(z_{j})\right).$ (16) $$

Then, for each $m$,

$$ $\begin{split}\mathrm{KL}\left(q_{\phi}(z\mid m)\,\|\,p(z)\right)=&\sum_{j}\mathrm{KL}_{j}(m)\\ \quad+&\underbrace{\mathrm{TC}\left(q_{\phi}(z\mid m)\right)}_{\mathrm{KL}\left(q_{\phi}(z\mid m)\,\|\,\prod_{j}q_{\phi}(z_{j}\mid m)\right)}.\end{split}$ (17) $$

So the total KL contains a per-dimension rate term plus a per-sample total correlation. In the Gaussian (no-flow) case, $\mathrm{KL}_{j}(z\mid m)=\tfrac{1}{2}\left(\mu_{j}(m)^{2}+\sigma_{j}(m)^{2}-\log\sigma_{j}(m)^{2}-1\right)$, whose gradients drive $\mu_{j}\to 0$, $\sigma_{j}\to 1$ under posterior collapse. We therefore impose a capacity constraint (“free nats”) on the dim-wise term to prevent collapse:

$$ $\sum_{j=1}^{d}\underbrace{\max\bigl(\delta,\;\mathbb{E}_{p_{\mathrm{data}}}\mathrm{KL}_{j}(z)\bigr)}_{\text{free-nats}},$ (18) $$

which enforces a minimum information budget $\delta=0.1$ per coordinate (zero subgradient below $\delta$). This allows us to fix the empirically observed imbalance where one latent carried most information and the other collapsed. An aggregated alternative consistent with the KL decomposition is $\max\bigl(\phi\cdot d,\ \sum_{j}\mathrm{KL}(q_{\phi}(z_{j})\,\|\,p(z_{j}))\bigr)$.

### F.2 Network Design

We now present our network architecture.

##### MatVAE.

The *encoder* architecture begins by projecting the 3-dimensional material triplet through a linear transformation into a 256-dimensional hidden space, followed by SiLU activation. The resulting representation then passes through three "ResidualBlocks", each using a bottleneck design that compresses the 256-dimensional vector to 128 dimensions via LayerNorm and SiLU activation, applies another linear transformation, and restores the original dimensionality through a second LayerNorm-SiLU sequence. Each "ResidualBlock" maintains a skip connection that adds the input directly to the final output. The encoder finally has separate linear heads that project the processed representation into the latent space parameters: one head predicts the posterior mean $\mu_{\phi}(m)$ and another predicts the log-variance $\log\sigma^{2}_{\phi}(m)$ for the 2-dimensional latent code $z$.

The *decoder* mirrors this architecture in reverse, beginning with a linear projection from the 2-dimensional latent space back to the 256-dimensional hidden representation, followed by SiLU activation. The latent encoding then goes through three "ResidualBlocks" with an identical bottleneck structure and skip connections as the encoder. Finally, three separate linear heads decode the processed representation into the reconstructed material properties: Young’s modulus, Poisson’s ratio, and density, each predicted as scalar values in the normalized space.

##### Geometry Transformer.

Our model is based on TRELLIS xiang2025structured3dlatentsscalable. We use a transformer-based architecture specifically designed for processing sparse voxel representations with associated material properties. The model operates on a 64^3 resolution voxel grid, accepting 1024-dimensional DINOv2 visual features as input and compressing them to a compact 2-dimensional latent representation through a 12-layer transformer backbone. Each transformer block utilizes 12 attention heads with a 4:1 MLP expansion ratio, using Swin attention with 8$\times$8 local windows. During training, the Geometry Transformer operates in conjunction with a frozen MatVAE that decodes the latent into material properties.

### F.3 Training

We present our voxelization scheme for training on meshes in [Algorithms˜2](https://arxiv.org/html/2510.22975v1#alg2) and [3](https://arxiv.org/html/2510.22975v1#alg3). We present the hyperparameters used for training MatVAE and Geometry Transformer in [Tb.˜12](https://arxiv.org/html/2510.22975v1#A6.T12).

Figure: Algorithm 2 Segment-aware volumetric voxelization for meshes.

Figure: Algorithm 3 Voxelization and flood fill primitives for meshes.

**Table 12: Training Hyperparameters. We show the hyperparameters for the MatVAE and Geometry Transformer.**
| MatVAE |  |
| --- | --- |
| Training Precision | FP-32 |
| Hidden Width | 256 |
| Network Depth | 3 ($\times 2$) |
| Latent Dimensions | 2 |
| Dropout Rate | 0.05 |
| Epochs | 850 |
| Batch Size | 256 |
| Optimizer | AdamW |
| Learning Rate | $10^{-4}$ |
| Weight Decay | $10^{-4}$ |
| LR Scheduler | Cosine Annealing |
| Final Learning Rate | $10^{-5}$ |
| Gradient Clipping | 5.0 |
| $\beta$-TC Loss Weights | $\alpha=1.0$ (KL) |
| $\beta=2.0$ (TC) |  |
| $\gamma=1.0$ (MI) |  |
| Free Nats | 0.1 |
| KL Annealing Epochs | 200 |
| Data Normalization | Log Min-Max |

##### Voxelization For Training.

Our training dataset contains Universal Scene Description (USD) files with multi-segment meshes.
Each mesh is normalized to the range $[-0.5,0.5]$ using a global bounding box computed across all segments to preserve relative spatial relationships. We use volumetric voxelization using a regular 3D grid with a resolution of $\frac{1}{64}$, where each voxel center is tested for interior containment within the mesh volume through point-in-polyhedron testing, followed by volumetric filling to generate solid voxel representations rather than surface-only discretizations. All the voxels inside a given segment receive the material properties of the segment they lie in.

##### Rendering for Training.

For multi-view image rendering of meshes, we use a path-tracing renderer to produce photorealistic renderings of 3D objects. Camera viewpoints are sampled using a quasi-random Hammersley sequence distributed uniformly across a sphere. For training and testing, we render 150 views, though our method can work by rendering as many views as needed, with cameras positioned at a fixed radius of 2 units from the object center and configured with a 40-degree field of view. Images are rendered at 512$\times$512 pixel resolution. The rendering pipeline outputs both the RGB images and the corresponding camera extrinsics and intrinsics. For rendering splats, we simply replace the renderer with the 3D Gaussian Splat renderer kerbl3Dgaussians in our workflow. For rendering SDFs, we render meshes using many points collected from the SDF. For rendering NeRFs mildenhall2020nerf, we simply replace the renderer with nerfstudio nerfstudio in our workflow.

##### Feature Aggregation.

For visual feature extraction, we employ DINOv2-ViT-L/14 oquab2024dinov2learningrobustvisual with registers. We use a patch size of 14$\times$14 pixels and process input images resized to 518$\times$518 pixels, resulting in a 37$\times$37 patch. We use the nv-dinov2(^2^22[https://build.nvidia.com/nvidia/nv-dinov2](https://build.nvidia.com/nvidia/nv-dinov2)) nvidia_nv_dinov2_2025 implementation.

### F.4 Simulation and Rendering

For our mesh simulations ([Fig.˜5](https://arxiv.org/html/2510.22975v1#S6.F5)), we simulate with the finite-element method (FEM) using the libuipc 10.1145/3735126; gipc2024 implementation, and we render the simulations in a path-tracing-based renderer. While comparing with other simulators ([Fig.˜2](https://arxiv.org/html/2510.22975v1#S1.F2)) we use MPM mpm using taichi-mpm TaichiMPM, XPBD xpbd using PositionBasedDynamics PositionBasedDynamics, and FEM using Warp warp2022.

For our large-scale splat simulations or splat + mesh simulations, we use Simplicits 10.1145/3658184 using the sparse simplicits implementation using Kaolin 10.1145/3658184; KaolinLibrary. For rendering our large-scale splat simulations or splat + mesh simulations, we use Polyscope polyscope and composite splat renders from gsplat gsplat. For these simulations, we apply material property tolerances to reduce numerical noise: voxels with Young’s modulus differing by less than $10^{1}$ Pa, Poisson’s ratio by less than $10^{-3}$, or density by less than $10^{1}$ kg/m³ are assigned identical values for the respective property. We present additional details for deforming and rendering deformed Gaussian Splats in [§˜G.5](https://arxiv.org/html/2510.22975v1#A7.SS5).

### F.5 Baselines

##### Converting Hardness to Young’s Modulus.

NeRF2Physics zhai2024physicalpropertyunderstandinglanguageembedded does not estimate a numerical value of Young’s Modulus, but instead predicts Shore A-Shore D hardness. Thus, to compare our method with NeRF2Physics zhai2024physicalpropertyunderstandinglanguageembedded we convert these Shore hardness values to average Young’s Modulus values.

- Shore A. For Shore A hardness, we follow astm2240 and use:
$E_{\text{MPa}}=e^{(S_{A}\times 0.0235)-0.6403}$
(19)
where $S_{A}$ is the Shore A hardness value and $E_{\text{MPa}}$ is Young’s modulus in megapascals.
- Shore D. For Shore D hardness, we follow astm2240 and use:
$E_{\text{MPa}}=e^{((S_{D}+50)\times 0.0235)-0.6403}$
(20)
where $S_{D}$ is the Shore D hardness value and $E_{\text{MPa}}$ is Young’s modulus in megapascals.

##### Point or Voxel Sampling.

The baselines NeRF2Physics zhai2024physicalpropertyunderstandinglanguageembedded and PUGS shuai2025pugszeroshotphysicalunderstanding in their methods sample points from the NeRF or Gaussian splat, respectively, and predict mechanical properties at those points. To ensure fair comparisons in [Tb.˜2](https://arxiv.org/html/2510.22975v1#A1.T2) and [6](https://arxiv.org/html/2510.22975v1#S6.F6), we explicitly make these methods work on the same set of points in the object on which our method is evaluated.

##### Implementation details of Baselines.

The baseline NeRF2Physics zhai2024physicalpropertyunderstandinglanguageembedded uses gpt-3.5-turbo for certain parts of their pipeline. We replace gpt-3.5-turbo in their pipeline with a better performing model, GPT-4o openai2024gpt4technicalreport. The baseline Phys4DGen liu2024physgen does not have code available. Thus, we faithfully reproduce the parts, "Material Grouping and Internal Discovery" and "MLLMs-Guided Material Identification". We reproduce these parts of their pipeline using GPT-4o openai2024gpt4technicalreport for the MLLMs-Guided Material Identification. Furthermore, we obtained the prompts from the authors of Phys4DGen liu2024physgen and use the same prompts.

## Appendix G Additional Details on the Simulations

We experiment with Simplicits 10.1145/3658184, a reduced-order simulator ([Fig.˜1](https://arxiv.org/html/2510.22975v1#S0.F1), [5](https://arxiv.org/html/2510.22975v1#S6.F5), [8](https://arxiv.org/html/2510.22975v1#A1.F8) and [8](https://arxiv.org/html/2510.22975v1#A1.F8)) and an accurate finite-element method (FEM) simulator ([Fig.˜1](https://arxiv.org/html/2510.22975v1#S0.F1), [5](https://arxiv.org/html/2510.22975v1#S6.F5) and [8](https://arxiv.org/html/2510.22975v1#A1.F8)) with our material properties. We also use a FEM simulator for our experiments on interpreting errors in properties ([§˜D.4](https://arxiv.org/html/2510.22975v1#A4.SS4)). We use a material point method (MPM) mpm, and an Extended Position Based Dynamics (XPBD) xpbd simulator for our experiments to compare between simulators ([Fig.˜2](https://arxiv.org/html/2510.22975v1#S1.F2)). We share details on these simulations. We also share details on the interpolation we use across all our simulations. We share the hyperparameters used for all the FEM simulations in [Tb.˜13](https://arxiv.org/html/2510.22975v1#A7.T13).

**Table 13: Hyperparameters for FEM simulation.**
| Hyperparameter | Value | Hyperparameter | Value |
| --- | --- | --- | --- |
| Time Integrator | Backward Euler | Linear Solver | pre-conditioned CG |
| Nonlinear Solver | Newton’s w/ line search | Linear tolerance | $10^{-3}$ |
| Newton max iters. | 1024 | Line search |  |
| Velocity tol. | 0.05 $ms^{-1}$ | max iters | 8 |
| CCD tol. | 1.0 | Collision |  |
| Transform rate tol. | 0.1/s | Friction | 0.5 |
| $dt$ | 0.02 | Contact Resistance | 1.0 |
| Gravity | $[0.0,-9.8,0.0]$ | $\hat{d}$ | 0.01 |

### G.1 Interpolation Scheme

Our simulations receive a material field sampled on a voxel grid predicted by VoMP, i.e., values $m(\mathbf{X}_{i})$ given at lattice points $\{\mathbf{X}_{i}\}\subset\Omega$. When the simulator needs material values at arbitrary query locations $\mathbf{X}$ (e.g., element centroids or vertices), we evaluate a nearest-neighbour interpolation of the voxel field:

$$ $\begin{split}i^{*}(\mathbf{X})&=\arg\min_{i}\lVert\mathbf{X}-\mathbf{X}_{i}\rVert_{2},\\ m^{*}(\mathbf{X})&=m\big(\mathbf{X}_{\,i^{*}(\mathbf{X})}\big).\end{split}$ (21) $$

We intentionally avoid higher-order interpolation of material fields since real objects are piecewise-constant across label regions, and convex blending across parts of the objects invents intermediate materials. These intermediate materials might not be physically present or admissible, while our outputs fall into a valid material due to the MatVAE ([§˜3](https://arxiv.org/html/2510.22975v1#S3)). Nearest-neighbour preserves sharp interfaces and is usually robust for arbitrary query locations.

### G.2 Preparing Scenes and Assigning Materials for the FEM Solver

Mechanical properties are set either uniformly (like in [§˜D.4](https://arxiv.org/html/2510.22975v1#A4.SS4)) or heterogeneously from a voxel field. For uniform assignment, given $E$ and $\nu$ we compute Lamé parameters

$$ $\lambda\,=\,\frac{E\,\nu}{(1+\nu)(1-2\nu)}\,,\qquad\mu\,=\,\frac{E}{2(1+\nu)}\,,$ (22) $$

which are used elementwise together with a constant mass density $\rho$.

For heterogeneous assignment, a voxel lattice provides $E(\mathbf{X})$, $\nu(\mathbf{X})$, and $\rho(\mathbf{X})$ at voxel centers. After applying the same rigid/scale transform as the mesh, each tetrahedron takes $\lambda,\mu$ from the nearest voxel to its centroid, and each vertex takes $\rho$ from the nearest voxel to its position. This produces per-tetrahedron $\lambda,\mu$ and per-vertex $\rho$ fields that are directly used in the elastic strain energy density per unit reference volume ($W$), first variation of the incremental potential ($R$), and Newton-Jacobian ($\mathcal{K}$).

During simulation, a visual mesh is embedded into the physics mesh by assigning each visual vertex $\mathbf{x}_{v}$ to a containing (or nearest) tetrahedron with vertices $\{\mathbf{X}_{a}\}_{a=1}^{4}$ and barycentric weights $\{w_{a}\}_{a=1}^{4}$ satisfying $\sum_{a}w_{a}=1$ and $\sum_{a}w_{a}\,\mathbf{X}_{a}=\mathbf{x}_{v}$; its deformed position is then the barycentric interpolation of current nodal positions $\{\mathbf{x}_{a}\}_{a=1}^{4}$:

$$ $\mathbf{x}_{v}^{\,\mathrm{def}}\,=\,\sum_{a=1}^{4}w_{a}\,\mathbf{x}_{a}\,.$ (23) $$

The state update in our simulation experiments is computed time-step by time-step, and we also deform and move the visual mesh according to the physics mesh at each time step.

### G.3 Details of the FEM Solver

For FEM simulations, we use a simulator based on the libuipc 10.1145/3735126; gipc2024 implementation and the Warp (warp.fem) warp2022 implementation. We first explain the details for our simulations in §[D.4](https://arxiv.org/html/2510.22975v1#A4.SS4).

We consider a deformable continuum body with reference configuration $\Omega\subset\mathbb{R}^{3}$ and boundary $\partial\Omega=\Gamma_{D}\cup\Gamma_{N}$, where $\Gamma_{D}$ denotes boundary points with Dirichlet boundary conditions, and $\Gamma_{N}$ denotes boundary points with Neumann boundary conditions. The unknown to solve for is the displacement field $\mathbf{u}:\Omega\to\mathbb{R}^{3}$. Time is discretized into frames with a fixed step $\Delta t$. At each frame we compute an increment $\Delta\mathbf{u}$ that advances the configuration $\mathbf{u}\leftarrow\mathbf{u}+\Delta\mathbf{u}$ while enforcing Dirichlet constraints on $\Gamma_{D}$. The deformation map is $\varphi(\mathbf{X})=\mathbf{X}+\mathbf{u}(\mathbf{X})$, with deformation gradient $\mathbf{F}(\mathbf{u})=\mathbf{I}+\nabla\mathbf{u}$, Jacobian $J=\det\mathbf{F}$, and isochoric invariant $I_{c}=\mathrm{tr}(\mathbf{F}^{\top}\mathbf{F})$. For corotational modeling, we use the stretch tensor $\mathbf{S}$ from the polar/SVD decomposition: if $\mathbf{F}=\mathbf{U}\,\mathrm{diag}(\bm{\sigma})\,\mathbf{V}^{\top}$ then $\mathbf{S}=\mathbf{V}\,\mathrm{diag}(\bm{\sigma})\,\mathbf{V}^{\top}$. Given Young’s modulus $E$ and Poisson ratio $\nu$, the Lamé parameters are $\lambda=E\nu/((1+\nu)(1-2\nu))$ and $\mu=E/(2(1+\nu))$. Here $\nabla$ denotes the gradient with respect to reference coordinates, $\mathbf{A}{:}\mathbf{B}=\mathrm{tr}(\mathbf{A}^{\top}\!\mathbf{B})$ is the Frobenius inner product, and $\lVert\cdot\rVert$ is the Euclidean norm.

The elastic response we use is the corotational Hookean model. Define the small strain $\bm{\varepsilon}=\mathbf{S}-\mathbf{I}$. The strain energy density and Kirchhoff stress are

$$ $\begin{split}W_{\mathrm{CR}}(\mathbf{S})&=\underbrace{\mu\,\bm{\varepsilon}\!:\!\bm{\varepsilon}}_{\text{shear (deviatoric)}}+\underbrace{\tfrac{\lambda}{2}\,\mathrm{tr}(\bm{\varepsilon})^{2}}_{\text{volumetric}},\\ \bm{\tau}(\mathbf{S})&=\underbrace{2\mu\,\bm{\varepsilon}}_{\text{shear}}+\underbrace{\lambda\,\mathrm{tr}(\bm{\varepsilon})\,\mathbf{I}}_{\text{volumetric}},\end{split}$ (24) $$

with a consistent linearization obtained via the variation of $\mathbf{S}$ with respect to $\mathbf{F}$ and projected to maintain symmetry and positive semidefiniteness.

Each frame solves an incremental variational problem. Given the previous increment $\Delta\mathbf{u}^{\,n-1}$, we seek $\Delta\mathbf{u}^{\,n}$ that approximately minimizes the incremental potential

$$ $\begin{split}\Pi(\Delta\mathbf{u})&=\underbrace{\int_{\Omega}\rho\left(\tfrac{1}{2}\,\frac{\lVert\Delta\mathbf{u}-\Delta\mathbf{u}^{\,n-1}\rVert^{2}}{\Delta t^{2}}\right)\,\mathrm{d}V}_{\text{inertial regularization}}\\ &\quad+\underbrace{\int_{\Omega}\big(-\rho\,\mathbf{g}\cdot\Delta\mathbf{u}-\mathbf{f}_{\!\mathrm{ext}}\cdot\Delta\mathbf{u}\big)\,\mathrm{d}V}_{\text{body and external work}}\\ &\quad+\underbrace{\int_{\Omega}W_{\mathrm{CR}}\big(\mathbf{S}(\mathbf{u}^{n-1}+\Delta\mathbf{u})\big)\,\mathrm{d}V}_{\text{elastic energy}}\\ &\quad+\underbrace{\Pi_{\mathrm{int}}(\Delta\mathbf{u})}_{\text{interior/boundary regularization}},\end{split}$ (25) $$

where $\rho$ is the mass density, $\mathbf{g}$ is the gravitational acceleration vector, $\mathbf{f}_{\!\mathrm{ext}}$ denotes prescribed volumetric loads, and $\Pi_{\mathrm{int}}$ denotes any interior/boundary regularization term (e.g., a jump penalty in discontinuous settings). The admissible test function $\mathbf{v}$ is any sufficiently smooth virtual displacement that vanishes on $\Gamma_{D}$. The first variation $\delta\Pi(\Delta\mathbf{u};\mathbf{v})=0$ for all such $\mathbf{v}$ yields the residual functional

$$ $\begin{split}R(\Delta\mathbf{u})[\mathbf{v}]&=\underbrace{\int_{\Omega}\rho\,\frac{\Delta\mathbf{u}-\Delta\mathbf{u}^{\,n-1}}{\Delta t^{2}}\cdot\mathbf{v}\,\mathrm{d}V}_{\text{inertia}}\\ &\quad+\underbrace{\int_{\Omega}\big(-\rho\,\mathbf{g}\cdot\mathbf{v}-\mathbf{f}_{\!\mathrm{ext}}\cdot\mathbf{v}\big)\,\mathrm{d}V}_{\text{body/external}}\\ &\quad-\underbrace{\int_{\Omega}\bm{\tau}\big(\mathbf{S}(\mathbf{u}^{n-1}+\Delta\mathbf{u})\big):\nabla\mathbf{v}\,\mathrm{d}V}_{\text{elastic (internal) virtual work}}\\ &\quad+\underbrace{R_{\mathrm{int}}(\Delta\mathbf{u})[\mathbf{v}]}_{\text{regularization}},\end{split}$ (26) $$

which is set to zero for all $\mathbf{v}$. Newton’s method is applied to $R(\Delta\mathbf{u})=0$. At iterate $\Delta\mathbf{u}^{\,(k)}$ we assemble the consistent tangent operator $\mathcal{K}=\mathrm{D}R\big[\Delta\mathbf{u}^{\,(k)}\big]$ (the Gàteaux derivative of $R$) and solve the linear system

$$ $\begin{split}\mathcal{K}\,\delta\mathbf{u}&=-R\big(\Delta\mathbf{u}^{\,(k)}\big),\\ \Delta\mathbf{u}^{\,(k+1)}&=\Delta\mathbf{u}^{\,(k)}+\alpha\,\delta\mathbf{u}\end{split}$ (27) $$

where $\alpha\in(0,1]$ is chosen by a backtracking Armijo rule to guarantee sufficient decrease of $\Pi$. The operator $\mathcal{K}$ contains an inertial mass-like term $\int_{\Omega}\rho\,\Delta t^{-2}\,\delta\mathbf{u}\cdot\mathbf{v}\,\mathrm{d}V$, the consistent elastic tangent from the linearization of $\bm{\tau}(\mathbf{S}(\cdot))$, and any interior/boundary penalty contributions. This procedure is repeated until the update norm or residual falls below a prescribed tolerance.

For all our other simulations (i.e. except the simulations in §[D.4](https://arxiv.org/html/2510.22975v1#A4.SS4)) we use a closely related variant whose differences are in the constitutive law, material assignment, mesh preparation/interpolation, and contact handling. First, the stored energy and stress are taken to be compressible Neo-Hookean with volumetric regularization. Writing $\mathbf{C}=\mathbf{F}^{\top}\!\mathbf{F}$ and $\mathbf{B}=\mathbf{F}\,\mathbf{F}^{\top}$, the energy and Kirchhoff stress are

$$ $\begin{split}W_{\mathrm{NH}}(\mathbf{F})\,&=\,\tfrac{\mu}{2}\big(\mathrm{tr}\,\mathbf{C}-3-2\,\ln J\big)+\tfrac{\lambda}{2}\,(\ln J)^{2},\\ \bm{\tau}_{\mathrm{NH}}(\mathbf{F})\,&=\,\mu\,(\mathbf{B}-\mathbf{I})+\lambda\,\ln J\,\mathbf{I}.\end{split}$ (28) $$

This change only affects the elastic terms in $\Pi$, $R$, and $\mathcal{K}$; the kinematics and inertial terms remain the same. For the simulation experiments, we also use IPC 10.1145/3386569.3392425 for collision handling.

### G.4 Preparing Scenes and Assigning Materials for the Simplicits Solver

Each object is specified by a set of quadrature points $\mathcal{Q}=\{\mathbf{X}_{q}\}$ that sample its volume (used for elasticity and inertia), a set of collision particles $\mathcal{C}=\{\mathbf{X}_{c}\}$ for contact, and a set of visual vertices for rendering. We position objects with a rigid transform (origin and rotation) and an object scale; these transforms are applied consistently when evaluating kinematics, gravity, and material fields.

We embed all objects into a regular grid domain and attach to this grid a low-dimensional Simplicits subspace. The displacement basis is the product of a trilinear grid shape and a per-object handle shape, with multiple duplicated handles per grid vertex. At each quadrature point, we evaluate and cache per-node subspace weights and their spatial gradients. These weights modulate the duplicated handle functions during assembly, instantiating the Simplicits subspace on the grid.

Material parameters are assigned per quadrature point from a voxel lattice providing $E(\mathbf{X})$, $\nu(\mathbf{X})$, and $\rho(\mathbf{X})$. After applying the same rigid/scale transform as the object, each quadrature location $\mathbf{X}_{q}$ takes its material from the nearest voxel to $\mathbf{X}_{q}$. We compute Lamé parameters per point as

$$ $\begin{split}\lambda_{q}\,&=\,\frac{E(\mathbf{X}_{q})\,\nu(\mathbf{X}_{q})}{(1+\nu(\mathbf{X}_{q}))(1-2\nu(\mathbf{X}_{q}))}\,,\\ \qquad\mu_{q}\,&=\,\frac{E(\mathbf{X}_{q})}{2(1+\nu(\mathbf{X}_{q}))}\,,\end{split}$ (29) $$

and use $\rho_{q}=\rho(\mathbf{X}_{q})$ in the inertial terms.

The quadrature weights are set uniformly as

$$ $w_{q}\,=\,\frac{v}{\lvert\mathcal{Q}\rvert},$ (30) $$

where $v$ is the object volume estimate and $\lvert\mathcal{Q}\rvert$ is the number of quadrature points. This makes elastic and inertial energies invariant to the sampling density.

Collision particles $\mathcal{C}$ are used for detecting and resolving contact against other particles and registered kinematic triangle meshes (containers and obstacles). We use an IPC-style barrier with Coulomb friction, and scale the contact stiffness by object volume and the number of collision particles to obtain comparable penalties across scenes.

### G.5 Deforming Splats and Rebndering Deformed Splats

We render each object as a set of anisotropic Gaussian splats. At rest, a splat is parameterized by its mean $\bm{\mu}_{0}\in\mathbb{R}^{3}$, a unit quaternion $\mathbf{q}_{0}$ (with rotation $\mathbf{R}_{0}\in\mathrm{SO}(3)$), and axis scales $\mathbf{s}_{0}\in\mathbb{R}_{>0}^{3}$. We define the rest-frame shape operator

$$ $\underbrace{\mathbf{L}}_{\text{rest anisotropy}}\,=\,\mathbf{R}_{0}\,\mathrm{diag}(\mathbf{s}_{0})\,.$ (31) $$

During simulation, the displacement field yields a world-space deformation gradient $\underbrace{\mathbf{F}}_{\text{local deformation}}$ at the splat center and a world-space position $\bm{\mu}$ (obtained by evaluating the embedded deformation at the visual vertex). We map the rest anisotropy through the local deformation to obtain the world-space covariance of the splat as

$$ $\underbrace{\Sigma}_{\text{world covariance}}\,=\,\underbrace{(\mathbf{F}\,\mathbf{L})}_{\text{deformed axes}}\;\overbrace{(\mathbf{F}\,\mathbf{L})^{\top}}^{\text{deformed axes}^{\top}}\;+\;\underbrace{\varepsilon\,\mathbf{I}}_{\text{SPD padding}}\,.$ (32) $$

Here $\varepsilon>0$ is a small scalar that guarantees positive-definiteness under extreme compression.

For rasterization we pass $\bm{\mu}$ and $\Sigma$ to the Gaussian renderer. The renderer (gsplat gsplat) expects a symmetric 6-vector parameterization; we therefore pack the lower-triangular entries as

$$ $\underbrace{\mathbf{c}}_{\text{packed covariance}}\,=\,\big[\,\Sigma_{11},\;\Sigma_{12},\;\Sigma_{13},\;\Sigma_{22},\;\Sigma_{23},\;\Sigma_{33}\,\big]^{\top}\,.$ (33) $$

Color appearance (spherical-harmonic coefficients) and opacity are carried from the rest representation; only the mean $\bm{\mu}$ and covariance $\Sigma$ change over time. We also support a scalar scale multiplier applied to $\mathbf{s}_{0}$ for interactive control in qualitative visualizations.

Given the view (extrinsic) matrix $\mathbf{V}$ and vertical field-of-view, we synthesize camera intrinsics

$$ $\begin{split}\mathbf{K}\,&=\,\begin{bmatrix}f_{x}&0&c_{x}\\ 0&f_{y}&c_{y}\\ 0&0&1\end{bmatrix},\\ f_{x}\,&=\,\tfrac{W}{2\tan(\tfrac{\mathrm{fov}_{x}}{2})},\;f_{y}\,=\,\tfrac{H}{2\tan(\tfrac{\mathrm{fov}_{y}}{2})},\;\\ c_{x}\,&=\,\tfrac{W}{2},\;c_{y}\,=\,\tfrac{H}{2},\end{split}$ (34) $$

with image size $(W,H)$. We then render the set of splats $\{\bm{\mu},\,\mathbf{c}\}$ under $(\mathbf{V},\mathbf{K})$ to produce RGB (and depth) frames. At every frame, we interpolate $\bm{\mu}$ and $\mathbf{F}$ at the visual vertices, form $\Sigma=\mathbf{F}\,\mathbf{L}\,\mathbf{L}^{\top}\mathbf{F}^{\top}+\varepsilon\,\mathbf{I}$, pack it as $\mathbf{c}$, and feed the Gaussian rasterizer together with the stored colors and opacities.

### G.6 Details of the Simplicits Solver

We use the Simplicits 10.1145/3658184 simulator based on the implementation in Kaolin KaolinLibrary. Simplicits solves for a displacement field represented in a low-dimensional subspace attached to a regular grid. This subspace is a product basis between trilinear grid polynomials and duplicated per-vertex “handle” functions whose influence is modulated by per-point weights and weight gradients evaluated at quadrature points. We assemble inertia and compressible Neo-Hookean elasticity on this subspace, using the per-quadrature Lamé parameters $(\lambda_{q},\mu_{q})$ and measures $w_{q}$. Each frame performs Newton steps on the incremental potential with backtracking line search. Linear systems are solved by preconditioned conjugate gradients.

In the Kaolin implementation, splat–splat contact uses particle pairs: for a pair $(a,b)$ with current positions $\mathbf{x}_{a},\mathbf{x}_{b}$ and contact radius $r$, we set $\mathbf{n}_{c}=(\mathbf{x}_{a}-\mathbf{x}_{b})/\lVert\mathbf{x}_{a}-\mathbf{x}_{b}\rVert$, $r_{c}=r$, and use the relative offset $\mathbf{o}_{c}(\Delta\mathbf{u})=\Delta\mathbf{u}_{a}-\Delta\mathbf{u}_{b}$, so that $d_{c}=\mathbf{n}_{c}\!\cdot(\mathbf{x}_{a}-\mathbf{x}_{b})$ and $\mathbf{v}_{t,c}$ measures tangential slip between the two.

However, in our implementation, splat–mesh contact is handled differently than splat-splat contact. For splat-mesh contact, instead of using the collision points, we use triangle meshes as kinematic colliders: for a particle at $\mathbf{x}$ and its closest point $\mathbf{p}$ on a nearby triangle (with interpolated mesh normal/velocity $\mathbf{n}_{c},\mathbf{v}_{m}$), we take $r_{c}=2r$ and

$$ $\mathbf{o}_{c}(\Delta\mathbf{u})\,=\,\Delta\mathbf{u}+\big(\lVert\mathbf{x}-\mathbf{p}\rVert-\mathbf{n}_{c}\!\cdot\Delta\mathbf{u}\big)\,\mathbf{n}_{c}-\mathbf{v}_{m}\,,\qquad d_{c}\,=\,\lVert\mathbf{x}-\mathbf{p}\rVert-\mathbf{n}_{c}\!\cdot\mathbf{v}_{m}\,.$ (35) $$

Only the simulated-object DOFs enter $\mathbf{o}_{c}$; mesh or splat motion appears through $\mathbf{v}_{m}$.

These terms are used in the Newton system as

$$ $\mathbf{K}\;\leftarrow\;\mathbf{K}\;+\;\underbrace{\alpha\,\mathbf{H}^{\top}\mathbf{C}\,\mathbf{H}}_{\text{contact stiffness}}\,,\qquad\mathbf{r}\;\leftarrow\;\mathbf{r}\;-\;\underbrace{\alpha\,\mathbf{H}^{\top}\mathbf{g}}_{\text{contact force}}\,,$ (36) $$

where $\mathbf{H}$ is the Jacobian of contact offsets, and $\mathbf{g},\mathbf{C}$ are the per-contact gradient/Hessian with respect to $\mathbf{o}_{c}$. For splat–mesh contacts only the simulated-object block of $\mathbf{H}$ is present; for splat–splat contacts the two object blocks appear with opposite signs.

### G.7 Preparing Scenes and Assigning Materials for the XPBD Solver

We also use an Extended Position-Based Dynamics (XPBD) solver xpbd based on PositionBasedDynamics PositionBasedDynamics.

##### Particles and initialization.

Objects are represented by particles positioned at rest locations $\{\mathbf{X}_{i}\}$. For each particle we initialize position $\mathbf{x}_{i}\!\leftarrow\!\mathbf{X}_{i}$ and a previous position equal to $\mathbf{X}_{i}$ (Verlet), set mass $m_{i}$ and inverse mass $w_{i}=1/m_{i}$ (pinned points use $w_{i}=0$). A soft sphere uses one center particle and a set of surface particles sampled on a UV sphere; the center forms simple tetrahedra with nearby surface points for volume preservation.

##### Material parameters and compliance.

To target an elastic behavior with Young’s modulus $E$ and Poisson ratio $\nu$ (bulk modulus $K=E/(3(1-2\nu))$), we choose distance- and volume-constraint compliances $\alpha_{\mathrm{dist}},\alpha_{\mathrm{vol}}$ inversely proportional to $E$ and $K$. XPBD uses the scaled compliance $\alpha_{\Delta}=\alpha/\Delta t^{2}$ so that smaller $\alpha$ yields stiffer response.

##### Prediction, projections, and updates.

Each frame predicts positions with Verlet integration, then iteratively projects distance, volume, and collision constraints. For a pairwise distance constraint, the XPBD update displaces endpoints along the edge with a factor

$$ $\gamma\;=\;\frac{\alpha_{\Delta}}{\alpha_{\Delta}+w_{i}+w_{j}},\quad\Delta\mathbf{x}_{i}\propto-\gamma\,C_{ij}\,\frac{\mathbf{x}_{i}-\mathbf{x}_{j}}{\lVert\mathbf{x}_{i}-\mathbf{x}_{j}\rVert},\quad\Delta\mathbf{x}_{j}=-\frac{w_{i}}{w_{j}}\,\Delta\mathbf{x}_{i},$ (37) $$

with the analogous formulation for volume constraints using their gradients. After projections, we update positions and apply ground-plane contact with Coulomb friction.

##### Visualization.

We track the evolving surface by updating a triangular mesh whose vertices coincide with the surface particles.

### G.8 Preparing Scenes and Assigning Materials for the MPM Solver

We use a material point method (MPM) simulator mpm based on taichi-mpm TaichiMPM. Scenes are specified by a uniform Cartesian grid, a set of particles sampling each object’s volume, and per-object mechanical properties.

##### Domain, grid, and timestep.

We embed all objects in a unit cube domain discretized by an $n_{\mathrm{grid}}\times n_{\mathrm{grid}}\times n_{\mathrm{grid}}$ grid (cell size $\Delta x=1/n_{\mathrm{grid}}$). We use a fixed time step $\Delta t$ small enough for stability (e.g., $\Delta t=5\times 10^{-5}$ in our experiments).

##### Particles and initialization.

Each object is sampled with material points (particles) positioned at rest locations $\{\mathbf{X}_{p}\}$. For each particle we initialize position $\mathbf{x}_{p}\!\leftarrow\!\mathbf{X}_{p}$, velocity $\mathbf{v}_{p}\!\leftarrow\!\mathbf{0}$, affine velocity field $\mathbf{C}_{p}\!\leftarrow\!\mathbf{0}$, and deformation gradient $\mathbf{F}_{p}\!\leftarrow\!\mathbf{I}$. Mass is set as $m_{p}=\rho\,V_{p}$ with density $\rho$ and particle volume $V_{p}$ consistent with the grid resolution. In the simple drop test, we sample a sphere at a height and let gravity act.

##### Material parameters and constitutive law.

We assign per-object Young’s modulus $E$ and Poisson ratio $\nu$, compute Lamé parameters $\mu=E/(2(1+\nu))$ and $\lambda=E\nu/((1+\nu)(1-2\nu))$, and use a fast corotated (FCR) elastic model. With polar/SVD decomposition $\mathbf{F}=\mathbf{U}\,\mathrm{diag}(\bm{\sigma})\,\mathbf{V}^{\top}$ and rotation $\mathbf{R}=\mathbf{U}\mathbf{V}^{\top}$, the Kirchhoff stress is

$$ $\bm{\tau}_{\mathrm{FCR}}(\mathbf{F})\;=\;2\mu\,(\mathbf{F}-\mathbf{R})\,\mathbf{F}^{\top}\;+\;\lambda\,J\,(J-1)\,\mathbf{I},\qquad J=\det\mathbf{F}.$ (38) $$

##### Transfers and updates.

Each step performs Particle-to-Grid (P2G) transfers using quadratic B-spline weights: we scatter particle mass, momentum, and internal forces $-V_{p}\,\bm{\tau}\,\nabla w$ to grid nodes. On the grid, we (i) convert momentum to velocity, (ii) add gravity, and (iii) enforce box boundary conditions by clamping outward normal velocities to zero. We then perform Grid-to-Particle (G2P) to interpolate grid velocities back to particles, update the affine field, and integrate

$$ $\mathbf{x}_{p}\leftarrow\mathbf{x}_{p}+\Delta t\,\mathbf{v}_{p},\qquad\mathbf{F}_{p}\leftarrow(\mathbf{I}+\Delta t\,\nabla\mathbf{v})\,\mathbf{F}_{p}.$ (39) $$

##### Visualization.

We embed a coarse surface at rest, align it to the particle center of mass, and update its vertices via interpolation of nearby particle displacements.

## Appendix H Other Related Works

For completeness, we include other tangentially related works here.
A different setting from ours is inferring physical properties given additional observations, such as video davis2015visual; Mottaghi_2016_CVPR; 10.1007/3-540-47969-4_37; chen2025vid2sim; liu2024physics3d; xue20233dintphysgeneralized3dgroundedvisual; li2025freegave; 5459407; yildirim2interpreting; li2023pacnerfphysicsaugmentedcontinuum; pmlr-v119-li20j; BMVC2016_39; NIPS2015_d09bf415; NIPS2017_4c56ff4c; Xia_2024_CVPR; xu2019densephysnetlearningdensephysical; feng2024pienerfphysicsbasedinteractiveelastodynamics; lin2024phy124fastphysicsdriven4d
or physical manipulation of real objects yu2024octopiobjectpropertyreasoning; 10.1145/383259.383268; lang2003scanning; lloyd2001robotic; pai2008acme; pai2000robotics; 10160731; pinto2016curiousrobotlearningvisual.
Other related works focus on generating new physically plausible shapes, e.g. stable under gravity or other interactions,
but cannot augment existing 3D assets with mechanical properties, which is our goal lin2025omniphysgs; NEURIPS2024_d7af02c8; chen2024atlas3dphysicallyconstrainedselfsupporting; ni2024phyreconphysicallyplausibleneural; Yang_2024_CVPR; Mezghanni_2022_CVPR; Chen_2025_CVPR; cao2025physx-3d; cao2025sophylearninggeneratesimulationready.
Other methods predict displacements zhang2024adaptigraph; li2023robocook, bypassing mechanical properties, or focus on other aspects such as articulation Xia_2025_CVPR.

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 8
presentation: 4
questions: 1. Generalizability of Voxelization: The feature aggregation step requires solid voxelization, especially challenging for sparse representations like NeRFs and GSs which are fundamentally surface or density fields. Given the extensive training data is derived from high-quality segmented meshes (GVM), how reliable is the voxelization scheme for novel, noisy, or incomplete real-world captured NeRFs/GSs? Does the current method of voxelizing 3D Gaussian splats via rendering depth maps and carving space introduce artifacts or inconsistencies that affect the predicted internal properties in practice?
soundness: 3
strengths: 1.  New Dataset: The GVM pipeline combines part segmentation + PBR textures + VLM prompts constrained by a curated materials range table to annotate 37M voxels, a scale jump over earlier works that only annotated sparse points. This will be useful to the community and encourage future research in this under-explored area.
2. Volumetric Representation: Unlike previous models that focus heavily on surface properties or struggle with interior prediction (e.g., NeRF2Physics and PUGS due to feature field limitations), VoMP is designed to predict materials throughout the object volume. The system uses a unified multi-view rendering and voxelization pipeline, successfully handling meshes, SDFs, NeRFs, and 3D Gaussian Splats.
3. MatVAE: The introduction of MatVAE- a VAE used to learn a low-dimensional latent space of valid real-world material triplets- is a significant contribution. This component ensures that the decoded per-voxel materials are physically plausible, even during interpolation or sampling, a guarantee largely absent in prior methods that may output simulator-specific or non-physical parameters. The paper explicitly shows that interpolation in the latent space yields valid materials, unlike naive linear interpolation in the material space.
summary: The paper proposes VoMP, a feed-forward system that predicts volumetric mechanical property fields including Young's modulus, Poisson's ratio and density- for 3D objects across multiple 3D representations (meshes, Gaussian splats, SDFs, NeRFs). The pipeline first voxelizes the object, lifts multi-view DINO image features into the volume, and feeds per-voxel features to a transformer trained to output material latent codes. MatVAE is a VAE trained on a database of real-world materials so that the decoding process guarantees materials lie on a learned manifold of physically plausible parameters. The authors construct a Geometry with Volumetric Materials (GVM) dataset via a VLM-assisted annotation pipeline that leverages part segmentations, PBR textures, and real-material ranges. VoMP generalizes across 3D formats and produces simulation-ready fields that drive high-fidelity FEM / simplicits simulations without tuning, outperforming recent baselines both in accuracy and speed.
weaknesses: I do not have a major concern, several minor points:
1. Reliance on Approximate Input and Voxelization Resolution: The final output resolution is limited by the fixed-grid voxelization. This can lead to oversmoothing in highly heterogeneous regions and approximation errors when mapping results back to highly detailed input geometry, especially for thin structures or internal complexities. Also the authors do not provide a comprehensive study on the impact of input & output resolution.
2. Assumption of Isotropic Materials: The annotation pipeline and the resulting model fundamentally assume that materials at the part level are isotropic (properties are uniform in all directions). This is a critical limitation for common composite or natural materials, such as wood or carbon fiber, whose physical behavior depends strongly on direction (anisotropy). The model cannot currently capture this crucial complexity.
3. While the VLM is carefully guided by MTD and visual cues, the core physical property assignment for 37M voxels relies on the VLM's inference capabilities. Failure cases in similar VLM-based approaches (like Phys4DGen's dependency on MLLM consensus) highlight the brittleness of this approach. The use of a VLM introduces a reliance on an external, possibly inaccurate, source of physical knowledge, rather than deriving parameters purely from visual features and learned physics principles. I would like to see more discussions with regard to this.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 4
questions: To what extent does the method capture discontinuities at part boundaries without “bleeding” material across seams, and is there any explicit penalty encouraging jumps where appropriate?

Does the model predict uncertainties per voxel (e.g., variance over E, ν, ρ)
soundness: 3
strengths: - Representation-agnostic, feed-forward pipeline that predicts per-voxel mechanical properties in seconds, with outputs directly usable in simulators.
- Data contribution: a VLM-based pipeline that annotates ~1.6K part-segmented 3D shapes with **volumetric** materials; unlike PIXIE’s Pixelverse (surface-biased), this provides volumetric supervision.
- Extensive quantitative and qualitative results with detailed ablations, plus realistic FEM simulations.
- Clear writing and thorough explanations.
summary: VoMP predicts spatially varying volumetric mechanical properties—Young’s modulus, Poisson’s ratio, and density—from 3D assets, taking any renderable and voxelizable representation as input and outputting per-voxel fields ready for accurate simulation. It aggregates multi-view DINOv2 features into a voxel grid and uses a TRELLIS Geometry Transformer to produce per-voxel material latents. These latents are decoded by a 2D MatVAE trained on real-world triplets, which enforces physically valid material values. A data pipeline combines part segmentations, material databases, textures, and a vision–language model to create large-scale volumetric supervision and a new benchmark. Experiments show lower errors than NeRF2Physics, PUGS, Phys4DGen, and Pixie, while running in a few seconds per object and enabling realistic deformable simulations. Overall, the work delivers a fast, representation-agnostic, feed-forward approach to simulation-ready material fields via multi-view voxel features and a physically grounded latent material prior.
weaknesses: - Scalability is limited by the size and diversity of the training set (≈1.3K shapes) and the need for part-segmented annotations, making it harder to scale than methods that avoid volumetric labeling.
- Most experiments use author-curated data; because the test set follows a similar distribution, generalization to independent datasets (e.g., Objaverse) is uncertain.
- Resolution is bounded by fixed-grid voxelization, which restricts fine-detail fidelity.

---

confidence: 3
contribution: 3
initial_rating: 8
presentation: 3
questions: 1. The Geometry Transformer's architecture uses $64^3$ voxel grid, which seems a limitation. What are the bottlenecks preventing scaling this transformer up to $128^3$ or even $256^3$? Would this influence fine-grained detail prediction? It's interesting to analyze whether the performance could also scale up with respect to the resolution.
2. A remaining concern is the material annotation accuracy. Although there is external knowledge to help with the annotation, how can to guarantee the correctness for training and evaluation? Is there any small expert-annotated dataset including these three values that can be used to serve as an independent test of physical accuracy? This problem seems implied in Figure 6, where in (b), VoMP outperforms all previous approaches significantly, while in (c) ABO-500 benchmark, the improvement is not obvious. How to demonstrate that the model is not mimicking the VLM-predicted behavior but truly learning from physics.
3. The model allows internal structure prediction. How to deal with the situations of occlusion?
soundness: 3
strengths: 1. The core design is novel and effective. The problem is decoupled into two parts, which guarantee valid physical latent space learning and fast plausible prediction. The ablation study confirms the superiority of the proposed architecture
2. The feedforward nature achieves inference in seconds compared to the previous optimization approach, making it a useful tool for scalable simulation pipeline setup
3. The paper includes a comprehensive set of experiments, including strong quantitative comparisons, ablation studies that justify the design choices.
summary: This paper presents a novel feed-forward model (VoMP) to automatically predict the physical properties for 3D objects, which is an interesting and core problem for setting up the physical simulations requiring these properties. It proposes two important modules, MatVAE and Geometry Transformer, to map the interior voxel to mechanical values in a physically valid manner. The experiments demonstrate the effectiveness and speed of the proposed model.
weaknesses: 1. One remaining concern is the reliance on VLM-generated ground-truth. An expert-annotated dataset is hard to get; hence, the usage of VLMs and external knowledge is an interesting point for data construction. But how to guarantee the correctness of the VLM predicted value is also crucial, especially for real future downstream task usage. 
2. The Geometry Transformer is trained using a fixed $64^3$ voxel grid, which is rather low resolution for complex 3D assets. This seems like a major bottleneck that prevents the model from capturing fine-grained material details.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: A bit of concern around L244: Since transformers inherently support variable-length sequences, why is cropping necessary in your implementation? What specific limitation or efficiency concern motivates this design choice? The explanation that “dynamic resampling ensures the model is exposed to different parts of the asset over epochs and has a larger number of effective max voxels” seems counterintuitive. Random subsampling may lead to distribution shift during full-sequence inference, effectively making inference out-of-distribution compared to training. Moreover, if subsampling changes which spatial regions of the object are visible or centered, the geometric semantics of each cropped region could shift, potentially confusing the model. How do you ensure consistency of geometry–material correspondence under such random cropping or resampling?
soundness: 3
strengths: - Proposes a framework for predicting voxel-level material properties from arbitrary 3D representations, defining a novel and important task in 3D physical modeling.
- Introduces a MatVAE that maps material parameters into a physically valid latent space, effectively smoothing predictions and constraining outputs to realistic ranges.
- Employs a VLM to automatically construct large-scale annotated datasets, with the resulting GVM dataset being valuable for future research and other methods.
- Demonstrates higher overall quality than prior methods, with notable improvements in physical consistency and simulation fidelity.
summary: This paper proposes VoMP, an end-to-end framework that predicts voxel-wise physical property fields from a voxelized 3D representation aggregated from unprojected 2D features. The method employs a MatVAE trained on real material databases to ensure physically plausible outputs, a Geometry Transformer that transforms the input features to predict per-voxel latent codes in the material space, and uses a decoder to reconstruct continuous volumetric material fields from these latents. To train this model, the authors construct a large-scale GVM dataset with voxel-level annotations generated via VLM guidance. Experiments across various 3D inputs demonstrate generalization and physically consistent simulation results.
weaknesses: - Although interior structure plays a crucial role in physical simulation, the current voxelization approach (projecting 2D multi-view features) is not well-suited for capturing internal geometry variations, leading to limited understanding of interior geometry.
- The voxelization process may cause significant information loss in high-frequency or fine geometric regions, reducing the spatial precision of the predicted material fields.
- The transformer design uses only Swin-Attention without the hierarchical downsampling (and upsampling) scheme of a full Swin-Transformer, which restricts the receptive field and may limit the model’s ability to capture long-range structural dependencies. The VAE of TRELLIS is only for local geometry encoding and compression, and using that structure for global material property prediction is not suitable.
- The task and dataset inherently probably contain inconsistencies due to the semantic labeling process based on VLM and the task’s inherent ambiguity. A pure regression formulation may not be ideal for modeling such uncertainty, even with the help of a latent MatVAE. Would a generative or probabilistic approach better capture these ambiguities?


Decision: Accept (Poster)