H2OFlow: Grounding Human-Object Affordances with 3D Generative Models and Dense Diffused Flows
Authors: Harry Zhang
Luca Carlone
Venue: ICLR 2026 Poster (2026)
Keywords: affordance
3d vision
generative model
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Understanding how humans interact with the surrounding environment, and specifically reasoning about object interactions and affordances, is a critical challenge in computer vision, robotics, and AI. Current approaches often depend on labor-intensive, hand-labeled datasets capturing real-world or simulated human-object interaction (HOI) tasks, which are costly and time-consuming to produce. Furthermore, most existing methods for 3D affordance understanding are limited to contact-based analysis, neglecting other essential aspects of human-object interactions, such as orientation (e.g., humans might have a preferential orientation with respect certain objects, such as a TV) and spatial occupancy (e.g., humans are more likely to occupy certain regions around an object, like the front of a microwave rather than its back). To address these limitations, we introduce **H2OFlow**, a novel framework that comprehensively learns 3D HOI affordances ---encompassing contact, orientation, and spatial occupancy--- using only synthetic data generated from 3D generative models. H2OFlow employs a dense 3D-flow-based representation, learned through a dense diffusion process operating on point clouds. This learned flow enables the discovery of rich 3D affordances without the need for human annotations. Through extensive quantitative and qualitative evaluations, we demonstrate that H2OFlow generalizes effectively to real-world objects and surpasses prior methods that rely on manual annotations or mesh-based representations in modeling 3D affordance.

Published as a conference paper at ICLR 2026
H20FLOW: GROUNDING HUMAN-OBJECT AFFOR-
DANCES WITH 3D GENERATIVE MODELS AND DENSE
DIFFUSED FLOWS
Harry Zhang
Luca Carlone
MIT Cambridge, MA 02139 harryzêmi edu
MIT Cambridge, MAA 02139 1carlone@mic t edu
ABSTRACT
Understanding how humans interact with the surrounding environment, and specifically reasoning about object interactions and affordances, a critical chal- lenge in computer vision, robotics, and AI. Current approaches often depend labor-intensive, hand-labeled datasets capturing real-world or simulated human- object interaction (HOI) tasks, which costly and time-consuming produce. Furthermore, most existing methods for 3D affordance understanding are lim- ited to contact-based analysis, neglecting other essential aspects of human-object interactions, such as orientation (e.g., humans might have preferential orienta- tion with respect certain objects, such as a TV) and spatial occupancy (e.g., hu- mans are more likely to occupy certain regions around an object, like the front microwave rather than its back). address these limitations, we introduce H2OFlow, a novel framework that comprehensively learns 3D HOI affordances encompassing contact, orientation, and spatial occupancy- using only synthetic data generated from 3D generative models. H2OFlow employs dense 3D-flow- based representation, learned through dense diffusion process operating on point clouds. This learned flow enables the discovery of rich 3D affordances without the need for human annotations. Through extensive quantitative and qualitative evalu- ations, we demonstrate that H2OFlow generalizes effectively real-world objects and surpasses prior methods that rely on manual annotations or mesh-based rep- resentations in modeling 3D affordance.
INTRODUCTION
and robotics demands next-generation agents that can perceive and interact with the world seamlessly humans do. A key aspect of human intelligence is the innate ability to recognize the functionalities offered by objects and environments -allowing us to effort- lessly adapt to unstructured settings like homes. For Al agents achieve similar generalization, they must learn how to interact with objects based on their intended purpose -a a concept known as affordance. First introduced by psychologist James Gibson Gibson (2014). the concept of af- fordance has become an important topic for advancing Al and robot capabilities in our daily life. plethora of studies have been conducted on affordances for visual recognition Hou et al. (202T): );; Hong et al. (2023), action prediction Roy & Fernando (2021): Chen et al. 2023). and functionality understanding Li al. (2023a) Zhang al. al. (2023 Kim et al. (2024). Understanding affordances through the lens of human-object interactions (HOls) also offers compelling approach for teaching Al agents. By observing how humans manipulate and interact with objects, we can extract rich cues about objects' functionality, thus enabling a broader set of interactions for Al agents.
The rapid advancement
However, prior work in HOI affordance learning has largely focused on contact-based affordances, which is restrictive subset of all possible affordances. For instance, recent methods estimate contact scores from RGB images Bahl et al. (2023] 2022); Li et al. (2023a). 3D point clouds Chu et al. (2025) ; Yang al. (2023). ) human models Hassan al. (2021). relying densely annotated human-object contact labels. This manual supervision is not only aabor-intensive but also fails to generalize novel objects and broader classes of interaction.

---

Published as a conference paper at ICLR 2026
Synthetic
Contact
Orientational Affordance
Spatial Affordance
3D HOIs
Affordance
Figure 1: H2OFlow learns comprehensive affordances from synthetic 3D HOI data generated by 3D generative models using a novel representation. The learned affordance captures contact, orienta- tional, and occupancy information based on input object point clouds.
We observe that human-object interactions (HOIs) involve 3D spatial relationships beyond simple contact. For example, human faces, torsos, and arms often maintain characteristic distances and orientations relative to objects, with natural variations across interactions. For instances, humans would grasp different tools with different hand configurations: hammer typically held specific distance from its head, with the wrist angled to allow effective striking, while pen is gripped closer to the tip for finer control. complete understanding of affordances in HOIs should incorporate these geometric patterns, including relative positioning and orientational tendencies.
recent work by Kim et al., (2024) introduces the concept of comprehensive affordance, which captures these relationships probabilistically. Instead of binary contact labels, their method mod- els distribution over possible 3D spatial and orientational relations between every pair of object and human surface points. This approach generalizes affordance reasoning beyond contact, en- abling finer-grained understanding of interaction geometry. As shown in Kim et al. (2024), learning comprehensive affordances in HOIs typically relies synthetic RGB images uplitted to 3D using DD-to-3D techniques. However, this approach requires intricate masking methods to achieve high- quality results, introducing multiple potential failure modes. Furthermore, the learned affordances often fail to generalize to novel real-world objects, and the dependency on well-defined watertight meshes for better-quality affordance computation severely limits real-world applicability.
To address these challenges, we leverage recent advances in 3D generative models for HOIs Li et al. (2024a); Diller Dai (2024); |Png et al. (2023). Our key innovation is a pipeline that directly generates plausible 3D HOI samples using generative models, eliminating the need for error-prone novel geometries,
subsample points from the generated data and employ dense diffused flows Eisner et al. (202)) -a technique proven effective for modeling multi-modality-a to reconstruct 3D humans from the HOI samples. For comprehensive affordance learning, we introduce novel probabilistic formulation operating directly on human- object point cloud pairs, circumventing the need for watertight mesh.
2D-to-3D uplifting. To ensure generalization
This culminates in Human-Obin Object Flow (H2OFlow), a framework for learning rich affordance knowledge HOIs (Fig. D. Our key contributions are:
1. A point-cloud-based affordance representation that efficiently captures both explicit contact and implicit non-contact interaction patterns in HOIs from raw point clouds inputs.
2. A synthetic data generation and learning pipeline, which leverages 3D generative models and dense diffused flows, that learns flexible affordances from synthetic 3D point clouds.
3. Extensive quantitative and qualitative experiments demonstrating the effectiveness and practical utility of the learned affordances on both synthetic datasets and real-world data.
RELATED WORK
Affordance Learning. First introduced in Gibson 2014). affordance learning has emerged as a crit-
ical capability for Al and robotic systems. Modern approaches focus enhancing agents' ability for better visual recognition/Hov Hou et al. (202T); Hong et al. (2033), action prediction| Roy & Fernando
2021 Chen et al. 2023), functionality understanding Li et al. (202a)); Eisner et al. (2022) Kim
2

---

Published as a conference paper at ICLR 2026
mimicking scene- conditoond human-object and hand-object interactions Bhhtnaggr Lu et al. ; Nguyen et al. (2024); :; Jiang et al. (2022): Huang et al. (2022) : Diang Petrov al. (2023): Hassan al. ) Zhang et al. (202)) With the adnenc of L.Ms, more works have been proposed to explore open-vocabulary affordances in point clouds Nguyen et al.] (2024); Chu et al. (025). However, most works foeus exclusively on contact-based affordances, neglecting crucial spatial and orientational aspects of interactions. Moreover, the re- quirement of manual labeling of the contact regions Do al. (2018); [Jia et al. (2023); (Tripahh et al. (203)) Delitzas et al. (2024); Yang et al. (023) is deemed cumbersome and restrictive when generalizing to the real world. More recently, Kim al. (2024) proposed a comprehensive set of affordance representations that captures both contact and non-contact knowledge in HOIs without manual labels. While such comprehensive affordances work well capturing both contact and spatial relations, they require to calculate the normal directions of each vertex and the inferred af- fordances have limited generalization to novel objects. We instead propose a novel set of affordance representations that operates on (partially observed) point cloud data, bypassing the need of water- tight meshes, and generalizes to unseen objects via learned dense diffused flows.
et al.
(2024).
et al. 2022;;
et a.. (2022)
3D Flows in Visual Learning. 3D flows have emerged powerful representation in visual learn- ing, playing a key role in both policy learning Hu et al. (2017); Bahl et al. (2022) 2023) and object understanding Bisner al.] (2022); Xu et al. (2024); Cai al. (2024). By capturing how points in 3D space move over time, 3D flows inherently encode affordances under external forces. For instance. predicting flow on articulated objects reveals how individual parts might move when inter- acted with by a human. While prior work has largely focused on learning 3D flows for rigid objects, we extend this intuition to the human body. Specifically, we propose learn 3D flows that predict how each point on the human body moves when interacting with an object. Given the multi-modke and highly deformable nature of human-object interactions (HOIs), we leverage diffusion models Ho et al. (2020); Roobaach et al. 2022);; Ramesh et al.] (2022) ; Nakayama et al. (2023); PPeblee & Xiel (2023) to learn these flows in a dense and expressive manner. We refer to this representation as dense diffused flows. As we show later, dense diffused flows generalize well to unseen objects and we are able to infer comprehensive affordance knowledge using such a representation.
HOI Data Synthesis in 3D. With the growing availability of paired scene-motion datasets Araújo ct al. (203); Hassan et al. (2019); Wang et al. (2022b); Zheng et al. (2022), a range of methods has been developed to synthesize human interactions in 3D environments Brahmbhatt et al. (2019a bj
2020 : Araújo et al. (2023); Hassan et al.) (202T); Wang et al.] (2022)): Taheri et al.] a (2020); Zhou
al. 2022); Ye et al. (2023). Another line of research leverages reinforcement learning to train scene-aware policies that generate navigation and interaction motions in static 3D scenes Xiao et al.
(2023); Lee Joo (0023). More recently, with the rise of large language models and the avail- ability of paired human-object motion data [Bhatnagar et al. (2022); Li ct al. (2023b), several works have demonstrated the ability to predict human- object interactions (HOls) from sparse waypoints or textual descriptions Li et al.] (024a);; Diller & Dail (224); |Png et al. (2023), enabling direct generation 3D HOI data from language. In H2OFlow, we leverage the pre-trained model from Li et al. (2024a) to synthesize a diverse set of HOI sequences from text. These sequences are rich in affordance cues that go beyond mere contact information. We then subsample vertices from the resulting human-objec meshes to generate point clouds for downstream learning of dense diffused flows. At inference time, our model requires only a partially observed object point cloud to infer affordances.
PROBLEM FORMULATION
We address the problem of learning comprehensive human-object interactions (HOIs) from point cloud data. Given a human point cloud H = { Nor I N and an object point cloud O {o, Na E Nox3, our goal is to infer novel affordance representation that captures three key aspects of interaction: contact, orientation, and spatial configuration. Figure provides an overview of H2OFlow.
We define an affordance score for each pair of human-object points (i. The contact affordance, denoted as Cj R. reflects the likelihood of contact between human point h; and object point oj, with higher values indicating actual contact. The orientational affordance. denoted as R,g- E R, captures the characteristic orientation patterns of human body parts relative to the object (e.g., the
3

---

Published as a conference paper at ICLR 2026
forearms' rotation relative to the table top is more uniform than the feet's in Figure 2) A higher Riy value indicates a consistent and meaningful orientation pattern observed during interactions. The spatial affordance, denoted as Siy E #xxx*X, over a voxel grid of size H x W x L, characterizes the spatial occupancy of human body parts around the object, assigning higher scores to regions frequently occupied during interactions in 3D space (e.g., the orange region in the spatial affordance of Figure tendd to get occupied by human more than the purple region).
Ho (
Connect
Orierraianal
DIT
Ho
Sprial
Otject
Training
Object
Sampling
Inferring
Figure 2: H2OFlow overview. We generate synthetic 3D HOI mesh samples, process the meshes into point cloud and train DiT learn dense diffused flow distribution for human goal configuration prediction. Upon seeing unseen object, H2OFlow samples learned dense flows to reconstruct goal humans. Using the flows and point clouds, we are able infer comprehensive affordances. Note the "Object" affordance here is the transpose of the human contact affordance matrix.
METHOD
To learn affordance knowledge from point clouds in generalizable manner, propose H2OFlow, a framework that first synthesizes diverse human-object interaction (HOI) samples using a pre- trained 3D generative model the training data. Then, train a diffiusion model that takes as input an object point cloud and predicts human interactions in the form of dense diffused flows Xu al. (2024), probabilistic representation that predicts per-point displacement on the human point cloud conditioned on the HOI. During inference, these flows are then used to comprehensively infer HOI affordances -contact, orientation, and spatial directly from the object point cloud.
4.1 TRAINING DATA: SYNTHETIC HOI SAMPLES GENERATION
We employ pretrained 3D generative model to generate diverse and realistic HOI mesh sequences. Given an initial object- humenn configuration and language prompt, the pre-trained generative model generates temporally synchronized object and human motions. The outputs are long mesh sequences comprising varied and rich interaction dynamics across different object categories. Please refer to the Training Data illustration of Figure for examples.
One might ask: why can't we directly learn affordances from generative model outputs? There are two main problems that hinder the generalizability of directly inferring affordances from synthetic HOIs Kim et al. (2024). First, such generative models are trained with object meshes, while in- puts from raw-sensor data contain noisy point clouds, making such approach incompatible with real-world data. Second, it is costly to generate and analyze 3D HOI meshes, creating a large com- putational and memory bottleneck. Thus, it is imperative for to find way that generalizes well to unseen point clouds for practicality, while maintaining lower computational cost. We make use of dense diffused flow, representation that lends itself well point cloud learning.
4.2
AN INTERMEDIATE REPRESENTATION: DENSE FLOWS
To better generalize to unseen objects, H2OFlow reconstructs plausible human configurations from given object point cloud O using intermediate, point-based representation, dense flows /hang
4

---

Published as a conference paper at ICLR 2026
et al. (2023); Xu et al. )2024); Cai et al. (2024), which can be applied to both rigid and deformable objects. Dense flows represent how each point transitions from its initial to its target configuration.
We assume the initial human pose is given by standard O-pose (T-pose) SMPL mest From this mesh, we sample NH points (1), π (2).... T [NN to construct the initial human point cloud Ho. where π(-) denotes the sampling operator. To obtain the goal human configuration from synthesized HOI mesh, we sample the same NH points to create the goal human point cloud H, ensuring one-to-one correspondence.
Using this setup, compute the dense flow field F (f the per-point displacement between the goal and initial configurations of the human:
f: := h; ho.s:
Vi (H....,NH):
which can be compactly written F := : - Ho. As illustrated in the Learned Dense Diffused
Flows in Figure we can sample F conditioned on the object point cloud to reconstruct diverse goal human point clouds. We provide more detailed dense flows visualization in Figure of Section C
6 In summary, given a generic O-pose human point cloud Ho and an input object point cloud O
H2OFlow predicts dense flow field that displaces He into realistic interaction configuration H,
effectively modeling the human-object interaction through spatial deformation
4.3 LEARNING THE DENSE FLOWS REPRESENTATION
Human-object interactions (HOIs) in both real-world scenarios and synthesized samples exhibit strong multimodality. For instance, human may contact an object using either the left or right hand, or interact with different regions of the same object. This diversity highlights the need for distributional representation of dense flow that captures continuous spectrum of plausible human configurations, rather than a single deterministic outcome.
To this end, we aim to learn a distribution over dense flows conditioned on an object point cloud: gHo = Pa (F O), where Ho is the initial human point cloud and F is the dense flow field. At inference time, we can sample plausible dense flow F Po( F O) and reconstruct goal human configuration via
Hu F.
To effectively model this complex distribution, adopt diffusion models Ho al. 2020 Sohl- Dickstein et al. (2015); Peebles & Xie (203)) which learn data distributions through iterative for- ward noising and reverse denoising processes. By applying this framework to dense flow prediction, we introduce the concept of dense diffused flow, enabling our model to generate diverse and plausible human poses in interaction with given object.
Diffusion Process. Given a synthetic HOI sample point cloud pair (H,O) and a canonical O-pose human point cloud Ho. we train a diffusion model to learn the distribution over dense flows F. The ground-truth dense flow is defined as the per-point displacement between the goal and initial configurations:
Fcr H Ho.
(2)
Following standard diffusion modeling practices Ho et al. (2020); Song al. (2020), we construct noisy version of the clean dense flow Fo Fer by sampling at time step t (1.....Ty
F, ===vo Vo + vI âGE, where E ~ NO, I) is Gaussian noise, and ā; is the cumulative product of noise scheduling parameters 3, The forward process adds Gaussian noise progressively over time steps, while the reverse process learns to denoise and recover the original Fn. We parameterize the reverse process as: (F F.) NNFF (F.),. Eg (F))), and supervise the model using the hybrid loss from Nichol & Dhariwal (2021) that combines the noise loss with a new cumulative KL- loss using the derived Eg (F.).
During inference, given an object point cloud O and a generic O-pose human point cloud Ho- we initialize the dense diffused flows as Gaussian noise: Fr NN(0, I). The dense diffused flows are iteratively denoised via the reverse process. The final denoised flows Fo are then used to transform the points of H into a predicted interaction configuration H o + Fu with respect the object O.
Please refer to Appendix Section for details on placing the O-pose human relative to the object. ? Dense flows representation is the fundamental reason for H2OFlow's generalizability, and we discuss this design and advantages over prior works in more details in Appendix Section G
5

---

Published as a conference paper at ICLR 2026
Dense Diffused Flows from Diffusion Transformer. Diffusion Transformers (DiT) Peebles & Xie (023) have demonstrated strong capability in modeling multi-modal point cloud distributions for deformable objects Cai et al. (2024). We adopt DiT the backbone for predicting dense diffused flows. At each diffusion timestep. the model takes as input the noised flow Ft, the human point cloud H, the object point cloud O, and the timestep t. Using MIP encoders with shared weights, we extract per-point features from each input: dense flow features f from Fr, human features fH from H, and object features fo from O. The dense flow and human features are concatenated to form joint features fPH, which serve as the input to the DiT model, conditioned on the object features fo Within each DiT block, self-attention first applied to the joint human-flow features IFH to enable local reasoning across the human point cloud and coordinate flow predictions. Then, cross-attention applied between fFH and the object features f to capture global human-object interaction patterns. This process is repeated across N DiT blocks, after which the network outputs the predicted noise Eg and the interpolation vector Ug. We explain the training objective (hybrid loss) and details in Section
4.4 TEST TIME: COMPREHENSIVE AFFORDANCE INFERENCE
During inference, with the learned diffusion model, we can sample flows conditioned on an object point cloud, resulting in a distribution over possible human goal configurations. Given an initial human point cloud Hu and sampled flow po (F O). the goal human given by H Ho + F and each point of the sampled predicted human h; ha, + f. For each predicted human point le; we define a conditional probability distribution with respect to each object point o,:
Pi == p(h; oj))
Thus, Pij defines the possible human points locations in diverse HOI samples. In practice, this distribution defined over large set of generated HOI samples. Our three affordance types contact, orientational, and spatial- are then defined over this pairwise distribution Pij. resulting in per-point-pair evaluation of affordance.
Contact Affordance. We define the contact affordance score Cij between human point h, and object point 0j as:
exp (-1 [db))))))
C33 = EEENP,
(4)
where diy h; 0j denotes the per-pair displacement between human point and object point, Wij denotes the cross-attention weight between h; and oj from the DiT model, and т is a temperature hyperparameter that controls sensitivity distance.
Intuitively, the contact affordance score Cij is higher when the human and object points are likely to be spatially close during HOI. The inclusion of the cross-attention weight Way further enhances contact prediction by leveraging semantic alignment from the DiT model, especially in cases where contact not perfectly captured in the sampled HOI configurations.
Orientational Affordance. Following the intuition from prior work Kim et al. 2024 we aim to capture the consistency and pattern of human body part orientations relative to object geometry using an entropy-based formulation. The key idea is that a lower entropy in the orientation distri- bution implies stronger, more consistent orientational pattern during interaction, indicating a high orientational affordance. However, unlike Kim et al. (2024), which computes surface normals to measure orientation-ols often computationally expensive and unstable under noisy meshes-we lever- age the predicted dense diffused flows directly as proxy for directional motion. Specifically, for each human-object point pair (i, we compute a relative orientation vector using the cross product between the displacement vector d:y (from human point h; object point o,) and the diffused flow vector f.:
Ty = di x f
(5)
ddy x flll
The cross pross-product wyy, intuitively, represents the relative displacement direction between the human and the object given the human dense flow direction, efficiently grounding the per-pair information on the overall flow direction. To evaluate the distribution of these orientation vectors, discretize

---

Published as a conference paper at ICLR 2026
Dease Diffused Flow J,
Object Elamen Displacemment d.,
Cross-Product Zy
Orientational
S²
Spaiial
Figure 3: Visual illustration of affordance inference. Given predicted human point clouds, contact affordance assigns high scores to human-object point pairs that are close. Orientational affordances give higher scores to point pairs that yield more uniform cross-product directions (i.e., hand points) and vice versa (i.e., foot points). The spatial affordances output higher scores to regions surrounding the object that often occupied by human parts. A video of the figure is available at this website
the unit sphere S into ni bins with representative directions {n1,-- J. The discrete probabil- ity of æyy falling into bin n is computed using a Gaussian kernel:
| nn 1
Pn,ai (n) x exp
n1
nb:
(6)
20
where is hyperparameter. This defines distribution over orientation bins on the sphere. We then compute the negated Shannon entropy of this distribution:
Hij Eg [log Pa,ii(n))
which becomes higher when orientations concentrate around specific directions.
Finally, we define the orientational affordance score Rij as the expectation of this negated entropy over the distribution of possible human configurations:
Hij
Rig
Eni-Pis
Wij
where thy is the cross-attention weight from the DiT model and is temperature hyperparameter.
Since a uniform distribution has high entropy, while struetured behavior has low entropy, a low Ry indicates that the orientation distribution Pa (n) is nearly uniformly random -i.e. no domi- nant pattern exists- whereas a high Riş reflects consistent and structured orientational behavior in human-object interaction
Spatial Affordance. Lastly, we aim to capture the 3D spatial occupancy pattern of human surface points with respect to object geometry, following ideas from prior work Han & Joo| (2023): : Kim al. (2024). This affordance measures the likelihood that a specific region space is occupied by a part of the human body during interaction with the object.
We define voxel grid G H covering the spatial region around the object. For each voxel E G. we introduce an indicator function ō, that equals if the voxel g contains the human point h, and otherwise. The spatial affordance score is then defined the expected occupancy of voxel g by point h; conditioned on the interaction with object point Oy:
Siz En,-P., [dag]
This formulation results in a discrete occupancy map over the voxel grid, which can be further analyzed as a spatial probability distribution. Learning spatial affordance helps us understand the typical spatial arrangement or positioning of the human body relative to the object during interaction.
We propose advanced cases of orientational affordance in Section N.2

---

Published as a conference paper at ICLR 2026
Baseline
SIM-H f
SIM-O
MAE-H V
MAE-O
Precksio@@K
MSE.
COMA
2.2%
= 1.49%
0.22 = 0.07
0.14 0.03
42.9 7.2%
0.14
COMA-Res Recon
2..1%%
31.8 1 1.9%
0.62 0.11
0.51 1 0.05
9.112.45
0.66 1 0.12
H20SMPL
01.3::220 2.1%
68.0
+ 3.3%
0.21 0000 0.03
0.15 = 0.03
53.6 = 1.9%
0.04
H20Flow-NoAtm
67.3 = 1.6%
76.4 + 2.4%
0.15 = 0.02
0.09 = 0.01
69.2 1.1%
0.12
H2OFlow
72.3
81.0 2.4%
0.11 0.03
0.07 0.01
75.6 3.1%
0.12 0.01
Table 1: Quantitative comparisons with various baselines on OMOMO dataset. Note that -H and -O represent human and object contact results.
In practice, this representation avoids reliance on high-quality surface meshes and is highly efficient: operations are parallelizable GPUs, and memory usage is minimized by sampling only small subset of points from both the human and object point clouds.
EXPERIMENTS
We present both quantitative and qualitative results to evaluate H2OFlow. We use pretrained CHOIS Li et al. (2024a) as the 3D generative model backbone to generate diverse HOIs. During training. apply random perturbation and occlusion to the objects point cloud to achieve real- world robustness. We compare against baseline methods in terms of affordance learning quality, memory efficiency, and runtime performance. For the qualitative evaluation, demonstrate how H2OFlow surpasses traditional contact-based affordances via distributions over orientational and spatial information across diverse range of object categories.
5.1 QUANTITATIVE RESULTS
Baselines. We compare H2OFlow against COMA Kim et al. (2024) using objects from the OMOMO test set Li et (203b)). Since COMA requires 2D object images to generate inpainted HOI samples, we render each OMOMO object from 50 camera views. To ensure a fair comparison, we also reconstruct object meshes from H2OFlow's 's point cloud inputs and render them from the same views as input to COMA this serves as the COMA-Reco baseline. We include a variant of our method, H2OSMPL, where we learn a direct SMPL predictor using diffusion conditioned on the object input. Additionally, we include variant of our method, 120Flow-NoAAue which moves the cross- cross-attention mechanism used for aggregating affordance scores. All methods generate 50 HOI samples per object for evaluation.
Metrics. For contact affordance, we compute the similarity (SIM) Swain & Ballard (199I) and mean absolute error (MAE) between the normalized predicted and ground-trun truth contact distribu- tions. For orientational affordance, we rank human vertices by the average entropy of their relative orientations in ground-truth HOIs, and compare these rankings based the predicted orienta- tional scores. We report Precision @ K by measuring the overlap between the top-K ranked sets. For spatial affordance. we calculate the mean squared error (MSE) between predicted and ground- truth voxel occupancy grids.
Results. As seen in Table T for all metrics, H2OFlow outperforms other baselines by very notice- able margin. We note that COMA's performance breaks when the input 2D rendered mesh images are reconstructed from point clouds. Moreover, learning dense diffused flows results in better per- formance than learning SMPL parameters directly. We analyze why this is the case in Appendix Section H2OFlow performs better with attention weights in contact and orientational affordances aggregation. We provide results on the BEHAVE dataset and contact-only baselines in Appendix Section and Section [ Note that COMA, to our knowledge, is the only prior work addressing different types of affordances. We test H2OFlow's robustness against occlusion in Appendix Sec-
tion
Memory and Runtime Comparisons. Experiments suggest that H2OFlow utilizes significantly less memory and runs faster than COMA. This expected in that H2OFlow operates on sparse point clouds. We document quantitative comparisons in Appendix Section
8

---

Published as a conference paper at ICLR 2026
Store
6.2
Objects
Predicted HOI
Object Contacts
Orientational
Sparial
Figure 4:
Visualizations of affordances inferred from flows prediction with color maps. H2OFlow infers diverse affordance distributions from predicted HOI samples unseen objects.
5.2 QUALITATIVE RESULTS
Learned Affordances Visualizations. We showcase sample inferred affordances in Figure The qualitative results are also run on the test objects of the OMOMO dataset, unseen during the training of H20Flow. For each object, we pick two points on the object that give us interesting interaction information. Both contact affordance and orientation affordance reflect diverse, multimodal distri- butions from the predicted HOI samples. Depending on the points on the object, human contact affordances reflect different heatmaps, and different parts of the human also exhibit different orien- tational tendencies. Figure 8 the selected bottom (orange) point makes more contact with the side of the body while the center (blue) point makes frequent contact with the whole torso, which reflects real-world contact tendency when moving monitor. For the tripod in Figure human legs tend to exhibit more uni- form orientation relative to the bottom of the tripod (orange) than their hands, while the orientation of the hands relative the top part of the tripod (blue) is uniform. For spatial affordance, can see the high-level human occupancy around the object during HOIs: the more frequently occupied by the human body parts, consistent with real-world interactions (in some cases, full human silhouettes are observed).
Figure we provide more examples. Specifically, in the monitor example of
regions
With Wapi
Object
PoiRe Cau
Cantert
DoFiox Ontretatlen
(a) Atten- tion usage.
(b) Real point clouds results.
Figure 5: (a) Ablations cross-attention weights and (b) results real-world point clouds. Objects shown are: monitor, trashcan, backpack handle & panel, chair, yoga ball, table, box, and suitcase.
Cross-Attention Weights Ablation. We ablate the effect of incorporating cross-attention weights into the computation of affordance scores, as shown in Figure 5a With cross-attention weights, both the contact and orientational affordances exhibit greater symmetry compared to the variant without attention. This is particularly valuable in low-sample scenarios, where sampling only few instances from the diffusion model may result in limited diversity and fail to fully capture the underlying multi- modal distribution. Cross-attention weights mitigate this issue by learned geometric associations between human and object. Even when the sampled outputs are sparse, the attention weights act as corrective signal, producing plausible and semantically aligned affordance estimations.

---

Published as a conference paper at ICLR 2026
Affordance Types Ablation. We are interested in studying the contribution of orientational and spatial affordances beyond contact. We design two downstream HOI inference tasks and vary only which affordance terms are computed and used for downstream scoring. We evaluate downstream HOI inference tasks on unseen objects and test on the following variants of H2OFlow affordances output: (1) C: use Cjj only, (2) C+ O: cij and Rej. (3) C+SS: use Cij and Siy. (4) Shuffled: keep Cij but randomly permute Ray across human indices or Siy across voxels per object, and (5) C+O+S: use all three affordances (H2OFlow default). To combine terms for downstream scoring we use a normalized linear fusion dhg = A + Ao Rej + A₂ S.j. We design two downstream HOI inference tasks: (1) HOI Region Retrieval: Given an object query point 0j. rank human points by Øizi compute mAP@ {1,5,10} against GT contact points, and (2) Pose Selection: Given sampled HOI hypotheses per object, select arg max; Σig P1 Report Top-k accuracy GT pose clusters, collision rate with object, and contact leakage that measures contacts on implausible parts.
(b)
Variant
mAP@ 11
mAP@51
mAP@ 10
Top-5 Acc.1
Collision
Leakage|
C
0.52 + 0.03
0.60 + 0.02
0.66 + 0.02
0,41 + 0.04
0.31 + 0.02
0.27 + 0.03
C+0
0.57 + 0.03
0.63 0.02
0.69 + 0.02
0.55 0.03
0.26 + 0.02
0.22 + 0.02
C+S
0,56 + 0.03
0.62
0.68 + 0.02
0.51 + 0.04
0.21
0.20 + 0.02
C+0+S
0.63 + 0.02
0.76 + 0.03
0.64 + 0.03
0.01
0.17 + 0.01
Shuffled
0.54 + 0.03
0.61 0.02
0.66 + 0.02
0.44 + 0.03
0.29 0.02
0.26 + 0.02
Table 2: Downstream HOI inference results. Left: Region Retrieval (mAP@ (0,0@(11,,,00 (1,5,10);; Right: Pose Selection (Top-5 accuracy, collision rate 4, and contact leakage L).
We record the results in Table 2] As results suggest, against C, C+O and C-S significantly im- prove the metrics, and C+O+S yields further gains. Shuffled controls eliminate these improve- ments, confirming that structured orientational and spatial affordances indeed improve performance for affordance learning, not merely because of additional feature capacity.
Unseen Real-World Objects. We evaluate H2OFlow on real-world point clouds captured using a cheap depth camera on an iPhone, collected by the RealityKit and subsampled via FPS Qi et al. (017), in Figure [b) Due to training-time perturbation and occlusion, H2OFlow does not require full object scan and is robust to real-world occlusions (e.g. bottom). H20Flow produces highly plau- sible affordance scores these real inputs, effectively capturing meaningful interaction patterss -particularly the orientational tendencies around the head region. For example, we observe that clean, multi-modal affordances are inferred in the backpack examples (different parts). While the objects were unseen during training, H2OFlow learns local geometric cues via dense diffused flows instead of memorizing global mesh templates. Thus, the output affordances are semantically mean- ingful and consistent with the actual usage of the interacted parts on the objects. In contrast, as the full comparison shows in Figure| 7 COMA relies on 2D renderings from clean meshes, and thus struggles with noisy, reconstructed meshes derived from point clouds. This limitation severely degrades COMA's performance, resulting in oversimplified and unimodal affordance score maps.
CONCLUSION
We introduced H2OFlow, novel framework for learning comprehensive 31D affordances from syn- thetic data using dense diffused flows. H2OFlow demonstrates strong generalization to unseen ob- jects and is capable of capturing diverse contact, orientational, and spatial relationships underlying human-object interactions. Looking forward, we aim to extend this framework to support more fine- grained interaction tasks and downstream applications such as robot policy learning. In particular, incorporating more diverse interaction data and exploring robot-human affordance correspondence will be key directions for future research.
ETHICS STATEMENT
We take ethics very seriously and research conforms to the ICLR Code of Ethics. Affordance learning is a well-established research area, and this paper inherits all the impacts of the research area, including potential for dual use of the technology in both civilian and military applications.
10

---

Published as a conference paper at ICLR 2026
We believe that the work does not impose high risk for misuse. Furthermore, the paper does not involve crowdsoureing or research with human subjects.
REPRODUCIBILITY STATEMENT
Our paper makes use of publicly available open-source datasets, ensuring that the data required for reproducing our results is accessible to all rescarchers. We have thoroughly documented all aspects of our model's training. including the architecture, hyperparameters, optimizer settings, learning rate schedules, and any other implementation details for achieving the reported results. Additionally, specify the hardware and software configurations used for our experiments to facilitate replication. We anticipate that it should not be challenging for other researchers to reproduce the results and findings presented in this paper.
ACKNOWLEDGEMENTS
This work was partially funded by ONR RAPID Program and by Carlone's NSF CAREER Award.
REFERENCES
Joao Pedro Araūjo, Jiaman Li, Karthik Vetrivel, Rishi Agarwal, Jiajun Wu, Deepak Gopinath, Alexan- der William Clegg, and Karen Liu. Circle: Capture in rich contextual environments. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pp. 21211-21221 2023.
Shikhar Bahl, Abhinav Gupta, and Deepak Pathak.
Human-to-robot imitation in the wild. arXiv preprint
ar arXhv:2207.09550 2022.
Shikhar Bahl, Russell Mendonca, Lili Chen, Unnat Jain, and Deepak Pathak. Affordances from human videos as versatile representation for robotics. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 13778 13790, 2023.
Bharat Lal Bhatnagar, Xianghui Xie, Ilya A Petrov, Cristian Sminchisescu, Christian Thcobalt, and Gerard Pons-Moll. Behave: Dataset and method for tracking human object interactions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15935-15946, 2022.
Samarth Brahmbhatt, Cusuh Ham, Charles C Kemp, and James Hays. Contactdb:
Analyzing and predicting grasp contact via thermal imaging. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 1709-8719. 2019a.
Samarth Brahmbhatt, Ankur Handa, James Hays, and Dieter Fox. Contactgrasp: Funetional multi- multi--inger grasp synthesis from contact. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2386 2393. JEEE, 2019b.
Samarth Brahmbhatt, Chengeheng Tang, Christopher D Twigg, Charles C Kemp, and James Hays. Contactpose: dataset of grasps with object contact and hand pose. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XIII 16, pp. 361-378. Springer, 2020.
Eric Cai. Octavian Donea, Ben Eisner, and David Held. Non-rigid relative placement through 3d dense diffuu- sion. arXiv preprint urXiv:2410.192477 2024.
Joya Chen, Difei Gao, Kevin Qinghong Lin, and Mike Zheng Shou. Affordance grounding from demonstra- tion video to target image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6799-6808, 2023.
Hengshuo Chu, Xiang Deng, Xiaoyang Chen, Yinchuan Li, Jianye Hao, and Liqiang Nie. 3d-affordancellm: Harnessing large language models for open-vocabulary affordance detection in 3d worlds. arXiv preprint arXiv:2502.2004 2025.
Alpár Cseke, Shashank Tripathi, Sai Kumar Dwivedi, Arjun Lakshmipathy, Agniv Chatterjee, Michael J Black, and Dimitrios Tzionas. Pico: Reconstructing 3d people in contact with objects. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 1783-1799 1794, 2025.
Alexandros Delitzas, Ayca Takmaz, Federico Tombari, Robert Sumner, Mare Pollefeys, and Francis Engel- mann. Scenefun3d: fine-grained functionality and affordance understanding in 3d scenes. In Proceedings of the IEFF/CVF Conference Computer Vision and Pattern Recognition, pp. 14531-14542, 2024.

---

Published as a conference paper at ICLR 2026
Christian Diller and Angela Dai. Cg-hoi: Contact-guided 3d human-object interaction generation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19888-199011 2024,
Thanh-Toan Do, Anh Nguyen, and Ian Reid. Affordancenet: An end-to-end deep learning approach for object affordance detection. In 2018 TEFF international conference on rohotics and automation (ICRA), pp. 5882- 5889. IEEE, 2018.
Ben Eisner, Harry Zhang, and David Held. Flowbot3d: Learning 3d articulation flow to manipulate articulated objects, arXiv preprint arXiv:2205.044: 043322 2022.
James J Gibson. The theory of affordances:(1979) In The people, place, and space reader, pp. 56-60. Rout- ledge, 2014.
Sookwan Han and Hanbyul Joo. Chorus: Learning canonicalized 3d numan-obbec spatial relations from un- bounded synthesized images. In Proceedings of the IEEE/CVF International Conference Computer Vi- sion, pp. 15835-15846, 2023.
Mohamed Hassan, Vasieeios Choutas, Dimitrios Tzionas, and Michael J Black. Resolving 3d human pose am- biguities with 3d scene constraints, In Proceedings of the IEEE/CVF international conference on computer vision, pp. 2282-2292, 2019.
Mohamed Hassan, Partha Ghosh, Joachim Tesch, Dimitrios Tzionas, and Michael Black. Populating 3d scenes by learning human-scene interaction. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 14708-14718. 2021.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12590 2022.
Jonathan Ho, Ajay Jain, and Pieter Ahheel. Denoising diffusion probabilistic models. Conf. on Neural Infor- mation Processing Systems (NeurIPS), 33:6840-6851, 2020.
Yining Hong, Haoyu Zhen, Peihao Chen, Shuhong Zheng, Yilun Du Zhenfang Chen, and Chuang Gan. 3d-llm: Injecting the 3d world into large language models. Advances in Neural Information Processing Systems, 36: 20482 20494, 2023.
7hi Hou, Baosheng Yu, Yu Qiao, Xiaojiang Peng, and Dacheng Tao. Affordance transfer learning for human- object interaction detection. In Proceedings of the IEFF/CVF conference on computer vision and pattern recognition, pp. 495-504, 2021.
Ruizhen Hu, Wenchao Li, Oliver Van Kaick, Ariel Shamir, Hao Zhang, and Hui Huang. Learning to predict part mobility from single static snapshot. ACM Transactions On Graphics (TOG), 36(6): 13, 2017.
Chun-Hao P Huang, Hongwei Yi, Markus Höschle, Matvey Safroshkin, Tsvetelina Alexiadis, Senya Po- likovsky. Daniel Scharstein, and Michael J Black. Capturing and inferting dense full-body human-scene contact. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
13274-13285, 2022.
Juntao Jian, Xiuping Liu, Manyi Li, Ruizhen Hu, and Jian Liu. Affordpose: large-scale dataset of hand-ohject interactions with affordance-driven hand pose. In Proceedings of the IEFF/CVF International Conference on Computer Vision pp. 14713-14724, 2023.
Yuheng Jiang, Suyi Jiang, Guoxing Sun, Zhuo Su, Kaiwen Guo, Minye Wu, Jingyi Yu, and Lan Xu. Neural- hofusion: Neural volumetric rendering under human-object interactions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6155-6165, 2022.
Hyeonwoo Kim, Sookwan Han, Patrick Kwon, and Hanbyul Joo. Beyond the contact: Discovering comprehen- sive affordance for 3d objects from pre-trained 2d diffusion models. In European Conference Computer Vision, pp. 400-419. Springer. 2024.
Jiye Lee and Hanbyul Joo. Locomotion action- manipulation: Synthesizing human-scent scene interactions com- plex 3d environments. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9663-9674, 2023.
Gen Li, Varun Jampani, Deqing Sun, and Laura Sevilla-Lara. Locate: Localize and transfer object parts for weakly supervised affordance grounding. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 10922-10931, 2023a.
Jiaman Li, Jinjun Wu, and C Karen Liu. Object motion guided human motion synthesis. ACM Transactions on Graphics (TOG), 42(6):1-11 2023b.
12

---

Published as a conference paper at ICLR 2026
Jiaman Li, Alexander Clegg, Roozbeh Mottaghi, Jiajun Wu, Xavier Puig, and C Karen Liu. Controllable human-object interaction synthesis. In European Conference on Computer Vision, pp. 54-72. Springer, 2024a.
Lei Li and Angela Dai. Hoi-page: sero-shot human-object interaction generation with part affordance guid- ance, arXiv preprini arXiv:2500.07209 2025.
Yishu Li, Wen Hui Leng. Yiming Fang. Ben Eisner, and David Held. Flowbothd: History-ave aware diffuser handling ambiguities in articulated objects manipulation. arXiv preprint arXiv24410.07078, 2024b.
Liangsheng Lu, Wei Zhai, Hongchen Luo, Yu Kang, and Yang Cao. Phrase-based affordance detection via cyclic bilateral interaction. IEEE Transactions on Artificial Intelligence, 4(5):1 1198, 2022.
George Kiyohiro Nakayama, Mikacla Angelina Uy Jiahui Huang, Shi-Min Hu, Ke Li, and Lconidas Guibas. Dififacto: Controllable part-based 3d point cloud generation with cross diffusion. In Proceedings of the IEEE/CVE International Conference on Computer Vision. pp. 14257-14267, 2023.
Toan Nguyen, Minh Nhat Vu, Baoru Huang, Tuan Van Vo. Vy Truong, Ngan Le, Thieu Vo. Bac Le, and Anh Nguyen. Lunguage-conditions affordance-pose detection in 3d point clouds. In 2024 IEEP International Conference on Robotics and Automation (ICRA), pp. 3071-3078. IEEE, 2024.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In Inter- national conference on machine learning, pp. 8162 8171. PMLR, 2021.
Chuer Pan, Brian Okor, Harry Zhang, Ben Eisner, and David Held. Tax-pose: Task-specific cross-pose estimation for robot manipulation. pp. 1783-1792. PMLR, 2023.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference computer vision. pp. 4195-4205, 2023.
Xiaogang Peng, Yiming Xie, Zizhao Wu, Varun Jampani, Deqing Sun, and Huaizu Jiang. Hoi-diff: Text-driven synthesis of 3d human-object interactions using diffusion models. arXiv preprint arXiv:2312.0655: 2023.
Ilya Petrov, Riccardo Marin, Julian Chibane, and Gerard Pons-Moll. Object pop-up: Can we infer 3d objects and their poses from human interactions alone? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4726 4736, 2023.
Charles Ruizhongtai Qi. Yi, Hao Su, and Leonidas Guibas. Pointnet+++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems, 30, 2017.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sas- try, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PmLR, 2021.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen, Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125 1())3, 2022.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. JEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 10684-10695, 2022.
Debaditya Roy and Basura Fernando. Action anticipation using pairwise human-object interactions and trans- formers. IEEE Transactions on Image Processing, 30:81 16 8129, 2021.
Jascha Sohl- Bohh-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. Inil. Conf. on Machine Learning (ICML). PMLR, 2015.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv:2010.02502, 2020.
arXiv preprint
Michael J Swain and Dana II Ballard, Color indexing. International journal of computer vision, 701::11-3,, 1991.
Omid Taheri, Nima Ghorbani, Michael Black, and Dimitrios Tzionas. Grab: A dataset of whole-hody human grasping of objects. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part IV 16, pp. 581-600. Springer, 2020.
Shashank Tripathi, Agniv Chatterjee, Jeau-Claude Passy, Hongwei Yi, Dimitrios Tzionas, and Michael J Black. Deco: Dense estimation of 3d human-scene contact in the wild. In Proceedings of the IEFF/CVF Interna- tional Conference Computer Vision, pp. 8001-8013, 2023.
13

---

Published as a conference paper at ICLR 2026
Jingbo Wang, Yu Rong. Jingyuan Liu, Sijie Yan, Dahua Lin, and Bo Dai. Towards diverse and natural aware 3d human motion synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20460- 00660, 2022a.
Wang. Yixin Chen, Tengyu Liu, Yixin Zhu, Wei Liang, and Siyuan Huang. Humanise: Language- conditioned human motion generation 3d scenes, Advances in Neural Information Processing Systems, 1591999914 2022b.
Zeqi Xiao, Tai Wang, Jingbo Wang. Jinkun Cao, Wenwei Zhang, Bo Dai, Dahua Lin, and Jiangmiao Pang. Unified human-scent scene interaction via prompted chain of contacts. arXiv preprint arXiv:2309.07911 2023.
Mengda Xu, Zhenjia Xu, Yinghao Xu, Cheng Chi, Gordon Wetzstein, Manuela Veloso, and Shuran Song. Flow as the cross- doss-dommin manipulation interface, Oct 2024. htt.h:tps////i 15:88
/ttp://://axxlvooaaaaa/////// /
Yuhang Yang. Wei Zhai, Hongchen Luo, Yang Cao, Jiebo Luo, and Zheng-Ju Jun Zha. Grounding 3d object affordance from 2d interactions in images. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 10905-10915, 2023.
Yufei Ye, Xucting Li, Abhinav Gupta, Shalini De Mello, Stan Birchfield, Jiaming Song. Shubham Tulsiani, and Sifei Liu. Affordance diffusion: Synthesizing hand-ohject interactions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22479-22489, 2023.
Harry Zhang, Ben Eisner, and David Held. Flowbot++: Learning generalized articulated objects manipulation via articulation projection. arXiv preprint arXiv:2306. /2893, 2023.
Jinlu Zhang, Yixin Chen, 7an Wang, Jie Yang, Yizhou Wang, and Siyuan Huang. Interactanything: Zero-shot human object interaction synthesis via IIm feedhack and object affordance parsing. Proceedings of the Computer Vision and Pattern Recognition Conference, pp- 7015-7025, 2025.
Xiaohan Zhang, Bharat Lal Bhatnagar, Sebastian Starke, Vladimir Guzov, and Gerard Pons-Moll. Couch: Towards controllable human-chair interactions. In European Conference on Computer Vision, pp. 518-535. Springer, 2022.
Yang Zheng, Yanchao Yang, Kaichun Mo, Jiaman Li, Tao Yu, Yebin Liu, C Karen Liu, and Lconidas J Guibas. Gimo: Gaze-informed human motion prediction in context. In European Conference Computer Vision, pp. 676-694. Springer, 2022.
Keyang 7hou, Bharat Lal Bhatnagar, Jan Fric Lenssen, and Gerard Pons-Moll. Toch: Spatio-temporal object correspondence hand for motion refinement. arXiv preprint arXiv:2205.0795. 2022.
14

---

Published as a conference paper at ICLR 2026
A PROMPTING THE 3D GENERATIVE MODEL
To prompt the 3D generative model CHOIS et al. (2024a), we follow standard practices docu- mented in Li et al. 22024a) where the model first randomly generates series of waypoints for the human to follow and we prompt HOI generation using suggested prompts from et al. (2024a) We list some examples below in Table 3
Raw Prompt
Normalized Prompt
Facing the back of the chair, lift the chair and then place the chair onto the floor. Lift and move the chair.
Facing the back of the chair, lift the chair, move the chair, and then place the chair on the floor.
Lift the chair, move the chair, and put down the chair.
Grab the top of the chair, swing the chair.
Grab the top of the chair, swing the chair, and put down the chair.
Lift the chair over your head, walk and place the chair onto the floor.
Lift the chair over your head, walk and then place the chair on the floor.
Put your hand on the back of the chair at the top. Pull it to move it across the floor. Lift the chair, rotate the chair and set it back down. Use the foot scoot the chair
Put your hand on the back of the chair, pull the chair, and set it back down.
Lift the chair, rotate the chair, and set it back down.
Use the foot scoot the chair change its orien- tation.
to change its orientation. Push the chair, then turn yourself around you can then drag the chair behind you. Hold the chair and turn it around to face a diffferent orientation.
Push the chair, release the hands, then drag the chair, and set it back down.
Hold the chair and turn it around to face a different orientation.
Grab one of the chair's legs and tilt it at angle. Kick the chair across the room. Lift the chair, flip it upside down and place it on top of the table.
Grab the chair's legs, tilt the chair.
Kick the chair, and set it back down. Lift the chair, flip it upside down and place it on top of the table.
Move the chair upside down from the table to the floor.
Move the chair upside down from the table to the floor.
lift the chair, flip it upside down and place it on top of the table. And then move the chair upside down from the table to the floor.
Lift the chair, flip it upside down and place it on top of the table. And then move the chair upside down from the table to the floor.
Lift and move the chair. Then
Lift the chair, move the chair, then kick the chair move, and set it back down.
kick the chair to move.
Table 3: Examples of raw and normalized action prompts used in CHOIS.
B ZERO-POSE HUMAN CONFIGURATION
We center the zero-pose human and the object into the same canonical frame. Specifically, we subtract the centroid of the object point cloud from both the object and the human point cloud. To make sure the model is onation-equivariant, we apply random perturbations to the object point cloud during training data generation. In this way, the zero-shape human will be agnostic to the object location and rotation during inference. Note that in Figure we move the human point cloud to the side for better visibility. In reality, the object and zero-pose human will overlap.
15

---

Published as a conference paper at ICLR 2026
C
DENSE FLOWS GROUND TRUTH
No
F - 1fil
FPS
GT Flows Culculation
Subsampling
(had
Ma
Figure 6: Dense flow training data generation visualization. Given pair of HOI mesh generated from CHOIS, we first subsample the mesh vertices into point clouds using furthest point sampling (FPS) Qi et al. (2017). We then calculate the ground truth dense flows using Equation
As shown in Figure the ground-truth dense flows are caleulated the per-point displacement from zero-pose SMPL. model to the ground-truth HOI sample human, both of which are subsampled using the same set of indices.
D DATASET DETAILS
H2OFlow trains on OMOMO objects Li et al. (2023b), where the training set comprises 12 object categories while the testing dataset has object categories. For each object in the training dataset, we generate 100 HOI sequences, where each sequence has 200 frames.
E HYPERPARAMETERS AND TRAINING DETAILS
We document the choice of hyperparmeters used in H2OFlow. For training the diffusion model, use a learning rate of le-4, a weight decay of 1e-5, a batch size 32, and training epochs number of 20,000. Both human and object point clouds training, we apply random rotation to the objects and random occlusion as augmentation in order to ensure robustness to real-world variability. The model is trained with the A optimizer, and the total number of steps is set to T 100 the diffusion process. Following Cai et al. (2024); Nichol 128 the hidden size per DIT block. We have heads per block and
downsampled to 512 points via FPS. During
& Dhariwal (2021), ), blocks in total.
In the inference of comprehensive affordances, we temperature hyperparameter of 20 for contact affordance Cij. For orientational affordance, we use variance of o² 1 and temperature hyperparameter of 10.
During training and testing, we center the object coordinates and produce ground-truie truth in object frame (i.e., the object is always upright).
F DIFFUSION MODEL. DETAILS
We parameterize the reverse process as: (F F.) = ((F Eo)),), where
the mean o4 and variance Σn derived from predicted noise term EF Ho, O,t) and an interpolation vector vo v F,, ,, Ho, O,t). Following Nichol & Dhariwal (2021). the interpolation vector contains value per dimension and is used parameterize the covariance: Ep F
exp log 3, + (1 v) log - (e). We supervise the model using the hybrid loss from Nichol
& Dhariwal (202T) that combines the regular noise loss with new cumulative KL-divergence loss using the derived Eg (F.).
16

---

Published as a conference paper at ICLR 2026
G COMPARISON WITH COMA
G.1 METHODOLOGY
We emphasize that H2OFlow is fundamentally different from COMA as H2OFlow introduces four technical advances over COMA.
First, more generalizability and flexibility. Most fundamentally, COMA directly uses reconstructed 3D imputs and has learned components in their pipeline. While COMA lays out comprehensive affordances in a nice way, the lack of learning- based methods lacks its generalizability and flexibility when it comes unseen objects (as noted in the quantitative results of the paper).
Second, a point-cloud based affordance learning paradigm with dense diffused flows as an effective intermediate representation. In COMA, affordances are inferred from reconstructed meshes instead of learned flow representations, which lack the generalizability to real-world visual inputs. More- over, COMA'S per-vertex- affordance calculation between meshes consumes lot more memory and time than H2OFlow's 's sparser per-point-pair formulation. In H2OFlow, with flows, no water- tight meshes or surface normals are needed, which tend be noisy supported by results in Table [ and Table where COMA struggles to generalize to unseen objects and reconstructed meshes from noisy point clouds while H2OFlow performs well in both cases.
worddsceccrrsss This is
Third, diffusion- blffusioo-bbsed multi mudti-modal dense-flow flow predictor based on per-point encoding, This learning paradigm handles intrinsic ambiguity due to molliitoddlltyy and also learns to reason about geomet- ric information on different regions (local information) of the object-human interaction. With dense diffused flows, H2OFlow's pipeline provides a new method for modeling human pose with a more flexible representation. At the same time, this representation sidesteps the need of normal vectors from meshes (Equation (5). Equation (6)) which are costly to compute for real-world applications, while achieving better results.
Lastly, cross-attention aggregation & partial-scan robustness. We improve the affordance aggrega- tion via learned cross-attention weights (see ablations in Table During training, we apply random rotation to the objects and random occlusion as augmentation in order to ensure robustness to real- world variability, which ensures the robustness occlusion in the real world, opposed COMA that requires a clean mesh of the object.
G.2 REAL-WORLD RESULTS
In Figure 7 we show comparisons with COMA on real-world unseen objects. COMA relies on 2D renderings from clean meshes, and thus struggles with noisy, reconstructed meshes derived from point clouds. This limitation severely degrades COMA's performance, resulting in oversimplified and unimodal affordance score maps.
H MORE QUALITATIVE RESULTS ON OMOMO DATASET
In Figure g
we provide more examples. Specifically, in the monitor example, the selected bottom (orange) point makes more contact with the side of the body while the center (blue) point makes frequent contact with the whole torso, which reflects real-world contact tendency when moving a monitor. For the tripod, human legs tend exhibit of the tripod (orange) than their hands, while the orientation of the hands relative to the top part of the tripod (blue) is more uniform. For spatial affordance, we can the high-level human occupancy around the object during HOIs: the high-pne high-probability regions are more frequently occupied by the human body parts, consistent with real- world interactions (in some cases, full human silhouettes are observed).
uniform orientation relative the bottom
I RESULTS ON BEHAVE DATASET
We also test method BEHAVE dataset. To test the generalizability of H2OFlow, we provide baseline version by testing on BEHAVE objects directly without fine-tuning (H20Flow-NoFTT) We also have another baseline that finetuned on the BEHAVE objects (H20Flow-FT). The results
17

---

Published as a conference paper at ICLR 2026
Object
Point Cloud
H2OFlow Contact
H2OFlow Orientation
COMA Contact
COMA Orientation
Figure Comparison with COMA on real point clouds.
18

---

Published as a conference paper at ICLR 2026
Objects
Predicted HOIS
Objject Contacts
Human Contacts
Orientational
Spatial
Visualizations of the affordances inferred from dense diffused flows prediction. H2OFlow infers diverse affordance distributions from predicted HOI samples on unseen objects.
Figure 8:
Baseline
SIM-H f
SIM-O
MAE-H J
MAE-O L
Precision
MSE
COMA
52.8 + 1.5%
70.6 1.1%
0.13 + 0.04
0.03 + 0.01
5.1%%
0.13 + 0.05
COMA-Recon
42.5 1 3.2%
55.8 1 1.7%
0.22 0.10
0.155
42.4 1. 2.2%
0.36 0.21
H2OFlow-NoFT
55.3 - 3.6%
71.2
1.4%
0.13 0.13=000 0.03
0.03
72.2 1.2%
0.14 = 0.02
H20Flow-FT
11.41120 1.2%
80.1
0.03
0.02 0.01
79.1 5.2%
0.11 = 0.02
Table 4: Quantitative comparisons with various baselines on BEHAVE dataset. Note that -H and -O represent human and object contact results.
are shown in Table Without any fine-tuning, H2OFlow performed comparably with COMA's full-mesh version. After fine-tuning on the BEHAVE dataset, H2OFlow's performance exceeded COMA's by noticeable margin.
J COMPARISONS WITH OTHER CONTACT-ONLY BASELINES
While few other methods focused comprehensive affordances, provide comparisons with other contact-fforrdance-onf baselines in Table[5 Specifically, we compare with IAGNet Yang al. (0033) and DECO Tripathi al. (2023) which respectively only measure contact affordances for human and object on BEHAVE test images.
K ABLATION ON OCCLUSION
Random masking during training lets the model accept incomplete seans. In Table and Table we also see that COMA struggles with outputting high-quality affordances reconstructed meshes from partially observed point clouds, while H2OFlow is agnostic to occlusion due to training-time augmentation. New experiments also support the fact that H2OFlow is robust to out-of-distribution occlusion due to the random masking introduced in training. In Table 6 we evaluate H2OFlow's sensitivity to occlusion test objects and show that the performance loss due to occlusion and partial observability minimal, indicating robustness to commodity depth cameras or monocular depth-completion pipelines.
19

---

Published as a conference paper at ICLR 2026
Baseline
SIM-H
SIM-O f
MAE-H
MAE-O.
IAGNet
64.34 1.2%
0.03 = 0.01
DECO
23.18 = 2.1%
0.23 = 0.07
COMA
52.8 1.5%
70.6 1066+1.11 1.1%
0.13 + 0.04
0.03 + 0.01
COMA-Recon
42.5
3.2%
0.22
0.15
I120Flo-NoFTT
55.3
3.6%
71.2
1.4%
0.13
0.03
0.03 0.01
H20Flow-FT
74.4 + 1.2%
80.1 = 1.8%
0.10 = 0.03
0.02 1 0.01
Table Quantitative comparisons of contact affordances only with various baselines on BEHAVE dataset. Note that -H and -O represent human and object contact results.
Baseline
SIM-H
SIM-O1
MAE-H,
MAE-O1
Precision@K1
MAE,
No Occl.
72.3%
81.0%
0.11
0.07
75.6%
0.12
10% Ocel.
72.3%
80.8%
0.12
0.07
75.5%
0.12
30% Occl.
71.2%
80.7%
0.12
0.08
75.3%
0.13
50% Ocel.
70.9%
79.5%
0.13
0.08
74.2%
0.13
Table 6: Performance under different occlusion levels.
L FLOW PREDICTION DESIGN CHOICE
An interesting question is why we learn to predict dense flows instead of SMPL parameters? We answer the question below.
Local-geometry awareness. A flow vector originates at every human vertex and therefore directly observes local object geometry: SMPL pose parameters do not. This makes flows more suitable for fine-grained, multimodal affordances.
Lower computational cost. Flow prediction needs only the object cloud and canonical human point cloud; SMPLL-parameter regression would additionally require reconstructing the human and sampling vertices a separate task. Moreover, for orientational affordance, normal directions would have been required if we directly learned SMPL without the intermediate dense flow repre- sentation. As pointed out in COMA, this is the bottleneck for computation.
Multi-modality. Flows allow the diffusion model to sample multiple valid endpoints (left-/right- hand grasp, frontal/back sitting). We conducted smaller-scale experiment, where we learn direct SMPL predictor using diffusion conditioned on the object input. For affordance scores, direct SMPL regression does not support cross-attention weights during learning, so weighting is not available in aggregation here.
per-point human information learned
Performance. Results in Table I suggest that the direct SMPL formulation tends to perform lot worse. One potential reason is that human pose parameter, especially rotation, is a lot harder to learn. Previous work Eisner al. (2022); Pan al. (2023); Zhang al. (2023); Li et al. (2024b);
Cai et al. (2024) on dense-flow learning designed flows to sidestep the rotation learning issue.
M MEMORY AND RUNTIME COMPARISONS
H2OFlow, on average, utilizes 8416 + 513 MB of GPU memory and 7619 = 882 MB of CPU mem- ory. On single V100 GPU, it takes H2OFlow 6.7 + 1.2 seconds infer affordances for unseen point cloud. In contrast, our experiments with COMA indicate that it takes 9771 +: 1190 MB of GPU memory and 15812 + 2314 MB of CPU memory. On single V100 GPU, it takes COMA 65.2 2.1 seconds to infer affordances for an unseen object. When given point cloud, the time spent on cre- ating watertight mesh for COMA becomes the bottleneck. This is expected, as COMA analyzes per-pair mesh vertex normal directions and requires extensive inpainting operations. H2OFlow by- passes the large memory consumption by predicting point clouds directly.
N APPLICATIONS TO OTHER DOMAINS VIA DENSE OPTIMIZATION
We discuss two potential cases of the learned diffused flows and affordance scores.
20

---

Published as a conference paper at ICLR 2026
N.1 RECONSTRUCTING FULL SMPL PARAMETERS FROM DENSE DIFFUSED FLOWS
One straightforward application is to reconstruct the full human SMPL parameters from the point cloud generated from the learned diffused flows model. Specifically, SMPL. pose and shape parameters via the following optimization problem:
able recover the full
0, B. R. i arg min 8,
3, R,
with C Σ (0.3)) +t hall + λo 10112 + X3 1|31
(10)
vertices reconnsrucciionnnnccmm
where is the set of sampled vertices from the dense diffused flow model (e, 3) is the vertex location from using the SMPI parameterm,, |Ø]] and || act as priors because the data term alone under-constrained when only sparse subset of vertices are seen. Using dense diffused flow alone, we are able to reconstruct the full SMPL mesh. This again illustrates that the flows are an implicit form of affordance.
N.2
RECONSTRUCTION
Another interesting aspect of the affordance scores to reconstruct different embodiments based the predicted human point cloud. For example, suppose embodiment correspondence between a robot point rı to human point h, then we are able to recon- struct the full robot configuration based on the reconstructed HOI samples and affordance scores.
able to obtain dense cross-
Specifically, we are given a set of pre-computed human-object scores {chum R } that capture the contact and orientational affordances between human surface points h; and object points oj. Our goal is to find robot configuration that reproduces these scores as faithfully as possible.
First, we define the following parameters:
RRPr joint-space parameters that generate nominal robot surface points (r)())) through FK function.
(R, global rigid transform (R E SO(3) and IR3) that aligns the robot the human coordinate frame: the aligned points are r/ (2, R.t) t) = ( +
Crobot the feasible set defined by the robot's 'ss joint limits, self-collision constraints, and object-penetration avoidance.
Robot-object contact score.
For each robot point & and object point j we define Ckyy (b, R, t) = exp( exp(-I1r' R,,RR, t) - o,1l /T).
(11)
The score increases as Euclidean distance between the aligned robot point and the object point decreases.
Robot-object orientational score. We first form unit direction vector
Let f; the dense diffused flow attached to human point h;
( R,t) 0,) f; ( 0,) fll
Øky ( R. t)
(12)
where the cross product couples the displacement T O, with the flow f.. We discretize the unit sphere 2 into cells with representative normals [n,,) j and compute the probability that æıj falls into cell
Pa.ky (m:; 4b R, t) x exp exp (-1la11ss R, 12.112/220).
(13)
The orientation is then the negative Shannon entropy
RRy (2, R, Σ
Pa.k (n; d R. log (n; R. t).
(14)
A low entropy indicates strongly preferred orientation and hence large Raj-
21

---

Published as a conference paper at ICLR 2026
Cruss-embodiment matching loss. using a weighted squared loss
We force the robot scores to agree with the human scores
N
Na
R
Mk
(Cky (b) R. t) hum 2
3=1
R hum 2
Rky
4 R.t).
(15)
The correspondence weight Mys transfers each human score (i,j) to its associated robot point k.
Optimization problem. Our final objective to minimize the loss 15 subject the kinematic constraints:
min ECoobas RESO(3); tER*
C(Ø, R,,R,
(16)
Solving (16) yields robot pose (Φ* t*) whose contact and orientational affordance fields best
imitate those observed for the human demonstrator, while remaining physically feasible for the robot.
O REAL-WORLD QUANTITATIVE RESULTS
We design set of experiments to quantitatively assess H2OFlow's performance real-world ob- jects. While real-world ground truth is not available, we devise two complementary lines of met- rics to compare baseline performance. We select six real-world objects that have similar simulated counterparts in the OMOMO and BEHAVE datasets. Each real object point cloud is aligned to a canonical object frame using ICP with its closest simulated counterpart.
We report two types of quantitative metrics: (1) SMPL-based metrics, which compare recon- structed goal human poses against affordance-based metries, which estimate relative plausibility and spatial consistency of predicted affordances. Since affordance ground truth is not available real-world data, these are used only as complementary evidence.
synthetic human-object interactions, and (2)
Given a real object O, sample N goal human configurations from H2OFlow and reconstruct corresponding SMPL poses SH20 { using the optimization defined in Eq For the COMA baseline, we use N generated SMPLs inferred from multi-view 2D renderings of the same object. We collect M reference SMPL poses from CHOIS on a geometrically similar simulated IM object, denoted SCHOIS [HOIS 1m All SMPLs centered in the object coordinate frame with pelvis translation removed.
O.1 DEFINING METRICS FOR REAL-WORLD EXPERIMENTS
We compute three set-to-set distances between generated and reference SMPL sets. First, Minimum Matching Distance (MMD) measures the average minimum per-sample joint distances d(S,
(1 JSS))
1 min d'S, |S|
MMD(S, S
(17)
S'e
Next, Coverage measures the percentage of candidate set that covers the modes of reference set:
COV- | E S 3SES, d(s, (')
(18)
SSall
Lastly, Frechet Pose Distance (FPD) measures the distributional similarity between the prediction sets:
FPD = ||fe μ4 2 + Tr(E, + E,
2(23/",,, 1B-
(19)
where (S) denotes the 3D joint positions of pose S. and (Hr, E.), (Hr: Σ,) are the means and covariances of joint vectors from the candidate and reference sets, respectively. Lower MMD and
22

---

Published as a conference paper at ICLR 2026
Method
MMD
coV@ 15em (%)
FPD L
SIM-H (%)
MAE-H.
Precision @ (%e)
MSE
COMA
12.8
43.2
61.5
49.7
40.9
H2OFlow
8,9
64.5
47.8
68.6
0.13
71.3
0.13
H2OFlow (None)
10.6
58.9
50.6
65.4
0.16
67.2
0.16
H2OFlow (Light)
9.2
63.1
48.6
0,14
70.8
H2OFlow (Medium)
8.9
64.5
47.8
68.6
0.13
71.3
0.13
H2OFlow (Aggressive)
9.9
60.2
49.5
65.9
0.15
68.1
0.17
Table 7: Quantitative comparison of SMPL-bbsed distances between real-world predictions and CHOIS reference poses. Lower MMD/FPD and higher Coverage indicate better alignment with synthetic interaction distributions. "Ligh/Meddiu//Agreesiii correspond to the presets in Sec-
tion 0.2
FPD and higher COV, indicate closer alignment between predicted and synthetic interaction distri- butions.
We additionally report similarity between predicted contact and spatial occupancy distributions de- rived from real scans and the corresponding simulated object using the same metrics used in simu- lated experiments. These metrics capture how well the learned affordance distributions transfer to real geometry.
Across six real-world objects, H2OFlow achieves significantly lower MMD/FPD and higher cover- age than COMA, confirming its ability to generalize to real-world objects without manual labeling.
O.2 ROBUSTNESS TO NOISE LEVEL
Our real-world captures undergo post-processing, which can yield relatively clean point clouds. To quantify robustness, we test H2OFlow on noisy single-object point clouds and define a reproducible denoising pipeline with controllable strength. We assume single, segmented object point clouds, captured via a commodity RGB-D camera (in our case, an iPhone camera).
We adopt standard point-cloud pipeline available in Open3D Library. Each stage and its control parameter is listed so denoising strength is explicit and reproducible.
Statistical Outlier Removal statisticon sttt wel.iirrrrrvallleeee (nb_nnilllliiiiieeeeele nei ghbors =
k, std_ratio r), which removes isolated points whose mean neighbor distance exceeds standard deviations. Controls: k (neighbors), r (aggressiveness).
Radius Outlier Removal (ROR):
m, radius R), which prunes sparse regions lacking neighbors within radius R. Controls: m, R (m).
We then apply FPS to the cleaned point clouds, yielding a denoised, subsampled result. We define four presets that sweep aggressiveness:
23

---

Published as a conference paper at ICLR 2026
Preset
m
Visualization
None
Light
20
2.0
8
0.01
Medium
30
1.5
12
0.012
Aggressive
50
1.0 16
0.015
In our evaluation, we use the same H2OFlow weights across all conditions. For each object, we use RealityKit to collect raw noisy scan, denoised with same real-world objects in Section O.I with the same real-world evaluation protocols.
We use the
H2OFlow, by default, cloud-native and remains stable under Light and Medium denoising. By contrast, the Aggressive preset over-prune thin structures (e.g., handles, rims), degrading affordance performance near those parts and increasing collisions during pose selection. We also observe that the drop of perfor- mance is not substantial: although both None and Aggressive settings reduce performance relative to Light/Medium, they still outperform the COMA baseline, underscoring the effectiveness of flows as the intermediate representation.
the medium denoising setting. As shown in Table it is point-
P SCALABILITY
P.I SCALING VIA DATA AUGMENTATION
For objects like chairs, the dominant affordance is often hips-on-seat (sitting) rather than hand- mediated manipulation. Because training set is synthesized from CHOIS (object motion-guide interactions). it over-represents moving/lifting patterns. Fortunately, H2OFlow's training recipe is model-agnostic and point-cloud-centric. everyday usage (sit, lean, rest, place, open/close) and retrain the same dense-flow learner.
expand the HOI source models to better cover
We augment the synthetic HOI pool with recent 3D HOI generation models that explicitly produce usage-centric human-object interactions such as InteractAnything Zhang et al. (2025), HOI-PAGE Li& Dai (023), and PICO Cseke al. (2025). All generators standardized by our existing pre- processing: mesh + FPS subsampling + paired point clouds, identical to our current pipeline. No watertight meshes normals are required downstream. By adding more training data, we effec- tively expand the actions used: sit on, perch, lean back, rest arm, place on seal, kneel, step onto, lie on... We assemble balanced mixture of manipulation and usage episodes per category and filter collisions or self-eenctrations We keep the same object splits (train/test categories) in the main paper. We retrain the same DiT flow model with identical losseshtyperparametros only the HOI source distribution changes. Dense diffused flows remain the intermediate representation, preserv- ing compatibility with our affordance inference.
As the chair example (same real-world chair from Figure 5 shows in Figure without training data augmentation from Zhang et al. (2025), the contact affordance mostly concentrates on the hands areas with limited contact in the hip area due to the fact that most chair interactions from CHOIS
24

---

Published as a conference paper at ICLR 2026
Point Cloud
Contact
Orientation
Spatial
Contact
Orientation
Spatial
Before Augmentation
After Augmentation
Figure 9: Comparison of the three affordance representations before and after dataset augmentation with usage data. After augmentation, we observe more symmetry and meaningful interaction pat- terns that reflect actual object usage (e.g., hip-on-seat).
data are moving the chair. Similarly, for orientation, not much trend is seen in the hip area and for spatial affordance, the oceupancy is concentrated the back of the chair, indicating how human moves the chair. While the above predictions valid for moving the chair, more expressive affordances that indicate sitting on the chair are more informative. Augmented with synthetic data generated from Zhang et al. (202)). the training dataset. Thus, the flow predictor will learn to predict both moving and using the chair. After this augmentation, we a bi-modal concentration of contact affordance in both hands and hip areas, indicating the model now learns to output contact based on actual usage. For the orientation affordance, we now better symmetry well as concentrated orientational pattern in the legs area. Lastly, for the spatial affordance, the front side of the chair is now more frequently occupied.
have more HOI meshes representing sitting on the chair in
Because H2OFlow learns from point-cloud flows rather than annotation-heavy labels or normals, broadening generators directly broadens learned usage affordances (e.g., hips-on-seat) without ar- chitectural change.
P.2
DENSE DIFFUSED FLOWS
Beyond dataset diversification, can condition the dense-flow generator a textual intent ("sit on the chair"). This steers sampling toward usage-consistent interactions (seat occupancy, torso orientation) at test time, even for unseen objects, while keeping H2OFlow's point-based formulation intact.
Let t be tokenized prompt. We encode text with frozen CLIP text encoder Radford et al. (2021) (taa() E R fxx We then condition the DiT backbone used for dense-flow denoising with cross- attention adapters: in each DiT block, after self-attention the joint human-flow tokens (as in the main model), add cross-attention from human-flow tokens to text tokens. Object tokens remain as in the main model and hybrid diffusion loss as our current DiT.
cross-attention). We also retain the same coise-prediction targets
As standard in conditional diffusion Ho Salimans (022), we drop text with probability Párop dur- ing training and learn unconditinnal/oondttooa with guidance scale y: & (1 + 7) co (F, O, 0,t,2) 2) y co (F, O E
branches jointly. At inference, we sample flows
During training we attach short textual prompts to each synthetic HOI episode and automatically normalize existing action descriptions into concise templates (e.g.. "sit on chair". "lean back on backrest"), preserving the same category splits. At test time, language steers the sampled human goal configurations H Ho + F; our affordance inference (contact Cjj. orientational Ray. spatial Si,) remains unchanged.
terms of implementation, the change is minimal, as only a few pieces in the DiT are changed.
class BiTBlock ( (nn (
def iniit_ (self hidden_size. num hemdseads: super -nit.. -)
self. self init..
SA( hidden-size. nea-heads)
CA(hidden size. heads self. self = Mip(hidden.size. int Intdddeenn.iiz 4))
self. cross_attn-o
# adalN-Zero modulation
# outputs shifts scales gates for submodules chunks
25

---

Published as a conference paper at ICLR 2026
self adaLN Sequential
nn S))U()
nn Linear (hidden_size, 9 hidden_size, bias=Truc)
def forward x. y y-obj cond, x x-pos=None, y-pos=None): (sh-msa, se. se.msa, gt-msa.
sh mo. sc so. go.
sh.mlp. sc-mlp. gt.mlp) = self (cond ). chunk dim=1)
modulate self. norm_msa x_hf), sh_msa, sc_msa)
x = x gtmmsa.
msa, unsqueeze (1) * self self_attn query=x, key=x, valuc=x, rotary potary-pe=(xxo pos, x.pos)
[o]
x_o modulate (self. norm mx).mm)a((()) sh.mo, sc_mo)
x = x + gt_mo. unsqueeze (1) sels..ross.................s cross_attnnn....
query=x_o, key=y.obj, value=y-obj, rotary-pe=(x-pot y-pos)
[0]
x_ff = modulate (self. nor mx))_ sh_mlp, sc_mlp)
x = x gt mlp (nsqueeze(1) * self mifpm ff)
return
Figure 10: Original H2OFlow's DiT Block Implementation in Py Torch
class DiTBlockTex( Module):
def -.initt-(self hidden-size. num.heads):
super ()_err)...ittt... SA( siideen.siee... num_heads)
sclf
self. cross-attn_o CA( hidden.size. num_heads self. crosscccossaaan... self. mlp = Mlp (Mphidden-size. int(hidden 4))
CA(hidden_size num_heads)
# adaLN-Zero modulation
# outputs shifts scales /gates for submodules 12 chunks self adaLN an. Sequential nn n (iL.U),
nn Linear hidden_size, 12 hidden_size, bias=True)
def (se... x.hf, y.obj. 2.1xt, cond, x.pos, y-pos, z.pos):
msh.msa, se.msa. gt-msa
sh_mo, sc.mo. gt.mo.
sh_mt, sc_mt. gt-mt.
sh-mlp. se.mip. gt.mlp) self adaLN (cond) ). chunk (12, dim=l) x = modulate (self.. norm_msa x_hf), sh_msa, sc_msa)
x = x gt_msa.. (1) * self. self_atnn
query=x, key=x, value=x, rotary pe=(x pos, pos)
[0]
x.o = modulate (self. norm.mca.o(x). shamo, sc_mo)
= x + gt_mo. unsqueeze (1) * cross.attosttt
query=x_o, key=y-obj, valuc=y-obj, rotary_pe=(x_pos, y-pos)
[00]
x.t = podulate(sell) norm [em_ sh,ml, sc_mt)
gt.mt. (nsqueeze()) * self.crose
query=x.t, key=z.ixt, value=z-txt, rotary pexty--p=xxxp..... z.pos)
[0]
x_ff = modulate(selff morm.mlp(x)., sh_mlp. sc.mlp)
= unsquezzee & self mlp(x_ff)
return
26

---

Published as a conference paper at ICLR 2026
Point Cloud
Contact
Orientation "Sit on the chair"
Spatial
Contact
Orientation
Sputial
"Move the chair"
Figure 12: Chair example of prompt-conditioned H2OFlow
Figure 11: Prompt-Conditioned H2OFlow's DIT Block Implementation in Py/Torch As shown in the implementation of Figure 10 and Figure T1 with minimal changes to the DiT block architecture, are able condition the output flows on the tokenized prompts. In Figure 12 use the same example as above but test it on the text-conditioned H2OFlow. As one can observe, with the prompt conditioning, the different usages of the chair yield really different affordances. The contact now is not bimodal between hands and hips and is now concentrated on one body part based on the usage. Similarly, for the orientation affordance, we observe high concentration on hips when sitting and on arms when moving. More interestingly, we see sitting silhouette for sitting and standing silhouette for moving.
Thus, small, modular text head yields promptable dense flows that align with language-speccffed affordances, while preserving the core point-cloud training and inference H2OFlow.
Q LIMITATIONS
While H2OFlow learns comprehensive affordances from synthetic data and generalizes to unseen objects' noisy point clouds, it has a few limitations. First, the underlying generative model does not have a sufficiently large variety of objects-more fine-grained interactions on smaller, articulated objects not captured. While H2OFlow foundation models and datasets such fine-gr ime-grrined HOI tasks. Second, H2OFlow could be ex- tended to physical robots, by warmstarting the manipulation policies based on the affordance score while constructing a correspondence map between human point clouds and robot points. We leave this extension to future work.
learn from arbitrary HOI samples, there limited
R
LLM USAGE
We primarily used LLMs to check grammar and spelling. We also used LLMs for formatting tables and figures.
27


Reviews:
[{"review_id": "qQkHA2oE9w", "reviewer": "Reviewer_qngF", "summary": "The paper studies the problem of understanding the affordance of human object interaction. To this end, the authors propose a representation that captures not only contact affordance, but also orientational affordance and spatial affordance. The authors propose a pipeline to generate synthetic data using 3D generative models and employ a dense 3D-flow-based representation. The authors claim that, through extensive quantitative and qualitative experiments, the effectiveness and practical utility of the learned affordances on both synthetic datasets and real-world data are demonstrated.", "questions": "Please see the weaknesses part of my review. I would suggest the authors to address my concerns in the rebuttal.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The introduction of the affordance representation that captures both explicit contact and implicit non-contact interaction patterns is novel to me.\n\n2. The problem the paper studies is important. If we want to move to spatial and physical AI in the future, it is important to understand human-object affordance.", "weaknesses": "1. The organisation of the paper needs to be improved. The overview figure as referred in Section 3 is very important for the understanding of the problem formulation, but the authors put it in the appendix. In Section 4, the authors describe a lot about their method, and it would be a lot better if there is a figure to demonstrate the whole method. \n\n2. Insufficient real-world results. The authors claim that their method can be well generalised to unseen real-world objects. It would be more convincing if the authors can provide quantitative results, given that at the moment there is only qualitative results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MjSMYlf1N5", "reviewer": "Reviewer_LAsr", "summary": "This paper presents H2OFlow, a framework for learning comprehensive 3D human-object interaction (HOI) affordances that goes beyond simple contact-based analysis. The authors address a key limitation in current affordance understanding methods, which typically focus only on contact regions while neglecting other critical aspects of human-object interactions, such as spatial orientation preferences and occupancy patterns.\n\nThe core motivation stems from the observation that human-object interactions involve rich 3D spatial relationships—for example, humans maintain characteristic distances and orientations when interacting with objects. Building on the concept of \"comprehensive affordance\" from Kim et al. (2024), which models probabilistic distributions over 3D spatial and orientational relations, the authors propose a novel approach that operates directly on point clouds rather than requiring mesh-based representations or 2D-to-3D uplifting.\n\nKey contributions include:\n\nA point-cloud-based affordance representation that captures both explicit contact and implicit non-contact interaction patterns from raw point cloud inputs\nA synthetic data generation pipeline leveraging 3D generative models and dense diffusion flows (inspired by Eisner et al. 2022) to learn flexible affordances without manual annotations\nA probabilistic formulation that operates directly on human-object point cloud pairs, eliminating the dependency on watertight meshes\nThe authors claim that H2OFlow generalizes effectively to real-world objects and outperforms existing methods that rely on manual annotations or mesh-based representations. The framework uses only synthetic data generated from 3D generative models, potentially addressing the scalability issues associated with labor-intensive dataset creation in this domain.", "questions": "1. In Section 4.1, in the first sentence, the pretrained 3D model is used to generate meshes. However, in the last sentence of the same paragraph, the outputs are videos. So what does the pretrained model actually output?\n2. Why use FPS to sample points? Have you tried other sampling policies?\n3. For unseen objects, can you generate reasonable affordances for this object? For example, in Figure 5, most of the training data is human hands interacting with the object, and then for the unseen object, the chair, it also gives information about how hands interact with it. However, for chairs, maybe \"Hips sitting on the chair\" is a more common affordance. How can the method handle this situation?\n4. Are the additional affordance representations (except contact) useful? Where is the ablation study about this?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 3, "strengths": "Originality: \n1. This paper introduces a novel paradigm shift from manual contact annotation to direct learning from point cloud datasets.\n2. The integration of dense diffusion flows with 3D generative models represents an innovative approach to synthetic data generation that eliminates dependency on high-quality mesh inputs. \n3. It extends beyond the traditional binary contact-based affordance definition by incorporating spatial orientation and occupancy patterns, providing a more comprehensive and nuanced understanding of human-object interactions.\n\nQuality: This paper achieves better qualitative and quantitative results on affordance learning compared with previous methods.\n\nClarity: The training method, inference time method, and dataset acquisition methods are clear.\n\nSignificance: This work addresses a critical challenge in computer vision, robotics, and AI by providing a more scalable and comprehensive approach to affordance learning. The elimination of manual annotation requirements and mesh dependencies has substantial practical implications for real-world applications. The comprehensive affordance representation (contact + orientation + spatial occupancy) offers significantly more flexibility for downstream tasks and could enable more sophisticated robotic manipulation and scene understanding capabilities. The potential impact extends beyond affordance learning to broader areas of 3D scene understanding and human behavior modeling.", "weaknesses": "1. Although it uses the comprehensive affordance representations for better affordance definition, all three representations are proposed in previous methods, lacking original definitions and contributions. Additionally, there is no ablation study on whether the two additional representations actually yield better results for affordance learning. \n2. Compared with only one previous baseline COMA, lacking experiments.\n3. Most of the main figures are in the supplementary, indicating that the paper is not well organized.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JqHzB8KPx1", "reviewer": "Reviewer_Y1sC", "summary": "This paper proposes H2OFlow, a novel framework for learning comprehensive 3D human-object interaction (HOI) affordances, encompassing contact, orientation, and spatial occupancy. The method leverages a pre-trained 3D generative model to create a synthetic dataset, thus avoiding manual annotation. Its core innovation is a diffusion model trained to predict dense flow, a probabilistic, point-based representation of human interaction poses conditioned on an object's point cloud. In inference, sampling these flows generates a distribution of plausible interactions, from which the three affordance maps are statistically derived.", "questions": "See the Weaknesses", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The paper introduces an innovative framework that learns comprehensive affordances from synthetic HOI samples generated by 3D generative models. This approach cleverly eliminates the need for manual annotation and avoids the error-prone 2D-to-3D uplifting process used in prior work.\n* A key contribution is the use of \"dense diffused flows\" as a probabilistic, point-based representation for human interaction. This design, learned via a diffusion model, elegantly circumvents the dependency on manifold meshes and is directly responsible for the method's outstanding robustness to the noisy and partial point clouds typical of real-world sensors. The superiority of this representation is further validated by strong experimental results and thorough ablations.", "weaknesses": "* The framework's performance is fundamentally tethered to the quality and diversity of a single upstream generative model (CHOIS). The paper appears to neglect any strategy for analyzing, filtering, or augmenting synthetic data. This raises critical questions: How does the model mitigate the risk of inheriting and amplifying potential biases from its sole data source—such as limitations in interaction patterns, insufficient object diversity, and unnatural poses? What are the upper limits of the affordance knowledge defined by CHOIS?\n* The proposed method excels at exploring the general distribution of possible geometric interactions but lacks semantic control or decoupling mechanisms. This leads to two issues: (a) whether it is conditioned on specific task instructions (real-world scenarios should prioritize concrete interactions); (b) whether it can distinguish between objects that are geometrically similar yet functionally distinct. The model appears to learn more about geometric matching between humans and objects than a deeper, semantic understanding of functional accessibility.\n* This method relies on a single standard human pose (H0) as a fixed baseline for flow prediction. However, significant variations in human morphology and dimensions directly influence how individuals interact with objects. Can the method be generalized to different people's post H0?  How would the predictability and reliability change if the standard pose H0 represented individuals of varying body types?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7NRQjeMMVy", "reviewer": "Reviewer_TEPZ", "summary": "The paper proposes a pipeline for learning object affordances from 3D Human-Object interactions obtained using 3D generative AI. First, the method uses a pre-trained generation pipeline to generate human-object interaction motions, then uses it to train a diffusion model that, given an object, predicts plausible humans interacting with it using Dense Flow. The paper studies three kinds of affordances: contact, spatial, and orientational-based, and compare against several baselines and objects with geometry unseen at training time, showing good generalization.", "questions": "- Could you comment on your choice of using generative AI, instead of directly using the data on which they are trained?\n- Is it possible to provide an example on a more noisy point cloud, maybe discussing how much outliers cause degradation in the network prediction?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- I found the paper interesting, proposing a direction orthogonal to the ones in the previous methods. The study of different kinds of affordances could serve to define object classes more effectively based on their spatial and orientation qualities.\n- Improvement over competitors is significant, and the results on real objects acquired with a phone camera look exciting", "weaknesses": "- Generate interaction data using a pre-trained 3D network specifically designed for generating 3D HOI might actually incorporate biases from such methods. For example, as mentioned in the paper, the majority of methods focus on contact-based interactions, and hence, the generated data are also affected. Additionally, one might wonder why not use the original datasets used to train CHOIS as training data for the Dense Flow. Finally, the Dense Flow module works at a frame level, while CHOIS generates sequences, which can be quite redundant in terms of nearby frames.\n\n- The point clouds used in the real-world setting look particularly clean. I assume it is part of the post-processing done by the capturing device, but it would also be useful to test on more noisy data, such as point clouds from kinect fusions, as those available in the BEHAVE dataset. \n\n- Qualitative results are not well presented. Figure 3 illustrates, for example, contact over the object; however, although the table is symmetric, the legs have different affordances, highlighting a skewed distribution in the dataset. On the human body, contact is spread on legs, which I do not find intuitive. Orientational and spatial are difficult to decode. I would suggest reporting a colormap for the visualized colors, describing better what is visualized, and using a mesh when available (e.g., SMPL)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)