SpatialHand: Generative Object Manipulation from 3D Prespective
Authors: Zehan Wang
Jialei Wang
Siyu Chen
Ziang Zhang
Luping Liu
Xize Cheng
Kaihang Pan
Hengshuang Zhao
Zhou Zhao
Venue: ICLR 2026 Poster (2026)
Keywords: AIGC Application; Image Editing
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: We introduce SpatialHand, a novel framework for generative object insertion with precise 3D control. Current generative object manipulation methods primarily operate within the 2D image plane, but often fail to grasp 3D scene complexities, leading to ambiguities in an object's 3D position, orientation, and occlusion relations. SpatialHand addresses this by conceptualizing object insertion from a true ``3D perspective," enabling manipulation with a complete 6 Degrees-of-Freedom (6DoF) controllability. Specifically, our solution naturally and implicitly encodes the 6DoF pose condition by decomposing it into 2D location (via masked image), depth (via composited depth map), and 3D orientation (embedded into latent features). To overcome the scarcity of paired training data, we develop an automated data construction pipeline using synthetic 3D assets, rendering, and subject-driven generation, complemented by visual foundation models for pose estimation. We further design a multi-stage training scheme to progressively drive SpatialHand to robustly follow multiple complex conditions. Extensive experiments reveal our approach's superiority over existing alternatives and its great potential for enabling more versatile and intuitive AR/VR-like object manipulation within images.

Published as a conference paper at ICLR 2026
SPATIALHAND: GENERATIVE OBJECT MANIPULA- TION FROM 3D PRESPECTIVE
Zehan Wang', Jialei Wang'*, Siyu Chen', Ziang Zhang', Luping Liu?, Xize Cheng', Kaihang Pan', Hengshuang Zhao?, Zhou Zhao 1
Zhejiang University; The University of Hong Kong
https://spatiallie grthaaanngggttttu sithub.iol
6DoF Object Insertion
Ori
Any-Degree Rotation
Ori
Precise Translation
Occlusion-robust Swapping
Ori
Ori
Figure I: Illustration of how the "SpatialHand" can manipulate objects in images from 3D prespec- tive. With the basic ability to insert objects with 6DoF control (top row), allowing for any any-degree rotation (middle row), and precise 3D movement (bottom row).
ABSTRACT
We introduce SpatialHand, a novel framework for generative object insertion with precise 3D control. Current generative object manipulation methods primarily op- erate within the 2D image plane, but often fail to grasp 3D scene complexities, leading to ambiguities in an object's ' 3D position, orientation, and occlusion rela- tions. SpatialHand addresses this by conceptualizing object insertion from a true "3D perspective," " enabling manipulation with a complete 6 Degrees-of-Freedonn (6DoF) controllability. Specifically, our solution naturally and implicitly encodes the 6DoF pose condition by decomposing it into 2D location (via masked image), depth (via composited depth map), and 3D orientation (embedded into latent fea- tures). To overcome the scarcity of paired training data, develop automated data construction pipeline using synthetic 3D assets, rendering, and subject-driven
"Equal Contribution. Corresponding Author.

---

Published as a conference paper at ICLR 2026
generation, complemented by visual foundation models for pose estimation. We further design a multi-stage training scheme to progressively drive SpatialHand to robustly follow multiple complex conditions. Extensive experiments reveal our approach's 's superiority over existing alternatives and its great potential for enabling more versatile and intuitive AR/VR-like object manipulation within images.
INTRODUCTION
Manipulating an object within the 3D scene of images is fundamental capability in AR/VR envi-
ronments and real-world content creation workflows (Biener et al., 2020 Mendes et al. 201 9 Yu
et al. 2021 Besançon et al. 2021 Monteiro et al., 2021 Gardony et al. 2021 Zhang et al. 2025
Wang et al.
2025b 2023 Huang et al., 2024 Wang et al. al., 2025c; Linwei et al. 2026). Despite cur-
rent generative object insertion and movement methods (Xie et al. 2023 Yang et al. 2023 Huang
ctt al., 2025 Chen et al.
2024a Podell et al. 2023 Rombach et al. 2022 having achieved Signifi-
cant advancements in object identity preservation and contextual blending, they primarily operate in
the 2D plane and lack the understanding or control over the underlying 3D scene layout within the image.
In this work, we revisit generative object manipulation from a 3D perspective. Here, "3D per- spective" refers to inserting or moving an object with a specified 6DoF Pose (3D location and 3D orientation). forming correct spatial alignment and occlusion relation with other objects in the scene.
To enable efficient and effective object manipulation in 3D perspective, we propose SpatialHand, a practical solution that implicitly encodes an object's 6DoF pose. Although it's challenging for image generation models directly understand 3D location, we observe that they can effectively follow 2D masks and depth maps. Based on this, we represent the 3D location as combination of 2D position and depth. By introducing geometry-aware masked images and depth maps, we can precisely control object placement and ensure realistic occlusion relationships. Besides, the 3D orientation embedded into the latent features as additional guidance for object pose.
Despite our architecture modification enabling the model to natively encode specified 6DoF pose information, the lack of paired training data remains a challenge. To address this, we first leverage advanced 3D generation models (Yang et al. 2024c] Zhao al. 2025) to create large collection of high-quality 3D assets. Based on these assets, we introduce two strategies to complementarily pro- duce the target images: 1) Rendering the assets in virtual scenes using rendering engine (Blender]
2025). 2) Blending the assets with subject-driven image generation models (OpenAT 2025 Wu
2024 Ren al., 2024]] Yang al., 2024b a Wang et al. 2024b) Kirillov et al., 2023 Wang et al.
al. 2025). 3) we apply visual foundation models (Liu et al.
2025a
2026) to estimate the object's
6DoF pose from these synthetic images.
Given the multiple types of spatial guidance our model should follow, we design a multi-stage training scheme to progressively teach the model to understand and follow complex spatial cues. We initialize our model with a subject-driven generation model (Wu et al.) (2025) to inherit the identity preservation capabilities. Then, we first train on single-object images to drive the model generate novel views of objects with specified 3D orientation. Afterwards, we use multi-object complex scenes to learn further controllable 3D positioning and achieve realistic occlusion relationships.
We evaluate our method both quantitatively and qualitatively across various tasks and scenes. The re- sults demonstrate a new level of controllability in object insertion from the 3D perspective, enabling intuitive AR/VR-like object manipulation and several novel 3D-aware content creation applications.
In summary, our contributions are four-fold:
We highlight the "3D perspective" of the generative object manipulation task, aligning with how humans manipulate objects in the AR/VR environment. Our approach enables precise control over object 6DoF pose and eliminates the ambiguity in 2D image inpainting.
We decompose the 6DoF pose into 2D location, depth, and 3D orientation, allowing our model to naturally and implicitly encode spatial condition information.
We develop a stable and automated training data curation pipeline using high-quality syn- thetic 3D assets, rendering engine, and subject-driven image generation models.
2

---

Published as a conference paper at ICLR 2026
a) Task
b) Challenges
c) Solutions
Object Reference
Location Ambiguity
Depth Condition
"Whiee gull bull spa, yellow legs.
or
Scene Background
Orientation Ambiguity
Orientation Condition
or
Figure 2: Motivation of SpatialHand. 2D inpainting for object insertion/movemmnt suffers from location ambiguity (in front of or behind existing elements?) and orientation ambiguity (facing right or left?). SpatialHand resolves these by adding extra depth and orientation conditions for spatially controlled object manipulation.
We employ multi-stage, decoupled training approach that enables the model progres- sively adapt to complex spatial conditions, achieving robust spatial control.
RELATED WORK
2.1
GENERATIVE OBJECT MANIPULATION
With the advancement of generative models (Rombach et al. 2022, Podell et al. 2023 Peebles &
Xie 2023 Labs 2024). many methods now use impainting place objects into scene images, guided by visual or textual conditions. Text-guided inpainting methods (Zhang et al.) 2023a Avrahami al. 2022 Yu et al. 2023) can easily generate objects described in text within the inpainting mask of existing images. Paint-by-Example (Yang et al., (023) and AnyDoor (Chen et al.. 2024a) enable visual reference objects by using visual features as additional conditions. UniReal (Chen et al., 2024b) and ObjectMover (Yu et al.) 2025) take it a step further by leveraging prior knowledge of video generation models, using data from videos or game engines to achieve more realistic and plausible object replacement.
Although these methods perform well preserving object identity and environmental interactions, their reliance on simple 2D inpainting masks leads to ambiguity in the results. As shown in Fig, [ they cannot determine whether an inserted object should appear in front of or behind existing objects, or whether it should face left or right. To address this, our work considers the object manipulation task from the 3D perspective, enabling precise object insertion movement with a specified 3D
location and pose.
2.2 3D-AWARE IMAGE EDITING
Recently, several works Michel et al. 2023 Wu al. 2024 Yenphraphai al. 2024 Wang al.
2024a Pandey et al. 2024) have tried to edit and manipulate the content in 2D images from the 3D space, like location translation and object rotation. Object-3DIT (Michel et al. (023) collects synthetic dataset and learns 3D-aware image editing guided by language instructions. Diffusion Handles (Pandey et al. 2024) and Diff3DEdit (Wang et al. 2024a) convert 2D images into 3D point clouds to manipulate objects directly in 3D space. Image Sculpting (Yenphraphai et al., (024), the other hand, models the object's 'ss 3D mesh and performs edits on the mesh itself. These point cloud- mesh-based methods complicated techniques to maintain consistency in the resulting images, such as feature injection, DreamBooth tuning (Ruiz et al. 2023). ControlNet (Zhang et al. 2023b), and noise inversion (Song al. 2020).
Although existing methods offer explicit 3D control, they introduce significant complexity and la- tency, and often fail to capture the full geometric structure of the 2D image. For example, point

---

Published as a conference paper at ICLR 2026
Diffusion Transformer
Noise
VAE Encoder
Text Encoder
MLP
"White and grey gull, yellow bill with red spoz. yellowlags
Target Image
3D Location
Object Reference
3D Orientation
Figure 3: Overall pipeline of SpatialHand. We focus object insertion
primary task due to
its flexibility. SpatialHand decomposes the 6DoF object pose into 3D location (2D mask and depth map) and 3D orientation. These spatial conditions, along with free-vier object reference images and text captions, are incorporated into the diffusion transformer's input tokens.
clouds cannot represent the back side objects, limiting large rotations, while synthetic meshes struggle with reshaping occlusion relationships. In contrast, our method leverages implicit 3D guid- ance and a one-stage image generation model, significantly simplifying the whole process while supporting diverse and comprehensive 3D editing capabilities.
METHOD
Our main goal is place an object into image with specific 6DoF pose (3D location and ori- entation), and form realistic occlusion relationship. To this end, we modify the standard diffusion transformer model enable decomposed spatial condition input (Sec. curate large-scale train- ing data pairs (Sec. (3.2] and introduce progressive training scheme (Sec. 3.3) to effectively teach the model follow spatial guidance.
3.1 MODEL DESIGN
Preliminary Our framework is built upon the advanced open-sourced text-to-image model, FLUX-Dev.1 (Labs) (024), which employs the MM-DIT structure. The original MM-DiT block takes the concatenation of noisy image tokens X E NxD and text condition tokens C7 E R as input, where N, are the length image and text tokens, and D the latent dimension. The combined token sequence is projected into Query Q Key K, and Value V representations, and is globally interacted with the multi-modal attention mechanism:
QK" Vd
Attn([X, Cr.) Softmax
V
To repurpose the standard text-to-image generation model for object insertion, we first extend the input sequence with background and object reference conditions. Specifically, we embed the masked scene image Cmask and the reference object Cobj (provided as image and caption) into tokens with the pretrained VAE and text encoder, and concatenate them with the noisy image tokens additional conditional inputs. The results sequence can be formulate as X, Cuask: Cobjl.
3D Location Condition Using only the masked scene and reference object, as previous methods, provides limited 2D positional control. A simple 2D mask cannot determine object's precise location in actual 3D space, leading to both location and orientation ambiguities, as shown in Fig
To resolve these ambiguities and enable precise 3D positioning, we further introduce an additional composited depth map to explicitly define the desired insertion depth. We first predict the scene's depth map using Depth Anything (Yang et al. 2024b), then modify the depth values within the masked region to match the target placement. real practice, this target depth can user-frit seer-friendly and intuitively specified by simply clicking the desired ground position.
4

---

Published as a conference paper at ICLR 2026
Step Synthesis
Step Object Placement
Step 3: Information Annolation
Dogg
Rendering Engine
Grounding DINO
"Boy" "Car
Hunyuan-3d & QwenVL
Segment Anything
2D Location
Depth Map 3D Orientation
Depth Anything
"Sofa"
Subject-driven Generation
Orient Anything
Figure 4: Pipeline of training data curation. We start with high-quality synthetic 3D assets. Using rendering engine and subject-driven generation, simulate how humans place objects in 3D space. Then, we employ series of visual foundation models to estimate the 3D information within images.
To preserve realistic occlusions and foreground consistency, further propose geometry-awant composition. By comparing the scene's depth map and the intended placement depth, we identify masked foreground objects that should remain visible. These objects, in front of the insertion place, are preserved in both the masked scene and depth map, ensuring correct occlusion (i.c.. foreground objects stay front of the inserted object).
In summary, our method extracts the scene's 's depth map, marks the depth value of the desired location on it. Besides, we further geometry-awand composition refine the occlusion relationship and masked image Cmask and depth map Cdepth- The input token sequence further extend to [ Crusko Cdepth: Cobyl.
produce the geometry-
3D Orientation Condition After determining the 3D location, method additionally enables specification of the object's s 3D orientation to achieve full 6DoF controllability. Previous novel view synthesis methods (Liu et al., 2023 Wu et al. 2024) primarily operate through relative rotations from reference views, limiting their effectiveness textual-only scenarios. In contrast, we focus on modeling semantic object orientation that aligns with an object's canonical front-facing direction. Our method incorporates absolute target orientations as direct conditions rather than relying on relative rotation.
We describe the 3D orientation of an object with three parameters: azimuth yp, elevation 0 and in- plane rotation 8, following Orient Anything (Wang al., 2024b 2026). To introduce the orientation parameters as a condition for the diffusion transformer, we apply a zero-initialized MLP projector P(-) map them into the latent dimension and add them into the object reference conditions. The final input token sequence can formulated as [X, Čaask:
Cdepth: Cobj P([s,, 0))]].
3.2 TRAINING DATA CURATION
For placing object with specific 6DoF pose, ideal training data pairs should cover: 1) Object Con- dition: textual caption and visual image from difference view for the reference object, 2) Targer Image: target scenes with placed object, and 3) 6DoF Guidance: object location, depth and ori- entation annotations. The lack of paired data is another obstacle for 3D-aware object replacement. To overcome this, develop an automated data curation pipeline that simulates real-world object placements through three steps:
Step 3D Object Assets Synthesis In the real world, the first step of placing objects obtaining the actual 3D- 3D-form object. Inspired by recent advances in 3D generation models (Yang et al.) 2024cc Zhao et al. 2025 Xiang et al.) 2024). we synthesize comprehensive data coverage. With the high-quality 3D mesh, the multi-view renderings can serve as visual object conditions, and associated captions are native textual object conditions. Specifically, we leverage Hunyuan-3D 2.0 (Zhao et al.) 2025) to generate 43k 3D assets from 10k common object category names, render 20 random views for each object, and employ Qwen-2.5-VL (Bai et al. 2025) to produce corresponding captions.
object assets from category names for
Step 2: Object Placement Simulation To mimic the process of placing 3D objects in the real world, design two complementary object placement methods: Simulated Placement: Using
5

---

Published as a conference paper at ICLR 2026
we randomly arrange and render 3D assets to generate diverse 6DoF poses and occlusion patterns across scenes. This approach ensures perfect object identity preservation but sacrifices scene diversity and complexity. 2) Generative Placement: Leveraging subject-driven image generation models, UNO (Wu et al. 2025) and ChatGPT-40 (Ope-) nAl 2025), blend multiple objects into realistic scenes with natural arrangements. This method enhances contextual diversity and plausibility, albeit with slightly weaker identity consistency. By combining the two complementary strategies, we balance geometric and identity precision (Blender) with rich environmental variation (generative models), providing scalable, diverse, and reliable tar- get images.
the Blender simulation platform (Blender 2025
Step 3: 3D Information Estimation Given the visual and textual reference object along with the larget image containing the reference object, the final step involves estimating the object's spatial pose within the target image as 6DoF guidance. First, we use Grounding-DDin -Diin (Liu et al. 2024) and Segment Anything (Kirillov et al. 2023) to detect the 2D position (bounding box and segmentation mask). Second, the target image" s depth map is predicted using Depth Anything (Yang et al. 2024b), and the placed objects' average depths are computed within its segmented region. Finally, we infer the 3D orientation of the reference object using Orient Anything (Wang al. 2024b 2026).
Statistic We collect 43k diverse 3D assets spanning a wide range of object categories. Using sim- ulated placement and generative placement techniques, we generate 100k and 450k target images, respectively. To ensure data quality, we implement a rigorous filtering process that removes: (1) samples with low DINO similarity scores to reference objects and (2) cases with low bounding box or orientation confidence. This quality control process results in 370k high-fidelity object-image- pose training pairs.
3.3 TRAINING SCHEMES
Achieving generative 3D-aware object placement and manipulation necessitates that the model ad- here to several conditions: object identity, 2D location, depth map, and 3D orientation. To facilitate robust learning amidst these intricate constraints, we introduce progressive training scheme.
Stage O: Prrentty-rresevattion Pre-training Object identity preservation is the most fundamen- tal ability of our tasks. Therefore, we initialize our model with the pre-trained FLUX-1 dev (Labs) 2024) model and the LoRA adapter (Wu et al.) 2025) trained for subject-driven generation tasks, ensuring the model inherently possesses object identity preservation capabilities from the beginning of training.
Stage 1: Novel View Synthesis Fine-Tuning Then focus training the model comprehend object orientations and generate novel views with specific 3D orientations. To this end, we randomly select two renderings of the same 3D object as object reference and target images, and incorporate estimated orientation predictions from Orient Anything as guidance for training.
Stage 2: 3D-aware Insertion Fine-Tuning Finally, we refine the model's ability to realistically insert objects into scenes with specified poses. Using the dataset introduced in Section 3.2 the model to accurately position objects while maintaining background.
we train
Implementation Details We initialize our model using the pre-trained FL.UX-1 dev (Labs 2024)
framework, along with the LoRA adapter provided in UNO Ww et al. 2025). For stages and 2 in Section 3.3 employ the AdamW (Loshchilov & Hutter 2017) optimizer with a learning rate of le-4 and batch size of 8, using a cosine scheduler. We utilize LoRA (Hu et al. 2022) with the rank of 512 for fine-tuning, and the model is trained for 60k steps in Stage and 20k steps in Stage 2. All the experiments are conducted on 8 x A100 GPUs.
4 EXPERIMENTS
4.1 3D-AWARE OBJECT INSERTION
Alternative Baseline Considering there is no existing object insertion method that supports pose
condition, select the advancing image editing model, GPT-4o (OpenAl 2025), Gemini-2.0-

---

Published as a conference paper at ICLR 2026
Textull Condition
Textual-only Condition
Objective
Suhjective
Objective
Subjective
DINOT AbsRell Acc Fidelity? Adherence
CLIPt AbsRell Acci 30"1 Fidelity? Adherencef
Bermini-2.0-Flaae 81.2
33.2
16.0
3.51
2.10
65.7
32.3
18.6
4.10
1.87
GPT-4o 80.5
38.6
20.2
4.13
2.67
66.1
47.5
19.1
4.31
2.13
Nano Banana
80.9
32.1
17.5
3.63
2.25
66.5
32.5
19.5
4.25
1.97
SpatialHand 81.7
19.8
52.0
4.27
72.5
18.7
47.0
4.25
4.53
Table 1: Quantitative comparison results 3D-aware object insertion. Object similarity in the generated image to the reference measured using DINO1 and CLIPt. AbsRell evaluates the accuracy of the inserted object's 'ss depth position, and Acc 30"1 measures its orientation accuracy.
Visual Textual Condition
Textual-onl Condition
modde
Gemini
Nano
Gemini
Nano Banama
Object
Object
Banana
Qualitative comparison on 3D-aware object insertion. Red arrow indicates the desired orientation, and green arrow denotes the left side of the corresponding pose. 7oom in for best view.
Figure 5:
Flash (Google 2025b) and Nano Banana (Gemini-2.5: 2.5-Flash Image) Gooole 2025a). as our base- lines. To incorporate pose guidance, we input the background scene, composited depth map, and reference object into the unified image generation models, along with a detailed instruction specify- ing the desired 3D location and orientation. Detailed examples are provided in the Appendix.
Benchmark Specifically, we manually select 20 images (including both real and synthetic sam- ples) scene images, and choose 20 high-quality 3D objects from Objaverse (Deitke et al.) 20233 as reference objects. We use the renderings and captions from Qwen2.5VL of 3D assets as visual and textual object conditions, respectively. For each scene image, we manually annotate two suitable poses as insertion targets. Our benchmark contains 800 test samples for different kinds of object conditions input (visual+textus) and textual-only). amounting 1,600 test samples in total.
Evaluation Metric For identity preservation, compute the DINO (Oquab et al. 2023 feature similarity between the result image and the rendering under the target pose of the reference object, and the CLIP (Radford et al.) 2021) feature similarity between the result image and the textual conditions. For depth alignment, we compute the absolute relative depth error (AbsRel) between the target depth and the inserted object's depth. For orientation alignment, within a 30 angular error tolerance (Aco@ 30") between the target and the actual orientation of the inserted object. We also conduct human subjective study, where annotators rate each result image from to 5 based on two criteria: "Object Fidelity" and "Pose Adherence".
calculate the accuracy
Main Results Comparative results are provided in Tab. and Fig. 5 First, SpatialHand shows state-of-the-art object ID consistency in both visual+textual and textual-only inputs. More impor- tantly, objects inserted by our method adhere better to the specific pose regarding depth and orienta- tion (lower AbsRel and higher Ace 30%). Even the most advanced instruction-bast model. GPT-40, struggles to understand and maintain the desired spatial state from textual instruc- tion, highlighting the necessity of our carefully designed 3D location and orientation conditions.
image editing

---

Published as a conference paper at ICLR 2026
Rotation
Translation
Occlusion
Objective
Subjective
Objective
Subjective
Objective
Subjective
Acc 30
Fidelity?
Adherenoef
mloUt
AbsRell
Fidelity?
Adherenoef
VLM-Aoct
Fidelity?
Adherencef
ObJoct3DiT
31.4
2.78
3.22
0.45
35.4
2.36
3.56
53.2
1.67
2.31
Diffusion Handles
20.7
3.01
0.28
2.76
3.34
49.7
1.78
2.05
Gemini-2.00 2.0-Flas Flash
18.3
3.43
2.56
0.22
27.3
3.27
2.65
49.2
3.13
2.67
GPT-4o
3.74
3.16
Nana Banana
20.5
3.53
2.73
0.25
26.3
3.53
2.72
50.1
3.21
2.78
Saat
47.8
3.89
4.17
0.72
17.9
4.12
4.56
82.6
3.57
4.53
Table 2: Quantitative comparison on 3D-aware object movement.
Diffusion Handles
Gemini 2.0 Flash
Nano Banana
Scene
Ours
Object3DIT
GPT-4o
Rotation
Translation
Occlusion
Figure 6: Qualitative comparison on 3D-aware object movement.
4.2 3D-AWARE OBJECT MOVEMENT
Alternative Baseline For 3D-aware object movement task, we compare several 3D-aware image editing methods, including Object3DiT (Michel et al.) 2023) and Diffusion Handles (Pandey et al.) 2024), which support 3D object rotation and translation in existing images. Additionally, we also in- clude advanced unified instruction-bassee
image editing models, Gemini-2.0-Flash (Google) 2025b), GPT-4o (OpenAl 2025) and Nano Banana 2ecmini-2.5-Flastt Image) (Google 2025a), as baselines.
Our SpadialHand method achieves 3D movement by following two processes: removing the object, then inserting it back with the target pose.
We divide 3D-aware object movement into three sub-tasks: 1) horizontal rotation, 2) 3D translation, and 3) handling occlusion (moving objects between foreground and occluded posi- tions). For each sub-task, we collect 50 images and annotate manipulations per image, resulting in 100 samples per sub-task and 300 samples in total.
Benchmark
Evaluation Metric
We measure accuracy within 30 angular error (Acc @ 30%) with Orient Any- thing, and translation adherence, average box IoU (mloU) and absolute relative depth error (AbsRel), using DINO-grounding and Depth Anything. For occlusion, advanced VI.M, Qwen2.5V1 (Bai al.)
8

---

Published as a conference paper at ICLR 2026
Row
Geometry
Training stage
Input condition
DINO
AbsRell
Acc@30%1
1
Yes
Stage 1&2
Visual+Texxee
81.7
19.8
52.0
2
Yes
Stage &&2
Visual Only
78.8
18.5
46.8
3
No
Stage 1&2
Textual
80.2
25.5
48.8
4
Yes
Stage 2
Textual
79.8
18.2
28.7
Table 3: Ablation study on 3D-aware object insertion task. "Geometry" indicates whether geometry-
aware composition used for 3D location condition (Sec. 3.T. "Training Stage" refers different training strategies (Sec. 3.3
"Input Condition" represents different forms of object reference.
Object
Scene
w/o Geometry-aware Composition
with Geometry-avare Composition
Figure 7: Effect geometry-awee composition. We sequentially present the masked scene, depth map, and synthesized images, with and without applying geometry-awart composition.
2025,, tasked check occlusion correctness (VLM-Acc). Human studies about "Object Fidelity" and "Pose Adherence" are also employed.
Main Results Tab. presents the quantitative results for different 3D-aware object movement tasks. Our model exhibits superior advantages each sub-task. On representative subjective metrics, our method show significantly better object fidelity and spatial state adherence compared to both specialized 3D-aware image editing methods and general image editing methods. Fig. further provides qualitative comparisons of different methods on various subtasks. Methods that explicitly operate on point clouds, like Diffusion Handlers, struggle with large rotations and occlusions due missing back structure in monocular point clouds. Furthermore, methods relying on textual instruc- tions often fail describe the complex spatial editing intentions. In contrast, our method implicitly encodes the desired spatial state, enabling diverse 3D-aware editing while avoiding ambiguity.
4.3 ABLATION STUDY
Impact of Textual Condition First, we examine the effect the textual object condition. Com- paring rows 1 and in Tab, fssows that extra caption information significantly helps maintain object identity during insertion. Considering that VLMs perform well and are widely used for captioning tasks, using VLM captions for visual-only scenarios is practical choice for real-world applications.
Geometry-awand aware Composition Fig. visualizes the geometry-aware composition process de- seribed in Sec. 3.1 It shows that the geometry- aware masked image and depth map better capture occlusion between inserted and existing objects, and also preserve foreground object identity. More- over, the higher depth error of row compared geometry-aware composition is effective for controlling object position.
1 in Tab. quantitatively demonstrates that
Progressive Training Scheme Row 4 in Tab. shows the result of removing stage from progressive training (Sec. [33).. We observe that skipping the fine-tuning for novel view synthesis markedly harms the model's accuracy, Ace@ 30% This confirms our progressive training scheme's effectiveness in helping the model understand object orientation and handle complex conditions.
capability, as indicated by the lower orientation
CONCLUSION
We introduce SpatialHand to virtually insert objects in existing images, operating from "3D perspective." uniquely encodes the 6DoF object pose as 2D location, depth, and 3D orien- tation cues. To overcome data scarcity and ensure robust adherence complex spatial conditions,

---

Published as a conference paper at ICLR 2026
develop automated data curation pipeline and progressive training scheme. Extensive periments demonstrate SpatialHand's superiority over existing approaches, showcasing its strong potential to advance more intuitive and versatile VR/AR-like object manipulation within images.
ACKNOWLEDGEMENTS
This work was supported by National Natural Science Foundation of China under Grant NN.66252422, No.624B2128, No.62422606), Hong Kong Research Grant Council General Re- search Fund (N0.17213925) and State Grid Zhejiang Electric Power Cooperation Technology Project (No.B311DS240012)
REFERENCES
Omri Avrahami, Dani Lischinski, and Ohad Fried. Blended diffusion for text-driven editing of natural images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 18208-18218, 2022.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, al. Qwen2. 5-vl technical report. arXiv preprint 2332 2025.
Lonni Besançon, Anders Ynnerman, Daniel Keefe, Lingyun Yu, and Tobias Isenberg. The state of the art of spatial interfaces for 3d visualization. In Computer Graphics Forum, volume 40, pp- 293-326. Wiley Online Library, 2021.
Verena Biener, Daniel Schneider, Travis Gesslein, Alexander Otte, Bastian Kuth, Per Ola Kris- tensson, Eyal Ofek, Michel Pahud, and Jens Grubert. Breaking the screen: Interaction across touchscreen boundaries in virtual reality for mobile knowledge workers. IEEE transactions on visualization and computer graphics, 26(12):3490-3502, 2020.
Blender. Blender, 2025. https:://// www http:://wwwwwweee..c.. www.beeeeder.oggg.
Xi Chen, Lianghua Huang, Yu Liu, Yujun Shen, Deli Zhao, and Hengshuang Zhao. Anydoor: Zero- shot object-level image customization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 6602, 2024a.
Xi Chen, Zhifei Zhang, He Zhang. Yuqian Zhou, Soo Ye Kim, Qing Liu, Yijun Li, Jianming Zhang, Nanxuan Zhao, Yilin Wang, et al. Unireal: Universal image generation and editing via learning real-world dynamics. arXiv preprint 2xXxv:242..07744 2024b.
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of anno- tated 3d objects. Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 13142-13153, 2023.
Aaron L Gardony, Shaina B Martis, Holly A Taylor, and Tad T Brunyé. Interaction strategies for effective augmented reality reo-visualization: Insights from spatial cognition. Sllman-Computte Interaction, 36(): 2021.
Google. Gemini-2.5-flash. 2025a.
httess//aistuuoooooollleeo/oooot
-emini--2-S--lahh-imne http::///istaaao..oooolee/////op
Google. Gemini-2.0-flashh 2025b.
chat wwwddmmmooeiiiii.e 2-0- exp
Edward J Hu, Yelong Shen. Phillip Wallis, Zeyuan Allen-Zhu. Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3 2022.
Haifeng Huang, Yilun Chen, Zehan Wang, Rongjie Huang, Runsen Xu, Tai Wang, Luping Liu, Xize Cheng, Yang Zhao, Jiangmiao Pang, et al. Chat-scene: Bridging 3d scene and large language models with object identifiers. Advances in Neural Information Processing Systems, 37:1 11:111991-- 114017, 2024.
10

---

Published as a conference paper at ICLR 2026
Yi Huang, Jiancheng Huang, Yifan Liu, Mingfu Yan, Jiaxi Lv, Jianzhuang Liu, Wei Xiong, He Zhang, Liangliang Cao, and Shifeng Chen. Diffusion model-based image editing: A survey. IEEE Transactions Pattern Analysis and Machine Intelligence, 2025.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Yen Lo, et al. Segment anything. In Proceed- ings of the IEEE/CVF international conference on computer vision, pp. 401 4015-4026, 2023.
forectt-fooott-to Black Forest Labs. Flux. htttp:://gitub..oom fott:s////.ttaaa...ooooooooaoaoa
2024.
Chen Linwei, Fu Ying, Shangguan Jingyu, Xu Jinglin, and Peng Yuxin. A survey on degraded im- age segmentation. Chinese Journal of Electronics, 35:1-21, 2026. doi: 10.23999/cee.2225.00.. 245. URL
2025. 00. 245]
Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl Vondrick. 1eto-1- Zero-shot one image to 3d object. In Proceedings of the IEEE/CVF international conference on computer vision, pp.
2023.
Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European Conference Computer Vision, pp. 38-55. Springer, 2024.
llya Loshchilov and Frank Hutter. 1711v:1111.05 2017.
Decoupled weight decay regularization. arXiv preprint
Daniel Mendes, Fabio Marco Caputo, Andrea Giachetti, Alfredo Ferreira, and Joaquim Jorge. A survey on 3d virtual object manipulation: From the desktop to immersive virtual environments. In Computer graphics forum, volume 38, pp. 21-45. Wiley Online Library, 2019.
Oscar Michel, Anand Bhattad, Eli VanderBilt, Ranjay Krishma, Aniruddha Kembhavi, and Tanmay Gupta. Object 3dit: Language-guided 3d-aware image editing. Advances in Neural Information Processing Systems, 36:3497-3516, 2023.
Pedro Monteiro, Guilherme Gonçalves, Hugo Coelho, Miguel Melo, and Maximino Bessa. Hands- free interaction immersive virtual reality: A systematic review. IEEE Transactions on Visual- ization and Computer Graphics, 27(5) 2021.
OpenAl.
Gpt-4o, 2025.
http:///penaa..com/ii
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023.
Karran Pandey, Paul Guerrero, Matheus Gadelha, Yannick Hold-Geoffroy, Karan Singh, and Niloy J Mitra. Diffusion handles enabling 3d edits for diffusion models by lifting activations to 3d. Pro- ceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 7695- 7704, 2024.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the JEEE/CVF international conference on computer vision, pp. 4195 4205, 2023.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhom, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint 2rXiv:2307.01952, 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- PmLR, 2021.
11

---

Published as a conference paper at ICLR 2026
Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, et al. Grounded sam: Assembling open- open-world models for diverse visual tasks. arXiv preprint arXiv:2401.14155 2024.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- on computer vision and pattern recognition, pp. 10684- 10684-10695, 2022.
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Pro- ceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 22500- 22510, 2023.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502 2020.
Ruicheng Wang. Jianfeng Xiang, Jiaolong Yang, and Xin Tong. Diffusion models are geometry critics: Single image 3d editing using pre-trained diffusion priors. European Conference on Computer Vision, pp. 441-458. Springer, 2024a.
Zehan Wang, Haifeng Huang, Yang Zhao, Ziang Zhang, and Zhou Zhao. Chat-3d: Data- efficiently tuning large language model for universal dialogue of 3d scenes. arXiv preprint vrXiv:2308.08744 2023.
Zehan Wang, Ziang Zhang, Tianyu Pang, Chao Du, Hengshuang Zhao, and Zhou Zhao. Orient anything: Learning robust object orientation estimation from rendering 3d models. arXiv preprint arXiv:2412. 18605, 2024b.
Zehan Wang, Siyu Chen, Lihe Yang, Jialei Wang, Ziang Zhang, Hengshuang Zhao, and Zhou Zhao Depth anything with any prior. arXiv preprint arXiv:2505. 10565, 2025a.
Zehan Wang, Jiayang Xu, Ziang Zhang, Tianyu Pang, Chao Du, Hengshuang Zhao, and Zhou Zhao. Genspace: Benchmarking spatially-aware image generation. arXiv preprint arXiv:2505.24870, 2025b.
Zehan Wang. Sashuai Zhou, Shaoxuan He, Haifeng Huang, Lihe Yang. Ziang Zhang, Xize Cheng. Shengpeng Ji, Tao Jin, Hengshuang Zhao, et al. Spatialclip: Learning 3d-aware image representa- tions from spatially discriminative language. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 29656-29666, 2025c.
Zehan Wang, Ziang Zhang, Jiayang Xu, Jialei Wang, Tianyu Pang, Chao Du, Hengshuang Zhao and Zhou Zhao. Orient anything v2: Unifying orientation and rotation understanding. arXiv preprint
arXiv:2601.05573 2026.
Shaojin Wu, Mengqi Huang, Wenxu Wu, Yufeng Cheng, Fei Ding, and Qian He. Less-to- more generalization: Unlocking more controllability by in-context generation. arXiv preprint arXiv:2504.021660 2025.
Ziyi Wu, Yulia Rubanova, Rishabh Kabra, Drew Hudson, Igor Gilitschenski, Yusuf Aytar, Sjoerd van Steenkiste, Kelsey Allen, and Thomas Kipf. Neural assets: 3d-aware multi-object scene synthesis with image diffusion models. Advances in Neural Information Processing Systems, 37: 76289-76318, 2024.
Jianfeng Xiang, Zelong Lv, Sicheng Xu, Yu Deng, Ruicheng Wang, Bowen Zhang, Dong Chen, Xin Tong, and Jiaolong Yang. Structured 3d latents for scalable and versatile 3d generation. arXiv preprint arXiv:2412.01506, 2024.
Shaoan Xie, Zhifei Zhang, Zhe Lin, Tobias Hinz, and Kun Zhang. Smartbrush: Text and shape guided object inpainting with diffusion model. Proceedings of the IEEE/CVF conference on computer vision and patlern recognition, pp.
2023.
12

---

Published as a conference paper at ICLR 2026
Binxin Yang, Shuyang Gu, Bo Zhang, Ting Zhang, Xuejin Chen, Xiaoyan Sun, Dong Chen, and Fang Wen. Paint by example: Exemplar-bar based image editing with diffusion models. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 18381-18391, 2023.
Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10371-10381, 2024a.
Lihe Yang. Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. Advances in Neural Information Prucessing Systems, 37:21875-21911,, 2024b.
Xianghui Yang, Huiwen Shi, Bowen Zhang, Fan Yang, Jiacheng Wang, Hongxu Zhao, Xinhai Liu, Xinzhou Wang, Qingxiang Lin, Jiaao Yu, et al. Hunyuan3d 1.0: A unified framework for text-to- 3d and image-to-3d generation. arXiv preprint 2xXiv:2411.022 1.02293, 2024c.
Jiraphon Yenphraphai, Xichen Pan, Sainan Liu, Daniele Panozzo, and Saining Xie. Image sculpting: Precise object editing with 3d geometry control. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patten Recognition, pp. 4241-4251. 2024.
Difeng Yu, Xueshi Lu, Rongkai Shi, Hai-Ning Liang, Tilman Dingler, Eduardo Velloso, and Jorge Goncalves. Gaze-supported 3d object manipulation in virtual reality. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1-13, 2021.
Tao Yu, Runseng Feng, Ruoyu Feng, Jinming Liu, Xin Jin, Wenjun Zeng, and Zhibo Chen. Inpaint anything: Segment anything meets image inpainting. arXiv preprint arXiv:2304. 000900099 2023.
Xin Yu, Tianyu Wang, Soo Ye Kim, Paul Guerrero, Chen, Qing Liu, Zhe Lin, and Xiaojuan Qi. Objectmover: Generative object movement with video prior. arXiv preprint arX0v:2503.08037, 2025.
Kai Zhang, Lingbo Mo, Wenhu Chen, Huan Sun, and Yu Su. Magicbrush: A manually annotated dataset for instruction-guided image editing. Advances in Neural Information Processing Systems, 36:3142--31449 2023a.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 3836-3847, 2023b.
Ziang Zhang. Zehan Wang, Guanghao Zhang, Weilong Dai, Yan Xia, Ziang Yan, Minjie Hong, and Zhou Zhao Dsi-bench: benchmark for dynamic spatial intelligence. arXiv preprint 1rXiv:2510.18873 2025.
Zibo Zhao, Zeqiang Lai, Qingxiang Lin, Yunfei Zhao, Haolin Liu, Shuhui Yang, Yifei Feng. Mingxin Yang. Sheng Zhang, Xianghui Yang, et al. Hunyuan3d 2.0: Scaling diffusion models for high resolution textured 3d assets generation. arXiv preprint 1222, 2025.
13

---

Published as a conference paper at ICLR 2026
A
MORE VISUALIZATION COMPARISON
Visual-Vextual Textual Condition
Textual-only Condition
dnd ealp sau nendow
vightet
WIN
Jalle and A
Tyrke atpler eedonding pear
by
Gemini
Nano
Gemini
Nano Banana
Object
Banana
Object
Scene
Ours
Figure 8: More qualitative comparison on 3D-aware object insertion
More visualization results on 3D-aware object insertion are shown in Figure 8 It shows that our method achieves excellent performance both 3D position and orientation control during object insertion, while maintaining the consistency of the inserted objects. In contrast, GPT-4o and Gemini- 2.0-Flash perform poorly in terms of orientation control and background preservation.
Figure 9 shows more visualization results about object movement. Our method can perform better in rotation, translation, and occlusion handling, while the other four models show lower success rates in operations. Diffusion Handles and GPT-4o perform poorly in background preservation. Gemini-2.0 demonstrates a lack of understanding of bounding boxes.
B PROMPT FOR GPT-40, SAMINNI-.00-LASSE AND NANO-BANANA
For fair evaluation of GPT-4o and Gemini-2.0 2.0h-sash's performance, we use the following prompt for object insertion: "Insert the item in the second picture into the first picture(background) and depth map is provided as the third picture, the boxed area's depth the average depth the item should be placed, the caption of the item is {caption]. the item should orient camera with azimuth [orientation[O], polar [orientation[]]} rotation [orientation(2]), ccommon knowledge' }r where "comon knowledge' includes the coordinate system and definitions of the three rotation angles. This prompt contains all the information that we input into our model.
For GPT-4o, utilize the mask functionality in the API to constrain the regions where objects be modified during insertion. However, from the results, GPT-4o does not strictly follow this mask, exhibiting cases where objects extend beyond the bounding box and background changes occur.
For object movement, we draw the bounding box of the object in the background to indicate the target position with the prompt: "Move the item in the picture the new position, which is marked by the red box. Depth map is provided as the third picture, the boxed area's depth is the average depth of the item should be placed. The caption of the item is (caption), the item should orient to camera with azimuth {orientation(O]), polar {orientation[1]] rotation (orientation[2] "common knowledge") }"
C QUANTITATIVE ABLATION STUDY
For the ablation study of Geometry aware Composition, have shown the results in Figure 7. Figure 10 results for the ablation study of the Impact Textual Condition and Progressive Training Scheme.
14

---

Published as a conference paper at ICLR 2026
Diffusion
Nano
Scene
Ours
Object3DIT
GPT-4o
Gemini-2.0
Handles
Banana
Rotation
Translation
Occlusion
Figure 9: More qualitative comparison on 3D-aware object insertion
15

---

Published as a conference paper at ICLR 2026
Ablation Study
Object
Scene
Ours
w/o Text
w/o Stage I
Figure 10: More qualitative comparison on 3D-aware object insertion
The results show that Textual Condition plays crucial role in maintaining object consistency when changing object orientation. Without the Textual Condition, the features of the inserted object can- not be fully preserved. Additionally, single-stage training shows poor performance in orientation control, which demonstrates the effectiveness of our Progressive Training Scheme.
D MORE VISUALIZATION UNDER CHALLENGING SCENARIOS
We provide more visualization results for object movement involving human and artistic subjects in Figure I and object insertion in scenes with more diverse lighting conditions in Figure [2] The results show that even though our method was not specifically trained for these situations, still demonstrates generalization ability these challenging cases.
E USAGE OF LARGE LANGUAGE MODELS
We utilized Large Language Models (LLMs) for sentence-level refinement during the drafting of this manuscript. Our experiments featured several key baselines that are Multimodal Large Language Models (MLLMs), including GPT-4o, Gemini-2.0-flast and Nano-Banana. Section thoroughly explains how we prompted and used these MILMs.
F LIMITATION AND FUTURE WORK
Under the current paradigm, an object's orientation and its 2D bounding box are somewhat inter- dependent. The box's 's aspect ratio should generally match the object's expected shape under the specific orientation. For instance, car's side view requires a wider inpainting box than its front view. A significant conflict between these can make harder for the model to apply the orienta- tion correctly. Making location and orientation conditions more independent is meaningful future direction.
16

---

Published as a conference paper at ICLR 2026
Figure 11: Qualitative results of moving human and artistic subjects.
Figure 12: Qualitative results under different and challenging lighting conditions.
17

---

Published as a conference paper at ICLR 2026
Scene
LoossControl
Ours
Figure 13: Qualitative results against depth-based methods. Visual comparison demonstrating the inherent ambiguity of the Box Depth Map. Our method accurately executes the 3D) rotation while preserving the object's s structure.
Scene
DragDiffusion
Drag Your Noise
Ours
Figure 14: Qualitative results against drag-based methods. Visual comparison showing that drag- based methods tend to cause object deformation when moving objects. Our method successfully performs realistic object translation while maintaining structural integrity.
18


Reviews:
[{"review_id": "nCOXR5FjGT", "reviewer": "Reviewer_bQWY", "summary": "SpatialHand proposes a practical, single-stage diffusion-transformer framework for 3D-aware object insertion and movement in images by decomposing an object’s 6DoF pose into three conditions: (1) 2D location via a geometry-aware masked scene, (2) target depth via a composited depth map, and (3) 3D orientation embedded as latent tokens (azimuth/elevation/in-plane rotation). They add additional tokens to a pretrained FLUX model, using LoRA to adapt to additional conditions. The authors build a large paired training corpus via (a) synthetic 3D assets + renderer and (b) generative placement using subject-driven models, then use visual foundation models to estimate pose for supervision. They train progressively (identity pretrain → novel-view fine-tune → insertion fine-tune) and show quantitative and qualitative gains for both insertion and movement tasks versus several baselines.", "questions": "See weaknesses", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The method presented is elegant and solves critical problems for object manipulation: 3D translation and rotation + in-context generation. \n* The scalable data generation pipeline using pretrained 3D model generators + traditional rendering is clearly effective for generating varied data", "weaknesses": "* Missing evaluation: I believe the authors should evaluate against LooseControl (SIGGRAPH 2024) and Build-a-Scene (ICLR 2025) which also can move/rotate objects in 3D space. \n* I believe the DragDiffusion line of work [CVPR 2024], more recently Dragin3D [CVPR 2025] can also rotate objects and should be evaluated. \n* This one is less important, but a recent frontier model Reve.ai can also move objects by placing 2D boxes and could be part of the evaluation. \n* The method can handle simple objects, but not humans or stylized artwork (according to the results and nature of the training data). This point should be discussed in the limitation. \n* Basic shadows of the rendered object look good, but the object tends to stick out/not always harmonize lighting-wise. Could natural lighting harmonization be evaluated? Could results be presented on diverse/colored lighting conditions? The current scenes all have daytime lighting that is similar to the object’s studio lighting.\n* Could practical runtime speed/memory footprint be reported?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4TgiX7sLva", "reviewer": "Reviewer_9QNt", "summary": "This paper presents a method that enable object insertion with 3D control.\nThe main contribution from my point of view is the dataset curation pipeline and the ability to achieve better result of 3D control.", "questions": "- I think overall, composition is a difficult task with a long history in computer graphics and vision. For inserting an object, while the pose is vital, the appearance is also important. Have the authors considered how to better improve the composited appearances? For example, by inputting additional conditions? I think this is not critical but worth discussing in the paper.\n- I am curious about how different datasets affect the results? In the paper, there are both simulated placement and generative placement datasets. But it is not clear shown and discussed why both datasets are needed.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The dataset generation process is reasonable and serve the purpose of the proposed method well.\n- The proposed method is straightforward and easy to understand.\n- The demonstrated results seem better than other existing methods.", "weaknesses": "- The need of additional training is a bit complicated compared to recent training-free method.\n- The computational overhead compared to original base model (e.g., FLUX) is not discussed properly.\n- The composited objects sometimes have weird projections or appearance and did not match the scene well, e.g., the airplane in Fig. 5. Regarding that case, I feel the composited results generated by GPT-4o is more convincing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "D1qkjVqZjz", "reviewer": "Reviewer_XPg4", "summary": "The paper proposes a method for image manipulation to control the 6DoF of objects appearing in it. To do so, a transformer diffusion model is modified to take a combination of depth and orientation control signals. First, the method extracts the scene depth using DepthAnything, and computes the occlusion on the desired edited region, determining the location of the 3D object. The object orientation is described with three parameters (azimuth, elevation, in-plane rotation), and added to the object reference condition. To train the transformer, the paper proposes to rely on 3D generative AI data, render the assets in images and generate captions, and then rely on rendering engines' simulations and generative AI placement to synthesize realistic scenes. Final 3D information (depth of generated scene, 2D masks, and positions) is obtained by piping ad-hoc foundational models. Experiments are conducted on 1600 test samples in total, and evaluated using AI-based metrics (e.g., DINO and CLIP similarity) as well as a human user study, surpassing the competitors.", "questions": "- Will the code be released?\n- How long does it take for the training in wall-clock time?\n- Could you provide further details on the training, e.g., what are the losses used for? How are the different stages balanced?\n- Is the material of the assets considered in the training set creation through Blender? How does the method perform for objects with high reflectance/specularity?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper provides a careful combination of different foundational models to generate data, showing impressive results in terms of consistency.\n- The paper is well presented and clear\n- The data, benchmark, and method could be useful for several downstream tasks and of interest for general ICLR audience", "weaknesses": "- The paper relies on an intricate combination of different tools, and not all the details are reported. I believe replicating the results could be particularly challenging, and code release is not mentioned in the paper. The method also requires a non-negligible computational effort for training, as it uses 8xA100 GPUs\n\n- The papers rely on a test set that also uses generated background, DINO features, and other foundational models as judges to assess the consistency of the provided generation. However, since the method itself heavily relies on generative AI, I would argue that an evaluation using more grounded geometrical data would be informative. For example, relying on synthetic rendering or lightstage captured objects could provide an exact ground truth on how the edited image and the rotated object should look, which would also be a nice qualitative result for the reader to inspect the result of the method. On a similar note, it would be interesting to use the method to perform 3D reconstruction of image objects by generating controlled multi-views of it, and then compare the obtained result with the GT Objaverse asset.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)