Tactile DreamFusion: Exploiting Tactile Sensing for 3D Generation

Authors: ruihan  gao
kangle  deng
gengshan  yang
wenzhen  yuan
jun-yan  zhu
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-02 05:54:23

Abstract: 3D generation methods have shown visually compelling results powered by diffusion image priors. However, they often fail to produce realistic geometric details, resulting in overly smooth surfaces or geometric details inaccurately baked in albedo maps. To address this, we introduce a new method that incorporates touch as an additional modality to improve the geometric details of generated 3D assets. We design a lightweight 3D texture field to synthesize visual and tactile textures, guided by diffusion-based distribution matching losses on both visual and tactile domains. Our method ensures the consistency between visual and tactile textures while preserving photorealism. We further present a multi-part editing pipeline that enables us to synthesize different textures across various regions. To our knowledge, we are the first to leverage high-resolution tactile sensing to enhance geometric details for 3D generation tasks. We evaluate our method on both text-to-3D and image-to-3D settings. Our experiments demonstrate that our method provides customized and realistic fine geometric textures while maintaining accurate alignment between two modalities of vision and touch.

Main paper: 
 introduction: 
 Generating high-fidelity 3D assets is essential for various applications, from content creation in gaming and VR/AR to developing realistic simulations for robotics. Recently, a surge in emerging methods has enabled the creation of 3D assets from a single image [1] or a short text prompt [2, 3] . Their success can be attributed to advances in generative models [4, 5] and neural rendering [6, 7] , as well as the availability of extensive 2D and 3D datasets [8, 9] .While existing methods can effectively capture the overall object shape and visual appearance, they often struggle with synthesizing fine-grained geometric details, such as stochastic patterns or bumps from the material. As a result, the final mesh output tends to be either overly smooth or have geometric details baked into the albedo map. But why is that?We argue that data is the major bottleneck. First, high-resolution geometric data are largely absent in current 2D and 3D datasets. For image datasets like LAION [8] , obtaining geometric details is challenging due to cameras' limited resolution in capturing real-world objects. In the case of 3D asset datasets like Objaverse [9] , although the assets often include visual textures, they rarely feature high-resolution geometric textures. Second, it is difficult for humans to precisely describe fine geometric textures in natural language, which makes text-based applications even more challenging.To address the above issues, we propose exploring tactile sensing to capture high-resolution data for 3D generation. Given a text prompt or an input image, we first generate a view-consistent (but potentially smooth) 3D representation and output it as a base mesh with an albedo map. We then collect a few tactile patches that capture the high-resolution surface geometry of the target texture with GelSight [10, 11] , a high-resolution tactile sensor. Given data from multiple modalities, aligning Our method leverages tactile sensing to improve the existing 3D generation pipeline. Left: Given a text prompt, we first generate an image using SDXL [12] and then run Wonder3D [13] to generate mesh from the image. This process often results in a mesh with an overly smooth surface. Right: Our method takes a text prompt and several tactile patches as input and generates high-fidelity coherent visual and tactile textures that can be transferred to different meshes. Our method can easily adapt to image-to-3D tasks, as shown in the rightmost column, with the reference image's thumbnail displayed at the bottom right corner. Please visit our webpage for video results.them is crucial. To achieve that, we learn a lightweight 3D texture field to co-optimize the albedo map and the normal map, guided by diffusion-based distribution matching losses on both visual and tactile domains. Our optimization ensures that the final refined textures contain high-resolution, coherent details for both visual appearance and fine geometry, with accurate alignment between the two modalities. Furthermore, we extend our method to synthesizing multiple textures in different local regions.Our contributions can be summarized as follows: (1) We are the first to leverage tactile sensing to synthesize high-fidelity geometric details for 3D generation. (2) We present a new technique to learn high-quality 3D texture fields with aligned visual and geometry textures, with a multi-part extension that allows us to synthesize multiple textures across various regions. (3) Our experiments show that our method outperforms existing text-to-3D and image-to-3D methods. Our code and dataset are available on our project website.2 Related Work 3D generative models. Following the success of text-to-image models [14, 15, 16, 17] , we have witnessed a booming development of 3D generative models conditioned on text or image. Recent works have used 2D diffusion priors in 3D generation [2, 18, 3, 19, 20, 21, 22, 13, 23] , with notable methods like DreamFusion [2] introducing Score Distillation Sampling (SDS) to optimize a 3D representation using gradients from 2D diffusion models. Follow-up works have further extended the SDS optimization using multi-view diffusion models, improving both 3D generation and single-view reconstruction [1, 24, 25, 26, 27, 28, 13, 29, 30] .Another line of research [31, 32, 33, 34] trains large-scale transformers to generate 3D shapes in a feed-forward manner, requiring high-quality large-scale 3D asset datasets. While these models effectively capture global shapes, they often fail to accurately render fine-grained geometric details, which are often either incorrectly baked in color or lost entirely. In contrast, our method incorporates tactile information to enhance the detail and realism of local geometry in 3D assets, addressing the challenges observed in current 3D generative models.Geometry representation in 3D generation. Researchers have explored various 3D representations for generation, such as voxel grids [35, 36, 37] , point clouds [38, 39, 40] , meshes [3, 41, 23, 42 , V d M p Q 9 g / N H R S Z E = " > A A A B 7 3 i c b V D L S s N A F L 2 p r 1 p f V Z d u B o t Q Q U o i v j Z C 0 Y 3 L C v Y B b S i T 6 a Q d O p n E m Y k Y Q 3 / C j Q t F 3 P o 7 7 v w b p 2 0 W 2 n r g w u G c e 7 n 3 H i / i T G n b / r Z y C 4 t L y y v 5 1 c L a + s b m V n F 7 p 6 H C W B J a J y E P Z c v D i n I m a F 0 z z W k r k h Q H H q d N b 3 g 9 9 p s P V C o W i j u d R N Q N c F 8 w n x G s j d R 6 u v T L j 0 f J Y b d Y s i v 2 B G i e O B k p Q Y Z a t / j V 6 Y U k D q j Q h G O l 2 o 4 d a T f F U j P C 6 a j Q i R W N M B n i P m 0 b K n B A l Z t O 7 h 2 h A 6 P 0 k B 9 K U 0 K j i f p 7 I s W B U k n g m c 4 A 6 4 G a 9 c b i f 1 4 7 1 v 6 F m z I R x Z o K M l 3 k x x z p E I 2 f R z 0 m K d E 8 M Q Q T y c y t i A y w x E S b i A o m B G f 2 5 X n S O K 4 4 Z 5 X T 2 5 N S 9 S q L I w 9 7 s A 9 l c O A c q n A D N a g D A Q 7 P 8 A p v 1 r 3 1 Y r 1 b H 9 P W n J X N 7 M I f W J 8 / G p G P X w = = < / l a t e x i t > z = f (x, y) < l a t e x i t s h a 1 _ b a s e 6 4 = " N h u N V I q Z Z 7 u k b K G d C J R z W d q I D M o = " > A A A C L H i c f Z D L S g M x F I Y z X m u 9 j b p 0 E y x C h V p m x N t G K H b j s o K 9 Q K e U T J p p Q z O Z I c m I Z Z g H c u O r C O L C I m 5 9 D j P t g N q K B w I f / 3 9 O k v O 7 I a N S W d b Y W F h c W l 5 Z z a 3 l 1 z c 2 t 7 b N n d 2 G D C K B S R 0 H L B A t F 0 n C K C d 1 R R U j r V A Q 5 L u M N N 1 h N f W b 9 0 R I G v A 7 N Q p J x 0 d 9 T j 2 K k d J S 1 6 z y q + K x 4 w m E Y y d E Q l H E o J d 8 8 0 N S + s c d J S V o H 3 X N g l W 2 J g X n w c 6 g A L K q d c 0 X p x f g y C d c Y Y a k b N t W q D p x e i l m J M k 7 k S Q h w k P U J 2 2 N H P l E d u L J s g k 8 1 E o P e o H Q h y s 4 U X 9 O x M i X c u S 7 u t N H a i B n v V T 8 y 2 t H y r v s x J S H k S I c T x / y I g Z V A N P k Y I 8 K g h U b a U B Y U P 1 X i A d I Z 6 N 0 v n k d g j 2 7 8 j w 0 T s r 2 e f n s 9 r R Q u c 7 i y I F 9 c A C K w A Y X o A J u Q A 3 U A Q a P 4 B m 8 g b H x Z L w a 7 8 b H t H X B y G b 2 w K 8 y P r 8 A d p G o V g = = < / l a t e x i t > n = ( @f @x , @f @y , 1)Figure 2 : Tactile data capture. We collect one patch by pressing GelSight Mini on an object surface. We use Poisson integration to estimate the contact depth from the sensor output, apply high-pass filtering to extract the high-frequency texture information, and then run the 2D texture synthesis method of Image Quilting [77] to obtain an initial texture map. Finally, we convert the height map back to a normal map. 43, 44, 45, 19] , neural radiance fields (NeRF) [46, 47, 2, 20, 21] , neural surfaces [48, 22, 13, 25] ,and Gaussians [49, 50] . Among these, meshes support faster and more efficient rendering with rasterization than volumetric rendering. Meshes also seamlessly integrate with graphics engines for downstream applications. In contrast, volume representations require separate post-processing to extract meshes, often resulting in a loss of quality. Our approach uses meshes to represent coarse geometry while embedding local geometric details into a 3D texture field of tactile normal.3D texture generation and transfer. Simultaneously generating geometry and texture often results in blurry textures due to surface smoothing or averaging across views. To address this, many approaches propose a subsequent stage that focuses on refining high-resolution textures [22, 13] or treats texture generation as a separate problem entirely [51, 52, 53, 54, 55, 56] . Additionally, various texture transfer methods enable the transfer of texture to new shapes using images [57, 58, 59, 60] or other material assets [61] . While most methods focus solely on visual appearance, our approach produces high-fidelity geometric details. Closely related to our method, NeRF-Texture [62] also transfers textures with geometric variations but requires a collection of 100-200 images of the same object. However, their method, constrained by visual resolution, can only model geometry at a coarse level. In contrast, our method leverages high-resolution tactile sensing at the millimeter scale and provides accurate surface normal details, with richer details at fine scales.Tactile sensing in 3D. Tactile data is useful in providing contact information and are widely used in 3D perception and robotics tasks. Early works start with reconstructing simple objects [63, 64, 65, 66] , and then evolve to more complicated scenes with latest 3D neural representations [67, 68] , as well as robotic in-hand reconstruction with multi-finger contacts [69, 70] . Vision-based tactile sensing [10, 71, 11] , a particular sensing mechanism based on the photometric stereo principle, can provide highresolution surface texture information like surface normals and thus can be useful for high-quality 3D synthesis and generation. Several recent works have also used it for 2D generation [72, 73, 74] and 3D scene reconstruction [75, 67, 76] . In contrast, to our knowledge, our work is the first to use tactile sensing for 3D generation. 
tactile data acquisition: 
 We use GelSight [10, 11] , to acquire high-resolution geometric information by touching the object's surface. The raw sensor data are pre-processed to obtain high-frequency signals, as shown in Figure 2 .GelSight tactile sensor. The GelSight sensor is a vision-based tactile sensor that uses photometric stereo to measure geometry at a high spatial resolution at the contact surface. It can capture the fine details on the surface, such as bumps on avocados' skin and patterns of crochet yarns. In our work, we use the GelSight mini with a sensing area of 21mm × 25mm (H × W ) and a pixel resolution of 240×320, equivalent to about 85 micrometers per pixel. The sensor captures an RGB image, which can be mapped to the surface normals and then used to reconstruct a surface height map.Tactile data pre-processing and contact mask. We manually press the GelSight sensor against the object's surface once to obtain a tactile image, which covers a small area. The derived height map contains both a low-frequency component, representing the global shape of the surface, and high-frequency geometric textures. We first apply a high-pass filter to the height map to extract texture information and then center-crop the masked region to a square patch to remove the non-contact E K y D M D i v A z z h v N x U w 2 6 Q I 8 x g 2 3 o = " > A A A B 6 H i c b V D L S g N B E O z 1 G e M r 6 t H L Y B A 8 h V 3 x d Q x 6 8 Z i A e U C y h N l J b z J m d n a Z m R X C k i / w 4 k E R r 3 6 S N / / G S b I H T S x o K K q 6 6 e 4 K E s G 1 c d 1 v Z 2 V 1 b X 1 j s 7 B V 3 N 7 Z 3 d s v H R w 2 d Z w q h g 0 W i 1 i 1 A 6 p R c I k N w 4 3 A d q K Q R o H A V j C 6 m / q t J 1 S a x / L B j B P 0 I z q Q P O S M G i v V k 1 6 p 7 F b c G c g y 8 X J S h h y 1 X u m r 2 4 9 Z G q E 0 T F C t O 5 6 b G D + j y n A m c F L s p h o T y k Z 0 g B 1 L J Y 1 Q + 9 n s 0 A k 5 t U q f h L G y J Q 2 Z q b 8 n M h p p P Y 4 C 2 x l R M 9 S L 3 l T 8 z + u k J r z x M y 6 T 1 K B k 8 0 V h K o i J y f R r 0 u c K m R F j S y h T 3 N 5 K 2 J A q y o z N p m h D 8 B Z f X i b N 8 4 p 3 V b m s X 5 S r t 3 k c B T i G E z g D D 6 6 h C v d Q g w Y w Q H i G V 3 h z H p 0 X 5 9 3 5 m L e u O P n M E f y B 8 / k D 3 V m M / g = = < / l a ti h U d Z D 2 v l s x i z x j A I G 2 H 3 u 2 T 4 m o = " > A A A B 9 X i c b V D L S s N A F L 2 p r 1 p f V Z d u B o v g q i T i a 1 l 0 4 8 J F B f u A N p a b 6 a Q d O p m E m Y l S Q v / D j Q t F 3 P o v 7 v w b k z Y L b T 0 w c D j n X u 6 Z 4 0 W C a 2 P b 3 1 Z h a X l l d a 2 4 X t r Y 3 N r e K e / u N X U Y K 8 o a N B S h a n u o m e C S N Q w 3 g r U j x T D w B G t 5 o + v M b z 0 y p X k o 7 8 0 4 Y m 6 A A 8 l 9 T t G k 0 k M 3 Q D O k K J L b S Q 9 L v X L F r t p T k E X i 5 K Q C O e q 9 8 l e 3 H 9 I 4 Y N J Q g V p 3 H D s y b o L K c C r Y p N S N N Y u Q j n D A O i m V G D D t J t P U E 3 K U K n 3 iZ A + V 4 w a M U 4 J U s X T r I Q O U S E 1 a V F Z C c 7 8 l x d J 8 6 T q n F f P 7 k 4 r t a u 8 j i I c w C E c g w M X U I M b q E M D K C h 4 h l d 4 s 5 6 s F + v d + p i N F q x 8 Z x / + w P r 8 A T Q G k l Q = < / l a t e x i t > L a < l a t e x i t s h a 1 _ b a s e 6 4 = " p K e 1 5 f U z g v E 7 l A b q S x 8 F f 4 1 F Q W U = " > A A A B 9 X i c b V D L S s N A F L 2 p r 1 p f V Z d u B o v g q i T i a 1 l 0 4 8 J F B f u A N p b J d N I O n U z C z I 1 S Q v / D j Q t F 3 P o v 7 v w b k z Y L b T 0 w c D j n X u 6 Z 4 0 V S G L T t b 6 u w t L y y u l Z c L 2 1 s b m 3 v l H f 3 m i a M N e M N F s p Q t z 1 q u B S K N 1 C g 5 O 1 I c x p 4 k r e 8 0 X X m t x 6 5 N i J U 9 z i O u B v Q g R K + Y B R T 6 a E b U B w y K p P b S Q 9 L v X L F r t p T k E X i 5 K Q C O e q 9 8 l e 3 H 7 I 4 4 A q Z p M Z 0 H D t C N 6 E a B Z N 8 U u r G h k e U j e i A d 1 K q a M C N m 0 x T T 8 h R q v S J H + r 0 K S R T 9 f d G Q g N j x o G X T m Y p z b y X i f 9 5 n R j 9 S z c R K o q R K z Y 7 5 M e S Y E i y C k h f a M 5 Q j l N C m R Z p V s K G V F O G a V F Z C c 7 8 l x d J 8 6 T q n F f P 7 k 4 r t a u 8 j i I c w C E c g w M X U I M b q E M D G G h 4 h l d 4 s 5 6 s F + v d + p i N F q x 8 Z x / + w P r 8 A V D l k m c = < / l a t e x i t > L t< l a t e x i t s h a 1 _ b a s e 6 4 = " 9 e e a D 6 r S d g r 0 o 6 x t N N f a l 5 x P 5H g = " > A A A C A H i c b V C 5 T s N A E F 2 H K 4 T L Q E F B s y J C o o p s x F V G U E B B E S R y S L F l r T e b Z J X 1 o d 0 x I r L c 8 C s 0 F C B E y 2 f Q 8 T e s E x e Q 8 K S R n t 6 b 0 c w 8 P x Z c g W V 9 G 6 W F x a X l l f J q Z W 1 9 Y 3 P L 3 N 5 p q S i R l D V p J C L Z 8 Y l i g o e s C R w E 6 8 S S k c A X r O 2 P r n K / / c C k 4 l F 4 D + O Y u Q E Z h L z P K Q E t e e a e E x A Y U i L S 2 8 x L H W C P k L a u s 8 w z q 1 b N m g D P E 7 s g V V S g 4 Z l f T i + i S c B C o I I o 1 b W t G N y U S O B U s K z i J I r F h I 7 I g H U 1 D U n A l J t O H s j w o V Z 6 u B 9 J X S H g i f p 7 I i W B U u P A 1 5 3 5 u W r W y 8 X / v G 4 C / Q s 3 5 W G c A A v p d F E / E R g i n K e B e 1 w y C m K s C a G S 6 1 s x H R J J K O j M K j o E e / b l e d I 6 r t l n t d O 7 k 2 r 9 s o i j j P b R A T p C N j p H d X S D G q i J K M r Q M 3 p F b 8 a T 8 W K 8 G x / T 1 p J R z O y i P z A + f w C y p 5 c i < / l a t e x i t > L VG < l a t e x i t s h a 1 _ b a s e 6 4 = " L M k d R 9 c v G l f U t U O d c K t H x c E 1 r 5 8 = " > A A A C A H i c b V C 7 T s N A E D z z D O F l o K C g s Y i Q q C I b 8 S o j K K C g C F J e U m x Z 5 8 s l O e V 8 t u 7 W i M h y w 6 / Q U I A Q L Z 9 B x 9 9 w T l x A w k g r j W Z 2 t b s T x J w p s O 1 v Y 2 F x a X l l t b R W X t / Y 3 N o 2 d 3 Z b K k o k o U 0 S 8 U h 2 A q w o Z 4 I 2 g Q G n n V h S H A a c t o P R d e 6 3 H 6 h U L B I N G M f U C / F A s D 4 j G L T k m / t u i G F I M E / v M j 9 1 g T 5 C 2 r j J M t + s 2 F V 7 A m u e O A W p o A J 1 3 / x y e x F J Q i q A c K x U 1 7 F j 8 F I s g R F O s 7 K b K B p j M s I D 2 t V U 4 J A q L 5 0 8 k F l H W u l Z / U j q E m B N 1 N 8 T K Q 6 V G o e B 7 s z P V b N e L v 7 n d R P o X 3 o p E 3 E C V J D p o n 7 C L Y i s P A 2 r x y Q l w M e a Y C K Z v t U i Q y w x A Z 1 Z W Y f g z L 4 8 T 1 o n V e e 8 e n Z / W q l d F X G U 0 A E 6 R M f I Q R e o h m 5 R H T U R Q R l 6 R q / o z X g y X o x 3 4 2 P a u m A U M 3 v o D 4 z P H 6 + Z l y A = < / l a t e x i t > L TG Figure 4: Method overview.Given an input image or a text prompt, our method generates a mesh with highquality visual and normal texture. We first generate a base mesh with albedo texture using a text-or image-to-3D method. We use a 3D texture field with hash encoding to represent albedo and tactile normal textures and train it with loss functions on rendered images. To capture the scale differences between visual and tactile modalities, we sample distinct camera views, Pv for visual rendering and Pt for tactile rendering. For texture refinement, we train the texture field with a visual matching loss La, to ensure fidelity to the input mesh, and a visual guidance loss with normal-conditioned ControlNet, LVG, to enhance photorealism and cross-modal alignment. We further apply a tactile matching loss, Lt, and a tactile guidance loss, LTG, using a customized Texture Dreambooth, to achieve high-quality geometric details aligned with the distribution of tactile input V* texture exemplars.area. Finally, we convert the masked height map patch back to a normal map patch by computing its gradients.Figure 3 shows our dataset TouchTexture collected from 16 daily objects with diverse tactile textures. Our dataset is available on our project website. 
method: 
 Generating 3D assets with high-resolution geometric details is challenging due to the difficulty of acquiring low-level texture details from text prompts or a single image. Therefore, we incorporate an additional input modality, tactile normal maps, to enhance the geometric details in 3D asset generation. Figure 4 shows our overall pipeline. Our method takes an input image and a tactile normal patch as input and generates a 3D asset with a high-fidelity, color-aligned normal map. The input image can be a real image or generated by a text-to-image model such as SDXL [12] . Below, we first describe how we generate the base mesh in Section 4.1. We then introduce our core texture refinement algorithm in Section 4.2 and extend it to objects with multiple materials in Section 4.3.Similar to prior works with stage-wise geometry sculpting and texture refinement [22, 49] , we first generate a base mesh with albedo texture using a text-or image-to-3D method. We then unwrap a UV map of the exported mesh M and project the vertex albedo onto an albedo UV map.Direct texture transfer. To incorporate the tactile information, one could synthesize a large 2D texture map with an exemplar of the preprocessed normal patch using image quilting algorithm [77] and use it as a normal UV map of the object mesh. However, this would result in inconsistencies between visual and tactile textures, especially when transition in visual color indicates geometric change, as shown in Figure 9 . Therefore, we synthesize a texture map that roughly matches the physical scale of real materials and use it for initialization. Then, we jointly optimize the albedo and the normal tactile texture to ensure alignment.Texture representation. For ease of optimization, we represent the visual and tactile normal textures as a 3D texture field. In practice, we use a multi-resolution hash grid [78] β(•), whose input is a 3D spatial coordinate p on the mesh:EQUATIONwhere c ∈ R 3 is the albedo and n t ∈ R 3 is the tactile normal defined in the tangent space on the mesh surface.Given the base mesh M , the texture field β(•), a camera pose P , and a lighting condition L, we can render a color image Îc of the object using a differentiable rasterizer R, nvdiffrast [79] :EQUATIONSpecifically, for each 3D point on the mesh, we composite the base normal n b from the mesh geometry and the tactile normal n t from the texture to get the shading normal n s :EQUATIONwhere n b is the global geometric normal defined by the base mesh, t is the tangent vector, and Q TBN is the Tangent-Bitangent-Normal (TBN) Matrix for every surface point. With the calculated shading normal, we use a point light and a simple diffuse shading model to produce the color image Îc following prior works [2, 3] given a sampled camera pose P . We can also obtain Îa (P ), Ît (P ), and În (P ) by projecting the albedo, tactile normal, and shading normal onto a sample view P .Learning 3D texture field. Given the scale discrepancy between vision and touch, we sample camera views differently for two modalities. To supervise visual renderings Îc and Îa , we sample camera poses P v orbiting the base mesh looking at the object center with perspective projection. To supervise tactile renderings Ît , we sample camera poses P t close to mesh surfaces based on vertex normals to emulate the captured data using a real sensor. Specifically, we randomly sample a vertex, place a camera along this vertex's normal direction, and set the camera to look at the sampled vertex with orthographic projection.We initialize the neural texture field of albedo with a reconstruction loss of rendered images:EQUATIONwhere I a is rendered using the exported albedo UV map in Section 4.1 from the same camera P v . Similarly, we initialize the tactile normal texture field with a tactile matching loss:EQUATIONwhere I t is rendered using the tactile normal UV map from the image quilting results from a tactile camera pose P t .To refine the visual texture, we use a diffusion guidance loss inspired by the multi-step denoising process in SDEdit [80] and DreamGaussian [49] . Importantly, we compute the Visual Guidance loss L VG based on a normal-conditioned ControlNet [81] to ensure the refined visual texture is consistent with the tactile normal. Given the rendered color image Îc from P v , we perturb it with random noise ϵ(t) to get a noisy image x t and apply a multi-step denoising process f ϕ (•) using the 2D diffusion prior to obtaining a refined image:EQUATION)where y is the input text prompt, and f ϕ is Stable Diffusion with a normal-conditioned ControlNet. We denoise the image from timestep t to a completely clean image I ϕ , which is different from the typical single-step SDS loss in Dreamfusion [2] . The starting timestep t is carefully chosen to limit the noise strength so the clean image can enhance details without breaking the original content. This refined image is then used to optimize the texture through the L1 and LPIPS [82] loss:EQUATIONWhile refining the visual textures, we jointly optimize the tactile texture with a Tactile Guidance loss L TG . Similar to L VG , we add random noise to a rendered tactile normal map I t (P t ), generated from a tactile camera pose P t , and then apply a multi-step denoising process to obtain a refined normal image I ψ . To capture the distribution of tactile normals, we replace the standard diffusion prior with a customized Texture Dreambooth model, created by fine-tuning the Stable Diffusion model f ψ (•) for each tactile texture using a Low-Rank Adapter (LoRA) [83] as in Dreambooth [84] . We leave the details of training LoRAs to the appendix. We use the output I ψ to refine the tactile texture:EQUATIONOptimization. We start by only optimizing L a and L t for 150 iterations. After that, we add L VG and L TG for another 50 iterations. The overall loss function is:EQUATIONWe use λ a = 500, λ VG = 5, λ TG = 0.05 to balance L a , L VG and L TG . We reduce λ t from 1 to 0.05 and change L a from per-pixel error to mean-color error to allow more spatial editing by diffusion priors in color space.Another advantage of our 3D texture field is its ability to easily define non-uniform textures in 3D, allowing different tactile textures to be assigned to distinct parts of an object. For instance, when generating a 3D asset of "a cactus in a pot", we can apply different textures to the cactus and the pot. This is achieved by incorporating two tactile inputs along with a text prompt that specifies the texture assignment, e.g., "cactus with texture A, pot with texture B".Part segmentation based on diffusion features. Our method can automatically segment object parts based on the text prompt without requiring manual annotation. We leverage the internal attention maps of diffusion models to segment the rendered views I a (P v ) of the object. Specifically, we add a random noise of level t to I a and applying one denoising step with the SD UNet. Inspired by DiffSeg [85] , we perform unsupervised part segmentation by aggregating and spatially clustering self-attention maps from multiple layers.. To assign part labels, we aggregate cross-attention maps corresponding to different parts and match them with the unlabeled segmentation maps based on KL distance. This results in a list of labeled segmentation masks S i (I a ), i ∈ {1, . . . , N } where each i denotes one of the N object parts. For example, given the prompt "cactus in a pot," we aggregate cross-attention maps for "cactus" and "pot" from different layers, then assign each part segmented from the clustered self-attention maps to either cactus or pot by sorting its KL distance to respective cross-attention maps. More details are included in the appendix.3D part label field. Since the 2D part segmentations S(I a (P v )) may contain noise and lack multiview consistency, we integrate them from different viewpoints into a coherent 3D part label field. We achieve this by incorporating a part label s ∈ R N into our texture field β(•). We render part label maps Ŝ(P v ) from sampled camera poses P v and supervise them using a cross entropy loss:EQUATION)Optimization with multi-part textures. To learn multi-part textures, we keep the same L a and L VG loss functions while modifying L t and L TG to incorporate part-specific tactile supervision: L t = N i=1Ŝi (P t ) ⊙ 1cos Ît (P t ), I t(i) (P t ) ,EQUATIONwhere ⊙ is the pixel-wise product, I t(i) (P t ) is the rendered reference view using the i-th part's tactile data, I ψ(i) is the refined tactile normal generated by Texture Dreambooth trained on the i-th part texture, and Ŝi (P t ) is the rendered binary mask of the i-th part from our learned 3D label field. 
experiment: 
 We present comprehensive experiments to verify the efficacy of our method. We perform qualitative and quantitative comparisons with existing baselines and ablation studies on the major components.Dataset. We collected 16 diverse types of textures from daily objects in our TouchTexture dataset, five tactile patches per texture, and pair them with a short description. Please see our appendix for the full list. We randomly sample one patch to initialize the tactile UV map with image quilting and use five patches to train the customized Texture Dreambooth. For base mesh generation given a text/image input, while any mesh generation method is applicable, we use Wonder3D [13] in the main experiments but include results using InstantMesh [86] and RichDreamer [87] in the appendix as well. Specifically, Wonder3D takes an image as input and outputs a mesh with albedo stored as vertex colors. We then convert the vertex albedo into an albedo UV map as described in Section 4. optimization in Section 4.2, we refine the albedo and tactile textures with the "textured prompts", i.e., A [object name] with V* texture, where V* represents the text description corresponding to the selected tactile texture map.3D generation with a single texture. Figure 5 shows our 3D generation results for a single texture, showing albedo, normal, and full-color rendering from two viewpoints for each object. Our method works for both text input (the corn and the football) and image input (the potato and the strawberry), generating realistic, coherent, and high-resolution visual and geometry details. Figure 6 shows the results of applying different textures to the same object (a coffee cup), with normal rendering on top and color rendering below. The joint optimization of our method helps produce coherent visual and tactile textures that provide smooth and natural shadowing effects.Multi-part texture generation. As introduced in Section 4.3, users can specify an object (via text or image) and assign different textures to two distinct parts. Figure 7 shows results for multi-part texture synthesis. The left column shows the text/image input and the tactile input. The image input can be either real or generated from a text prompt. The color-coded text prompts correspond to text descriptions for two textures. The right columns show the albedo, normal, and full-color rendering results. Our method segments parts effectively with our 3D label field learned by aggregated attention maps. The joint optimization successfully applies the textures to each corresponding part designated by the user and generates an overall coherent visual appearance and tactile geometry.Baselines. To our knowledge, this work is the first to leverage high-resolution tactile sensing to enhance geometric details for 3D generation tasks. Thus, we compare our method with existing text/image-to-3D methods. For text-to-3D, we compare with DreamCraft3D [22] and use the same textured prompt as input. DreamCraft3D focuses on geometry sculpting using neural field and DMTet [88] , followed by texture boosting through fine-tuning DreamBooth with augmented multiview rendering, requiring 10x computational cost compared to our method. For image-to-3D, we compare with DreamGaussian [49] and Wonder3D [13] . DreamGaussian employs fast generation using 3D Gaussian primitives and refines an albedo map in Stage 2 with a multi-step denoising MSE loss. Wonder3D generates paired multi-view RGB and normal images, utilizing a geometry fusion process to generate a 3D result. The input images remain the same as ours for these two baselines. We use the official open-source implementation for all baselines. Table 1 : Human perceptual study. For all paired comparisons, our method is preferred (≥ 50%) over the baselines for both texture appearance and geometric details. Qualitative evaluation. Figure 8 shows qualitative results of our method compared against three baselines. For each example, we show color and normal rendering with zoomed-in patches at the same location for detailed comparison. Our method achieves higher visual fidelity, more realistic details in normal space, and better color-geometry alignment than the baselines. In particular, our textures exhibit sharper details than those of Wonder3D, which tend to be overly smooth. In contrast to DreamGaussian and DreamCraft3D, our generated geometry details are more realistic and align well with color appearance. In the avocado example, DreamCraft3D suffers from the "Janus problem", generating a brown-colored core on both sides.Quantitative evaluation. We perform a human perceptual study using Amazon Mechanical Turk (AMTurk). We set up a paired test, showing a reference prompt and two rendering results, one generated with our method and the other generated with one of the baselines.We conduct two separate surveys to evaluate the texture appearance and geometric details respectively. For texture appearance, we render full-color RGB images and ask users "Which of the following views has more realistic textures?". For geometric details, we render shaded colorless images that only demonstrate the mesh geometry and ask users "Which of the following views has more realistic geometry details?". Each user has two practice rounds followed by 20 test rounds to evaluate our method against DreamGaussian, Wonder3D, and DreamCraft3D. All samples are randomly selected and permuted, and we collect 1,000 responses.Table 1 shows the mean and standard deviation of users' preference score; our method is preferred over all baselines. While DreamCraft3D has a close performance regarding texture appearance, it fails to obtain high-resolution geometric details that align well with color texture.Ablation study. We perform an ablation study to evaluate key aspects of our method. We render both front and back views of a sampled object for each experiment. To illustrate details and crossmodal alignment, we prsent a global full-color view on the left, a global normal view on the right, and patch views of the full-color, albedo, and normal renderings in between. Figure 9 shows results of ablating diffusion-based guidance losses for texture refinement, specifically the visual guidance L VG and tactile guidance L TG . Removing the visual guidance introduces misaligned visual and tactile normal textures. For example, the bumps in the normal map are misaligned with the locations of white seeds in the albedo rendering, as shown in the zoomed-in patches. Our method encourages the generated color to be consistent with the tactile normal using ControlNet-guided visual refinement loss while also enhancing the details in the tactile texture. Omitting L TG reduces tactile texture details, while omitting L VG introduces misalignment between visual and tactile normal textures; for instance, bumps in the normal map do not match the white seeds in the albedo rendering. Our method ensures color consistency with tactile normals via ControlNetguided visual refinement and enhances tactile texture detail.We also study the efficacy of tactile data processing by comparing our method with the texture map synthesized using the original tactile data without preprocessing stated in Section 3. Figure 10 shows that using the original data produces much more flattened textures since the low-frequency deformation of the gel pad due to uneven contact during the data collection would dominate the tactile signal and thus degrade the details of synthesized texture maps. Figure 11 shows the results of removing tactile input and only optimizing the albedo map with "textured prompts" using L a and L VG . Without tactile input, the output mesh is overly smooth, demonstrating the insufficiency of using text prompts only for fine geometry texture generation. 
discussion: 
 Recent methods in 3D generation often struggle with unrealistic geometric details. To address this, we have introduced a novel approach that incorporates tactile information. Our method synthesizes both visual and tactile textures using a 3D texture field. Additionally, we have introduced a multi-part texturing pipeline for controllable region-wise texture generation. To our knowledge, this is the first use of high-resolution tactile sensing to improve 3D generation. Our method produces realistic, fine-grained geometric textures while maintaining accurate visual-tactile alignment.Limitations. The quality of the generated coarse shape depends on the existing 3D generative models, which struggle with complex geometry. Additionally, slight seams may appear in our results due to UV unwrapping. Addressing these issues presents a meaningful direction for future work. At each iteration, we run a forward pass of the diffusion model for the target albedo image, aggregate its self-attention and cross-attention maps, and compute KL distance to merge the masks into N parts. We show an example of "a cactus in a pot", where we extract the masks corresponding to two tokens "cactus" and "pot", shown as Mask A and Mask B. The segmentation masks are then used to supervise the label field training to enable multi-part synthesis. 13 : Additional materials in tactile normal dataset TouchTexture. We collect tactile normal data from 16 daily objects featuring diverse tactile textures. We show the tactile normal map and a 3D height map for each object. This library of tactile samples covers a wide range of diverse materials commonly found in daily life.Regarding the network, we use TCNN encoding with two base linear layers, which then branches out into three output layers, one linear layer followed by sigmoid activation for albedo output, one linear layer followed by tanh activation for tactile normal output, and one optional linear layer for label field. We follow DreamBooth [84] to train texture LoRAs with Stable Diffusion (SD) V1.4 for the tactile guidance loss. We find empirically that SD V1.4 works better for generating tactile normal maps. We use ControlNet (v1.1 -normalbae version) with SD V1.5 for the diffusion loss. We follow Wonder3D to generate the base mesh and train the texture field network using Adam optimizer with lr = 0.01. We train all models on A6000 GPUs and each experiment takes about 10 mins and 20G RAM to run.Given a text or image input, our method seamlessly integrates with a wide range of mesh generation approaches to enhance the 3D output with refined details. Here, we presents results using RichDreamer [87] for base mesh generation in Figure 14a and results using InstantMesh [86] in Figure 14b . RichDreamer generates detailed and consistent 3D meshes from text prompts by incorporating depth and normal priors during pre-training, while InstantMesh employs a feed-forward avocado "avocado skin's " strawberry "strawberry" canvas bag "canvas bag" striped steel "fine striped steel" corn "corn" rubber "heat-resistant rubber with heart-shape pattern" woven "woven crochet" orange "orange" cantaloupe "cantaloupe skin's" potato "potato" football "football" chopping board 'cutting board" table tennis handle "smooth wood" table cloth "patterned table cloth" metal grid "metal grid" cork "cork mat" image-to-3D approach with a transformer-based reconstruction model and FlexiCubes for efficient mesh production. Both methods provide solid base meshes, and our approach effectively adapts to each, refining the output meshes with intricate texture details for greater realism and fidelity. We perform a human perceptual study on Amazon Mechanical Turk (AMTurk). The full text of instructions given to participants and the webpage setup are shown in Figure 15 and Figure 16 . The estimated time for each test is about 5 minutes, and we pay users a corresponding amount of compensation higher than the minimum wage in the country of the data collector. The online survey does not pose any explicit potential risks that are expected to be incurred by participants. Our method of incorporating tactile information into 3D generation enhances the geometric details of the generated results. This advancement helps automatically create high-fidelity assets, which are highly applicable in game and film production. However, there is also a potential risk of misuse, as 3D assets could be employed to generate fake content for misinformation. Despite this concern, we believe humans can currently distinguish our synthesized objects from real ones. Answer: [Yes] Justification: We describe the implementation details in the paper to ensure reproducibility. The code and data will be released on our project website. Guidelines:• The answer NA means that paper does not include experiments requiring code. Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes]Justification: Yes, we discuss societal impact in the supplementary material.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: We do not release data or models that have a high risk of misuse and thus pose no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We cite all the original paper/code package/dataset to our best efforts.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.• If this information is not available online, the authors are encouraged to reach out to the asset's creators. 13. New Assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [NA] Justification: The paper does not release new assets. The code and data will be released upon publication.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix:  


Decision: This submission has garnered 1 accept, 2 weak accepts, and 1 borderline reject. Reviewers zBVq, MUQX, and a9oZ support the paper, recognizing its novelty and the convincing nature of its contributions. The authors have successfully addressed the concerns raised during the review process, which include issues of repetitive textures, adaptability to complex objects, and the role of tactile sensing. Reviewer ecYi expressed reservations, comparing the proposed text-driven 3D generation method to traditional 3D reconstruction approaches; however, the authors' rebuttal effectively clarified the distinctions between these methods. Given the supportive majority among the reviewers, the AC recommends acceptance of this manuscript.

Reviews: review: paper_summary: paper_summary: This paper proposes a method for generating 3D assets with detailed geometry through inputs from a tactile sensor. More specifically, given a bump-map as input from a tactile sensor (just a small patch is enough), the method uses it as regularization while maximizing the likelihood using a normals-conditioned stable diffusion model. The albedo is also optimized alongside the normals, though limited results are shown. Additionally, using diffusion based segmentation maps, different parts of the image can be given different textures. While each component of the method itself is not too novel, the sum total is and the results are pretty good.
strength_weakness: strength_weakness: 1) There are no results on the albedo provided, it would be great if the authors could explain why they were omitted. I strongly urge them to include it in the rebuttal. 

2) I may be mistaken, but it seems this method works only for repetitive textures (though the diffusion model is able to change it through optimization). It would be great if the authors could provide the results of an experiment where the object has two very different textures but those textures are only learnt through the diffusion prior. For example, in the cactus pot case, this would correspond to optimizing L_{tactile} only for the pot and let the diffusion prior decide what the normals for the cactus must look like (or vice versa).
questions: questions: Just reiterating what I already mentioned in the weaknesses: It would be great if the authors could provide the results of an experiment where the object has two very different textures but those textures are only learnt through the diffusion prior. For example, in the cactus pot case, this would correspond to optimizing L_{tactile} only for the pot and let the diffusion prior decide what the normals for the cactus must look like (or vice versa). 

I believe this experiment would give us insights on how the diffusion prior itself would perform if a part of the image already has detailed normals (via L_{tactile})
limitations: limitations: Yes
paper_summary: paper_summary: This submission addresses the long-standing challenge of enhancing geometric details in results produced by text-to-3D and image-to-3D pipelines. The approach introduces a novel method that leverages tactile normal modality to synthesize high-fidelity geometric details. Additionally, it employs attention maps during the diffusion process to segment input images based on text prompts, allowing for the synthesis of multiple textures across various regions. The results demonstrate that this method effectively recovers geometric details and ensures alignment between geometry and color.
strength_weakness: strength_weakness: 1. The paper claims compatibility of the proposed method with both text-to-3D and image-to-3D pipelines. However, DreamCraft3D, selected as a text-to-3D pipeline, requires both an image and a text caption as inputs. Although DreamCraft3D can be considered as a 'text-to-image, text & image-to-3D' process, it differs from a purely text-to-3D approach. Thus, the compatibility between the proposed method and purely text-to-3D pipelines remains questionable.

2. The pipeline overview in Figure 4 indicates the need for a reference image and tactile input. The paper does not address how to select an appropriate tactile input for the reference image, nor how to ensure the tactile details are compatible with the object in the image.

3. The paper presents an intriguing text-guided segmentation strategy that leverages attention maps during the diffusion process based on text prompts. However, lacking expertise in diffusion-based segmentation, I am unfamiliar with the efficiency and success rate of this method, but I am positively impressed by it.

4. The generalization and diversity of the proposed method are also of concern. The objects presented in the results are not very complex, and the dataset comprises 16 popular categories. It remains unclear how well the method would perform on more complex objects, such as those in sci-fi or fantasy genres.
questions: questions: 1. I am curious about your criteria for selecting 3D generation baselines. For image-to-3D tasks, to my knowledge, more advanced baselines such as 'InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Models' and 'TripoSR: Fast 3D Object Reconstruction from a Single Image' offer superior quality and may yield stronger results. For text-to-3D tasks, 'RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3D' is a noteworthy pure text-to-mesh method. 

2. The proposed method in the paper first refines normals and then optimizes colors based on the refined normals. Does this imply that the geometric details introduced by your method are not derived from the original colors (i.e. not very necessary to original colors) of the objects? Additionally, could the color optimization process potentially disrupt the alignment between the original colors and the refined colors, potentially failing to meet the initial requirements?

3. I am curious about the generalization and diversity capabilities of the proposed method. Can it handle more complex objects? Furthermore, with more powerful baselines, can this method achieve higher quality results on more complex cases?
limitations: limitations: The authors acknowledge their limitations in the main paper and address potential social impacts in the supplementary materials. Regarding the first limitation, the implementation of new 3D generative models, such as 'Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Image' and 'Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attention', is recommended. For the second limitation, utilizing a more powerful computer graphics tool could be beneficial. Concerning the social impact, issues related to deepfakes and the potential for misinformation are noteworthy. The authors assert that humans can currently distinguish their synthesized objects from real ones, a claim with which I concur. Although it is of low priority, it would be preferable for the authors to include a comparison between a generated object and a real one; a single case would suffice.
paper_summary: paper_summary: This paper proposes a lightweight 3D texture field that ensures the consistency between visual and tactile textures while preserving photorealism. The experiments demonstrate that quantitative and qualitative results show good generation quality.
strength_weakness: strength_weakness: 1. This paper suggests that existing methods struggle to achieve fine-grained geometric details. However, methods like Neuralangelo and PermutoSDF can recover highly detailed geometric information.

2. The TouchTexture dataset presented in Figure 3 and the supporting materials seemingly do not capture the "local geometric intricacies."

3. There are only very limited objects been shown in the main paper and supplementary material. Are those objects cherry-picked? It would be great if more results on in-the-wild object can be provided to show the generalisation ability of the model.

4. Ablation Study is an important part in paper and it would be more convincing with both quantitative and qualitative experiments, and cannot be simply summarized with only several sentences. If there are any figures and tables in the article, please indicate the specific table number or figure number in the analysis and analyze according to the specific visualization results.

- [Neuralangelo: High-Fidelity Neural Surface Reconstruction. CVPR2023]
- [PermutoSDF: Fast Multi-View Reconstruction with Implicit Surfaces using Permutohedral Lattices. CVPR2023]
questions: questions: See my previous sections.
limitations: limitations: I'm not confident about voting for accepting this paper because of the potential similarity to existing methods and the limited novelty. 
There is more engineering effort than novelty. The novelty might be limited.
paper_summary: paper_summary: This paper proposes to use the tactile sensing signal (height map and normal map) to improve the 3D generation quality, especially the geometric details. The authors use a 3D mesh generation guided by a normal-conditioned ControlNet to ensure the consistency between the  visual textures and the tactile textures. They also develop a multi-part editing pipeline to generate objects with different texture parts. Experiments and ablation studies demonstrate the effectiveness of the proposed method.
strength_weakness: strength_weakness: The connection between the tactile sensing and the 3D generation is not strong and critical. The tactile signals are only used to generate the normal maps of certain textures. It may be replaced by simpler alternatives such as texture reterival from a normal map database. It needs further elaboration why this combination is necessary.
questions: questions: 1. What is the generation time of the multi-part geneation? 
2. Can the authors add an experiment of real image to 3D mesh with real corresponding tactile signals?
limitations: limitations: Limitaions have been addressed in the paper, which include complex geometry generation and slight seams.

Comments: comment: I would like to thank the authors for the rebuttal, my concerns were well addressed. Looking at other reviews and the rebuttal, I have decided to raise my score.
After reading the rebuttal, most of my concerns are addressed. Hence, I decide to keep my original score and am leaning to accept this paper.
Thank you for taking time and preparing a rebuttal which I read carefully.
Thank you for the response. The explaintation is reasonable to me.