TexTailor: Customized Text-aligned Texturing via Effective Resampling
Authors: Suin Lee
Daeshik Kim
Venue: ICLR 2025 Poster (2025)
Keywords: 3D texture synthesis
diffusion model
resampling
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: We present TexTailor, a novel method for generating consistent object textures from textual descriptions. Existing text-to-texture synthesis approaches utilize depth-aware diffusion models to progressively generate images and synthesize textures across predefined multiple viewpoints. However, these approaches lead to a gradual shift in texture properties across viewpoints due to (1) insufficient integration of previously synthesized textures at each viewpoint during the diffusion process and (2) the autoregressive nature of the texture synthesis process. Moreover, the predefined selection of camera positions, which does not account for the object's geometry, limits the effective use of texture information synthesized from different viewpoints, ultimately degrading overall texture consistency. In TexTailor, we address these issues by (1) applying a resampling scheme that repeatedly integrates information from previously synthesized textures within the diffusion process, and (2) fine-tuning a depth-aware diffusion model on these resampled textures. During this process, we observed that using only a few training images restricts the model's original ability to generate high-fidelity images aligned with the conditioning, and therefore propose an performance preservation loss to mitigate this issue. Additionally, we improve the synthesis of view-consistent textures by adaptively adjusting camera positions based on the object's geometry. Experiments on a subset of the Objaverse dataset and the ShapeNet car dataset demonstrate that TexTailor outperforms state-of-the-art methods in synthesizing view-consistent textures.

Published as a conference paper at ICLR 2025
TEXTAILOR: CUSTOMIZED TEXT-ALIGNED TEXTUR- ING VIA EFFECTIVE RESAMPLING
Suin Lee, Dae-Shik Kim
KAIST
Daejeon, South Korea
(suinlee, daeshik)ekaist.ac.u
ABSTRACT
We present TexTailor, novel method for generating consistent object textures from textual descriptions. Existing text-to-texture synthesis approaches utilize depth-aware diffusion models progressively generate images and synthesize textures across predefined multiple viewpoints. However, these approaches lead gradual shift in texture properties across viewpoints due to (I) insufficient integration of previously synthesized textures at each viewpoint during the dif- fusion process and (2) the autoregressive nature of the texture synthesis process. Moreover, the predefined selection of camera positions, which does not account for the object's geometry, limits the effective use of texture information synthe- sized from different viewpoints, ultimately degrading overall texture consistency. In TexTailor, we address these issues by (1) applying a resampling scheme that repeatedly integrates information from previously synthesized textures within the diffusion process, and (2) fine-tuning a depth-aware diffusion model on these re- sampled textures. During this process, observed that using only few train- ing images restricts the model's 's original ability to generate high-fidelity images aligned with the conditioning, and therefore propose an performance preserva- tion loss to mitigate this issue. Additionally, we improve the synthesis of view- consistent textures by adaptively adjusting camera positions based on the object's geometry. Experiments on subset of the Objaverse dataset and the ShapeNet car dataset demonstrate that TexTailor outperforms state-of-the-an methods in syn- thesizing view-consistent textures. The source code for Tex'Taior is available at https::/githuni com Adios 42xxttailxx
INTRODUCTION
Realistic, high-quality 3D creatures are critical for creating immersive experiences in vidco games, films, and AR/VR applications, making them essential part of modern digital media. While ad- vancements in graphics engines and technical expertise have enabled the production high-quality 3D content, the process remains labor-intensive, requiring multiple iterations and adjustments as well as substantial creative input.
To alleviate these challenges, the computer vision community has focused on breakthroughs in im- plicit neural representations (Mildenhall et al., 2021 Barron et al. 2021 Chen et al. [2022] Wang
al. 202I) and diffusion models based on textual descriptions, which provide an intuitive approach
to 3D content generation (Rombach et al. 2022; Sabaria et al., 2022, Nichol et al. 202 No-
tably, the introduction of the score distillation sampling (SDS) loss function (Poole et al. 2022 has
enabled the generation of diverse, high-quality 3D content by combining implicit neural representa-
tions with the strong priors provided by diffusion models (Wang et al. 2024 Lin et al. 2023).
While these methodologies provide both geometry and texture, converting implicit neural represen- tations into explicit formats, such as meshes, remains necessary for integration into graphics engines and real-time applications. Recently, DMTet (Shen et al.) (222]) has enabled precise mesh geome- try extraction from implicit representations by leveraging signed distance field and the Marching Tetrahedra algorithm. However, in texture synthesis, texture unwrapping often leads to inconsis- tent mappings, which can degrade the visual quality of the output or necessitate additional texture synthesis steps (Lin et al. 2023 Chen al. 2023b).
With significant advances in 3D geometry generation (Shen et al. 2021 Vahdat et al. 2022 Chen &
Zhang 2019 Nash et al. 2020 Müller al. 2023) and geometry optimization process (Shen al.

---

Published as a conference paper at ICLR 2025
TexTailor (Ours)
Text2Tex
TEXTure
Figure 1: (a): The illustration of definition of viewpoint. Following the red arrow, the pencil case mesh is painted from left to right in each row in (b). (b): This visualization illustrates the gradual shift in texture properties that becomes more pronounced the viewpoint changes. Compared to Text2Tex (Chen et al. (223a) and TEXTure (Richardson et al. 2023). TexTailor exhibits signifi- cantly less of this gradual shift.
2021
recent research (Chen et al. 2023a Richardson et al. 2023 Tang et al.
2024 Youwang et al.,
2024 Metzer et al.) [223) has focused on texture synthesis strategies for textureless meshes using
language cues. Among these approaches, several works (Tang et al. 2024 Chen et al.) 2023a
Richardson al. 2023) utilize inpainting techniques (Lugmayr et al.) [2022] within pre-trained
depth-aware image diffusion models (Zhag et al., 2023 Rombach al. 2022) to progressively
generate images, projecting them back onto the mesh for specific regions from predefined view- points. However, achieving coherent texture synthesis across all viewpoints remains a challenge for two key reasons: (1) The current inpainting techniques applied to texture synthesis are incomplete, information from previously synthesized visible textures at the current viewpoint is reflected onto the untextured areas only once per timestep in the diffusion process. This results in inconsisten- cies between adjacent viewpoints. (2) Additionally, progressively generating textures from multiple viewpoints introduces inherent sequential and temporal biases, gradually obscuring the texture infor- mation from the initial viewpoint. As a result, the characteristics of the texture synthesized from the initial viewpoint gradually degrade as the viewpoints shift, a phenomenon similar to the short-term dependency problems (Sutskever 2014) observed language models.
For instance, assuming the Y-axis is the vertical axis in the world coordinate system, we define the viewpoint as e (8, vh p). shown in Fig I (a). [ (b) shows the generated texture images of 'a pencil box" mesh as 0 decreases, with and fixed, following red arrow in Fig. (a). In comparison to TexTailor
Fig- [b), the texture properties (e.g., color and pattern) of the other methods, synthesized from the initial viewpoint, progressively degrade decreases. This gradual degradation intensifies over time, ultimately undermining texture consistency.
To address these challenges, present novel texture synthesis method, TexTailor. (1) TexTailor utilizes the resampling scheme proposed by Lugmayr et al. (002)) for texture synthesize fields, which helps repeatedly integrate information from previously synthesized textures. To enhance its effectiveness, we implement this scheme within the non-Markovian diffusion process (DDIM, Song et al. (2020)). And (2) we finetunes depth-aware text-to-image diffusion model using images generated through this resampling. During this process, we observe that finetuning with limited sample set risks reducing the model's ability to generate high-fidelity images conditioned on depth maps and textual descriptions. To mitigate this, we introduce an performance preservation loss, ensuring that the model learns the resampled distributions without compromising its generalization capabilities. Additionally, since the inpainting technique relies on previously synthesized visible textures, poorly predefined viewpoint locations that fail to consider the complexity the mesh ge- ometry can lead to suboptimal texture generation for the current viewpoint. To resolve this, we propose an adaptive viewpoint refinement scheme, which dynamically adjusts the viewpoint based on the amount of previously generated texture. This method enables the synthesis of coherent tex- tures across all viewpoints by adaptively positioning the viewpoint according to the geometry of the mesh.
2

---

Published as a conference paper at ICLR 2025
Experiments subset of the Objaverse dataset (Deitke et al. 2022) and the ShapeNet dataset (Chang et al., 2015) demonstrate the superior performance of TexTailor in synthesizing coherent 3D textures guided by textual descriptions. It surpasses state-of- -o-----ftthtttt art texture synthesis methods driven by language cues, achieving better results terms of LPIPS (Zhang et al. 2018) and FID (Heusel et al. 2017) subset of the Objaverse dataset. Our contributions are as follows:
We extend the resampling technique, previously used only in 2D) image inpainting, to the field of 3D texture synthesis by applying it to the DDIM MarkManno process, enabling consistent texture generation within single view with significantly fewer steps.
We analyze the gradual shift phenomenon in the texture synthesis process and propose a novel approach that trains the model using only few resampled images, removing the need for an external dataset of 3D meshes, textures, or text. By incorporating the performance preservation loss, this method effectively mitigates the gradual shift across multiple angles.
To address the catastrophic forgetting phenomenon, we propose the performance preserva- tion loss to mitigate this issue and maintain the model's performance. Experimental results demonstrate that this loss effectively prevents forgetting and maintains high texture quality across diverse viewpoints.
We introduce adaptive method that dynamically adjusts camera positions based on tex- ture coverage, eliminating manual configuration and ensuring consistent texture synthesis for complex geometries.
2 BACKGROUND
2.1 DIFFUSION MODELS
Diffusion models generate high-fidelity images by learning to iteratively convert a sample from
simple Gaussian distribution æт (0, I) into complex data distribution 2o q(<n) through diifsod(diffusion) and generative process (Ho et al., 2020 Sohl-Dickstein et al. 2015 Song et al.)
2020). The forward process, which approximates the posterior q( (o). is modeled as a Markov chain that progressively adds Gaussian noise to the data using coefficients ā1.7 E (0, 1]":
((1))) = II ((1 where q< == 1=1
مم- ( ) (I)
āi (v ai 0z-1
āe _
-1-1;
The generative process, modeled by the joint distribution po( gradually denoises noisy data, starting from p(ær) == (xT:; 0. T). During this process. a noise predictor < t), typically implemented using a U-Net architecture (Ronneherger et al., (015), predicts the noise in 24, [Song et al. (2020) introduce DDIM, non-Markovian diffusion process that enables different generative samplers by adjusting the generative noise variance, allowing for deterministic mappings. This approach reduces the number of sampling steps while maintaining the same marginals as DDPM.
However, training diffusion models in high-dimensional pixel space can be computationally ex- pensive. To address this. Rombach et al. (2022) proposed Stable Diffusion Model. which first use auto-encoding models to transform the data into a lower-dimensional semantic space before apply- ing the forward and generative processes. This dimensionality reduction significantly decreases the computational cost. Consequently, the noise predictor is trained in the semantic space as follows:
LLDM (0, 20) := E =
[w Vorzo + v1 oct <[1ē] (2)
where 2n is the latent vector of the input image To, obtained from the auto-encoder, e is condi- tioning vector (e.g., from textual descriptions), and w(t) is weighting term indexed by timestep Zhang et al. (202)) extends this approach by adding trainable encoder blocks connected to convolu- tional layers with zero weights (ControlNet) to the pre-trained noise predictor network cg(zz, f, c). This architecture ensures that the pre-trained diffusion model remains fixed during training, while normal, the newly added trainable blocks and layers learn conditional information, such as depth, and edge maps. This approach enables the model to generate data distributions controlled by addi- tional 2D spatial inputs including depth maps, allowing training with fewer parameters on smaller datasets without compromising pre-trained capabilities. In this paper, we utilize Stable Diffusson with ControlNet as a depth-aware T2I diffusion model, fine-tuning only the ControlNet component to synthesize partial textures conditioned on depth maps.
3

---

Published as a conference paper at ICLR 2025
2.2 TEXI-TO-TEXTURE GENERATION
Texture synthesis procedure. In re- cent works (Richardson et al. 2023
Ignore
Keep
Chen et al. 2023a; Tang et al. 2024).
New
the partial mesh surface viewed from single viewpoint
segmented into several regions and undergo incremen- tal texturing from one viewpoint the next to ensure both local and global consistency. Specifically, Chen al. (2023a) divides the partial surface into four regions ("keep". "new". "update", and "ignore") as illustrated in Fig. (b). and synthesizes partial texture using an image inpainting technique. The portion of the surface in the current viewpoint that has already been textured from the previous viewpoint(see Fig. 2aa) is labeled the "keep" re- gion, while the part that lacks texture called the "new" region. Only the "new" region undergoes partial texturing using the inpainting strategy, while the "keep" region remains unchanged. Addi- tionally, from the same viewpoint, surfaces where view direction is parallel to the normal vector of the object's visible faces are categorized the "update" region. This "update" region re-textured and projected again since it is viewed from a better angle than other regions. Consequently, partial mesh surfaces corresponding the "new" region are textured using information from the "keep" region across multiple viewpoints.
Update
(a) Current view point image
(b) Four regions
Figure 2: Left:
The image 2o rendered from the current viewpoint before being processed by the depth- dif- fusion model. Right: An illustration the four regions the partial mesh surface.
Image inpainting for texturing. During the texture synthesize process, an image inpainting strat- egy (Lugmayr et al. 2022) is applied to a depth-aware diffusion model (Rombach et al., 2022 Zhang et al. (203)) to effectively texture the missing regions of the mesh surface corresponding to the "new" "new"rreiion:
2t-1
known
N(V (1 āt).),
(3)
unknown
N(uul(2 oz((,,, t));
(4)
2 Zt-1 (1 (IMMtett)))
known
unkknown @ Mintents
(5)
where zo is the latent vector of the input image æp. which rendered from the current viewpoint of an object with texture applied from previous viewpoint (see Fig. a)). x is the latent vector sampled timestep / by adding noise 2o, while zunknown is the model's output the same timestep. Miatent represents a mask, with value of 1 for parts corresponding to the unknown region("new) U "ignore") on the image plane(see Fig. 2). resized match the size of the latent space. In the latent space, by combining zjinknown for the unknown region and known for the known region("keeg) the result is the composite ž,-1-
This process generates partial texture for the current viewpoint based on information from previous viewpoints. However, simply merging the known and unknown regions once at each timestep insufficient to fully harmonize the texture information synthesized from previous viewpoints during the denoising process the diffusion model.
3 METHOD
3.1 RESAMPLING
As mentioned Sec. 2 texturing an object involves progressively generating images using a depth- aware diffusion model and compositing them across multiple viewpoints. During the denoising process at each timestep, simply merging the known and unknown regions once is insufficient to seamlessly harmonize the unknown regions with the known ones, leading to inconsistencies be- tween adjacent viewpoints. Additionally, discordant textures synthesized from a single viewpoint accumulate through subsequent viewpoints, further degrading overall 3D texture consistency.
4

---

Published as a conference paper at ICLR 2025
"a hippopotamus"
Stable Diffusion
ControlNet
20
Known
Z
Known
ZTz
Known
Generative Process
9
2
7
22
2++1
Z,
ControlNet
Back-project
Lyine
Beff x((.)) (a)
-
Resampling
Lyre == E. Eefirccs (Wt)]le (2P,
corig corg
Figure 3: Overview of TexTailor. TexTailor synthesizes textures for given 3D mesh without tex- tures, based on a textual description, such as "a hippopotamus". We add additional camera positions to the predefined set properly condition the previously synthesized textures from each viewpoint, climinating the need for manually coordinating camera locations. Based on these viewpoints, we progressively generate textures using image inpainting techniques, including resampling scheme within non-Markovian process. To prevent the gradual shift in texture properties, we fine-tune ControlNet with small set of resampled images, incorporating performance preservation loss.
In 2D image inpainting field, Lugmayr et al. (2022) propose a resampling method that iteratively repeats adding noise and denoising, along with merging step (similar Eq, 5). multiple times each timestep to more thoroughly mix the known and unknown regions. However, directly apply- ing this method texture synthesis requires large number of timesteps, it operates within Markovian process, similar to DDPM, and must be repeated across multiple viewpoints. To gen- erate higher quality, harmonious images in fewer steps, we propose a resampling method within non-Markovian process using the following formula:
25 N
(v
at
21-11 (1
d1
11)
0t-1
ōt-1
2f vor
_ N
v / + v Ōi-1
V1 C2e .0
3/ = known
MiMMettt 2 ④ Mistents
(8)
where r denotes the resampling step at each timestep, r E [1..... R). and ED_ corresponds to ě; from E The resampling process at each timestep. as described in the equations above, occurs after the merging process defined by Eq, 5 Specifically, adding noise to _ the previously merged sample at timestep 1, as shown in Eq. 6 Eq. 2r-1 is sampled by denoising क according to the DDIM non-Markovian process. The resulting ž is then combined with z,nown, as described in Eq. where zimon! is derived from Eq. The timestep is part of the sequence (T...., J1), which is sub-sequence (T.... 1}. This approach allows high-quality texture synthesis by merging the "known" regions and "unknown" regions R
the r-th resampling, 2n sampled by
times with only 30 steps. This is significantly fewer than the 250 steps required by the original resampling method (Lugmayr al. 2022) for single view.
5

---

Published as a conference paper at ICLR 2025
3.2 FINE-TUNING WITH ORIGINAL CAPABILITIES PRESERVED
Using the resampling method (Sec. 3.1 within the depth-aware diffusion model, the partial mesh surface from the current viewpoint are coherently painted based solely on the pre- viously synthesized textures visible from that same viewpoint(corresooddng to "keep" re- gion). However, the autoregressive nature of the texture synthesis process, which relies only previously synthesized visible textures, leads to a gradual shift texture properties, such material pattern (see bb), ul- timately compromising the consistency the texture. This is because, the synthesis pro- gresses from the first viewpoint across multiple viewpoints, the texture information from the first viewpoint becomes increasingly obscured.
Furthermore, ControlNet (Zhang al. 2023) is responsible for extracting data distributions that align with depth maps the output domain of
Wo preservation
TexTailor
the diffusion model. However, it is challenging guide resampled data distributions from the diffusion model through ControlNet without in- corporating previously synthesized visible tex- tures (i.e., without using image inpainting techniques) in the current viewpoint. In other words, ControlNet itself does not retain or reflect the characteristics of previously synthesized resampled textures the current viewpoint.
Figure 4: Illustration of the loss of original gen- erative ability when fine-tuning ControlNet with- out the performance preservation loss.
These limitations hinder the ability to infer texture information for viewpoints distant from the first viewpoint using resampled textures from viewpoints close the first viewpoint. To address this challenge, we fine-tune ControlNet with a small set of resampled images near the first viewpoint to extract images of the same object from different angles in the output domain the diffusion model, allowing it to retain the texture properties across multiple viewpoints:
L Fine == Exffee- (00 [wo()]leo(z cr ca) -
(9)
where and are the text prompts and depth condition, respectively.
While this approach helps preserve texture properties during synthesis, it compromises ControlNet's original capability to extract images aligned with depth conditions and text prompts, phenomenon known as catastrophic forgetting. For instance, the generated textures from the depth-aware diffu- sion model finetuned with exhibit significant noise and reduced quality, as shown in Fig To address this issue, we introduce an performance preservation loss:
Tipre == Fg e (00000000000
[w<(t]]co<<]" Ge Ga) corig (=
R ,, Ct2 ca)
(10)
where Corig is the output of the fixed pre- trained network. This loss ensures that the noise prediction network does not deviate significantly from the original pre- trained parameters. Consequently, Our final loss is as follows:
LFinal LFine + XLpres
(11)
where À adjusts the strength of the performance preservation loss. Fig. iillustrates the entire pro- cess.
3.3 ADAPTIVE VIEWPOINT REFINEMENT
To achieve consistent texture synthesis using the inpainting method (Sec. 2.2 it is crucial set the camera's 's position for each viewpoint appropriately. This is because the outcomes of images generated by the inpainting technique are highly dependent the previously synthesized visible

---

Published as a conference paper at ICLR 2025
textures at the current viewpoint. A simple solution is to distribute as many viewpoints possible evenly around the object in 3D space. However, this approach presents two problems.
First, unnecessary predefined viewpoints make the overall texture synthesis process inefficient. Dis- tributing too many viewpoints is akin to sequentially painting the entire mesh surface in small, redundant sections, which adds unnecessary steps and even exacerbates the gradual shift in texture properties, even with the introduction of [T Second, complex geometries can lead to camera positions where textures from previous viewpoints are not sufficiently visible. Finding evenly dis- tributed viewpoints v = (0, e; p) in 3D space often requires numerous trials and errors to achieve precise viewpoint settings. Furthermore, optimal camera positions vary depending on the object's geometry, making manual configuration particularly difficult, especially for large-scale objects.
To select optimal camera positions, we utilize the ratio of the "keep" region to the "new" region at each viewpoint. We render each region from the meshes onto the image planes and convert them into masks where a value of one represents the corresponding region. Then, we calculate the proportion of pixels corresponding to the "keep" region relative to the total of the "keep" and "new" regions on the image plane:
Mi 1 (k.))
p Eig 1 (new) + Li (k.))
(12)
If the proportion p calculated in Eq. 12 is smaller than a threshold 8, we add an additional camera position( ( (0prev +- Pprev)) 1/pree + (Peurrent ppree)), p)) the predefined camera position sequence [u),-.., }. Here, E (0, 1) is an interpolation parameter, and we interpolate the azimuth (0) and elevation (1)) angles between the previous and current viewpoints. This adaptive approach ensures that the camera positions are optimized based on the object's geometry and the
amount of previously generated texture.
4
EXPERIMENTS
4.1 EXPERIMENT SETUP
Implementation Details. We select a subset of the Objaverse dataset (Deitke et al. 2022) to eval- uate our model, following the approach Chen et al. (2023a). In this dataset, Chen et al. (2023a)
filter out low-quality or misaligned meshes from the designated categories, resulting in 410 tex- tured meshes across 225 categories for experiments. Notably, the original textures exclusively for evaluation. Recent text-driven texture synthesis methods (Richardson et al. 2023 Metzer al., 2023 Youwang et al. 2024), which rely on gradient-based rendering using a differen- tial renderer (Fuji Tsang et al. 2022). encounter difficulties when optimizing texture synthesis for 'car' objects from the ShapeNet dataset (Chang al., 2015). Therefore, we present only qualitative results for our approach on this dataset to demonstrate its performance on fine-grained categories compared to Text2Tex.
used
For both datasets, the number of resampling steps, as well as the parameters X, B, and 7, are set to 3, 2.5, 0.5, and 0.5, respectively. We finetune the ControlNet using five images rendered from viewpoints close to and including the first viewpoint: V1 (0%, 15", 1), V/2 (0", 35, 1). Va- (0" 1). 24 = (20', 15", 1), and v; = (340", 1150, 1). For rendering and texture projection, we utilize the PyTorch framework (Paszke al. 2017) along with PyTorch3D (Ravi al. 2020).
Evaluation metrics. To quantitatively measure the view-consistency of synthesized textures, we report the average LPIPS (Zhang et al. 2018) between images rendered from 3D textured meshes across multiple viewpoints. A gradual shift texture properties or the presence of artifacts as- sumed to increase the perceptual loss between any two viewpoints. To calculate average LPIPS, We sample 25 uniformly spaced camera positions from both the upper and lower hemispheres of fixed- radius sphere. All cameras within the upper hemisphere share the same clevation, and similarly, all cameras within the lower hemisphere share the same elevation, with each set directed towards the center of the sphere. An image is rendered from each viewpoint a resolution of 512 x 512. We then compute the average LPIPS values for all pairs of images in the 3D scene and sum the averages across all evaluated categories.
Additionally, we assess the quality and diversity of the generated textures using the Frechet Incep- tion Distance (FID) (Heusel et al. 2017). The synthesized texture distribution consists of rendered
7

---

Published as a conference paper at ICLR 2025
"a sushi"
"a hammer"
"a dumpster"
"a CD player"
Latent-Paint
Text2Tex
TEXTure
Paint-it
Ours
Figure 5: Qualitative comparisons on Objaverse. We compare TexTailor with state-of-the-ar baselines Metzer al. 2023 Chen et al. 2023a Richardson et al. 2023 Youwang et al., (024) on Objaverse meshes. Compared to other methods, Tex Tailor produces textures that are more view- consistent and better aligned with object geometries.
images from the above camera points, while the real distribution is composed of renders of the meshes under the same conditions, using artist-designed textures.
Quantitative comparisons. In Tab. I we compare our approach against recent SOTA text-driven texture synthesis methods: Lattent-Paint (Metzer al. 2023). Text2Tex (Chen et al. 2023a). TEXTure (Richardson et al. 2023), and Paint- it Youwang et al. 2024), on a subset the Objaverse dataset. For texture descriptions, template, "a (category)", is utilized consistently across all approaches. The results demonstrate that our method synthesizes high-quality and more consistent textures than all baselines across various categories.
Table Quantitative comparison Objaverse subset.
Method
LPIPS
FID
Latent-Paint
54.429
45.334
Text2Tex
38.931
33.487
TEXTure
54.138
43.337
Paint-it
38.316
39.823
Ours
37.889
29.998
Qualitative comparisons. In Fig. 5 present the dered results using TexTailor alongside other baselines on the Objaverse dataset. The qualitative results demonstrate that our approach significantly improves texture quality, particularly in terms of
8

---

Published as a conference paper at ICLR 2025
"AudiAS A3"
"Modified Aston Martin DB9"
Ours
Text2Tex
Figure 6: Qualitative comparisons ShapeNet Our approach synthesizes more view-
consistent and higher-quality textures for fine-grained categories compared to the baseline.
w/ Resampling
w/ Training
w/Perf.LLss
w/ View Refine
LPIPS
FID
38.89
30.924
XX
39.85
53,855
38.00
30.567
V
37.89
29.998
Table 2: Table of Ablation Studies.
new-connistency. Specifically, some methods based on autoregressive processes for texture synthe- sis exhibit gradual shift in texture properties (see Fig. T. leading reduced view consistency over- all. Other approaches that employ score distillation sampling loss (SDS loss) enhance texture quality by optimizing networks that generate texture maps the texture maps themselves. However, they cannot fully overcome the inherent limitations of SDS loss, such as oversaturation, over-smoothing, and low diversity (Wang al. 2024; Poole al., 2022). For example, the case of "a hammer" (second column in 5p. our method successfully textures both the handle and head of the hammer mesh without gradual shift in each part's texture properties, and avoiding artifacts across multiple viewpoints. These limitations in the other baselines are also reflected in the quantitative results in Tab. 1 Fig. [6] due optimization problems with the differential renderer, we only report the texture results for ShapeNet cars using our method and Text2Tex. Notably, even with fine-grained categories, our results show high-quality, view-consistent textures without the aforementioned prob- lems. Additional qualitative comparisons for texture synthesis Objaverse dataset objects and human meshes can be found in the appendix.
4.2
ABLATION STUDIES
Effects of resampling. First, we incorporate the resampling method into our baseline, as shown in Fig. 7 (a). The resampling method better preserves the texture properties of the chocolate-covered cherry and muffin liner from the first viewpoint (#1) at the subsequent viewpoints, U2 and Va, com- pared to the baseline. However, we still observe gradual change in the pattern of the muffin liner from U2 to v4, and this change becomes more pronounced at tio which is far from vn, in left ex- ample of Fig. (b), compromising view-consistency. This tendency is clearly reflected in the results shown in Tab. 2
Effects of training with resampled texture. To address this problem, we fine-tune ControlNet on the resampled textures from five viewpoints (U1, V2 V3. Ug, U3) with the performance preservation loss. as shown in Fig. 7 (b). As result, the ability to preserve texture properties is significantly improved. We now observe that the muffin textures from #11-a viewpoint diametrically opposite to vn with respect to the origin-an are similar to the textures from vn.
Effects of adaptive view refinement. In Fig. c), we illustrate the texture synthesis process from the side view to the bottom view of the muffin mesh. If the user does not define appropriate mid-

---

Published as a conference paper at ICLR 2025
()) Effeets of resampling
Baseline
+Resampling
V1
V2
V
1/2
D4
(b) Effects of training with resampled textures +Resampling
+Training
V00
174
V10
V11
V11
(c) Effects of adaptive view refinement Training
+ Adaptive view refinement
V11
V12
V11
Vi
Figure 7: Ablation studies. The illustration demonstrates the effectiveness of the key components of TexTailor. As each component applied, the muffin's texture becomes more consistent across multiple viewpoints, and the overall texture quality improves.
point between these views, as shown in the left example, the model generates a texture that is entirely misaligned with the previously synthesized in terms of geometry. Conversely, proper compensa- tion at predefined viewpoints effectively prevents this phenomenon. Our method eliminates the need for such cumbersome manual efforts, including tracking and precisely defining camera positions for different geometries.
CONCLUSION
In this paper, we propose TexTailor, novel method for synthesizing high-quality textures based on language cues for 3D meshes across multiple categories. Our methodology utilizes a resampling scheme successfully harmonize unknown regions with known regions inpainting techniques, thereby improving view consistency between adjacent viewpoints. To address the gradual shift in texture properties, fine-tune small set of resampled texture images using ControlNet with an performance preservation loss. Additionally, the adaptive view refinement technique enhances quality and view consistency by leveraging previously synthesized textures, eliminating the need for manual coordination of camera positions based on the object's geometry. Experiments on datasets from various categories demonstrate the superior performance of our approach in synthesizing high- quality, view-consistent textures.
ACKNOWLEDGEMENTS
This research has been supported the LG Electronics Corporation. (Project No. G01230381)
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Renderpeople. 2023. URL Btt.:p/ss//eeeeepppee.....
Jonathan Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-ruualla, and Pratul Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. In Proceedings of the IEEE/CVP international conference computer vision, pp. 5855-5864, 2021.
Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, al. Shapenet: An information-rich 3d model reposi- tory. arXiv preprint Xiv: Xiv: 1512.03012, 2015.
Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su. Tensorf: Tensorial radiance fields. In European conference computer vision, pp. 333-350. Springer, 2022.
Dave Zhenyu Chen, Yawar Siddiqui, Hsin- Ying Lee, Sergey Tulyakov, and Matthias Nießner. Text2tex: Text- driven texture synthesis via diffussin models. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 18558-18568, 2023a.
Rui Chen, Yongwei Chen, Ningxin Jiao, and Kui Jia. Fantasia3d: Disentangling geometry and appearance for high-quality text-to-3d content creation. In Proceedings of the IEEE/CVF international conference on computer vision, pp.
2023b.
Zhiqin Chen and Hao Zhang. Learning implicit fields for generative shape modeling. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5939-5948, 2019.
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. arXiv 2022.
preprint
Clement Fuji Tsang, Maria Shugrina, Jean Francois Lafleche, Towaki Takikawa, Jiehan Wang, Charles Loop, Wenzheng Chen, Krishna Murthy Jatavallabhula, Edward Smith, Artem Rozantsev, Or Perel, Tianchang Shen, Jun Gao, Sanja Fidler, Gavriel State, Jason Gorski, Tommy Xiang, Jianing Li, Michael Li, and Rev Leharedian. Kaolin: A pytorch library for accelerating 3d deep learning research. httpa://githh: 2022.
Com Com/NVINH IDI A.aameworkskkao"
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Berhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rulc converge to local nash equilibrium. 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Edward Hu, Yclong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint 106.:2200.009655, 2021.
Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming- Yu Liu, and Tsung-Y Yi Lin. Magic3d: High-resolution text-to-3d content creation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 300 800-309, 2023.
Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Lue Van Gool. Repaint: Inpainting using denoising diffusion prohabilistic models. In Proceedings of the IEFE/CVF conference on computer vision and puttern recognition, pp. 11461-11471, 2022.
Gal Metzer, Elad Richardson, Patashnik, Raja Giryes, and Daniel Cohen-Or. Latent-nerf supe-guided generation of 3d shapes and textures. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12663-12673, 2023.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes neural radiance fields for view synthesis. Communications of the ACM, 65(1): 99-1006 2021.
Norman Müller, Yawar Siddiqui, Lorenzo Porzi, Samuel Rota Bulo, Peter Kontschieder, and Matthis NieBner. Diffif: Rendering-guided 3d radiance field diffusion. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pp. 4328 4338,, 2023.
Charlic Nash, Yaroslav Ganin, SM Ali Eslami, and Peter Battaglia. Polygen: An autoregressive generative model of 3d meshes. In International conference machine learning, pp. 7220-7229. PMI.R, 2020.
11

---

Published as a conference paper at ICLR 2025
Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorcalistic image generation and editing with text-guided diffusion models, arXiv preprint arXiv:21 2021.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang. Zachary Vitito. Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation pytorch. 2017.
Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209. 14988, 2022.
Nikhila Ravi, Jeremy Reizenstein, David Novotny, Taylor Gordon, Wan- Yen Lo, Justin Johnson, and Georgia Gkioxari. Accelerating 3d deep learning with pytorch3d. arXiv:2007.08501, 2020.
Elad Richardson, Gal Metzer, Yuval Alaluf, Raja Giryes, and Daniel Cohen-Or, Texture: Text-guided texturing of 3d shapes. In ACM SIGGRAPH 2023 conference proceedings, pp. 1-11, 2023.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, Bjöm Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10684-10695
June 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted Iitervention-MICCA 2015: 18th inter- national conference, Mumich, Germany, October 5-9, 2015, proceedings, part III 18, pp. 234-241. Springer, 2015.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li. Jay Whang, Emily L Denton, Kamyar Chasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information processing systems, 35:36479- 36494, 2022.
Tianchang Shen, Jun Gao, Kangxue Yin, Ming- Yu Liu, and Sanja Fidler. Deep marching tetrahedra: a hybrid representation for high-resolution 3d shape synthesis. Advances in Neural Information Processing Systems, 34:6087-6101, 2021.
Jascha Sohl-Dickstein, Eric Woiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised lcarning using nonequilibrium thermodynamies. In International conference machine learning, pp. 2256-2265. PMLR, 2015.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv20000002502, 2020.
arXiv preprint
I Sutskever. Sequence to sequence learning with neural networks. arXiv preprint arXiv: 1409.321, 2014.
Jiaxiang Tang, Ruijic Lu, Xiaokang Chen, Xiang Wen, Gang Zeng, and Ziwei Liu, Intex: Interactive text-to- texture synthesis via unified depth-aware inpainting, arXiv preprint arXiv:2403. 11878, 2024.
Arash Vahdat, Francis Williams, Zan Gojeic, Or Litany, Sanja Fidler, Karsten Kreis, et al. Lion: Latent point diffusion models for 3d shape generation. Advances in Neural Information Processing Systems, 35:10021- 10039, 2022.
Peng Wang, Lingjic Liu, Yuan Liu, Christian Thcobalt, Taku Komura, and Wenping Wang. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. NeurlPS, 2021.
Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolifiedreamer: High- fidelity and diverse text-to-3d generation with variational score distillation. Advances in Neural Information Processing Systems, 36, 2024.
Kim Youwang. Tae-Hyun Oh, and Gerard Pons-Moll. Paint-it: Text-to-texture synthesis via deep convolutional texture map optimization and physically-based rendering. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 4347-4356, 2024.
Lymin Zhang, Anyi Rao, and Mancesh Agrawala. Adding conditional control to text-to-image diffusion mod- els. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3836-3847, 2023.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unrcasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018.
12

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1
ADDITIONAL QUALITATIVE COMPARISON
"a minivan"
"a money"
"a basketball"
desk chair"
Latent-Paint
Paint-it
TEXTure
Text2 Tex
Ours
Figure 8: Additional qualitative comparisons on Objaverse. We compare Teiloi with state- of-the-art baselines (Metzer et al. 2023 Chen et al. 2023a Richardson et al. 2023 Youwang al.) 2024) on Objaverse meshes. Compared to other methods, Tex Tailor produces textures that are more view-consistent and better aligned with object geometries.
To further validate the effectiveness of our proposed methodology, we provide qualitative comparisons for four additional objects from the Objaverse dataset, complementing the four objects already showcased in the qualitative results of the paper (see Fig. 5 Specifically, methods that rely on auto-regressive processes for texture synthesis often encounter gradual shift in texture properties, leading the emergence of texture seams that significantly degrade overall texture quality across multiple viewpoints. For instance, in the cases of Text2Tex (Chen et al.) 2023ab and TEXTure (Richardson al., 2023) with the prompt "a basketball", the texture patterns, colors, and materials of the basketball are inconsistently generated, resulting in noticeable variations within single viewpoint. Similarly, with the prompt "a desk chair," " the chair exhibits patches of paint in colors such as white or blue, further emphasizing these inconsistencies.
Objaverse Dataset
In contrast, approaches based on SDS loss face challenges stemming from intrinsic limitations the loss itself, such oversaturation and low diversity. For example, while Latent-Paint (Metzer et al.) 2023) generates textures that appear relatively well-synthesized, it frequently displays strong bias toward red-dominated textures. Similarly, Paint-it (Youwang et al. (024) often suffers from visible oversaturation artifacts, detracting from the overall realism of the generated textures.
13

---

Published as a conference paper at ICLR 2025
Latent-Paint
Paint-it
TEXTure
Text2
Figure 9: Additional qualitative comparisons RenderPeople (ren) 2023). Qualitative com- parison of generated textures for clothed human meshes using various text prompts. Each column corresponds to different text prompt: (Ist col) 'A woman wearing white blouse with ribbon detail, light beige pants, nude-tone heels, and neatly tied blonde hair', (2nd col) 'A man, wearing black vest, black formal trousers, black tie, black belt, white dress shirt, and dark formal shoes, with short neatly styled hair', (3rd col) 'A man, wearing a gray short-sleeve T-shirt, blue jeans, white sneakers, and short, dark brown hair styled neatly', and (4th row) 'A woman with dark skin tone, wearing a black blazer, a black top, gray pants with gray tied belt, black heels, and having dark hair'. The results demonstrate the performance of our method across diverse prompts and viewpoints.
Clothed Human Dataset we provide qualitative comparison results for clothed human meshes from RenderPeople (ren 2023). The text prompts for RenderPeople's clothed human meshes were created based on the ground truth images provided by RenderPeople, which were designed by pro- fessional designers. Texture synthesis for human meshes demonstrates the superiority of our ap- proach in terms of consistency and quality. For instance, when examining the texture results for the male mesh corresponding the prompt the second column 'A man, wearing black vest, black formal trousers, a black tic, a black belt, a white dress shirt, and dark formal shoes, with short neatly styled hair' other methods exhibit phenomenon of textual gradual shift. This causes the clothing colors shifting inconsistently between white, black, and gray, making it challenging for the models to faithfully reproduce the clothing described in the prompt. In contrast, Tailor effectively elim- inates this issue, accurately representing the prompt with textures like "a black vest" and "a white dress shirt", ensuring precise alignment with the described attire. In addition to clothing, the head region also presents challenges. For instance, in the top view, textures applied to other regions (such as the face, upper body, or lower body) are often not visible, leading to decline texture quality for the top view. However, Tex Tailor addresses this issue by utilizing the adaptive view refinement
14

---

Published as a conference paper at ICLR 2025
technique, which transfers texture information from regions like the face and clothing. This ensures seamless texture generation without visible texture seams, even in challenging viewpoints like the top or bottom view.
A.2
ADDITIONAL QUALITATIVE RESULTS
(a) Glossy objects
(b) Metallic objects
(c) Translucent objects
"a bowl"
"a faucet"
"a light bulb"
"a cup"
"an aerosol can"
"a candle"
Figure 10:
Additional qualitative comparisons non-diffuse objects from the Objaverse dataset. We present the texture synthesis results categorized into three types: (a) Glossy objects, (b) Metallic objects, and (c) Translucent objects.
We further demonstrate the texture quality of not only diffuse objects but also non-diffuse objects from the Objaverse dataset, categorized into three types: (a) glossy objects, (b) metallic objects, and (c) translucent objects. The results generally show satisfactory texture quality across all categories. However, for regions where light reflection or shadows occur, these features are pre-generated based on the synthesis process rather than dynamically determined by the lighting direction in the user's s graphics software. This could pose challenges for practical applications. Nevertheless, this limita- tion is not unique to approach and is observed in other methods well. Developing techniques to prevent such biases in light effects during the texture synthesis stage will be a focus of our future work.
A.3 ADDITIONAL ABLATION STUDIES
Effects of resampling. When observing the texture of "a basket," the transition from vy to tn reveals a change in the texture of the inside of the basket in the left part. This occurs in the baseline method because the previously synthesized texture visible from the current viewpoint only merged once per timestep during the diffusion process. In contrast, our methodology improves this by utilizing the resampling strategy proposed in (Lugmayr et al. 2022), which allows the texture be merged multiple times per timestep. resulting in a more consistent outcome.
Effects of training with resampled texture. When observing the object "a briefcase." applying the resampling method generates consistent textures for viewpoints that are close to the initial view- point Vo, such as the nearby viewpoint VG. However, as the viewpoint moves further away from tip. variations in texture properties begin to appear, and at the opposite viewpoint Vo. the texture prop- erties have completely changed. In contrast, when the depth-aware T2I model is fine-tuned using five resampled texture images from viewpoints adjacent vo with performance preservation loss, the model fits to the distribution of the resampled textures, leading to noticeable improvements in consistency.
15

---

Published as a conference paper at ICLR 2025
(a) Effects resampling
"a basket"
Baseline
+Resampling
Vo (b) Effects training with resampled textures
V
V1
"a briefease"
+Resampling
+Training
Vo (c) Effects adaptive view refinement
V5
Vg
Vs
Vg
+Training
+ Adaptive view refinement
V11
V22
V11
V13
V16
Figure 11: Ablation studies. The illustration highlights the effectiveness of the key components of TexTailor. As each component is applied, the textures various objects become increasingly con- sistent across multiple viewpoints, demonstrating improved texture quality regardless of the object type.
Effeets adaptive view refinement. When observing the object cappuccino", transitioning from V1 to U12 the left part reveals an issue where the bottom of the cup is generated incorrectly. Since there is no texture information from the previous viewpoint, the model generates the top of the cappuccino instead of the bottom. However, by using the adaptive view refinement technique the right part, an intermediate viewpoint (W1s) is automatically added. This not only provides the texture information from V11 to guide more natural texture synthesis but also eliminates the need for the tedious process of manually configuring optimal camera positions.
A.4 LIMITATIONS
TexTailor faces several limitations. First, the overall texture quality heavily relies the quality of the five images used to fine-tune the depth-aware T2I diffusion model. Even with the resam- pling method, certain object angles, even from viewpoints close the initial one, may still produce textures with inconsistent properties or images misaligned with the depth condition. This reliance on suboptimal training images can sometimes degrade texture quality rather than enhance Ad- ditionally, patterns frequently observed in viewpoints near the initial one often repeat across the object, leading to unrealistic and repetitive textures. For instance, in the case an alarm clock, the clock hands learned from the initial viewpoint might appear as texture patterns the sides even the back. Furthermore, TexTailor's 's fine-tuning process time-intessive, requiring approximately NVIDIA TITAN RTX. Finally, LPIPS does not adequately
hour and half per 3D mesh capture consistency across multiple viewpoints due to spatial misalignments in overlapping sections between adjacent views. To address these limitations, integrating lightweight fine-tuning approaches
16

---

Published as a conference paper at ICLR 2025
such LoRA et 2 and developing improved metrics for measuring consistency between arbitrary pairs of views offer promising directions for future work.
A.5 HIGHLIGHTING DIFFERENCES: A ZOOMED-IN VIEW
Tex
Ours
Figure 12: Zoomed-in qualitative comparisons between TexTailor and the baseline Text2Tex The red boxes highlight regions where visual differences in texture consistency and quality are more apparent, helping to illustrate the effectiveness of TexTailor in preserving texture properties and minimizing artifacts.
We aim to enhance the clarity of the qualitative comparisons between TexTailor and the baseline Text2Tex by providing zoomed-in patches for four challenging cases ("a sushi," "a hammer," "a minivan," " and "a desk chair") where visual differences are harder to discern in the qualitatve com- parison sections(Sec. 5 and Sec. 8).
A.6 ABLATION STUDY ON THE THRESHOLD B
We compare our results in Tab. by varying the threshold 3 value from Eq. 12 For this experiment, randomly select 100 meshes from the Objaverse dataset, ensuring no overlapping categories among the 400 meshes, and evaluate them using FID and LPIPS metrics. When the threshold is set too low (e.g., 0.25), the information for the keep region is not properly reflected, resulting in lower scores. Conversely, when the threshold is set too high (e.g., 0.75), objects tend to be painted in overly fragmented parts rather than as cohesive units. This intensifies the gradual shift effect, also leading to lower scores. Based on these observations, set to 0.5, balanced value, for all experiments in this paper.
Table 3: Quantitative effect varying threshold 3 on the adaptive view refinement technique.
0.25
0.5
0.75
LPIPS + FID 1
8,99464 58.567
8,99176
8.99468
57.799
59.503
17


Reviews:
[{"review_id": "XJZS0RTOPd", "reviewer": "Reviewer_uECh", "summary": "This paper proposes TexTailor, a method for text-to-texture synthesis utilizing an inpainting approach to achieve view-consistent textures. To address common challenges in texture generation, TexTailor introduces a resampling scheme and fine-tuning to maintain texture consistency across viewpoints. Furthermore, it employs adaptive viewpoint refinement for efficient viewpoint sampling.", "questions": "1. Could the authors clarify the novel aspects of TexTailor? The current version of TexTailor appears to be a combination of existing methods. It would be helpful if they could elaborate on any unique modifications within the resampling scheme or improvements made to accelerate the fine-tuning process.\n2. While the paper discusses performance preservation loss qualitatively, a quantitative analysis of its impact on quality would clarify its specific role. Including an ablation study of the performance preservation loss in Table 2 could better highlight its contribution to TexTailor’s performance.\n3. Since the viewpoint refinement uses a fixed threshold, how sensitive is the model’s performance to changes in this parameter?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper proposes TexTailor to address view-consistent texture synthesis by combining inpainting with resampling and fine-tuning.\n2. Method and results are presented clearly and logically, making the paper easy to follow.", "weaknesses": "1. While effective, the approach primarily combines existing techniques, with limited emphasis on novel contributions. The paper could be strengthened by enhancing the resampling scheme or accelerating the fine-tuning phase.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZqRiAGQrUf", "reviewer": "Reviewer_3JDj", "summary": "This paper introduces a method for text-driven 3D object texturing. Previous work on this important topic fail in some areas, according to the paper, including: Consistency and the gradual change in textures assigned to the object. The paper aims to solve these issues, by introducing 2 ideas: First, the model leverages a resampling scheme for better integration of previously generated texture during the diffusion process, and second, the model fine-tunes a depth-aware diffusion model with these resampling textures. With these contributions, the method is said to achieve higher quality and consistency than previous work, measured on a set of datasets and perceptual metrics.", "questions": "- Can the authors provide a user study that measures consistency, alignment, quality, and realism? This should provide a better idea on the quality of the results on the actual goals that the paper aims to achieve. \n- I believe that computational costs should be compared more explictly with previous work, so as to better understand the quality/cost pareto frontier in this line of work.  \n- I suggest the paper should provide a CLIP-guided text-image alignment metric. \n- I suggest the paper should provide a more upfront discussion of its limitations.\n- The paper should include a detailed analysis of 2D texturing models.\n- The paper should include a detailed analysis of text-to-avatar models, as well as quantitative and qualitative comparisons. \n- How does the model behave with non-diffuse objects? Very few glossy, metallic, or translucent objects are shown. \n- The paper should include many more results, at least in the supplementary material.\n- Results should include standard deviations to better understand the differences between methods in terms of LPIPS, FLIP, etc.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- This paper adequately identifies problems in previous methods for text-driven object texturing, including lack of texture consistency and graduality in texture changes. The origin of this problems are identified as being caused by insufficient integration, predefinition of camera positions, and autorregresion. The paper introduces changes to these methods, to enhance their quality and consistency. This is an important line of research, as these works are becoming more prevalent in the literature and industrial applications. \n- This paper tackles an important and salient problem in the literature. \n- The results shown in the paper indeed suggest that the proposed method provides less gradual changes in texture properties. In objects with different parts, TexTailor shows superior performance in assigning different texture to different parts, than previous work do. \n- The proposed method is sound, and the ideas proposed here are very well suited for the task the paper is aiming to solve. In this sense, the paper is correct as far as I am familiar with the literature and the problems in 3D content generation.\n- This paper is well written and easy to follow. The problems identified in previous work are clearly stated, and the ideas to solve them are easy to understand and very well explained. \n- Code is provided as a supplementary material, which should greatly enhance reproducibility.", "weaknesses": "- While sound, the ideas introduced in this work are somewhat limited in scope and the paper fails to be compelling that they are particularly effective. In this sense, I am not convinced about the extent upon which these contributions will be impactful in the literature. Furthermore, the resampling scheme introduced in this paper is not new, as it is borrowed from previous work. Therefore, the ideas introduced here are not particularly novel nor signficant.\n- Insufficient results are shown on the paper. It is hard to understand the capabilities of the model with the amount of results shown here. In particular, only four results are shown in the comparisons on Objaverse, and these results are not particularly compelling (for example, in the hammer, the method assigns a metal texture to the handle and a wooden texture to the head, which is not correct and arguably a worse result than TEXTure). Only two results are shown in ShapeNet car, and the ablation study is shown exclusively on a single object. Significant more results should be provided to convince the reader that the method is more effective than previous work.\n- I am unconvinced about the metrics used in this paper. While standard for 3D object texturing work, LPIPS and FID do not adequately measure text-to-image alignment. CLIP-based metrics should be used in conjunction to the ones shown in this paper, to be more informative about how well this model is generating results aligned with the prompts. While visually more consistent than previous work, this model seems to struggle more than previous methods (particularly TEXTure and Text2Text) in asigning the correct texture to each part of the object. This is not something that LPIPS and FID can measure correctly. \n- A user study should be provided for better comparisons between methods, across a bunch of dimensions, including alignment, realism, quality, consistency, etc. \n- The quantitative results are not particularly convincing. The ablation study does not show significant improvements across the metrics used, particularly LPIPS, and without standard deviations of the errors it is hard to understand whether the improvements are actually statistically significant. Therefore, the ablation fails to convince that the proposed contributions are actually valuable and effective. \n- No results are shown on 3D human avatar texturing, which is a very closely related and relevant line of work. \n- Related to the previous point, the analysis of the related work is lacking on a set of areas. The most relevant is the work on 3D human texturing. Relevant work include: SMPLitex: A Generative Model and Dataset for 3D Human Texture Estimation from Single Image (BMVC 2023), UVMap-ID: A Controllable and Personalized UV Map Generative Model (ACMMM 2024), TexDreamer: Towards Zero-Shot High-Fidelity 3D Human Texture Generation (ECCV 2024), etc. Besides, 2D texturing methods with generative models should also be included as part of the related work. \n- Limitations are not adequately addressed or discussed. From the paper, it seems like the only limitation of the proposed method is its computational cost. However, the results shown in the paper indicate that the method is by no means perfect and it struggles with consistently assigning the appropriate texture to different parts of the object, among other limitations. These should be mentioned more explicitly. \n- Contributions are very overstated. Sentences like \"... demonstrate the superior performance of TexTailor in .... \" or \" ... surpases SOTA texture synthesis methods driven by language cues\" should be empirically demonstrated or removed altogether.\n- The paper suggests some reasons why previous methods fail (autorregressive inference, integration of previous information, fixed camera positions, etc), but it fails to provide adequate evidence that these actually limiting factors.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zXhu56q5lj", "reviewer": "Reviewer_ujQC", "summary": "The paper focuses on **consistent texture synthesis**. The authors analyze the artifacts in the current approaches and propose a new approach, **TexTailor**, to keep synthesized textures consistent across different viewpoints. TexTailor equips with a resampling scheme for integrating previous textures, a finetuned depth-aware T2I model trained with performance preservation loss, and an adaptive viewpoint refinement strategy for inpainting. \n\nThe authors evaluate the performance of TexTailor on a subset of Objaverse dataset, and showcases that TexTailor outperforms state-of-the-art methods.", "questions": "1. What are the difficulties for the text-driven methods mentioned in Lines 373 and 374? \n2. Is LPIPS (Section 4.1, Evaluation metrics) a good metric to evaluate view consistency, as LPIPS is sensitive to spatial information? Given that the view angles are known, would it make more sense to reproject one of the views to another and then compute LPIPS between the projected view and the other one?\n3. What does the performance preservation loss do in Eqn. (10)? Why would it be effective at a high level? \n\nSome of the questions may have been entangled with the Weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 1, "contribution": 2, "strengths": "## Motivation\nThe paper starts with an analysis of the limitations of previous methods. It hypothesizes those inconsistent results from previous methods are mainly coming from an inappropriate way of integrating information from previously synthesized textures. Given this agile insight, it tries to addresses the inconsistency issue by proposing a new approach to better use information across different viewpoints and previously synthesized textures. \n\nThe motivation of the paper is more about a technical aspect. The analysis of previous approaches makes sense. \n\n## Method\n- In Section 3.2, the problem of ControlNet for incorporating multi-views is interesting. \n- In Section 3.3, the analysis of setting viewpoints sounds interesting (Line 303-310). Using a proportion (Eqn. (12)) is an intuitive way. \n\n## Experiments\n- TexTailor outperforms the previous methods in terms of view consistency and quality, as shown in Table 1. \n- The ablation study shows a progressive improvement of each component. \n\nThe authors also show the limitation of TexTailor - the processing time could be further improved.", "weaknesses": "What concerns me the most in this paper is the motivation behind some technical parts and its unclear writing. \n\n## Motivation\n- In Line 93, it is not clear to me why finetuning a depth-aware T2I model matters. Maybe including a brief explanation could be helpful. \n\n## Method\n- In Section 3.1, the authors propose a non-Markov process to reduce the sampling steps. However, the benefits of it is confusing to me. Would it involve a faster sampling speed? If it would, there is not result to support it. On the other hand, the authors mainly show the effects of resampling is to \"preserve the texture properties\" (Line 480). This makes me confused about the motivation of newly proposed resampling trick. \n\n## Experiments\n- It does not make sense to me the authors choose to not compare with text-driven methods (Line 373-374) just because they have \"difficulties\" when optimizing textures for \"cars\". Wouldn't it be a good chance to showcase the superiority of TexTailor?  \n- The authors do not show any viewpoint-varying results in video format, making it less convincing that TexTailor achieves a good view consistency. \n- It is hard to see obvious improvement from TexTailor in Figure 5, especially comparing with Text2Tex. Perhaps including some bounding boxes and zoom-in patches would help. \n\n## Writing/Delivery\nThe writing of the papers can be further improved. For example,\n- Most of the figures in the paper are compressed, resulting in blurriness and sometimes hard to read. \n- In Fig.1, citing previous methods (i.e., Text2tex and Texture) might make readers easier to check the idea of them. \n- It is challenging for readers to digest Eqn. (6) - (8). A good strategy to improve it might be similar to what Repaint shows in their paper: demonstrating all the terms in a figure with pictures for a vivid demonstration. Current delivery of newly proposed resampling way in Section 3.1 is hard for readers to understand, especially about the main difference between it and Repaint. \n- Fig.3 does not deliver a clear message for each component. For example, simply giving readers two equations does not help them to understand what is going on. It might be helpful if the authors can name these two equations in high level.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WpfNOs2ZxC", "reviewer": "Reviewer_NBoV", "summary": "This paper proposed a novel architecture that can generate more consistent 3D texture than TEXTure and Text2Tex.", "questions": "1. I do not get the correlation of the third paragraph in Section I with the main content. I think the geometry conversion is not a problem to be solved in this paper. And the SDS can be directly applied on the mesh (DMTET) which does not need conversion.\n2. The authors mentioned that \" we can achieve high-quality texture with only 30 steps, significantly fewer than the 250 steps required by the original resampling method for a single view.\" Other methods like texture and text2tex only sampled no more than 50 steps for each view. Where is this \"250\" from.\n3. What is the meaning of resampling steps? Does it mean you have to sample R steps for each view at each timestep?\n4. The authors used \"resampled images near the first viewpoint to extract images of the same object from different angles in the output domain of the diffusion model\" . How to make sure that the viewpoints near the first viewpoint maintain the same style as the first view.\n5. In the loss function of Eqn. 10, the target is constraining the new noise estimation to be the same as the original noise estimation. The what is the meaning of training? The optimal case is keeping the original model unchanged.\n6. Any 3D results of the method? I prefer to see the rendered 360-degree videos of results.\n7. The attention feature injection as in [Text-Guided Texturing by Synchronized Multi-View Diffusion] can help to reduce the problem of the autoregressive inpainting. Have you tried this?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Proposed a better approach for viewpoint sampling.\n2. The performance of the proposed method is better than listed SOTAs.", "weaknesses": "1. The resampling strategy is similar to the [TexFusion:Synthesizing 3D Textures with Text-Guided Image Diffusion Models] and [TexGen: Text-Guided 3D Texture Generation with Multi-view Sampling and Resampling]. Please explain the difference.\n2. In Fig. 1b, it seems that the proposed method is over-smoothed. Please explain the reason.\n3. Please answer the following questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)