UV-free Texture Generation with Denoising and Geodesic Heat Diffusion
Authors: Simone Foti
Stefanos Zafeiriou
Tolga Birdal
Venue: NeurIPS 2024 poster (2024)
Keywords: Texture Generation
Heat diffusion
DDPM
Primary area: machine_vision

Abstract: Seams, distortions, wasted UV space, vertex-duplication, and varying resolution over the surface are the most prominent issues of the standard UV-based texturing of meshes. These issues are particularly acute when automatic UV-unwrapping techniques are used. For this reason, instead of generating textures in automatically generated UV-planes like most state-of-the-art methods, we propose to represent textures as coloured point-clouds whose colours are generated by a denoising diffusion probabilistic model constrained to operate on the surface of 3D objects. Our sampling and resolution agnostic generative model heavily relies on heat diffusion over the surface of the meshes for spatial communication between points. To enable processing of arbitrarily sampled point-cloud textures and ensure long-distance texture consistency we introduce a fast re-sampling of the mesh spectral properties used during the heat diffusion and introduce a novel heat-diffusion-based self-attention mechanism. Our code and pre-trained models are available at github.com/simofoti/UV3-TeD.

UV-free Texture Generation with Denoising and Geodesic Heat Diffusions
Simone Foti
Stefanos Zafeiriou Department of Computing Imperial College London
Tolga Birdal
Figure 1: Random textures generated by our method, Uv3-TeD, on the surface of general objects from the Amazon Berkeley Object dataset and of chairs from ShapeNet (miniatures on the shelves).
Abstract
Seams, distortions, wasted UVspace, vertex-duplication, and varying resolution over the surface are the most prominent issues of the standard UV-based texturing of meshes. These issues are particularly acute when automatic UV-unwrapping techniques are used. For this reason, instead of generating textures automatically generated UV UV-planes like most state-of-the-an methods, propose to represent textures as coloured point-clouds whose colours are generated by a denoising diffusion probabilistic model constrained operate on the surface of 3D objects. Our sampling and resolution agnostic generative model heavily relies on heat diffusion over the surface of the meshes for spatial communication between points. To enable processing of arbitrarily sampled point-cloud textures and ensure long- distance texture consistency we introduce fast re-sampling of the mesh spectral properties used during the heat diffusion and introduce a novel heat-diffusson- based self-attention mechanism. Our code and pre-trained models available gituuk.om//inofaa//U/31aal
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Introduction
Meshes are surface discretisations intentionally designed represent the geometry of 3D objects. Since real objects are more than just their geometry, meshes are frequently augmented with textures, representing appearances. Currently, textures mostly represented images that can be wrapped on the mesh via a Um-mapping procedure that maps every point on the surface of the mesh into point on the UV-image-plane where texture information are stored. Textures are therefore mostly generated on images, but considering that UV-maps intrinsically suffer from distortions, seams, wasted UV-space, certex-duplication, and varying resolution [9]. is UV- UV- -ampping really the best approach? In this work, we wonder: what if instead of generating texture on a plane and then wrapping it onto a shape we could directly generate a texture on the curved surface of the object?
The capability of generating textures by avoiding UV UV-unirapingg and mapping altogether alleviates the post-processing issues caused by the UV-mapping and can save time and resources for many seams, 3D artists, while generally improving the realism of the textures by avoiding distortions,
or stretching artefacts. In addition, it could enable the creation of priors for a variety of computer vision tasks ranging from shape and appearance reconstruction to object detection, identification, and tracking. More generally, a texture representation adhering to the actual geometry of the surface could result in smaller memory footprints without compromising on the rendering quality.
While most oftte-rf- methods focus generating textures in UV-space [23] 7 62 55
13 34
12 42 41] and thus inherit all the brawbacks of UV-mapping (Fig. 2 left), we propose to represent textures with unstructured point-clouds sampled on the surface of object and devise technique to render them without going through UV-mapping (Fig. 2 right). We generate point-cloud textures with denoising diffusion probabilistic model operating exclusively on the surface of the meshes. This is fundamentally different from generating coloured point-clouds (58] or triplane-basee implicit textures [8] while our work respects the geodesic information provided by the meshes, they operate in the Euclidean space and ignore the topology of the objects they seek to texturise. When compared to methods like (49), which can generate textures directly on the surface, our method has the advantage of not requiring any remeshing operation and being adaptable to different sampling resolutions. In addition, unlike many other texture generation methods 77 12 42 49). we generate albedo textures that by not factoring in the environment can be rendered with different lighting conditions to achieve more photorealistic results (Fig. D. Finally, while most methods are class-specific, our method can be trained on datasets containing objects of different classes (Fig. D Our key contributions are:
1. We create denoising diffusion probabilistic model generating point-cloud textures by operating only on the surface of the meshes,
2. We introduce novel attention layer based on heat diffusion and farthest point sampling to improve the recently proposed DiffusionNet blocks [88 by facilitating global communication across the entire surface of the object.
3. We propose a mixed Laplacian operator to ensure that heat diffusion can spread even in the presence of topological errors and disconnected components while still mostly relying on the provided topological information,
We devise an online sampling strategy that allows us to sample point-clouds and their spectral properties during training without requiring to recompute them from scratch.
2 Related Work
Texture Representations and Rendering, Textures are traditionally represented by images which mapped onto 3D shapes via UV-mapping. Since manual UV-mapping is complex and labour- intensive, many methods tried to perform it automatically while attempting to address some of the most common artefacts: seams and distortion [44] 39 Other texture representations such as 50) 61 have been proposed without finding level of adoption comparable to the one of UV-txxtures which are still the de facto standard for modelling the appearance of meshes. Although arguably the most convenient shape representation for computer graphics applications, meshes are not the only data structure used to represent 3D shapes. In fact, point-clouds are equally spread and they can also be associated with appearance information, stored as colour values associated to each point. Many techniques have tried to reduce their sparsity while rendering [46] 45 10], but when photo-realism is required, textured meshes are still a superior representation which can better approximate continuous surfaces. Given the strengths and limitations of both representations, we propose to adopt hybrid approach where the geometry is represented by mesh and its appearance

---

Figure 2: Qualitative comparison between point-cloud-texture (top-right halves) and automatically wrapped UV -tettures (bottom-left halves). All textures were generated by Point-UV Diffusion in order to showcase some of the most common issues of UV-mppingg. Although the method generated good quality textures point-clouds, projecting them in UV-space introduces significant artefacts.
by an unstructured point-cloud texture. This can prevent seams, distortions, unused UV-space, and varying resolution while still enabling the rendering of continuous surfaces. Hybrid approaches mixing meshes and point-clouds have also been proposed by [18] and 660 59]. However, in [8]] the appearance was present in both representations, which were both rendered, and 66 59 used highly structured point-clouds with points regularly positioned the mesh faces. The representation of [60] can be used only for high-resolution textures requiring significantly more points than the number of vertices, thus making it incompatible with current geometric deep learning models. For this reason, we use unstructured point-cloud textures arbitrarily sampled at any required resolution.
Texture Generation. Many texture generation techniques have been proposed, each relying on a different representation. Considering the wide adoption of UV-textures, and the maturity of deep-learning techniques operating on the image domain, it is not surprising that most methods
still rely on UV-mapping and generate UV-images to wrap on meshes [23] 62 56, [3]. or simultaneously optimise meshes and texture to achieve the desired result [4.. Alternatively, another
common approach consists in using a generative model to generate depth-conditioned images from multiple viewpoints. These images are then projected onto a mesh, refined, and stored as UV-
textures 12 42 41). Other methods try to map textures on UV-spheres [14, tri-planes [22] 53)
NeRFs (2], or implicit functions [35), but they either fail to operate on the real geometry of the object or they end up-projecting the textures
UVpllane Even Point-UV Diffusion (58).
whose coarse stage is conceptually similar to ours because generates coloured point-clouds using point-cloud operators, effectively projects points in Utttextures that are refined with image diffusion models. Unfortunately, especially at the coarse stage, this often results in the visible artefacts that are typical of this parametrisation (Fig. 2. A method capable of operating directly the surface of the objects is Texturify [4), which shows remarkable results adopting a shape- conditioned BANN convolving coloured quad-faces. The main limitation of this method is its need uniformly re-mesh the input shapes to a fixed resolution, potentially affecting the quality of the original mesh. Most of these methods trained class-specific shapes suggesting their difficulties in dealing with multi- class datasets. While our method can actually operate on datasets with shapes belonging to different classes, some methods exacerbate the single-class limitation, focusing exclusively on training their models on single shapes to then generate texture variations [54] Similarly, manifold diffusion fields [20] are capable of generating continuous functions -such as textures- over Riemannian manifolds. Unfortunately, they generate functions only on manifold meshes and they are always trained on single-shar shape datasets with multiple function variations. It is unclear whether their method would be capable of generalising to different geometries.
3 Notation and Background
We define a mesh as M F), with V E V33 representing the positions of the V vertices sampled on the surface (8) of a shape, and F FFx' the set of triangular faces describing the connectivity of the vertices. Throughout this work, we assume that the vertex positions and the mesh topology are given, but different for every shape we want to texturise. We think of textuers as continuous functions S + X' mapping points on S to a signal space X' that, without any loss of generality, corresponds to the albedo colours, i.e.,
R3 practice, we operate on textures defined as coloured point-clouds with colours X = (xP) E Px3, where P E R Px3 represents the P 3D coordinates of the points sampled on S. Note that, in general, we assume * (and P * V) as the texture point-clouds do not necessarily need to follow the same vertex distribution.
3

---

Before introducing the main contributions of our work in Sec. 4 we formalise two pillars on which we base our method: the denoising diffusion models and the Laplace Beltrami operator.
Denoinsing Difflusion Probabilistic Model. Denoising Diffusion Probabilistic Models (26] (DDPMs) are now a well-established class of generative models that rapidly found adoption across different fields II 3II. They are parameterised by a Markov chain trained using variational inference and are essentially characterised by three steps: a forward noising procedure, a backward denoising, and a sampling procedure that is used during inference. During the forward process, training sample X, ~ p(Xn) corresponding to a point-cloud texture and coming from the original tex- tures distribution at timestep 0 is iteratively perturbed (X, x,)l by progressively adding small amount of isotropic Gaussian noise 60 ~ (0, I). Being p(,,(x,-,) 1 (Nx;; vT X,,,, 8,T) single step in the discrete forward chain with noise schedule 3,, we represent the full chain as p(Xr Xx)) I P(X, (x,)) Similarly, a generic step in the chain can be obtained as pX, X1) = XX, Va, (1 σ+) I), where I is the identity matrix, Cr = =IT== Otg, and 0z = 1- Br. This implies that X = vā,Xo t -I d.cy- In DDPM models like ours, the reverse process is parameterised by neural network trained predict the noise that needs to be progressively removed.
This process is formulated as po - x,) == N/ (x. yoi (X, 1-ay
<T-a, co (X,, t)), (.)). where
the variance is empirically fixed and the mean is leveraging the noise prediction network eg (X,, t).
The variational inference objective can thus be simplified to C = Ex., [llc - co (X,, t)]]3] Finally, the sampling process follows the reverse process where the trained network transforms noise samples coming from the terminal distribution Xx ~ p(Xxr) into the denoised X,, po (Xn) = p(X.).
Eigenproperties of Laplace Beltrami Operator. The Laplace Beltrami operator (LBO) plays an essential role in geometry processing. For triangle meshes, the LBO usually based on cotangent formulation [38 63) derived from finite element analysis. The cotangent Laplacian L E R RVVV is a sparse matrix with elements proportional to the cotangent of the angles subtended by the edges, and it is associated to a diagonal mass matrix M E TR VXV whose diagonal elements are proportional to the total area of the faces surrounding each vertex 47 The eigendecomposition of the LBO,
L$ = AM$, determines set of orthonormal eigenvectors T == [ R RV corresponding to the K smallest eigenvalues A= == [ RRK of the weak Laplacian and its mass matrix. These eigenvalues and eigenvectors have been intensively studied in spectral geometry because not only can they be used as global and local shape descriptors, but they can also be used to formulate surface operations such as heat diffusion (63. As the name suggests, heat diffusion regulates the physical heat dispersion. This phenomenon can be modelled on any discrete surface representation with Laplacian operator and it is resolution, sampling, and representation independent.
4 UV-free Texture Diffusion (UV3-TeD)
We now describe UV3-TeD, our generative model for learning point-cloud textures built upon heat- diffusion- based operators specifically designed to operate on the surface of the input shapes (detailed in Sec. 1.1 and depicted in Fig. 4.. Our diffusion model UV3-TeD operates on noised version of
the colours, X,, and predicts a denoised X, through U-Net 43 shaped architecture (Fig. 9. backed by novel incention-enhnnce heat diffusion blocks (Sec. 4.1 We represent every mesh, M. by novel mixed LBO, informed both by the geometry and the topology of MM(Se.. 4.2) We further introduce an online sampling, in order to obtain a point-cloud P with corresponding albedo colours x and tailored spectral operators (Sec. 4.3). UV3-TeD is trained with denoising objective described in Sec. 3 using heterogeneous batching. Meshes with a point-cloud texture can finally be rendered by the neirest-n interpolation detail in Sec. 4.4
4.1 Attention-enhanced Heat Diffusion Blocks
Diffusion Blocks (DB). Our blocks are inspired by DiffusionNet [48), consisting of three separate learnable parts: heat diffusion, spatial gradient features, and a vertex-wise multi-layer perceptron (MLP). The heat diffusion process is used to disperse and aggregate information on surface and has closed-form solution leveraging the spectral properties of the LBO. Since we aim to operate on point-cloud textures while leveraging topological and geometric information provided by the mesh we use our tailored versions of M and & later described in Sec. 4.3 and named Φ, and Mp

---

PRECOMPUTE
ONLINE SAMPLING
(V, F}
(P, X}
(P)
AAA
X
X
UVD
SENAS
Figure 3: Framework of UV3-TeD. Given mesh M (V. we precompute the proposed mixed Laplacian (L and its eigendecomposition (A and 4). During the online sampling we compute coloured point-cloud (P.X) } alongside its spectral quantities and other information used by our network (Fig. 99. In particular, eigenvalues A, sampled eigenvectors Pp. and approximate mass Mp are used compute the heat diffusion operations (Eq. (I): the farthest point samples (Ps(P) are used in the proposed diffused farthest-sampler attention layers (Fig. 4. and the scale invariant heat kernel signatures sihks and alope-adjustedd eigenvalues A' are used as shape conditioning. UV3-TeD leverages these information to generate coloured point-clouds (Xp) from noise (Xr).
respectively. Being Y, E PxP a generic field defined on P. the heat diffusion layer is defined as:
A1h
diffuse( Yp, h) == P
Axch
where @ is the Hadamard product, T is the transpose operator, and h is the channel-specific learnable parameter indicating the heat diffusion time and effectively regulating the spatial support of the operator, which can range from local (h - 0) to global (h -> +xx). Since this block supports only radially symmetric filters about each point, it is combined with a spatial gradients features block that expands the space of possible filters by computing the inner product between pairs of feature gradients undergoing a learnable per-vertex rotation and scaling transformation in the tangent bundle (see [48) for more details). The spatial gradients are computed online on the sampled point-cloud with our faster implementation (see App. Then, the input is concatenated with the output of these two blocks and passed to per-vertex MLP (see Fig. 4 bottom).
We further add time embedding representing the denoising step of the DDPM and introduce group normalisation [55] to stabilise training after the time injection. We also add linear layer on the skip connection when input and output channels differ., e.g., when skip connections from the downstream branch of the U-Net are concatenated with the features of the upstream branch.
Enhancing DB via Farthest-Sampled Attention. As depicted in Fig. each of our Attention- enhanced Heat Diffusion Blocks concatenate three Diffusion Blocks and combine them with diffused farthest-sampled attention layer. Even though with the Diffusion Blocks alone it is theo- retically possible to achieve global support when h +00, the longer the heat diffusion time, the closer the diffused features become to the average of the input over the domain. This can result in less meaningful features causing texture inconsistencies between distant regions. To improve long-range consistency we introduce attention layers in each network's block alongside the other operators. Since directly performing the scaled dot product operation characterising attention modules on full-resolution point-cloud textures would be prohibitive, we build upon the heat diffusion concept and define more efficient attention operator (Fig. top).
We start by heat-diffusing the Y!(-1) features predicted by the previous layers over P to spread information across the surface geodesically. Then, we collect the spread information (i.e., diffuse(Y,$*-) h)) on a subset of the diffused features, S E RSC which is obtained by se- lecting the diffused features with C' channels corresponding to the S farthest samples [4] of P. S is then fed to multi-headed self-attention layer [51], where a set of linear layers first computes queries, keys, and values for each head, then computes scaled dot-product attention, concatenates the results across the different heads and, after going through an additional linear layer, produces new set of features over the farthest samples. These features still reside the farthest samples. To spread them
5

---

HEAT DIFFUSION
HEAT DIFFUSION
FARTHEST PARTHEST-SAMPLES SAMPLED MULTI-HEADED SELF-ATTENTION
Y
HEAT DIFFUSION
SPATIAL GRADIENTS
cat
Figure 4: Attention-enhanced Heat Diffusion block. Three consecutive Diffusion blocks (bottom) inspired by [48] and conditioned with a denoising time embedding are combined with a diffused farthest-sampled attention layer (top). The proposed attention, conditioned with local and global shape embeddings (sihks, and A²), first spreads information to all the points on the surface, before computing multi-headed self-attention on the features of the farthest samples (red points), and finally spreads them back to all the points with another heat diffusion.
across the entire surface, set the features of the other points to zero and perform another heat diffusion. The output the diffused farthest-sampler attention is re-combined with the output of the other blocks learning per-channel weighting constant.
Conditioning, While other point-cloud networks require additional inputs to represent the positions of the input points alongside their features, we just provide noised colours inputs because the diffusion process intrinsically operates on the surface of the shapes we want to texturise. Nevertheless, we do provide geometric and positional conditioning to the diffused farthest- sameled attention layers, which are otherwise unaware of the relative position of their inputs. Instead of using P to compute positional conditioning directly, we rely on the scale-invariant heat kernel signatures (sihks) [6] intrinsic local shape descriptors that are not only sampling and format agnostic but also isometry and scale-invariant. The geometry conditioning obtained from the eigenvalues A, which, like 24 are normalised by Area(M) and deprived of their slope as:
A
A' A'=
=
1/4 X Area(M)
4T * *h, for k
K
Eigenvalues processed in Eq. (2) can still be used global shape descriptors that besides having the advantage of being scale invariant also fluctuate over a straight line, becoming easier to process for neural network. Intuitively, sihks tell us the intrinsic coordinates of a point, while A' whether we are supposed to generate a texture on a chair, sofa, vase, or something else. Both are embedded with MLP and the resulting geometry embeddings are concatenated with the point features.
4.2 Mixed Robust Laplacian
To operate on real-works datasets we propose mixed Laplacian operator which is robust to any triangle mesh and can better diffuse heat in the presence of complex topological structures (see Fig. 5 left). Our mixed robust LBO (L RL is defined as:
L
R mix (1 L eL,
with 2 R.
Instead of using the cotan-LBO directly, we use the robust mesh Laplacian L 47 computed on the vertices of the mesh, as it provides robustness to non-orientable and non-manifold meshes. L ensures that heat is geodesically diffused, while L its point-cloud counterpart, enables communication between distinct or disconnected components of mesh. A small value leads to diffusing heat on the surface while allowing for some heat transmission to neighbouring regions (see Fig. right).
4.3 Online Sampling of Points, Colours, and Spectral Operators
Our sampling strategy is at the core of our method as it provides an efficient sampling strategy that can be used online without hindering training speeds. particular, Poisson Disk Sampling produces point-cloud with eegulally-saace points, enabling to approximate the mass matrix quickly.

---

MESH LBO
MIXED LBO
h
Figure 5: Heat diffusion Ted sliced the belly and topologically disconnected birdhouse. Using the mesh LBO prevents heat from spreading to disconnected regions, this is particularly visible on Ted as heat does not spread over the nose, mouth, and legs. Similarly, on the birdhouse heat spreads only on the right-hand side of the roof. Using our mixed LBQ formulation heat can spread over the entire shape even in the presence of topological errors and disconnected components.
To avoid recomputing the eigendecomposition of our Laplacian operator (L the sampled point-cloud, we recycle the spectral operators precomputed on the vertices the meshes. Finally, we describe how colours sampled during training.
Poisson Disk Sampling (PSD) (5] PDS produces a point-cloud P E Rxx3, by uniformly sampling points on the surface of mesh. This is achieved with a parallel dart-throwing algorithm that uses uniform radius r across the surface. The samples p; E P are randomly distributed on the surface but remain a minimum distance radius rather than desired number of points P", the radius can be estimated from the ideal quality measure expected from the radius statistics introduced in [5] (see also 4
away from each other. Since PDS designed operate given
P r 2V3P
RJ 0.7
(4)
Area(M))
Mass Matrix. Since the point-cloud textures have been sampled using PDS, we hypothesise that the distance between neighbouring points will equal the radius used by PDS. A triangulation of
such points would result in equilateral faces with area Arca (F iyte) 2 sin( 3 r'33 Therefore, said Q the number faces incident to each vertex p; and computing radius with Eq. we can approximate the mass matrix as:
Q 3
(0.7)²Q 6p*
M = Σ (Fea(F
R 2
Area(M), Vi.
(5)
Since never explicitly compute triangulation of the point-cloud texture, estimate Q on M. Also, considering that the mass matrix derived in Eq. ( has the same value on the diagonal elements, represent it with scalar, Mp.
Eigenvalues and Eigenvectors. The eigenvalues of LBO are considered global shape descriptors, as such, they are
indiiingiindeeddent.. Eigenvectors are on the other hand defined on the vertices of the mesh on which LBO was computed. However, as mentioned in Sec. 3 a mesh M is effectively discretising a continuous surface S. Similarly, signal on the vertices of the mesh can be thought of as discretisation of the continuous function defined on S. For this reason, eigenvectors can be resampled by interpolating the values of the eigenvectors defined the vertices of the mesh. We indicate these with Φ I PxK
Colour Sampling. Although we advocate for a new texture representation based on point-clouds, the most widely adopted representation is still based on UV UV-mapping. Hence, for every point sampled with PDS, we also query the colour stored the UVimage plane at its corresponding UV coooddnnasss.. When U Utextures are not provided, we sample the base colour instead. Following this procedure, we obtain coloured point-clouds {P. X} that we use as point-cloud textures during training.
When images have significantly higher resolution than the desired point-cloud texture resolution, we resize the image texture before sampling. We assume that properly textured meshes should intentionally have big UVtriangles where high texture resolution is required. Being A the N biggest triangles UVspace and A 3D the corresponding triangles on the mesh to estimate the scaling factor (s) needed to obtain the desired image texture size, compute the square root of the
7

---

ratio between the number of samples on A and the number of pixels in A
3D
E Area( (0000////////a )/Arca(F sye))
(6)
(W H) DN Areal ( (1)))
The number of samples in A 3D is estimated dividing their average area by the approximate area of the PD sampled point-cloud texture. The number of pixels in A n" by estimating the fraction of space occupied by the biggest triangles and multiplying it by the number of pixels in the image plane, which is computed the product between image width and height (W H). In practice, we set N = 250 to consider a significant number of triangles and use 3s instead of & to account for non-perfectly textured meshes which retain useful high-resolution content small UV Vtriangles.
OOMM ON
ZOOM
4.4 Rendering Point-Cloud Textures
RAY MESH II UTESSECIIION
We rely on Mitsuba3 (77. physically-based differentiable renderer, and implement a new class of textures: the point- cloud textures that we previously characterised with the {P, }} pair. When a ray intersection occurs and the point- cloud texture is queried, we compute the three nearest point- cloud neighbours to the point and interpolate their colour values. This is analogous to the standard texture querying that would occur in S'space. Note that the nearest neigh- bours are computed using Euclidean rather than geodesic distances. When enough points are sampled, this is a rea- sonable assumption that keeps rendering times low.
RAY
Figure 6: Rendering a point-cloud tex- tured cow [16]. When ray intersects the mesh, we interpolate the colours of the three nearest texture points.
Experiments
Datasets. We conduct experiments on two datasets, the chairs of ShapeNet 9 and the Amazon Berkeley Objects (ABO) dataset [15). The chair category of ShapeNet has often been used for texture generation on 3D shapes because, compared to the other categories. it has a high number of samples with relatively high texture resolutions. This motivated to train UV3-TeD on these data. However, driven by the objective of building a model capable of operating across multiple categories, we also decided to leverage the less widely used ABO. Despite its more limited adoption, this dataset contains multiple object categories with good-quality meshes and textures.
Since with UV3-TeD mesh and texture resolution independent, pre-filter data with more than 60, 000 vertices. This choice doesn't hinder the quality of the generated point-cloud textures but reduces the GPU memory consumption during training. As are interested in generating textures, we also discard meshes that have coloured parts, but no textures. both cases, we operate a 90 5 5 split between train, test, and validation sets. The filtering and data split leave with 4, 633 chairs for training, 266 for validation, and 317 for testing. On ABO we have 6, 476 shapes for training, 364 for validation, and 443 for testing.
Implementation Details. We implement our method using PyTorch 37). Pytorch Geometric 21 and
Diffusers [52]. We use 32 sihks, K 128 eigenvalues, and mixed-LBO weighting of 0.05. We train our models using the Adam W (30 optimiser for 400 epochs on chairs and 250 on ABO, with learning rate of 1e 4 and a cosine annealing with 500 warmup iteration steps. We use T 1, 000 DDPM timesteps. S = 250 farthest point samples in the attention layers, and P = 5, 000 target PDS samples. Since p* is used to estimate the mesh-specific PDS radius r. our point-cloud textures often have a slightly different amount of points P. Thus, we made all our layers suitable for heterogeneous batching. Our batch size is set to 8 on ShapeNet chairs and on ABO.
Unconditional Texture Generation. In Fig. 7 top we showcase our texture generation results on chairs from ShapeNet, in Fig. g we depict results on objects from ABO, and in Fig. I we combine textured object from both datasets rendered with more advanced lighting (more textured samples provided in App. A.2). Then, we proceed to compare our method against the state-of-the-art
Although we acknowledge that many state-of-the-ar methods operating in UV-space have generated impressive results, we want to highlight that they operate on images, a data type which has been vastly explored by the Deep Learning community and where models are well-engineered, mature in
8

---

UVV
Figure 7:
Qualitative comparison between PointUVDiff (pel-texture) and UV3-TeD (ours). Our textures are more diverse and more semantically meaningful. All shapes belonged to the test set.
terms of efficiency and quality, and trained on larger datasets. We find it important to note that we do not aim to compete against UV techniques, but rather attempt to prompt a paradigm shift towards direction that will not require UV UV-mapping. with its many unsolved issues. As detailed in Sec. 2 we consider the coarse stage of Point-UV Diffusion [58] and Texturify [49] to be the best non-UV-basee method currently available as well as the most relevant works to ours. The former already extensively proved its superiority over Texturify and [35) on the chairs of ShapeNet. On the same data, Texturify made additional comparisons against UV (157) and a UV Baseline), Implicit [35], Tri-plane [8], and focus sparse grid [17] methods, outperforming all of them across all metrics. For this reason, our comparison on Point-U'V Diffusion, which does not have shadows baked in the texture and is therefore better suited to be rendered with our pipeline (Sec. (.4).. Because we adopt physically- based rendering techniques rather than rasterisation, we re-compute FID and KID scores 225
36 3 for
Point-UV Diffusion. Each shape rendered from 5 random viewpoints with the camera pointing
at the object at an azimuth angle sampled from (/ 2m] and an elevation sampled from ([0 * */a]. We also compute the LPIPS [44] metric, measuring the diversity of the generated textures. For this, we generate 3 texture variations for each shape, render them, and compute the LPIPS values for all the possible pairs of images with the same underlying shape. The results are then averaged for each method. We also evaluate two different scenarios: one in which we emulate the original formulation and render shapes whose point-cloud textures have been projected in L's-sp,e, and one where we do not project their point-clouds in UV-space using our rendering technique instead. As we can observe from the quantitative results reported in Tab. [] our method significantly outperforms Point-UP V Diffusion across all metrics. In addition, as we can observe from the samples reported in Fig. [ and Fig. 7 our method can generate more diverse samples, which are also more capable capturing the semantics of the different object parts. Interestingly, this is achieved without providing any semantic segmentation. UV3-TeD also significantly outperforms DiffusionNet DDPM in sample quality while maintaining comparable diversity (Tab. [ This baseline mimics UV3-TeD by leveraging a DDPM model with a U-Net-like architecture having as many layers as ours, but without shape conditioning and using the point operators of [48). Therefore, not only there were no farthest- attention layers, but no online sampling strategy was used and the point-cloud Laplacian and mass matrix were used instead. Colours were still sampled like for UV3-TeD. All the hyperparameters matched ours.
Table I: Quantitative comparison the chairs of ShapeNet.
Method
FID (L)
KID x10-4 (4)
LPIPS (t)
Poin Poif
58 (uv)
63.35
83.19
0.08
58
(pel-texture)
65,09
126.18
0.09
DiffusionNet DDPM
116.58
468.09
0.24
UV3-Tel (ours)
54.20
42.17
0..1
Ablation Studies. We here perform multiple ablations to examine how much each model component, conditioning, and choice contributes to the overall performance. The ablations are performed training the model for 50 epochs on the ABO (multi-class) dataset. Ablations are quantitatively evaluated on 100 test shapes using the three main metries previously used during the comparisons: the FID, KID, and LPIPS scores. Results are reported in Tab. 2 Overall, our final model (UV3-TeD) reports the best quality scores while maintaining good diversity scores. A detailed discussion of the different ablations is provided in App. A.2

---

Table 2: Ablation studies on our UV3-TeD on ABO. Models were trained for 50 epochs.
FID (4)
KID x10 (1)
LPIPS (t)
UV3-TeD (ours)
77.14
58.59
0.14
w/o farthest-sampled attention
83.16
103.42
0.10
w/o A'
79.96
66.29
0.15
w/o sihks
78,46
70.90
0.14
Φ p instead of sihks
79.62
65.17
0.14
@ 0 (L instead of L L
78.07
62.64
0.15
= (L instead of L L
78,47
64.53
0.14
w/o GT-texture resizing
83.72
97.00
0.16
Conclusion
We introduced UV3-TeD, new method for representing and generating textures on sparse unstruc- tured point-clouds constrained lie on the surface of an input mesh. Our framework is based upon denoising diffusion over surfaces, in which we introduce new farthest-sampler multi medti-headdd attention layer diffusing and capturing features over distant regions, required for coherent texture synthesis. To perform diffusion meshes of arbitrary topologies, we proposed a mixed Laplacian, fusing both geometric and topological cues. In addition, we proposed online sampling strategies for efficiently working with different quantities related to the shape spectra. Acknowledging that rendering is as equally important as the texture representation, we proposed a path-tracing renderer tailored for our point-cloud textures living on shape surfaces.
Limitations & Future Work. Existing UV-based texturing pipelines are heavily engineered, leverag- ing the recent advances in image generation. We expect that approach will similarly benefit from the advances in 3D foundation models. Learning high-frequency texture details requires significant training effort, usually exceeding thousands of epochs. More efficient architectures, utilising pooling are required to overcome the drawback and increase the resolution of the generated textures. To enhance quality even further, we recommend extending our method to support BRDFs generation and encourage additional research into sampling strategies capable of ensuring crisp texture borders re-think efficient
between parts. With UV3-TeD TeD and its promising results, we invite the community texture representations, and pave the way to seam- -fee high-quality coding of appearances on surfaces. As such, we believe our work opens up ample room for future studies texture generation and other applications requiring the generation of signals that reside on surfaces. For instance, our framework could be easily adapted to applications ranging from HDRT environment map generation, shape matching, and weather forecasting, to molecular shape analysis and generation.
We believe our approach will have a predominantly positive impact, fostering research in generating UV-free textures and ultimately improving creative processes across various industries and empowering individuals with limited artistic skills to participate in content creation. Also, we do not expect nor wish to replace artists due advancements in the field. Instead, aim to make their work more efficient, allowing them to unlock their creativity faster.
Broader Impact.
Figure Random samples generated by UV3-TeD (our method) on ABO set.
10

---

Acknowledgmeetts and Disclosure of Funding
This work was supported by the UK Engineering and Physical Sciences Research Council (EPSRC)
Project GNOMON (EP/X01 1364) for Imperial College London, Department of Computing.
References
[1] H. Baatz, J. Granskog, M. Papas, F. Rousselle, and J. Novák. Nerf-tex: Neural reflectance field textures. Computer Graphics Forum, 41(6))287-301, 2022.
[2] D. Benson and Davis. Octree textures. ACM Transactions on Graphics 213)
2002.
[3] M. Binkkwski, D. Sutherland, M. Arbel, and A. Gretton. Demystifying mmd gans. arXiv preprint 1801v01000001400 2018.
[4] T. Birdal and S. Ili.. A point sampling algorithm for 3d matching of irregular geometries, In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1871-6878 IEEE, 2017.
[5] J. Bowers, R. Wang, L.-Y. Wei, and D. Maletz. Parallel poisson disk sampling with spectrum analysis on surfaces. ACM Transactions on Graphics (TOG), 1966)1-10, 2010.
[6] M. M. Bronstein and Kokkinos. Scale-invariant heat kernel signatures for non-rigid shape recognition. In 2010 IEEE computer society conference computer vision and pattern recognition, pages 1704-1711. IEEE, 2010.
[7] T. Cao, K. Kreis, S. Fidler, N. Sharp, and K. Yin. Texfusion: Synthesizing 3d textures with text-guided image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4169-4181, 2023.
[8] E. R. Chan, C. Z. Lin, M. A. Chan, K. Nagano, B. Pan, S. De Mello, O. Gallo, L. J. Guibas, J. Tremblay, S. Khamis, et al. Efficient geometry-awar 3d generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16123-16133, 2022.
[9] A. X. Chang, Funkhouser, L. Guibas, P. Hanrahan, Q. Huang. Li, S. Savarese, M. Savva, S. Song,, H. Su, J. Xiao, L. Yi, and F. Yu. ShapeNet: An Information-Rice -icch 3D Model Repository. Technical Report
[cs.GR], Stanford University - Princeton University Toyota Technological Institute Chicago, 2015.
[10] J.-H. R. Chang, W.Y Chen, A. Ranjan, K. M. Yi, and O. Tuzel. Pointersect: Neural rendering with cloud-ray intersection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8359-8369, 2023.
[11] Z. Chang, G. A. Koulieris, and H. P. Shum. On the design fundamentals of diffusion models: survey, arXiv preprint 2.XX::22306005542,, 2023.
[12] D. Z. Chen, Y. Siddiqui, H.-Y. Lee, S. Tulyakov, and M. Nießner. Text2tex: Text-driven texture synthesis via diffusion models. ICCV, 2023.
[13] Z. Chen, K. Yin, and S. Fidler. Auv-net: Learning aligned uv maps for texture transfer and synthesis. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 1465-1474, 2022.
[14] A.-C. Cheng, X. Li, S. Liu, and X. Wang. Tuvf: Learning generalizable texture uv radiance fields. ICLR, 2024.
[15] Collins, S. Goel, K. Deng. Luthra. L. Xu, Gundogdu, X. Zhang, T. F. Yago Vicente, T. Dideríksen, H. Arora, M. Guillaumin, and J. Malik. Abo: Dataset and benchmarks for real-world 3d object understanding. CVPR, 2022.
[16] K. Crane, U. Pinkall, and P. Schröder. Robust fairing via conformal curvature flow. ACM Transactions Graphics (TOG), 32(4):1-10, 2013.

---

[17] A. Dai, Y. Siddiqui, J. Thies, J. Valentin, and M. Nießner. Spsg: Self-supervised photometric scene generation from rgb-d scans. Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 1747-1756, 2021.
[18] A. Devaux and M. Brédif. Realtime projective multi-textuuing of pointclouds and meshes for realistic street-view web navigation. In Proceedings of the 21 International Conference on Web3D Technology, pages 105-108, 2016.
[19] D. Dolonius, E. Sintorn, and U. Assarsson. Uv-free texturing using sparse voxel dags. Computer Graphics Forum, 39(2):121-132 2020.
[20] A. A. Elhag, J. M. Susskind, and M. A. Bautista. Manifold diffusion fields. In ICLR, 2024.
[21] M. Fey and E. Lenssen. Fast graph representation learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019.
[22] J. Gao, T. Shen, Z. Wang, W. Chen, K. Yin, D. Li, O. Litany, Z. Gojeic, and S. Fidler. Get3d: A generative model of high quality 3d textured shapes learned from images. Advances In Neural Information Processing Systems, 15:31841-333
2022.
[23] L. Gao, T. Wu, Y.-J. Yuan, M.-X. Lin, Y.-K. Lai, and H. Zhang. Tm-net: Deep generative networks for textured meshes. ACM Transactions on Graphics (TOG), 40(6);1-15, 2021.
[24] Z. Gao. Z. Yu, and X. Pang. A compact shape descriptor for triangular surface meshes. Computer-Aided Design, 53:62-69, 2014.
[25] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
[26] Ho, Jain, and P. Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6855 2020.
[27] Jakob, S. Speierer, N. Roussel, and D. Vicini. Dr. jit: just-in-time compiler for differentiable rendering. ACM Transactions on Graphics (TOG), -144::1-19, 2022.
[28] A. Lacoste, A. Luccioni, Schmidt, and Dandres. Quantifying the carbon emissions of machine learning, arXiv preprint arXiv:1910.09700, 2019.
[29] Z. Liu, P. Luo, X. Wang, and X. Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
[30] I. Loshchilov and F. Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations, 2018.
[31] C. Luo. Understanding diffusion models: A unified perspective. arXiv:2208. / 1970, 2022.
arXiv preprint
[32] G. Metzer, E. Richardson, O. Patashnik, R. Giryes, and D. Cohen-Or. Latent-nerf for shape- guided generation of 3d shapes and textures. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 12663-12673, 2023.
[33] T. W. Mitchel, C. Esteves, and A. Makadia. Single mesh diffusion models with field latents for texture generation. arXiv preprint arXiv:2312.09255 2023.
[34] N. Mohammad Khalid, T. Xie, E. Belilovsky, and T. Popa. Clip-mesh: Generating textured meshes from text using pretrained image-text models. In SIGGRAPH Asia 2022 conference papers, pages 1-8, 2022.
[35] M. Oechsle, Mescheder, M. Niemeyer, Strauss, and A. Geiger. Texture fields: Learning texture representations in function space. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 4531-4540 2019.
[36] G. Parmar, R. Zhang, and J.-Y. Zhu. On aliased resizing and surprising subtleties in gan evaluation. In CVPR, 2022.
12

---

[37] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang. Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer. Automatic differentiation in pytorch. In NIPS 2017 Workshop Autodiff Submission, 2017.
[38] U. Pinkall and K. Polthier. Computing discrete minimal surfaces and their conjugates. Experi- mental mathematics, 2(1): 1993.
[39] R. Poranne, M. Tarini, S. Huber, D. Panozzo, and O. Sorkooe---Hee Sorkine-Hornung. Autocuts: simultaneous distortion and cut optimization for uv mapping. ACM Transactions on Graphics (TOG), 36(6): 1-11, 2017.
[40] C. R. Qi, L. Yi, H. Su, and L. J. Guibas. Pointnet++: Deep hierarchical feature learning on point sets in metric space, Advances neural information processing systems, 30, 2017.
[41] A. Raj, C. Ham, C. Barnes, V. Kim, Lu, and J. Hays. Learning to generate textures on 3d meshes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 32-38, 2019.
[42] E. Richardson, G. Metzer, Y. Alaluf, R. Giryes, and D. Cohen-Or. Texture: Text-guided texturing of 3d shapes. In ACM SIGGRAPH 2023 Conference Proceedings, SIGGRAPH '23, New York, NY, USA, 2023. Association for Computing Machinery. ISBN 9798400701597. doi: URL pattps///do.. org.:...rg/11... 1145/3588432 35915885222391153333
10.1145/5884432.5591
[43] O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention-MICE 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pages 234-241. Springer, 2015.
[44] N. Schertler, D. Panozzo, S. Gumhold, and M. Tarini. Generalized motorcycle graphs for imperfect quad-dominant meshes. ACM Transactions on Graphics, 37(4), 2018.
[45] M. Schütz, K. Krösl, and M. Wimmer. Real-time continuous level of detail rendering of point clouds. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pages 103-110. IEEE, 2019.
[46] M. Schütz, B. Kerbl, and M. Wimmer. Rendering point clouds with compute shaders and vertex order optimization. Computer Graphics Forum, 40(4): 2021.
[47] N. Sharp and K. Crane. A laplacian for nonmanifold triangle meshes. In Computer Graphics Forum, volume 39, pages 69-80. Wiley Online Library, 2020.
[48] N. Sharp, S. Attaiki, K. Crane, and M. Ovsjanikov. Diffusionnet: Discretization agnostic learning on surfaces. ACM Transactions on Graphics (TOG), 41(3):1-16, 2022.
[49] Y. Siddiqui, J. Thies, F. Ma, Q. Shan, M. Nießner, and A. Dai. Texturify: Generating textures on
3d shape surfaces. In European Conference on Computer Vision, pages 72-88. Springer, 2022.
[50] M. Tarini, K. Hormann, P. Cignoni, and C. Montani. Polycube-mane maps. ACM transactions on graphics (TOG), 23(3):853-866 2004.
[51] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[52] von Platen, S. Patil, A. Lozhkov, P. Cuenca, N. Lambert, K. Rasul, M. Davaadorj, D. Nair, S. Paul, W. Berman, Y. Xu, S. Liu, and T. Wolf. Diffusers: State-of-the-ar diffusion models. https://ggt htwgggthhuu.....e com/ com/huue htt.:hhggggngggg ace/ htte//fffuuserrre 2022.
[53] Wei, H. Wang, Feng, G. Lin, and K.-H. Yap. Taps3d: Text-guided 3d textured shape generation from pseudo supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16805-16815 2023.
[54] R. Wu, R. Liu, C. Vondrick, and C. Zheng. Sin3dm: Learning a diffusion model from a single 3d textured shape. arXiv preprint arXiv:2305. 2023.
13

---

[55] Y. Wu and K. He. Group normalization. In ECCV, pages 3-19, 2018.
[56] K. Yin, J. Gao, M. Shugrina, S. Khamis, and S. Fidler. 3dstylenet: Creating 3d shapes with geometrie and texture style variations. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 12456-12465, 2021.
[57] R. Yu, Y. Dong, P. Peers, and X. Tong. Learning texture generators for 3d shape collections from internet photo sets. In British Machine Vision Conference, 2021.
[58] X. Yu, P. Dai, W. Li, L. Ma, Z. Liu, and X. Qi. Texture generation on 3d meshes with point-uv diffusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4206-4216, 2023.
[59] C. Yuksel. Mesh color textures. In Proceedings of High Performance Graphics, pages 1-11. ACM New York, NY, USA, 2017.
[60] C. Yuksel, Keyser, and D. H. House. Mesh colors. ACM Transactions on Graphics (TOG), 29(2):1-11, 2010.
[61] C. Yuksel, S. Lefebvre, and M. Tarini. Rethinking texture mapping. In Computer graphics forum, volume 38, pages 535-551. Wiley Online Library, 2019.
[62] X. Zeng. Paint3d: Paint anything 3d with lighting-less texture diffusion models. arXiv preprint arXiv:2312.33913, 2023.
[63] H. Zhang. O. Van Kaick, and R. Dyer. Spectral mesh processing. In Computer graphics forum, volume 29, pages 1865-1894. Wiley Online Library, 2010.
[64] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang. The unreasonable effectiveness of deep features as a perceptual metric. Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586-595, 2018.
14

---

A Appendix
This document supplements our paper entitled UV-free Mesh Texture Generation with Denoising and Geodesic Heat Diffusion by providing further information on our architecture and design choices, additional experiments and ablations for our mesh diffusion framework as well as qualitative results in the form of texture mesh renderings. We also provide the failure cases of UV3-TeD.
A.1
Additional Information on UV3-TeD
K
A'
sthks
M sthke
A
ENHANCED DIFFUSION BLOCK
ENHANCED DIFFUSION BLOCK
ENHANCED DIFFUSION BLOCK
ENHANCED DIFFUSION BLOCK
ENHANCED DIFFUSION BLOCK
X
E
:
EEMB
Figure 9: Architecture of the proposed UV3-TeD. The noised input colours (X2) go through U-Net- shaped network with multiple Attention-enhancee Heat Diffusion Blocks (detailed in Fig. 4). Blue arrows depict how the blocks are connected in a U-Net fashion. Each Diffusion block is conditioned on a time embedding obtained by embedding the timestep and processing it with an MLP and multiple block-specific linear layers. The attention part of the attention-enhanced blocks is conditioned on the signal obtained by processing A' and sihs with two separate MLPs. All the conditioning is depicted in pink arrows.
Architecture. As mentioned in Sec. 4 although we do not have pooling and unpooling operators, the skip-connections of our UV3 are inspired by U-Net. Therefore, the output of the first block will be fed to the last block alongside the features coming from the previous layer, the output of the second block will be fed to the penultimate block, et cetera. Given the resemblance to U-Net, we refer to the first half of the network as the downstream branch and the second half as the upstream one. The block between these two branches is simply referred to as the middle block, which is the only one just receiving a set of features from the previous block and passing its output features to the following block. In our experiments, we use a single middle block as well as 5 blocks in both the down- and up-stream branches (Fig. 2 Each block is built as attention-enhanced heat diffusion block, which was previously depicted in Fig. 4 and described in Sec. 4.1 Each attention-enhanced diffusion block receives three conditioning signals: DDPM timestep, global shape descriptor and local set of intrinsic coordinates. Each timestep is first converted into a sinusoidal embedding (26), then it goes through a MLP and block-specific linear layer and it is used to condition the DiffusionNet block part of our enhanced blocks (see Fig. 4 The global shape descriptor conditioning is obtained by processing the normalised and straightened eigenvalues A' (Eq. 2 with a MLP, while the intrinsic coordinate conditioning by passing the scale-invare invariant heat kernel signatures (sihks) through a per-point MLP. These two geometric embeddings are concatenated and passed onto every Diffused Parthest-Sample Attention layer (Sec. 4.1 Point-wise linear layers are also used as first and last layers of the whole architecture.
The noised input colours (X,) and the predicted noise e = <= L, A', sihks) have 3 colour channels each (R, G, B). The first linear layer converts the 3 channels into 256 features, while the last one does the opposite. All the MLPs in the functions and 3 layers of size 256. In the upstream branch, the linear layer in the skip connection (Fig. 4 reduces the 512 incoming features to 256. This is not necessary for the downstream blocks as they do not receive additional inputs. The farthest-sample
diffusion blocks have ReLU activation
multi-heade self-attention layers have 8 heads with 64 channels each and operate on 250 farthest samples. The time embedding module produces time embeddings of size 256 and goes through a MLP with SiLU activations and 3 layers of size [256, 1024, 1024). The MLPs for the geometric embeddings also use SiLU activations, but the one processing A' has layers of size [128, 64, 16). while the one processing the sihks has 3 layers of size [32, 64, 16).
15

---

Figure 10: Visual representation of the mean learned diffusion times for the heat diffusion operations in each block of our Network. Conditioning information were omitted for sake of clarity. Refer to Fig. foor a more detailed representation of the architecture and to Fig. for the content of each block. The orange part of each block illustrates the diffused farthest- sampled attention layer with its two heat diffusions. The light-blue parts illustrate the three DiffusionNet blocks with their diffusion operations. These mean diffusion times are referred to UV3-TeD trained on ABO and they are displayed by diffusing heat always on the same shape.
Learned Diffusion Times. As mentioned in Sec. 3 each heat diffusion operation is performed channel-wise with learned diffusion time that controls the support of the neural operator. In Fig. 10 we visualise the learned diffusion times of each heat diffusion. In particular, we average the learned time across channels and diffuse heat using the mean time from 8 farthest sampled sources. Interestingly, most Diffusion blocks have a similar support, with the exception of the last two operators, which have a slightly bigger support. In addition, the first diffusion of each diffused farthest-sampled attention layer has a support that is comparable to the Diffusion blocks, while the second diffusion learns longer diffusion times. These longer diffusion times prove the correct operation of the proposed diffused farthest-sampled attention layer (Sec. which is supposed to spread the output of the multi-headed self-attention from the farthest samples to all the neighbouring points.
Fast Gradient Computation. As briefly mentioned in Sec. 4.1 we propose a fast gradient computa- tion implementation leveraging batched tensor operations. Our efficient implementation, avoiding the usage of multiple nested loops, more suitable for online computation and does not require to precompute gradients like in the original DiffusionNet [48].
With the objective of constructing a sparse matrix that represents the spatial gradients of the mesh we start by creating a batched tensor containing information about the neighbours of each vertex and the tangent vectors of the edges connecting them. Since we operate on a point-cloud, the neighbours are estimated with a kNN search with & 30 (like in 48) ). Having fixed number of neighbours facilitates the creation of a batched tensor.
Next, we construct the column and row indices for the sparse matrix. The column indices are created by concatenating array of vertex indices with the index of their neighbours retrieved from the batched tensor previously computed. The row indices are created by repeating vertex indices times. We then calculate the values to be stored in the sparse matrix. This involves computing the least squares solution of a linear system between the batched tensor and a matrix constructed by concatenating a column vector of -I's to an identity matrix. The result is split into two parts that are stored as a sparse complex tensor representing the gradients of the mesh, where each row corresponds to vertex, each column corresponds to a neighbouring vertex, and the value at specific row and column represents the gradients of the edge connecting the two vertices.
The computational speedup with respect to the original implementation leveraging the multiple nested loops is significant: with approximately 100k points our implementation is on average 35x faster, while with approximately 25k points it is on average 38x faster.
Runtimes and Computational Resources. We run our models on a single Nvidia A100 with 40GB
of dedicated memory. The time required to generate a single point-cloud texture (P* = 5, 000) is approximately 1 minute and 30 seconds. One training epoch takes approximately 23 minutes with the ABO dataset and 10 minutes with the chairs of ShapeNet. Note that in order to keep a constant GPU utilisation and prevent data loading and processing bottlenecks during online sampling it is advisable to use multiple workers. We use 12 workers on a computer with an AMD EPYC 64-Core Processor, which has max CPU frequency of 3.5 GHz.
16

---

Experiments were conducted using private infrastructure, which has a carbon efficiency of 0.166 kgCO,ey/Wh. cumulative of 640 hours (110 GPU-days) of computation was performed on hardware of type A A100 PCIe 40GB (TDP of 250W). Total emissions are estimated to be 109.56 kgCOzeq of which 0 percent was directly offset. Estimations were conducted using the Machine- Learning Impact calculator presented in (2 while the carbon efficiency was estimated using the following electricity maps
A.2 Additional Experiments
MESH LBO
MIXED LBO
PCL LBO
th
th
MESH LBO
MIXED LBO
PCL LBO
Figure 11: Top: heat diffusion chair whose legs are disconnected from the of the structure. Diffusing heat with the mesh LBO, heat cannot spread beyond the leg where heat was applied. With the point-cloud LBO heat diffuses also across the of the structure, but it quickly travels also horizontally across the vertical bars. With our mixed LBO formulation heat correctly spread over the rest of the structure and appears better follow geodesic paths. Bottom: Heat diffusion on an object with thin structures using different LBO formulations. Using the proposed mixed LBO heat diffuses geodesically, closely mimicking the behaviour of heat diffusion with the mesh LBO. On the contrary, with the point-cloud LBO heat would be immediately transferred from the back of the hanger its front because of the Euclidean proximity of the two parts. This is an undesirable behaviour not shown by our method.
More on Heat Diffusion with the Mixed Laplacian. As detailed in Sec. we propose hybrid formulation of the Laplace Beltrami Operator that we call the Mixed LBO and which is obtained as convex combination of the mesh and point-cloud LBOs (computed on the vertices of the mesh). As we already explained, by leveraging our LBO we can diffuse heat on surfaces with topological errors and disconnected components (Fig. 5). Fig- TT (top) shows another example where the legs of a chair were not topologically connected to the rest of the structure. Unlike the mesh LBO, our mixed LBO formulation allows heat to spread over the rest of the chair. In addition, when compared to the point-cloud LBO, our formulation retains the topological information provided by the mesh and lets the heat diffuse geodesically. This is particularly evident in Fig. II (bottom), where some heat diffused starting from the back-side on the back of the object, and partially starts to diffuse towards the front. On the contrary, with the point-cloud LBO heat is equally diffused on the front and on the back of the object. Therefore, the presence of thin structures, it is clear that heat is not geodesically diffused. On the other hand, with our formulation, we avoid spreading heat on the front of the object and we closely mimic the correct behaviour of heat diffusion with the mesh LBO. small proportion of heat spreads over a screw placed close to the heat source. We consider this to be an acceptable behaviour. Intuitively, this information-shatee -harrin may carry some useful insights on how the screw may be coloured with respect to the surrounding material used to build the hanger. Yet, the amount of heat is lower than the portion of the hanger touching the screw, facilitating the distinction between different structures.
coat hanger. With the mesh LBO heat diffuses mostly
also be noted that with Mixed IBOO some
Test Online Sampling Strategies. In Sec. we introduced a set of sampling strategies to re-use much possible pre-computed operators, make possible the online sampling of point-clouds,
17

---

DUO
Figure 12: Comparison between the eigenvectors of the shape with two different sampling
densities. The cigenvectors of the original mesh are represented as colours on the surface of the mesh, while the eigenvectors of mesh obtained subdividing the faces of the original mesh are reported on a point-cloud. The point colours match those of the underlying mesh, suggesting that sampling the eigenvectors of the mesh the point locations would produce the same result.
and ensure that heat can be diffused following known surface information of the mesh. After computing and eigendecomposing our Mixed LBO, we store its eigenvectors, eigenvalues, and mass matrix. The eigenvalues remain unchanged when sampling and therefore are re-used without any modification. The mass matrix, being proportional to the distance between the sampled points, be approximated as described in Sec. [4.3 from the PDS radius. Finally, the cigenvectors are recomputed by interpolating their values the vertices of the mesh. In Fig. [2] empirically show that this interpolation operation is possible and produces the same results as recomputing the eigenvectors. In particular, after computing the eigenvectors of mesh, of the subdivided mesh, which are displayed as a coloured point-cloud. As it can be observed in Fig. 12 the values of the coloured point-cloud are in agreement with the colours on the surface of the mesh, which are obtained as a linear interpolation between the vertex colours. For this reason, it is not necessary re-compute the eigenvectors for every point-cloud that sample and can interpolate the mesh values instead.
subdivide it and compute the eigenvectors
Established that can interpolate the eigenvectors, still need to prove that diffusing heat point-cloud sampled with PDS, whose eigenvectors have been interpolated and the mass has been approximated from the PDS radius, produces the same results diffusing heat the surface of the mesh. Therefore, we compare the heat diffused -from the same source- on a mesh and on an naline-sampled point-cloud. As the same results.
observe from Fig. 13 the two diffusion processes produce
Additional Samples. More textures generated on test shapes by our method are reported in Fig. 14 and Fig. 15 top. Fig. shows more textures generated by UV3-TeD on ABO objects and rendered from multiple viewpoints. This proves how, unlike methods relying on multi-view images for texture generation, UV3-TeD generates textures directly on the surface of the objects, making them multi- view consistent by construction. We also report more chairs generated by Point-UV Diffusion (58] and rendered with our rendering method (Sec. 4.4) for fairness of comparison. Finally, Fig, reportss some failure cases of our method.
h
Figure 13: Heat diffusion computed on the vertices of mesh and with our online sampling point-cloud. The heat depicted on the surface of the mesh computed with the traditional method using Eq. T and our Mixed LBO (Sec. 4.2). The heat depicted on the point-cloud was computed using online sampled operators like described Sec. 44.3 Heat diffuses in the same way with both techniques proving the correctness of our sampling strategy.
18

---

Figure 14; Additional random samples generated by our UV3-TeD trained ABO.
19

---

UV3-TeD
Point- UV Diff
Figure 15: Additional random samples generated by our UV3-TeD (lop), and Point-UV Diffusion (boltom) trained on chairs from ShapeNet. The point-clouds generated by Point-UV Diffusion were rendered with our method for a more fair comparison with no projection artefacts. Our method generates more diverse textures and better distinguishes the different parts.
20

---

Figure 16: Multi-view consistency of ABO shapes textured with UV3-Tel)
Higher Frequency Point-cloud Textures on CelebA. The low-frequency nature of the results on ShapeNet chairs and ABO mostly stem from the datasets used. In fact, most objects in ShapeNet have plain per-part colours and can be considered mostly low-frequency textures. ABO the other hand has more detailed textures, but these textures are much higher in frequency than our sampling resolution (e.g. wood grain, rubber pours, etc.). To demonstrate the ability of complex figh-freeuency details, we have trained UV3-TeD on CelebA images [29) projected on plane deformed by a 3D ripple effect. UV3-TeD was trained for 50 epochs, with a learning rate of 5e and batch size
method to handle
Figure 17: Plane perturbed by rip- ple effect and textured with point- cloud textures generated by UV3-TeD trained on CelebA for only 50 epochs.
The farthest point samples were reduced to self-attention layers to 64. The target PDS samples S = 200 and the channels in the farthest- (P* 5, 000) as well all the other hyperparameters were the same as those previously detailed in the Implementation Details (Sec. (5) and Architecture (App.A.J) paragraphs. This experiment shows promising results (Fig. 17 considering that it capable generating diverse CelebA textures even prior to reaching full convergence.
Considering that Diffusion Blocks resemble the original operators of 48 results also prove the unproven claim of neat-diffusionn-base operators being capable of operating at high-frequencies.
believe
Ablation Studies Detailed Discussion. In Sec. we performed multiple ablations studies to examine how much each model component, conditioning. and choice contributes to overall performance. Ablations were quantitatively evaluated using three main metrics: the FID, KID, and LPIPS scores. The first two evaluate the visual quality of generated samples rendered from random viewpoints, while LPIPS evaluates the perceptual dissimilarity between different objects consistently rendered from the same viewpoint. Results were reported Tab. 2
We started by investigating the effects of the pro- posed farthest-sampled attention layers, removing the proposed attention causes the most significant drop performance across metrics, showing the positive impact of our proposed attention mechanism. The benefits provided by the attention mechanism can be observed also in Fig. [8] It is clear that this mecha- nism makes the generated textures more realistic and uniform across different parts.
Attention
No Attention
Figure 18: ABO shapes textured by UV3- TeD with and without the proposed farthest- sampled attention layer.
We then proceeded remove the normalised and straightened eigenvalue (A') conditioning as well the scale-invariant heat kernel signatures (sihks) conditioning. Both cause drop in visual quality. However, the absence of A', which holds class-related information, appears to slightly improve the diversity of the generated point-cloud textures at the expense of sample quality. We believe that this result still signals the importance of providing both conditioning signals. Also directly using the online-sampled cigenvectors o instead of sihks causes small drop in performance. Note that
21

---

ShapNet chairs
ABO
Figure 19: Failure cases of random samples generated by our UV3-TeD trained on ShapeNet chairs (top) and ABO (bottom). Most failures exhibit issues in correctly recognising the different object parts, Long-range inconsistencies, uniform yet unreasonable colours, or blotchy patterns.
the diffused farthest-sampled attention layers contain a group normalisation layer after the first heat diffusion. Since it expects a specific number of channels. when either sihks A' are removed, to make the ablation more controllable, instead of modifying the group normalisation layer we duplicate the remaining conditioning thus providing redundant information.
Although we have extensively proved the validity and the advantages of our mixed LBO operator (e.g., Fig. and Fig. TTp. compare the performance of network with the model trained to compute heat diffusions with our online sampling and using the mesh (L or point-cloud (L,) LBOs. Our method performs slightly worse in terms of LPIPS when compared using the mesh Laplacian, but it also exhibits a far greater performance improvement in terms of FID and KID. This, and the previous geometrical evaluations of the mixed LBO, make the adoption of mixed operator preferable. Similarly, we can reach the same conclusion by observing the results with the point-cloud operator, which is also not improving over the final UV3-TeD model in terms of LPIPS score. is worth noting that ABO is a well curated dataset with reduced number of topological defects. Therefore, we expect performance deteriorate even further on less curated datasets.
Finally, we evaluate the effects of our texture resizing Sec. 4.3 Although the LPIPS score improves, showing increased diversity of the generated samples, the blotchy patches depicted in some of the failure cases on Fig. 19 become present on most generated point-cloud textures. This results in worse FID and KID scores when compared our full model.
As already mentioned Sec. 5 our final model (UV3-TeD) reports the best quality scores while maintaining good diversity scores.
A Note on Photorealism. All the result images of this paper except Fig. [ were rendered with a constant emitter to better showcase the generated albedos. Nevertheless, our generated albedo textures are intrinsically relightable and can be photo- realistically rendered. More photorealistic renderings can be obtained by rendering objects with environment maps and train- ing our model generate full BRDFs. Fig. 20 shows how just using an environment map can improve the realism of one of our chairs.
Figure 20: Chair rendered with constant emitter (left) and an envi- ronment map (right).
22

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our contributions have been explicitly listed in Sec. detailed in Sec. 4 and proved in Sec. S
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: As its name suggests, the Limitations & Future Work paragraph of Sec. 6 is dedicated the limitations and the potential future directions.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sellsspeciiiationn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
23

---

Answer: [NA]
Justification: Our paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We describe in detail our contributions in Sec. 4 and App. A. while all the architectural choices and implementation details of our model are outlined in Sec. [ which is complemented by App. A.2 All the datasets used (Sec. [3] are open source. In addition, code and pre-traines trained models will be made publicly available.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
24

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: Our code and the instructions
are provided following the submission
guidelines. The code of VoiffuuDiiiisiin (58) and the datasets [15 9 used are available
from their respective project pages. In addition, code and pre-trained models will be made publicly available.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (httpss:/miis.cc))
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nips. httpppscccpbbllgggguiiceeeeppppp dooddiiii CCooeeSumisssiooniii ssiliii for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e-g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All the experimental settings are provided in Sec. Additional details available A.I
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Error bars are reported because would be too computationally expensive given our available resources.
Guidelines:
The answer NA means that the paper does not include experiments.
25

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
In the Runtimes and Computational Resources paragraph of App. A.
detail our computational resources, training times, total GPU usage throughout the project well as an estimate of the total kgCO2eq emissions caused by our work.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code Of Fthics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics httppp////eurrrpssccc/p/o///ttts
Answer: [Yes]
Justification: Our research does not involve human subjects or participants. The datasets used are all available to use and do not leverage data scraped from the internet without the artists" consent. We do not expect our research have safety, security, discrimination, surveillance, deception, harassment, human rights, bias, and fairness consequences. Further research into this topic will cause further kgCO2eq emissions produced by GPU servers or computers. We outlined such consumption, which is still limited compared to most state-of- art generative models. All the information about the datasets and models was communicated in the manuscript and in the README instructions. Upon public release, the code and the essential elements for reproducibility will be licensed with CC BY-NC-ND license.
Guidelines:
The answer NA means that the authors have reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
26

---

The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The impact of our work has been discussed in the Broader Impact paragraph of Sec.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymeen of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-us should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
27

---

Justification:
All assets are open source and have been properly cited.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperssi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The code attached to the submission includes detailed instructions for setup and training. The code will be open-sourced with CC BY license.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/codeemmodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
28

---

Answer:
[NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
29


Reviews:
[{"review_id": "IFHWY1CdWR", "reviewer": "Reviewer_Yhxo", "summary": "This paper proposes UV3-TeD, a 3D mesh texturing method without explicit UV mapping. To circumvent the challenges of using explicit UV mapping, the authors propose to represent texture as a point cloud with color features. While there were some methods that used similar representation, the authors emphasize that their representation can effectively constrain the point clouds to operate only on the mesh surface. To generate the colored point cloud in a generative manner, the authors propose a point cloud DDPM. When designing the diffusion model, the authors design a heat diffusion block augmented with the farthest sampled attention mechanism. The authors evaluate their method on 3D shape datasets such as ShapeNet-Chairs and ABO.", "questions": "- How does UV3-TeD perform when applied to meshes with highly-detailed surfaces or meshes that require high-frequency textures, e.g., 3D clothed human scans?", "limitations": "- The texture generation results seem to be more cartoonish or blurred rather than photorealistic, while it is trained with some near-real datasets, e.g., ShapeNet or ABO.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper proposes an interesting approach for texturizing 3D meshes. Although the concept of colored point clouds has been vastly explored, the proposed heat-diffusion-based DDPM seems to be effective in constraining the points on the 3D mesh surface.\n\n- The paper presents nice and effective visualizations of the contributions and effects of their proposed methods.\n\n- The idea of farthest-sampled attention seems to be valid and fresh.", "weaknesses": "- The visualized results for the generation results seem to be too low-frequency. The concept of heat diffusion may guide the generated textures to be coherent and locally smooth, but it seems to be inherently limited in representing high-frequency details, such as stripes or repeated patterns. \n\n- While the authors proposed farthest-sampled attention to enhance long-range texture smoothness, this reviewer could not find any results that show multi-view or long-range consistent generation results. It'd be more convincing if the authors could include multi-view rendered results of a single mesh, with or without farthest-sampled attention.\n\n- The authors mainly compared their method with PointUV-Diff. However, to show the benefit of using colored point clouds rather than explicit UV mapping or texture maps, it is highly recommended to show some comparison results (both quantitative & qualitative) with existing UV-mapping-based texture generation methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their constructive and interesting comments. We start by addressing two related points and then proceed to provide a detailed response to the remaining comments.\n\n> The visualized results for the generation results seem to be too low-frequency. The concept of heat diffusion may guide the generated textures to be coherent and locally smooth, but it seems to be inherently limited in representing high-frequency details, such as stripes or repeated patterns.\n    \n> How does UV3-TeD perform when applied to meshes with highly-detailed surfaces or meshes that require high-frequency textures, e.g., 3D clothed human scans?\n    \nWe indeed agree that there is a trade-off between high frequency details and local coherence. However, the impact is not as dramatic. In fact, the low-frequency nature of our results mostly stem from the datasets used. In fact, most objects in ShapeNet have plain per-part colours and can be considered mostly low-frequency textures. ABO on the other hand has more detailed textures, but these textures are much higher in frequency than our sampling resolution (e.g. wood grain, rubber pours, etc.). Note that the as discussed in the \\textit{Remarks} paragraph of Sec 3.3 in [54], the concept of heat diffusion is used for communication across points while the MLP and the gradient features enable learning high-frequency outputs. Since our Diffusion Blocks resemble the original operators of [54] we expect them to be capable of producing high-frequency results. While [54] never proved this claim, we have now conducted a texture generation experiment by training UV3-TeD on CelebA images projected on a plane deformed by a 3D ripple effect. This experiment showed promising results demonstrating that our model is capable of generating diverse CelebA textures even without reaching full convergence (see Fig. D in the attached rebuttal PDF). We have added this experiment and its detailed description in the appendix. \n        \n> While the authors proposed farthest-sampled attention to enhance long-range texture smoothness, this reviewer could not find any results that show multi-view or long-range consistent generation results. It'd be more convincing if the authors could include multi-view rendered results of a single mesh, with or without farthest-sampled attention.\n        \nWe thank the reviewer for pointing this out. In the attached PDF we have provided additional figures to support this. In particular, we would like to emphasise that unlike methods relying on multi-view images for texture generation, our method generates textures directly on the surface of the objects,  making it multi-view consistent by construction (Fig. C). The benefits provided by the attention mechanism can be observed from Fig. B2 in the PDF. It is clear that this mechanism makes the generated textures more realistic and uniform across different parts. We have added both figures in the Appendix of our revision.\n        \n> The authors mainly compared their method with PointUV-Diff. However, to show the benefit of using colored point clouds rather than explicit UV mapping or texture maps, it is highly recommended to show some comparison results (both quantitative \\& qualitative) with existing UV-mapping-based texture generation methods.\n        \nWe thank the reviewer for pointing this out. It is worth noting that Point-UV Diffusion already performed a comparison against Texture Fields and Texturify on the chairs of ShapeNet. Not only their full pipeline (see Tab. 1 of the Point-UV Diffusion paper [54]), but also their coarse stage (see Ablation metrics in Tab. 2 of [54]) outperformed the competing methods in terms of KID and FID. Although neither of these methods operate in UV space, Texturify made comparisons against UV (LTG and UV Baseline), Implicit (Texture Fields), Tri-plane (EG3D), and sparse grid (SPSG) methods, outperforming all these methods across all metrics on the chairs of ShapeNet (see Tab. 1 in Texturify paper). Our method, outperforming Point-UV Diffusion, is expected to surpass all these baselines as well. We have added this discussion to our paper. \n        \n        \n        \n> The texture generation results seem to be more cartoonish or blurred rather than photorealistic, while it is trained with some near-real datasets, e.g., ShapeNet or ABO.\n    \nThis is a good observation. Besides the quality of the textures in the datasets, our method, albeit being agnostic to the sampling strategy and resolution, is affected by these choices, especially in terms of photorealism and when it comes to generating crisp borders between parts.\n        \nMore importantly, we generate only albedo textures and render them with a constant emitter to better showcase the generated albedos. More photorealistic renderings can be obtained by rendering objects with environment maps and training our model to generate full BRDFs. Note that this is also a key difference with many texture generation methods like Texturify where the textures are not albedos but rather bake-in the contributions of the self-shadows as well as the light interactions between the BRDF properties and environment maps. We have attached a rendering (Fig. B1) showing how just using an environment map can improve the realism of one of our chairs. We thank the reviewer for bringing this up and now mention this key intrinsic advantage of our method in the revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "50cGjLTbPT", "reviewer": "Reviewer_cPWU", "summary": "This paper proposes to apply denoising diffusion generative model directly on mesh surfaces, with a focus on texture generation.\nIt achieves this goal by utilizing the DiffusionNet method which utilizes heat diffusion to enable 'convolution' and message passing within the surface.\nThis work modify and extend the original DiffusionNet mainly in the following way:\n\n**Mixing the extrinsic and intrinsic adjacency**.\nDiffusionNet uses pure intrinsic connectivity (Mesh LBO) by default, where the information only flow within the surface without considering the actual adjacency in 3D space. \nThis may introduce problems for apply DDPM on arbitrary meshes. Instead, the proposed method mixes intrinsic adjacency with extrinsic adjacency (Point cloud LBO) so that touching parts will also have mutual information flow.\n\n**Adding attention mechanism to facilitate long-range consistency**\nSince heat diffusion typically only disperse or aggregate information locally, this paper adds an attention mechanism to better model dependencies that are far apart.", "questions": "What is the DiffusionNet DDPM baseline? \nIt seems the paper does not include a description for this baseline.\n\nWhat is exactly the benefit of using diffusion on the surfaces vs in the embedded 3D space? \nIt seems the results are better, but why? \nWhy providing the manifold neibourhood information neccessarily facilitate the texture generation? \nHow do you compare by simply using a fully transformer architecture? It can directly model long range dependency quite well, without the need for LBO. The proposed method has an emphasis on incorporating attention into the DiffusionNet framework. But why not directly use the Transformer that are full of attention blocks?\n\nMaybe transformation invariance is one advantage of intrinsic message passing, but one can always enhance this by using data augmentation of random transformation. \n\nI would suggest to **show some qualitative analysis examples** that conveys intrinsic message passing indeed is vital in some cases for colorizing surface points.", "limitations": "The authors discuss the limitation of generating textures with high-frequency.", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "**Denosing Diffusion on Manifold via Heat Diffusion**.\nAlthough the two diffusion mechanism have very different nature, it is very interesting to see these two diffusion can work together to enable direct generative modeling on manifold. I believe this paradigm **has greater potentials** than just mesh texturing. For example, one can easily adopt this framework for generating spherical or toric images (for example, generate HDRI environment map) or any signal that resides on geometric structures that are more complex than regular 2D grid.\n\n**Good paper exposition**.\nThe paper presentation is clear and the paper figures explain the concepts in an easy-to-follow manner.", "weaknesses": "**No much explanations on the results / comparisons**\n\n**The neccessity of texturing the points using an intrinsic (hybrid) architecture**\n\nPlease see the questions section to see further details.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their positive and encouraging feedback. We are glad the reviewer believes our work has greater potential than just mesh texturing. We agree that our approach can easily adapt to other applications requiring the generation of signals that reside on geometric structures that are more complex than regular 2D grids. We will mention the suggested applications as potential future work. We also address the major questions below.\n\n> What is the DiffusionNet DDPM baseline? It seems the paper does not include a description for this baseline.\n    \nWe thank the reviewer for bringing this to our attention. We have now added more details on this baseline. The DiffusionNet DDPM was built as a DDPM model with a U-Net-like architecture with as many layers as ours, but without shape conditioning and using the Vanilla DiffusionNet blocks introduced in [54]. Therefore, not only there were no farthest-sampled attention layers, but no online sampling strategy was used and the point-cloud Laplacian and mass matrix were used. Colours were still sampled like for UV3-TeD. All the hyperparameters matched ours.\n        \n> What is exactly the benefit of using diffusion on the surfaces vs in the embedded 3D space? It seems the results are better, but why? Why providing the manifold neibourhood information neccessarily facilitate the texture generation? \n\nThe main benefit of using heat diffusion on the surface is enabling geodesic information sharing by construction. \nBy looking at Fig. 10 in Appendix A2 we can observe how heat diffusion performed with a point cloud Laplacian struggles to correctly aggregate information. Note that point cloud Laplacians are built by creating a new triangulation that attempts to capture the surface properties of the shape on which they are built. Therefore, when they are used in heat diffusion we can expect the heat to spread quasi-geodesically. Performing diffusion operations directly in the embedded 3D space would then further exacerbate the issues showcased in Fig. 10 and described in Appendix A2. \n        \n> How do you compare by simply using a fully transformer architecture? It can directly model long range dependency quite well, without the need for LBO. The proposed method has an emphasis on incorporating attention into the DiffusionNet framework. But why not directly use the Transformer that are full of attention blocks?\n    \nGlobal information sharing provided by transformers is indeed powerful, particularly for capturing long-range dependencies. However, for tasks like texture generation, which resemble image generation, local information sharing is equally crucial. Our approach integrates both attention mechanisms and convolution-like operations, leveraging the strengths of both. Mixing attention and convolutions is a standard approach also in many state-of-the-art diffusion models operating on images, providing a balance between capturing global context and preserving local details while keeping low computational and memory overhead. \n\nMoreover, unlike images, there is no de-facto standard to choosing a 3D representation. In a variety of cases, ignoring the 2-manifold nature of 3D objects yields inferior results. Hence, any transformer of choice should be adapted to operate in restriction to the object surface, requiring significant engineering efforts (see for instance [i]).\nOur approach provides a natural way to achieve this. Our novel Laplacian also enables us to span a range of representations from point clouds to pure mesh topology.\n         \n\n\n\n        \n > Maybe transformation invariance is one advantage of intrinsic message passing, but one can always enhance this by using data augmentation of random transformation.\n\nData augmentation is certainly a strong alternative to transformation invariance, which is indeed a desirable by-product of our intrinsic message passing. This is potentially useful in a variety of applications like texturing digital humans. Nonetheless, our primary motivation for choosing heat diffusion lies not in its invariance properties, but in its ability to effectively propagate information across the object surface.\n\n[i] Chandranet al. Shape Transformers: Topology‐Independent 3D Shape Models Using Transformers. Computer Graphics Forum 2022", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4uhfY9LwmW", "reviewer": "Reviewer_CcU9", "summary": "The paper proposed a new way to represent the texture to replace the UV-map and trained a model for texturing generation of objects that are not limited on a specific category.", "questions": "1. Any intuitive explanation of what the \"h\" is?\n2. Is it possible to extend the proposed method to text-conditioned texture generation?\n3. Can this method generate textures with diverse textures like objects in Objaverse?", "limitations": "1. It seams that the generated textures are almost in pure colour or with limited details. I am really concerned about this point. There has already been many texture generation methods that can produce diverse, detailed and photo-realistic textures.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The proposed new representation avoid per-category training and seams on UV maps.", "weaknesses": "1. Missing the experiments on the Objaverse dataset.\n2. The Figure 3 is not detailed enough to describe how the entire pipeline work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their time and efforts invested in our paper. Our detailed responses are given below.\n\n> Fig. 3 is not detailed enough to describe how the entire pipeline work.\n\nWe thank the reviewer for suggesting us to include the full pipeline figure, which we provided in the supplementary attached PDF (Fig. A). We have also included this in our revision, nicely complementing Fig. 8 in section A.1 depicting our architecture and Fig. 3 showing its core building block. We hope that this eases the understanding of our method. \n\n> It seams that the generated textures are almost in pure colour or with limited details. I am really concerned about this point. There has already been many texture generation methods that can produce diverse, detailed and photo-realistic textures.\n        \nAlthough we acknowledge that many state of the art methods operating in UV-space (such as those mentioned in Sec. 1, line 31) have generated impressive results, we want to highlight that they operate on images, a data type which has been vastly explored by the Deep Learning community and where models are well-engineered, mature in terms of efficiency and quality, and trained on larger datasets.\nWe find it important to note that we do not aim to compete against UV techniques, but rather attempt to prompt a paradigm shift towards a direction that will not require UV-mapping, with its many unsolved issues. \nThe best non-UV-based method currently available is arguably the coarse stage of Point-UV diffusion. As can be observed in Fig. 6 and Fig. 14, our method generates more diverse and realistic textures that better respect the semantics of different objects' parts. In addition, it is worth noting that most objects in ShapeNet and ABO do have limited texture details at a resolution that could be captured by current non-UV models like ours. For these reasons, we consider our results to be an important, promising step towards the generation of UV-free textures and we are confident that improving the efficiency of our architecture with the use of pooling (line 343) will enable the generation of high-resolution point cloud textures. \n        \nAlso, unlike many other texture generation methods, we generate textures that do not factor in the environment. Our renderings were performed with a constant emitter to better showcase the generated albedo textures. Nevertheless, photorealism can be significantly improved rendering the textured objects with environment maps (see Fig. B1 in the attached PDF).\n        \nFinally, to demonstrate the ability of our method to handle more complex high-frequency details, we have trained UV3-TeD on CelebA images projected on a plane deformed by a ripple effect. This experiment showed promising results considering that it is capable of generating diverse CelebA textures even without reaching full convergence (see Fig. D in the attached PDF). We have added this experiment and its detailed description in the appendix.\n\n> Any intuitive explanation of what the \"h\" is?\n\nAs detailed in lines 162-164: \"$h$ is the channel-specific learnable parameter indicating the heat diffusion time and effectively regulating the spatial support of the operator, which can range from local ($h \\rightarrow 0$) to global ($h \\rightarrow +\\infty$).\" In other words, it is the heat diffusion time.\n        \n> Is it possible to extend the proposed method to text-conditioned texture generation?\n\nAlthough this is not something we have experimented with, we believe it is possible to extend UV3-TeD to support text conditioning. \nWe believe there are two potential approaches to providing text conditioning, each with its challenges. The first approach would require re-training the model using existing 3D datasets with text descriptions and concatenating CLIP embeddings of texture descriptions to the shape conditioning vectors. Since the captions in these datasets usually combine shape and texture information, the main challenge is the potential need to remove global shape information from the ground truth textual descriptions to prevent any clash with our global shape descriptors, which proved to be quite effective (Tab.2). The second and more promising approach would be to use UV3-TeD as a prior and run an optimisation according to an auxiliary differentiable constraint like in [a]. The constraint could be implemented with a CLIP loss between the desired text prompt and the rendered image of the textured object. The mild challenge here is in making our rendering pipeline fully differentiable. \n        \n> Can this method generate textures with diverse textures like objects in Objaverse?\n\nWe believe it is possible, but it is not trivial. Only a subset of data would be suitable for training. Data with no texture, low quality scans, and scenes with multiple objects (e.g., entire houses with all the furniture inside) need to be removed. Successfully automating this process is essential yet extremely complex. More importantly, the current training procedure is prohibitively slow given our computational resources. Training on approximately 200k shapes (corresponding to 1/4 of Objaverse) takes approximately 12h per epoch. As mentioned in the limitations and future works section, we believe that ''more efficient architectures, utilising pooling are required to overcome the drawback\" (line 342-343). Given our current computational limitations, we have conducted additional experiments aimed at showcasing the model's ability to work with higher frequencies on CelebA (see above).\n\n\n\n[a] Graikos et al. Diffusion models as plug-and-play priors. NeurIPS2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely thank all the reviewers for their constructive feedback. We appreciate that reviewers find our work interesting, valid, and fresh (Yhxo), with greater potential than just mesh texturing (cPWU), clearly presented (Yhxo, cPWU), and with nice and effective visualisations (Yhxo, cPWU). We fully acknowledge the reviewers’ concerns regarding the need for more compelling experiments. In the individual responses, we have provided robust evidence and explanations, addressing all the major issues raised thoroughly. 

The Rebuttal PDF here attached complements the individual responses by providing a framework figure, more photorealistic renderings, additional evidence of our intrinsic multi-view consistency and the benefits of our proposed attention, and the results of high-frequency CelebA texture generation.

We trust that the supplementary clarifications and data we have provided effectively address all the concerns and queries raised by the reviewers.

Meta-review: The paper introduces a novel approach for 3D texture generation by avoiding traditional UV mapping and instead using colored point clouds constrained to mesh surfaces, combined with the DDPM and heat diffusion. This method addresses common UV-based texturing issues like seams and varying resolution. The reviewers appreciated the innovation, particularly the integration of a heat-diffusion-based self-attention mechanism for ensuring spatial consistency, and noted the potential for broader applications beyond mesh texturing.

However, there were concerns about the method's ability to generate high-frequency details, with some results appearing cartoonish or blurred. The evaluation could be more compelling with additional comparisons to state-of-the-art UV-based methods. The authors effectively addressed these concerns during the rebuttal, providing additional experiments and clarifications that satisfied most reviewers, leading to an improved score from one reviewer.

Given the paper's innovative contributions and the authors' engagement in addressing feedback, the paper is recommended for acceptance as a poster presentation. This format will allow the authors to showcase their work while gathering further insights for future improvements.
Decision: Accept (poster)