S-PIFu: Integrating Parametric Human Models with PIFu for Single-view Clothed Human Reconstruction
Authors: Kennard Chan
Guosheng Lin
Haiyu Zhao
Weisi Lin
Venue: NeurIPS 2022 Accept (2022)
Keywords: Single-view clothed human reconstruction
parametric human body models
pixel-aligned implicit models

Abstract: We present three novel strategies to incorporate a parametric body model into a pixel-aligned implicit model for single-view clothed human reconstruction. Firstly, we introduce ray-based sampling, a novel technique that transforms a parametric model into a set of highly informative, pixel-aligned 2D feature maps. Next, we propose a new type of feature based on blendweights. Blendweight-based labels serve as soft human parsing labels and help to improve the structural fidelity of reconstructed meshes. Finally, we show how we can extract and capitalize on body part orientation information from a parametric model to further improve reconstruction quality. Together, these three techniques form our S-PIFu framework, which significantly outperforms state-of-the-arts methods in all metrics. Our code is available at https://github.com/kcyt/SPIFu.

S-PIFu: Integrating Parametric Human Models with PIFu for Single-view Clothed Human Reconstruction
Kennard Yanting Chan
Guosheng Lin
S S-Lab, Nanyang Technological University kenn00420e. ntu. edu. sg
Nanyang Technological University gslinentu. edu. sg
Haiyu Zhao
Weisi Lin
Sense Research zhaohailyu@@ensetinn.c. com
S-Lab, Nanyang Technological University wslinentu. cawaenttu.du...... sg
Abstract
We present three novel strategies incorporate parametric body model into a pixel-aligned implicit model for single-view clothed human reconstruction. Firstly, we introduce ray-based sampling. novel technique that transforms parametric model into set of highly informative, pixel-aligned 2D feature maps. Next, we propose a new type feature based on blendweights. Blendweight- labels serve soft human parsing labels and help to improve the structural fidelity of reconstructed meshes. Finally, we show how we can extract and capitalize on body part orientation information from a parametric model to further improve reconstruction quality. Together, these three techniques form our S-PIFu framework, which significantly outperforms state-of-the-arts methods in all metrics. Our code is available https://githu...om/k///////
1 Introduction
Human digitization an important topic that has application in areas like virtual reality, robot navigation, and medical imaging. While there exist high-end, multi-view capturing systems that can accurately digitalize and reconstruct 3D human bodies (4, 5 13). they are largely inaccessible to general consumers. In search of more accessible methods, the research community has delved into human digitization approaches that use simple inputs such as single image (2 16 22 21).
Various methods have been proposed for single-image clothed human digitization, but one class of approaches that has gained notable attention is pixel-aligned implicit models (20) 21, which are relatively lightweight and could capture fine-level geometric details such as clothes wrinkles. But these models are prone depth ambiguity (causing unnaturally elongated body parts) and to producing human meshes with broken limbs (8, 18).
In order overcome these two limitations, He al. (7) designed ARCH++, model which combines a pixel-aligned implicit model with a parametric human body model. However, ARCH++ depends heavily on having near-perfect mapping between the canonical and posed spaces (especially during training). which is often infeasible to attain due to the varied clothing of human meshes. Another problem with ARCH++ is that only uses the (x.y.z) coordinate information of the parametric model, ignoring other information that could well prove useful.
In order to overcome these issues, we propose S-PIFu, pixel-aligned implicit model that utilizes parametric human body model without requiring a mapping between the canonical and posed spaces. S-PIFu transforms parametric body model into set of -xxeledlggeed 2D feature maps, which can contain more information than just the (x.y.z) coordinate information. In its essence, S-PIFu is a new
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

(a) Groundtruth
(b) PIFu
(c) +RCH++
(d) PIFuHD
(c) Ours- SPIFu
Figure I: Compared with SOTA methods, including (b) PIFu 20). (c) ARCH++ 7 and (d) PIFuHD (21), our proposed model can precisely reconstruct the fine details of a human subject without floating artefacts unnaturally elongated body parts. In the last row, the reconstruction outputs are colored by projecting the RGB values from the imput image onto them. The purpose of the coloring is only to aid in visualization. Subject's face censored as required by dataset's owner.
method of incorporating SMPL-X (17) parametric body model into a PIFu (which is a pixel-aligned implicit model). The 'S' stands for SMPL-X.
The main contributions of our work are:
1.
We introduce ray-based sampling (RBS), a novel technique that transforms a parametric model into set of highly informative, pixel-aligned 2D feature maps. We find that ap- plying RBS on coordinate information alone can already propel a vanilla PIFu to SOTA performances.
2. Furthermore, we propose new type of feature based on blendweights. Blendweight- baaed labels serve as soft human parsing labels and lead significant improvement over the baseline model.
3. Thirdly, we show how we can extract and capitalize on body part orientation information from parametric model. We show this can also improve the baseline model significantly.
Each of our contributions can be added to any existing pixel-aligned implicit model.
2 Related Work
2.1 Single-view Human Reconstruction
Single-view human reconstruction (or digitalization) methods can be classified into parametric and non-parametric approaches. Parametric methods, such as 12 ID. recover human body shapes by estimating the parameters of parametric human body model (e.g. SMPL (14) SMPLX (I7) from an input image. However, these parametric methods can only reconstruct clothless human meshes.
On the other hand, non-parametric approaches do not require parametric body model. For example, (22) utilizes 3D CNN to directly estimate volumetric body shape from single RGB image. subclass of non- parametric approaches that have generated significant interest is pixel-aligned implicit models (20: 8 21). These methods model human body as an implicit function, from which a mesh can be obtained using Marching Cubes algorithm (15). One of the first pixel-aligned implicit models is the Pixel-Aligned Implicit Function (PIFu) (20). PIFu uses a multi-layer perceptron to estimate the occupancy of query points in 3D space. Using Marching Cubes, PIFu can reconstruct
clothed human meshes effectively. A short illustration of PIFu shown in Fig. 2
(See dotted box).
However, as aforementioned, pixel-aligned implicit models are prone to generating human meshes with broken limbs or unnaturally elongated human body parts. Recent pixel-aligned implicit models like IntegratedPIFu (3) propose techniques that successfully mitigate these problems, but the problems still exist. In contrast, parametric methods do not suffer from these problems. This observation led to hybrid methods that combine pixcl-aligned implicit model with parametric method.

---

Frontal
Nomm Nap
Nommal Vep
6122812
Ewdicated rnsooriion
Encoder
Deooder
Image (812x512)
Inout RGB
Concore
MLP
predictiona (icleu))
SMPI XX
512x512
20 Feature Maps:
1. Coordinate (x.z2) "C"
model
Sampling
2. Blendweighi-bape Soft Human parsing maps
Part Orientation maps-
Groundruhh Mesh
Figure 2: Illustration of our S-PIFu framework.
2.2 +RCH++
ARCH (2) is hybrid method that parametric body model determine how posed human mesh (posed space) could be mapped into common pose (canonical space). A PIFu is then used to train and infer body mesh in the canonical space. Another hybrid method, PaMIR (26), voxelizes an estimated SMPL (14) parametric body model before passing it through a 3D encoder to get voxel-aligned features, which are passed into a PIFu.
ARCH++ 7 improves upon ARCH and PaMIR by jointly estimating the occupancy of points in both the posed and canonical spaces. In addition, ARCH++ proposes using PointNet+++119) to capture and encode the structural information of a parametric model. But, as aforementioned, ARCH++ relies heavily having near-perfect mapping between the canonical and posed spaces and only uses the (x.y.z) coordinate information of the parametric model, ignoring other meaningful information.
2.3 Novelty of our Method
Our paper introduces Ray-based sampling (RBS), which is completely different way of extracting information from SMPL-like mesh. RBS directly generates pixel-aligned features from a SMPL-like mesh using rays that penetrate through the mesh. While the ray penetrates through the mesh, collects meaningful information from the faces that it hit. We will elaborate on this later.
Compared with ARCH and ARCH++, RBS does not need the hard-to-obtain near- ear-peefeect mapping between canonical space and posed space (infeasible when human subject wears loose clothing like jacket coat). Compared with PaMIR, RBS can generate features at much higher resolution (For height and width, RBS features has a 512x512 resolution, which is times bigger than the 128x 128 voxel features used in PaMIR). The resolution of PaMIR's voxel features is limited to 128x due to GPU memory constraints (26).
Moreover, we also introduce blendweight-basee based labels and body orientation labels (See later).
3 Method
An illustration of our S-PIFu shown in Fig. 2 Similar to 21 and (7), we use a pix2pixHI network (23) to predict frontal and rear normal maps from an input RGB image. In addition, S-PIFu uses SMPLify-X (17) to estimate SMPL-X parametric body model (17) from the input RGB image.
In contrast to PaMIR and ARCH++, our method does not need train separate encoder to encode parametric model. Instead, we directly transform parametric model into set of 2D feature maps (called coordinate maps) using what call ray-based sampling (See Section 3.I.. The set of 2D feature maps is significantly more lightweight than voxels used by PaMIR, and, unlike ARCH and ARCH++, does not require an error-free mapping between canonical and posed spaces.
Ray-based sampling captures (x.y.z) coordinate information from the SMPL-X model. But can also be used to capture dlendweight-basee labels that serves proxy to soft human parsing labels (See

---

Section 3.2
The blendweight-bas labels are represented as another set of 2D feature maps (called blendweight- based maps).
Beside soft human parsing labels, S-PIFu also leverages on ray-based sampling to extract body part orientation information from SMPL-X model (See Section 3.3 giving additional set of 21) feature maps called body part orientation maps.
We concatenate coordinate maps, blendweight-based maps, body part orientation maps, frontal and rear normal maps, and a RGB input image together before feeding them into a PIFu, which will output oceupancy predictions of query points in 3D space. The PIFu will be used to reconstruct clothed human meshes, which are then refined with our normal maps using the mesh refinement technique employed in (7)
3.1 Ray-based Sampling
After estimating a SMPL-X model from a RGB image, we use Ray-based Sampling (RBS) to transform the SMPL-X model into set of 2D feature maps. See an illustration of RBS in Fig. 3 In RBS, we project the SMPL-X model into an image plane. We interpret the projection lines as rays (in blue in Fig. 3) that move towards the SMPL-X, and each ray associated with pixel location.
Any ray that hits the SMPL-X mesh will simply penetrate through the mesh and continue moving in its original direction. Each ray will record the faces (more specifically, the face indices) of the mesh that it has penetrated through. Using the face indices, we can identify the SMPL-X vertices that form the penetrated faces. The (x.y.z) values of the vertices then placed in 2D feature maps.
To be clearer, we will look at an example. Let us denote the lower ray (in blue) illustrated in 'Close-up" of Fig. Bas We see that ray r penetrates 4 faces (f1, f2, fs. fa) of the SMPL-X mesh. Each face in SMPL-X is formed by three vertices. Thus, we can identify the 12 vertices that form f1, f2. fa. and fa- If fi is formed by vertices 111, U12, and 013, then we will average up the coordinates of U11, U12, and U13 to obtain the face average coordinates T1, D1. 21. With 4 faces, the ray r will have total of 4 sets of face average coordinates. Recall that each ray is associated with exactly one pixel location in the image plane (the image plane actually corresponds to a 2D feature map). Thus, will need 4x3 12 2D feature maps to store the information captured by the ray (4 refers the number of face average coordinates, and refers to the x.y.z coordinates). Terminology: In some literature, the 12 feature maps are also referred as a feature map with 12 channels.
We limit the number of faces that a ray can penetrate through to 6 faces as it is very unlikely for a ray to penetrate more than 6 faces. In total, RBS will compute 6x3 18 2D feature maps (if we only use coordinate information). If we specify the 2D feature maps to have resolution of 512x512, then total of 512x512 rays will be used. This is actually relatively lightweight in practice because RBS only need to be done once, and it can be done in pre-training. The 2D feature maps can be efficiently stored using sparse matrices at all (Also, this means that zeroes will be inserted onto a pixel on the 2D feature map if a ray that corresponds to that pixel does not hit any surface.)
significant number of rays will not hit the SMPL-X mesh
memory-intensive intensive 3D voxels that are prone to quantization errors caused by the necessary use of low spatial resolution (to preserve memory) (7.. Compare to voxels, our set of 2D feature maps is significantly more lightweight, making it possible to utilize a much higher resolution of the maps.
PaMIR uses memory
In addition, unlike ARCH++ RBS does not require mapping between canonical and posed spaces and is thus not prone errors in the mapping. In Section 4.2.2 we show that RBS with coordinate information alone is able to significantly outperform ARCH++ in almost all metrics.
In short, RBS generates 18 2D feature maps from SMPL-X model. These feature maps are concatenated with the original inputs of a PIFu and then fed into the PIFu as originally designed. The 18 feature maps inform the PlFu of the structural information of SMPL-X mesh.
3.2 Using Blendweight-based labels as proxy to Soft Human Parsing Labels
In existing hybrid models, such as ARCH 9. PAMIR 26) and ARCH++ 7., the only information that was extracted from the predicted parametric human body model the (x.yz) coordinate information. It is a waste parametric body model contains much more information than just the (x.y.z)

---

A my pointing inco the page
Fread Mi penccrige SWPPL modes)
Logeod
intortationn
Cansern
Foaltoo
Frant-Viow
Side-Viow
Close-up
Figure 3: Illustration of Ray-basse Sampling. As in 20 weak peak-pperpeeetiee camera is assumed. The diagram assumes the SMPL-X model has been scaled, thus rays and projection lines are parallel.
coordinate information. One such information is the human parsing information contained in a parametric body model.
Parametric human models, such as SMPL(14) and SMPL-X(17), are formed by predefined and fixed number of vertices. Each of these vertices can be easily mapped to discrete human parsing label (e.g. head, upper left arm, hip etc.). While the pose and shape of a parametric model can change, the discrete human parsing label for each of these vertices remain largely unchanged. For example, if the 50th vertex of the SMPL-X model has a human parsing label of 'head', it will retain this human parsing label regardless of what pose and shape the SMPL-X model takes.
Using Ray-based Sampling (RBS), we can map the SMPL-X vertices to pixel locations of 2D feature map. Since we know the discrete human parsing label of each SMPL-X vertex, we can essentially create 2D feature map that consists of the discrete human parsing labels that given to each S.MLL-X vertex. Please refer to Fig. 4a on more detail on how the 2D feature map is computed. This 2D feature map can then be fed into a PIFu. (In reality, as explained in Section 3.1 total of 6 such 2D feature map because a ray can penetrate up to faces.)
we need
However, assigning discrete human parsing label to each SMPL-X vertex can cause unnecessary noise at the boundary between two different human body parts. For example, looking at Fig. 4., we observe that at the boundary between two different body parts, the vertices can be assigned two very different human parsing labels despite being very similar to one another. Moreover, vertices that are assigned the same human parsing label (e.g. lower left leg), can actually be very different from one another.
In order to resolve these shortcomings, we would require a method to give a form of soft human parsing labels to each of the SMPL-X vertices. This leads us to the concept of blendweight in the SMPL-X model. Blendweight is matrix of shape (N x K) where N is the total number of SMPL-X vertices, and K is the number of joints SMPL-X. Essentially, the blendweight matrix contains a K-length vector of values for each SMPL-X vertex. This vector determines how much each joint in SMPL-X would influence the position of a specific vertex when the SMPL-X model is transformed from default pose to an arbitrary pose. We can interpret this K-length vector as vector of weights that determine the degree of association that a specific SMPL-X vertex has with the K different joints in the SMPL-X body. In other words, if we consider joint locations be a rough proxy for body part locations, then we can use this K-length vector as a soft human parsing label for a vertex.
A simplistic visualization of how blendweight can be used as soft human parsing labels is shown in Fig. 4p. In reality, the blendweight more complicated (and useful) than that as each vertex is given a K-length vector where K-55 in SMPL-X model. Thus, unlike what is shown in the figure, the soft human parsing label given to a vertex does not merely weigh the association of that vertex with two different body parts (or joints). The soft human parsing label weighs the association of that vertex with 55 different joints. In broader sense, blendweight- based labelling is simply giving each vertex an unique ID, except that the ID is made up of mathematically meaningful values that tell us how vertex is similar to another vertex and in what aspects are they similar (e.g. Are two vertices both influenced by the 10th joint?). In addition, parsing labels and blendweight saadde soft parsing labels in supplementary materials.
provide comparison between using discrete human
In order to reduce computational cost, we utilize principal component analysis (PCA) to reduce the dimensions of the blendweight matrix from (N x 55) to (N x 24). Thus, we can map each vertex to 24-length vector. Using RBS, we will generate 6 x 24 or 144 2D feature maps (As explained in
5

---

The trienia
Each venaa has discrtte hurran parsing label h,
based Sampling fazze, verlose, of mesh 21 Thus, we co compure the nt of face by
By face Ch,
(a) Discrete Human Parsing
git-bresed by. lengt 55 (below PCA)
Simmlar discwele furmen pansiny lobels, can compute h2 ort by:
N of boow - 3 (h. ofyy
N of%
(b) Blendwaights
Figure 4: Comparison of Discrete Human Parsing labels with Blendweight Human Parsing labels. In
(a), the ha of vertex a one-hot vector that has length of 27 if we decide partition the human body into 27 different body parts. In (b), each vertex has a h2 label. Each element in a h2 vector is a real number (unlike discrete human parsing label, which is a one-hot vector)
Section 3.1 ray can penetrate up to 6 faces.). An illustration on how we compute the 2D feature maps is shown in Fig. 4b.
3.3 Encoding the Orientation of Body Parts using Vertex Normal information
With bendwwightt-assee labels, our model will have fine- grained body part labels of a human body, However, both 2D and 3D spaces, the same body part can appear very different when is oriented differently. For example, a closed fist and a wide open hand appear very differently despite belonging to the same body part. This limits the usefulness of having human parsing labels (even if is soft or fine-grained labels) as any model would be confused by how the exact same human parsing label can have vastly different appearances. Thus, we argue that it important for human parsing labels to be complemented with body part orientation information.
To extract body part orientation information from predicted SMPL-X model, we start by applying Ray-based Sampling (RBS). For each SMPL-X face / that is penetrated by ray, we collect the vertex normal vectors of the three vertices that constitute that face. We then compute the face average normal of f by taking the average of the three vertex normal vectors. This face average normal vector tells us the current orientation of the body part (more accurately, the orientation of face on that body part). In order to provide our model with more information about the body parts, we also provide the initial orientation of the body parts (i.e. the face average normal of the same face but when the SMPL-X model is in the initial or rest pose. See Fig. for an illustration). The reason for including the initial orientation is to allow our model to compute how different the current orientation (of body part) is from its initial orientation. If the difference is small, it tells our model that the body part should appear similar to how appears at rest pose. But if the difference is big, the body parts should appear more different from its rest pose. (Refer to the short example in Fig. 5 In other words, to help our model interpret face average normals, we are giving our model a reference face average normal for every face in the SMPL-X.
In addition, the inclusion of face normals in RBS is important because, in RBS, ray does not know whether face it penetrated is a face that is facing towards the camera face that is facing away from the camera. In the context of a PIFu, which the base architecture of our S-PIFu, it is important to know whether a face is facing towards or away from the camera. This is because PIFu works by predicting whether a query point is inside or outside a mesh. So when there is a face that is facing towards the camera, it means that query points in front of this face is outside of the mesh. Likewise, when there a face facing away from camera, it means query points in front of the face surface are inside of the mesh. The face normals thus provide important hints to PIFu to aid its predictions.

---

1) The rod trlangle represents face (la.
2) Black anow represents the face normal
ny] off
maggotood
(a) Rest Pose (We use the rest reference pose)
Fiether
After Posing
Hare, the sere faow in (A) But face has changed from ton;, change
(b) Currrent Pose (i.o. the pose of given train or test example)
magrited
ni f. compute n; fg
Figure 5: Illustration of the change in orientation for a face on SMPL-X body model. n'o Do tells S-PIFu how far face's orientation has been changed when rest-pose mesh (a) being posed the current pose (b). Thus, n'a o ny does contain information about the current pose (i.e. if n'o ny is equal zero vector, then current pose the rest pose.)
Experiments
4.1 Datasets
We train our models and the competing models the THuman2.0 dataset (24). which consists of high-quality scans of human subjects. The subjects are primarily young Chinese adults. We exclude scans meshes that have very obvious scanning errors. Meshes that suffer from extreme self-occlusion are also excluded because we do not hope to reduce the task into a guessing game. This data cleaning process is conducted before any of the models is trained as ensure fair comparison between the different models. In all, we use 362 human meshes from the THuman2.0 dataset. A 80-20 train-test split is used, and for each mesh, render 10 RGB images at different yaw angles with weak-perse perespeesseeeieee camera.
In addition, include the BUFF dataset (25) additional evaluation dataset. No model is trained using the BUFF dataset. Systematic sampling based on sequence number carried out on the BUFF dataset obtain total of 93 human meshes for testing. We systematic sampling to ensure that, for the same human subject, meshes different poses are obtained and repeated (or near identical) poses omitted. More implementation details is included in our supplementary materials.
4.2 Comparison with State-of-the-art
compare S-PIFu against the existing models on single-view clothed human reconstruction qualitatively and quantitatively. For our quantitative evaluation, we follow the same metrics as used in (20) 21 7 Specifically, we use Chamfer distance (CD), Point-to-Surface (P2S), and Normal reprojection error (Normal).
Existing methods that we compared against include PIFu (20), ARCH++ 77) and PIFuHD (1... We included PIFu as it is our base architecture and thus serve an important baseline. ARCH++ is used as it is the state-of- of-toe--tthhteree the-arts for incorporating a parametric human body model into aixel-aligned implicit model. Also, in (7), He et al. demonstrated that ARCH+ is an improvement over both ARCH 9 and PaMIR (26). and this our explanation for not including ARCH and PaMIR. In addition, another reason for not including PaMIR is that its authors did not demonstrate that PaMIR able to outperform PIFuHD, which is the state-of-the-arrs that time. We include PIFuHD still one of the most effective methods for single-view clothed human reconstruction.
7

---

(u) Groundtruth
(b) PIFu
(c) ARCH++
(d) PIFulID
(c) Ours SPIFu
Figure 6: Qualitative evaluation with tfae-ooftthe-ar methods, including (b) PIFu (20), (c) ARCH++
7 and (d) PIFuHD (1 For each row, the first object the input RGB image given the models.
4.2.1 Qualitative Evaluation
We present qualitative comparison of our S-PIFu with the other existing methods in Fig and Fig In Fig [] we observe that only our approach is able to capture the correct structure of the human subject body. Other methods tend to produce unnaturally elongated body parts, missing body parts, or floating artefacts. Moreover, S-PIFu successfully reconstructs the fine details of the human body, such the wrinkles the long pants and the precise facial features.
In 6 we observe that PIFu, ARCH++, and PIFuHD tend to generate broken limbs, floating artefacts, missing body parts. In contrast, S-PIFu is able generate clean, natural structure of human body. We believe that ARCH++'s tendency to produce missing body parts is due to the imperfect mapping between the canonical and posed spaces. In (7), ARCH+ trained using the RenderPeople dataset (I, which mainly consists meshes in upright standing poses (6). THuman2.0 dataset, however, has more varied and complicated poses, making perfect mapping harder achieve.
4.2.2 Quantitative Evaluation
In addition, we provide quantitative evaluation of our methods against the state-of-the-arts methods in Tab. []] The table shows that when PIFu is incorporated with coordinates information extracted using our ray-based sampling ('PIFu + M C"), it can achieve significant improvements and even outperforms current state-of-the-arts models in the THuman2.0 dataset. However, PIFu + M + C does not do as well the BUFF dataset, indicating a lesser generalizability to foreign dataset.
In contrast, S-PIFu, which combine coordinates information (C), blendweight-baad based labels (B), and body part orientation information (N) into a PIFu (together with frontal and rear normal maps M), outperforms the existing state-of of-the-arts models in all metrics in both datasets. This suggests that the inclusion and N helps our model to learn more generalizable features from the training data.
4.3
Ablation Studies
4.3.1
Evaluation of using Ray-based Sampling to extract coordinate information
In order to evaluate the benefits of using Ray-based sampling (RBS) to extract coordinate information from a SMPL-X mesh, we train a PIFu that uses not only the RGB image, frontal normal map, and rear normal map, but also the additional input of coordinates (x.y.z) extracted via RBS. We compare this model with vanilla PlFu that trained with only the RGB image and the frontal and rear normal maps. See qualitative results in Fig. 7 The PIFu with coordinates information can better capture

---

Table 1: Quantitative evaluation of our models against the state-of-the-arts in the THuman2.66 test set and BUFF dataset. (M frontal and rear normal maps. C,B.N Coordinates information, Blendweight-base labels, Body part orientation information from Ray- bay-based Sampling respectively. S-PIFu combines C, B, and N into a PIFu with M i.e. S-PIFu PIFu
B++
THuman2.0
BUFF
Methods
CD (10+t)
P2S (10*)
Normal
CD (10")
P2S (10')
Normal
PIFu
10.83
6,671
2.211
3,427
3.926
2.580
PTFu M
3.085
2.673
1.731
1.979
1.973
1.947
ARCII+
3.670
1.979
2.131
3,050
1.811
3,436
PIFuHD
2.758
2.215
1.698
1.964
1.845
2,010
(Ours) PiFu + M + C
1,999
1.626
1.413
2.022
1840
1,536
(Ours) PIFu + +
2.595
2.079
1.411
2.050
1.866
1.559
(Ours) PIFu + + N
2.818
2.275
1.416
2.060
1.557
(Ours) S-PIFu
2.035
1.629
1.440
1.923
1.726
1.564
the structure of human meshes compared to the other PIFu, which often produces broken or missing limbs, and floating artefacts.
Quantitatively, we can compare the CD and P2S of the two models using the second row ("PIFu + M') and the fifth row ('PIFu M C.) of Tab. [T] We find that although 'PIFu M C' fared marginally worse than 'PlFu + M' in the CD of datasataset, it significantly outperformed 'PIFu + M' in the CD and P2S of the THuman2.0 dataset and in the P2S of the BUFF dataset.
As mentioned Section 4.2.2.
'PIFu + M + C' also outperforms ARCH+ and other methods.
(a) Groundrruth
(b) PIFu
(c) PiFu # Coordinate information from RBS
Figure 7: Effect of incorporating SMPL-X coordinates extracted using Ray-based sampling (RBS).
4.3.2 Evaluation of using flendweight-based labels
Similarly, in order to evaluate the effect of using Blendweight-based labels, we train a PIFu that takes Blendweight- based labels as the only additional inputs (besides RGB image and frontal and rear normal maps). We compare this model with a PIFu that is given only the RGB image and the frontal and rear normal maps. See the qualitative results in Fig. g We find that using blendweight-basee labels helps to improve the structural fidelity of reconstructed meshes. Specifically, it tackles the problems of unnaturally elongated human parts and floating artefacts.
Quantitatively, we can compare the CD and P2S of the two models using the second row ('PIFu + M') and the sixth row ('PIFu + M B') of Tab. [ Similarly, marginally worse than 'PIFu + M' in the CD of BUFF dataset. For the CD and P2S in THuman2.0 and P2S in BUFF, 'PFuu M + B' significantly outperformed 'PIFu + M'. Also, in our Supp. Mat., we show the results when we replace the 'B' in 'PIFu + M + B' with discrete human parsing labels.
find that 'PIFu M B only fared
4.3.3 Evaluation of using Body Part Orientation Information
Lastly, we evaluate the merits of using body part orientation information by comparing a PIFu that given only RGB image and frontal and rear normal maps with another PIFu that is given the additional

---

(a) Groundtruth
(b) PIFu
(c) PIFu Blendweight
Figure 8: Evaluation of the effect of incorporating Blendweight based Human Parsing Information
input of body part orientation information. See the qualitative results in Fig. 9 We find that using
body part orientation information helps further reinforce the structural accuracy of reconstructed meshes by mitigating the occurrence of noisy artefacts in the reconstruction outputs.
Quantitatively, can compare the CD and P2S of the two models using the second row ('PIFu + M') and the seventh row ('PIFu + M + N') Tab. [ Except for the CD in Buff dataset, for which 'PIFu + M N' performed marginally worse than 'IIFu M', 'PIFu + M N' significantly outperformed 'PIFu + M' in the CD and P2S of the two datasets. Also, in our Supp. Mat., we compare the results when replace the 'N' in 'IFuu M N' with 'N' but without the initial orientation of body parts.
(a) Groundtruth
(b) PiFu
(c) PIFu Body Part Orientation
Figure 9: Evaluation of the effect of incorporating Body Part Orientation Information
Conclusion
Our S-PIFu framework consists of three novel strategies designed to incorporate a parametric human body model into pixel-aligned implicit model. Firstly, we proposed ray-based sampling, novel technique that transforms parametric model into highly informative 2D feature maps. Next, we introduced type of feature based blendweights. Finally, showed how body part orientation information can be extracted from a SMPL.-X model to further improve reconstruction quality.
Acknowledgments and Disclosure of Funding
This study is supported under the RIE2020 Industry Alignment Fund Industry Collaboration Projects (IAF-ICP) Funding Initiative, as well as cash and in-kind contribution from the industry partner(s). G. Lin's participation is supported by the Ministry of Education, Singapore, under Academic Research Fund Tier 2 (MOE-T2PP2022-0000)
10

---

References
[1] RENDERPEOPLE, RENDERPEOPLEE,RCP: http:/P/i...ttpeeeeeererrppooo
[2] Alldieck, T., Magnor, M., Bhatnagar, B.L., Theobalt, C., Pons-Moll, G.: Learning reconstruct people in clothing from a single rgb camera. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Patter Recognition, pp. 1 1175-1186 (2019)
[3] Chan, Y., Lin, G., Zhao, H., Lin, W.: Integratedpifu: Integrated pixel aligned implicit function for single-view human reconstruction. In: European conference computer vision (2022)
[4] Collet, A., Chuang, M., Sweeney, P., Gillett, D., Evseev, D., Calabrese, D., Hoppe, H., Kirk, A., Sullivan, S.: High-quality streamable free-viewpoint video. ACM Transactions on Graphics (ToG) 34(4), 1-13 (2015)
[5] Guo, K., Lincoln, P., Davidson, P., Busch, J., Yu, X., Whalen, M., Harvey, G., Orts-Escolano, S., Pandey, R., Dourgarian, J., al.: The relightables: Volumetric performance capture of humans with realistic relighting. ACM Transactions on Graphics (ToG) 38(6), 1-19 (2019)
[6] He, T., Collomosse, J., Jin, H., Soatto, S.: Geo-pifu: Geometry and pixel aligned implicit functions for single-view human reconstruction. Advances in Neural Information Processing Systems 33, 927699277999 (2020)
[7] He, T., Xu, Y., Saito, S., Soatto, S., Tung, T.: Arch++: Animation-ready clothed human recon- struction revisited. In: Proceedings of the IEEE/CVF International Conference Computer Vision. pp. 11046-11056 (2021)
[8] Hong, Y., Zhang, J., Jiang. B., Guo, Y., Liu, L.., Bao, H.: Stereopifu: Depth aware clothed human digitization via stereo vision. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 535-545 (2021)
[9] Huang, Z., Xu, Y., Lassner, C., Li, H., Tung, T.: Arch: Animatable reconstruction of clothed humans. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition. pp. 3093-3102 (2020)
[10] Kanazawa, A., Black, M.J., Jacobs, D.W., Malik, J.: End-to-end recovery of human shape and pose. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7122-7131 (2018)
[11] Kolotouros, N., Pavlakos, G., Black, M.J., Danillidis, K.: Learning to reconstruct 3d human pose and shape via model-fitting in the loop. In: Proceedings of the IEEE/CVF International Conference Computer Vision. pp. 2252 222221261222 (2019)
[12] Kolotouros, N., Pavlakos, G., Daniilidis, K.: Convolutional mesh regression for single-image human shape reconstruction. In: Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 4501-4510 (2019)
[13] Lombardi, S., Saragih, J., Simon, T., Sheikh, Y.: Deep appearance models for face rendering. ACM Transactions on Graphics (ToG) 37(4), 1-13 (2018)
[14] Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., Black, M.J.: Smpl: A skinned multi- person linear model. ACM transactions on graphics (TOG) 34(6), 1-16 (2015)
[15] Lorensen, W.E., Cline, H.F.: Marching cubes: high resolution 3d surface construction algorithm. ACM siggraph computer graphics 24)) 163-169 (1987)
[16] Natsume, R., Saito, S., Huang, Z., Chen, W., Ma. C., Li, H., Morishima, S.: Siclope: Silhouette- based clothed people. In: Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition. pp. 4480-4490 (2019)
[17] Pavlakos, G., Choutas, V., Ghorbani, N., Bolkart, T., Osman, A.A.A., Tzionas, D., Black, M.J.: Expressive body capture: 3D hands, face, and body from single image. In: Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). pp. 10975-10985 (2019)

---

[18] Peng, S., Zhang, Y., Xu, Y., Wang, Q., Shuai, Q.. Bao, H.. Zhou, X.: Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 9054-9063 (2021)
[19] Qi, C.R., Yi, L.,, Su, H., Guibas, L.J.: Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems 30 (2017)
[20] Saito, S., Huang, Z., Natsume, R., Morishima, S., Kamazawa, A., Li, H.: Pifu: Pixel-aligned im- plicit function for high-resolution clothed human digitization. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 2304-2314 (2019)
[21] Saito, S., Simon, T., Saragih, J., Joo, H.: Pifuhd: Multi-level pixel-aligned implicit function for high-resolution 3d human digitization. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 84-93 (2020)
[22] Varol, G., Ceylan, D., Russell, B., Yang. J., Yumer, E.. Laptev, I., Schmid, C.: Bodynet: Volumetric inference of 3d human body shapes. In: Proceedings of the European Conference on Computer Vision (ECCV). pp. 20-36 (2018)
[23] Wang, T.C., Liu, M.Y., Zhu, J.Y., Tao, A., Kautz, J., Catanzaro, B.: High-resolution image synthesis and semantic manipulation with conditional gans. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8798-8807 (2018)
[24] Yu, T., Zheng, Z., Guo, K., Liu, P., Dai, Q., Liu, Y.: Function4d: Real-time human volumetric capture from very sparse consumer rgbd sensors. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR2021) (June 2021)
[25] Zhang, C., Pujades, S., Black, M.J., Pons-Moll, G.: Detailed, accurate, human shape estimation from clothed 3d scan sequences. In: The IEEE Conference Computer Vision and Pattern Recognition (CVPR) (July 2017)
[26] Zheng. Z., Yu, T., Liu, Y., Dai, Q.: Pamir: Parametric model-conditioned implicit representation for amage-based human reconstruction. IEEE transactions on pattern analysis and machine intelligence (2021)
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material or as URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] Training details mentioned here. Readers may refer to supplementary materials for more details
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [Yes] All results are obtained after using three pre-specified different random seeds. We do not have space here for the exact error bars diagram
12

---

(d) Did you include the total amount of compute and the type of resources used (e.g.. type of GPUs, internal cluster, or cloud provider)? [Yes] Refer to our supplementary materials
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes]
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you're 're using/curating? [Yes]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "VRMU-rVXc1r", "reviewer": "Reviewer_49L7", "summary": "This paper presents several input features derived from parametric human model to PIFu-like 3D reconstruction task of clothed humans. At each pixel from the input view, this work propose to compute the multiple ray intersection with the parametric body model and extract positions, skinning weights, and surface normal at the intersections as auxiliary pixel-aligned features. The experiments show that the proposed features significantly improve the robustness and accuracy of reconstruction over prior methods in THuman2.0 and BUFF datasets. ", "questions": "- Please address the concerns above.\n- After reading through the paper, I still couldn’t figure out what “S” in S-PIFu stands for. Please clarify.\n- Since intersection locations can be obtained via barycentric interpolation, why is simply averaging of 3 vertices used? Does the performance improve by taking barycentric interpolation?\n- What value is inserted if the ray does not hit surface? \n- What is the motivation behind conditioning vertex positions as input features? Please elaborate.\n\nOther comments:\n- Citation format is not the official one. \n- L5: blendweight-based soft human parsing label is merely skinning weights . Please use common term.\n- L87: The method that proposes to use front and back normals are PIFuHD not PIFu.\n- L168: principal component analysis", "limitations": "Neither limitation nor its societal impact is discussed in the paper.", "rating": 3, "confidence": 5, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The strengths of the paper can be summarized as follows:\n- The simplicity of the proposed method is a great advantage. Especially the plug and play nature of the proposed feature encoding allows us to boost the performance regardless of the baseline method (e.g, PIFu, PIFuHD).\n- The experimental results support the effectiveness of the proposed approach. \n\nOn the other hand, the paper has the following weaknesses:\n- It is not clear how this approach performs for the pixels that do not belong to SMPLX body. Non-overlapped regions are quite common for clothed human as silhouette of clothed humans is larger than minimally clothed SMPLX. Intuitively, the proposed approach provide no information for these pixels. \n- Related to this, the proposed approach seems to require very accurate underlying SMPLX fitting. However, as PaMIR and ICON mentioned, off-the-shelf body pose estimation networks do not provide very accurate fitting and thus additional optimization is required. It is not clear how well the proposed approach handles inaccurate pose parameters. Qualitative results from images in the wild and evaluation with noise perturbed pose parameters are highly recommended. \n- The paper misses one important work, ICON, which improves robustness over PIFu by conditioning SMPL body. Please discuss and compare.\n\nICON: Implicit Clothed Humans Obtained From Normals\nYuliang Xiu, Jinlong Yang, Dimitrios Tzionas, Michael J. Black; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 13296-13306\n\nDespite its promising performance, I think the paper is not ready for publication due to its incomplete evaluations and comparisons. Also the illustration and exposition can be further improved.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "RmT7tLbq56L", "reviewer": "Reviewer_FRso", "summary": "This paper proposes a new method for single-view clothed human reconstruction by incorporating a parametric body model into a pixel-aligned implicit model. The proposed method has a ray-based sampling method to transform a parametric model into a set of feature maps, proposes blendweight-based soft human parsing labels to improve the structural fidelity, and a method to extract body part orientation information from a parametric model to improve the reconstruction quality.", "questions": "Please see the above section.", "limitations": "It seems not.", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths.\n\n1. A ray-based sampling method is proposed to transform a parametric model into a set of pixel-aligned 2D feature maps.\n2. Blendweight-based soft human parsing labels are used.\n3. A method that extract and capitalize on body part orientation information from a parametric model is proposed.\n\nWeakness.\n\nThe main idea of this paper is to enrich the 2D features with more information derived from SMPL-X parametric body model, before feeding the 2D features into PIFU model for reconstruction. However, three proposed methods seem more like tricks than novel techniques to me.\n\nI am not sure why estimation of body parts orientation can help correct errors caused by shape variations of a body part (explained in L175-176).\n\nThe proposed method relies on [16] to generate the SMPL-X parametric body model. What happens if [16] fails?\n\nBased on Table 1, it seems that PIFU+C performs best on the THuman2.0 dataset than the S-PIFu method. Besides, on the BUFF dataset, all three ablated versions (PIFu+C, B, N) perform (slightly) worse than PIFu or PIFuHD. Can the authors explain this?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Fvcrhtf5wnM", "reviewer": "Reviewer_sxwr", "summary": "This paper introduces S-PIFu which incorporates a SMPL-X model into PIFu for single-view clothed human reconstruction. Specifically, the authors propose a ray-based sampling technique to transform a SMPL-X model (fitted to the input image) into a set of 2D feature maps embedding coordinate, parsing label, and surface normal information. These feature maps are concatenated with the input image (together with the estimated front and rear normal maps) and fed to the encoder of PIFu. Occupancy prediction then follows the normal pipeline of PIFu. Both quantitative and qualitative evaluations show SOTA results.", "questions": "In addition to the points listed as cons above, \n- The original PIFu does not take estimated front and rear normal maps as input. They are only being used in PIFuHD. Please clarify your implementation of PIFu in the comparison and update the results if necessary.\n- Average coordinates of the face vertices are being used in producing the coordinate feature map. It is actually possible to compute the exact intersection of a ray with a face. How does this compare with using average coordinates?\n- Similarly, average normal is computed from the face vertices in producing the normal feature map. How does this compare with simply using normal of a face (triangle)? Alternatively, normal at the point of intersection can be computed using some shading model (e.g., Phong shading). How does this compare with using an average normal?\n- Following the last two comments, it seems average parsing label (although not explicit mentioned) is computed from the face vertices in producing the parsing feature map. It is possible to compute an interpolated parsing label by modifying a shading model to work with parsing labels (e.g., Phong shading). How does this compare with using an average paring label?\n- Table 1 shows that S-PIFu performs worse than PIFu+C on THuman2.0. This suggests that most improvement is actually brought by the coordinate information, and the parsing and normal information may actually harm the occupancy prediction (due to the increased channel number?). Further analysis and discussion are needed. It might be desirable to show the comparison between S-PIFu, S-PIFu - C, S-PIFU - B, and S-PIFu - N in the ablation study. Since the estimated front and rear normal maps are not used in PIFu, another variant to be considered is S-PIFu - 'front & rear normal'.\n", "limitations": "No specific limitation and negative societal impact need to be addressed.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n+ The proposed ray-based sampling technique for transforming a SMPL-X model into a set of 2D feature maps sounds novel. It provides a simple and effective way to encode both geometric and semantic information of the 3D parametric model. This 2D feature map representation works seamlessly with the PIFu framework.\n+ The proposed soft parsing labels based on blending weights sounds novel. It is more informative than discrete parsing label and can help to improve the labels around the boundary between two body parts.\n+ Other than spatial information (i.e., 3D coordinates) which have been exploited by previous works, this work also extract human parsing labels and surface normals from the parametric model which provide further semantic and geometric information for occupancy prediction.\n+ Quantitative and qualitative evaluations show the proposed method outperforms PIFu, PIFuHD, and ARCH++.\n+ Experiments are included to demonstrate the improvement brought by the coordinate, parsing label, and surface normal feature maps, respectively, over PIFu.\n+ The paper is well written and easy to follow.\n\nCons:\n- The authors argue that ARCH++ depends on having a near-perfect mapping between the canonical and posed spaces. Note that such a mapping is defined by the skinning weights and hence it depends on the accuracy of the model fitting. Likewise, this work constructs 2D feature maps from the fitted SMPL-X model and therefore it also depends on the accuracy of the model fitting. However, there is no analysis showing how the model fitting accuracy/error will affect the reconstruction.\n- Unlike ARCH++ where pixel-aligned features (256 channels) and spatial features (96 channels) are computed independently before concatenation for occupancy prediction, 2D feature maps (~198 channels) are constructed from the fitted SMPL-X model which are concatenated with the input image (3 channels) and the estimated front and back normal maps (6 channels), and fed to the encoder of PIFu. Note that only 3 out of ~207 channels in the input come from the RGB image. This suggests the occupancy prediction may heavily depend on the fitted SMPL-X model. There is no analysis on any possible bias on the source of input.\n- Meshes with extreme self-occlusion are excluded in training. This reduces the number of usable meshes in THuman2.0 from 526 to 362. With reduced training data, the performance of the models (both the proposed model and existing models for comparison) will be affected (to different extends), particularly in handling self-occlusion.\n- For each mesh in THuman2.0, only 10 RGB images at different yaw angles have been used in training. Previous works typically use 360 images around each models for training. This difference in training setting makes it not possible to directly/easily compare the quality of models with previous works. For instance, the results of ARCH++ presented in this paper look much worse than in the original paper.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept