Towards Hard-pose Virtual Try-on via 3D-aware Global Correspondence Learning
Anonymous Author(s)
AffiliationAddress email
Abstract
In this paper, we target image-based person-to-person virtual try-on in the presenceof diverse poses and large viewpoint variations.
Existing methods are restrictedin this setting as they estimate garment warping flows mainly based on 2D posesand appearance, which omits the geometric prior of the 3D human body shape.
Moreover, current garment warping methods are confined to localized regions,which makes them ineffective in capturing long-range dependencies and results ininferior flows with artifacts.
To tackle these issues, we present 3D-aware globalcorrespondences, which are reliable flows that jointly encode global semantic correlations, local deformations, and geometric priors of 3D human bodies.
Particularly,given an image pair depicting the source and target person, (a) we first obtain their pose-aware and high-level representations via two encoders, and introduce a coarse-to-fine decoder with multiple refinement modules to predict the pixel-wiseglobal correspondence.
(b) 3D parametric human models inferred from imagesare incorporated as priors to regularize the correspondence refinement process sothat our flows can be 3D-aware and better handle variations of pose and viewpoint.(c)
Finally, an adversarial generator takes the garment warped by the 3D-awareflow, and the image of the target person as inputs, to synthesize the photo-realistictry-on result.
Extensive experiments on public benchmarks and our HardPosetest set demonstrate the superiority of our method against state-of-the-art try-onapproaches.
Introduction
Current image-based virtual try-on (VTON) methods typically include two key modules, namely awarping module that transfers a given garment to the desired pose and a blending module/generatorthat synthesizes the try-on result given the target person and the warped garment.
To facilitatehigh-quality VTON, it is therefore crucial to accurately preserve garment details (e g., color, texture,shape, pattern) in the warping stage.
This, however, is a difficult problem which most existingmethods try to address by relying on thin plate splines (TPS) [1] or appearance flows [2].
While TPS is a well-studied and efficient method that performs the warping based on a set of controlpoints and has a closed-form solution, it is prone to inaccurate transformations under large geometricdeformations [3].
To further improve the warping quality, there has been a shift towards directlyestimating appearance flow fields by leveraging the capacity of deep neural networks.
These methodsperform well in relatively easy try-on scenarios, such as transferring flat, in-shop garments with purecolors or simple patterns to desired poses, but still suffer from severe artifacts in unconstrained sceneslike person-to-person VTON, where varying poses and viewpoints commonly exist.
These artifacts are caused by the following two reasons: First, due to the lack of 3D geometric guidance in current methods, flows estimated merely based on appearance and 2D poses are unable
Pose
[Figure]
Appearance
Try-on
Pose & Appearance
Try-on
Pose & Appearance
Try-on
Pose & Appearance
Try-on to represent the deformable, non-rigid characteristics of garments, which results in texture distortionsand improper warping.
Second, for the purpose of efficiency, mainstream flow-based networks areusually constructed by stacking modules with limited receptive fields, e.g., ClothFlow [4] simplyuses a convolutional layer to predict its initial cloth flow.
As the effective area of a network modulemay be a small fraction of its theoretical receptive field [5], flows estimated by existing methods arede facto restricted to local areas and cannot leverage long-range dependencies effectively to tacklepose variations and occlusions.
Visual correspondences [6, 7] that depict cross-instance/domain semantic mappings could be apotential alternative to appearance flows for tackling pose and viewpoint variations, because theyexploit correlations among high-level concepts, instead of appearance-oriented pixel-wise offsets.
However, without extra guidance/regularization we observe that these approaches tend to focuson large and apparent structures (e.g., human pose) while ignoring subtle details and thus have adetrimental effect on the VTON user experience.
Moreover, the complexity of calculating highresolution correspondences directly is expensive ( O ( N 2 ) , where N is the resolution), which furtherhinders its subsequent processes and downstream applications.
A feasible solution is to leverage 3D human models (e.g. SMPL [8]) to generate 3D-aware correspondences that are less ambiguous.
However, a naive implementation that simply replaces the 2D poserepresentation in a network with its 3D counterpart is ineffective.
This is because in our hard-pose
VTON task, these 3D human models fail to predict the body mesh correctly and inevitably introducenoise to the estimated flow.
Besides, employing 3D human models will lead to considerably lowerinference speed [9].
Therefore, we propose a 3D-aware global correspondence learning (3D-GCL) method to tackle theaforementioned issues.
The key idea of our 3D-GCL method is to leverage geometric priors of 3Dhuman bodies to guide the correspondences, so that modelling non-rigid deformations of garmentscan be more effective.
Unlike previous methods that incorporate 3D information into their inputsexplicitly, we consider the implicit supervision of 3D priors in our correspondence learning process .
In this way, not only our correspondences can be 3D-aware, but also the side effects of improper 3Dinformation inferred from images can be alleviated.
Specifically, given a source image and a targetimage, we first obtain their high-level and pose-aware feature maps via two parallel encoder networks,as well as the global correspondence via computing the correlation between the feature maps.
Afterthat, multiple geometry-aware correspondence refinement modules (GACRMs) are introduced toupsample and refine the global correspondence stage-by-stage, thereby efficiently constructing thefinal full-resolution correspondences.
More importantly, for each of the GACRMs, an elaboratelydesigned 3D regularization loss is adopted to provide the GACRMs with the ability to leverage 3Dgeometric priors.
At last, we warp the garment with the final correspondence, combine it with theimage of the target person, and synthesize the try-on result via an adversarial generator.
Experimentson public benchmarks and our manually selected HardPose test set demonstrate that our 3D-GCL network synthesizes photo-realistic virtual try-on results that accurately preserve garment structuresand textures even for hard-pose cases.
Our contribution can be summarize as follows:
• We elaborately design a novel 3D-aware global correspondence learning framework for virtualtry-on in the presence of diverse pose variations and viewpoint changes.
• We present a geometry-aware correspondence refinement module and a 3D correspondence regularization loss, which together facilitate the understanding of geometric information and ensure thepreservation of garment textures in our network.• We conduct extensive experiments to validate the superiority of our method against state-of-the-artapproaches, in terms of both garment warping and try-on synthesis.
Related Work
Virtual Try-on The most challenging aspect of VTON is the preservation of diverse textures andlogos, which is addressed in current approaches by heavily relying on warping modules.
One populartechnique to warp the in-shop garment that has been employed by several early approaches [10, 11]is to utilize the TPS [1].
However, while these models can maintain garment details, they tendto introduce undesired distortions and artifacts under large pose deformations.
To alleviate theseartifacts, extra constraints and visual cues have been exploited [12, 13] to refine the control points ofthe TPS.
However, these approaches are still limited by the sparse control-point-based representationof the TPS.
Hence, there has been a shift to utilize networks to predict dense flow fields directly[4, 13, 14], which has considerably improved in-shop VTON performance.
Nevertheless, as we haveemphasized, existing methods are unable to precisely represent long-range and local deformationsvia a single flow, and hence they are restricted in the hard-pose VTON task.3D-aware Synthesis Models Recent advances in 3D pose estimation and reconstruction (e.g.,[8, 15, 16]) facilitate the use of 3D human models in synthesis methods [3, 9, 17, 18, 19].
Despitethe effectiveness of 3D human models in modeling local dependencies, current 3D-aware synthesismodels struggle to predict long-range deformations due to their unstable gradient propagation.
Thismight be caused by inaccurate 3D model fitting, intertwined flow and feature gradients, and the useof bilinear sampling [20].
Contrary to existing methods, we first predict a global correspondence, andthen augment it via the regularization provided by 3D human models.
Hence we can circumvent thedifficulty of learning long-range flows merely based on 3D human models.
Correspondence Learning Several cross-domain image synthesis methods based on correspondencelearning [7, 18, 21, 22] have been proposed recently, among which [18] adopts the same poserepresentation as in the proposed method.
However, its learned flows are restricted to a predefined UVspace, which limits its scalability and performance.
The proposed method instead does not includesuch restrictions, as its correspondences refined by 3D human models are robust to various poses andbody shapes.
3D-aware Global Correspondence Learning
Our goal is to overcome the limitations of current try-on methods in handling large pose and viewpointvariations.
This requires us to estimate flows that 1) can model long-range correspondences to tackleocclusions, rare poses, and novel views; and 2) can preserve textures and local structures of garmentsto ensure accurate try-on results.
To this end, given a pair of person images, our proposed 3D-GCLnetwork (Sec. 3.1) first estimates a global flow via the correlation between their high-level 3Dpose-aware features (Sec. 3.2), and then adopts multiple GACRMs to guide the global flow indepicting local deformations (Sec. 3.3).
Our training objectives are provided in Sec. 3.4.
3.1 Network Architecture
Given an image I s of the source person wearing a garment G and a target person image I t , our task isto generate a photo-realistic try-on image where the target person is wearing G .
As shown in Fig.
2,our 3D-GCL network tackles this task in a two-stage manner:
In the first stage, we employ an efficientencoder-decoder architecture f
( I s , I t ) to estimate the 3D-aware global correspondence for warping
Source Person
IUV Clothes
IUV Target Person
Encoder
Encoder …
Feature Pyramid …
Correlation
Flow
GACRM
Flow
GACRM …
Output Flow
Warping
Generator
Try-on Output
Stage-I: 3D-aware Global Correspondence Estimation
Stage-II: Try-on Synthesis
Feature Pyramid x L x 1 x y L y 1 y
G , i.e., f [Equation]
2 , where H and W denote the image height and width.
In the second stage, we adopt a conditional generator g ( I t | G ′ ) to synthesize the final try-on result I ′ t ,where G ′ ∈
R H × W × 3 is the warped garment.
Choices for realizing g are flexible, and we simplyemploy a modified StyleGAN2 [23], where we replace the original style normalization strategy withspatially-adaptive normalization [24] to better maintain the spatial structures in garments.
The detailsof our network architecture are provided in the supplementary material.
3.2 Pose-aware Global Correspondence Estimation
Without the proper guidance of 3D pose information, existing appearance flows often fail to distinguishbody parts (e.g., torso and arms) under heavy occlusions and pose variations.
Moreover, full-resolutionvisual correspondences are computationally expensive.
Therefore, we propose to exploit featuresthat are pose-aware and compact for correlation learning, so that these two issues can be tackledsimultaneously.
Specifically, we first employ DensePose [15] on I s
and I t to obtain their corresponding IUV maps
P s , P t ∈
R
H
× W × 3 , which are the pixel-wise estimations of a 3D parametric human body (i.e., SMPL[8]) projected on images.
In contrast to previous appearance flow methods [2, 4], we then extractfeatures directly from P s and P t rather than from the input images or segmentation masks.
1 Thisis reasonable, as the I channel and the UV channels of P s and P t have encoded semantic parts andpositions of the 3D human body, respectively.
Furthermore, this allows our network to disentangleirrelevant appearance information (e.g., faces and skin tones) and focus on pose and shape variations.
Following the same motivation, we also leverage a human parser [25] to obtain the segmentation ofthe target garment G .
Two parallel fully convolutional encoders are adopted for feature extraction,i.e., a source branch that takes the concatenation of P s and G as input and a target branch withinput P t .
Let x , y ∈ R N × C denote the output features of the source branch and the target branch,where N = HS × WS is the scaled spatial resolution and C is the number of feature channels.
S = 8throughout this paper, so that N ≪ H × W , which ensures that the estimation of the correspondenceis computationally efficient.
We calculate the correlation between x and y as follows:
[Equation]
where o ij , 1 ≤ i, j ≤ N measures the semantic and pose similarity between the i -th feature vectorin x and the j -th feature vector in y .
As o contains the similarities between all possible spatialcombinations of feature vectors in x and y , it is able to capture long-range variations.
Hence we
[Figure]
consider as down section,
Comparison with other correlation based methods .
Several recently proposed methods [7, 22] alsoconsider feature correlation for correspondence estimation.
However, they target image translationacross different domains (e.g., from sketches to photos) while our focus is on estimating globalcorrespondences to warp garments (from poses to poses).
Besides, they rely on a 2D keypoint-basedrepresentation of poses [27] that is sparse and cannot represent body shapes, while we adopt the 3Ddense representation of poses that leads to more robust estimated correspondences.
More importantly,our estimated global correspondences serve as the base flows and are refined by GACRMs to tackleboth long-range and local pose variations.
3.3 Geometry-aware Correspondence Refinement
A potential issue of correlation based global appearances is that they ignore the spatial structures offeatures (as in Eq.
Consequently, it is hard to ensure local consistency in the global appearances.
Even worse, the correlation between high-level semantic features tends to generate over-smooth flowsthat cannot preserve garment details such as patterns and logos.
Therefore, we present the GACRMand a 3D regularization loss to leverage the geometric prior of 3D human bodies and tackle theseissues.
To begin with, we convert o into a flow f 0 of size HS × WS via applying the argmax function.
Toincorporate local details into f 0 , we exploit the early-stage feature pyramids in both the source branchand the target branch.
Assume both feature pyramids consist of L layers (as shown in Fig. 2), weutilize a GACRM for each layer, which can be formulated as follows:
[Equation]
where 0 ≤ l < L is the layer index.
x l and y l denote feature maps of the l -th layer of the source andtarget feature pyramids and f ul is the up-scaled flow from the previous layer.
h is responsible forpredicting a residual flow to enhance f ul , which is realized simply by a convolutional gated recurrentunit [22] (ConvGRU) taking x l warped by f ul and y l as inputs, as shown in Fig.
To incorporate the geometric prior of 3D human bodies into our correspondence refinement process,we propose a novel 3D regularization loss for the global correspondence and its corresponding refinedflows.
Specifically, we adopt an off-the-shelf 3D mesh regressor [28] to obtain the SMPL models ofthe source person and the target person.
Following [9], we render the two SMPL models to obtain avisibility mask, and calculate a SMPL flow measuring the disparity between the source person meshand the target person mesh projected on the 2D image plane.
The SMPL flow is utilized as the pseudoground-truth in our 3D regularization loss, which is defined as follows:
[Equation]
where L o and L f are loss terms for the global correspondence and the refined flows.
In the remainderof this section, for the conciseness of notation and presentation, we assume that all related variablesin the 3D regularization loss are resized accordingly.
To define L o , note that it is common practiceto conduct interpolations (e.g., bilinear sampling) in flow based warping, which are similar to theaforementioned coarse warping based on the global correspondence.
Therefore, we generate apseudo ground-truth correspondence by scattering the interpolation weights of each pixel to itscorresponding position in the correlation matrix, making the image warped by our pseudo groundtruth correspondence visually close to its counterpart warped by the SMPL flow.
Formally, we obtainthe ground-truth correspondence o gt ∈ R N × N as follows:
[Equation]
where Ω( i ) denotes the reference points given i as the target interpolation point, w j is the interpolationweight of the reference point j , and G gt denotes the ground-truth warped garment.
We then define
L o as follows:
[Equation]
where M ∈ { 0 , 1 } N × N is a binary mask with M ij =
[ o gtij > 0] .
[ · ] and ⊙ denote the Iverson bracketand element-wise product, respectively.
For L f , we consider the l 1 loss between the refined flowsand the SMPL flow as follows:
[Equation]
where V ∈ { 0 , 1 } H × W and f gt denote the visibility mask and the SMPL flow, respectively.
The above 3D regularization loss encourages our network to predict flows that agree well withgeometric deformations of 3D human bodies.
This avoids the need of introducing complicatedmodules for imposing local constraints on flows, such as PatchMatch [22], local attention [20], ormulti-stage correlations [13].
Consequently, we can adopt an elegant and efficient architecture for the
GACRM, while obtaining pixel-wise and precise global correspondences.
3.4 Objective Functions
Since the 3D-GCL network consists of two stages, we train its correspondence estimation subnetworkin the first stage and the generator for try-on synthesis in the second stage separately.
In addition tothe proposed 3D regularization loss L r , we follow [18, 23] and use the l 1 loss L l 1 , the perceptual loss
L perc , and the adversarial loss L adv for training.
The total loss for the correspondence estimationstage L c is therefore defined as follows:
[Equation]
where λ r , λ cl 1 , and λ cperc are weighting hyperparameters.
L cl 1 and L cperc are applied to the garmentswarped by f 0 , ..., f L .
The total loss for the generator L g is defined as follows:
[Equation]
where λ adv , λ gl 1 , and λ gperc are hyperparameters.
Experiments
In this section, we conduct extensive experiments to validate the effectiveness of the proposed 3DGCL network.
Due to the page limitation, the interested reader is refer to the supplemental materialfor more experimental settings and results.
Datasets Our experiments are conducted on two open-source datasets, DeepFashion [29] and
MPV [30], which contain 52,712 and 37,723 fashion images, respectively.
To ensure a fair comparison, we follow the train/test split of [18, 20, 31] on the DeepFashion dataset and use the original splitof MPV.
Additionally, pairs without properly detected IUV maps or SMPL models are filtered out.
In Garment [Figure]
Person
Garment wFlow
Dior
PBAFN
PWS
Ours
Person
Garment wFlow
Dior
PBAFN
PWS
Ours the test phase, query persons on the test list are further shuffled to match the task setting of VTON.
Inthis way, we get 101,622/8,564 train/test pairs for DeepFashion and 52,236/10,544 train/test pairs for
MPV.
Furthermore, as this paper targets hard-pose VTON, we manually select test image pairs from
DeepFashion and MPV to construct the HardPose test set.
The HardPose test set consists of 2,406image pairs, where each test pair contains diverse pose changes or large viewpoint variations.
Evaluation Metrics We evaluate the performance of both garment warping and final try-on synthesis.
For the warping stage, we apply LPIPS [32] to evaluate the similarity between the warped garmentsand the corresponding ground-truths extracted by the human parser [25].
We also adopt the mean
Intersection over Union (mIoU) metric to evaluate the semantic-correctness of the estimated flows bycomparing the segmentation maps of the warped garments with their ground-truths.
Furthermore,following [12, 13, 19, 33], we perform a human evaluation by inviting 40 volunteers to select thewarped clothes with the best visual quality among our method and the baseline approaches.
For the try-on stage, note that ground-truth try-on results are not available.
Thus we select the widelyused unpaired Fr ´e chet Inception Distance (FID) to quantify the quality of the synthesis results.
Wealso perform a human evaluation study following recent virtual try-on methods [12, 13, 19].
40volunteers are invited to fill in a questionnaire, in which they are presented with the try-on results ofall methods and are asked to select the most realistic one.
Implementation Details
The proposed 3D-GCL network is implemented in PyTorch and trainedwith 4 Tesla V100 GPUs.
We first train the correspondence estimation subnetwork for 20 epochswith a batch-size of 8, and then follow the settings of [18, 23] to train the try-on generator.
Baselines We compare the warping and try-on performance of our model with publicly-availablestate-of-the-art methods.
For the warping stage, we leverage the FlowNet backbone [26] for allmethods and consider the warping models of AFWM [13], CCM [18], and wFlow [19] as the baselinemethods.
For the evaluation on VTON, we select four cutting-edge methods, including PBAFN [13],
Dior [34], Pose-with-Style [18], and wFlow [19]. [Table] [Table]
4.1 Qualitative results
[Table] As demonstrated in Fig. 4, conventional warping strategies adopted in PBAFN [13] and Dior [34] failto capture correct correspondences in images.
Although they employ an extra smoothness constraint[13], their warped garments are often violently distorted to fill the target silhouette facing complexposes and view-point changes.
Moreover, they also ignore 3D information such as orientations andbody geometry, and thus harm the visual quality of their warped garments.
The hybrid approaches
Pose-with-Style [18] and wFlow [19] struggle to fuse different flow-based warping methods ina harmonious way.
wFlow produces considerably blurry results since its composition masks areunreliable in low confidence regions.
Pose-with-Style on the other hand only warps pixels in thepre-defined UV space of DensePose, which limits its application for diverse garment types such asskirts and dresses.
On contrary, our method produces favourable results even in challenging scenarioswith large pose and viewpoint changes.
Note that with the help of global correspondence learning,our method has a larger receptive field and is able to also generate more compelling results in frontview transformations.
Fig.
5 illustrates the visual comparison of our try-on results with the baselinemethods.
As mentioned in Section 1, the try-on results are significantly affected by the quality of thedeformed garments.
Our method, with more accurate and harmonious estimated correspondences,outperforms the other four baseline methods in visual quality considerably.
4.2 Quantitative results
The quantitative comparison between our correspondence estimation method against other baselines isreported in Table 1.
Our method outperforms existing advanced warping methods such as CoCosNetv2 [22] and CCM [18], and achieves comparable performance to wFlow [19] on the LPIPS metric.
Note that AFWM [13] obtains considerably lower LPIPS scores.
This is mainly because AFWMadopts the expensive multi-stage feature correlations and encourages extremely smooth flows.
Yetsuch a strategy tends to ignore the spatial and semantic structures in garments, and consequentlyyields inferior visual quality in complex scenarios.
To validate this, we also measure the mIoU metricand assess the human evaluation (HE) score.
As reported in Table 1, our method obtains the highestmIoU scores, indicating that our warped garments are more semantically accurate.
Our method alsoobtains the highest human evaluation percentage, which indicates that it produces the best visualquality for warped garments under large viewpoint changes and pose variations.
As shown in Table 2, our method also outperforms other state-of-the-art VTON methods persistentlyacross different test sets.
Our method obtains the lowest FID scores, which indicates that our methodcan produce high-fidelity try-on results.
This is also validated through the human evaluation.
Note thatmost of the presented methods perform better on the MPV dataset than on the DeepFashion dataset. This is because images on the MPV dataset contain less pose variations and viewpoint changes. Onthe DeepFashion dataset, our method still surpasses the second best method (Pose-with-Style) by aconsiderable margin, which suggests that our method can generalize well in complex scenarios.
Person
Garment 3D-GCL ∗ 3D-GCL † 3D-GCL ⋆ 3D-GCL
Person
Garment 3D-GCL ∗ 3D-GCL † 3D-GCL ⋆ 3D-GCL
[Table]
4.3 Ablation Study
To validate the effectiveness of the global correspondence estimation strategy and the 3D regularization loss, we implement three variants of the proposed method and evaluate them on DeepFashion.
The experimental results of this ablation study is reported in Table 3.
Effect of Global Correspondence Compared with the vanilla variant of the proposed 3D-GCLnetwork (3D-GCL ∗ ), adopting global correspondences (3D-GCL † ) provides us with the performancegains persistent on two metrics.
This is expected, as global correspondences are effective in capturinglong-range deformations.
Effect of 3D Regularization Loss Compared with global correspondences, the advantages of the 3Dregularization loss (3D-GCL ⋆ ) are even more significant, especially according to the mIoU metric.
As discussed in Sec. 3.3, correspondence learning only matches pose features semantically, whichmeans that, on their own, they are unable to preserve fine-grained structures and textures.
This resultsuggests that the 3D regularization loss leveraging SMPL flows to provide geometric guidance, is anecessary and effective component of our network.
Finally, our full model obtains the best performance compared with other variants, which furthersupports that flows encoding both long-range and local deformations are critical in producingcompelling try-on results in challenging scenarios.
The visual comparison of different variants of ourmethod is shown in Fig.
Conclusion
In this paper, we propose a 3D-aware global correspondence learning (3D-GCL) network to tacklehard-pose person-to-person visual try-on.
Our 3D-GCL network adopts an efficient architecturefor global correspondence estimation and refinement, and introduces a novel 3D regularizationloss to exploit geometric priors of 3D human bodies to guide the refinement process of the globalcorrespondences.
In this way, our full-resolution 3D-aware global correspondences can representboth long-range and local deformations precisely, and hence our method can handle large poseand viewpoint variations, while preserving garment textures and structures effectively.
Quantitativeand qualitative comparison validate the effectiveness of the proposed 3D-GCL network.
Due to itseffectiveness and flexibility, the proposed 3D-GCL network constitutes a new baseline for hard-posetry-on, and more advanced components, like attention models, can be integrated into the network inthe future for further improvements.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] We have discussed the limitations
in the supplementary material.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] We have
discussed the potential negative societal impacts in the supplementary material.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] Our code and data are provided in the supplementary material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes] All training details are specified in the supplementary material.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[N/A]
We fix the random seed so that our model runs stably
and our results are reproducible.
(d) Did you include the total amount of compute and the type of resources used (e.g., type
of GPUs, internal cluster, or cloud provider)?
[Yes] Computational resources used are
included in the supplementary material.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[Yes] All assets used in our work are
publicly released under an MIT License or a license restricted to non-commercialresearch and educational purposes.
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
We have constructed the HardPose subset from DeepFashion and MVP, and we include
the list of this subset in the supplemental material.
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[Yes] The data and models used in this work are publicly released.
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[Yes] Faces in our visual examples are masked out,
therefore our data do not contains identifiable information or offensive content.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots,
[Table]
if applicable?
[Yes] The instructions given to the participants are provided in the
[Table]
supplementary material.
[Table]
(b) Did you describe any potential participant risks, with links to Institutional Review
[Table]
Board (IRB) approvals, if applicable?
[N/A]
[Table]
(c) Did you include the estimated hourly wage paid to participants and the total amount
[Table]
spent on participant compensation?
[N/A]
The participants consist of volunteers.
[Table]