pwc-archive/papers-with-abstracts huggingface dataset:
Learning to Estimate the Body Shape Under Clothing from a Single 3D Scan
Authors: Pengpeng Hu, Nastaran Nourbakhsh Kaashki, Vasile Dadarlat, Adrian Munteanu
Date: 2020-08-13 00:00:00
Tasks: Virtual Try-on

Estimating the 3D human body shape and pose under clothing is important for many applications, including virtual
try-on, non-contact body measurement, and avatar creation for
virtual reality. Existing body shape estimation methods formulate
this task as an optimization problem by fitting a parametric body
model to a single dressed-human scan or a sequence of dressed human meshes for a better accuracy. This is impractical for many
applications that require fast acquisition such as gaming and
virtual try-on due to the expensive computation. In this paper, we
propose the first learning-based approach to estimate the human
body shape under clothing from a single dressed-human scan,
dubbed Body PointNet. The proposed Body PointNet operates
directly on raw point clouds and predicts the undressed body in
a coarse-to-fine manner. Due to the nature of the data – aligned
paired dressed scans and undressed bodies, and genus-0 manifold
meshes (i.e. single-layer surfaces) – we face a major challenge of
lacking training data. To address this challenge, we propose a
novel method to synthesize the dressed-human pseudo-scans and
corresponding ground truth bodies. A new large-scale dataset,
dubbed BUG (Body Under virtual Garments) is presented,
employed for the learning task of body shape estimation from 3D
dressed-human scans. Comprehensive evaluations show that the
proposed Body PointNet outperforms the state-of-the-art methods
in terms of both accuracy and running time.
Personalized 3D mannequin reconstruction based on 3D scanning
Authors: Pengpeng Hu, Duan Li, Ge Wu, Taku Komura, Dongliang Zhang, Yueqi Zhong
Date: 2018-04-16 00:00:00

Personalized customization is a new manufacturing trend in high-end products (e.g. senior
custom clothing). Traditional apparel customization (made-to-measure & bespoken) highly
depends on experienced tailors. A personalized mannequin is essential for apparel customization
using CAD technologies. Currently, a common method of reconstructing a mannequin is based on
body measurements or body features. It only preserves the body size instead of preserving the
accurate user’s stature. However, the same human body measurement does not equal the
same body shape. This may result in an unfit garment for the user. This paper proposes a novel
scanning-based pipeline to reconstruct a personalized mannequin, which preserves both body
size and body stature information. We first capture the body of a user via 3D scanning, and a
statistical body model is fit to the scanned data. This results in a skinned articulated model of
the user. The scanned body is then adjusted to be pose-symmetric via linear blending skinning.
The updated pose-symmetric body is then segmented to initialize the stature symmetry
processing. Finally, a slice-based method is proposed to generate a symmetric 3D mannequin.
The process of apparel customization can be easily digitalized with the help of the proposed
mannequin reconstruction system and the corresponding existing clothing CAD software.
Automatic three-dimensional-scanned garment fitting based on virtual tailoring and geometric sewing
Authors: Duan Li, Yueqi Zhong, Ge Wu, Pengpeng Hu
Date: 2018-12-19 00:00:00

In this article, we presented a new automatic three-dimensional-scanned garment fitting method for A-Pose-scanned
human models. Both the garment and the human body were decomposed based on feature lines defined by various
landmarks. The patches of the three-dimensional garment were automatically positioned around the human model by
setting up the correspondence via feature matching. Virtual sewing was engaged to obtain the final results of virtual
dressing. The penetration between cloth model and human model was solved by a geometrical method constrained by
Laplacian-based deformation. The experimental results indicated that the proposed method was an efficient way for
redressing various garments onto various human models while maintaining the original geometrical features of garments.
Automatic foot scanning and measurement based on multiple RGB-depth cameras
Authors: Ge Wu, Duan Li, Pengpeng Hu, Yueqi Zhong, Ning Pan
Date: 2016-11-25 00:00:00

In this paper, a system with six depth cameras was built to scan both feet simultaneously. An improved calibration method based on a T-shaped checkerboard was used to calculate the extrinsic parameters of the cameras. T-shaped virtual checkerboards were introduced to further fine-tune the accuracy of calibration based on the iterative closest point algorithm. Based on the proposed foot scanner, a complete procedure was introduced to measure the foot automatically by locating the anatomical landmarks without manual intervention. Various experiments were presented to validate the performance of the scanner and the measurements. The results verified that the proposed methods were efficient and versatile for three-dimensional foot scanning and measurement.
Scanning and animating characters dressed in multiple-layer garments
Authors: Pengpeng Hu, Taku Komura, Daniel Holden, Yueqi Zhong
Date: 2017-05-09 00:00:00

Despite the development of user-friendly interfaces for modeling garments and putting them onto characters, preparing a character dressed in multiple layers of garments can be very time-consuming and tedious. In this paper, we propose a novel scanning-based solution for modeling and animating characters wearing multiple layers of clothes. This is achieved by making use of real clothes and human bodies. We first scan the naked body of a subject by an RGBD camera, and a statistical body model is fit to the scanned data. This results in a skinned articulated model of the subject. The subject is then asked to put on one piece of garment after another, and the articulated body model dressed up to the previous step is fit to the newly scanned data. The new garment is segmented in a semi-automatic fashion and added as an additional layer to the multi-layer garment model. During runtime, the skinned character is controlled based on the motion capture data and the multi-layer garment model is controlled by blending the movements computed by physical simulation and linear blend skinning, such that the cloth preserves its shape while it shows realistic physical motion. We present results where the character is wearing multiple layers of garments including a shirt, coat and a skirt. Our framework can be useful for preparing and animating dressed characters for computer games and films.
Foot shape prediction using elliptical Fourier analysis
Authors: Ge Wu, Duan Li, Pengpeng Hu, Yueqi Zhong, Ning Pan
Date: 2017-12-17 00:00:00
Tasks: Prediction

In this paper, a new method was proposed to establish the relationship between three-dimensional (3D) foot shapes and their two-dimensional (2D) foot silhouettes, through which a complete 3D foot shape can be predicted by simply inputting its two 2D silhouettes. 3D foot scans of 80 participants were randomly selected as the training set, and those of another 20 participants were used as the testing set. Elliptical Fourier analysis (EFA) and principle component analysis (PCA) were adopted to parameterize the 3D foot shapes. A linear regressive model was then developed to predict the 3D foot shape with the foot silhouettes. Experiment results indicated individual 3D foot shape can be predicted with a mean error between 1.21 and 1.27 mm, which can provide enough accuracy for the fit evaluation of footwear.
A new method to evaluate the dynamic air gap thickness and garment sliding of virtual clothes during walking
Authors: Pengpeng Hu, Edmond SL Ho, Nauman Aslam, Taku Komura, Hubert PH Shum
Date: 2019-12-13 00:00:00

With the development of e-shopping, there is a significant growth in clothing purchases online. However, the virtual
clothing fit evaluation is still under-researched. In the literature, the thickness of the air layer between the human body
and clothes is a dominant geometric indicator to evaluate the clothing fit. However, such an approach has only been
applied to the stationary positions of the mannequin/human body. Physical indicators such as the pressure/tension of a
virtual garment fitted on the virtual body in a continuous motion are also proposed for clothing fit evaluation. Neither
geometric nor physical evaluations consider the interaction of the garment with the body, e.g., the sliding of the garment
along the human body. In this study, a new framework was proposed to automatically determine the dynamic air gap
thickness. First, the dynamic dressed character sequence was simulated in three-dimensional (3D) clothing software via
importing the body parameters, cloth parameters, and a walking motion. Second, a cost function was defined to convert
the garment in the previous frame to the local coordinate of the next frame. The dynamic air gap thickness between
clothes and the human body was determined. Third, a new metric, the 3D garment vector field was proposed to
represent the movement flow of the dynamic virtual garment, whose directional changes are calculated by cosine
similarity. Experimental results show that our method is more sensitive to the small air gap thickness changes compared
with state-of-the-art methods, allowing it to more effectively evaluate clothing fit in a virtual environment.
Automatic body measurement based on slicing loops
Authors: Yueqi Zhong, Duan Li, Ge Wu, Pengpeng Hu
Date: 2016-06-04 00:00:00

Purpose
The automatic body measurement is the key of tailoring, mass customization and fit/ease evaluation. The major challenges include finding the landmarks and extracting the sizes accurately. The purpose of this paper is to propose a new method of body measurement based on the loop structure.
Design/methodology/approach
The scanned human model is sliced equally to layers consist of various shapes of loops. The semantic feature analysis has been regarded as a problem of finding the points of interest (POI) and the loop of interest (LOI) according to the types of loop connections. Methods for determining the basic landmarks have been detailed.
Findings
The experimental results validate that the proposed methods can be used to locate the landmarks and to extract sizes on markless human scans robustly and efficiently.
3D textile reconstruction based on KinectFusion and synthesized texture
Authors: Pengpeng Hu, Taku Komura, Duan Li, Ge Wu, Yueqi Zhong
Date: 2017-11-06 00:00:00

Purpose
The purpose of this paper is to present a novel framework of reconstructing the 3D textile model with synthesized texture.
Design/methodology/approach
First, a pipeline of 3D textile reconstruction based on KinectFusion is proposed to obtain a better 3D model. Second, “DeepTextures” method is applied to generate new textures for various three-dimensional textile models.
Findings
Experimental results show that the proposed method can conveniently reconstruct a three-dimensional textile model with synthesized texture.
A study on improving the calibration of body scanner built on multiple RGB-Depth cameras
Authors: Ge Wu, Duan Li, Yueqi Zhong, Pengpeng Hu
Date: 2017-06-05 00:00:00

Purpose
The calibration is a key but cumbersome process for 3D body scanning using multiple depth cameras. The purpose of this paper is to simplify the calibration process by introducing a new method to calibrate the extrinsic parameters of multiple depth cameras simultaneously.
Design/methodology/approach
An improved method is introduced to enhance the accuracy based on the virtual checkerboards. Laplace coordinates are employed for a point-to-point adjustment to increase the accuracy of scanned data. A system with eight depth cameras is developed for full-body scanning, and the performance of this system is verified by actual results.
Findings
The agreement of measurements between scanned human bodies and the real subjects demonstrates the accuracy of the proposed method. The entire calibration process is automatic.
Two-step registration in 3D human body scanning based on multiple RGB-D sensors
Authors: Pengpeng Hu, Yueqi Zhong, Ge Wu, Duan Li
Date: 2015-12-01 00:00:00

In this paper, we propose a universal registration method for the body scanner consists of multiple fixed sensors. Firstly, a new double-faced checkerboard is designed to apply the calibration method into the initial estimations of the relative poses of point clouds. Secondly, ICP (Iterative Closest Point) algorithm is utilized for further registration. Since the ICP algorithm may fail in situations of outliers, ambiguity, or missing data, we make full use of the geometry feature of creased papers to solve these problems. A new body scanning system with fast capture speed is also proposed to validate our method. The experiment results demonstrate our approach is robust and fast.
MaskLayer: Enabling scalable deep learning solutions by training embedded feature sets
Authors: Remco Royen, Leon Denis, Quentin Bolsee, Pengpeng Hu, Adrian Munteanu
Date: 2021-01-20 00:00:00

Deep learning-based methods have shown to achieve excellent results in a variety of domains, however, some important assets are absent. Quality scalability
is one of them. In this work, we introduce a novel and generic neural network
layer, named MaskLayer. It can be integrated in any feedforward network, allowing quality scalability by design by creating embedded feature sets. These
are obtained by imposing a specific structure of the feature vector during training. To further improve the performance, a masked optimizer and a balancing
gradient rescaling approach are proposed. Our experiments show that the cost
of introducing scalability using MaskLayer remains limited. In order to prove its
generality and applicability, we integrated the proposed techniques in existing,
non-scalable networks for point cloud compression and semantic hashing with
excellent results. To the best of our knowledge, this is the first work presenting a
generic solution able to achieve quality scalable results within the deep learning
framework.
A generic method of wearable items virtual try-on
Authors: Pengpeng Hu, Nastaran Nourbakhsh, Jing Tian, Stephan Sturges, Vasile Dadarlat, Adrian Munteanu
Date: 2020-03-24 00:00:00
Tasks: Virtual Try-on

Virtual try-on synthesizes garments for the target bodies in 2D/3D domains. Even though existing virtual try-on methods focus on redressing garments, the virtual try-on hair, shoes and wearable accessories are still under-reached. In this paper, we present the first general method for virtual try-ons that is fully automatic and suitable for many items including garments, hair, shoes, watches, necklaces, hats, and so on. Starting with the pre-defined wearable items on a reference human body model, an automatic method is proposed to deform the reference body mesh to fit a target body for obtaining dense triangle correspondences. Then, an improved fit metric is used to represent the interaction between wearable items and the body. For the next step, with the help of triangle correspondences and the fit metric, the wearable items can be fast and efficiently inferred by the shape and posture of the targeted body. Extensive experimental results show that, besides automation and efficiency, the proposed method can be easily extended to implement the dynamic try-on by applying rigging and importing motion capture data, being able to handle both tight and loose garments, and even multi-layer clothing.
Multi-modal deep network for RGB-D segmentation of clothes
Authors: Boris Joukovsky, Pengpeng Hu, Adrian Munteanu
Date: 2020-04-30 00:00:00
Tasks: Semantic Segmentation

In this Letter, the authors propose a deep learning based method to perform semantic segmentation of clothes from RGB-D images of people. First, they present a synthetic dataset containing more than 50,000 RGB-D samples of characters in different clothing styles, featuring various poses and environments for a total of nine semantic classes. The proposed data generation pipeline allows for fast production of RGB, depth images and ground-truth label maps. Secondly, a novel multi-modal encoder–ecoder convolutional network is proposed which operates on RGB and depth modalities. Multi-modal features are merged using trained fusion modules which use multi-scale atrous convolutions in the fusion process. The method is numerically evaluated on synthetic data and visually assessed on real-world data. The experiments demonstrate the efficiency of the proposed model over existing methods.
Constrained Relational Topic Models
Authors: Silvia Terragni, Elisabetta Fersini, Enza Messina
Date: 2020-02-01 00:00:00
Tasks: Document Classification, Topic Models

Relational topic models (RTM) have been widely used to discover hidden topics in a collection of networked documents. In this paper, we introduce the class of Constrained Relational Topic Models (CRTM), a semi-supervised extension of RTM that, apart from modeling the structure of the document network, explicitly models some available domain knowledge. We propose two instances of CRTM that incorporate prior knowledge in the form of document constraints. The models smooth the probability distribution of topics such that two constrained documents can either share the same topics or denote distinct themes. Experimental results on benchmark relational datasets show significant performances of CRTM on a semi-supervised document classification task.
DepthComp: real-time depth image completion based on prior semantic scene segmentation
Authors: Amir Atapour-Abarghouei, Toby P Breckon
Date: 2017-09-04 00:00:00
Tasks: Scene Segmentation, Segmentation

We address plausible hole filling in depth images in a computationally lightweight methodology that leverages recent advances in semantic scene segmentation. Firstly, we perform such segmentation over a co-registered color image, commonly available from stereo depth sources, and non-parametrically fill missing depth values based on a multipass basis within each semantically labeled scene object. Within this formulation, we identify a bounded set of explicit completion cases in a grammar inspired context that can be performed effectively and efficiently to provide highly plausible localized depth continuity via a case-specific non-parametric completion approach. Results demonstrate that this approach has complexity and efficiency comparable to conventional interpolation techniques but with accuracy analogous to contemporary depth filling approaches. Furthermore, we show it to be capable of fine depth relief completion beyond that of both contemporary approaches in the field and computationally comparable interpolation strategies.
A Distributed Implementation of Steady-State Kalman Filter
Authors: Jiaqi Yan, Xu Yang, Yilin Mo, Keyou You
Date: 2021-01-26 00:00:00
Tasks: State Estimation

This paper studies the distributed state estimation in sensor network, where $m$ sensors are deployed to infer the $n$-dimensional state of a linear time-invariant (LTI) Gaussian system. By a lossless decomposition of optimal steady-state Kalman filter, we show that the problem of distributed estimation can be reformulated as synchronization of homogeneous linear systems. Based on such decomposition, a distributed estimator is proposed, where each sensor node runs a local filter using only its own measurement and fuses the local estimate of each node with a consensus algorithm. We show that the average of the estimate from all sensors coincides with the optimal Kalman estimate. Numerical examples are provided in the end to illustrate the performance of the proposed scheme.
RESPER: Computationally Modelling Resisting Strategies in Persuasive Conversations
Authors: Ritam Dutt, Sayan Sinha, Rishabh Joshi, Surya Shekhar Chakraborty, Meredith Riggs, Xinru Yan, Haogang Bao, Carolyn Penstein Rosé
Date: 2021-01-26 00:00:00
Proceeding: EACL 2021 2
Tasks: Persuasion Strategies

Modelling persuasion strategies as predictors of task outcome has several real-world applications and has received considerable attention from the computational linguistics community. However, previous research has failed to account for the resisting strategies employed by an individual to foil such persuasion attempts. Grounded in prior literature in cognitive and social psychology, we propose a generalised framework for identifying resisting strategies in persuasive conversations. We instantiate our framework on two distinct datasets comprising persuasion and negotiation conversations. We also leverage a hierarchical sequence-labelling neural architecture to infer the aforementioned resisting strategies automatically. Our experiments reveal the asymmetry of power roles in non-collaborative goal-directed conversations and the benefits accrued from incorporating resisting strategies on the final conversation outcome. We also investigate the role of different resisting strategies on the conversation outcome and glean insights that corroborate with past findings. We also make the code and the dataset of this work publicly available at https://github.com/americast/resper.
Domain-Adaptive Object Detection via Uncertainty-Aware Distribution Alignment
Authors: Dang-Khoa Nguyen, Wei-Lun Tseng, Hong-Han Shuai
Date: 2020-10-31 00:00:00
Tasks: Domain Adaptation, Object, object-detection, Object Detection, Unsupervised Domain Adaptation, Weakly Supervised Object Detection

Domain adaptation aims to transfer knowledge from the sourcedata with annotations to scarcely-labeled data in the target domain,which has attracted a lot of attention in recent years and facilitatedmany multimedia applications. Recent approaches have shown theeffectiveness of using adversarial learning to reduce the distribu-tion discrepancy between the source and target images by aligningdistribution between source and target images at both image and in-stance levels. However, this remains challenging since two domainsmay have distinct background scenes and different objects. More-over, complex combinations of objects and a variety of image stylesdeteriorate the unsupervised cross-domain distribution alignment.To address these challenges, in this paper, we design an end-to-endapproach for unsupervised domain adaptation of object detector.Specifically, we propose a Multi-level Entropy Attention Alignment(MEAA) method that consists of two main components: (1) LocalUncertainty Attentional Alignment (LUAA) module to acceleratethe model better perceiving structure-invariant objects of interestby utilizing information theory to measure the uncertainty of eachlocal region via the entropy of the pixel-wise domain classifierand (2) Multi-level Uncertainty-Aware Context Alignment (MUCA)module to enrich domain-invariant information of relevant objectsbased on the entropy of multi-level domain classifiers. The proposedMEAA is evaluated in four domain-shift object detection scenarios.Experiment results demonstrate state-of-the-art performance onthree challenging scenarios and competitive performance on onebenchmark dataset.