bluuebunny/crossref_metadata_2025_split huggingface dataset:
Voronoi Filters for Simulation Enrichment
Authors: Casafranca Juan J. , Otaduy Miguel A. 
Published: March 2023

AbstractThe simulation of complex deformation problems often requires enrichment techniques that introduce local high‐resolution detail on a generally coarse discretization. The use cases include spatial or temporal refinement of the discretization, the simulation of composite materials with phenomena occurring at different scales, or even codimensional simulation. We present an efficient simulation enrichment method for both local refinement of the discretization and codimensional effects. We dub our method Voronoi filters, as it combines two key computational elements. One is the use of kinematic filters to constrain coarse and fine deformations, and thus provide enrichment functions that are complementary to the coarse deformation. The other one is the use of a centroidal Voronoi discretization for the design of the enrichment functions, which adds high‐resolution detail in a compact manner while preserving the rigid modes of coarse deformation. We demonstrate our method on simulation examples of composite materials, hybrid triangle‐based and yarn‐level simulation of cloth, or enrichment of flesh simulation with high‐resolution detail.
Differentiable Simulation for Outcome‐Driven Orthognathic Surgery Planning
Authors: Dorda D. , Peter D. , Borer D. , Huber N.B. , Sailer I. , Gross M. , Solenthaler B. , Thomaszewski B. 
Published: March 2023

AbstractAlgorithms at the intersection of computer graphics and medicine have recently gained renewed attention. A particular interest are methods for virtual surgery planning (VSP), where treatment parameters must be carefully chosen to achieve a desired treatment outcome. FEM simulators can verify the treatment parameters by comparing a predicted outcome to the desired one. However, estimating the optimal parameters amounts to solving a challenging inverse problem. In current clinical practice it is solved manually by surgeons, who rely on their experience and intuition to iteratively refine the parameters, verifying them with simulated predictions.We prototype a differentiable FEM simulator and explore how it can enhance and simplify treatment planning, which is ultimately necessary to integrate simulation‐based VSP tools into a clinical workflow. Specifically, we define a parametric treatment model based on surgeon input, and with analytically derived simulation gradients we optimise it against an objective defined on the visible facial 3D surface. By using sensitivity analysis, we can easily explore the solution‐space with first‐order approximations, which allow the surgeon to interactively visualise the effect of parameter variations on a given treatment plan. The objective function allows landmarks to be freely chosen, accommodating the multiple methodologies in clinical planning. We show that even with a very sparse set of guiding landmarks, our simulator robustly converges to a feasible post‐treatment shape.
A Fusion Crowd Simulation Method: Integrating Data with Dynamics, Personality with Common
Authors: Mao Tianlu , Wang Ji , Meng Ruoyu , Yan Qinyuan , Liu Shaohua , Wang Zhaoqi 
Published: March 2023

AbstractThis paper proposes a novel crowd simulation method which integrates not only modelling ideas but also advantages from both data‐driven methods and crowd dynamics methods. To seamlessly integrate these two different modelling ideas, first, a fusion crowd motion model is developed. In this model the motion of crowd are driven dynamically by different forces. Part of the forces are modeled under a universal interaction mechanism, which describe the common parts of crowd dynamics. Others are modeled by examples from real data, which describe the personality parts of the agent motion. Second, a construction method for example dataset is proposed to support the fusion model. In the dataset, crowd trajectories captured in the real world are decomposed and re‐described under the structure of the fusion model. Thus, personality parts hidden in the real data could be locked and extracted, making the data understandable and migratable for our fusion model. A comprehensive crowd motion generation workflow using the fusion model and example dataset is also proposed. Quantitative and qualitative experiments and user studies are conducted. Results show that the proposed fusion crowd simulation method can generate crowd motion with the great motion fidelity, which not only match the macro characteristics of real data, but also has lots of micro personality showing the diversity of crowd motion.
Local Scale Adaptation to Hand Shape Model for Accurate and Robust Hand Tracking
Authors: Kalshetti P. , Chaudhuri P. 
Published: March 2023

AbstractThe accuracy of hand tracking algorithms depends on how closely the geometry of the mesh model resembles the user's hand shape. Most existing methods rely on a learned shape space model; however, this fails to generalize to unseen hand shapes with significant deviations from the training set. We introduce local scale adaptation to augment this data‐driven shape model and thus enable modeling hands of substantially different sizes. We also present a framework to calibrate our proposed hand shape model by registering it to depth data and achieve accurate and robust tracking. We demonstrate the capability of our proposed adaptive shape model over the most widely used existing hand model by registering it to subjects from different demographics. We also validate the accuracy and robustness of our tracking framework on challenging public hand datasets where we improve over state‐of‐the‐art methods. Our adaptive hand shape model and tracking framework offer a significant boost towards generalizing the accuracy of hand tracking.
Tiled Characteristic Maps for Tracking Detailed Liquid Surfaces
Authors: Narita F. , Ando R. 
Published: March 2023

AbstractWe introduce tiled characteristic maps for level set method that accurately preserves both thin sheets and sharp edges over a long period of time. Instead of resorting to high‐order differential schemes, we utilize the characteristics mapping method to minimize numerical diffusion induced by advection. We find that although a single characteristic map could be used to better preserve detailed geometry, it suffers from frequent global re‐initialization due to the strong distortions that are locally generated. We show that when multiple localized tiled characteristic maps are used, this limitation is constrained only within tiles; enabling long‐term preservation of detailed structures where little distortion is observed. When applied to liquid simulation, we demonstrate that at a reasonably amount of added computational cost, our method retains small‐scale high‐fidelity (e.g., splashes and waves) that is quickly smeared out or deleted with purely grid‐based or particle level set methods.
Voice2Face: Audio‐driven Facial and Tongue Rig Animations with cVAEs
Authors: Villanueva Aylagas Monica , Anadon Leon Hector , Teye Mattias , Tollmar Konrad 
Published: March 2023

AbstractWe present Voice2Face: a Deep Learning model that generates face and tongue animations directly from recorded speech. Our approach consists of two steps: a conditional Variational Autoencoder generates mesh animations from speech, while a separate module maps the animations to rig controller space. Our contributions include an automated method for speech style control, a method to train a model with data from multiple quality levels, and a method for animating the tongue. Unlike previous works, our model generates animations without speaker‐dependent characteristics while allowing speech style control. We demonstrate through a user study that Voice2Face significantly outperforms a comparative state‐of‐the‐art model in terms of perceived animation quality, and our quantitative evaluation suggests that Voice2Face yields more accurate lip closure in speech with bilabials through our speech style optimization. Both evaluations also show that our data quality conditioning scheme outperforms both an unconditioned model and a model trained with a smaller high‐quality dataset. Finally, the user study shows a preference for animations including tongue. Results from our model can be seen athttps://go.ea.com/voice2face.
Learning Physics with a Hierarchical Graph Network
Authors: Chentanez Nuttapong , Jeschke Stefan , Müller Matthias , Macklin Miles 
Published: March 2023

AbstractWe propose a hierarchical graph for learning physics and a novel way to handle obstacles. The finest level of the graph consist of the particles itself. Coarser levels consist of the cells of sparse grids with successively doubling cell sizes covering the volume occupied by the particles. The hierarchical structure allows for the information to propagate at great distance in a single message passing iteration. The novel obstacle handling allows the simulation to be obstacle aware without the need for ghost particles. We train the network to predict effective acceleration produced by multiple sub‐steps of 3D multi‐material material point method (MPM) simulation consisting of water, sand and snow with complex obstacles. Our network produces lower error, trains up to 7.0X faster and inferences up to 11.3X faster than [SGGP*20]. It is also, on average, about 3.7X faster compared to Taichi Elements simulation running on the same hardware in our tests.
PERGAMO: Personalized 3D Garments from Monocular Video
Authors: Casado‐Elvira Andrés , Trinidad Marc Comino , Casas Dan 
Published: March 2023

AbstractClothing plays a fundamental role in digital humans. Current approaches to animate 3D garments are mostly based on realistic physics simulation, however, they typically suffer from two main issues: high computational run‐time cost, which hinders their deployment; and simulation‐to‐real gap, which impedes the synthesis of specific real‐world cloth samples. To circumvent both issues we propose PERGAMO, a data‐driven approach to learn a deformable model for 3D garments from monocular images. To this end, we first introduce a novel method to reconstruct the 3D geometry of garments from a single image, and use it to build a dataset of clothing from monocular videos. We use these 3D reconstructions to train a regression model that accurately predicts how the garment deforms as a function of the underlying body pose. We show that our method is capable of producing garment animations that match the real‐world behavior, and generalizes to unseen body motions extracted from motion capture dataset.
Context‐based style transfer of tokenized gestures
Authors: Kuriyama Shigeru , Mukai Tomohiko , Taketomi Takafumi , Mukasa Tomoyuki 
Published: March 2023

AbstractGestural animations in the amusement or entertainment field often require rich expressions; however, it is still challenging to synthesize characteristic gestures automatically. Although style transfer based on a neural network model is a potential solution, existing methods mainly focus on cyclic motions such as gaits and require re‐training in adding new motion styles. Moreover, their per‐pose transformation cannot consider the time‐dependent features, and therefore motion styles of different periods and timings are difficult to be transferred. This limitation is fatal for the gestural motions requiring complicated time alignment due to the variety of exaggerated or intentionally performed behaviors.This study introduces a context‐based style transfer of gestural motions with neural networks to ensure stable conversion even for exaggerated, dynamically complicated gestures. We present a model based on a vision transformer for transferring gestures' content and style features by time‐segmenting them to compose tokens in a latent space. We extend this model to yield the probability of swapping gestures' tokens for style‐transferring. A transformer model is suited to semantically consistent matching among gesture tokens, owing to the correlation with spoken words. The compact architecture of our network model requires only a small number of parameters and computational costs, which is suitable for real‐time applications with an ordinary device.We introduce loss functions provided by the restoration error of identically and cyclically transferred gesture tokens and the similarity losses of content and style evaluated by splicing features inside the transformer. This design of losses allows unsupervised and zero‐shot learning, by which the scalability for motion data is obtained.We comparatively evaluated our style transfer method, mainly focusing on expressive gestures using our dataset captured for various scenarios and styles by introducing new error metrics tailored for gestures. Our experiment showed the superiority of our method in numerical accuracy and stability of style transfer against the existing methods.
WTFM Layer: An Effective Map Extractor for Unsupervised Shape Correspondence
Authors: Liu Shengjun , Xu Haojun , Yan Dong‐Ming , Hu Ling , Liu Xinru , Li Qinsong 
Published: March 2023

AbstractWe propose a novel unsupervised learning approach for computing correspondences between non‐rigid 3D shapes. The core idea is that we integrate a novel structural constraint into the deep functional map pipeline, a recently dominant learning framework for shape correspondence, via a powerful spectral manifold wavelet transform (SMWT). As SMWT is isometrically invariant operator and can analyze features from multiple frequency bands, we use the multiscale SMWT results of the learned features as function preservation constraints to optimize the functional map by assuming each frequency‐band information of the descriptors should be correspondingly preserved by the functional map. Such a strategy allows extracting significantly more deep feature information than existing approaches which only use the learned descriptors to estimate the functional map. And our formula strongly ensure the isometric properties of the underlying map. We also prove that our computation of the functional map amounts to filtering processes only referring to matrix multiplication. Then, we leverage the alignment errors of intrinsic embedding between shapes as a loss function and solve it in an unsupervised way using the Sinkhorn algorithm. Finally, we utilize DiffusionNet as a feature extractor to ensure that discretization‐resistant and directional shape features are produced. Experiments on multiple challenging datasets prove that our method can achieve state‐of‐the‐art correspondence quality. Furthermore, our method yields significant improvements in robustness to shape discretization and generalization across the different datasets. The source code and trained models will be available at https://github.com/HJ-Xu/WTFM-Layer.
Exploring Contextual Relationships in 3D Cloud Points by Semantic Knowledge Mining
Authors: Chen Lianggangxu , Lu Jiale , Cai Yiqing , Wang Changbo , He Gaoqi 
Published: March 2023

Abstract3D scene graph generation (SGG) aims to predict the class of objects and predicates simultaneously in one 3D point cloud scene with instance segmentation. Since the underlying semantic of 3D point clouds is spatial information, recent ideas of the 3D SGG task usually face difficulties in understanding global contextual semantic relationships and neglect the intrinsic 3D visual structures. To build the global scope of semantic relationships, we first propose two types of Semantic Clue (SC) from entity level and path level, respectively. SC can be extracted from the training set and modeled as the co‐occurrence probability between entities. Then a novel Semantic Clue aware Graph Convolution Network (SC‐GCN) is designed to explicitly model each SC of which the message is passed in their specific neighbor pattern. For constructing the interactions between the 3D visual and semantic modalities, a visual‐language transformer (VLT) module is proposed to jointly learn the correlation between 3D visual features and class label embeddings. Systematic experiments on the 3D semantic scene graph (3DSSG) dataset show that our full method achieves state‐of‐the‐art performance.
SPCNet: Stepwise Point Cloud Completion Network
Authors: Hu Fei , Chen Honghua , Lu Xuequan , Zhu Zhe , Wang Jun , Wang Weiming , Wang Fu Lee , Wei Mingqiang 
Published: March 2023

AbstractHow will you repair a physical object with large missings? You may first recover its global yet coarse shape and stepwise increase its local details. We are motivated to imitate the above physical repair procedure to address the point cloud completion task. We propose a novel stepwise point cloud completion network (SPCNet) for various 3D models with large missings. SPCNet has a hierarchical bottom‐to‐up network architecture. It fulfills shape completion in an iterative manner, which 1) first infers the global feature of the coarse result; 2) then infers the local feature with the aid of global feature; and 3) finally infers the detailed result with the help of local feature and coarse result. Beyond the wisdom of simulating the physical repair, we newly design a cycle loss to enhance the generalization and robustness of SPCNet. Extensive experiments clearly show the superiority of our SPCNet over the state‐of‐the‐art methods on 3D point clouds with large missings. Code is available at https://github.com/1127368546/SPCNet.
Real‐time Deep Radiance Reconstruction from Imperfect Caches
Authors: Huang Tao , Song Yadong , Guo Jie , Tao Chengzhi , Zong Zijing , Fu Xihao , Li Hongshan , Guo Yanwen 
Published: March 2023

AbstractReal‐time global illumination is a highly desirable yet challenging task in computer graphics. Existing works well solving this problem are mostly based on some kind of precomputed data (caches), while the final results depend significantly on the quality of the caches. In this paper, we propose a learning‐based pipeline that can reproduce a wide range of complex light transport phenomena, including high‐frequency glossy interreflection, at any viewpoint in real time (> 90 frames per‐second), using information from imperfect caches stored at the barycentre of every triangle in a 3D scene. These caches are generated at a precomputation stage by a physically‐based offline renderer at a low sampling rate (e.g., 32 samples per‐pixel) and a low image resolution (e.g., 64×16). At runtime, a deep radiance reconstruction method based on a dedicated neural network is then involved to reconstruct a high‐quality radiance map of full global illumination at any viewpoint from these imperfect caches, without introducing noise and aliasing artifacts. To further improve the reconstruction accuracy, a new feature fusion strategy is designed in the network to better exploit useful contents from cheap G‐buffers generated at runtime. The proposed framework ensures high‐quality rendering of images for moderate‐sized scenes with full global illumination effects, at the cost of reasonable precomputation time. We demonstrate the effectiveness and efficiency of the proposed pipeline by comparing it with alternative strategies, including real‐time path tracing and precomputed radiance transfer.
Targeting Shape and Material in Lighting Design
Authors: Usta B. , Pont S. , Eisemann E. 
Published: March 2023

AbstractProduct lighting design is a laborious and time‐consuming task. With product illustrations being increasingly rendered, the lighting challenge transferred to the virtual realm. Our approach targets lighting design in the context of a scene with fixed objects, materials, and camera parameters, illuminated by environmental lighting. Our solution offers control over the depiction of material characteristics and shape details by optimizing the illuminating environment‐map. To that end, we introduce a metric that assesses the shape and material cues in terms of the designed appearance. We formalize the process and support steering the outcome using additional design constraints. We illustrate our solution with several challenging examples.
SO(3)‐Pose: SO(3)‐Equivariance Learning for 6D Object Pose Estimation
Authors: Pan Haoran , Zhou Jun , Liu Yuanpeng , Lu Xuequan , Wang Weiming , Yan Xuefeng , Wei Mingqiang 
Published: March 2023

Abstract6D pose estimation of rigid objects from RGB‐D images is crucial for object grasping and manipulation in robotics. Although RGB channels and the depth (D) channel are often complementary, providing respectively the appearance and geometry information, it is still non‐trivial on how to fully benefit from the two cross‐modal data. From the simple yet new observation, when an object rotates, its semantic label is invariant to the pose while its keypoint offset direction is variant to the pose. To this end, we present SO(3)‐Pose, a new representation learning network to explore SO(3)‐equivariant and SO(3)‐invariant features from the depth channel for pose estimation. The SO(3)‐invariant features facilitate to learn more distinctive representations for segmenting objects with similar appearance from RGB channels. The SO(3)‐equivariant features communicate with RGB features to deduce the (missed) geometry for detecting keypoints of an object with the reflective surface from the depth channel. Unlike most of existing pose estimation methods, our SO(3)‐Pose not only implements the information communication between the RGB and depth channels, but also naturally absorbs the SO(3)‐equivariance geometry knowledge from depth images, leading to better appearance and geometry representation learning. Comprehensive experiments show that our method achieves the state‐of‐the‐art performance on three benchmarks. Code is available at https://github.com/phaoran9999/SO3-Pose.
Learning 3D Shape Aesthetics Globally and Locally
Authors: Chen Minchan , Lau Manfred 
Published: March 2023

AbstractThere exist previous works in computing the visual aesthetics of 3D shapes “globally”, where the term global means that shape aesthetics data are collected for whole 3D shapes and then used to compute the aesthetics of whole 3D shapes. In this paper, we introduce a novel method that takes such “global” shape aesthetics data, and learn both a “global” shape aesthetics measure that computes aesthetics scores for whole 3D shapes, and a “local” shape aesthetics measure that computes to what extent a local region on the 3D shape surface contributes to the whole shape's aesthetics. These aesthetics measures are learned, and hence do not consider existing handcrafted notions of what makes a 3D shape aesthetic. We take a dataset of global pairwise shape aesthetics, where humans compares between pairs of shapes and say which shape from each pair is more aesthetic. Our solution proposes a point‐based neural network that takes a 3D shape represented by surface patches as input and jointly outputs its global aesthetics score and a local aesthetics map. To build connections between global and local aesthetics, we embed the global and local features into the same latent space and then output scores with the weights‐shared aesthetics predictors. Furthermore, we designed three loss functions to supervise the training jointly. We demonstrate the shape aesthetics results globally and locally to show that our framework can make good global aesthetics predictions while the predicted aesthetics maps are consistent with human perception. In addition, we present several applications enabled by our local aesthetics metric.