Miracle-dz/UniFAR-FTUs huggingface dataset:
GesGPT: Speech Gesture Synthesis With Text Parsing from GPT

Gesture synthesis has gained significant attention as a critical research area, focusing on producing contextually appropriate and natural gestures corresponding to speech or textual input. Although deep learning-based approaches have achieved remarkable progress, they often overlook the rich semantic information present in the text, leading to less expressive and meaningful gestures. We propose GesGPT, a novel approach to gesture generation that leverages the semantic analysis capabilities of Large Language Models (LLMs), such as GPT. By capitalizing on the strengths of LLMs for text analysis, we design prompts to extract gesture-related information from textual input. Our method entails developing prompt principles that transform gesture generation into an intention classification problem based on GPT, and utilizing a curated gesture library and integration module to produce semantically rich co-speech gestures. Experimental results demonstrate that GesGPT effectively generates contextually appropriate and expressive gestures, offering a new perspective on semantic co-speech gesture generation.
Boosting Reinforcement Learning and Planning with Demonstrations: A Survey

Although reinforcement learning has seen tremendous success recently, this kind of trial-and-error learning can be impractical or inefficient in complex environments. The use of demonstrations, on the other hand, enables agents to benefit from expert knowledge rather than having to discover the best action to take through exploration. In this survey, we discuss the advantages of using demonstrations in sequential decision making, various ways to apply demonstrations in learning-based decision making paradigms (for example, reinforcement learning and planning in the learned models), and how to collect the demonstrations in various scenarios. Additionally, we exemplify a practical pipeline for generating and utilizing demonstrations in the recently proposed ManiSkill robot learning benchmark.
Synthetic Aperture Radar Imaging Below a Random Rough Surface

Motivated by applications in unmanned aerial based ground penetrating radar for detecting buried landmines, we consider the problem of imaging small point like scatterers situated in a lossy medium below a random rough surface. Both the random rough surface and the absorption in the lossy medium significantly impede the target detection and imaging process. Using principal component analysis we effectively remove the reflection from the air‐soil interface. We then use a modification of the classical synthetic aperture radar imaging functional to image the targets. This imaging method introduces a user‐defined parameter, δ, which scales the resolution by δ $\sqrt{\delta }$ allowing for target localization with sub wavelength accuracy. Numerical results in two dimensions illustrate the robustness of the approach for imaging multiple targets. However, the depth at which targets are detectable is limited due to the absorption in the lossy medium.
Marching-Primitives: Shape Abstraction from Signed Distance Function

Representing complex objects with basic geometric primitives has long been a topic in computer vision. Primitive-based representations have the merits of compactness and computational efficiency in higher-level tasks such as physics simulation, collision checking, and robotic manipulation. Unlike previous works which extract polygonal meshes from a signed distance function (SDF), in this paper, we present a novel method, named Marching-Primitives, to obtain a primitive-based abstraction directly from an SDF. Our method grows geometric primitives (such as superquadrics) iteratively by analyzing the connectivity of voxels while marching at different levels of signed distance. For each valid connected volume of interest, we march on the scope of voxels from which a primitive is able to be extracted in a probabilistic sense and simultaneously solve for the parameters of the primitive to capture the underlying local geometry. We evaluate the performance of our method on both synthetic and real-world datasets. The results show that the proposed method out-performs the state-of-the-art in terms of accuracy, and is directly generalizable among different categories and scales. The code is open-sourced at https://github.com/ChirikjianLab/Marching-Primitives.git.
Online search is more likely to lead students to validate true news than to refute false ones

With the spread of high-speed Internet and portable smart devices, the way people access and consume information has drastically changed. However, this presents many challenges, including information overload, personal data leakage, and misinformation diffusion. Across the spectrum of risks that Internet users can face nowadays, this work focuses on understanding how young people perceive and deal with false information. Within an experimental campaign involving 261 students, we presented to the participants six different news items and invited them to browse the Internet to assess the veracity of the presented information. Our results suggest that online search is more likely to lead students to validate true news than to refute false ones. We found that students change their opinion related to a specific piece of information more often than their global idea about a broader topic. Also, our experiment reflected that the majority of participants rely on online sources to obtain information and access the news, and those getting information from books and Internet browsing are the most accurate in assessing the veracity of a news item. This work provides a principled understanding of how young people perceive and distinguish true and false pieces of information, identifying strengths and weaknesses amidst young subjects and contributing to build tailored digital information literacy strategies for youth.
Considerations on the Evaluation of Biometric Quality Assessment Algorithms

Quality assessment algorithms can be used to estimate the utility of a biometric sample for the purpose of biometric recognition. “Error versus Discard Characteristic” (EDC) plots, and “partial Area Under Curve” (pAUC) values of curves therein, are generally used by researchers to evaluate the predictive performance of such quality assessment algorithms. An EDC curve depends on an error type such as the “False Non Match Rate” (FNMR), a quality assessment algorithm, a biometric recognition system, a set of comparisons each corresponding to a biometric sample pair, and a comparison score threshold corresponding to a starting error. To compute an EDC curve, comparisons are progressively discarded based on the associated samples’ lowest quality scores, and the error is computed for the remaining comparisons. Additionally, a discard fraction limit or range must be selected to compute pAUC values, which can then be used to quantitatively rank quality assessment algorithms. This paper discusses and analyses various details for this kind of quality assessment algorithm evaluation, including general EDC properties, interpretability improvements for pAUC values based on a hard lower error limit and a soft upper error limit, the use of relative instead of discrete rankings, stepwise vs. linear curve interpolation, and normalisation of quality scores to a [0, 100] integer range. We also analyse the stability of quantitative quality assessment algorithm rankings based on pAUC values across varying pAUC discard fraction limits and starting errors, concluding that higher pAUC discard fraction limits should be preferred. The analyses are conducted both with synthetic data and with real face image and fingerprint quality assessment data, with a focus on general modality-independent conclusions for EDC evaluations. Various EDC alternatives are discussed as well. Open source evaluation software is provided at https://github.com/dasec/quality-assessment-evaluation. Will be made available upon acceptance.
Human behavior in the time of COVID-19: Learning from big data

Since the World Health Organization (WHO) characterized COVID-19 as a pandemic in March 2020, there have been over 600 million confirmed cases of COVID-19 and more than six million deaths as of October 2022. The relationship between the COVID-19 pandemic and human behavior is complicated. On one hand, human behavior is found to shape the spread of the disease. On the other hand, the pandemic has impacted and even changed human behavior in almost every aspect. To provide a holistic understanding of the complex interplay between human behavior and the COVID-19 pandemic, researchers have been employing big data techniques such as natural language processing, computer vision, audio signal processing, frequent pattern mining, and machine learning. In this study, we present an overview of the existing studies on using big data techniques to study human behavior in the time of the COVID-19 pandemic. In particular, we categorize these studies into three groups—using big data to measure, model, and leverage human behavior, respectively. The related tasks, data, and methods are summarized accordingly. To provide more insights into how to fight the COVID-19 pandemic and future global catastrophes, we further discuss challenges and potential opportunities.
Phonetic and acoustic differences in child and adult laughter

This is a preliminary study comparing the acoustic differences in recordings of child and adult spontaneous laughter. Altogether, 100 laughter calls where analyzed from one male and female adult and one male and female child. Results indicate that bout and call durations of children and adult laughter are similar in duration; however, segmental durations show developmental differences. Children show variation between vowel and consonant durations unlike adults. Moreover, child vowels are longer in duration when compared to the adult vowels. Surprisingly, between the adult and child vowels, no difference was observed in mean pitch and mean intensity values. The most prominent difference between child and adult laughter is observed in vowel quality where children's F1 values of laughter vowels are relatively higher than adult's. The consonantal resonance in children is similar to their vowels. However, in adults, the consonant resonant frequencies are much higher than the vowel resonances. Therefore, while .
Theoretical Model Construction of Deformation-Force for Soft Grippers Part I: Co-rotational Modeling and Force Control for Design Optimization

Compliant grippers, owing to adaptivity and safety, have attracted considerable attention for unstructured grasping in real applications, such as industrial or logistic scenarios. However, accurately modeling the bidirectional relationship between shape deformation and contact force for such grippers, the Fin-Ray grippers as an example, remains stagnant to date. To address this research gap, this article devises, presents, and experimentally validates a universal bidirectional force-displacement mathematical model for compliant grippers based on the co-rotational concept, which endows such grippers with an intrinsic force sensing capability and offers a better insight into the design optimization. In Part I of the article, we introduce the fundamental theory of the co-rotational approach, where arbitrary large deformation of beam elements can be modeled. Its intrinsic principle allows taking materials with varying stiffness, various connection types, and key design parameters into consideration with few assumptions. Further, the force-displacement relationship is numerically derived, providing accurate displacement estimations of the gripper under external forces with minor computational loads. The performance of the proposed method is experimentally verified through comparison with Finite Element Analysis (FEA) in simulation, obtaining a fair degree of accuracy (6%), and design optimization of Fin-Ray grippers is systematically investigated. Part II of this article demonstrating the force sensing capabilities and the effects of representative co-rotational modeling parameters on model accuracy is released in Arxiv.
Golden Ratio Base Expansions of the Logarithm and Inverse Tangent of Fibonacci and Lucas Numbers

Let $\alpha=(1+\sqrt 5)/2$, the golden ratio, and $\beta=-1/\alpha=(1 - \sqrt 5)/2$. Let $F_n$ and $L_n$ be the Fibonacci and Lucas numbers, defined by $F_n=(\alpha^n -\beta^n)/\sqrt 5$ and $L_n=\alpha^n + \beta^n$, for all non-negative integers. We derive base~$\alpha$ expansions of $\log F_n$, $\log L_n$, $\arctan\dfrac1{F_n}$ and $\arctan\dfrac1{L_n}$ for all positive integers $n$.
CMG-Net: An End-to-End Contact-based Multi-Finger Dexterous Grasping Network

In this paper, we propose a novel representation for grasping using contacts between multi-finger robotic hands and objects to be manipulated. This representation significantly reduces the prediction dimensions and accelerates the learning process. We present an effective end-to-end network, CMG-Net, for grasping unknown objects in a cluttered environment by efficiently predicting multi-finger grasp poses and hand configurations from a single-shot point cloud. Moreover, we create a synthetic grasp dataset that consists of five thousand cluttered scenes, 80 object categories, and 20 million annotations. We perform a comprehensive empirical study and demonstrate the effectiveness of our grasping representation and CMG-Net. Our work significantly outperforms the state-of-the-art for three-finger robotic hands. We also demonstrate that the model trained using synthetic data perform very well for real robots.
Evolving Populations of Diverse RL Agents with MAP-Elites

Quality Diversity (QD) has emerged as a powerful alternative optimization paradigm that aims at generating large and diverse collections of solutions, notably with its flagship algorithm MAP-ELITES (ME) which evolves solutions through mutations and crossovers. While very effective for some unstructured problems, early ME implementations relied exclusively on random search to evolve the population of solutions, rendering them notoriously sample-inefficient for high-dimensional problems, such as when evolving neural networks. Follow-up works considered exploiting gradient information to guide the search in order to address these shortcomings through techniques borrowed from either Black-Box Optimization (BBO) or Reinforcement Learning (RL). While mixing RL techniques with ME unlocked state-of-the-art performance for robotics control problems that require a good amount of exploration, it also plagued these ME variants with limitations common among RL algorithms that ME was free of, such as hyperparameter sensitivity, high stochasticity as well as training instability, including when the population size increases as some components are shared across the population in recent approaches. Furthermore, existing approaches mixing ME with RL tend to be tied to a specific RL algorithm, which effectively prevents their use on problems where the corresponding RL algorithm fails. To address these shortcomings, we introduce a flexible framework that allows the use of any RL algorithm and alleviates the aforementioned limitations by evolving populations of agents (whose definition include hyperparameters and all learnable parameters) instead of just policies. We demonstrate the benefits brought about by our framework through extensive numerical experiments on a number of robotics control problems, some of which with deceptive rewards, taken from the QD-RL literature.
Anharmonic theory of superconductivity and its applications to emerging quantum materials.

The role of anharmonicity on superconductivity has often been disregarded in the past. Recently, it has been recognized that anharmonic decoherence could play a fundamental role in determining the superconducting properties (electron-phonon coupling, critical temperature, etc) of a large class of materials, including systems close to structural soft-mode instabilities, amorphous solids and metals under extreme high-pressure conditions. Here, we review recent theoretical progress on the role of anharmonic effects, and in particular certain universal properties of anharmonic damping, on superconductivity. Our focus regards the combination of microscopic-agnostic effective theories for bosonic mediators with the well-established BCS theory and Migdal-Eliashberg theory for superconductivity. We discuss in detail the theoretical frameworks, their possible implementation within first-principles methods, and the experimental probes for anharmonic decoherence. Finally, we present several concrete applications to emerging quantum materials, including hydrides, ferroelectrics and systems with charge density wave instabilities.
Reinforcement Learning with Exogenous States and Rewards

Exogenous state variables and rewards can slow reinforcement learning by injecting uncontrolled variation into the reward signal. This paper formalizes exogenous state variables and rewards and shows that if the reward function decomposes additively into endogenous and exogenous components, the MDP can be decomposed into an exogenous Markov Reward Process (based on the exogenous reward) and an endogenous Markov Decision Process (optimizing the endogenous reward). Any optimal policy for the endogenous MDP is also an optimal policy for the original MDP, but because the endogenous reward typically has reduced variance, the endogenous MDP is easier to solve. We study settings where the decomposition of the state space into exogenous and endogenous state spaces is not given but must be discovered. The paper introduces and proves correctness of algorithms for discovering the exogenous and endogenous subspaces of the state space when they are mixed through linear combination. These algorithms can be applied during reinforcement learning to discover the exogenous space, remove the exogenous reward, and focus reinforcement learning on the endogenous MDP. Experiments on a variety of challenging synthetic MDPs show that these methods, applied online, discover large exogenous state spaces and produce substantial speedups in reinforcement learning.
Vortices in dipolar Bose–Einstein condensates

Quantized vortices are the hallmark of superfluidity, and are often sought out as the first observable feature in new superfluid systems. Following the recent experimental observation of vortices in Bose-Einstein condensates comprised of atoms with inherent long-range dipole-dipole interactions [Nat. Phys. 18, 1453-1458 (2022)], we thoroughly investigate vortex properties in the three-dimensional dominantly dipolar regime, where beyond-mean-field effects are crucial for stability, and investigate the interplay between trap geometry and magnetic field tilt angle.
Enabling Design Space Exploration of DRAM Caches for Emerging Memory Systems

The increasing growth of applications’ memory capacity and performance demands has led the CPU vendors to deploy heterogeneous memory systems either within a single system or via disaggregation. DRAM caches are one way to enable heterogeneity and disaggregation in such systems. While there is significant research investigating the designs of DRAM caches, there has been little research investigating DRAM caches from a full system point of view, because there is not a suitable model available to the community to accurately study large-scale systems with DRAM caches at a cycle-level. In this work we describe a new cycle-level DRAM cache model in the gem5 simulator which can be used for emerging heterogeneous and disaggregated memory systems.
Stochastic Graph Neural Network-Based Value Decomposition for MARL in Internet of Vehicles

Autonomous driving has witnessed incredible advances in the past several decades, while Multi-Agent Reinforcement Learning (MARL) promises to satisfy the essential need of autonomous vehicle control in a wireless connected vehicle networks. In MARL, how to effectively decompose a global feedback into the relative contributions of individual agents belongs to one of the most fundamental problems. However, the environment volatility due to vehicle movement and wireless disturbance could significantly shape time-varying topological relationships among agents, thus making the Value Decomposition (VD) challenging. Therefore, in order to cope with this annoying volatility, it becomes imperative to design a dynamic VD framework. Hence, in this article, we propose a novel Stochastic Value Mixing (SVMIX) methodology by taking account of dynamic topological features during VD and incorporating the corresponding components into a multi-agent actor-critic architecture. In particular, Stochastic Graph Neural Network (SGNN) is leveraged to effectively capture underlying dynamics in topological features and improve the flexibility of VD against the environment volatility. Finally, the superiority of SVMIX is verified through extensive simulations.
The brighter-fatter effect in the JWST MIRI Si:As IBC detectors. I. Observations, impact on science, and modeling

The Mid-Infrared Instrument (MIRI) on board the James Webb Space Telescope (JWST) uses three Si:As impurity band conduction (IBC) detector arrays. The output voltage level of each MIRI detector pixel is digitally recorded by sampling-up-the-ramp. For uniform or low-contrast illumination, the pixel ramps become non-linear in a predictable way, but in areas of high contrast, the non-linearity curve becomes much more complex. The origin of the effect is poorly understood and currently not calibrated. We provide observational evidence of the Brighter-Fatter Effect (BFE) in MIRI conventional and high-contrast coronographic imaging, low-resolution spectroscopy, and medium-resolution spectroscopy data and investigate the physical mechanism that gives rise to the effect on the MIRI detector pixel raw voltage integration ramps. We use public data from the JWST MIRI commissioning and Cycle 1 phase. We also develop a numerical electrostatic model of the MIRI detectors using a modified version of the public Poisson_CCD code. We find that the physical mechanism behind the BFE manifesting in MIRI data is fundamentally different to that of CCDs and photodiode arrays such as the Hawaii-XRG (HXRG) near-infrared detectors used by the NIRISS, NIRCam, and NIRSpec instruments on board JWST. Observationally, the BFE makes the JWST MIRI data yield 10-25 % larger point sources and spectral line profiles as a function of the relative level of debiasing of neighboring detector pixels. This broadening impacts the MIRI absolute flux calibration, time-series observations of faint companions, and PSF modeling and subtraction. We also find that the intra-pixel 2D profile of the shrinking Si:As IBC detector depletion region directly impacts the accuracy of the pixel ramp non-linearity calibration model.
Numerical simulation of low-Reynolds-number turbulent flow through a straight square duct

The mean flow and turbulent statistics obtained from the numerical simulation of the fully developed turbulent flow through a straight duct of square cross-section are reported. The Reynolds number based on the bulk velocity and hydraulic diameter is 4410. Spatial and temporal approximations of the equations of motion were derived from standard finite-difference techniques. To achieve sufficient spatial resolution 16.1 × 106 grid nodes were employed. Turbulent statistics along the wall bisectors show good agreement with plane channel data despite the influence of the sidewalls in the former flow. The mean secondary flow field consists of two counter-rotating cells symmetrically placed about the corner bisectors with their common flow towards each corner with strong evidence for the existence of a smaller and much weaker pair situated about the wall bisectors. The mean streamwise vorticity of each corner cell is found to be associated with a stronger vorticity distribution of the opposite sign having an absolute maximum on the nearest duct wall.