OpenRaiser/Intern-Atlas huggingface dataset:
A Comprehensive Review of Data-Driven Co-Speech Gesture Generation
Venue: arXiv cs.GR 2023
Year: 2023

Gestures that accompany speech are an essential part of natural and efficient embodied human communication. The automatic generation of such co-speech gestures is a long-standing problem in computer animation and is considered an enabling technology in film, games, virtual social spaces, and for interaction with social robots. The problem is made challenging by the idiosyncratic and non-periodic nature of human co-speech gesture motion, and by the great diversity of communicative functions that gestures encompass. Gesture generation has seen surging interest recently, owing to the emergence of more and larger datasets of human gesture motion, combined with strides in deep-learning-based generative models, that benefit from the growing availability of data. This review article summarizes co-speech gesture generation research, with a particular focus on deep generative models. First, we articulate the theory describing human gesticulation and how it complements speech. Next, we briefly discuss rule-based and classical statistical gesture synthesis, before delving into deep learning approaches. We employ the choice of input modalities as an organizing principle, examining systems that generate gestures from audio, text, and non-linguistic input. We also chronicle the evolution of the related training data sets in terms of size, diversity, motion quality, and collection method. Finally, we identify key research challenges in gesture generation, including data availability and quality; producing human-like motion; grounding the gesture in the co-occurring speech in interaction with other speakers, and in the environment; performing gesture evaluation; and integration of gesture synthesis into applications. We highlight recent approaches to tackling the various key challenges, as well as the limitations of these approaches, and point toward areas of future development.
An Empirical Study on Software Bill of Materials: Where We Stand and the Road Ahead
Venue: arXiv cs.SE 2023
Year: 2023

The rapid growth of software supply chain attacks has attracted considerable attention to software bill of materials (SBOM). SBOMs are a crucial building block to ensure the transparency of software supply chains that helps improve software supply chain security. Although there are significant efforts from academia and industry to facilitate SBOM development, it is still unclear how practitioners perceive SBOMs and what are the challenges of adopting SBOMs in practice. Furthermore, existing SBOM-related studies tend to be ad-hoc and lack software engineering focuses. To bridge this gap, we conducted the first empirical study to interview and survey SBOM practitioners. We applied a mixed qualitative and quantitative method for gathering data from 17 interviewees and 65 survey respondents from 15 countries across five continents to understand how practitioners perceive the SBOM field. We summarized 26 statements and grouped them into three topics on SBOM's states of practice. Based on the study results, we derived a goal model and highlighted future directions where practitioners can put in their effort.
Quick Minimization of Tardy Processing Time on a Single Machine
Venue: arXiv cs.DS 2023
Year: 2023

We consider the problem of minimizing the total processing time of tardy jobs on a single machine. This is a classical scheduling problem, first considered by [Lawler and Moore 1969], that also generalizes the Subset Sum problem. Recently, it was shown that this problem can be solved efficiently by computing $(\max,\min)$-skewed-convolutions. The running time of the resulting algorithm is equivalent, up to logarithmic factors, to the time it takes to compute a $(\max,\min)$-skewed-convolution of two vectors of integers whose sum is $O(P)$, where $P$ is the sum of the jobs' processing times. We further improve the running time of the minimum tardy processing time computation by introducing a job ``bundling'' technique and achieve a $\tilde{O}\left(P^{2-1/α}\right)$ running time, where $\tilde{O}\left(P^α\right)$ is the running time of a $(\max,\min)$-skewed-convolution of vectors of size $P$. This results in a $\tilde{O}\left(P^{7/5}\right)$ time algorithm for tardy processing time minimization, an improvement over the previously known $\tilde{O}\left(P^{5/3}\right)$ time algorithm.
Designing losses for data-free training of normalizing flows on Boltzmann distributions
Venue: arXiv cs.LG 2023
Year: 2023

Generating a Boltzmann distribution in high dimension has recently been achieved with Normalizing Flows, which enable fast and exact computation of the generated density, and thus unbiased estimation of expectations. However, current implementations rely on accurate training data, which typically comes from computationally expensive simulations. There is therefore a clear incentive to train models with incomplete or no data by relying solely on the target density, which can be obtained from a physical energy model (up to a constant factor). For that purpose, we analyze the properties of standard losses based on Kullback-Leibler divergences. We showcase their limitations, in particular a strong propensity for mode collapse during optimization on high-dimensional distributions. We then propose strategies to alleviate these issues, most importantly a new loss function well-grounded in theory and with suitable optimization properties. Using as a benchmark the generation of 3D molecular configurations, we show on several tasks that, for the first time, imperfect pre-trained models can be further optimized in the absence of training data.
Scalable Batch Acquisition for Deep Bayesian Active Learning
Venue: arXiv cs.LG 2023
Year: 2023

In deep active learning, it is especially important to choose multiple examples to markup at each step to work efficiently, especially on large datasets. At the same time, existing solutions to this problem in the Bayesian setup, such as BatchBALD, have significant limitations in selecting a large number of examples, associated with the exponential complexity of computing mutual information for joint random variables. We, therefore, present the Large BatchBALD algorithm, which gives a well-grounded approximation to the BatchBALD method that aims to achieve comparable quality while being more computationally efficient. We provide a complexity analysis of the algorithm, showing a reduction in computation time, especially for large batches. Furthermore, we present an extensive set of experimental results on image and text data, both on toy datasets and larger ones such as CIFAR-100.
UserSimCRS: A User Simulation Toolkit for Evaluating Conversational Recommender Systems
Venue: arXiv cs.IR 2023
Year: 2023

We present an extensible user simulation toolkit to facilitate automatic evaluation of conversational recommender systems. It builds on an established agenda-based approach and extends it with several novel elements, including user satisfaction prediction, persona and context modeling, and conditional natural language generation. We showcase the toolkit with a pre-existing movie recommender system and demonstrate its ability to simulate dialogues that mimic real conversations, while requiring only a handful of manually annotated dialogues as training data.
STAR-RIS Assisted Over-the-Air Vertical Federated Learning in Multi-Cell Wireless Networks
Venue: arXiv cs.IT 2023
Year: 2023

Vertical federated learning (FL) is a critical enabler for distributed artificial intelligence services in the emerging 6G era, as it allows for secure and efficient collaboration of machine learning among a wide range of Internet of Things devices. However, current studies of wireless FL typically consider a single task in a single-cell wireless network, ignoring the impact of inter-cell interference on learning performance. In this paper, we investigate a simultaneous transmitting and reflecting reconfigurable intelligent surface (STAR-RIS) assisted over-the-air computation based vertical FL system in multi-cell networks, in which a STAR-RIS is deployed at the cell edge to facilitate the completion of different FL tasks in different cells. We establish the convergence of the proposed system through theoretical analysis and introduce the Pareto boundary of the optimality gaps to characterize the trade-off among cells. Based on the analysis, we then jointly design the transmit and receive beamforming as well as the STAR-RIS transmission and reflection coefficient matrices to minimize the sum of the gaps of all cells. To solve the non-convex resource allocation problem, we introduce a successive convex approximation based algorithm. Numerical experiments demonstrate that compared with conventional approaches, the proposed STAR-RIS assisted vertical FL model and the cooperative resource allocation algorithm achieve much lower mean-squared error for both uplink and downlink transmission in multi-cell wireless networks, resulting in improved learning performance for vertical FL.
DINF: Dynamic Instance Noise Filter for Occluded Pedestrian Detection
Venue: arXiv cs.CV 2023
Year: 2023

Occlusion issue is the biggest challenge in pedestrian detection. RCNN-based detectors extract instance features by cropping rectangle regions of interest in the feature maps. However, the visible pixels of the occluded objects are limited, making the rectangle instance feature mixed with a lot of instance-irrelevant noise information. Besides, by counting the number of instances with different degrees of overlap of CrowdHuman dataset, we find that the number of severely overlapping objects and the number of slightly overlapping objects are unbalanced, which may exacerbate the challenges posed by occlusion issues. Regarding to the noise issue, from the perspective of denoising, an iterable dynamic instance noise filter (DINF) is proposed for the RCNN-based pedestrian detectors to improve the signal-noise ratio of the instance feature. Simulating the wavelet denoising process, we use the instance feature vector to generate dynamic convolutional kernels to transform the RoIs features to a domain in which the near-zero values represent the noise information. Then, soft thresholding with channel-wise adaptive thresholds is applied to convert the near-zero values to zero to filter out noise information. For the imbalance issue, we propose an IoU-Focal factor (IFF) to modulate the contributions of the well-regressed boxes and the bad-regressed boxes to the loss in the training process, paying more attention to the minority severely overlapping objects. Extensive experiments conducted on CrowdHuman and CityPersons demonstrate that our methods can help RCNN-based pedestrian detectors achieve state-of-the-art performance.
Microfluidic Pulse Shaping Methods for Molecular Communications
Venue: arXiv cs.ET 2023
Year: 2023

Molecular Communication (MC) is a bio-inspired communication modality that utilizes chemical signals in the form of molecules to exchange information between spatially separated entities. Pulse shaping is an important process in all communication systems, as it modifies the waveform of transmitted signals to match the characteristics of the communication channel for reliable and high-speed information transfer. In MC systems, the unconventional architectures of components, such as transmitters and receivers, and the complex, nonlinear, and time-varying nature of MC channels make pulse shaping even more important. While several pulse shaping methods have been theoretically proposed for MC, their practicality and performance are still uncertain. Moreover, the majority of recently proposed experimental MC testbeds that rely on microfluidics technology lack the incorporation of programmable pulse shaping methods, which hinders the accurate evaluation of MC techniques in practical settings. To address the challenges associated with pulse shaping in microfluidic MC systems, we provide a comprehensive overview of practical microfluidic chemical waveform generation techniques that have been experimentally validated and whose architectures can inform the design of pulse shaping methods for microfluidic MC systems and testbeds. These techniques include those based on hydrodynamic and acoustofluidic force fields, as well as electrochemical reactions. We also discuss the fundamental working mechanisms and system architectures of these techniques, and compare their performances in terms of spatiotemporal resolution, selectivity, system complexity, and other performance metrics relevant to MC applications, as well as their feasibility for practical MC applications.
Sem@$K$: Is my knowledge graph embedding model semantic-aware?
Venue: arXiv cs.LG 2023
Year: 2023

Using knowledge graph embedding models (KGEMs) is a popular approach for predicting links in knowledge graphs (KGs). Traditionally, the performance of KGEMs for link prediction is assessed using rank-based metrics, which evaluate their ability to give high scores to ground-truth entities. However, the literature claims that the KGEM evaluation procedure would benefit from adding supplementary dimensions to assess. That is why, in this paper, we extend our previously introduced metric Sem@K that measures the capability of models to predict valid entities w.r.t. domain and range constraints. In particular, we consider a broad range of KGs and take their respective characteristics into account to propose different versions of Sem@K. We also perform an extensive study to qualify the abilities of KGEMs as measured by our metric. Our experiments show that Sem@K provides a new perspective on KGEM quality. Its joint analysis with rank-based metrics offers different conclusions on the predictive power of models. Regarding Sem@K, some KGEMs are inherently better than others, but this semantic superiority is not indicative of their performance w.r.t. rank-based metrics. In this work, we generalize conclusions about the relative performance of KGEMs w.r.t. rank-based and semantic-oriented metrics at the level of families of models. The joint analysis of the aforementioned metrics gives more insight into the peculiarities of each model. This work paves the way for a more comprehensive evaluation of KGEM adequacy for specific downstream tasks.
A LiDAR-Inertial-Visual SLAM System with Loop Detection
Venue: arXiv cs.RO 2023
Year: 2023

We have proposed, to the best of our knowledge, the first-of-its-kind LiDAR-Inertial-Visual-Fused simultaneous localization and mapping (SLAM) system with a strong place recognition capacity. Our proposed SLAM system is consist of visual-inertial odometry (VIO) and LiDAR inertial odometry (LIO) subsystems. We propose the LIO subsystem utilizing the measurement from the LiDAR and the inertial sensors to build the local odometry map, and propose the VIO subsystem which takes in the visual information to construct the 2D-3D associated map. Then, we propose an iterative Kalman Filter-based optimization function to optimize the local project-based 2D-to-3D photo-metric error between the projected image pixels and the local 3D points to make the robust 2D-3D alignment. Finally, we have also proposed the back-end pose graph global optimization and the elaborately designed loop closure detection network to improve the accuracy of the whole SLAM system. Extensive experiments deployed on the UGV in complicated real-world circumstances demonstrate that our proposed LiDAR-Visual-Inertial localization system outperforms the current state-of-the-art in terms of accuracy, efficiency, and robustness.
Investigating the Combination of Planning-Based and Data-Driven Methods for Goal Recognition
Venue: arXiv cs.AI 2023
Year: 2023

An important feature of pervasive, intelligent assistance systems is the ability to dynamically adapt to the current needs of their users. Hence, it is critical for such systems to be able to recognize those goals and needs based on observations of the user's actions and state of the environment. In this work, we investigate the application of two state-of-the-art, planning-based plan recognition approaches in a real-world setting. So far, these approaches were only evaluated in artificial settings in combination with agents that act perfectly rational. We show that such approaches have difficulties when used to recognize the goals of human subjects, because human behaviour is typically not perfectly rational. To overcome this issue, we propose an extension to the existing approaches through a classification-based method trained on observed behaviour data. We empirically show that the proposed extension not only outperforms the purely planning-based- and purely data-driven goal recognition methods but is also able to recognize the correct goal more reliably, especially when only a small number of observations were seen. This substantially improves the usefulness of hybrid goal recognition approaches for intelligent assistance systems, as recognizing a goal early opens much more possibilities for supportive reactions of the system.
Channel Measurement for Holographic MIMO: Benefits and Challenges of Spatial Oversampling
Venue: arXiv cs.IT 2023
Year: 2023

In this paper, the channel of an indoor holographic multiple-input multiple-output (MIMO) system is measured. It is demonstrated through experiments for the first time that the spatial oversampling of holographic MIMO systems is able to increase the capacity of a wireless communication system significantly. However, the antenna efficiency is the most crucial challenge preventing us from getting the capacity improvement. An extended EM-compliant channel model is also proposed for holographic MIMO systems, which is able to take the non-isotropic characteristics of the propagation environment, the antenna pattern distortion, the antenna efficiency, and the polarization characteristics into consideration.
Predictions of photophysical properties of phosphorescent platinum(II) complexes based on ensemble machine learning approach
Venue: arXiv cs.LG 2023
Year: 2023

Phosphorescent metal complexes have been under intense investigations as emissive dopants for energy efficient organic light emitting diodes (OLEDs). Among them, cyclometalated Pt(II) complexes are widespread triplet emitters with color-tunable emissions. To render their practical applications as OLED emitters, it is in great need to develop Pt(II) complexes with high radiative decay rate constant ($k_r$) and photoluminescence (PL) quantum yield. Thus, an efficient and accurate prediction tool is highly desirable. Here, we develop a general protocol for accurate predictions of emission wavelength, radiative decay rate constant, and PL quantum yield for phosphorescent Pt(II) emitters based on the combination of first-principles quantum mechanical method, machine learning (ML) and experimental calibration. A new dataset concerning phosphorescent Pt(II) emitters is constructed, with more than two hundred samples collected from the literature. Features containing pertinent electronic properties of the complexes are chosen. Our results demonstrate that ensemble learning models combined with stacking-based approaches exhibit the best performance, where the values of squared correlation coefficients ($R^2$), mean absolute error (MAE), and root mean square error (RMSE) are 0.96, 7.21 nm and 13.00 nm for emission wavelength prediction, and 0.81, 0.11 and 0.15 for PL quantum yield prediction. For radiative decay rate constant ($k_r$), the obtained value of $R^2$ is 0.67 while MAE and RMSE are 0.21 and 0.25 (both in log scale), respectively. The accuracy of the protocol is further confirmed using 24 recently reported Pt(II) complexes, which demonstrates its reliability for a broad palette of Pt(II) emitters.We expect this protocol will become a valuable tool, accelerating the rational design of novel OLED materials with desired properties.
Streaming Lower Bounds and Asymmetric Set-Disjointness
Venue: arXiv cs.CC 2023
Year: 2023

Frequency estimation in data streams is one of the classical problems in streaming algorithms. Following much research, there are now almost matching upper and lower bounds for the trade-off needed between the number of samples and the space complexity of the algorithm, when the data streams are adversarial. However, in the case where the data stream is given in a random order, or is stochastic, only weaker lower bounds exist. In this work we close this gap, up to logarithmic factors. In order to do so we consider the needle problem, which is a natural hard problem for frequency estimation studied in (Andoni et al. 2008, Crouch et al. 2016). Here, the goal is to distinguish between two distributions over data streams with $t$ samples. The first is uniform over a large enough domain. The second is a planted model; a secret ''needle'' is uniformly chosen, and then each element in the stream equals the needle with probability $p$, and otherwise is uniformly chosen from the domain. It is simple to design streaming algorithms that distinguish the distributions using space $s \approx 1/(p^2 t)$. It was unclear if this is tight, as the existing lower bounds are weaker. We close this gap and show that the trade-off is near optimal, up to a logarithmic factor. Our proof builds and extends classical connections between streaming algorithms and communication complexity, concretely multi-party unique set-disjointness. We introduce two new ingredients that allow us to prove sharp bounds. The first is a lower bound for an asymmetric version of multi-party unique set-disjointness, where players receive input sets of different sizes, and where the communication of each player is normalized relative to their input length. The second is a combinatorial technique that allows to sample needles in the planted model by first sampling intervals, and then sampling a uniform needle in each interval.
From stage to page: language independent bootstrap measures of distinctiveness in fictional speech
Venue: arXiv cs.CL 2023
Year: 2023

Stylometry is mostly applied to authorial style. Recently, researchers have begun investigating the style of characters, finding that the variation remains within authorial bounds. We address the stylistic distinctiveness of characters in drama. Our primary contribution is methodological; we introduce and evaluate two non-parametric methods to produce a summary statistic for character distinctiveness that can be usefully applied and compared across languages and times. Our first method is based on bootstrap distances between 3-gram probability distributions, the second (reminiscent of 'unmasking' techniques) on word keyness curves. Both methods are validated and explored by applying them to a reasonably large corpus (a subset of DraCor): we analyse 3301 characters drawn from 2324 works, covering five centuries and four languages (French, German, Russian, and the works of Shakespeare). Both methods appear useful; the 3-gram method is statistically more powerful but the word keyness method offers rich interpretability. Both methods are able to capture phonological differences such as accent or dialect, as well as broad differences in topic and lexical richness. Based on exploratory analysis, we find that smaller characters tend to be more distinctive, and that women are cross-linguistically more distinctive than men, with this latter finding carefully interrogated using multiple regression. This greater distinctiveness stems from a historical tendency for female characters to be restricted to an 'internal narrative domain' covering mainly direct discourse and family/romantic themes. It is hoped that direct, comparable statistical measures will form a basis for more sophisticated future studies, and advances in theory.