Miracle-dz/UniFAR-FTUs huggingface dataset:
Automatic Evaluation of Turn-taking Cues in Conversational Speech Synthesis

Turn-taking is a fundamental aspect of human communication where speakers convey their intention to either hold, or yield, their turn through prosodic cues. Using the recently proposed Voice Activity Projection model, we propose an automatic evaluation approach to measure these aspects for conversational speech synthesis. We investigate the ability of three commercial, and two open-source, Text-To-Speech (TTS) systems ability to generate turn-taking cues over simulated turns. By varying the stimuli, or controlling the prosody, we analyze the models performances. We show that while commercial TTS largely provide appropriate cues, they often produce ambiguous signals, and that further improvements are possible. TTS, trained on read or spontaneous speech, produce strong turn-hold but weak turn-yield cues. We argue that this approach, that focus on functional aspects of interaction, provides a useful addition to other important speech metrics, such as intelligibility and naturalness.
CTC-based Non-autoregressive Speech Translation

Combining end-to-end speech translation (ST) and non-autoregressive (NAR) generation is promising in language and speech processing for their advantages of less error propagation and low latency.In this paper, we investigate the potential of connectionist temporal classification (CTC) for non-autoregressive speech translation (NAST). In particular, we develop a model consisting of two encoders that are guided by CTC to predict the source and target texts, respectively.Introducing CTC into NAST on both language sides has obvious challenges: 1) the conditional independent generation somewhat breaks the interdependency among tokens, and 2) the monotonic alignment assumption in standard CTC does not hold in translation tasks.In response, we develop a prediction-aware encoding approach and a cross-layer attention approach to address these issues.We also use curriculum learning to improve convergence of training.Experiments on the MuST-C ST benchmarks show that our NAST model achieves an average BLEU score of 29.5 with a speed-up of 5.67\times, which is comparable to the autoregressive counterpart and even outperforms the previous best result of 0.9 BLEU points.
STL: Surprisingly Tricky Logic (for System Validation)

Much of the recent work developing formal methods techniques to specify or learn the behavior of autonomous systems is predicated on a belief that formal specifications are interpretable and useful for humans when checking systems. Though frequently asserted, this assumption is rarely tested. We performed a human experiment $(\mathbf{N}=62)$ with a mix of people who were and were not familiar with formal methods beforehand, asking them to validate whether a set of signal temporal logic (STL) constraints would keep an agent out of harm and allow it to complete a task in a gridworld capture-the-ftag setting. Validation accuracy was 45% $\pm$ 20% (mean $\pm$ standard deviation). The ground-truth validity of a specification, subjects' familiarity with formal methods, and subjects' level of education were found to be significant factors in determining validation correctness. Participants exhibited an affirmation bias, causing significantly increased accuracy on valid specifications, but significantly decreased accuracy on invalid specifications. Additionally, participants, particularly those familiar with formal methods, tended to be overconfident in their answers, and be similarly confident regardless of actual correctness. Our data do not support the belief that formal specifications are inherently human-interpretable to a meaningful degree for system validation. We recommend ergonomic improvements to data presentation and validation training, which should be tested before claims of interpretability make their way back into the formal methods literature.
DPFormer: Learning Differentially Private Transformer on Long-Tailed Data

The Transformer has emerged as a versatile and effective architecture with broad applications. However, it still remains an open problem how to efficiently train a Transformer model of high utility with differential privacy guarantees. In this paper, we identify two key challenges in learning differentially private Transformers, i.e., heavy computation overhead due to per-sample gradient clipping and unintentional attention distraction within the attention mechanism. In response, we propose DPFormer, equipped with Phantom Clipping and Re-Attention Mechanism, to address these challenges. Our theoretical analysis shows that DPFormer can reduce computational costs during gradient clipping and effectively mitigate attention distraction (which could obstruct the training process and lead to a significant performance drop, especially in the presence of long-tailed data). Such analysis is further corroborated by empirical results on two real-world datasets, demonstrating the efficiency and effectiveness of the proposed DPFormer.
Bypassing H\"older super-critcality barriers in viscous, incompressible fluids

This is the second in a series of papers where we analyze the incompressible Navier-Stokes equations in H\"older spaces. We obtain, to our knowledge, the very first genuinely super-critical regularity criterion for this system of equations in any dimension $d\geq3$ and in the absence of physical boundaries. For \emph{any} $\beta\in(0,1)$, we show that $L_t^1C_x^{0,\beta}$ solutions emanating from smooth initial data do not develop any singularities. The novelty stems from obtaining new bounds on the fundamental solution associated with a one-dimensional drift-diffusion equation in the presence of destabilizing singular lower order terms. Such a bound relies heavily on the symmetry and pointwise structure of the problem, where the drift term is shown to ``enhance'' the parabolic nature of the equation, allowing us to break the criticality barrier. Coupled with a subtle regularity estimate for the pressure courtesy of Silvestre, we are able to treat the (incompressible) Navier-Stokes equation as a perturbation of the classical drift-diffusion problem. This is achieved by propagating moduli of continuity as was done in our previous work, based on the elegant ideas introduced by Kiselev, Nazarov, Volberg and Shterenberg.
On the Privacy-Robustness-Utility Trilemma in Distributed Learning

The ubiquity of distributed machine learning (ML) in sensitive public domain applications calls for algorithms that protect data privacy, while being robust to faults and adversarial behaviors. Although privacy and robustness have been extensively studied independently in distributed ML, their synthesis remains poorly understood. We present the first tight analysis of the error incurred by any algorithm ensuring robustness against a fraction of adversarial machines, as well as differential privacy (DP) for honest machines' data against any other curious entity. Our analysis exhibits a fundamental trade-off between privacy, robustness, and utility. To prove our lower bound, we consider the case of mean estimation, subject to distributed DP and robustness constraints, and devise reductions to centralized estimation of one-way marginals. We prove our matching upper bound by presenting a new distributed ML algorithm using a high-dimensional robust aggregation rule. The latter amortizes the dependence on the dimension in the error (caused by adversarial workers and DP), while being agnostic to the statistical properties of the data.
Screening with Disadvantaged Agents

Motivated by school admissions, this paper studies screening in a population with both advantaged and disadvantaged agents. A school is interested in admitting the most skilled students, but relies on imperfect test scores that reflect both skill and effort. Students are limited by a budget on effort, with disadvantaged students having tighter budgets. This raises a challenge for the principal: among agents with similar test scores, it is difficult to distinguish between students with high skills and students with large budgets. Our main result is an optimal stochastic mechanism that maximizes the gains achieved from admitting ``high-skill"students minus the costs incurred from admitting ``low-skill"students when considering two skill types and $n$ budget types. Our mechanism makes it possible to give higher probability of admission to a high-skill student than to a low-skill, even when the low-skill student can potentially get higher test-score due to a higher budget. Further, we extend our admission problem to a setting in which students uniformly receive an exogenous subsidy to increase their budget for effort. This extension can only help the school's admission objective and we show that the optimal mechanism with exogenous subsidies has the same characterization as optimal mechanisms for the original problem.
Fully gapped superconductivity and topological aspects of the noncentrosymmetric superconductor TaReSi

We report a study of the noncentrosymmetric TaReSi superconductor by means of muon-spin rotation and relaxation ($\mu$SR) technique, complemented by electronic band-structure calculations. Its superconductivity, with $T_c$ = 5.5 K and upper critical field $\mu_0H_\mathrm{c2}(0)$ $\sim$ 3.4 T, was characterized via electrical-resistivity- and magnetic-susceptibility measurements. The temperature-dependent superfluid density, obtained from transverse-field $\mu$SR, suggests a fully-gapped superconducting state in TaReSi, with an energy gap $\Delta_0$ = 0.79 meV and a magnetic penetration depth $\lambda_0$ = 562 nm. The absence of a spontaneous magnetization below $T_c$, as confirmed by zero-field $\mu$SR, indicates a preserved time-reversal symmetry in the superconducting state. The density of states near the Fermi level is dominated by the Ta- and Re-5$d$ orbitals, which account for the relatively large band splitting due to the antisymmetric spin-orbit coupling. In its normal state, TaReSi behaves as a three-dimensional Kramers nodal-line semimetal, characterized by an hourglass-shaped dispersion protected by glide reflection. By combining non\-triv\-i\-al electronic bands with intrinsic superconductivity, TaReSi is a promising material for investigating the topological aspects of noncentrosymmetric superconductors.
Near-Field Communications: A Tutorial Review

Extremely large-scale antenna arrays, tremendously high frequencies, and new types of antennas are three clear trends in multi-antenna technology for supporting the sixth-generation (6G) networks. To properly account for the new characteristics introduced by these three trends in communication system design, the near-field spherical-wave propagation model needs to be used, which differs from the classical far-field planar-wave one. As such, near-field communication (NFC) will become essential in 6G networks. In this tutorial, we cover three key aspects of NFC. 1) Channel Modelling: We commence by reviewing near-field spherical-wave-based channel models for spatially-discrete (SPD) antennas. Then, uniform spherical wave (USW) and non-uniform spherical wave (NUSW) models are discussed. Subsequently, we introduce a general near-field channel model for SPD antennas and a Green’s function-based channel model for continuous-aperture (CAP) antennas. 2) Beamfocusing and Antenna Architectures: We highlight the properties of near-field beamfocusing and discuss NFC antenna architectures for both SPD and CAP antennas. Moreover, the basic principles of near-field beam training are introduced. 3) Performance Analysis: Finally, we provide a comprehensive performance analysis framework for NFC. For near-field line-of-sight channels, the received signal-to-noise ratio and power-scaling law are derived. For statistical near-field multipath channels, a general analytical framework is proposed, based on which analytical expressions for the outage probability, ergodic channel capacity, and ergodic mutual information are obtained. Finally, for each aspect, topics for future research are discussed.
The category of MSO transductions

MSO transductions are binary relations between structures which are defined using monadic second-order logic. MSO transductions form a category, since they are closed under composition. We show that many notions from language theory, such as recognizability or tree decompositions, can be defined in an abstract way that only refers to MSO transductions and their compositions.
On the structure of set-theoretic polygon equations

Polygon equations generalize the prominent pentagon equation in very much the same way as simplex equations generalize the famous Yang-Baxter equation. In particular, they appeared as"cocycle equations"in Street's category theory associated with oriented simplices. Whereas the (N-1)-simplex equation can be regarded as a realization of the higher Bruhat order B(N,N-2), the N-gon equation is a realization of the higher Tamari order T(N,N-2). The latter and its dual, associated with which is the dual N-gon equation, have been shown to arise as suborders of B(N,N-2) via a"three-color decomposition". There are two different reductions of T(N,N-2) and its dual, to T(N-1,N-3), respectively its dual. In this work, we explore the corresponding reductions of (dual) polygon equations, which lead to relations between solutions of neighboring (dual) polygon equations. We also elaborate (dual) polygon equations in this respect explicitly up to the octagon equation.
Transformer Language Models Handle Word Frequency in Prediction Head

Prediction head is a crucial component of Transformer language models. Despite its direct impact on prediction, this component has often been overlooked in analyzing Transformers. In this study, we investigate the inner workings of the prediction head, specifically focusing on bias parameters. Our experiments with BERT and GPT-2 models reveal that the biases in their word prediction heads play a significant role in the models' ability to reflect word frequency in a corpus, aligning with the logit adjustment method commonly used in long-tailed learning. We also quantify the effect of controlling the biases in practical auto-regressive text generation scenarios; under a particular setting, more diverse text can be generated without compromising text quality.
Improving Generalization in Language Model-based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-based Techniques

Compositional and domain generalization present significant challenges in semantic parsing, even for state-of-the-art semantic parsers based on pre-trained language models (LMs). In this study, we empirically investigate improving an LM’s generalization in semantic parsing with two simple techniques: at the token level, we introduce a token preprocessing method to preserve the semantic boundaries of tokens produced by LM tokenizers; at the sequence level, we propose to use special tokens to mark the boundaries of components aligned between input and output. Our experimental results on two text-to-SQL semantic parsing datasets show that our token preprocessing, although simple, can substantially improve the LM performance on both types of generalization, and our component boundary marking method is particularly helpful for compositional generalization.
Learning Capacity: A Measure of the Effective Dimensionality of a Model

We exploit a formal correspondence between thermodynamics and inference, where the number of samples can be thought of as the inverse temperature, to define a"learning capacity'' which is a measure of the effective dimensionality of a model. We show that the learning capacity is a tiny fraction of the number of parameters for many deep networks trained on typical datasets, depends upon the number of samples used for training, and is numerically consistent with notions of capacity obtained from the PAC-Bayesian framework. The test error as a function of the learning capacity does not exhibit double descent. We show that the learning capacity of a model saturates at very small and very large sample sizes; this provides guidelines, as to whether one should procure more data or whether one should search for new architectures, to improve performance. We show how the learning capacity can be used to understand the effective dimensionality, even for non-parametric models such as random forests and $k$-nearest neighbor classifiers.
Synfeal: A Data-Driven Simulator for End-to-End Camera Localization

Collecting real-world data is often considered the bottleneck of Artificial Intelligence, stalling the research progress in several fields, one of which is camera localization. End-to-end camera localization methods are still outperformed by traditional methods, and we argue that the inconsistencies associated with the data collection techniques are restraining the potential of end-to-end methods. Inspired by the recent data-centric paradigm, we propose a framework that synthesizes large localization datasets based on realistic 3D reconstructions of the real world. Our framework, termed Synfeal: Synthetic from Real, is an open-source, data-driven simulator that synthesizes RGB images by moving a virtual camera through a realistic 3D textured mesh, while collecting the corresponding ground-truth camera poses. The results validate that the training of camera localization algorithms on datasets generated by Synfeal leads to better results when compared to datasets generated by state-of-the-art methods. Using Synfeal, we conducted the first analysis of the relationship between the size of the dataset and the performance of camera localization algorithms. Results show that the performance significantly increases with the dataset size. Our results also suggest that when a large localization dataset with high quality is available, training from scratch leads to better performances. Synfeal is publicly available at https://github.com/DanielCoelho112/synfeal.
Acceleration of stochastic gradient descent with momentum by averaging: finite-sample rates and asymptotic normality

Stochastic gradient descent with momentum (SGDM) has been widely used in many machine learning and statistical applications. Despite the observed empirical benefits of SGDM over traditional SGD, the theoretical understanding of the role of momentum for different learning rates in the optimization process remains widely open. We analyze the finite-sample convergence rate of SGDM under the strongly convex settings and show that, with a large batch size, the mini-batch SGDM converges faster than the mini-batch SGD to a neighborhood of the optimal value. Additionally, our findings, supported by theoretical analysis and numerical experiments, indicate that SGDM permits broader choices of learning rates. Furthermore, we analyze the Polyak-averaging version of the SGDM estimator, establish its asymptotic normality, and justify its asymptotic equivalence to the averaged SGD. The asymptotic distribution of the averaged SGDM enables uncertainty quantification of the algorithm output and statistical inference of the model parameters.
WhitenedCSE: Whitening-based Contrastive Learning of Sentence Embeddings

This paper presents a whitening-based contrastive learning method for sentence embedding learning (WhitenedCSE), which combines contrastive learning with a novel shuffled group whitening. Generally, contrastive learning pulls distortions of a single sample (i.e., positive samples) close and push negative samples far away, correspondingly facilitating the alignment and uniformity in the feature space. A popular alternative to the “pushing” operation is whitening the feature space, which scatters all the samples for uniformity. Since the whitening and the contrastive learning have large redundancy w.r. t. the uniformity, they are usually used separately and do not easily work together. For the first time, this paper integrates whitening into the contrastive learning scheme and facilitates two benefits. 1) Better uniformity. We find that these two approaches are not totally redundant but actually have some complementarity due to different uniformity mechanism. 2) Better alignment. We randomly divide the feature into multiple groups along the channel axis and perform whitening independently within each group. By shuffling the group division, we derive multiple distortions of a single sample and thus increase the positive sample diversity. Consequently, using multiple positive samples with enhanced diversity further improves contrastive learning due to better alignment. Extensive experiments on seven semantic textual similarity tasks show our method achieves consistent improvement over the contrastive learning baseline and sets new states of the art, e.g., 78.78% (+2.53% based on BERT{pasted macro ‘BA’}) Spearman correlation on STS tasks.
High-Fidelity Image Compression with Score-based Generative Models

Despite the tremendous success of diffusion generative models in text-to-image generation, replicating this success in the domain of image compression has proven difficult. In this paper, we demonstrate that diffusion can significantly improve perceptual quality at a given bit-rate, outperforming state-of-the-art approaches PO-ELIC and HiFiC as measured by FID score. This is achieved using a simple but theoretically motivated two-stage approach combining an autoencoder targeting MSE followed by a further score-based decoder. However, as we will show, implementation details matter and the optimal design decisions can differ greatly from typical text-to-image models.
Is Silent eHMI Enough? A Passenger-Centric Study on Effective eHMI for Autonomous Personal Mobility Vehicles in the Field

Autonomous Personal Mobility Vehicle (APMV) is a miniaturized autonomous vehicle designed to provide short-distance mobility to everyone in pedestrian-rich environments. By the characteristic of the open design, passengers on the APMV are exposed to the communication between the eHMI deployed on APMVs and pedestrians. Therefore, to ensure an optimal passenger experience, eHMI designs for APMVs must consider the potential impact of APMV-pedestrian communications on passengers' subjective feelings. To this end, this study discussed three external human-machine interface (eHMI) designs, i.e., 1) graphical user interface (GUI)-based eHMI with text message (eHMI-T), 2) multimodal user interface (MUI)-based eHMI with neutral voice (eHMI-NV), and 3) MUI-based eHMI with affective voice (eHMI-AV), from the perspective of APMV passengers in the communication between APMV and pedestrians. In the riding field experiment (N=24), we found that eHMI-T may be less suitable for APMVs. This conclusion was drawn based on passengers' feedback, as they expressed an awkward feeling during the"silent time"when the eHMI-T provided information only to pedestrians but not to passengers. Additionally, these two MUI-based eHMIs with voice cues had their own advantages, i.e., eHMI-NV has an advantage in pragmatic quality, while eHMI-AV has an advantage in hedonic quality. The study also highlights the necessity of considering passengers' personalities when desig