Miracle-dz/UniFAR-FTUs huggingface dataset:
A low latency attention module for streaming self-supervised speech representation learning

The transformer is a fundamental building block in deep learning, and the attention mechanism is the transformer's core component. Self-supervised speech representation learning (SSRL) represents a popular use-case for the transformer architecture. Due to transformers' acausal behavior, the use of transformers for SSRL has been predominantly focused on acausal applications. However, several media processing problems, such as speech processing, require real-time solutions. In this paper, we present an implementation of the attention module that enables training of SSRL architectures with low compute and memory requirements, while allowing real-time inference with low and fixed latency. The attention module proposed in this paper includes two components, streaming attention (SA) and low-latency streaming attention (LLSA). The SA represents our proposal for an efficient streaming SSRL implementation, while the LLSA solves the latency build-up problem of other streaming attention architectures, such as the masked acausal attention (MAA), guaranteeing a latency equal to one layer even when multiple layers are stacked. We present a comparative analysis between the vanilla attention, which we will refer here as acausal attention (AA), the SA, and the LLSA, by training a streaming SSRL with automatic speech recognition as downstream task. When training on librispeech-clean-100 and testing on librispeech-test-clean, our low-latency attention module has a word error rate (WER) of 5.84%, which represents a significant improvement over the MAA (WER = 13.82%). Our implementation also reduces the inference latency from 1.92 to 0.16 seconds. The proposed low-latency module preserves many of the benefits of conventional acausal transformers, but also enables latency characteristics that make it applicable to real-time streaming applications.
gmmsslm : Semi-supervised Gaussian mixture modeling with a missing data mechanism in R

Semi-supervised learning is extensively applied these days to estimate classiﬁers from training data in which not all of the labels of the feature vectors are available. With the use of generative models that propose a form for the joint distribution of a feature vector and its ground-truth label, the Bayes’ classiﬁer can be estimated via maximum likelihood on partially classiﬁed training data. To increase the accuracy of this sample classiﬁer, Ahfock and McLachlan (2020) proposed that a missing-label mechanism be adopted and that the Bayes’ classiﬁer be estimated on the basis of the full likelihood formed in the framework that models the probability of a missing label given its observed feature vector in terms of its entropy. In the case of two Gaussian classes with a common covariance matrix, it was shown that the accuracy of the classiﬁer so estimated from the partially classiﬁed training data can even have lower error rate than if it were estimated from the sample completely classiﬁed. Here, we focus on an algorithm for estimating the Bayes’ classiﬁer via the full likelihood in the case of multiple Gaussian classes with arbitrary covariance matrices. Diﬀerent strategies for initializing the algorithm are discussed and illustrated. A new R package with these tools, gmmsslm , is demonstrated on real data.
Improved Best-of-Both-Worlds Guarantees for Multi-Armed Bandits: FTRL with General Regularizers and Multiple Optimal Arms

We study the problem of designing adaptive multi-armed bandit algorithms that perform optimally in both the stochastic setting and the adversarial setting simultaneously (often known as a best-of-both-world guarantee). A line of recent works shows that when configured and analyzed properly, the Follow-the-Regularized-Leader (FTRL) algorithm, originally designed for the adversarial setting, can in fact optimally adapt to the stochastic setting as well. Such results, however, critically rely on an assumption that there exists one unique optimal arm. Recently, Ito (2021) took the first step to remove such an undesirable uniqueness assumption for one particular FTRL algorithm with the $\frac{1}{2}$-Tsallis entropy regularizer. In this work, we significantly improve and generalize this result, showing that uniqueness is unnecessary for FTRL with a broad family of regularizers and a new learning rate schedule. For some regularizers, our regret bounds also improve upon prior results even when uniqueness holds. We further provide an application of our results to the decoupled exploration and exploitation problem, demonstrating that our techniques are broadly applicable.
How far is almost strong compactness from strong compactness

Bagaria and Magidor introduced the notion of almost strong compactness, which is very close to the notion of strong compactness. Boney and Brooke-Taylor asked whether the least almost strongly compact cardinal is strongly compact. Goldberg gives a positive answer in the case $\mathrm{SCH}$ holds from below and the least almost strongly compact cardinal has uncountable cofinality. In this paper, we give a negative answer for the general case. Our result also gives an affirmative answer to a question of Bagaria and Magidor.
Fluctuation Relations to Calculate Protein Redox Potentials from Molecular Dynamics Simulations

The tunable design of protein redox potentials promises to open a range of applications in biotechnology and catalysis. Here, we introduce a method to calculate redox potential changes by combining fluctuation relations with molecular dynamics simulations. It involves the simulation of reduced and oxidized states, followed by the instantaneous conversion between them. Energy differences introduced by the perturbations are obtained using the Kubo-Onsager approach. Using a detailed fluctuation relation coupled with Bayesian inference, these are postprocessed into estimates for the redox potentials in an efficient manner. This new method, denoted MD + CB, is tested on a de novo four-helix bundle heme protein (the m4D2 “maquette”) and five designed mutants, including some mutants characterized experimentally in this work. The MD + CB approach is found to perform reliably, giving redox potential shifts with reasonably good correlation (0.85) to the experimental values for the mutants. The MD + CB approach also compares well with redox potential shift predictions using a continuum electrostatic method. The estimation method employed within the MD + CB approach is straightforwardly transferable to standard equilibrium MD simulations and holds promise for redox protein engineering and design applications.
A Deep Neural Network Based Reverse Radio Spectrogram Search Algorithm

Modern radio astronomy instruments generate vast amounts of data, and the increasingly challenging radio frequency interference (RFI) environment necessitates ever-more sophisticated RFI rejection algorithms. The ‘needle in a haystack’ nature of searches for transients and technosignatures requires us to develop methods that can determine whether a signal of interest has unique properties, or is a part of some larger set of pernicious RFI. In the past, this vetting has required onerous manual inspection of very large numbers of signals. In this paper we present a fast and modular deep learning algorithm to search for lookalike signals of interest in radio spectrogram data. First, we trained a β-Variational Autoencoder on signals returned by an energy detection algorithm. We then adapted a positional embedding layer from classical Transformer architecture to a embed additional metadata, which we demonstrate using a frequency-based embedding. Next we used the encoder component of the β-Variational Autoencoder to extract features from small (∼715 Hz, with a resolution of 2.79 Hz per frequency bin) windows in the radio spectrogram. We used our algorithm to conduct a search for a given query (encoded signal of interest) on a set of signals (encoded features of searched items) to produce the top candidates with similar features. We successfully demonstrate that the algorithm retrieves signals with similar appearance, given only the original radio spectrogram data. This algorithm can be used to improve the efficiency of vetting signals of interest in technosignature searches, but could also be applied to a wider variety of searches for ‘lookalike’ signals in large astronomical datasets.
Robust High-Dimensional Time-Varying Coefficient Estimation

In this paper, we develop a novel high-dimensional coefficient estimation procedure based on high-frequency data. Unlike usual high-dimensional regression procedure such as LASSO, we additionally handle the heavy-tailedness of high-frequency observations as well as time variations of coefficient processes. Specifically, we employ Huber loss and truncation scheme to handle heavy-tailed observations, while $\ell_{1}$-regularization is adopted to overcome the curse of dimensionality. To account for the time-varying coefficient, we estimate local coefficients which are biased due to the $\ell_{1}$-regularization. Thus, when estimating integrated coefficients, we propose a debiasing scheme to enjoy the law of large number property and employ a thresholding scheme to further accommodate the sparsity of the coefficients. We call this Robust thrEsholding Debiased LASSO (RED-LASSO) estimator. We show that the RED-LASSO estimator can achieve a near-optimal convergence rate. In the empirical study, we apply the RED-LASSO procedure to the high-dimensional integrated coefficient estimation using high-frequency trading data.
Estimating Fiscal Multipliers by Combining Statistical Identification with Potentially Endogenous Proxies

Different proxy variables used in fiscal policy SVARs lead to contradicting conclusions regarding the size of fiscal multipliers. We show that the conflicting results are due to violations of the exogeneity assumptions, i.e. the commonly used proxies are endogenously related to the structural shocks. We propose a novel approach to include proxy variables into a Bayesian non-Gaussian SVAR, tailored to accommodate for potentially endogenous proxy variables. Using our model, we show that increasing government spending is a more effective tool to stimulate the economy than reducing taxes.
Temporally distributed parent body accretion in the C reservoir of the solar system

Accretion processes in protoplanetary disks produce a diversity of small bodies that played a crucial role in multiple reshuffling events throughout the solar system and in both early and late accretion of planets. Application of thermo-chronometers to meteorites provides precise dating of the formation age of mineralogical components. Nucleosynthetic anomalies indicating a dichotomy between NC and C meteorites and precise parent body (PB) chronology can be combined with thermal evolution models to constrain the timescale of accretion and dynamical processes in the early solar system. Achondrite PBs are considered to have accreted early and mostly in the NC region, while late accretion in the C region produced mostly undifferentiated PBs, such as the CR chondrite PB that formed as late as 4 Ma after CAIs. However, presence of more evolved CR-related meteorites suggests also an earlier accretion timing. We present modeling evidence for a temporally distributed accretion of parent bodies of CR-related meteorite groups that originate from a C reservoir and range from aqueously altered chondrites to partially differentiated primitive achondrites. The PB formation times derived range from<1 Ma to ~4 Ma after solar system formation, with ~3.7 Ma, ~1.5-2.75 Ma,<~0.6 Ma, and<~0.7 Ma for CR, Flensburg, NWA 6704, and NWA 011. This implies that accretion processes in the C reservoir started as early as in the NC reservoir and produced differentiated PBs with carbonaceous compositions in addition to undifferentiated C chondritic PBs. The accretion times correlate inversely with the degree of the meteorites' alteration, metamorphism, or differentiation. Accretion times for CI/CM, Ryugu, and Tafassites PBs of ~3.75 Ma, ~1-3 Ma, and 1.1 Ma, respectively, fit well into this correlation in agreement with the thermal and alteration conditions suggested by the meteorites.
Dust ion acoustic rogue waves in superthermal warm ion plasma

The properties of dust ion acoustic rogue waves (DIARWs) in an unmagnetized collisionless plasma system composed of charged dust grains, superthermal electrons and warm ions as a fluid are studied. The multiple scale perturbation method is used to derive a nonlinear Schrodinger equation (NLSE) for DIARWs. From the coefficients of nonlinearity and dispersion, we have determined the critical wave number threshold kcr at which modulational instability sets in. This critical wave number depends on the various plasma parameters, viz. superthermality of electrons, ion temperature and dust concentration. Within the modulational instability region, a random perturbation of amplitude grows and thus, creates DIARWs. It is found that DIARWs are significantly affected by electron superthermality (via κ), ion temperature (via σ) and dust concentration (via f). In view of the crucial importance of DIARWs in space environments, our results may be useful in understanding the basic features of DIARWs that may occur in space plasmas.
Numerical modelling of complex turbulent free‐surface flows with the SPH method: an overview

The gridless smoothed particle hydrodynamics (SPH) method is now commonly used in computational fluid dynamics (CFD) and appears to be promising in predicting complex free‐surface flows. However, increasing flow complexity requires appropriate approaches for taking account of turbulent effects, whereas some authors are still working without any turbulence closure in SPH. A review of recently developed turbulence models adapted to the SPH method is presented herein, from the simplistic point of view of a one‐equation model involving mixing length to more sophisticated (and thus realistic) models like explicit algebraic Reynolds stress models (EARSM) or large eddy simulation (LES). Each proposed model is tested and validated on the basis of schematic cases for which laboratory data, theoretical or numerical solutions are available in the general field of turbulent free‐surface incompressible flows (e.g. open‐channel flow and schematic dam break). They give satisfactory results, even though some progress should be made in the future in terms of free‐surface influence and wall conditions. Recommendations are given to SPH users to apply this method to the modelling of complex free‐surface turbulent flows. Copyright © 2006 John Wiley & Sons, Ltd.
Identifying the source term in the potential equation with weighted sparsity regularization

We explore the possibility for using boundary measurements to recover a sparse source term f(x) in the potential equation. Employing weighted sparsity regularization and standard results for subgradients, we derive simple-to-check criteria which assure that a number of sinks (f(x)<0) and sources (f(x)>0) can be identified. Furthermore, we present two cases for which these criteria always are fulfilled: a) well-separated sources and sinks, and b) many sources or sinks located at the boundary plus one interior source/sink. Our approach is such that the linearity of the associated forward operator is preserved in the discrete formulation. The theory is therefore conveniently developed in terms of Euclidean spaces, and it can be applied to a wide range of problems. In particular, it can be applied to both isotropic and anisotropic cases. We present a series of numerical experiments. This work is motivated by the observation that standard methods typically suggest that internal sinks and sources are located close to the boundary.
Using Auxiliary Tasks In Multimodal Fusion of Wav2vec 2.0 And Bert for Multimodal Emotion Recognition

The lack of data and the difficulty of multimodal fusion have always been challenges for multimodal emotion recognition (MER). In this paper, we propose to use pre-trained models as upstream network, wav2vec 2.0 for audio modality and BERT for text modality, and finetune them in downstream task of MER to cope with the lack of data. For the difficulty of multimodal fusion, we use a K-layer multi-head attention mechanism as a downstream fusion module. Starting from the MER task itself, we design two auxiliary tasks to alleviate the insufficient fusion between modalities and guide the network to capture and align emotion-related features. Compared to the previous state-of-the-art models, we achieve a better performance by 78.42% Weighted Accuracy (WA) and 79.71% Unweighted Accuracy (UA) on the IEMOCAP dataset.
Self-Guided Noise-Free Data Generation for Efficient Zero-Shot Learning

There is a rising interest in further exploring the zero-shot learning potential of large pre-trained language models (PLMs). A new paradigm called data-generation-based zero-shot learning has achieved impressive success. In this paradigm, the synthesized data from the PLM acts as the carrier of knowledge, which is used to train a task-specific model with orders of magnitude fewer parameters than the PLM, achieving both higher performance and efficiency than prompt-based zero-shot learning methods on PLMs. The main hurdle of this approach is that the synthesized data from PLM usually contains a significant portion of low-quality samples. Fitting on such data will greatly hamper the performance of the task-specific model, making it unreliable for deployment. Previous methods remedy this issue mainly by filtering synthetic data using heuristic metrics(e.g., output confidence), or refining the data with the help of a human expert, which comes with excessive manual tuning or expensive costs. In this paper, we propose a novel noise-robust re-weighting framework SunGen to automatically construct high-quality data for zero-shot classification problems. Our framework features the ability to learn the sample weights indicating data quality without requiring any human annotation. We theoretically and empirically verify the ability of our method to help construct good-quality synthetic datasets. Notably, SunGen-LSTM yields a 9.8% relative improvement than the baseline on average accuracy across eight different established text classification tasks.
Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants

Large Language Models (LLMs) such as OpenAI Codex are increasingly being used as AI-based coding assistants. Understanding the impact of these tools on developers' code is paramount, especially as recent work showed that LLMs may suggest cybersecurity vulnerabilities. We conduct a security-driven user study (N=58) to assess code written by student programmers when assisted by LLMs. Given the potential severity of low-level bugs as well as their relative frequency in real-world projects, we tasked participants with implementing a singly-linked 'shopping list' structure in C. Our results indicate that the security impact in this setting (low-level C with pointer and array manipulations) is small: AI-assisted users produce critical security bugs at a rate no greater than 10% more than the control, indicating the use of LLMs does not introduce new security risks.
Physical Momentum in the Indian Stock Market

Our study focuses on determining the presence of abnormal returns for physical momentum portfolios in the context of the Indian market. The physical momentum portfolios, comprising stocks from the NSE 500, are constructed for the daily, weekly, monthly, and yearly timescales. In the aforementioned timescales, we empirically evaluate the historical returns and varied risk profiles of these portfolios for the years 2014-2021. It has been observed that the best-performing physical momentum portfolios from each of the four timescales achieved higher returns and better risk measures when compared to the benchmark NIFTY 50 portfolio. We further find that the high-frequency daily time scale exhibits the strongest reversal in the physical momentum effect, wherein the portfolio yielded a 16-fold profit over the initial investment.
Searching for iron nanoparticles with a general-purpose Gaussian approximation potential

We present a general-purpose machine learning Gaussian approximation potential (GAP) for iron that is applicable to all bulk crystal structures found experimentally under diverse thermodynamic conditions, as well as surfaces and nanoparticles (NPs). By studying its phase diagram, we show that our GAP remains stable at extreme conditions, including those found in the Earth's core. The new GAP is particularly accurate for the description of NPs. We use it to identify new low-energy NPs, whose stability is verified by performing density functional theory calculations on the GAP structures. Many of these NPs are lower in energy than those previously available in the literature up to $N_\text{atoms}=100$. We further extend the convex hull of available stable structures to $N_\text{atoms}=200$. For these NPs, we study characteristic surface atomic motifs using data clustering and low-dimensional embedding techniques. With a few exceptions, e.g., at magic numbers $N_\text{atoms}=59$, $65$, $76$ and $78$, we find that iron tends to form irregularly shaped NPs without a dominant surface character or characteristic atomic motif, and no reminiscence of crystalline features. We hypothesize that the observed disorder stems from an intricate balance and competition between the stable bulk motif formation, with bcc structure, and the stable surface motif formation, with fcc structure. We expect these results to improve our understanding of the fundamental properties and structure of low-dimensional forms of iron, and to facilitate future work in the field of iron-based catalysis.
Individual based SIS models on (not so) dense large random networks

Starting from a stochastic individual-based description of an SIS epidemic spreading on a random network, we study the dynamics when the size n of the network tends to infinity. We recover in the limit an infinite-dimensional integro-differential equation studied by Delmas, Dronnier and Zitt (2022) for an SIS epidemic propagating on a graphon. Our work covers the case of dense and sparse graphs, provided that the number of edges grows faster than n, but not the case of very sparse graphs with O(n) edges. In order to establish our limit theorem, we have to deal with both the convergence of the random graphs to the graphon and the convergence of the stochastic process spreading on top of these random structures: in particular, we propose a coupling between the process of interest and an epidemic that spreads on the complete graph but with a modified infection rate. Keywords: random graph, mathematical models of epidemics, measure-valued process, large network limit, limit theorem, graphon.
Computing excluded minors for classes of matroids representable over partial fields

We describe an implementation of a computer search for the"small"excluded minors for a class of matroids representable over a partial field. Using these techniques, we enumerate the excluded minors on at most 15 elements for both the class of dyadic matroids, and the class of 2-regular matroids. We conjecture that there are no other excluded minors for the class of 2-regular matroids; whereas, on the other hand, we show that there is a 16-element excluded minor for the class of dyadic matroids.