Miracle-dz/UniFAR-FTUs huggingface dataset:
Wav-BERT: Cooperative Acoustic and Linguistic Representation Learning for Low-Resource Speech Recognition

Unifying acoustic and linguistic representation learning has become increasingly crucial to transfer the knowledge learned on the abundance of high-resource language data for low-resource speech recognition. Existing approaches simply cascade pre-trained acoustic and language models to learn the transfer from speech to text. However, how to solve the representation discrepancy of speech and text is unexplored, which hinders the utilization of acoustic and linguistic information. Moreover, previous works simply replace the embedding layer of the pre-trained language model with the acoustic features, which may cause the catastrophic forgetting problem. In this work, we introduce Wav-BERT, a cooperative acoustic and linguistic representation learning method to fuse and utilize the contextual information of speech and text. Specifically, we unify a pre-trained acoustic model (wav2vec 2.0) and a language model (BERT) into an end-to-end trainable framework. A Representation Aggregation Module is designed to aggregate acoustic and linguistic representation, and an Embedding Attention Module is introduced to incorporate acoustic information into BERT, which can effectively facilitate the cooperation of two pre-trained models and thus boost the representation learning. Extensive experiments show that our Wav-BERT significantly outperforms the existing approaches and achieves state-of-the-art performance on low-resource speech recognition.
Topology and geometry of Gaussian random fields II: on critical points, excursion sets, and persistent homology

This paper is second in the series, following Pranav et al. (2019), focused on the characterization of geometric and topological properties of 3D Gaussian random fields. We focus on the formalism of persistent homology, the mainstay of Topological Data Analysis (TDA), in the context of excursion set formalism. We also focus on the structure of critical points of stochastic fields, and their relationship with formation and evolution of structures in the universe. The topological background is accompanied by an investigation of Gaussian field simulations based on the LCDM spectrum, as well as power-law spectra with varying spectral indices. We present the statistical properties in terms of the intensity and difference maps constructed from the persistence diagrams, as well as their distribution functions. We demonstrate that the intensity maps encapsulate information about the distribution of power across the hierarchies of structures in more detailed than the Betti numbers or the Euler characteristic. In particular, the white noise (n = 0) case with flat spectrum stands out as the divide between models with positive and negative spectral index. It has the highest proportion of low significance features. This level of information is not available from the geometric Minkowski functionals or the topological Euler characteristic, or even the Betti numbers, and demonstrates the usefulness of hierarchical topological methods. Another important result is the observation that topological characteristics of Gaussian fields depend on the power spectrum, as opposed to the geometric measures that are insensitive to the power spectrum characteristics.
Unifying Design-based Inference: A New Variance Estimation Principle

This paper presents two novel classes of variance estimators with superior properties, in the absence of parametric or semi-parametricassumptions. The first new class of estimator is the Oblozene Chlebizky (OC) variance estimators as a novel alternative to the generalized sandwich in Paper 1 of 4. That the OC concept is unlikely to arise from other, more standard, frameworks is manifestly true in light of the 40 year lacuna since White (1980). For any member of the generalized sandwich variance estimator class, there is an OC with the same expected value. The this alternative replaces a random matrix at the center with a nonrandom one. The second type of estimator is guaranteed conservative for the variance of the estimator and is based upon a similar principle of replacing a random matrix with its expectation.
V-SlowFast Network for Efficient Visual Sound Separation

The objective of this paper is to perform visual sound separation: i) we study visual sound separation on spectrograms of different temporal resolutions; ii) we propose a new light yet efficient three-stream framework V-SlowFast that operates on Visual frame, Slow spectrogram, and Fast spectrogram. The Slow spectrogram captures the coarse temporal resolution while the Fast spectrogram contains the fine-grained temporal resolution; iii) we introduce two contrastive objectives to encourage the network to learn discriminative visual features for separating sounds; iv) we propose an audio-visual global attention module for audio and visual feature fusion; v) the introduced V-SlowFast model outperforms previous state-of-the-art in single-frame based visual sound separation on small- and large-scale datasets: MUSIC-21, AVE, and VGG-Sound. We also propose a small V-SlowFast architecture variant, which achieves 74.2% reduction in the number of model parameters and 81.4% reduction in GMACs compared to the previous multi-stage models. Project page: https://ly-zhu.github. io/V-SlowFast
Stability Analysis of Nonlinear Inviscid Microscopic and Macroscopic Traffic Flow Models of Bidirectional Cruise-Controlled Vehicles

The paper introduces a new bidirectional microscopic inviscid Adaptive Cruise Control (ACC) model that uses only spacing information from the preceding and following vehicles in order to select the proper control action to avoid collisions and maintain a desired speed. KL estimates that guarantee uniform convergence of the ACC model to the set of equilibria are provided. Moreover, the corresponding macroscopic model is derived, consisting of a conservation equation and a momentum equation that contains a nonlinear relaxation term. It is shown that, if the density is sufficiently small, then the macroscopic model has a solution that approaches exponentially the equilibrium speed (in the sup norm) while the density converges exponentially to a traveling wave. Numerical simulations are also provided, illustrating the properties of the microscopic and macroscopic inviscid ACC models.
A Studious Approach to Semi-Supervised Learning

The problem of learning from few labeled examples while using large amounts of unlabeled data has been approached by various semi-supervised methods. Although these methods can achieve superior performance, the models are often not deployable due to the large number of parameters. This paper is an ablation study of distillation in a semi-supervised setting, which not just reduces the number of parameters of the model but can achieve this while improving the performance over the baseline supervised model and making it better at generalizing. After the supervised pretraining, the network is used as a teacher model, and a student network is trained over the soft labels that the teacher model generates over the entire unlabeled data. We find that the fewer the labels, the more this approach benefits from a smaller student network. This brings forward the potential of distillation as an effective solution to enhance performance in semi-supervised computer vision tasks while maintaining deployability.
Harnessing the Power of Ego Network Layers for Link Prediction in Online Social Networks

Being able to recommend links between users in online social networks is important for users to connect with like-minded individuals as well as for the platforms themselves and third parties leveraging social media information to grow their business. Predictions are typically based on unsupervised or supervised learning, often leveraging simple yet effective graph topological information, such as the number of common neighbors. However, we argue that richer information about personal social structure of individuals might lead to better predictions. In this article, we propose to leverage well-established social cognitive theories to improve link prediction performance. According to these theories, individuals arrange their social relationships along, on average, five concentric circles of decreasing intimacy. We postulate that relationships in different circles have different importance in predicting new links. To validate this claim, we focus on popular feature extraction prediction algorithms (both unsupervised and supervised) and we extend them to include social circles’ awareness. We validate the prediction performance of these circle-aware algorithms against several benchmarks (including their baseline versions as well as node-embedding- and graph neural network (GNN)-based link prediction), leveraging two Twitter datasets comprising a community of video gamers and generic users. We show that social awareness generally provides significant improvements in prediction performance, beating also state-of-the-art solutions such as node2vec and learning from Subgraphs, Embeddings and Attributes for Link prediction (SEAL), and without increasing the computational complexity. Finally, we show that social awareness can be used in place of using a classifier (which may be costly or impractical) for targeting a specific category of users.
Regularize! Don't Mix: Multi-Agent Reinforcement Learning without Explicit Centralized Structures

We propose using regularization for Multi-Agent Reinforcement Learning rather than learning explicit cooperative structures called {\em Multi-Agent Regularized Q-learning} (MARQ). Many MARL approaches leverage centralized structures in order to exploit global state information or removing communication constraints when the agents act in a decentralized manner. Instead of learning redundant structures which is removed during agent execution, we propose instead to leverage shared experiences of the agents to regularize the individual policies in order to promote structured exploration. We examine several different approaches to how MARQ can either explicitly or implicitly regularize our policies in a multi-agent setting. MARQ aims to address these limitations in the MARL context through applying regularization constraints which can correct bias in off-policy out-of-distribution agent experiences and promote diverse exploration. Our algorithm is evaluated on several benchmark multi-agent environments and we show that MARQ consistently outperforms several baselines and state-of-the-art algorithms; learning in fewer steps and converging to higher returns.
On the Convergence of Tsetlin Machines for the AND and the OR Operators

—The Tsetlin Machine (TM) is a novel machine learning algorithm based on propositional logic, which has obtained the state-of-the-art performance on several pattern recognition problems. In previous studies, the convergence properties of TM for 1-bit operation and XOR operation have been analyzed. To make the analyses on the basic digital operations complete, in this article, we analyze the convergence when input training samples follow AND and OR operators respectively. Our analyses reveal that the TM can converge almost surely to reproduce AND and OR operators, which are learnt from training data over an inﬁnite time horizon. Speciﬁcally, by analysing the OR operator, we reveal the convergence property of TM when two sub-patterns can be jointly represented by one clause, which is quite distinct compared with the analysis of the XOR case. The analyses on AND and OR operators, together with the previously analysed 1-bit and XOR operations, complete the convergence analyses on basic operators in Boolean algebra.
Explaining Convolutional Neural Networks by Tagging Filters

Convolutional neural networks (CNNs) have achieved astonishing performance on various image classification tasks, but it is difficult for humans to understand how a classification comes about. Recent literature proposes methods to explain the classification process to humans. These focus mostly on visualizing feature maps and filter weights, which are not very intuitive for non-experts in analyzing a CNN classification. In this paper, we propose FilTag, an approach to effectively explain CNNs even to non-experts. The idea is that when images of a class frequently activate a convolutional filter, then that filter is tagged with that class. These tags provide an explanation to a reference of a class-specific feature detected by the filter. Based on the tagging, individual image classifications can then be intuitively explained in terms of the tags of the filters that the input image activates. Finally, we show that the tags are helpful in analyzing classification errors caused by noisy input images and that the tags can be further processed by machines.
Towards Zero and Few-shot Knowledge-seeking Turn Detection in Task-orientated Dialogue Systems

Most prior work on task-oriented dialogue systems is restricted to supporting domain APIs. However, users may have requests that are out of the scope of these APIs. This work focuses on identifying such user requests. Existing methods for this task mainly rely on fine-tuning pre-trained models on large annotated data. We propose a novel method, REDE, based on adaptive representation learning and density estimation. REDE can be applied to zero-shot cases, and quickly learns a high-performing detector with only a few shots by updating less than 3K parameters. We demonstrate REDE’s competitive performance on DSTC9 data and our newly collected test set.
Approximate Conditional Sampling for Pattern Detection in Weighted Networks

Assessing the statistical significance of network patterns is crucial for understanding whether such patterns indicate the presence of interesting network phenomena, or whether they simply result from less interesting processes, such as nodal-heterogeneity. Typically, significance is computed with reference to a null model. While there has been extensive research into such null models for unweighted graphs, little has been done for the weighted case. This article suggests a null model for weighted graphs. The model fixes node strengths exactly, and approximately fixes node degrees. A novel MCMC algorithm is proposed for sampling the model, and its stochastic stability is considered. We show empirically that the model compares favorably to alternatives, particularly when network patterns are subtle. We show how the algorithm can be used to evaluate the statistical significance of community structure.
CARL: Conditional-value-at-risk Adversarial Reinforcement Learning

In this paper we present a risk-averse reinforcement learning (RL) method called Conditional value-at-risk Adversarial Reinforcement Learning (CARL). To the best of our knowledge, CARL is the ﬁrst game formulation for Conditional Value-at-Risk (CVaR) RL. The game takes place between a policy player and an adversary that perturbs the policy player’s state transitions given a ﬁnite budget. We prove that, at the maximin equilibrium point, the learned policy is CVaR optimal with a risk tolerance explicitly related to the adversary’s budget. We provide a gradient-based training procedure to solve CARL by formulating it as a zero-sum Stackelberg Game, enabling the use of deep reinforcement learning architectures and training algorithms. Finally, we show that solving the CARL game does lead to risk-averse behaviour in a toy grid environment, also conﬁrming that an increased adversary produces increasingly cautious policies.
Deep Anomaly Generation: An Image Translation Approach of Synthesizing Abnormal Banded Chromosome Images

Advances in deep-learning-based pipelines have led to breakthroughs in a variety of microscopy image diagnostics. However, a sufficiently big training data set is usually difficult to obtain due to high annotation costs. In the case of banded chromosome images, the creation of big enough libraries is difficult for multiple pathologies due to the rarity of certain genetic disorders. Generative Adversarial Networks (GANs) have proven to be effective in generating synthetic images and extending training data sets. In our work, we implement a conditional GAN (cGAN) that allows generation of realistic single chromosome images following user-defined banding patterns. To this end, an image-to-image translation approach based on automatically created 2D chromosome segmentation label maps is used. Our validation shows promising results when synthesizing chromosomes with seen as well as unseen banding patterns. We believe that this approach can be exploited for data augmentation of chromosome data sets with structural abnormalities. Therefore, the proposed method could help to tackle medical image analysis problems such as data simulation, segmentation, detection, or classification in the field of cytogenetics.
Insights from a zeroth order analysis on steady streaming generated by an oscillating gas slug

Active micropumping and micromixing using oscillating bubbles form the basis for various Lab – on – chip applications. Acoustically oscillating bubbles are commonly used in active particle sorting and in micropumps/mixers. Slug flow is a commonly observed flow – regime in gas – liquid flows. In this work, we theoretically find the streaming intensity produced in a confined continuous liquid phase by an acoustically oscillating stationary gas slug undergoing surface mode oscillations. We explicitly specify the interface oscillation and compute the surface mode amplitudes by solving the momentum and continuity equations. The resonant frequency is found to be consistent with that reported experimentally in the literature. The variation of streaming intensity with tube diameter is found to be identical to that reported by K. Ryu, S. K. Chung, S. K. Cho; Journal of the Association for Laboratory Automation, Vol. 15 (3), 163 (2010). Although the leading order solution is purely oscillatory in nature, the streaming intensity estimated is found to be proportional to actual streaming velocity. The effect of slug size on resonant frequency of system is also studied.
Unsupervised Cycle-Consistent Generative Adversarial Networks for Pan Sharpening

Deep learning-based pan sharpening has received significant research interest in recent years. Most of the existing methods fall into the supervised learning framework in which they downsample the multispectral (MS) and panchromatic (PAN) images and regard the original MS images as ground truths to form training samples based on Wald’s protocol. Although impressive performance could be achieved, they have difficulties when generalizing to the original full-scale images due to the scale gap, which makes them lack of practicability. In this article, we propose an unsupervised generative adversarial framework that learns from the full-scale images without the ground truths to alleviate this problem. We first extract the modality-specific features from the PAN and MS images with a two-stream generator, perform fusion in the feature domain, and then reconstruct the pan-sharpened images. Furthermore, we introduce a novel hybrid loss based on the cycle-consistency and adversarial scheme to improve the performance. Comparison experiments with the state-of-the-art methods are conducted on GaoFen-2 (GF-2) and WorldView-3 satellites. Results demonstrate that the proposed method can greatly improve the pan-sharpening performance on the full-scale images, which clearly shows its practical value. Codes are available at https://github.com/zhysora/UCGAN.
From the Beginning: Key Transitions in the First 15 Years of DNSSEC

When the global rollout of the DNS Security Extensions (DNSSEC) began in 2005, a first-of-its-kind trial started: The complexity of a core Internet protocol was magnified in favor of better security for the overall Internet. Thereby, the scale of the loosely-federated delegation in DNS became an unprecedented cryptographic key management challenge. Though fundamental for current and future operational success, our community lacks a clear notion of how to empirically evaluate the process of securely transitioning keys. In this paper, we propose two building blocks to formally characterize and assess key transitions. First, the anatomy of key transitions, i.e., measurable and well-defined properties of key changes; and second, a novel classification model based on this anatomy for describing key transition practices in abstract terms. This abstraction allows for classifying operational behavior. We apply our proposed transition anatomy and transition classes to describe the global DNSSEC deployment. Specifically, we use measurements from the first 15 years of the DNSSEC rollout to detect and understand which key transitions have been used to what degree and which rates of errors and warnings occurred. In contrast to prior work, we consider all possible transitions and not only 1:1 key rollovers. Our results show measurable gaps between prescribed key management processes and key transitions in the wild. We also find evidence that such noncompliant transitions are needed in operations.
Spatiotemporal linear stability of viscoelastic Saffman-Taylor flows

Temporal and spatiotemporal linear stability analyses of a displaced Oldroyd-B fluid in a horizontally aligned, square, Hele-Shaw cell is reported to identify regions of topological transition of the advancing interface. Viscosity of the displacing fluid is negligible in comparison to the displaced fluid. All inertial terms in the model are retained. Interface evolves on a slow timescale compared with timescale of perturbation (i.e., `quasi-stationary' approximation). Parameters governing stability are Reynolds number, $Re = \frac{b^2 \rho \mathcal{U}_0}{12 \eta_2 L}$, elasticity number, $E = \frac{12 \lambda (1-\nu)\eta_2}{\rho b^2}$, and ratio of solvent to polymer solution viscosity, $\nu = \frac{\eta_s}{\eta_2}$, where $b, L, \mathcal{U}_0, \rho, \lambda$ are cell gap, cell length/width, mean flow velocity, density of the driven fluid and polymer relaxation time, respectively. Good agreement using the relative finger width data computed with our model and the experimental data in Stokes and inertial, Newtonian regime, is found. In asymptotic limit $E(1-\nu) \ll 1$, critical Reynolds number, $Re_c$ diverges as $Re_c \sim \left[E(1-\nu)\right]^{-5/3}$ and the critical wavenumber, $\alpha_c$ increases as $\alpha_c \sim \left[E(1-\nu)\right]^{-2/3}$. In a confined domain, temporal stability analysis indicate (a) destabilizing influence of inertial terms, (b) destabilizing impact of finite boundaries near wall, and (c) stabilizing impact of elasticity until a critical Reynolds number. Briggs idea of analytic continuation is deployed to classify regions of absolute and convective instabilities, as well as evanescent modes. Phase diagram reveals the presence of absolutely unstable region at high values of Reynolds and elasticity number, confirming the role of fluid inertia in triggering a pinch-off.
SYSML: StYlometry with Structure and Multitask Learning: Implications for Darknet Forum Migrant Analysis

Darknet market forums are frequently used to exchange illegal goods and services between parties who use encryption to conceal their identities. The Tor network is used to host these markets, which guarantees additional anonymization from IP and location tracking, making it challenging to link across malicious users using multiple accounts (sybils). Additionally, users migrate to new forums when one is closed further increasing the difficulty of linking users across multiple forums. We develop a novel stylometry-based multitask learning approach for natural language and model interactions using graph embeddings to construct low-dimensional representations of short episodes of user activity for authorship attribution. We provide a comprehensive evaluation of our methods across four different darknet forums demonstrating its efficacy over the state-of-the-art, with a lift of up to 2.5X on Mean Retrieval Rank and 2X on Recall@10.