Miracle-dz/UniFAR-FTUs huggingface dataset:
Improving Multimodal Accuracy Through Modality Pre-training and Attention

Training a multimodal network is challenging and it requires complex architectures to achieve reasonable performance. We show that one reason for this phenomena is the difference between the convergence rate of various modalities. We address this by pre-training modality-specific sub-networks in multimodal architectures independently before end-to-end training of the entire network. Furthermore, we show that the addition of an attention mechanism between sub-networks after pre-training helps identify the most important modality during ambiguous scenarios boosting the performance. We demonstrate that by performing these two tricks a simple network can achieve similar performance to a complicated architecture that is significantly more expensive to train on multiple tasks including sentiment analysis, emotion recognition, and speaker trait recognition.
Many-body Majorana-like zero modes without gauge symmetry breaking

Topological superconductors represent one of the key hosts of Majorana-based topological quantum computing. Typical scenarios for one-dimensional topological superconductivity assume a broken gauge symmetry associated to a superconducting state. However, no interacting one-dimensional many-body system is known to spontaneously break gauge symmetries. Here, we show that zero modes emerge in a many-body system without gauge symmetry breaking and in the absence of superconducting order. In particular, we demonstrate that Majorana zero modes of the symmetry-broken superconducting state are continuously connected to these zero-mode excitations, demonstrating that zero-bias anomalies may emerge in the absence of gauge symmetry breaking. We demonstrate that these many-body zero modes share the robustness features of the Majorana zero modes of symmetry-broken topological superconductors. We introduce a bosonization formalism to analyze these excitations and show that a ground state analogous to a topological superconducting state can be analytically found in a certain limit. Our results demonstrate that robust Majorana-like zero modes may appear in a many-body systems without gauge symmetry breaking, thus introducing a family of protected excitations with no single-particle analogs.
Author’s Sentiment Prediction

Even though sentiment analysis has been well-studied on a wide range of domains, there hasn’tbeen much work on inferring author sentiment in news articles. To address this gap, we introducePerSenT, a crowd-sourced dataset that captures the sentiment of an author towards the mainentity in a news article. Our benchmarks of multiple strong baselines show that this is a difficultclassification task. BERT performs the best amongst the baselines. However, it only achievesa modest performance overall suggesting that fine-tuning document-level representations aloneisn’t adequate for this task. Making paragraph-level decisions and aggregating over the entiredocument is also ineffective. We present empirical and qualitative analyses that illustrate thespecific challenges posed by this dataset. We release this dataset with 5.3k documents and 38kparagraphs with 3.2k unique entities as a challenge in entity sentiment analysis.
Evaluating the Intelligibility Benefits of Neural Speech Enrichment for Listeners with Normal Hearing and Hearing Impairment using the Greek Harvard Corpus

In this work we evaluate a neural based speech intelligibility booster based on spectral shaping and dynamic range compression (SSDRC), referred to as WaveNet-based SSDRC (wSSDRC), using a recently designed Greek Harvard-style corpus. The corpus has been developed according to the format of the Harvard/IEEE sentences and offers the opportunity to apply neural speech enhancement models and examine their performance gain for Greek listeners. wSSDRC has been successfully tested for English material and speakers in the past. In this paper we revisit wSSDRC to perform a full scale evaluation of the model with Greek listeners under the condition of equal energy before and after modification. Both normal hearing (NH) and hearing impaired (HI) listeners evaluated the model under speech shaped noise (SSN) at listener-specific SNRs matching their Speech Reception Threshold (SRT) - a point at which 50 % of unmodified speech is intelligible. The analysis statistics show that the wSSDRC model has produced a median intelligibility boost of 39% for NH and 38% for HI, relative to the plain unprocessed speech.
Enabling Interactive Transcription in an Indigenous Community

We propose a novel transcription workflow which combines spoken term detection and human-in-the-loop, together with a pilot experiment. This work is grounded in an almost zero-resource scenario where only a few terms have so far been identified, involving two endangered languages. We show that in the early stages of transcription, when the available data is insufficient to train a robust ASR system, it is possible to take advantage of the transcription of a small number of isolated words in order to bootstrap the transcription of a speech collection.
I-POST: Intelligent Point of Sale and Transaction System

We propose a novel solution for the cashier problem. Current cashier system/Point of Sale (POS) terminals can be inefficient, cumbersome and time-consuming for the users. There is a need for a solution dependent on modern technology and ubiquitous computing resources. We present I-POST (Intelligent Point of Sale and Transaction) as a software system that uses smart devices, mobile phone and state of the art machine learning algorithms to process the user transactions in automated and real time manner. I-POST is an automated checkout system that allows the user to walk in a store, collect his items and exit the store. There is no need to stand and wait in a queue. The system uses object detection and facial recognition algorithm to process the authentication of the client and the state of the object. At point of exit, the classifier sends the data to the backend server which execute the payments. The system uses Convolution Neural Network (CNN) for the image recognition and processing. CNN is a supervised learning model that has found major application in pattern recognition problem. The current implementation uses two classifiers that work intrinsically to authenticate the user and track the items. The model accuracy for object recognition is 97%, the loss is 9.3%. We expect that such systems can bring efficiency to the market and has the potential for broad and diverse applications.
Towards tight bounds for spectral sparsification of hypergraphs

Cut and spectral sparsification of graphs have numerous applications, including e.g. speeding up algorithms for cuts and Laplacian solvers. These powerful notions have recently been extended to hypergraphs, which are much richer and may offer new applications. However, the current bounds on the size of hypergraph sparsifiers are not as tight as the corresponding bounds for graphs. Our first result is a polynomial-time algorithm that, given a hypergraph on n vertices with maximum hyperedge size r, outputs an є-spectral sparsifier with O*(nr) hyperedges, where O* suppresses (є−1 logn)O(1) factors. This size bound improves the two previous bounds: O*(n3) [Soma and Yoshida, SODA’19] and O*(nr3) [Bansal, Svensson and Trevisan, FOCS’19]. Our main technical tool is a new method for proving concentration of the nonlinear analogue of the quadratic form of the Laplacians for hypergraph expanders. We complement this with lower bounds on the bit complexity of any compression scheme that (1+є)-approximates all the cuts in a given hypergraph, and hence also on the bit complexity of every є-cut/spectral sparsifier. These lower bounds are based on Ruzsa-Szemerédi graphs, and a particular instantiation yields an Ω(nr) lower bound on the bit complexity even for fixed constant є. In the case of hypergraph cut sparsifiers, this is tight up to polylogarithmic factors in n, due to recent result of [Chen, Khanna and Nagda, FOCS’20]. For spectral sparsifiers it narrows the gap to O*(r). Finally, for directed hypergraphs, we present an algorithm that computes an є-spectral sparsifier with O*(n2r3) hyperarcs, where r is the maximum size of a hyperarc. For small r, this improves over O*(n3) known from [Soma and Yoshida, SODA’19], and is getting close to the trivial lower bound of Ω(n2) hyperarcs.
Tree embeddings for hop-constrained network design

Network design problems aim to compute low-cost structures such as routes, trees and subgraphs. Often, it is natural and desirable to require that these structures have small hop length or hop diameter. Unfortunately, optimization problems with hop constraints are much harder and less well understood than their hop-unconstrained counterparts. A significant algorithmic barrier in this setting is the fact that hop-constrained distances in graphs are very far from being a metric. We show that, nonetheless, hop-constrained distances can be approximated by distributions over ``partial tree metrics. '' We build this result into a powerful and versatile algorithmic tool which, similarly to classic probabilistic tree embeddings, reduces hop-constrained problems in general graphs to hop-unconstrained problems on trees. We then use this tool to give the first poly-logarithmic bicriteria approximations for the hop-constrained variants of many classic network design problems. These include Steiner forest, group Steiner tree, group Steiner forest, buy-at-bulk network design as well as online and oblivious versions of many of these problems.
World Trade Center responders in their own words: predicting PTSD symptom trajectories with AI-based language analyses of interviews

Abstract Background Oral histories from 9/11 responders to the World Trade Center (WTC) attacks provide rich narratives about distress and resilience. Artificial Intelligence (AI) models promise to detect psychopathology in natural language, but they have been evaluated primarily in non-clinical settings using social media. This study sought to test the ability of AI-based language assessments to predict PTSD symptom trajectories among responders. Methods Participants were 124 responders whose health was monitored at the Stony Brook WTC Health and Wellness Program who completed oral history interviews about their initial WTC experiences. PTSD symptom severity was measured longitudinally using the PTSD Checklist (PCL) for up to 7 years post-interview. AI-based indicators were computed for depression, anxiety, neuroticism, and extraversion along with dictionary-based measures of linguistic and interpersonal style. Linear regression and multilevel models estimated associations of AI indicators with concurrent and subsequent PTSD symptom severity (significance adjusted by false discovery rate). Results Cross-sectionally, greater depressive language (β = 0.32; p = 0.049) and first-person singular usage (β = 0.31; p = 0.049) were associated with increased symptom severity. Longitudinally, anxious language predicted future worsening in PCL scores (β = 0.30; p = 0.049), whereas first-person plural usage (β = −0.36; p = 0.014) and longer words usage (β = −0.35; p = 0.014) predicted improvement. Conclusions This is the first study to demonstrate the value of AI in understanding PTSD in a vulnerable population. Future studies should extend this application to other trauma exposures and to other demographic groups, especially under-represented minorities.
Deep Learning-Based CSI Feedback for Beamforming in Single- and Multi-Cell Massive MIMO Systems

The potentials of massive multiple-input multiple-output (MIMO) are all based on the available instantaneous channel state information (CSI) at the base station (BS). Therefore, the user in frequency-division duplexing (FDD) systems has to keep on feeding back the CSI to the BS, thereby occupying large uplink transmission resources. Recently, deep learning (DL) has achieved great success in the CSI feedback. However, the existing works just focus on improving the feedback accuracy and ignore the effects on the following modules, e.g., beamforming (BF). In this paper, we propose a DL-based CSI feedback framework for BF design, called CsiFBnet. The key idea of the CsiFBnet is to maximize the BF performance gain rather than the feedback accuracy. We apply it to two representative scenarios: single- and multi-cell systems. The CsiFBnet-s in the single-cell system is based on the autoencoder architecture, where the encoder at the user compresses the CSI and the decoder at the BS generates the BF vector. The CsiFBnet-m in the multi-cell system has to feed back two kinds of CSI: the desired and the interfering CSI. The entire neural networks are trained by an unsupervised learning strategy. Simulation results show the great performance improvement and complexity reduction of the CsiFBnet compared with the conventional DL-based CSI feedback methods.
The CUHK-TUDELFT System for The SLT 2021 Children Speech Recognition Challenge

This technical report describes our submission to the 2021 SLT Children Speech Recognition Challenge (CSRC) Track 1. Our approach combines the use of a joint CTC-attention end-to-end (E2E) speech recognition framework, transfer learning, data augmentation and development of various language models. Procedures of data pre-processing, the background and the course of system development are described. The analysis of the experiment results, as well as the comparison between the E2E and DNN-HMM hybrid system are discussed in detail. Our system achieved a character error rate (CER) of 20.1% in our designated test set, and 23.6% in the official evaluation set, which is placed at 10-th overall.
Non-local optimization: imposing structure on optimization problems by relaxation

In stochastic optimization, particularly in evolutionary computation and reinforcement learning, the optimization of a function f : Ω → R is often addressed through optimizing a so-called relaxation θ ϵ Θ → Eθ(f) of f, where Θ resembles the parameters of a family of probability measures on Ω. We investigate the structure of such relaxations by means of measure theory and Fourier analysis, enabling us to shed light on the success of many associated stochastic optimization methods. The main structural traits we derive and that allow fast and reliable optimization of relaxations are the consistency of optimal values of f, Lipschitzness of gradients, and convexity. We emphasize settings where f itself is not differentiable or convex, e.g., in the presence of (stochastic) disturbance.
An AMUSING look at the host of the periodic nuclear transient ASASSN-14ko reveals a second AGN

We present Multi-Unit Spectroscopic Explorer (MUSE) integral-field spectroscopy of ESO 253−G003, which hosts a known active galactic nucleus (AGN) and the periodic nuclear transient ASASSN-14ko, observed as part of the All-weather MUse Supernova Integral-field of Nearby Galaxies survey. The MUSE observations reveal that the inner region hosts two AGN separated by $1.4\pm 0.1~\rm {kpc}$ (≈1${_{.}^{\prime\prime}}$7). The brighter nucleus has asymmetric broad permitted emission-line profiles and is associated with the archival AGN designation. The fainter nucleus does not have a broad emission-line component but exhibits other AGN characteristics, including $\hbox{$v_{\rm {FWHM}}$} \approx 700~\hbox{km~s$^{-1}$}$ forbidden line emission, $\rm{\log _{10}(\rm{[O\,\small {III}]}/\rm{H\beta})} \approx 1.1$, and high-excitation potential emission lines, such as [Fe vii] λ6086 and He ii λ4686. The host galaxy exhibits a disturbed morphology with large kpc-scale tidal features, potential outflows from both nuclei, and a likely superbubble. A circular relativistic disc model cannot reproduce the asymmetric broad emission-line profiles in the brighter nucleus, but two non-axisymmetric disc models provide good fits to the broad emission-line profiles: an elliptical disc model and a circular disc + spiral arm model. Implications for the periodic nuclear transient ASASSN-14ko are discussed.
Parameter Optimization for Loop Closure Detection in Closed Environments

Tuning parameters is crucial for the performance of localization and mapping algorithms. In general, the tuning of the parameters requires expert knowledge and is sensitive to information about the structure of the environment. In order to design truly autonomous systems the robot has to learn the parameters automatically. Therefore, we propose a parameter optimization approach for loop closure detection in closed environments which requires neither any prior information, e.g. robot model parameters, nor expert knowledge. It relies on several path traversals along the boundary line of the closed environment. We demonstrate the performance of our method in challenging real world scenarios with limited sensing capabilities. These scenarios are exemplary for a wide range of practical applications including lawn mowers and household robots.
Optimizing Large-Scale Fleet Management on a Road Network using Multi-Agent Deep Reinforcement Learning with Graph Neural Network

We propose a novel approach to optimize fleet management by combining multi-agent reinforcement learning with graph neural network. To provide ride-hailing service, one needs to optimize dynamic resources and demands over spatial domain. While the spatial structure was previously approximated with a regular grid, our approach represents the road network with a graph, which better reflects the underlying geometric structure. Dynamic resource allocation is formulated as multi-agent reinforcement learning, whose action-value function (Q function) is approximated with graph neural networks. We use stochastic policy update rule over the graph with deep Q-networks (DQN), and achieve superior results over the greedy policy update. We design a realistic simulator that emulates the empirical taxi call data, and confirm the effectiveness of the proposed model under various conditions.
Rank-based testing for semiparametric VAR models: A measure transportation approach

We develop a class of tests for semiparametric vector autoregressive (VAR) models with unspecified innovation densities, based on the recent measure-transportation-based concepts of multivariate center-outward ranks and signs. We show that these concepts, combined with Le Cam's asymptotic theory of statistical experiments, yield novel testing procedures, which (a) are valid under a broad class of innovation densities (possibly non-elliptical, skewed, and/or with infinite moments), (b) are optimal (locally asymptotically maximin or most stringent) at selected ones, and (c) are robust against additive outliers. In order to do so, we establish a Hajek asymptotic representation result, of independent interest, for a general class of center-outward rank-based serial statistics. As an illustration, we consider the problems of testing the absence of serial correlation in multiple-output and possibly non-linear regression (an extension of the classical Durbin-Watson problem) and the sequential identification of the order p of a vector autoregressive (VAR(p)) model. A Monte Carlo comparative study of our tests and their routinely-applied Gaussian competitors demonstrates the benefits (in terms of size, power, and robustness) of our methodology; these benefits are particularly significant in the presence of asymmetric and leptokurtic innovation densities. A real data application concludes the paper.
Scalable Querying of Nested Data

While large-scale distributed data processing platforms have become an attractive target for query processing, these systems are problematic for applications that deal with nested collections. Programmers are forced either to perform non-trivial translations of collection programs or to employ automated flattening procedures, both of which lead to performance problems. These challenges only worsen for nested collections with skewed cardinalities, where both handcrafted rewriting and automated flattening are unable to enforce load balancing across partitions. In this work, we propose a framework that translates a program manipulating nested collections into a set of semantically equivalent shredded queries that can be efficiently evaluated. The framework employs a combination of query compilation techniques, an efficient data representation for nested collections, and automated skew-handling. We provide an extensive experimental evaluation, demonstrating significant improvements provided by the framework in diverse scenarios for nested collection programs.
Analysis of the Matrix Event Graph Replicated Data Type

Matrix is a new kind of decentralized, topic-based publish-subscribe middleware for communication and data storage that is getting particularly popular as a basis for secure instant messaging. By comparison with traditional decentralized communication systems, Matrix replaces pure message passing with a replicated data structure. This data structure, which we extract and call the Matrix Event Graph (MEG), depicts the causal history of messages. We show that this MEG represents an interesting and important replicated data type for decentralized applications that are based on causal histories of publish-subscribe events: First, we prove that the MEG is a Conflict-Free Replicated Data Type for causal histories and, thus, provides Strong Eventual Consistency (SEC). With SEC being among the best known achievable trade-offs in the scope of the well-known CAP theorem, the MEG provides a powerful consistency guarantee while being available during network partition. Second, we discuss the implications of byzantine attackers on the data type’s properties. We note that the MEG, as it does not strive for consensus or strong consistency, can cope with <inline-formula> <tex-math notation="LaTeX">$n > f$ </tex-math></inline-formula> environments with <inline-formula> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula> participants, of which <inline-formula> <tex-math notation="LaTeX">$f$ </tex-math></inline-formula> are byzantine. Furthermore, we analyze scalability: Using Markov chains, we study the number of forward extremities of the MEG over time and observe an almost optimal evolution. We conjecture that this property is inherent to the underlying spatially inhomogeneous random walk. With the properties shown, a MEG represents a promising element in the set of data structures for decentralized applications, but with distinct trade-offs compared to traditional blockchains and distributed ledger technologies.
Further evidence to support a tidal disruption event in the changing-look AGN SDSS J0159

In this Letter, we report further evidence to support a tidal disruption event (TDE) in the known changing-look AGN SDSS J0159, through the unique variabilities of asymmetric broad H$\alpha$: broader but redder in 2010 than in 2000. Accepted the broad H$\alpha$ emission regions tightly related to accreting debris in a central TDE, the well-known relativistic accretion disk origination can be applied to well explain the asymmetric broad H$\alpha$ variabilities in SDSS J0159. Moreover, the model determined broad H$\alpha$ emission regions have the sizes not follow the empirical R-L relation valid in normal broad line AGN, but have locations basically similar to the regions of accreting debris from a central TDE in SDSS J0159, indicating tight connections between the broad H$\alpha$ emission materials and central TDE debris. Therefore, explanations of the asymmetric broad H$\alpha$ variabilities through the relativistic accretion disk origination provide further clues to support a central TDE in SDSS J0159.