Reviewerly/exHarmony huggingface dataset:
Transformer-based Streaming ASR with Cumulative Attention

In this paper, we propose an online attention mechanism, known as cumulative attention (CA), for streaming Transformer-based automatic speech recognition (ASR). Inspired by monotonic chunkwise attention (MoChA) and head-synchronous decoder-end adaptive computation steps (HS-DACS) algorithms, CA triggers the ASR outputs based on the acoustic information accumulated at each encoding timestep, where the decisions are made using a trainable device, referred to as halting selector. In CA, all the attention heads of the same decoder layer are synchronised to have a unified halting position. This feature effectively alleviates the problem caused by the distinct behaviour of individual heads, which may otherwise give rise to severe latency issues as encountered by MoChA. The ASR experiments conducted on AIShell-1 and Librispeech datasets demonstrate that the proposed CA-based Transformer system can achieve on par or better performance with significant reduction in latency during inference, when compared to other streaming Transformer systems in literature.
State Preparation Boosters for Early Fault-Tolerant Quantum Computation

Quantum computing is believed to be particularly useful for the simulation of chemistry and materials, among the various applications. In recent years, there have been significant advancements in the development of near-term quantum algorithms for quantum simulation, including VQE and many of its variants. However, for such algorithms to be useful, they need to overcome several critical barriers including the inability to prepare high-quality approximations of the ground state. Current challenges to state preparation, including barren plateaus and the high-dimensionality of the optimization landscape, make state preparation through ansatz optimization unreliable. In this work, we introduce the method of ground state boosting, which uses a limited-depth quantum circuit to reliably increase the overlap with the ground state. This circuit, which we call a booster, can be used to augment an ansatz from VQE or be used as a stand-alone state preparation method. The booster converts circuit depth into ground state overlap in a controllable manner. We numerically demonstrate the capabilities of boosters by simulating the performance of a particular type of booster, namely the Gaussian booster, for preparing the ground state of $N_2$ molecular system. Beyond ground state preparation as a direct objective, many quantum algorithms, such as quantum phase estimation, rely on high-quality state preparation as a subroutine. Therefore, we foresee ground state boosting and similar methods as becoming essential algorithmic components as the field transitions into using early fault-tolerant quantum computers.
Limits on atomic qubit control from laser noise

Technical noise present in laser systems can limit their ability to perform high fidelity quantum control of atomic qubits. The ultimate fidelity floor for atomic qubits driven with laser radiation is due to spontaneous emission from excited energy levels. The goal is to suppress the technical noise from the laser source to below the spontaneous emission floor such that it is no longer a limiting factor. It has been shown that the spectral structure of control noise can have a large influence on achievable control fidelities, while prior studies of laser noise contributions have been restricted to noise magnitudes. Here, we study the unique spectral structure of laser noise and introduce a new metric that determines when a stabilised laser source has been optimised for quantum control of atomic qubits. We find requirements on stabilisation bandwidths that can be orders of magnitude higher than those required to simply narrow the linewidth of a laser. We introduce a new metric, the $\chi$-separation line, that provides a tool for the study and engineering of laser sources for quantum control of atomic qubits below the spontaneous emission floor.
EDUCATION DURING A PANDEMIC

The paper describes the challenges of education during a pandemic. The transition to distance learning has had an impact on all of the participants of the education process. The purpose of this article is to study the social consequences and education outcomes of the work of the subjects of the educational process in the format of distance learning during 2020–2021. To achieve the goal the author used an interdisciplinary approach, emplying pedagogical and psychological research methods: observation, questioning, interviewing. It was revealed that distance learning format is characterized by the following features: flexible schedule, in which the student studies at a convenient time, quality management of the educational process, innovative learning technologies, and a modular system of learning. The study describes the positive and negative aspects of the organization of distance learning. The study concluded that during the coronavirus pandemic, it is necessary to take a set of measures for the socialization of all the participants in the virtual education space, to create the centers to improve the skills of teachers in the field of infocommunication technologies, and to organize continuing development courses to enhance professional pedagogical communication in a remote learning format.
Consistency of the maximum likelihood estimator in hidden Markov models with trends

A hidden Markov model with trends is a hidden Markov model whose emission distributions are translated by a trend that depends on the current hidden state and on the current time. Contrary to standard hidden Markov models, such processes are not homogeneous and cannot be made homogeneous by a simple de-trending step. We show that when the trends are polynomial, the maximum likelihood estimator is able to recover the trends together with the other parameters and is strongly consistent. More precisely, the supremum norm of the difference between the true trends and the estimated ones tends to zero. Numerical properties of the maximum likelihood estimator are assessed by a simulation study.
Maximum Entropy on Erroneous Predictions (MEEP): Improving model calibration for medical image segmentation

Modern deep neural networks achieved remarkable progress in medical image segmentation tasks. However, it has recently been observed that they tend to produce overconfident estimates, even in situations of high uncertainty, leading to poorly calibrated and unreliable models. In this work we introduce Maximum Entropy on Erroneous Predictions (MEEP), a training strategy for segmentation networks which selectively penalizes overconfident predictions, focusing only on misclassified pixels. Our method is agnostic to the neural architecture, does not increase model complexity and can be coupled with multiple segmentation loss functions. We benchmark the proposed strategy in two challenging segmentation tasks: white matter hyperintensity lesions in magnetic resonance images (MRI) of the brain, and atrial segmentation in cardiac MRI. The experimental results demonstrate that coupling MEEP with standard segmentation losses leads to improvements not only in terms of model calibration, but also in segmentation quality.
Hoax Detection Through Analysis of Modality: A Systemic - Functional Linguistics Study

Speakers' attitude which came from their doubts toward truth of an information influences the modality selection used.This study offered a step of hoax detection in a news text by paying attention on the use of its modality.This study was focused on the modality type, value, and orientation in Hoax Report of Legislative and President Election Internet Content Control Subdit for the period of August 2018 by Ministry of Communication and Information of Republic Indonesia which consisted of 60 news with a total of 284 clauses.The news was analyzed by using systemic functional linguistic approach with descriptive qualitative method using percentage technique.The study showed the type of modulation modality dominated in hoax; Based on the study, the modality dominating were middle degree obligation modulation and middle degree modalization probability, meanwhile middle degree modalization usuality and high degree of modulation inclination was rarely used; the modality orientation showed that the hoax revealed the involvement of suspect subjectively and used modality markers explicitly.Therefore, modality analysis through a functional systemic linguistic approach in a news text can be used as initial step of hoax detection..
Degree 2 Boolean Functions on Grassmann Graphs

We investigate the existence of Boolean degree $d$ functions on the Grassmann graph of $k$-spaces in the vector space $\mathbb{F}_q^n$. For $d=1$ several non-existence and classification results are known, and no non-trivial examples are known for $n \geq 5$. This paper focusses on providing a list of examples on the case $d=2$ in general dimension and in particular for $(n, k)=(6,3)$ and $(n,k) = (8, 4)$. We also discuss connections to the analysis of Boolean functions, regular sets/equitable bipartitions/perfect 2-colorings in graphs, $q$-analogs of designs, and permutation groups. In particular, this represents a natural generalization of Cameron-Liebler line classes.
Robustness of a universal gate set implementation in transmon systems via Chopped Random Basis optimal control

We numerically study the implementation of a universal two-qubit gate set, composed of CNOT, Hadamard, phase and π/8 gates, for transmon-based systems. The control signals to implement such gates are obtained using the Chopped Random Basis optimal control technique, with a target gate infidelity of 10−2. During the optimization processes we account for the leakage toward non-computational states, an important non-ideality affecting transmon qubits. We also test and benchmark the optimal control solutions against the introduction of Gaussian white noise and spectral distortion, two key non-idealities that affect the control signals in transmon systems.
Story Point Effort Estimation by Text Level Graph Neural Network

Estimating the software projects' efforts developed by agile methods is important for project managers or technical leads. It provides a summary as a first view of how many hours and developers are required to complete the tasks. There are research works on automatic predicting the software efforts, including Term Frequency Inverse Document Frequency (TFIDF) as the traditional approach for this problem. Graph Neural Network is a new approach that has been applied in Natural Language Processing for text classification. The advantages of Graph Neural Network are based on the ability to learn information via graph data structure, which has more representations such as the relationships between words compared to approaches of vectorizing sequence of words. In this paper, we show the potential and possible challenges of Graph Neural Network text classification in story point level estimation. By the experiments, we show that the GNN Text Level Classification can achieve as high accuracy as about 80 percent for story points level classification, which is comparable to the traditional approach. We also analyze the GNN approach and point out several current disadvantages that the GNN approach can improve for this problem or other problems in software engineering.
A compound Poisson perspective of Ewens-Pitman sampling model

The Ewens-Pitman sampling model (EP-SM) is a distribution for random partitions of the set $\{1,\ldots,n\}$, with $n\in\mathbb{N}$, which is index by real parameters $\alpha$ and $\theta$ such that either $\alpha\in[0,1)$ and $\theta>-\alpha$, or $\alpha<0$ and $\theta=-m\alpha$ for some $m\in\mathbb{N}$. For $\alpha=0$ the EP-SM reduces to the celebrated Ewens sampling model (E-SM), which admits a well-known compound Poisson perspective in terms of the log-series compound Poisson sampling model (LS-CPSM). In this paper, we consider a generalization of the LS-CPSM, which is referred to as the negative Binomial compound Poisson sampling model (NB-CPSM), and we show that it leads to extend the compound Poisson perspective of the E-SM to the more general EP-SM for either $\alpha\in(0,1)$, or $\alpha<0$. The interplay between the NB-CPSM and the EP-SM is then applied to the study of the large $n$ asymptotic behaviour of the number of blocks in the corresponding random partitions, leading to a new proof of Pitman's $\alpha$ diversity. We discuss the proposed results, and conjecture that analogous compound Poisson representations may hold for the class of $\alpha$-stable Poisson-Kingman sampling models, of which the EP-SM is a noteworthy special case.
Generative Modeling Helps Weak Supervision (and Vice Versa)

Many promising applications of supervised machine learning face hurdles in the acquisition of labeled data in sufficient quantity and quality, creating an expensive bottleneck. To overcome such limitations, techniques that do not depend on ground truth labels have been studied, including weak supervision and generative modeling. While these techniques would seem to be usable in concert, improving one another, how to build an interface between them is not well-understood. In this work, we propose a model fusing programmatic weak supervision and generative adversarial networks and provide theoretical justification motivating this fusion. The proposed approach captures discrete latent variables in the data alongside the weak supervision derived label estimate. Alignment of the two allows for better modeling of sample-dependent accuracies of the weak supervision sources, improving the estimate of unobserved labels. It is the first approach to enable data augmentation through weakly supervised synthetic images and pseudolabels. Additionally, its learned latent variables can be inspected qualitatively. The model outperforms baseline weak supervision label models on a number of multiclass image classification datasets, improves the quality of generated images, and further improves end-model performance through data augmentation with synthetic samples.
Quantum Circuit Transformations with a Multi-Level Intermediate Representation Compiler

Quantum computing promises remarkable approaches for processing information, but new tools are needed to compile program representations into the physical instructions required by a quantum computer. Here we present a novel adaptation of the multi-level intermediate representation (MLIR) integrated into a quantum compiler that may be used for checking program execution. We first present how MLIR enables quantum circuit transformations for efficient execution on quantum computing devices and then give an example of compiler transformations based on so-called mirror circuits. We demonstrate that mirror circuits inserted during compilation may test hardware performance by assessing quantum circuit accuracy on several superconducting and ion trap hardware platforms. Our results validate MLIR as an efficient and effective method for collecting hardware-dependent diagnostics through automated transformations of quantum circuits.
FedGCN: Convergence-Communication Tradeoffs in Federated Training of Graph Convolutional Networks

Methods for training models on graphs distributed across multiple clients have recently grown in popularity, due to the size of these graphs as well as regulations on keeping data where it is generated. However, the cross-client edges naturally exist among clients. Thus, distributed methods for training a model on a single graph incur either significant communication overhead between clients or a loss of available information to the training. We introduce the Federated Graph Convolutional Network (FedGCN) algorithm, which uses federated learning to train GCN models for semi-supervised node classification with fast convergence and little communication. Compared to prior methods that require extra communication among clients at each training round, FedGCN clients only communicate with the central server in one pre-training step, greatly reducing communication costs and allowing the use of homomorphic encryption to further enhance privacy. We theoretically analyze the tradeoff between FedGCN's convergence rate and communication cost under different data distributions. Experimental results show that our FedGCN algorithm achieves better model accuracy with 51.7% faster convergence on average and at least 100X less communication compared to prior work.
Modality-Aware Triplet Hard Mining for Zero-shot Sketch-Based Image Retrieval

This paper tackles the Zero-Shot Sketch-Based Image Retrieval (ZS-SBIR) problem from the viewpoint of cross-modality metric learning. This task has two characteristics: 1) the zero-shot setting requires a metric space with good within-class compactness and the between-class discrepancy for recognizing the novel classes and 2) the sketch query and the photo gallery are in different modalities. The metric learning viewpoint benefits ZS-SBIR from two aspects. First, it facilitates improvement through recent good practices in deep metric learning (DML). By combining two fundamental learning approaches in DML, e.g., classification training and pairwise training, we set up a strong baseline for ZS-SBIR. Without bells and whistles, this baseline achieves competitive retrieval accuracy. Second, it provides an insight that properly suppressing the modality gap is critical. To this end, we design a novel method named Modality-Aware Triplet Hard Mining (MATHM). MATHM enhances the baseline with three types of pairwise learning, e.g., a cross-modality sample pair, a within-modality sample pair, and their combination.\We also design an adaptive weighting method to balance these three components during training dynamically. Experimental results confirm that MATHM brings another round of significant improvement based on the strong baseline and sets up new state-of-the-art performance. For example, on the TU-Berlin dataset, we achieve 47.88+2.94% mAP@all and 58.28+2.34% Prec@100. Code will be publicly available at: https://github.com/huangzongheng/MATHM.
On the Effectiveness of Interpretable Feedforward Neural Network

Deep learning models have achieved state-of-the-art performance in many classification tasks. However, most of them cannot provide an interpretation for their classification results. Machine learning models that are interpretable are usually linear or piecewise linear and yield inferior performance. Non-linear models achieve much better classification performance, but it is hard to interpret their classification results. This may have been changed by an interpretable feedforward neural network (IFFNN) proposed that achieves both high classification performance and interpretability for malware detection. If the IFFNN can perform well in a more flexible and general form for other classification tasks while providing meaningful interpretations, it may be of great interest to the applied machine learning community. In this paper, we propose a way to generalize the interpretable feedforward neural network to multi-class classification scenarios and any type of feedforward neural networks, and evaluate its classification performance and interpretability on intrinsic interpretable datasets. We conclude by finding that the generalized IFFNNs achieve comparable classification performance to their normal feedforward neural network counterparts and provide meaningful interpretations. Thus, this kind of neural network architecture has great practical use.
A NEW PARADIGM OF THE MODERN VIRTUAL ARTISTIC EDUCATION FROM OWN EXPERIENCE

The purpose of the work – the study of modern human educational evolution, with its eternal problem – what is a student? Is it just a biological creature, physically limited by skin and intellectually limited by the contents of the skull? Researching the topic, the author sought to take into account the peculiarities of the mental state of the modern young generation, which is totally dependent on digital technology. The emphasis is on the unexpected expectation of a meeting between a person (teacher) and a machine (student) at the point of obtaining artistic knowledge.The methodology is to transcend the essence of the student in the field of relations with teachers. Studying the role of the body as a tool to achieve the learning goal. Comparison of the tools of a living human teacher, which is in front of the monitor screen and the screen image of a student who is deprived of natural tools (hearing, sight, touch, etc.). Finding the equivalent of transferring virtual artistic knowledge to the other side of the screen, where the receiver is a digital image of the student. The invention of a new way to return the organic tools of the human body and its components. Scientific novelty in finding the root cause of the rejection in the human body of the ability of natural self-organization in modern digitalization and the reasons for drawing human consciousness as a source of energy in the off-screen mirror of the virtual art process. A new reading of the primary and analytical author's sources of mechanisms for instilling the primordial human awareness of the essence of the acquired knowledge as a transformative evolutionary educational bridge to future generations. Using one's own pedagogical artistic experience as a practical tool for implementing the proposed topic. Conclusions: What exactly will give rise to the transition and long stay of human consciousness where there is no language of the human body, which identifies our state in the living space of art and education? The final result of the possible synthesis of the primary source of human tools and the tools of the screen image of the virtual student is impressive. What to expect if we find ourselves in front of a horrible face of a robot machine that will dictate its own algorithms for obtaining virtual knowledge? Defining human quality as a subject of virtual new knowledge in the period of distance learning.
HLT-NUS SUBMISSION FOR 2020 NIST Conversational Telephone Speech SRE

This work provides a brief description of Human Language Technology (HLT) Laboratory, National University of Singapore (NUS) system submission for 2020 NIST conversational telephone speech (CTS) speaker recognition evaluation (SRE). The challenge focuses on evaluation under CTS data containing multilingual speech. The systems developed at HLT-NUS consider time-delay neural network (TDNN) x-vector and ECAPA-TDNN systems. We also perform domain adaption of probabilistic linear discriminant analysis (PLDA) model and adaptive s-norm on our systems. The score level fusion of TDNN x-vector and ECAPA-TDNN systems is carried out, which improves the final system performance of our submission to 2020 NIST CTS SRE.
Are Big Recommendation Models Fair to Cold Users?

Big models are widely used by online recommender systems to boost recommendation performance. They are usually learned on historical user behavior data to infer user interest and predict future user behaviors (e.g., clicks). In fact, the behaviors of heavy users with more historical behaviors can usually provide richer clues than cold users in interest modeling and future behavior prediction. Big models may favor heavy users by learning more from their behavior patterns and bring unfairness to cold users. In this paper, we study whether big recommendation models are fair to cold users. We empirically demonstrate that optimizing the overall performance of big recommendation models may lead to unfairness to cold users in terms of performance degradation. To solve this problem, we propose a BigFair method based on self-distillation, which uses the model predictions on original user data as a teacher to regularize predictions on augmented data with randomly dropped user behaviors, which can encourage the model to fairly capture interest distributions of heavy and cold users. Experiments on two datasets show that BigFair can effectively improve the performance fairness of big recommendation models on cold users without harming the performance on heavy users.