Reviewerly/exHarmony huggingface dataset:
Transfer Learning Approaches for Streaming End-to-End Speech Recognition System

Transfer learning (TL) is widely used in conventional hybrid automatic speech recognition (ASR) system, to transfer the knowledge from source to target language. TL can be applied to end-to-end (E2E) ASR system such as recurrent neural network transducer (RNN-T) models, by initializing the encoder and/or prediction network of the target language with the pre-trained models from source language. In the hybrid ASR system, transfer learning is typically done by initializing the target language acoustic model (AM) with source language AM. Several transfer learning strategies exist in the case of the RNN-T framework, depending upon the choice of the initialization model for encoder and prediction networks. This paper presents a comparative study of four different TL methods for RNN-T framework. We show ~10%-17% relative word error rate reduction with different TL methods over randomly initialized RNN-T model. We also study the impact of TL with varying amount of training data ranging from 50 hours to 1000 hours and show the efficacy of TL for languages with a very small amount of training data.
PerSpectron: Detecting Invariant Footprints of Microarchitectural Attacks with Perceptron

Detecting microarchitectural attacks is critical given their proliferation in recent years. Many of these attacks exhibit intrinsic behaviors essential to the nature of their operation, such as creating contention or misspeculation. This study systematically investigates the microarchitectural footprints of hardware-based attacks and shows how they can be detected and classified using an efficient hardware predictor. We present a methodology to use correlated microarchitectural statistics to design a hardware-based neural predictor capable of detecting and classifying microarchitectural attacks before data is leaked. Once a potential attack is detected, it can be proactively mitigated by triggering appropriate countermeasures.Our hardware-based detector, PerSpectron, uses perceptron learning to identify and classify attacks. Perceptron-based prediction has been successfully used in branch prediction and other hardware-based applications. PerSpectron has minimal performance overhead. The statistics being monitored have similar overhead to already existing performance monitoring counters. Additionally, PerSpectron operates outside the processor's critical paths, offering security without added computation delay. Our system achieves a usable detection rate for detecting attacks such as SpectreV1, SpectreV2, SpectreRSB, Meltdown, breakingKSLR, Flush+Flush, Flush+Reload, Prime+Probe as well as cache-attack calibration programs. We also believe that the large number of diverse microarchitectural features offers both evasion resilience and interpretability-features not present in previous hardware security detectors. We detect these attacks early enough to avoid any data leakage, unlike previous work that triggers countermeasures only after data has been exposed.
Quantum key establishment via a multimode fiber

Quantum communication aims to provide absolutely secure transmission of secret information. State-of-the-art methods encode symbols into single photons or coherent light with much less than one photon on average. For long-distance communication, typically a single-mode fiber is used and significant effort has been devoted already to increase the data carrying capacity of a single optical line. Here we propose and demonstrate a fundamentally new concept for remote key establishment. Our method allows high-dimensional alphabets using spatial degrees of freedom by transmitting information through a light-scrambling multimode fiber and exploiting the no-cloning theorem. Eavesdropper attacks can be detected without using randomly switched mutually unbiased bases. We prove the security against a common class of intercept-resend and beam-splitting attacks with single-photon Fock states and with weak coherent light. Since it is optical fiber based, our method allows to naturally extend secure communication to larger distances. We experimentally demonstrate this new type of key exchange method by encoding information into a few-photon light pulse decomposed over guided modes of an easily available multimode fiber.
Extractive Summarization of Telugu Documents using TextRank Algorithm

Reading large and lengthy documents is a tedious and time-consuming task. A summary of the same document gives us an overall idea of what the document is all about. Automated summaries can be generated using various algorithms. The summary can be generated for single or multiple document inputs. Here the proposed system carries out extractive summarization of multiple Telugu text documents. The algorithm applied here is text rank algorithm.
SCOP: Scientific Control for Reliable Neural Network Pruning

This paper proposes a reliable neural network pruning algorithm by setting up a scientific control. Existing pruning methods have developed various hypotheses to approximate the importance of filters to the network and then execute filter pruning accordingly. To increase the reliability of the results, we prefer to have a more rigorous research design by including a scientific control group as an essential part to minimize the effect of all factors except the association between the filter and expected network output. Acting as a control group, knockoff feature is generated to mimic the feature map produced by the network filter, but they are conditionally independent of the example label given the real feature map. We theoretically suggest that the knockoff condition can be approximately preserved given the information propagation of network layers. Besides the real feature map on an intermediate layer, the corresponding knockoff feature is brought in as another auxiliary input signal for the subsequent layers. Redundant filters can be discovered in the adversarial process of different features. Through experiments, we demonstrate the superiority of the proposed algorithm over state-of-the-art methods. For example, our method can reduce 57.8% parameters and 60.2% FLOPs of ResNet-101 with only 0.01% top-1 accuracy loss on ImageNet. The code is available at this https URL
Telescoping Density-Ratio Estimation

Density-ratio estimation via classification is a cornerstone of unsupervised learning. It has provided the foundation for state-of-the-art methods in representation learning and generative modelling, with the number of use-cases continuing to proliferate. However, it suffers from a critical limitation: it fails to accurately estimate ratios p/q for which the two densities differ significantly. Empirically, we find this occurs whenever the KL divergence between p and q exceeds tens of nats. To resolve this limitation, we introduce a new framework, telescoping density-ratio estimation (TRE), that enables the estimation of ratios between highly dissimilar densities in high-dimensional spaces. Our experiments demonstrate that TRE can yield substantial improvements over existing single-ratio methods for mutual information estimation, representation learning and energy-based modelling.
Investigating Self-Attention Network for Chinese Word Segmentation

Neural network has become the dominant method for Chinese word segmentation. Most existing models cast the task as sequence labeling, using BiLSTM-CRF for representing the input, and making output predictions. Recently, attention-based sequence models have emerged as a highly competitive alternative to LSTMs, which allow better running speed by parallelization of computation. We investigate self-attention network (SAN) for Chinese word segmentation, making comparisons between BiLSTM-CRF models. In addition, the influence of contextualized character embeddings is investigated using BERT, and a method is proposed for integrating word information into SAN segmentation. Results show that SAN gives highly competitive results compared with BiLSTMs, with BERT, and word information further improving segmentation for in-domain, and cross-domain segmentation. Our final models give the best results for 6 heterogenous domain benchmarks.
An Efficient Lightweight CNN and Ensemble Machine Learning Classification of Prostate Tissue Using Multilevel Feature Analysis

Prostate carcinoma is caused when cells and glands in the prostate change their shape and size from normal to abnormal. Typically, the pathologist’s goal is to classify the staining slides and differentiate normal from abnormal tissue. In the present study, we used a computational approach to classify images and features of benign and malignant tissues using artificial intelligence (AI) techniques. Here, we introduce two lightweight convolutional neural network (CNN) architectures and an ensemble machine learning (EML) method for image and feature classification, respectively. Moreover, the classification using pre-trained models and handcrafted features was carried out for comparative analysis. The binary classification was performed to classify between the two grade groups (benign vs. malignant) and quantile-quantile plots were used to show their predicted outcomes. Our proposed models for deep learning (DL) and machine learning (ML) classification achieved promising accuracies of 94.0% and 92.0%, respectively, based on non-handcrafted features extracted from CNN layers. Therefore, these models were able to predict nearly perfectly accurately using few trainable parameters or CNN layers, highlighting the importance of DL and ML techniques and suggesting that the computational analysis of microscopic anatomy will be essential to the future practice of pathology.
Teachers' gaze over space and time in a real-world classroom

Reading students’ faces and their body language, checking their worksheets, and keeping eye contact is a key trait of teacher competence. The new technology of mobile eye-tracking provides researchers with possibilities to explore teaching from the viewpoint of teacher gaze, but also introduces many new method questions. This study had the primary aim to investigate teachers´ attention distribution over space: the number and durations of several types of their gazes, and how their gaze depends on the factors of students´ gender, achievement, and position in the classroom. Results show that teacher gaze was distributed unevenly across both space and time. Teachers looked at the most-watched students 3-8 times more often than at the least-watched ones. Students sitting in the first row and the middle section received significantly more gaze than those sitting outside this zone. All three teachers made more single gaze visits - looking at the students but making no eye contact - than mutual gazes or student material gazes. The three teachers’ gaze distribution also varied substantially from lesson to lesson. Our results are important for understanding teacher behavior in real classrooms, but also point to the relevance of appropriate method design in future classroom studies with eye-tracking.
OS2D: One-Stage One-Shot Object Detection by Matching Anchor Features

In this paper, we consider the task of one-shot object detection, which consists in detecting objects defined by a single demonstration. Differently from the standard object detection, the classes of objects used for training and testing do not overlap. We build the one-stage system that performs localization and recognition jointly. We use dense correlation matching of learned local features to find correspondences, a feed-forward geometric transformation model to align features and bilinear resampling of the correlation tensor to compute the detection score of the aligned features. All the components are differentiable, which allows end-to-end training. Experimental evaluation on several challenging domains (retail products, 3D objects, buildings and logos) shows that our method can detect unseen classes (e.g., toothpaste when trained on groceries) and outperforms several baselines by a significant margin. Our code is available online: https://github.com/aosokin/os2d .
Dataset Artefacts in Anti-Spoofing Systems: A Case Study on the ASVspoof 2017 Benchmark

The Automatic Speaker Verification Spoofing and Countermeasures Challenges motivate research in protecting speech biometric systems against a variety of different access attacks. The 2017 edition focused on replay spoofing attacks, and involved participants building and training systems on a provided dataset (ASVspoof 2017). More than 60 research papers have so far been published with this dataset, but none have sought to answer why countermeasures appear successful in detecting spoofing attacks. This article shows how artefacts inherent to the dataset may be contributing to the apparent success of published systems. We first inspect the ASVspoof 2017 dataset and summarize various artefacts present in the dataset. Second, we demonstrate how countermeasure models can exploit these artefacts to appear successful in this dataset. Third, for reliable and robust performance estimates on this dataset we propose discarding nonspeech segments and silence before and after the speech utterance during training and inference. We create speech start and endpoint annotations in the dataset and demonstrate how using them helps countermeasure models become less vulnerable from being manipulated using artefacts found in the dataset. Finally, we provide several new benchmark results for both frame-level and utterance-level models that can serve as new baselines on this dataset.
About a criterion of successfully executing a circuit in the NISQ era: what <i>wd</i> ≪ 1/𝜖 <sub> <i>eff</i> </sub> really means

To evaluate classical software, a huge variety of software metrics exists. Similar metrics of quantum algorithms especially in context of near-term quantum computers are only rudimentary investigated. Hereby, metrics are particularly important to be able to estimate what is already possible with current quantum computers. Thus, in this paper, we discuss existing quantum performance metrics and focus on a metric that determines whether a quantum circuit is successfully executable on a given gate-based quantum computer. Thereby, we give an overview of various factors that can affect an execution and present our plan to concretize and apply the metric.
SongMASS: Automatic Song Writing with Pre-training and Alignment Constraint

Automatic song writing aims to compose a song (lyric and/or melody) by machine, which is an interesting topic in both academia and industry. In automatic song writing, lyric-to-melody generation and melody-to-lyric generation are two important tasks, both of which usually suffer from the following challenges: 1) the paired lyric and melody data are limited, which affects the generation quality of the two tasks, considering a lot of paired training data are needed due to the weak correlation between lyric and melody; 2) Strict alignments are required between lyric and melody, which relies on specific alignment modeling. In this paper, we propose SongMASS to address the above challenges, which leverages masked sequence to sequence (MASS) pre-training and attention based alignment modeling for lyric-to-melody and melody-to-lyric generation. Specifically, 1) we extend the original sentence-level MASS pre-training to song level to better capture long contextual information in music, and use a separate encoder and decoder for each modality (lyric or melody); 2) we leverage sentence-level attention mask and token-level attention constraint during training to enhance the alignment between lyric and melody. During inference, we use a dynamic programming strategy to obtain the alignment between each word/syllable in lyric and note in melody. We pre-train SongMASS on unpaired lyric and melody datasets, and both objective and subjective evaluations demonstrate that SongMASS generates lyric and melody with significantly better quality than the baseline method.
Self-Supervised Pose Adaptation for Cross-Domain Image Animation

Image animation is to animate a still image of the object of interest using poses extracted from another video sequence. Through training on a large-scale video dataset, most existing approaches aim to explore disentangled appearance and pose representations of training frames. Then, the desired output with a specific appearance and pose can be synthesized via recombining learned representations. However, in some real-world applications, test images may lack the corresponding video ground-truth or follow a different distribution than the distribution of the training video frames (i.e., different domains), which largely limit the performance of existing methods. In this paper, we propose domain-independent pose representations that are compatible with and accessible by still images from a different domain. Specifically, we devise a two-stage self-supervised pose adaptation framework for general image animation tasks. A domain-independent pose adaptation generative adversarial network (DIPA-GAN) and a shuffle-patch generative adversarial network (Shuffle-patch GAN) are proposed to penalize the rationality of the synthesized frame's pose and appearance, respectively. Finally, experiments evaluated on various image animation tasks, which include same/cross-domain moving objects, facial expression transfer and human pose retargeting, demonstrate the superiority of the proposed framework over prior literature. Impact Statement —Image animation is a popular technology in video production. Benefiting from the rapid development of artificial intelligence (AI), recent image animation algorithms have been widely used in real-world applications, such as virtual AI news anchor, virtual try-on, and face swapping. However, most existing methods are designed for specific cases. To animate a new portrait, users are asked to collect hundreds of images of the same person and train a new model. The technology proposed in this paper overcomes these training limitations and generalizes image animations. In the challenging cross-domain facial expression transfer task, the user study demonstrated that our technology achieved more than 20% increase in animation success rate. The proposed technology could benefit users in a wide variety of industries including movie production, virtual reality, social media and online retail.
Aspect-based Document Similarity for Research Papers

Traditional document similarity measures provide a coarse-grained distinction between similar and dissimilar documents.Typically, they do not consider in what aspects two documents are similar.This limits the granularity of applications like recommender systems that rely on document similarity.In this paper, we extend similarity with aspect information by performing a pairwise document classification task.We evaluate our aspect-based document similarity approach for research papers.Paper citations indicate the aspect-based similarity, i. e., the title of a section in which a citation occurs acts as a label for the pair of citing and cited paper.We apply a series of Transformer models such as RoBERTa, ELECTRA, XLNet, and BERT variations and compare them to an LSTM baseline.We perform our experiments on two newly constructed datasets of 172,073 research paper pairs from the ACL Anthology and CORD-19 corpus.According to our results, SciBERT is the best performing system with F1-scores of up to 0.83.A qualitative analysis validates our quantitative results and indicates that aspect-based document similarity indeed leads to more fine-grained recommendations.
Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classification

A recent approach for few-shot text classification is to convert textual inputs to cloze questions that contain some form of task description, process them with a pretrained language model and map the predicted words to labels. Manually defining this mapping between words and labels requires both domain expertise and an understanding of the language model’s abilities. To mitigate this issue, we devise an approach that automatically finds such a mapping given small amounts of training data. For a number of tasks, the mapping found by our approach performs almost as well as hand-crafted label-to-word mappings.
Multi-Subdomain Grouping-Based Particle Swarm Optimization for the Traveling Salesman Problem

Traveling salesman problem (TSP) has been widely applied to various fields of production and life. In order to solve the TSP, this work optimizes the architecture of the particle swarm optimization (PSO) by introducing a preprocessing multi-subdomain grouping approach to divide the entire search space into several sub-regions, based on the geographical center of planned cities. To solve subdomain problems, the global search ability of PSO is further improved using genetic mutation and particle competition mechanism to maintain the population diversity during late iterations. An effective simplified 4-opt is finally employed to eliminate path-crossing after connecting all subdomain paths in a time saving way. Comparison with three of our previous algorithms and seven algorithms from other reports reveals that the proposed algorithm shows good performance in both the computing efficiency and route quality. Particularly, it is suitable for small (or medium)-sized TSP problems with relatively percentage error values below 4.8%.
Neural networks and microelectronics parameters distribution measurements depending on sintering temperature and applied voltage

This research is based on the idea to design the interface structure around the grains and thin layers between two grains, as a possible solution for deep microelectronic parameters integrations. The experiments have been based on nano-BaTiO 3 powders with Y-based additive. The advanced idea is to create the new observed directions to network microelectronic characteristics in thin films coated around and between the grains on the way to get and compare with global results on the samples. Biomimetic similarities are artificial neural networks which could be original method and tools that we use to map input–output data and could be applied on ceramics microelectronic parameters. This mapping is developed in the manner like signals that are processed in real biological neural networks. These signals are processed by using artificial neurons, which have a simple function to process input signal, as well as adjustable parameter which represents sensitivity to inputs. The integrated network output presents practically the large number of inner neurons outputs sum. This original idea is to connect analysis results and neural networks. It is of the great importance to connect microcapacitances by neural network with the goal to compare the experimental results in the bulk samples measurements and microelectronics parameters. The result of these researches is the study of functional relation definition between consolidation parameters, voltage (U), consolidation sintering temperature and relative capacitance change, from the bulk sample surface down to the coating thin films around the grains.
Deep Reinforcement Learning with Robust Deep Deterministic Policy Gradient

Recently, Deep Deterministic Policy Gradient (DDPG) is a popular deep reinforcement learning algorithms applied to continuous control problems like autonomous driving and robotics. Although DDPG can produce very good results, it has its drawbacks. DDPG can become unstable and heavily dependent on searching the correct hyperparameters for the current task. DDPG algorithm risk overestimating the Q values in the critic (value) network. The accumulation of estimation errors as time elapse can result in the reinforcement agent trapping into a local optimum or suffering from disastrous forgetting. Twin Delayed DDPG (TD3) mitigated the overestimation bias problem but might not exploit full performance due to underestimation bias. In this paper Twin Average Delayed DDPG (TAD3) is proposed for specific adaption to TD3 and shows that the resulting algorithm perform better than TD3 in a challenging continuous control environment.