pwc-archive/papers-with-abstracts huggingface dataset:
HBert + BiasCorp -- Fighting Racism on the Web
Authors: Olawale Onabola, Zhuang Ma, Yang Xie, Benjamin Akera, Abdulrahman Ibraheem, Jia Xue, Dianbo Liu, Yoshua Bengio
Date: 2021-04-06 00:00:00

Subtle and overt racism is still present both in physical and online communities today and has impacted many lives in different segments of the society. In this short piece of work, we present how we're tackling this societal issue with Natural Language Processing. We are releasing BiasCorp, a dataset containing 139,090 comments and news segment from three specific sources - Fox News, BreitbartNews and YouTube. The first batch (45,000 manually annotated) is ready for publication. We are currently in the final phase of manually labeling the remaining dataset using Amazon Mechanical Turk. BERT has been used widely in several downstream tasks. In this work, we present hBERT, where we modify certain layers of the pretrained BERT model with the new Hopfield Layer. hBert generalizes well across different distributions with the added advantage of a reduced model complexity. We are also releasing a JavaScript library and a Chrome Extension Application, to help developers make use of our trained model in web applications (say chat application) and for users to identify and report racially biased contents on the web respectively.
Flexi-Transducer: Optimizing Latency, Accuracy and Compute forMulti-Domain On-Device Scenarios
Authors: Jay Mahadeokar, Yangyang Shi, Yuan Shangguan, Chunyang Wu, Alex Xiao, Hang Su, Duc Le, Ozlem Kalinli, Christian Fuegen, Michael L. Seltzer
Date: 2021-04-06 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

Often, the storage and computational constraints of embeddeddevices demand that a single on-device ASR model serve multiple use-cases / domains. In this paper, we propose aFlexibleTransducer(FlexiT) for on-device automatic speech recognition to flexibly deal with multiple use-cases / domains with different accuracy and latency requirements. Specifically, using a single compact model, FlexiT provides a fast response for voice commands, and accurate transcription but with more latency for dictation. In order to achieve flexible and better accuracy and latency trade-offs, the following techniques are used. Firstly, we propose using domain-specific altering of segment size for Emformer encoder that enables FlexiT to achieve flexible de-coding. Secondly, we use Alignment Restricted RNNT loss to achieve flexible fine-grained control on token emission latency for different domains. Finally, we add a domain indicator vector as an additional input to the FlexiT model. Using the combination of techniques, we show that a single model can be used to improve WERs and real time factor for dictation scenarios while maintaining optimal latency for voice commands use-cases
Dissecting User-Perceived Latency of On-Device E2E Speech Recognition
Authors: Yuan Shangguan, Rohit Prabhavalkar, Hang Su, Jay Mahadeokar, Yangyang Shi, Jiatong Zhou, Chunyang Wu, Duc Le, Ozlem Kalinli, Christian Fuegen, Michael L. Seltzer
Date: 2021-04-06 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

As speech-enabled devices such as smartphones and smart speakers become increasingly ubiquitous, there is growing interest in building automatic speech recognition (ASR) systems that can run directly on-device; end-to-end (E2E) speech recognition models such as recurrent neural network transducers and their variants have recently emerged as prime candidates for this task. Apart from being accurate and compact, such systems need to decode speech with low user-perceived latency (UPL), producing words as soon as they are spoken. This work examines the impact of various techniques - model architectures, training criteria, decoding hyperparameters, and endpointer parameters - on UPL. Our analyses suggest that measures of model size (parameters, input chunk sizes), or measures of computation (e.g., FLOPS, RTF) that reflect the model's ability to process input frames are not always strongly correlated with observed UPL. Thus, conventional algorithmic latency measurements might be inadequate in accurately capturing latency observed when models are deployed on embedded devices. Instead, we find that factors affecting token emission latency, and endpointing behavior have a larger impact on UPL. We achieve the best trade-off between latency and word error rate when performing ASR jointly with endpointing, while utilizing the recently proposed alignment regularization mechanism.
Attention Head Masking for Inference Time Content Selection in Abstractive Summarization
Authors: Shuyang Cao, Lu Wang
Date: 2021-04-06 00:00:00
Proceeding: NAACL 2021 4
Tasks: Abstractive Text Summarization, Decoder, Document Summarization

How can we effectively inform content selection in Transformer-based abstractive summarization models? In this work, we present a simple-yet-effective attention head masking technique, which is applied on encoder-decoder attentions to pinpoint salient content at inference time. Using attention head masking, we are able to reveal the relation between encoder-decoder attentions and content selection behaviors of summarization models. We then demonstrate its effectiveness on three document summarization datasets based on both in-domain and cross-domain settings. Importantly, our models outperform prior state-of-the-art models on CNN/Daily Mail and New York Times datasets. Moreover, our inference-time masking technique is also data-efficient, requiring only 20% of the training samples to outperform BART fine-tuned on the full CNN/DailyMail dataset.
Dynamic Encoder Transducer: A Flexible Solution For Trading Off Accuracy For Latency
Authors: Yangyang Shi, Varun Nagaraja, Chunyang Wu, Jay Mahadeokar, Duc Le, Rohit Prabhavalkar, Alex Xiao, Ching-Feng Yeh, Julian Chan, Christian Fuegen, Ozlem Kalinli, Michael L. Seltzer
Date: 2021-04-05 00:00:00
Tasks: speech-recognition, Speech Recognition

We propose a dynamic encoder transducer (DET) for on-device speech recognition. One DET model scales to multiple devices with different computation capacities without retraining or finetuning. To trading off accuracy and latency, DET assigns different encoders to decode different parts of an utterance. We apply and compare the layer dropout and the collaborative learning for DET training. The layer dropout method that randomly drops out encoder layers in the training phase, can do on-demand layer dropout in decoding. Collaborative learning jointly trains multiple encoders with different depths in one single model. Experiment results on Librispeech and in-house data show that DET provides a flexible accuracy and latency trade-off. Results on Librispeech show that the full-size encoder in DET relatively reduces the word error rate of the same size baseline by over 8%. The lightweight encoder in DET trained with collaborative learning reduces the model size by 25% but still gets similar WER as the full-size baseline. DET gets similar accuracy as a baseline model with better latency on a large in-house data set by assigning a lightweight encoder for the beginning part of one utterance and a full-size encoder for the rest.
What Will it Take to Fix Benchmarking in Natural Language Understanding?
Authors: Samuel R. Bowman, George E. Dahl
Date: 2021-04-05 00:00:00
Proceeding: NAACL 2021 4
Tasks: Benchmarking, Natural Language Understanding, Position

Evaluation for many natural language understanding (NLU) tasks is broken: Unreliable and biased systems score so highly on standard benchmarks that there is little room for researchers who develop better systems to demonstrate their improvements. The recent trend to abandon IID benchmarks in favor of adversarially-constructed, out-of-distribution test sets ensures that current models will perform poorly, but ultimately only obscures the abilities that we want our benchmarks to measure. In this position paper, we lay out four criteria that we argue NLU benchmarks should meet. We argue most current benchmarks fail at these criteria, and that adversarial data collection does not meaningfully address the causes of these failures. Instead, restoring a healthy evaluation ecosystem will require significant progress in the design of benchmark datasets, the reliability with which they are annotated, their size, and the ways they handle social bias.
Semantic Distance: A New Metric for ASR Performance Analysis Towards Spoken Language Understanding
Authors: Suyoun Kim, Abhinav Arora, Duc Le, Ching-Feng Yeh, Christian Fuegen, Ozlem Kalinli, Michael L. Seltzer
Date: 2021-04-05 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Intent Recognition, Language Modeling, Language Modelling, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Natural Language Understanding, Semantic Parsing, Sentence, Sentence Embedding, Sentence-Embedding, slot-filling, Slot Filling, speech-recognition, Speech Recognition, Spoken Language Understanding

Word Error Rate (WER) has been the predominant metric used to evaluate the performance of automatic speech recognition (ASR) systems. However, WER is sometimes not a good indicator for downstream Natural Language Understanding (NLU) tasks, such as intent recognition, slot filling, and semantic parsing in task-oriented dialog systems. This is because WER takes into consideration only literal correctness instead of semantic correctness, the latter of which is typically more important for these downstream tasks. In this study, we propose a novel Semantic Distance (SemDist) measure as an alternative evaluation metric for ASR systems to address this issue. We define SemDist as the distance between a reference and hypothesis pair in a sentence-level embedding space. To represent the reference and hypothesis as a sentence embedding, we exploit RoBERTa, a state-of-the-art pre-trained deep contextualized language model based on the transformer architecture. We demonstrate the effectiveness of our proposed metric on various downstream tasks, including intent recognition, semantic parsing, and named entity recognition.
End-to-End Speaker-Attributed ASR with Transformer
Authors: Naoyuki Kanda, Guoli Ye, Yashesh Gaur, Xiaofei Wang, Zhong Meng, Zhuo Chen, Takuya Yoshioka
Date: 2021-04-05 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Decoder, Speaker Identification, speech-recognition, Speech Recognition

This paper presents our recent effort on end-to-end speaker-attributed automatic speech recognition, which jointly performs speaker counting, speech recognition and speaker identification for monaural multi-talker audio. Firstly, we thoroughly update the model architecture that was previously designed based on a long short-term memory (LSTM)-based attention encoder decoder by applying transformer architectures. Secondly, we propose a speaker deduplication mechanism to reduce speaker identification errors in highly overlapped regions. Experimental results on the LibriSpeechMix dataset shows that the transformer-based architecture is especially good at counting the speakers and that the proposed model reduces the speaker-attributed word error rate by 47% over the LSTM-based baseline. Furthermore, for the LibriCSS dataset, which consists of real recordings of overlapped speech, the proposed model achieves concatenated minimum-permutation word error rates of 11.9% and 16.3% with and without target speaker profiles, respectively, both of which are the state-of-the-art results for LibriCSS with the monaural setting.
Efficient Attentions for Long Document Summarization
Authors: Luyang Huang, Shuyang Cao, Nikolaus Parulian, Heng Ji, Lu Wang
Date: 2021-04-05 00:00:00
Proceeding: NAACL 2021 4
Tasks: Decoder, Document Summarization

The quadratic computational and memory complexities of large Transformers have limited their scalability for long document summarization. In this paper, we propose Hepos, a novel efficient encoder-decoder attention with head-wise positional strides to effectively pinpoint salient information from the source. We further conduct a systematic study of existing efficient self-attentions. Combined with Hepos, we are able to process ten times more tokens than existing models that use full attentions. For evaluation, we present a new dataset, GovReport, with significantly longer documents and summaries. Results show that our models produce significantly higher ROUGE scores than competitive comparisons, including new state-of-the-art results on PubMed. Human evaluation also shows that our models generate more informative summaries with fewer unfaithful errors.
Streaming Multi-talker Speech Recognition with Joint Speaker Identification
Authors: Liang Lu, Naoyuki Kanda, Jinyu Li, Yifan Gong
Date: 2021-04-05 00:00:00
Tasks: Speaker Identification, speech-recognition, Speech Recognition, Speech Separation

In multi-talker scenarios such as meetings and conversations, speech processing systems are usually required to transcribe the audio as well as identify the speakers for downstream applications. Since overlapped speech is common in this case, conventional approaches usually address this problem in a cascaded fashion that involves speech separation, speech recognition and speaker identification that are trained independently. In this paper, we propose Streaming Unmixing, Recognition and Identification Transducer (SURIT) -- a new framework that deals with this problem in an end-to-end streaming fashion. SURIT employs the recurrent neural network transducer (RNN-T) as the backbone for both speech recognition and speaker identification. We validate our idea on the LibrispeechMix dataset -- a multi-talker dataset derived from Librispeech, and present encouraging results.
The Tangent Search Algorithm for Solving Optimization Problems
Authors: Abdesslem Layeb
Date: 2021-04-06 00:00:00

This article proposes a new population-based optimization algorithm called the Tangent Search Algorithm (TSA) to solve optimization problems. The TSA uses a mathematical model based on the tangent function to move a given solution toward a better solution. The tangent flight function has the advantage to balance between the exploitation and the exploration search. Moreover, a novel escape procedure is used to avoid to be trapped in local minima. Besides, an adaptive variable step size is also integrated in this algorithm to enhance the convergence capacity. The performance of TSA is assessed in three classes of tests: classical tests, CEC benchmarks, and engineering optimization problems. Moreover, several studies and metrics have been used to observe the behavior of the proposed TSA. The experimental results show that TSA algorithm is capable to provide very promising and competitive results on most benchmark functions thanks to better balance between exploration and exploitation of the search space. The main characteristics of this new optimization algorithm is its simplicity and efficiency and it requires only a small number of user-defined parameters.
Learning Latent and Hierarchical Structures in Cognitive Diagnosis Models
Authors: Chenchen Ma, Gongjun Xu
Date: 2021-04-05 00:00:00
Tasks: Attribute, cognitive diagnosis, Diagnostic

Cognitive Diagnosis Models (CDMs) are a special family of discrete latent variable models that are widely used in modern educational, psychological, social and biological sciences. A key component of CDMs is a binary $Q$-matrix characterizing the dependence structure between the items and the latent attributes. Additionally, researchers also assume in many applications certain hierarchical structures among the latent attributes to characterize their dependence. In most CDM applications, the attribute-attribute hierarchical structures, the item-attribute $Q$-matrix, the item-level diagnostic model, as well as the number of latent attributes, need to be fully or partially pre-specified, which however may be subjective and misspecified as noted by many recent studies. This paper considers the problem of jointly learning these latent and hierarchical structures in CDMs from observed data with minimal model assumptions. Specifically, a penalized likelihood approach is proposed to select the number of attributes and estimate the latent and hierarchical structures simultaneously. An efficient expectation-maximization (EM) algorithm and a latent structure recovery algorithm are developed, and statistical consistency theory is also established under mild conditions. The good performance of the proposed method is illustrated by simulation studies and a real data application in educational assessment.
Comparing Weak- and Unsupervised Methods for Resonant Anomaly Detection
Authors: Jack H. Collins, Pablo Martín-Ramiro, Benjamin Nachman, David Shih
Date: 2021-04-05 00:00:00
Tasks: Anomaly Detection, Sensitivity

Anomaly detection techniques are growing in importance at the Large Hadron Collider (LHC), motivated by the increasing need to search for new physics in a model-agnostic way. In this work, we provide a detailed comparative study between a well-studied unsupervised method called the autoencoder (AE) and a weakly-supervised approach based on the Classification Without Labels (CWoLa) technique. We examine the ability of the two methods to identify a new physics signal at different cross sections in a fully hadronic resonance search. By construction, the AE classification performance is independent of the amount of injected signal. In contrast, the CWoLa performance improves with increasing signal abundance. When integrating these approaches with a complete background estimate, we find that the two methods have complementary sensitivity. In particular, CWoLa is effective at finding diverse and moderately rare signals while the AE can provide sensitivity to very rare signals, but only with certain topologies. We therefore demonstrate that both techniques are complementary and can be used together for anomaly detection at the LHC.
Large-scale Sustainable Search on Unconventional Computing Hardware
Authors: Kirill P. Kalinin, Natalia G. Berloff
Date: 2021-04-06 00:00:00

Since the advent of the Internet, quantifying the relative importance of web pages is at the core of search engine methods. According to one algorithm, PageRank, the worldwide web structure is represented by the Google matrix, whose principal eigenvector components assign a numerical value to web pages for their ranking. Finding such a dominant eigenvector on an ever-growing number of web pages becomes a computationally intensive task incompatible with Moore's Law. We demonstrate that special-purpose optical machines such as networks of optical parametric oscillators, lasers, and gain-dissipative condensates, may aid in accelerating the reliable reconstruction of principal eigenvectors of real-life web graphs. We discuss the feasibility of simulating the PageRank algorithm on large Google matrices using such unconventional hardware. We offer alternative rankings based on the minimisation of spin Hamiltonians. Our estimates show that special-purpose optical machines may provide dramatic improvements in power consumption over classical computing architectures.
ST-PIL: Spatial-Temporal Periodic Interest Learning for Next Point-of-Interest Recommendation
Authors: Qiang Cui, Chenrui Zhang, Yafeng Zhang, Jinpeng Wang, Mingchen Cai
Date: 2021-04-06 00:00:00

Point-of-Interest (POI) recommendation is an important task in location-based social networks. It facilitates the relation modeling between users and locations. Recently, researchers recommend POIs by long- and short-term interests and achieve success. However, they fail to well capture the periodic interest. People tend to visit similar places at similar times or in similar areas. Existing models try to acquire such kind of periodicity by user's mobility status or time slot, which limits the performance of periodic interest. To this end, we propose to learn spatial-temporal periodic interest. Specifically, in the long-term module, we learn the temporal periodic interest of daily granularity, then utilize intra-level attention to form long-term interest. In the short-term module, we construct various short-term sequences to acquire the spatial-temporal periodic interest of hourly, areal, and hourly-areal granularities, respectively. Finally, we apply inter-level attention to automatically integrate multiple interests. Experiments on two real-world datasets demonstrate the state-of-the-art performance of our method.
一个新的超大范围混沌系统及其自适应滑模控制
Authors: 徐昌彪1，钟 德2，夏 诚2，黎 周1
Date: 2019-03-18 00:00:00

提出了一个超大范围的混沌系统，其中参数 b 的取值为［0，107］。理论分析了系统的动力学特性，考察了
系统的 Lyapunov 指数谱、分岔图以及 Poincare 映射。设计了系统的硬件电路，并用 Multisim 软件进行了电路仿真，构建了一个系统在未知参数条件下全局稳定的自适应控制器和一个对给定信号追踪与未知参数辨识的自适应滑模控制器。仿真结果表明所设计控制器是有效的
A view synthesis-based 360° VR caching system over MEC-Enabled C-RAN
Authors: Jianmei Dai, Zhilong Zhang, Shiwen Mao, Danpu Liu
Date: 2020-10-01 00:00:00
Tasks: Edge-computing

With the development of virtual reality (VR) technology, the future of VR systems is evolving from single-user wired connections to multi-user wireless connections. However, wireless online rendering and transmission incur extra processing and transmission latency, as well as higher bandwidth requirements. To meet the requirements of wireless VR applications and enhance the quality of the VR user experience, this paper designs a view synthesis-based 360° VR caching system over Cloud Radio Access Network (C-RAN), where both mobile edge computing (MEC) and hierarchical caching are supported. In the system, an MEC-Cache Server is deployed in the pooled Base band Units (BBU pool) and used for view synthesis and caching. In addition, the remote radio heads (RRHs) can also cache some video contents. If the requested content of a specific view is cached in the BBU pool or RRHs, or can be synthesized with the aid of the cached adjacent views, it is unnecessary to request the content from the remote VR video source server. Therefore, the transmission latency and backhaul traffic load for VR services can be decreased. We formulate a hierarchical collaborative caching problem aiming to minimize the transmission latency, which is proved NP-hard. To address the impractical expenses of the offline optimal method, an online MaxMinDistance caching algorithm with low complexity is proposed. Numerical simulation results demonstrate that the proposed caching strategy provides significantly improved cache hit rate, backhaul traffic load, transmission latency, and Quality of Experience (QoE) performances relative to conventional caching strategies