pwc-archive/papers-with-abstracts huggingface dataset:
End-to-end Deep Learning Pipeline for Microwave Kinetic Inductance Detector (MKID) Resonator Identification and Tuning
Authors: Neelay Fruitwala, Alex B Walter, John I Bailey III, Rupert Dodkins, Benjamin A Mazin
Date: 2021-04-03 00:00:00

We present the development of a machine learning based pipeline to fully automate the calibration of the frequency comb used to read out optical/IR Microwave Kinetic Inductance Detector (MKID) arrays. This process involves determining the resonant frequency and optimal drive power of every pixel (i.e. resonator) in the array, which is typically done manually. Modern optical/IR MKID arrays, such as DARKNESS (DARK-speckle Near-infrared Energy-resolving Superconducting Spectrophotometer) and MEC (MKID Exoplanet Camera), contain 10-20,000 pixels, making the calibration process extremely time consuming; each 2000 pixel feedline requires 4-6 hours of manual tuning. Here we present a pipeline which uses a single convolutional neural network (CNN) to perform both resonator identification and tuning simultaneously. We find that our pipeline has performance equal to that of the manual tuning process, and requires just twelve minutes of computational time per feedline.
Modeling Censored Mobility Demand through Quantile Regression Neural Networks
Authors: Frederik Boe Hüttel, Inon Peled, Filipe Rodrigues, Francisco C. Pereira
Date: 2021-04-02 00:00:00
Tasks: Decision Making, quantile regression, regression

Shared mobility services require accurate demand models for effective service planning. On the one hand, modeling the full probability distribution of demand is advantageous because the entire uncertainty structure preserves valuable information for decision-making. On the other hand, demand is often observed through the usage of the service itself, so that the observations are censored, as they are inherently limited by available supply. Since the 1980s, various works on Censored Quantile Regression models have performed well under such conditions. Further, in the last two decades, several papers have proposed to implement these models flexibly through Neural Networks. However, the models in current works estimate the quantiles individually, thus incurring a computational overhead and ignoring valuable relationships between the quantiles. We address this gap by extending current Censored Quantile Regression models to learn multiple quantiles at once and apply these to synthetic baseline datasets and datasets from two shared mobility providers in the Copenhagen metropolitan area in Denmark. The results show that our extended models yield fewer quantile crossings and less computational overhead without compromising model performance.
Type Prediction Systems
Authors: Sarthak Dash, Nandana Mihindukulasooriya, Alfio Gliozzo, Mustafa Canim
Date: 2021-04-02 00:00:00
Tasks: Entity Disambiguation, Knowledge Base Question Answering, Prediction, Question Answering, Type prediction, Vocal Bursts Type Prediction

Inferring semantic types for entity mentions within text documents is an important asset for many downstream NLP tasks, such as Semantic Role Labelling, Entity Disambiguation, Knowledge Base Question Answering, etc. Prior works have mostly focused on supervised solutions that generally operate on relatively small-to-medium-sized type systems. In this work, we describe two systems aimed at predicting type information for the following two tasks, namely, a TypeSuggest module, an unsupervised system designed to predict types for a set of user-entered query terms, and an Answer Type prediction module, that provides a solution for the task of determining the correct type of the answer expected to a given query. Our systems generalize to arbitrary type systems of any sizes, thereby making it a highly appealing solution to extract type information at any granularity.
Scan Specific Artifact Reduction in K-space (SPARK) Neural Networks Synergize with Physics-based Reconstruction to Accelerate MRI
Authors: Yamin Arefeen, Onur Beker, Jaejin Cho, Heng Yu, Elfar Adalsteinsson, Berkin Bilgic
Date: 2021-04-02 00:00:00

Purpose: To develop a scan-specific model that estimates and corrects k-space errors made when reconstructing accelerated Magnetic Resonance Imaging (MRI) data. Methods: Scan-Specific Artifact Reduction in k-space (SPARK) trains a convolutional-neural-network to estimate and correct k-space errors made by an input reconstruction technique by back-propagating from the mean-squared-error loss between an auto-calibration signal (ACS) and the input technique's reconstructed ACS. First, SPARK is applied to GRAPPA and demonstrates improved robustness over other scan-specific models, such as RAKI and residual-RAKI. Subsequent experiments demonstrate that SPARK synergizes with residual-RAKI to improve reconstruction performance. SPARK also improves reconstruction quality when applied to advanced acquisition and reconstruction techniques like 2D virtual coil (VC-) GRAPPA, 2D LORAKS, 3D GRAPPA without an integrated ACS region, and 2D/3D wave-encoded images. Results: SPARK yields 1.5x - 2x RMSE reduction when applied to GRAPPA and improves robustness to ACS size for various acceleration rates in comparison to other scan-specific techniques. When applied to advanced reconstruction techniques such as residual-RAKI, 2D VC-GRAPPA and LORAKS, SPARK achieves up to 20% RMSE improvement. SPARK with 3D GRAPPA also improves performance by ~2x and perceived image quality without a fully sampled ACS region. Finally, SPARK synergizes with non-cartesian 2D and 3D wave-encoding imaging by reducing RMSE between 20-25% and providing qualitative improvements. Conclusion: SPARK synergizes with physics-based acquisition and reconstruction techniques to improve accelerated MRI by training scan-specific models to estimate and correct reconstruction errors in k-space.
Novel DNNs for Stiff ODEs with Applications to Chemically Reacting Flows
Authors: Thomas S. Brown, Harbir Antil, Rainald Löhner, Fumiya Togashi, Deepanshu Verma
Date: 2021-04-01 00:00:00
Tasks: CPU

Chemically reacting flows are common in engineering, such as hypersonic flow, combustion, explosions, manufacturing processes and environmental assessments. For combustion, the number of reactions can be significant (over 100) and due to the very large CPU requirements of chemical reactions (over 99%) a large number of flow and combustion problems are presently beyond the capabilities of even the largest supercomputers. Motivated by this, novel Deep Neural Networks (DNNs) are introduced to approximate stiff ODEs. Two approaches are compared, i.e., either learn the solution or the derivative of the solution to these ODEs. These DNNs are applied to multiple species and reactions common in chemically reacting flows. Experimental results show that it is helpful to account for the physical properties of species while designing DNNs. The proposed approach is shown to generalize well.
Neurological Status Classification Using Convolutional Neural Network
Authors: Mehrad Jaloli, Divya Choudhary, Marzia Cescon
Date: 2021-04-01 00:00:00
Tasks: Classification, EEG, Electroencephalogram (EEG), General Classification

In this study we show that a Convolutional Neural Network (CNN) model is able to accuratelydiscriminate between 4 different phases of neurological status in a non-Electroencephalogram(EEG) dataset recorded in an experiment in which subjects are exposed to physical, cognitiveand emotional stress. We demonstrate that the proposed model is able to obtain 99.99% AreaUnder the Curve (AUC) of Receiver Operation characteristic (ROC) and 99.82% classificationaccuracy on the test dataset. Furthermore, for comparison, we show that our models outperformstraditional classification methods such as SVM, and RF. Finally, we show the advantage of CNN models, in comparison to other methods, in robustness to noise by 97.46% accuracy on a noisy dataset.
Exploring Transformers in Emotion Recognition: a comparison of BERT, DistillBERT, RoBERTa, XLNet and ELECTRA
Authors: Diogo Cortiz
Date: 2021-04-05 00:00:00
Tasks: Emotion Recognition, Natural Language Understanding

This paper investigates how Natural Language Understanding (NLU) could be applied in Emotion Recognition, a specific task in affective computing. We finetuned different transformers language models (BERT, DistilBERT, RoBERTa, XLNet, and ELECTRA) using a fine-grained emotion dataset and evaluating them in terms of performance (f1-score) and time to complete.
Intent Detection and Slot Filling for Vietnamese
Authors: Mai Hoang Dao, Thinh Hung Truong, Dat Quoc Nguyen
Date: 2021-04-05 00:00:00
Tasks: Intent Classification and Slot Filling, Intent Detection, Natural Language Understanding, slot-filling, Slot Filling

Intent detection and slot filling are important tasks in spoken and natural language understanding. However, Vietnamese is a low-resource language in these research topics. In this paper, we present the first public intent detection and slot filling dataset for Vietnamese. In addition, we also propose a joint model for intent detection and slot filling, that extends the recent state-of-the-art JointBERT+CRF model with an intent-slot attention layer to explicitly incorporate intent context information into slot filling via "soft" intent label embedding. Experimental results on our Vietnamese dataset show that our proposed model significantly outperforms JointBERT+CRF. We publicly release our dataset and the implementation of our model at: https://github.com/VinAIResearch/JointIDSF
SPGISpeech: 5,000 hours of transcribed financial audio for fully formatted end-to-end speech recognition
Authors: Patrick K. O'Neill, Vitaly Lavrukhin, Somshubra Majumdar, Vahid Noroozi, Yuekai Zhang, Oleksii Kuchaiev, Jagadeesh Balam, Yuliya Dovzhenko, Keenan Freyberg, Michael D. Shulman, Boris Ginsburg, Shinji Watanabe, Georg Kucsko
Date: 2021-04-05 00:00:00
Tasks: speech-recognition, Speech Recognition, Speech-to-Text

In the English speech-to-text (STT) machine learning task, acoustic models are conventionally trained on uncased Latin characters, and any necessary orthography (such as capitalization, punctuation, and denormalization of non-standard words) is imputed by separate post-processing models. This adds complexity and limits performance, as many formatting tasks benefit from semantic information present in the acoustic signal but absent in transcription. Here we propose a new STT task: end-to-end neural transcription with fully formatted text for target labels. We present baseline Conformer-based models trained on a corpus of 5,000 hours of professionally transcribed earnings calls, achieving a CER of 1.7. As a contribution to the STT research community, we release the corpus free for non-commercial use at https://datasets.kensho.com/datasets/scribe.
What's the best place for an AI conference, Vancouver or ______: Why completing comparative questions is difficult
Authors: Avishai Zagoury, Einat Minkov, Idan Szpektor, William W. Cohen
Date: 2021-04-05 00:00:00
Tasks: World Knowledge

Although large neural language models (LMs) like BERT can be finetuned to yield state-of-the-art results on many NLP tasks, it is often unclear what these models actually learn. Here we study using such LMs to fill in entities in human-authored comparative questions, like ``Which country is older, India or ______?'' -- i.e., we study the ability of neural LMs to ask (not answer) reasonable questions. We show that accuracy in this fill-in-the-blank task is well-correlated with human judgements of whether a question is reasonable, and that these models can be trained to achieve nearly human-level performance in completing comparative questions in three different subdomains. However, analysis shows that what they learn fails to model any sort of broad notion of which entities are semantically comparable or similar -- instead the trained models are very domain-specific, and performance is highly correlated with co-occurrences between specific entities observed in the training set. This is true both for models that are pretrained on general text corpora, as well as models trained on a large corpus of comparison questions. Our study thus reinforces recent results on the difficulty of making claims about a deep model's world knowledge or linguistic competence based on performance on specific benchmark problems. We make our evaluation datasets publicly available to foster future research on complex understanding and reasoning in such models at standards of human interaction.
Mining Customers' Opinions for Online Reputation Generation and Visualization in e-Commerce Platforms
Authors: Abdessamad Benlahbib
Date: 2021-04-05 00:00:00
Tasks: Decision Making

Customer reviews represent a very rich data source from which we can extract very valuable information about different online shopping experiences. The amount of the collected data may be very large especially for trendy items (products, movies, TV shows, hotels, services...), where the number of available customers' opinions could easily surpass thousands. In fact, while a good number of reviews could indeed give a hint about the quality of an item, a potential customer may not have time or effort to read all reviews for the purpose of making an informed decision (buying, renting, booking...). Thus, the need for the right tools and technologies to help in such a task becomes a necessity for the buyer as for the seller. My research goal in this thesis is to develop reputation systems that can automatically provide E-commerce customers with valuable information to support them during their online decision-making process by mining online reviews expressed in natural language.
StarGAN-based Emotional Voice Conversion for Japanese Phrases
Authors: Asuka Moritani, Ryo Ozaki, Shoki Sakamoto, Hirokazu Kameoka, Tadahiro Taniguchi
Date: 2021-04-05 00:00:00
Tasks: Voice Conversion

This paper shows that StarGAN-VC, a spectral envelope transformation method for non-parallel many-to-many voice conversion (VC), is capable of emotional VC (EVC). Although StarGAN-VC has been shown to enable speaker identity conversion, its capability for EVC for Japanese phrases has not been clarified. In this paper, we describe the direct application of StarGAN-VC to an EVC task with minimal fundamental frequency and aperiodicity processing. Through subjective evaluation experiments, we evaluated the performance of our StarGAN-EVC system in terms of its ability to achieve EVC for Japanese phrases. The subjective evaluation is conducted in terms of subjective classification and mean opinion score of neutrality and similarity. In addition, the interdependence between the source and target emotional domains was investigated from the perspective of the quality of EVC.
Deep Neural Networks for Relation Extraction
Authors: Tapas Nayak
Date: 2021-04-05 00:00:00
Tasks: Decoder, Joint Entity and Relation Extraction, Knowledge Base Population, Relation, Relation Extraction

Relation extraction from text is an important task for automatic knowledge base population. In this thesis, we first propose a syntax-focused multi-factor attention network model for finding the relation between two entities. Next, we propose two joint entity and relation extraction frameworks based on encoder-decoder architecture. Finally, we propose a hierarchical entity graph convolutional network for relation extraction across documents.
A Heuristic-driven Uncertainty based Ensemble Framework for Fake News Detection in Tweets and News Articles
Authors: Sourya Dipta Das, Ayan Basak, Saikat Dutta
Date: 2021-04-05 00:00:00
Tasks: Articles, Fake News Detection

The significance of social media has increased manifold in the past few decades as it helps people from even the most remote corners of the world to stay connected. With the advent of technology, digital media has become more relevant and widely used than ever before and along with this, there has been a resurgence in the circulation of fake news and tweets that demand immediate attention. In this paper, we describe a novel Fake News Detection system that automatically identifies whether a news item is "real" or "fake", as an extension of our work in the CONSTRAINT COVID-19 Fake News Detection in English challenge. We have used an ensemble model consisting of pre-trained models followed by a statistical feature fusion network , along with a novel heuristic algorithm by incorporating various attributes present in news items or tweets like source, username handles, URL domains and authors as statistical feature. Our proposed framework have also quantified reliable predictive uncertainty along with proper class output confidence level for the classification task. We have evaluated our results on the COVID-19 Fake News dataset and FakeNewsNet dataset to show the effectiveness of the proposed algorithm on detecting fake news in short news content as well as in news articles. We obtained a best F1-score of 0.9892 on the COVID-19 dataset, and an F1-score of 0.9073 on the FakeNewsNet dataset.
Annotating Columns with Pre-trained Language Models
Authors: Yoshihiko Suhara, Jinfeng Li, Yuliang Li, Dan Zhang, Çağatay Demiralp, Chen Chen, Wang-Chiew Tan
Date: 2021-04-05 00:00:00
Tasks: Columns Property Annotation, Column Type Annotation, Management, Multi-Task Learning, Relation Prediction, Table annotation, Type prediction

Inferring meta information about tables, such as column headers or relationships between columns, is an active research topic in data management as we find many tables are missing some of this information. In this paper, we study the problem of annotating table columns (i.e., predicting column types and the relationships between columns) using only information from the table itself. We develop a multi-task learning framework (called Doduo) based on pre-trained language models, which takes the entire table as input and predicts column types/relations using a single model. Experimental results show that Doduo establishes new state-of-the-art performance on two benchmarks for the column type prediction and column relation prediction tasks with up to 4.0% and 11.9% improvements, respectively. We report that Doduo can already outperform the previous state-of-the-art performance with a minimal number of tokens, only 8 tokens per column. We release a toolbox (https://github.com/megagonlabs/doduo) and confirm the effectiveness of Doduo on a real-world data science problem through a case study.
BBAEG: Towards BERT-based Biomedical Adversarial Example Generation for Text Classification
Authors: Ishani Mondal
Date: 2021-04-05 00:00:00
Proceeding: NAACL 2021 4
Tasks: Decision Making, General Classification, text-classification, Text Classification

Healthcare predictive analytics aids medical decision-making, diagnosis prediction and drug review analysis. Therefore, prediction accuracy is an important criteria which also necessitates robust predictive language models. However, the models using deep learning have been proven vulnerable towards insignificantly perturbed input instances which are less likely to be misclassified by humans. Recent efforts of generating adversaries using rule-based synonyms and BERT-MLMs have been witnessed in general domain, but the ever increasing biomedical literature poses unique challenges. We propose BBAEG (Biomedical BERT-based Adversarial Example Generation), a black-box attack algorithm for biomedical text classification, leveraging the strengths of both domain-specific synonym replacement for biomedical named entities and BERTMLM predictions, spelling variation and number replacement. Through automatic and human evaluation on two datasets, we demonstrate that BBAEG performs stronger attack with better language fluency, semantic coherence as compared to prior work.
WhiteningBERT: An Easy Unsupervised Sentence Embedding Approach
Authors: JunJie Huang, Duyu Tang, Wanjun Zhong, Shuai Lu, Linjun Shou, Ming Gong, Daxin Jiang, Nan Duan
Date: 2021-04-05 00:00:00
Proceeding: Findings (EMNLP) 2021 11
Tasks: Retrieval, Sentence, Sentence Embedding, Sentence-Embedding, Sentence Embeddings

Producing the embedding of a sentence in an unsupervised way is valuable to natural language matching and retrieval problems in practice. In this work, we conduct a thorough examination of pretrained model based unsupervised sentence embeddings. We study on four pretrained models and conduct massive experiments on seven datasets regarding sentence semantics. We have there main findings. First, averaging all tokens is better than only using [CLS] vector. Second, combining both top andbottom layers is better than only using top layers. Lastly, an easy whitening-based vector normalization strategy with less than 10 lines of code consistently boosts the performance.
Paired Examples as Indirect Supervision in Latent Decision Models
Authors: Nitish Gupta, Sameer Singh, Matt Gardner, Dan Roth
Date: 2021-04-05 00:00:00
Proceeding: EMNLP 2021 11
Tasks: Out-of-Distribution Generalization, Question Answering, Question Generation, Question-Generation

Compositional, structured models are appealing because they explicitly decompose problems and provide interpretable intermediate outputs that give confidence that the model is not simply latching onto data artifacts. Learning these models is challenging, however, because end-task supervision only provides a weak indirect signal on what values the latent decisions should take. This often results in the model failing to learn to perform the intermediate tasks correctly. In this work, we introduce a way to leverage paired examples that provide stronger cues for learning latent decisions. When two related training examples share internal substructure, we add an additional training objective to encourage consistency between their latent decisions. Such an objective does not require external supervision for the values of the latent output, or even the end task, yet provides an additional training signal to that provided by individual training examples themselves. We apply our method to improve compositional question answering using neural module networks on the DROP dataset. We explore three ways to acquire paired questions in DROP: (a) discovering naturally occurring paired examples within the dataset, (b) constructing paired examples using templates, and (c) generating paired examples using a question generation model. We empirically demonstrate that our proposed approach improves both in- and out-of-distribution generalization and leads to correct latent decision predictions.
A New Approach to Overgenerating and Scoring Abstractive Summaries
Authors: Kaiqiang Song, Bingqing Wang, Zhe Feng, Fei Liu
Date: 2021-04-05 00:00:00
Proceeding: NAACL 2021 4
Tasks: Text Summarization

We propose a new approach to generate multiple variants of the target summary with diverse content and varying lengths, then score and select admissible ones according to users' needs. Abstractive summarizers trained on single reference summaries may struggle to produce outputs that achieve multiple desirable properties, i.e., capturing the most important information, being faithful to the original, grammatical and fluent. In this paper, we propose a two-staged strategy to generate a diverse set of candidate summaries from the source text in stage one, then score and select admissible ones in stage two. Importantly, our generator gives a precise control over the length of the summary, which is especially well-suited when space is limited. Our selectors are designed to predict the optimal summary length and put special emphasis on faithfulness to the original text. Both stages can be effectively trained, optimized and evaluated. Our experiments on benchmark summarization datasets suggest that this paradigm can achieve state-of-the-art performance.