pwc-archive/papers-with-abstracts huggingface dataset:
Optimal Cost Design for Model Predictive Control
Authors: Avik Jain, Lawrence Chan, Daniel S. Brown, Anca D. Dragan
Date: 2021-04-23 00:00:00
Tasks: Autonomous Driving, model, Model Predictive Control

Many robotics domains use some form of nonconvex model predictive control (MPC) for planning, which sets a reduced time horizon, performs trajectory optimization, and replans at every step. The actual task typically requires a much longer horizon than is computationally tractable, and is specified via a cost function that cumulates over that full horizon. For instance, an autonomous car may have a cost function that makes a desired trade-off between efficiency, safety, and obeying traffic laws. In this work, we challenge the common assumption that the cost we optimize using MPC should be the same as the ground truth cost for the task (plus a terminal cost). MPC solvers can suffer from short planning horizons, local optima, incorrect dynamics models, and, importantly, fail to account for future replanning ability. Thus, we propose that in many tasks it could be beneficial to purposefully choose a different cost function for MPC to optimize: one that results in the MPC rollout having low ground truth cost, rather than the MPC planned trajectory. We formalize this as an optimal cost design problem, and propose a zeroth-order optimization-based approach that enables us to design optimal costs for an MPC planning robot in continuous MDPs. We test our approach in an autonomous driving domain where we find costs different from the ground truth that implicitly compensate for replanning, short horizon, incorrect dynamics models, and local minima issues. As an example, the learned cost incentivizes MPC to delay its decision until later, implicitly accounting for the fact that it will get more information in the future and be able to make a better decision. Code and videos available at https://sites.google.com/berkeley.edu/ocd-mpc/.
Restoring degraded speech via a modified diffusion model
Authors: Jianwei Zhang, Suren Jayasuriya, Visar Berisha
Date: 2021-04-22 00:00:00
Tasks: model

There are many deterministic mathematical operations (e.g. compression, clipping, downsampling) that degrade speech quality considerably. In this paper we introduce a neural network architecture, based on a modification of the DiffWave model, that aims to restore the original speech signal. DiffWave, a recently published diffusion-based vocoder, has shown state-of-the-art synthesized speech quality and relatively shorter waveform generation times, with only a small set of parameters. We replace the mel-spectrum upsampler in DiffWave with a deep CNN upsampler, which is trained to alter the degraded speech mel-spectrum to match that of the original speech. The model is trained using the original speech waveform, but conditioned on the degraded speech mel-spectrum. Post-training, only the degraded mel-spectrum is used as input and the model generates an estimate of the original speech. Our model results in improved speech quality (original DiffWave model as baseline) on several different experiments. These include improving the quality of speech degraded by LPC-10 compression, AMR-NB compression, and signal clipping. Compared to the original DiffWave architecture, our scheme achieves better performance on several objective perceptual metrics and in subjective comparisons. Improvements over baseline are further amplified in a out-of-corpus evaluation setting.
A Machine Learning Framework for Real-time Inverse Modeling and Multi-objective Process Optimization of Composites for Active Manufacturing Control
Authors: Keith D. Humfeld, Dawei Gu, Geoffrey A. Butler, Karl Nelson, Navid Zobeiry
Date: 2021-04-22 00:00:00

For manufacturing of aerospace composites, several parts may be processed simultaneously using convective heating in an autoclave. Due to uncertainties including tool placement, convective Boundary Conditions (BCs) vary in each run. As a result, temperature histories in some of the parts may not conform to process specifications due to under-curing or over-heating. Thermochemical analysis using Finite Element (FE) simulations are typically conducted prior to fabrication based on assumed range of BCs. This, however, introduces unnecessary constraints on the design. To monitor the process, thermocouples (TCs) are placed under tools near critical locations. The TC data may be used to back-calculate BCs using trial-and-error FE analysis. However, since the inverse heat transfer problem is ill-posed, many solutions are obtained for given TC data. In this study, a novel machine learning (ML) framework is presented capable of optimizing air temperature cycle in real-time based on TC data from multiple parts, for active control of manufacturing. The framework consists of two recurrent Neural Networks (NN) for inverse modeling of the ill-posed curing problem at the speed of 300 simulations/second, and a classification NN for multi-objective optimization of the air temperature at the speed of 35,000 simulations/second. A virtual demonstration of the framework for process optimization of three composite parts with data from three TCs is presented.
On Geodesic Distances and Contextual Embedding Compression for Text Classification
Authors: Rishi Jha, Kai Mihata
Date: 2021-04-22 00:00:00
Proceeding: NAACL (TextGraphs) 2021 6
Tasks: Classification, Dimensionality Reduction, General Classification, text-classification, Text Classification

In some memory-constrained settings like IoT devices and over-the-network data pipelines, it can be advantageous to have smaller contextual embeddings. We investigate the efficacy of projecting contextual embedding data (BERT) onto a manifold, and using nonlinear dimensionality reduction techniques to compress these embeddings. In particular, we propose a novel post-processing approach, applying a combination of Isomap and PCA. We find that the geodesic distance estimations, estimates of the shortest path on a Riemannian manifold, from Isomap's k-Nearest Neighbors graph bolstered the performance of the compressed embeddings to be comparable to the original BERT embeddings. On one dataset, we find that despite a 12-fold dimensionality reduction, the compressed embeddings performed within 0.1% of the original BERT embeddings on a downstream classification task. In addition, we find that this approach works particularly well on tasks reliant on syntactic data, when compared with linear dimensionality reduction. These results show promise for a novel geometric approach to achieve lower dimensional text embeddings from existing transformers and pave the way for data-specific and application-specific embedding compressions.
Software-Defined Edge Computing: A New Architecture Paradigm to Support IoT Data Analysis
Authors: Di wu, XiaoFeng Xie, Xiang Ni, Bin Fu, Hanhui Deng, Haibo Zeng, Zhijin Qin
Date: 2021-04-22 00:00:00
Tasks: Anomaly Detection, Edge-computing

The rapid deployment of Internet of Things (IoT) applications leads to massive data that need to be processed. These IoT applications have specific communication requirements on latency and bandwidth, and present new features on their generated data such as time-dependency. Therefore, it is desirable to reshape the current IoT architectures by exploring their inherent nature of communication and computing to support smart IoT data process and analysis. We introduce in this paper features of IoT data, trends of IoT network architectures, some problems in IoT data analysis, and their solutions. Specifically, we view that software-defined edge computing is a promising architecture to support the unique needs of IoT data analysis. We further present an experiment on data anomaly detection in this architecture, and the comparison between two architectures for ECG diagnosis. Results show that our method is effective and feasible.
Assessing Validity of Static Analysis Warnings using Ensemble Learning
Authors: Anshul Tanwar, Hariharan Manikandan, Krishna Sundaresan, Prasanna Ganesan, Sathish Kumar Chandrasekaran, Sriram Ravi
Date: 2021-04-21 00:00:00
Tasks: Ensemble Learning

Static Analysis (SA) tools are used to identify potential weaknesses in code and fix them in advance, while the code is being developed. In legacy codebases with high complexity, these rules-based static analysis tools generally report a lot of false warnings along with the actual ones. Though the SA tools uncover many hidden bugs, they are lost in the volume of fake warnings reported. The developers expend large hours of time and effort in identifying the true warnings. Other than impacting the developer productivity, true bugs are also missed out due to this challenge. To address this problem, we propose a Machine Learning (ML)-based learning process that uses source codes, historic commit data, and classifier-ensembles to prioritize the True warnings from the given list of warnings. This tool is integrated into the development workflow to filter out the false warnings and prioritize actual bugs. We evaluated our approach on the networking C codes, from a large data pool of static analysis warnings reported by the tools. Time-to-time these warnings are addressed by the developers, labelling them as authentic bugs or fake alerts. The ML model is trained with full supervision over the code features. Our results confirm that applying deep learning over the traditional static analysis reports is an assuring approach for drastically reducing the false positive rates.
Evaluating Deception Detection Model Robustness To Linguistic Variation
Authors: Maria Glenski, Ellyn Ayton, Robin Cosbey, Dustin Arendt, Svitlana Volkova
Date: 2021-04-23 00:00:00
Proceeding: NAACL (SocialNLP) 2021 6
Tasks: Adversarial Defense, Deception Detection, Misinformation, model

With the increasing use of machine-learning driven algorithmic judgements, it is critical to develop models that are robust to evolving or manipulated inputs. We propose an extensive analysis of model robustness against linguistic variation in the setting of deceptive news detection, an important task in the context of misinformation spread online. We consider two prediction tasks and compare three state-of-the-art embeddings to highlight consistent trends in model performance, high confidence misclassifications, and high impact failures. By measuring the effectiveness of adversarial defense strategies and evaluating model susceptibility to adversarial attacks using character- and word-perturbed text, we find that character or mixed ensemble models are the most effective defenses and that character perturbation-based attack tactics are more successful.
Beyond Voice Activity Detection: Hybrid Audio Segmentation for Direct Speech Translation
Authors: Marco Gaido, Matteo Negri, Mauro Cettolo, Marco Turchi
Date: 2021-04-23 00:00:00
Proceeding: ICNLSP 2021 11
Tasks: Action Detection, Activity Detection, Segmentation, Translation

The audio segmentation mismatch between training data and those seen at run-time is a major problem in direct speech translation. Indeed, while systems are usually trained on manually segmented corpora, in real use cases they are often presented with continuous audio requiring automatic (and sub-optimal) segmentation. After comparing existing techniques (VAD-based, fixed-length and hybrid segmentation methods), in this paper we propose enhanced hybrid solutions to produce better results without sacrificing latency. Through experiments on different domains and language pairs, we show that our methods outperform all the other techniques, reducing by at least 30% the gap between the traditional VAD-based approach and optimal manual segmentation.
Claim Detection in Biomedical Twitter Posts
Authors: Amelie Wührl, Roman Klinger
Date: 2021-04-23 00:00:00
Proceeding: NAACL (BioNLP) 2021 6
Tasks: Fact Checking, Fake News Detection, Misinformation, Transfer Learning

Social media contains unfiltered and unique information, which is potentially of great value, but, in the case of misinformation, can also do great harm. With regards to biomedical topics, false information can be particularly dangerous. Methods of automatic fact-checking and fake news detection address this problem, but have not been applied to the biomedical domain in social media yet. We aim to fill this research gap and annotate a corpus of 1200 tweets for implicit and explicit biomedical claims (the latter also with span annotations for the claim phrase). With this corpus, which we sample to be related to COVID-19, measles, cystic fibrosis, and depression, we develop baseline models which detect tweets that contain a claim automatically. Our analyses reveal that biomedical tweets are densely populated with claims (45 % in a corpus sampled to contain 1200 tweets focused on the domains mentioned above). Baseline classification experiments with embedding-based classifiers and BERT-based transfer learning demonstrate that the detection is challenging, however, shows acceptable performance for the identification of explicit expressions of claims. Implicit claim tweets are more challenging to detect.
Understanding who uses Reddit: Profiling individuals with a self-reported bipolar disorder diagnosis
Authors: Glorianna Jagfeld, Fiona Lobban, Paul Rayson, Steven H. Jones
Date: 2021-04-23 00:00:00
Proceeding: NAACL (CLPsych) 2021 6

Recently, research on mental health conditions using public online data, including Reddit, has surged in NLP and health research but has not reported user characteristics, which are important to judge generalisability of findings. This paper shows how existing NLP methods can yield information on clinical, demographic, and identity characteristics of almost 20K Reddit users who self-report a bipolar disorder diagnosis. This population consists of slightly more feminine- than masculine-gendered mainly young or middle-aged US-based adults who often report additional mental health diagnoses, which is compared with general Reddit statistics and epidemiological studies. Additionally, this paper carefully evaluates all methods and discusses ethical issues.
QMUL-SDS at SCIVER: Step-by-Step Binary Classification for Scientific Claim Verification
Authors: Xia Zeng, Arkaitz Zubiaga
Date: 2021-04-23 00:00:00
Proceeding: NAACL (sdp) 2021 6
Tasks: Binary Classification, Claim Verification, General Classification, Sentence

Scientific claim verification is a unique challenge that is attracting increasing interest. The SCIVER shared task offers a benchmark scenario to test and compare claim verification approaches by participating teams and consists in three steps: relevant abstract selection, rationale selection and label prediction. In this paper, we present team QMUL-SDS's participation in the shared task. We propose an approach that performs scientific claim verification by doing binary classifications step-by-step. We trained a BioBERT-large classifier to select abstracts based on pairwise relevance assessments for each <claim, title of the abstract> and continued to train it to select rationales out of each retrieved abstract based on <claim, sentence>. We then propose a two-step setting for label prediction, i.e. first predicting "NOT_ENOUGH_INFO" or "ENOUGH_INFO", then label those marked as "ENOUGH_INFO" as either "SUPPORT" or "CONTRADICT". Compared to the baseline system, we achieve substantial improvements on the dev set. As a result, our team is the No. 4 team on the leaderboard.
Deep learning for sentence clustering in essay grading support
Authors: Li-Hsin Chang, Iiro Rastas, Sampo Pyysalo, Filip Ginter
Date: 2021-04-23 00:00:00
Tasks: Clustering, Deep Learning, Multiple-choice, Sentence

Essays as a form of assessment test student knowledge on a deeper level than short answer and multiple-choice questions. However, the manual evaluation of essays is time- and labor-consuming. Automatic clustering of essays, or their fragments, prior to manual evaluation presents a possible solution to reducing the effort required in the evaluation process. Such clustering presents numerous challenges due to the variability and ambiguity of natural language. In this paper, we introduce two datasets of undergraduate student essays in Finnish, manually annotated for salient arguments on the sentence level. Using these datasets, we evaluate several deep-learning embedding methods for their suitability to sentence clustering in support of essay grading. We find that the choice of the most suitable method depends on the nature of the exam question and the answers, with deep-learning methods being capable of, but not guaranteeing better performance over simpler methods based on lexical overlap.
3D Convolutional Neural Networks for Ultrasound-Based Silent Speech Interfaces
Authors: László Tóth, Amin Honarmandi Shandiz
Date: 2021-04-23 00:00:00
Tasks: Action Recognition, Temporal Action Localization

Silent speech interfaces (SSI) aim to reconstruct the speech signal from a recording of the articulatory movement, such as an ultrasound video of the tongue. Currently, deep neural networks are the most successful technology for this task. The efficient solution requires methods that do not simply process single images, but are able to extract the tongue movement information from a sequence of video frames. One option for this is to apply recurrent neural structures such as the long short-term memory network (LSTM) in combination with 2D convolutional neural networks (CNNs). Here, we experiment with another approach that extends the CNN to perform 3D convolution, where the extra dimension corresponds to time. In particular, we apply the spatial and temporal convolutions in a decomposed form, which proved very successful recently in video action recognition. We find experimentally that our 3D network outperforms the CNN+LSTM model, indicating that 3D CNNs may be a feasible alternative to CNN+LSTM networks in SSI systems.
Learning to Learn to be Right for the Right Reasons
Authors: Pride Kavumba, Benjamin Heinzerling, Ana Brassard, Kentaro Inui
Date: 2021-04-23 00:00:00
Proceeding: NAACL 2021 4
Tasks: Meta-Learning

Improving model generalization on held-out data is one of the core objectives in commonsense reasoning. Recent work has shown that models trained on the dataset with superficial cues tend to perform well on the easy test set with superficial cues but perform poorly on the hard test set without superficial cues. Previous approaches have resorted to manual methods of encouraging models not to overfit to superficial cues. While some of the methods have improved performance on hard instances, they also lead to degraded performance on easy instances. Here, we propose to explicitly learn a model that does well on both the easy test set with superficial cues and hard test set without superficial cues. Using a meta-learning objective, we learn such a model that improves performance on both the easy test set and the hard test set. By evaluating our models on Choice of Plausible Alternatives (COPA) and Commonsense Explanation, we show that our proposed method leads to improved performance on both the easy test set and the hard test set upon which we observe up to 16.5 percentage points improvement over the baseline.
Multimodal Fusion with BERT and Attention Mechanism for Fake News Detection
Authors: Nguyen Manh Duc Tuan, Pham Quang Nhat Minh
Date: 2021-04-23 00:00:00
Tasks: Fake News Detection

Fake news detection is an important task for increasing the credibility of information on the media since fake news is constantly spreading on social media every day and it is a very serious concern in our society. Fake news is usually created by manipulating images, texts, and videos. In this paper, we present a novel method for detecting fake news by fusing multimodal features derived from textual and visual data. Specifically, we used a pre-trained BERT model to learn text features and a VGG-19 model pre-trained on the ImageNet dataset to extract image features. We proposed a scale-dot product attention mechanism to capture the relationship between text features and visual features. Experimental results showed that our approach performs better than the current state-of-the-art method on a public Twitter dataset by 3.1% accuracy.
LeBenchmark: A Reproducible Framework for Assessing Self-Supervised Representation Learning from Speech
Authors: Solene Evain, Ha Nguyen, Hang Le, Marcely Zanon Boito, Salima Mdhaffar, Sina Alisamir, Ziyi Tong, Natalia Tomashenko, Marco Dinarelli, Titouan Parcollet, Alexandre Allauzen, Yannick Esteve, Benjamin Lecouteux, Francois Portet, Solange Rossato, Fabien Ringeval, Didier Schwab, Laurent Besacier
Date: 2021-04-23 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Emotion Recognition, Representation Learning, Self-Supervised Learning, speech-recognition, Speech Recognition, Spoken Language Understanding, Translation

Self-Supervised Learning (SSL) using huge unlabeled data has been successfully explored for image and natural language processing. Recent works also investigated SSL from speech. They were notably successful to improve performance on downstream tasks such as automatic speech recognition (ASR). While these works suggest it is possible to reduce dependence on labeled data for building efficient speech systems, their evaluation was mostly made on ASR and using multiple and heterogeneous experimental settings (most of them for English). This questions the objective comparison of SSL approaches and the evaluation of their impact on building speech systems. In this paper, we propose LeBenchmark: a reproducible framework for assessing SSL from speech. It not only includes ASR (high and low resource) tasks but also spoken language understanding, speech translation and emotion recognition. We also focus on speech technologies in a language different than English: French. SSL models of different sizes are trained from carefully sourced and documented datasets. Experiments show that SSL is beneficial for most but not all tasks which confirms the need for exhaustive and reliable benchmarks to evaluate its real impact. LeBenchmark is shared with the scientific community for reproducible research in SSL from speech.
BERT-CoQAC: BERT-based Conversational Question Answering in Context
Authors: Munazza Zaib, Dai Hoang Tran, Subhash Sagar, Adnan Mahmood, Wei E. Zhang, Quan Z. Sheng
Date: 2021-04-23 00:00:00
Tasks: Conversational Question Answering, Language Modelling, Natural Language Understanding, Question Answering, Reading Comprehension

As one promising way to inquire about any particular information through a dialog with the bot, question answering dialog systems have gained increasing research interests recently. Designing interactive QA systems has always been a challenging task in natural language processing and used as a benchmark to evaluate a machine's ability of natural language understanding. However, such systems often struggle when the question answering is carried out in multiple turns by the users to seek more information based on what they have already learned, thus, giving rise to another complicated form called Conversational Question Answering (CQA). CQA systems are often criticized for not understanding or utilizing the previous context of the conversation when answering the questions. To address the research gap, in this paper, we explore how to integrate conversational history into the neural machine comprehension system. On one hand, we introduce a framework based on a publically available pre-trained language model called BERT for incorporating history turns into the system. On the other hand, we propose a history selection mechanism that selects the turns that are relevant and contributes the most to answer the current question. Experimentation results revealed that our framework is comparable in performance with the state-of-the-art models on the QuAC leader board. We also conduct a number of experiments to show the side effects of using entire context information which brings unnecessary information and noise signals resulting in a decline in the model's performance.