pwc-archive/papers-with-abstracts huggingface dataset:
FAT ALBERT: Finding Answers in Large Texts using Semantic Similarity Attention Layer based on BERT
Authors: Omar Mossad, Amgad Ahmed, Anandharaju Raju, Hari Karthikeyan, Zayed Ahmed
Date: 2020-08-22 00:00:00
Tasks: Multiple-choice, Question Answering, Reading Comprehension, Semantic Similarity, Semantic Textual Similarity, Text Summarization

Machine based text comprehension has always been a significant research field in natural language processing. Once a full understanding of the text context and semantics is achieved, a deep learning model can be trained to solve a large subset of tasks, e.g. text summarization, classification and question answering. In this paper we focus on the question answering problem, specifically the multiple choice type of questions. We develop a model based on BERT, a state-of-the-art transformer network. Moreover, we alleviate the ability of BERT to support large text corpus by extracting the highest influence sentences through a semantic similarity model. Evaluations of our proposed model demonstrate that it outperforms the leading models in the MovieQA challenge and we are currently ranked first in the leader board with test accuracy of 87.79%. Finally, we discuss the model shortcomings and suggest possible improvements to overcome these limitations.
Cross-Utterance Language Models with Acoustic Error Sampling
Authors: G. Sun, C. Zhang, P. C. Woodland
Date: 2020-08-19 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

The effective exploitation of richer contextual information in language models (LMs) is a long-standing research problem for automatic speech recognition (ASR). A cross-utterance LM (CULM) is proposed in this paper, which augments the input to a standard long short-term memory (LSTM) LM with a context vector derived from past and future utterances using an extraction network. The extraction network uses another LSTM to encode surrounding utterances into vectors which are integrated into a context vector using either a projection of LSTM final hidden states, or a multi-head self-attentive layer. In addition, an acoustic error sampling technique is proposed to reduce the mismatch between training and test-time. This is achieved by considering possible ASR errors into the model training procedure, and can therefore improve the word error rate (WER). Experiments performed on both AMI and Switchboard datasets show that CULMs outperform the LSTM LM baseline WER. In particular, the CULM with a self-attentive layer-based extraction network and acoustic error sampling achieves 0.6% absolute WER reduction on AMI, 0.3% WER reduction on the Switchboard part and 0.9% WER reduction on the Callhome part of Eval2000 test set over the respective baselines.
A Stance Data Set on Polarized Conversations on Twitter about the Efficacy of Hydroxychloroquine as a Treatment for COVID-19
Authors: Ece Çiğdem Mutlu, Toktam A. Oghaz, Jasser Jasser, Ege Tütüncüler, Amirarsalan Rajabi, Aida Tayebi, Ozlem Ozmen, Ivan Garibay
Date: 2020-08-19 00:00:00
Tasks: Stance Detection

At the time of this study, the SARS-CoV-2 virus that caused the COVID-19 pandemic has spread significantly across the world. Considering the uncertainty about policies, health risks, financial difficulties, etc. the online media, specially the Twitter platform, is experiencing a high volume of activity related to this pandemic. Among the hot topics, the polarized debates about unconfirmed medicines for the treatment and prevention of the disease have attracted significant attention from online media users. In this work, we present a stance data set, COVID-CQ, of user-generated content on Twitter in the context of COVID-19. We investigated more than 14 thousand tweets and manually annotated the opinions of the tweet initiators regarding the use of "chloroquine" and "hydroxychloroquine" for the treatment or prevention of COVID-19. To the best of our knowledge, COVID-CQ is the first data set of Twitter users' stances in the context of the COVID-19 pandemic, and the largest Twitter data set on users' stances towards a claim, in any domain. We have made this data set available to the research community via GitHub. We expect this data set to be useful for many research purposes, including stance detection, evolution and dynamics of opinions regarding this outbreak, and changes in opinions in response to the exogenous shocks such as policy decisions and events.
Pursuing a Prospective Perspective
Authors: Steven Kearnes
Date: 2020-08-26 00:00:00

Retrospective testing of predictive models does not consider the real-world context in which models are deployed. Prospective validation, on the other hand, enables meaningful comparisons between data generation processes by incorporating trained models and considering the subjective decisions that affect reproducibility. Prospective experiments are essential for consistent progress in modeling.
Spatial Classification With Limited Observations Based On Physics-Aware Structural Constraint
Authors: Arpan Man Sainju, Wenchong He, Zhe Jiang, Da Yan, Haiquan Chen
Date: 2020-08-25 00:00:00
Tasks: General Classification, Imputation

Spatial classification with limited feature observations has been a challenging problem in machine learning. The problem exists in applications where only a subset of sensors are deployed at certain spots or partial responses are collected in field surveys. Existing research mostly focuses on addressing incomplete or missing data, e.g., data cleaning and imputation, classification models that allow for missing feature values or model missing features as hidden variables in the EM algorithm. These methods, however, assume that incomplete feature observations only happen on a small subset of samples, and thus cannot solve problems where the vast majority of samples have missing feature observations. To address this issue, we recently proposed a new approach that incorporates physics-aware structural constraint into the model representation. Our approach assumes that a spatial contextual feature is observed for all sample locations and establishes spatial structural constraint from the underlying spatial contextual feature map. We design efficient algorithms for model parameter learning and class inference. This paper extends our recent approach by allowing feature values of samples in each class to follow a multi-modal distribution. We propose learning algorithms for the extended model with multi-modal distribution. Evaluations on real-world hydrological applications show that our approach significantly outperforms baseline methods in classification accuracy, and the multi-modal extension is more robust than our early single-modal version especially when feature distribution in training samples is multi-modal. Computational experiments show that the proposed solution is computationally efficient on large datasets.
The Effect of Various Strengths of Noises and Data Augmentations on Classification of Short Single-Lead ECG Signals Using Deep Neural Networks
Authors: Faezeh Nejati Hatamian, Amirabbas Davari, Andreas Maier
Date: 2020-08-14 00:00:00
Tasks: Denoising

Due to the multiple imperfections during the signal acquisition, Electrocardiogram (ECG) datasets are typically contaminated with numerous types of noise, like salt and pepper and baseline drift. These datasets may contain different recordings with various types of noise [1] and thus, denoising may not be the easiest task. Furthermore, usually, the number of labeled bio-signals is very limited for a proper classification task.
Accelerated reactive transport simulations in heterogeneous porous media using Reaktoro and Firedrake
Authors: Svetlana Kyas, Diego Volpatto, Martin O. Saar, Allan M. M. Leal
Date: 2020-08-17 00:00:00
Tasks: BIG-bench Machine Learning

This work investigates the performance of the on-demand machine learning (ODML) algorithm introduced in Leal et al. (2020) when applied to different reactive transport problems in heterogeneous porous media. ODML was devised to accelerate the computationally expensive geochemical reaction calculations in reactive transport simulations. We demonstrate that the ODML algorithm speeds up these calculations by one to three orders of magnitude. Such acceleration, in turn, significantly accelerates the entire reactive transport simulation. The numerical experiments are performed by implementing the coupling of two open-source software packages: Reaktoro (Leal, 2015) and Firedrake (Rathgeber et al., 2016).
Spatial Information Considered Network for Scene Classification
Authors: Chao Tao, Weipeng Lu, Ji Qi and Hao Wang
Date: 2020-05-18 00:00:00
Proceeding: IEEE Geoscience and Remote Sensing Letters 2020 5
Tasks: Classification, Diversity, General Classification, Scene Classification

Remote sensing image (RSI) scene classification is a fundamental problem for understanding high-resolution RSIs. More recently, deep learning methods, especially convolutional neural networks (CNNs), and large datasets have greatly promoted the RSI scene classification. However, deep learning methods rely heavily on the visual features extracted from the patches cropped from original RSIs, so the intraclass diversity and interclass similarity are two big challenges. To address these problems, in this paper, we propose a spatial information considered model to learn more discriminative features. By combining CNN and recurrent neural network, the proposed method can exploit both local and long-range spatial relation information to enhance the representational ability of the learned features. As the initial visual features of a single patch are transformed into higher-level features with spatial information, the proposed method achieves more accurate scene classification. Besides, we present an RSI scene classification dataset named as CSU-RSISC10 dataset to preserve the spatial information between scenes in a new way of organization. Experiments demonstrate that the proposed method outperforms other three state-of-the-art methods in scene classification using CSU-RSISC10 dataset.
Continuously Index Domain Adaptation
Authors: Hao Wang, Hao He, Dina Katabi
Date: 2020-07-01 00:00:00
Proceeding: ICML 2020 7
Tasks: Continuously Indexed Domain Adaptation, Domain Adaptation

Existing domain adaptation focuses on transferring knowledge between domains with categorical indices (e.g., between datasets A and B). However, many tasks involve continuously indexed domains. For example, in medical applications, one often needs to transfer disease analysis and prediction across patients of different ages, where age acts as a continuous domain index. Such tasks are challenging for prior domain adaptation methods since they ignore the underlying relation among domains. In this paper, we propose the first method for continuously indexed domain adaptation. Our approach combines traditional adversarial adaptation with a novel discriminator that models the encoding-conditioned domain index distribution. Our theoretical analysis demonstrates the value of leveraging the domain index to generate invariant features across a continuous range of domains. Our empirical results show that our approach outperforms the state-of-the-art domain adaption methods on both synthetic and real-world medical datasets.
Reconciling emergences: An information-theoretic approach to identify causal emergence in multivariate data
Authors: Fernando E. Rosas, Pedro A. M. Mediano, Henrik J. Jensen, Anil. K. Seth, Adam B. Barrett, Robin L. Carhart-Harris, Daniel Bor
Date: 2020-04-17 00:00:00

The broad concept of emergence is instrumental in various of the most challenging open scientific questions -- yet, few quantitative theories of what constitutes emergent phenomena have been proposed. This article introduces a formal theory of causal emergence in multivariate systems, which studies the relationship between the dynamics of parts of a system and macroscopic features of interest. Our theory provides a quantitative definition of downward causation, and introduces a complementary modality of emergent behaviour -- which we refer to as causal decoupling. Moreover, the theory allows practical criteria that can be efficiently calculated in large systems, making our framework applicable in a range of scenarios of practical interest. We illustrate our findings in a number of case studies, including Conway's Game of Life, Reynolds' flocking model, and neural activity as measured by electrocorticography.
Neural Models for Information Retrieval
Authors: Bhaskar Mitra, Nick Craswell
Date: 2017-05-03 00:00:00
Tasks: BIG-bench Machine Learning, Information Retrieval, Learning-To-Rank, Retrieval

Neural ranking models for information retrieval (IR) use shallow or deep neural networks to rank search results in response to a query. Traditional learning to rank models employ machine learning techniques over hand-crafted IR features. By contrast, neural models learn representations of language from raw text that can bridge the gap between query and document vocabulary. Unlike classical IR models, these new machine learning based approaches are data-hungry, requiring large scale training data before they can be deployed. This tutorial introduces basic concepts and intuitions behind neural IR models, and places them in the context of traditional retrieval models. We begin by introducing fundamental concepts of IR and different neural and non-neural approaches to learning vector representations of text. We then review shallow neural IR methods that employ pre-trained neural term embeddings without learning the IR task end-to-end. We introduce deep neural networks next, discussing popular deep architectures. Finally, we review the current DNN models for information retrieval. We conclude with a discussion on potential future directions for neural IR.
Improvement of Quantum Circuits Using H-U-H Sandwich Technique with Diagonal Matrix Implementation
Authors: Iqra Naz, Muhammad Miqdad Khan
Date: 2020-08-23 00:00:00
Proceeding: Advanced Journal of Science and Engineering (Adv. J. Sci. Eng.); eISSN: 2717-0705 2020 8

Quantum circuits are ideal for most of the modern world problems. They are more efficient and reliable for many computations that are a challenge for classical computers. Beside this they are themselves more complex to deal with and too much costly to build. Cost is the most crucial factor for designing or implementation of any circuit but when it comes to quantum circuits it becomes the most unavoidable part of design technique. The hadamard-unitary sandwich technique has an amazing cost reduction potential in quantum circuits.  In this paper we have discussed about how H-U-H sandwich technique came up with a break through minimizing the circuit complexity and played leading role in cost reduction of quantum circuits by minimizing number of gates used. This method also helps in achieving high computation power and efficiency with more feasibility.
Spatial Information Inference Net: Road Extraction Using Road-Specific Contextual Information
Authors: Chao Tao, Ji Qi, Yansheng Li, Hao Wang, Haifeng Li
Date: 2019-10-28 00:00:00
Proceeding: ISPRS Journal of Photogrammetry and Remote Sensing 2019 10
Tasks: Road Segmentation, Segmentation, Semantic Segmentation

Deep neural networks perform well in road extraction from very high-resolution satellite imagery. A network with certain reasoning ability will give more satisfactory road network extraction results. In this study, we designed a spatial information inference structure, which enables multidirectional message passing between pixels when it is integrated to a typical semantic segmentation framework. Since the spatial information could be propagated and reinforced via inter layer propagation, the proposed road extraction network can learn both the local visual characteristics of the road and the global spatial structure information (such as the continuity and trend of the road). As a result, this method can effectively solve occlusions and preserve the continuity of the extracted road. The validation experiments using three large datasets of very high-resolution (VHR) satellite imagery show that the proposed method can improve road extraction accuracy and provide an output that is more in line with human expectations. Keywords: Road extraction, Semantic segmentation, Spatial information inference structure, Road-specific contextual information
Unmanned Aerial System Security using Real-time Autopilot Software Analysis
Authors: Christina Stracquodaine, Andrey Dolgikh, Matthew Davis, Victor Skormin
Date: 2016-06-07 00:00:00
Proceeding: 2016 International Conference on Unmanned Aircraft Systems (ICUAS) 2016 6
Tasks: Anomaly Detection

Given the proliferation of Unmanned Aerial Systems (UASs) for military and commercial purposes, there is a fast-approaching need for comprehensive and robust systems which secure the UAS for public safety and assurance of military goals. This paper puts forth a comprehensive system of protecting UASs against hardware and software attacks, at the lowest and highest semantic levels. This approach monitors the autopilot directly, as well as the lowest level of the operating system on-board the UAS to establish whether or not malicious software has altered the function of the aircraft. Coupled with the other recent work by the authors, this completes a system that also monitors the lowest semantic level of events on the UAS through ob-board sensors, allowing for the detection of hardware failures as well as malicious attacks. After testing a quadcopter testbed with this anomaly detection system against software attacks, it can be seen that the system is successful in the detection of changes to the control flow of the autopilot, as well as the detection of any malicious events on the UAS
Flow-edge Guided Video Completion
Authors: Chen Gao, Ayush Saraf, Jia-Bin Huang, Johannes Kopf
Date: 2020-09-03 00:00:00
Proceeding: ECCV 2020 8
Tasks: Video Inpainting

We present a new flow-based video completion algorithm. Previous flow completion methods are often unable to retain the sharpness of motion boundaries. Our method first extracts and completes motion edges, and then uses them to guide piecewise-smooth flow completion with sharp edges. Existing methods propagate colors among local flow connections between adjacent frames. However, not all missing regions in a video can be reached in this way because the motion boundaries form impenetrable barriers. Our method alleviates this problem by introducing non-local flow connections to temporally distant frames, enabling propagating video content over motion boundaries. We validate our approach on the DAVIS dataset. Both visual and quantitative results show that our method compares favorably against the state-of-the-art algorithms.
Tree Neural Networks in HOL4
Authors: Thibault Gauthier
Date: 2020-09-03 00:00:00
Tasks: BIG-bench Machine Learning

We present an implementation of tree neural networks within the proof assistant HOL4. Their architecture makes them naturally suited for approximating functions whose domain is a set of formulas. We measure the performance of our implementation and compare it with other machine learning predictors on the tasks of evaluating arithmetical expressions and estimating the truth of propositional formulas.
A Python Library for Exploratory Data Analysis on Twitter Data based on Tokens and Aggregated Origin-Destination Information
Authors: Mario Graff, Daniela Moctezuma, Sabino Miranda-Jiménez, Eric S. Tellez
Date: 2020-09-03 00:00:00

Twitter is perhaps the social media more amenable for research. It requires only a few steps to obtain information, and there are plenty of libraries that can help in this regard. Nonetheless, knowing whether a particular event is expressed on Twitter is a challenging task that requires a considerable collection of tweets. This proposal aims to facilitate, to a researcher interested, the process of mining events on Twitter by opening a collection of processed information taken from Twitter since December 2015. The events could be related to natural disasters, health issues, and people's mobility, among other studies that can be pursued with the library proposed. Different applications are presented in this contribution to illustrate the library's capabilities: an exploratory analysis of the topics discovered in tweets, a study on similarity among dialects of the Spanish language, and a mobility report on different countries. In summary, the Python library presented is applied to different domains and retrieves a plethora of information in terms of frequencies by day of words and bi-grams of words for Arabic, English, Spanish, and Russian languages. As well as mobility information related to the number of travels among locations for more than 200 countries or territories.
Fine-grained Early Frequency Attention for Deep Speaker Representation Learning
Authors: Amirhossein Hajavi, Ali Etemad
Date: 2020-09-03 00:00:00
Tasks: Deep Learning, Emotion Recognition, Representation Learning, Speaker Recognition, Speech Emotion Recognition, speech-recognition, Speech Recognition

Deep learning techniques have considerably improved speech processing in recent years. Speaker representations extracted by deep learning models are being used in a wide range of tasks such as speaker recognition and speech emotion recognition. Attention mechanisms have started to play an important role in improving deep learning models in the field of speech processing. Nonetheless, despite the fact that important speaker-related information can be embedded in individual frequency-bins of the input spectral representations, current attention models are unable to attend to fine-grained information items in spectral representations. In this paper we propose Fine-grained Early Frequency Attention (FEFA) for speaker representation learning. Our model is a simple and lightweight model that can be integrated into various CNN pipelines and is capable of focusing on information items as small as frequency-bins. We evaluate the proposed model on three tasks of speaker recognition, speech emotion recognition, and spoken digit recognition. We use Three widely used public datasets, namely VoxCeleb, IEMOCAP, and Free Spoken Digit Dataset for our experiments. We attach FEFA to several prominent deep learning models and evaluate its impact on the final performance. We also compare our work with other related works in the area. Our experiments show that by adding FEFA to different CNN architectures, performance is consistently improved by substantial margins, and the models equipped with FEFA outperform all the other attentive models. We also test our model against different levels of added noise showing improvements in robustness and less sensitivity compared to the backbone networks.