pwc-archive/papers-with-abstracts huggingface dataset:
Implicit Rank-Minimizing Autoencoder
Authors: Li Jing, Jure Zbontar, Yann Lecun
Date: 2020-10-01 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Decoder, Image Generation, Representation Learning, Semi-Supervised Image Classification

An important component of autoencoders is the method by which the information capacity of the latent representation is minimized or limited. In this work, the rank of the covariance matrix of the codes is implicitly minimized by relying on the fact that gradient descent learning in multi-layer linear networks leads to minimum-rank solutions. By inserting a number of extra linear layers between the encoder and the decoder, the system spontaneously learns representations with a low effective dimension. The model, dubbed Implicit Rank-Minimizing Autoencoder (IRMAE), is simple, deterministic, and learns compact latent spaces. We demonstrate the validity of the method on several image generation and representation learning tasks.
Beyond The Text: Analysis of Privacy Statements through Syntactic and Semantic Role Labeling
Authors: Yan Shvartzshnaider, Ananth Balashankar, Vikas Patidar, Thomas Wies, Lakshminarayanan Subramanian
Date: 2020-10-01 00:00:00
Tasks: Question Answering, Semantic Role Labeling

This paper formulates a new task of extracting privacy parameters from a privacy policy, through the lens of Contextual Integrity, an established social theory framework for reasoning about privacy norms. Privacy policies, written by lawyers, are lengthy and often comprise incomplete and vague statements. In this paper, we show that traditional NLP tasks, including the recently proposed Question-Answering based solutions, are insufficient to address the privacy parameter extraction problem and provide poor precision and recall. We describe 4 different types of conventional methods that can be partially adapted to address the parameter extraction task with varying degrees of success: Hidden Markov Models, BERT fine-tuned models, Dependency Type Parsing (DP) and Semantic Role Labeling (SRL). Based on a detailed evaluation across 36 real-world privacy policies of major enterprises, we demonstrate that a solution combining syntactic DP coupled with type-specific SRL tasks provides the highest accuracy for retrieving contextual privacy parameters from privacy statements. We also observe that incorporating domain-specific knowledge is critical to achieving high precision and recall, thus inspiring new NLP research to address this important problem in the privacy domain.
Near-imperceptible Neural Linguistic Steganography via Self-Adjusting Arithmetic Coding
Authors: Jiaming Shen, Heng Ji, Jiawei Han
Date: 2020-10-01 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Language Modeling, Language Modelling, Linguistic steganography

Linguistic steganography studies how to hide secret messages in natural language cover texts. Traditional methods aim to transform a secret message into an innocent text via lexical substitution or syntactical modification. Recently, advances in neural language models (LMs) enable us to directly generate cover text conditioned on the secret message. In this study, we present a new linguistic steganography method which encodes secret messages using self-adjusting arithmetic coding based on a neural language model. We formally analyze the statistical imperceptibility of this method and empirically show it outperforms the previous state-of-the-art methods on four datasets by 15.3% and 38.9% in terms of bits/word and KL metrics, respectively. Finally, human evaluations show that 51% of generated cover texts can indeed fool eavesdroppers.
Explaining Convolutional Neural Networks through Attribution-Based Input Sampling and Block-Wise Feature Aggregation
Authors: Sam Sattarzadeh, Mahesh Sudhakar, Anthony Lem, Shervin Mehryar, K. N. Plataniotis, Jongseong Jang, Hyunwoo Kim, Yeonjeong Jeong, Sangmin Lee, Kyunghoon Bae
Date: 2020-10-01 00:00:00
Tasks: Explainable Artificial Intelligence (XAI)

As an emerging field in Machine Learning, Explainable AI (XAI) has been offering remarkable performance in interpreting the decisions made by Convolutional Neural Networks (CNNs). To achieve visual explanations for CNNs, methods based on class activation mapping and randomized input sampling have gained great popularity. However, the attribution methods based on these techniques provide lower resolution and blurry explanation maps that limit their explanation power. To circumvent this issue, visualization based on various layers is sought. In this work, we collect visualization maps from multiple layers of the model based on an attribution-based input sampling technique and aggregate them to reach a fine-grained and complete explanation. We also propose a layer selection strategy that applies to the whole family of CNN-based models, based on which our extraction framework is applied to visualize the last layers of each convolutional block of the model. Moreover, we perform an empirical analysis of the efficacy of derived lower-level information to enhance the represented attributions. Comprehensive experiments conducted on shallow and deep models trained on natural and industrial datasets, using both ground-truth and model-truth based evaluation metrics validate our proposed algorithm by meeting or outperforming the state-of-the-art methods in terms of explanation ability and visual quality, demonstrating that our method shows stability regardless of the size of objects or instances to be explained.
Learning Variational Word Masks to Improve the Interpretability of Neural Text Classifiers
Authors: Hanjie Chen, Yangfeng Ji
Date: 2020-10-01 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Classification, General Classification, text-classification, Text Classification

To build an interpretable neural text classifier, most of the prior work has focused on designing inherently interpretable models or finding faithful explanations. A new line of work on improving model interpretability has just started, and many existing methods require either prior information or human annotations as additional inputs in training. To address this limitation, we propose the variational word mask (VMASK) method to automatically learn task-specific important words and reduce irrelevant information on classification, which ultimately improves the interpretability of model predictions. The proposed method is evaluated with three neural text classifiers (CNN, LSTM, and BERT) on seven benchmark text classification datasets. Experiments show the effectiveness of VMASK in improving both model prediction accuracy and interpretability.
Event Detection in Twitter by Weighting Tweet's Features
Authors: Parinaz Rahimizadeh, Mohammad Javad Shayegan
Date: 2020-10-01 00:00:00
Tasks: Event Detection

In recent years, people spend a lot of time on social networks. They use social networks as a place to comment on personal or public events. Thus, a large amount of information is generated and shared daily in these networks. Using such a massive amount of information can help authorities to react to events accurately and timely. In this study, the social network investigated is Twitter. The main idea of this research is to differentiate among tweets based on some of their features. This study aimed at investigating the performance of event detection by weighting three attributes of tweets; including the followers count, the retweets count, and the user location. The results show that the average execution time and the precision of event detection in the presented method improved 27% and 31%, respectively, than the base method. Another result of this research is the ability to detect all events (including hot events and less important ones) in the presented method.
Predicting User Engagement Status for Online Evaluation of Intelligent Assistants
Authors: Rui Meng, Zhen Yue, Alyssa Glass
Date: 2020-10-01 00:00:00
Tasks: Recommendation Systems

Evaluation of intelligent assistants in large-scale and online settings remains an open challenge. User behavior-based online evaluation metrics have demonstrated great effectiveness for monitoring large-scale web search and recommender systems. Therefore, we consider predicting user engagement status as the very first and critical step to online evaluation for intelligent assistants. In this work, we first proposed a novel framework for classifying user engagement status into four categories -- fulfillment, continuation, reformulation and abandonment. We then demonstrated how to design simple but indicative metrics based on the framework to quantify user engagement levels. We also aim for automating user engagement prediction with machine learning methods. We compare various models and features for predicting engagement status using four real-world datasets. We conducted detailed analyses on features and failure cases to discuss the performance of current models as well as challenges.
VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models
Authors: Zhisheng Xiao, Karsten Kreis, Jan Kautz, Arash Vahdat
Date: 2020-10-01 00:00:00
Proceeding: ICLR 2021 1
Tasks: Image Generation, Out-of-Distribution Detection

Energy-based models (EBMs) have recently been successful in representing complex distributions of small images. However, sampling from them requires expensive Markov chain Monte Carlo (MCMC) iterations that mix slowly in high dimensional pixel space. Unlike EBMs, variational autoencoders (VAEs) generate samples quickly and are equipped with a latent space that enables fast traversal of the data manifold. However, VAEs tend to assign high probability density to regions in data space outside the actual data distribution and often fail at generating sharp images. In this paper, we propose VAEBM, a symbiotic composition of a VAE and an EBM that offers the best of both worlds. VAEBM captures the overall mode structure of the data distribution using a state-of-the-art VAE and it relies on its EBM component to explicitly exclude non-data-like regions from the model and refine the image samples. Moreover, the VAE component in VAEBM allows us to speed up MCMC updates by reparameterizing them in the VAE's latent space. Our experimental results show that VAEBM outperforms state-of-the-art VAEs and EBMs in generative quality on several benchmark image datasets by a large margin. It can generate high-quality images as large as 256$\times$256 pixels with short MCMC chains. We also demonstrate that VAEBM provides complete mode coverage and performs well in out-of-distribution detection. The source code is available at https://github.com/NVlabs/VAEBM
Multiscale Detection of Cancerous Tissue in High Resolution Slide Scans
Authors: Qingchao Zhang, Coy D. Heldermon, Corey Toler-Franklin
Date: 2020-10-01 00:00:00
Tasks: Cell Detection, Vocal Bursts Intensity Prediction

We present an algorithm for multi-scale tumor (chimeric cell) detection in high resolution slide scans. The broad range of tumor sizes in our dataset pose a challenge for current Convolutional Neural Networks (CNN) which often fail when image features are very small (8 pixels). Our approach modifies the effective receptive field at different layers in a CNN so that objects with a broad range of varying scales can be detected in a single forward pass. We define rules for computing adaptive prior anchor boxes which we show are solvable under the equal proportion interval principle. Two mechanisms in our CNN architecture alleviate the effects of non-discriminative features prevalent in our data - a foveal detection algorithm that incorporates a cascade residual-inception module and a deconvolution module with additional context information. When integrated into a Single Shot MultiBox Detector (SSD), these additions permit more accurate detection of small-scale objects. The results permit efficient real-time analysis of medical images in pathology and related biomedical research fields.
Tabular GANs for uneven distribution
Authors: Insaf Ashrapov
Date: 2020-10-01 00:00:00
Tasks: Image Generation, Tabular Data Generation

GANs are well known for success in the realistic image generation. However, they can be applied in tabular data generation as well. We will review and examine some recent papers about tabular GANs in action. We will generate data to make train distribution bring closer to the test. Then compare model performance trained on the initial train dataset, with trained on the train with GAN generated data, also we train the model by sampling train by adversarial training. We show that using GAN might be an option in case of uneven data distribution between train and test data.
Universal consistency and rates of convergence of multiclass prototype algorithms in metric spaces
Authors: László Györfi, Roi Weiss
Date: 2020-10-01 00:00:00

We study universal consistency and convergence rates of simple nearest-neighbor prototype rules for the problem of multiclass classification in metric paces. We first show that a novel data-dependent partitioning rule, named Proto-NN, is universally consistent in any metric space that admits a universally consistent rule. Proto-NN is a significant simplification of OptiNet, a recently proposed compression-based algorithm that, to date, was the only algorithm known to be universally consistent in such a general setting. Practically, Proto-NN is simpler to implement and enjoys reduced computational complexity. We then proceed to study convergence rates of the excess error probability. We first obtain rates for the standard $k$-NN rule under a margin condition and a new generalized-Lipschitz condition. The latter is an extension of a recently proposed modified-Lipschitz condition from $\mathbb R^d$ to metric spaces. Similarly to the modified-Lipschitz condition, the new condition avoids any boundness assumptions on the data distribution. While obtaining rates for Proto-NN is left open, we show that a second prototype rule that hybridizes between $k$-NN and Proto-NN achieves the same rates as $k$-NN while enjoying similar computational advantages as Proto-NN. However, as $k$-NN, this hybrid rule is not consistent in general.
StreamSoNG: A Soft Streaming Classification Approach
Authors: Wenlong Wu, James M. Keller, Jeffrey Dale, James C. Bezdek
Date: 2020-10-01 00:00:00
Tasks: Classification, Clustering, General Classification

Examining most streaming clustering algorithms leads to the understanding that they are actually incremental classification models. They model existing and newly discovered structures via summary information that we call footprints. Incoming data is normally assigned a crisp label (into one of the structures) and that structure's footprint is incrementally updated. There is no reason that these assignments need to be crisp. In this paper, we propose a new streaming classification algorithm that uses Neural Gas prototypes as footprints and produces a possibilistic label vector (of typicalities) for each incoming vector. These typicalities are generated by a modified possibilistic k-nearest neighbor algorithm. The approach is tested on synthetic and real image datasets. We compare our approach to three other streaming classifiers based on the Adaptive Random Forest, Very Fast Decision Rules, and the DenStream algorithm with excellent results.
Discontinuous Constituent Parsing as Sequence Labeling
Authors: David Vilares, Carlos Gómez-Rodríguez
Date: 2020-10-01 00:00:00
Proceeding: EMNLP 2020 11

This paper reduces discontinuous parsing to sequence labeling. It first shows that existing reductions for constituent parsing as labeling do not support discontinuities. Second, it fills this gap and proposes to encode tree discontinuities as nearly ordered permutations of the input sequence. Third, it studies whether such discontinuous representations are learnable. The experiments show that despite the architectural simplicity, under the right representation, the models are fast and accurate.
An Empirical Investigation Towards Efficient Multi-Domain Language Model Pre-training
Authors: Kristjan Arumae, Qing Sun, Parminder Bhatia
Date: 2020-10-01 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Clustering, Language Modeling, Language Modelling, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Relation Extraction

Pre-training large language models has become a standard in the natural language processing community. Such models are pre-trained on generic data (e.g. BookCorpus and English Wikipedia) and often fine-tuned on tasks in the same domain. However, in order to achieve state-of-the-art performance on out of domain tasks such as clinical named entity recognition and relation extraction, additional in domain pre-training is required. In practice, staged multi-domain pre-training presents performance deterioration in the form of catastrophic forgetting (CF) when evaluated on a generic benchmark such as GLUE. In this paper we conduct an empirical investigation into known methods to mitigate CF. We find that elastic weight consolidation provides best overall scores yielding only a 0.33% drop in performance across seven generic tasks while remaining competitive in bio-medical tasks. Furthermore, we explore gradient and latent clustering based data selection techniques to improve coverage when using elastic weight consolidation and experience replay methods.
Towards Multi-Robot Task-Motion Planning for Navigation in Belief Space
Authors: Antony Thomas, Fulvio Mastrogiovanni, Marco Baglietto
Date: 2020-10-01 00:00:00
Tasks: Motion Planning, Navigate

Autonomous robots operating in large knowledgeintensive domains require planning in the discrete (task) space and the continuous (motion) space. In knowledge-intensive domains, on the one hand, robots have to reason at the highestlevel, for example the regions to navigate to or objects to be picked up and their properties; on the other hand, the feasibility of the respective navigation tasks have to be checked at the controller execution level. Moreover, employing multiple robots offer enhanced performance capabilities over a single robot performing the same task. To this end, we present an integrated multi-robot task-motion planning framework for navigation in knowledge-intensive domains. In particular, we consider a distributed multi-robot setting incorporating mutual observations between the robots. The framework is intended for motion planning under motion and sensing uncertainty, which is formally known as belief space planning. The underlying methodology and its limitations are discussed, providing suggestions for improvements and future work. We validate key aspects of our approach in simulation.
Deep Learning Based Computer-Aided Systems for Breast Cancer Imaging : A Critical Review
Authors: Yuliana Jiménez-Gaona, María José Rodríguez-Álvarez, Vasudevan Lakshminarayanan
Date: 2020-09-30 00:00:00
Tasks: Deep Learning

This paper provides a critical review of the literature on deep learning applications in breast tumor diagnosis using ultrasound and mammography images. It also summarizes recent advances in computer-aided diagnosis (CAD) systems, which make use of new deep learning methods to automatically recognize images and improve the accuracy of diagnosis made by radiologists. This review is based upon published literature in the past decade (January 2010 January 2020). The main findings in the classification process reveal that new DL-CAD methods are useful and effective screening tools for breast cancer, thus reducing the need for manual feature extraction. The breast tumor research community can utilize this survey as a basis for their current and future studies.
Towards Scalable Bayesian Learning of Causal DAGs
Authors: Jussi Viinikka, Antti Hyttinen, Johan Pensar, Mikko Koivisto
Date: 2020-09-30 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Bayesian Inference

We give methods for Bayesian inference of directed acyclic graphs, DAGs, and the induced causal effects from passively observed complete data. Our methods build on a recent Markov chain Monte Carlo scheme for learning Bayesian networks, which enables efficient approximate sampling from the graph posterior, provided that each node is assigned a small number $K$ of candidate parents. We present algorithmic techniques to significantly reduce the space and time requirements, which make the use of substantially larger values of $K$ feasible. Furthermore, we investigate the problem of selecting the candidate parents per node so as to maximize the covered posterior mass. Finally, we combine our sampling method with a novel Bayesian approach for estimating causal effects in linear Gaussian DAG models. Numerical experiments demonstrate the performance of our methods in detecting ancestor-descendant relations, and in causal effect estimation our Bayesian method is shown to outperform previous approaches.
A local geometry of hyperedges in hypergraphs, and its applications to social networks
Authors: Dong Quan Ngoc Nguyen, Lin Xing
Date: 2020-09-29 00:00:00
Tasks: Sociology

In many real world datasets arising from social networks, there are hidden higher order relations among data points which cannot be captured using graph modeling. It is natural to use a more general notion of hypergraphs to model such social networks. In this paper, we introduce a new local geometry of hyperdges in hypergraphs which allows to capture higher order relations among data points. Furthermore based on this new geometry, we also introduce new methodology--the nearest neighbors method in hypergraphs--for analyzing datasets arising from sociology.
Machine Learning in Generation, Detection, and Mitigation of Cyberattacks in Smart Grid: A Survey
Authors: Nur Imtiazul Haque, Md Hasan Shahriar, Md Golam Dastgir, Anjan Debnath, Imtiaz Parvez, Arif Sarwat, Mohammad Ashiqur Rahman
Date: 2020-09-01 00:00:00
Tasks: BIG-bench Machine Learning

Smart grid (SG) is a complex cyber-physical system that utilizes modern cyber and physical equipment to run at an optimal operating point. Cyberattacks are the principal threats confronting the usage and advancement of the state-of-the-art systems. The advancement of SG has added a wide range of technologies, equipment, and tools to make the system more reliable, efficient, and cost-effective. Despite attaining these goals, the threat space for the adversarial attacks has also been expanded because of the extensive implementation of the cyber networks. Due to the promising computational and reasoning capability, machine learning (ML) is being used to exploit and defend the cyberattacks in SG by the attackers and system operators, respectively. In this paper, we perform a comprehensive summary of cyberattacks generation, detection, and mitigation schemes by reviewing state-of-the-art research in the SG domain. Additionally, we have summarized the current research in a structured way using tabular format. We also present the shortcomings of the existing works and possible future research direction based on our investigation.
Flatness is a Flase Friend
Authors: Diego Granziol
Date: 2021-01-01 00:00:00

Hessian based measures of flatness, such as the trace, Frobenius and spectral norms, have been argued, used and shown to relate to generalisation. In this paper we demonstrate that, for feed-forward neural networks under the cross-entropy loss, low-loss solutions with large neural network weights have small Hessian based measures of flatness. This implies that solutions obtained without L2 regularisation should be less sharp than those with despite generalising worse. We show this to be true for logistic regression, multi-layer perceptrons, simple convolutional, pre-activated and wide residual networks on the MNIST and CIFAR-$100$ datasets. Furthermore, we show that adaptive optimisation algorithms using iterate averaging, on the VGG-$16$ network and CIFAR-$100$ dataset, achieve superior generalisation to SGD but are $30 \times$ sharper. These theoretical and experimental results further advocate the need to use flatness in conjunction with the weights scale to measure generalisation \citep{neyshabur2017exploring,dziugaite2017computing}.