pwc-archive/papers-with-abstracts huggingface dataset:
SentiLARE: Sentiment-Aware Language Representation Learning with Linguistic Knowledge
Authors: Pei Ke, Haozhe Ji, Siyang Liu, Xiaoyan Zhu, Minlie Huang
Date: 2019-11-06 00:00:00
Proceeding: EMNLP 2020 11
Tasks: Data Augmentation, Language Modeling, Language Modelling, Representation Learning, Sentiment Analysis, TAG

Most of the existing pre-trained language representation models neglect to consider the linguistic knowledge of texts, which can promote language understanding in NLP tasks. To benefit the downstream tasks in sentiment analysis, we propose a novel language representation model called SentiLARE, which introduces word-level linguistic knowledge including part-of-speech tag and sentiment polarity (inferred from SentiWordNet) into pre-trained models. We first propose a context-aware sentiment attention mechanism to acquire the sentiment polarity of each word with its part-of-speech tag by querying SentiWordNet. Then, we devise a new pre-training task called label-aware masked language model to construct knowledge-aware language representation. Experiments show that SentiLARE obtains new state-of-the-art performance on a variety of sentiment analysis tasks.
OpenML-Python: an extensible Python API for OpenML
Authors: Matthias Feurer, Jan N. van Rijn, Arlind Kadra, Pieter Gijsbers, Neeratyoy Mallik, Sahithya Ravi, Andreas Müller, Joaquin Vanschoren, Frank Hutter
Date: 2019-11-06 00:00:00
Tasks: BIG-bench Machine Learning

OpenML is an online platform for open science collaboration in machine learning, used to share datasets and results of machine learning experiments. In this paper we introduce OpenML-Python, a client API for Python, opening up the OpenML platform for a wide range of Python-based tools. It provides easy access to all datasets, tasks and experiments on OpenML from within Python. It also provides functionality to conduct machine learning experiments, upload the results to OpenML, and reproduce results which are stored on OpenML. Furthermore, it comes with a scikit-learn plugin and a plugin mechanism to easily integrate other machine learning libraries written in Python into the OpenML ecosystem. Source code and documentation is available at https://github.com/openml/openml-python/.
Designing Evaluations of Machine Learning Models for Subjective Inference: The Case of Sentence Toxicity
Authors: Agathe Balayn, Alessandro Bozzon
Date: 2019-11-06 00:00:00
Tasks: BIG-bench Machine Learning, Decision Making, Sentence

Machine Learning (ML) is increasingly applied in real-life scenarios, raising concerns about bias in automatic decision making. We focus on bias as a notion of opinion exclusion, that stems from the direct application of traditional ML pipelines to infer subjective properties. We argue that such ML systems should be evaluated with subjectivity and bias in mind. Considering the lack of evaluation standards yet to create evaluation benchmarks, we propose an initial list of specifications to define prior to creating evaluation datasets, in order to later accurately evaluate the biases. With the example of a sentence toxicity inference system, we illustrate how the specifications support the analysis of biases related to subjectivity. We highlight difficulties in instantiating these specifications and list future work for the crowdsourcing community to help the creation of appropriate evaluation datasets.
Don't Blame the ELBO! A Linear VAE Perspective on Posterior Collapse
Authors: James Lucas, George Tucker, Roger Grosse, Mohammad Norouzi
Date: 2019-11-06 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Variational Inference

Posterior collapse in Variational Autoencoders (VAEs) arises when the variational posterior distribution closely matches the prior for a subset of latent variables. This paper presents a simple and intuitive explanation for posterior collapse through the analysis of linear VAEs and their direct correspondence with Probabilistic PCA (pPCA). We explain how posterior collapse may occur in pPCA due to local maxima in the log marginal likelihood. Unexpectedly, we prove that the ELBO objective for the linear VAE does not introduce additional spurious local maxima relative to log marginal likelihood. We show further that training a linear VAE with exact variational inference recovers an identifiable global maximum corresponding to the principal component directions. Empirically, we find that our linear analysis is predictive even for high-capacity, non-linear VAEs and helps explain the relationship between the observation noise, local maxima, and posterior collapse in deep Gaussian VAEs.
Towards Large yet Imperceptible Adversarial Image Perturbations with Perceptual Color Distance
Authors: Zhengyu Zhao, Zhuoran Liu, Martha Larson
Date: 2019-11-06 00:00:00
Proceeding: CVPR 2020 6
Tasks: Image Classification

The success of image perturbations that are designed to fool image classifier is assessed in terms of both adversarial effect and visual imperceptibility. The conventional assumption on imperceptibility is that perturbations should strive for tight $L_p$-norm bounds in RGB space. In this work, we drop this assumption by pursuing an approach that exploits human color perception, and more specifically, minimizing perturbation size with respect to perceptual color distance. Our first approach, Perceptual Color distance C&W (PerC-C&W), extends the widely-used C&W approach and produces larger RGB perturbations. PerC-C&W is able to maintain adversarial strength, while contributing to imperceptibility. Our second approach, Perceptual Color distance Alternating Loss (PerC-AL), achieves the same outcome, but does so more efficiently by alternating between the classification loss and perceptual color difference when updating perturbations. Experimental evaluation shows PerC approaches outperform conventional $L_p$ approaches in terms of robustness and transferability, and also demonstrates that the PerC distance can provide added value on top of existing structure-based methods to creating image perturbations.
High-dimensional Black-box Optimization Under Uncertainty
Authors: Hadis Anahideh, Jay Rosenberger, Victoria Chen
Date: 2019-11-06 00:00:00
Tasks: global-optimization, Vocal Bursts Intensity Prediction

Optimizing expensive black-box systems with limited data is an extremely challenging problem. As a resolution, we present a new surrogate optimization approach by addressing two gaps in prior research -- unimportant input variables and inefficient treatment of uncertainty associated with the black-box output. We first design a new flexible non-interpolating parsimonious surrogate model using a partitioning-based multivariate adaptive regression splines approach, Tree Knot MARS (TK-MARS). The proposed model is specifically designed for optimization by capturing the structure of the function, bending at near-optimal locations, and is capable of screening unimportant input variables. Furthermore, we develop a novel replication approach called \emph{Smart-Replication}, to overcome the uncertainty associated with the black-box output. The Smart-Replication approach identifies promising input points to replicate and avoids unnecessary evaluations of other data points. Smart-Replication is agnostic to the choice of a surrogate and can adapt itself to an unknown noise level. Finally to demonstrate the effectiveness of our proposed approaches we consider different complex global optimization test functions from the surrogate optimization literature. The results indicate that TK-MARS outperforms original MARS within a surrogate optimization algorithm and successfully detects important variables. The results also show that although non-interpolating surrogates can mitigate uncertainty, replication is still beneficial for optimizing highly complex black-box functions. The robustness and the quality of the final optimum solution found through Smart-Replication are competitive with that using no replications in environments with low levels of noise and using a fixed number of replications in highly noisy environments.
Unfairness towards subjective opinions in Machine Learning
Authors: Agathe Balayn, Alessandro Bozzon, Zoltan Szlavik
Date: 2019-11-06 00:00:00
Tasks: BIG-bench Machine Learning

Despite the high interest for Machine Learning (ML) in academia and industry, many issues related to the application of ML to real-life problems are yet to be addressed. Here we put forward one limitation which arises from a lack of adaptation of ML models and datasets to specific applications. We formalise a new notion of unfairness as exclusion of opinions. We propose ways to quantify this unfairness, and aid understanding its causes through visualisation. These insights into the functioning of ML-based systems hint at methods to mitigate unfairness.
Automated Left Ventricle Dimension Measurement in 2D Cardiac Ultrasound via an Anatomically Meaningful CNN Approach
Authors: Andrew Gilbert, Marit Holden, Line Eikvil, Svein Arne Aase, Eigil Samset, Kristin McLeod
Date: 2019-11-06 00:00:00

Two-dimensional echocardiography (2DE) measurements of left ventricle (LV) dimensions are highly significant markers of several cardiovascular diseases. These measurements are often used in clinical care despite suffering from large variability between observers. This variability is due to the challenging nature of accurately finding the correct temporal and spatial location of measurement endpoints in ultrasound images. These images often contain fuzzy boundaries and varying reflection patterns between frames. In this work, we present a convolutional neural network (CNN) based approach to automate 2DE LV measurements. Treating the problem as a landmark detection problem, we propose a modified U-Net CNN architecture to generate heatmaps of likely coordinate locations. To improve the network performance we use anatomically meaningful heatmaps as labels and train with a multi-component loss function. Our network achieves 13.4%, 6%, and 10.8% mean percent error on intraventricular septum (IVS), LV internal dimension (LVID), and LV posterior wall (LVPW) measurements respectively. The design outperforms other networks and matches or approaches intra-analyser expert error.
An Information Theory Approach on Deciding Spectroscopic Follow Ups
Authors: Javiera Astudillo, Pavlos Protopapas, Karim Pichara, Pablo Huijse
Date: 2019-11-06 00:00:00
Tasks: General Classification, Time Series, Time Series Analysis

Classification and characterization of variable phenomena and transient phenomena are critical for astrophysics and cosmology. These objects are commonly studied using photometric time series or spectroscopic data. Given that many ongoing and future surveys are in time-domain and given that adding spectra provide further insights but requires more observational resources, it would be valuable to know which objects should we prioritize to have spectrum in addition to time series. We propose a methodology in a probabilistic setting that determines a-priory which objects are worth taking spectrum to obtain better insights, where we focus 'insight' as the type of the object (classification). Objects for which we query its spectrum are reclassified using their full spectrum information. We first train two classifiers, one that uses photometric data and another that uses photometric and spectroscopic data together. Then for each photometric object we estimate the probability of each possible spectrum outcome. We combine these models in various probabilistic frameworks (strategies) which are used to guide the selection of follow up observations. The best strategy depends on the intended use, whether it is getting more confidence or accuracy. For a given number of candidate objects (127, equal to 5% of the dataset) for taking spectra, we improve 37% class prediction accuracy as opposed to 20% of a non-naive (non-random) best base-line strategy. Our approach provides a general framework for follow-up strategies and can be extended beyond classification and to include other forms of follow-ups beyond spectroscopy.
weg2vec: Event embedding for temporal networks
Authors: Maddalena Torricelli, Márton Karsai, Laetitia Gauvin
Date: 2019-11-06 00:00:00
Tasks: Network Embedding

Network embedding techniques are powerful to capture structural regularities in networks and to identify similarities between their local fabrics. However, conventional network embedding models are developed for static structures, commonly consider nodes only and they are seriously challenged when the network is varying in time. Temporal networks may provide an advantage in the description of real systems, but they code more complex information, which could be effectively represented only by a handful of methods so far. Here, we propose a new method of event embedding of temporal networks, called weg2vec, which builds on temporal and structural similarities of events to learn a low dimensional representation of a temporal network. This projection successfully captures latent structures and similarities between events involving different nodes at different times and provides ways to predict the final outcome of spreading processes unfolding on the temporal structure.
The Naked Sun: Malicious Cooperation Between Benign-Looking Processes
Authors: Fabio De Gaspari, Dorjan Hitaj, Giulio Pagnotta, Lorenzo De Carli, Luigi V. Mancini
Date: 2019-11-06 00:00:00
Tasks: Behavioral Malware Detection, Malware Detection

Recent progress in machine learning has generated promising results in behavioral malware detection. Behavioral modeling identifies malicious processes via features derived by their runtime behavior. Behavioral features hold great promise as they are intrinsically related to the functioning of each malware, and are therefore considered difficult to evade. Indeed, while a significant amount of results exists on evasion of static malware features, evasion of dynamic features has seen limited work. This paper thoroughly examines the robustness of behavioral malware detectors to evasion, focusing particularly on anti-ransomware evasion. We choose ransomware as its behavior tends to differ significantly from that of benign processes, making it a low-hanging fruit for behavioral detection (and a difficult candidate for evasion). Our analysis identifies a set of novel attacks that distribute the overall malware workload across a small set of cooperating processes to avoid the generation of significant behavioral features. Our most effective attack decreases the accuracy of a state-of-the-art classifier from 98.6% to 0% using only 18 cooperating processes. Furthermore, we show our attacks to be effective against commercial ransomware detectors even in a black-box setting.
Energy Efficient Federated Learning Over Wireless Communication Networks
Authors: Zhaohui Yang, Mingzhe Chen, Walid Saad, Choong Seon Hong, Mohammad Shikh-Bahaei
Date: 2019-11-06 00:00:00
Tasks: Federated Learning

In this paper, the problem of energy efficient transmission and computation resource allocation for federated learning (FL) over wireless communication networks is investigated. In the considered model, each user exploits limited local computational resources to train a local FL model with its collected data and, then, sends the trained FL model to a base station (BS) which aggregates the local FL model and broadcasts it back to all of the users. Since FL involves an exchange of a learning model between users and the BS, both computation and communication latencies are determined by the learning accuracy level. Meanwhile, due to the limited energy budget of the wireless users, both local computation energy and transmission energy must be considered during the FL process. This joint learning and communication problem is formulated as an optimization problem whose goal is to minimize the total energy consumption of the system under a latency constraint. To solve this problem, an iterative algorithm is proposed where, at every step, closed-form solutions for time allocation, bandwidth allocation, power control, computation frequency, and learning accuracy are derived. Since the iterative algorithm requires an initial feasible solution, we construct the completion time minimization problem and a bisection-based algorithm is proposed to obtain the optimal solution, which is a feasible solution to the original energy minimization problem. Numerical results show that the proposed algorithms can reduce up to 59.5% energy consumption compared to the conventional FL method.
Doppler Spectrum Classification with CNNs via Heatmap Location Encoding and a Multi-head Output Layer
Authors: Andrew Gilbert, Marit Holden, Line Eikvil, Mariia Rakhmail, Aleksandar Babic, Svein Arne Aase, Eigil Samset, Kristin McLeod
Date: 2019-11-06 00:00:00
Tasks: Decision Making, Diagnostic, General Classification

Spectral Doppler measurements are an important part of the standard echocardiographic examination. These measurements give important insight into myocardial motion and blood flow providing clinicians with parameters for diagnostic decision making. Many of these measurements can currently be performed automatically with high accuracy, increasing the efficiency of the diagnostic pipeline. However, full automation is not yet available because the user must manually select which measurement should be performed on each image. In this work we develop a convolutional neural network (CNN) to automatically classify cardiac Doppler spectra into measurement classes. We show how the multi-modal information in each spectral Doppler recording can be combined using a meta parameter post-processing mapping scheme and heatmaps to encode coordinate locations. Additionally, we experiment with several state-of-the-art network architectures to examine the tradeoff between accuracy and memory usage for resource-constrained environments. Finally, we propose a confidence metric using the values in the last fully connected layer of the network. We analyze example images that fall outside of our proposed classes to show our confidence metric can prevent many misclassifications. Our algorithm achieves 96% accuracy on a test set drawn from a separate clinical site, indicating that the proposed method is suitable for clinical adoption and enabling a fully automatic pipeline from acquisition to Doppler spectrum measurements.
Predicting Long-Term Skeletal Motions by a Spatio-Temporal Hierarchical Recurrent Network
Authors: Junfeng Hu, Zhencheng Fan, Jun Liao, Li Liu
Date: 2019-11-06 00:00:00
Tasks: Decoder, motion prediction, Prediction

The primary goal of skeletal motion prediction is to generate future motion by observing a sequence of 3D skeletons. A key challenge in motion prediction is the fact that a motion can often be performed in several different ways, with each consisting of its own configuration of poses and their spatio-temporal dependencies, and as a result, the predicted poses often converge to the motionless poses or non-human like motions in long-term prediction. This leads us to define a hierarchical recurrent network model that explicitly characterizes these internal configurations of poses and their local and global spatio-temporal dependencies. The model introduces a latent vector variable from the Lie algebra to represent spatial and temporal relations simultaneously. Furthermore, a structured stack LSTM-based decoder is devised to decode the predicted poses with a new loss function defined to estimate the quantized weight of each body part in a pose. Empirical evaluations on benchmark datasets suggest our approach significantly outperforms the state-of-the-art methods on both short-term and long-term motion prediction.
Using Residual Dipolar Couplings from Two Alignment Media to Detect Structural Homology
Authors: Ryan Yandle, Rishi Mukhopadhyay, Homayoun Valafar
Date: 2019-11-06 00:00:00
Tasks: Protein Folding

The method of Probability Density Profile Analysis has been introduced previously as a tool to find the best match between a set of experimentally generated Residual Dipolar Couplings and a set of known protein structures. While it proved effective on small databases in identifying protein fold families, and for picking the best result from computational protein folding tool ROBETTA, for larger data sets, more data is required. Here, the method of 2-D Probability Density Profile Analysis is presented which incorporates paired RDC data from 2 alignment media for N-H vectors. The method was tested using synthetic RDC data generated with +/-1 Hz error. The results show that the addition of information from a second alignment medium makes 2-D PDPA a much more effective tool that is able to identify a structure from a database of 600 protein fold family representatives.
Guiding Variational Response Generator to Exploit Persona
Authors: Bowen Wu, Mengyuan Li, Zongsheng Wang, Yifu Chen, Derek Wong, Qihang Feng, Junhong Huang, Baoxun Wang
Date: 2019-11-06 00:00:00
Proceeding: ACL 2020 6
Tasks: Response Generation

Leveraging persona information of users in Neural Response Generators (NRG) to perform personalized conversations has been considered as an attractive and important topic in the research of conversational agents over the past few years. Despite of the promising progresses achieved by recent studies in this field, persona information tends to be incorporated into neural networks in the form of user embeddings, with the expectation that the persona can be involved via the End-to-End learning. This paper proposes to adopt the personality-related characteristics of human conversations into variational response generators, by designing a specific conditional variational autoencoder based deep model with two new regularization terms employed to the loss function, so as to guide the optimization towards the direction of generating both persona-aware and relevant responses. Besides, to reasonably evaluate the performances of various persona modeling approaches, this paper further presents three direct persona-oriented metrics from different perspectives. The experimental results have shown that our proposed methodology can notably improve the performance of persona-aware response generation, and the metrics are reasonable to evaluate the results.
The Speed Submission to DIHARD II: Contributions & Lessons Learned
Authors: Md Sahidullah, Jose Patino, Samuele Cornell, Ruiqing Yin, Sunit Sivasankaran, Hervé Bredin, Pavel Korshunov, Alessio Brutti, Romain Serizel, Emmanuel Vincent, Nicholas Evans, Sébastien Marcel, Stefano Squartini, Claude Barras
Date: 2019-11-06 00:00:00
Tasks: Action Detection, Activity Detection, Clustering, speaker-diarization, Speaker Diarization, Speech Enhancement

This paper describes the speaker diarization systems developed for the Second DIHARD Speech Diarization Challenge (DIHARD II) by the Speed team. Besides describing the system, which considerably outperformed the challenge baselines, we also focus on the lessons learned from numerous approaches that we tried for single and multi-channel systems. We present several components of our diarization system, including categorization of domains, speech enhancement, speech activity detection, speaker embeddings, clustering methods, resegmentation, and system fusion. We analyze and discuss the effect of each such component on the overall diarization performance within the realistic settings of the challenge.