pwc-archive/papers-with-abstracts huggingface dataset:
Exploiting Clinically Available Delineations for CNN-based Segmentation in Radiotherapy Treatment Planning
Authors: Louis D. van Harten, Jelmer M. Wolterink, Joost J. C. Verhoeff, Ivana Išgum
Date: 2019-11-12 00:00:00
Tasks: Image Segmentation, Medical Image Segmentation, Segmentation, Semantic Segmentation

Convolutional neural networks (CNNs) have been widely and successfully used for medical image segmentation. However, CNNs are typically considered to require large numbers of dedicated expert-segmented training volumes, which may be limiting in practice. This work investigates whether clinically obtained segmentations which are readily available in picture archiving and communication systems (PACS) could provide a possible source of data to train a CNN for segmentation of organs-at-risk (OARs) in radiotherapy treatment planning. In such data, delineations of structures deemed irrelevant to the target clinical use may be lacking. To overcome this issue, we use multi-label instead of multi-class segmentation. We empirically assess how many clinical delineations would be sufficient to train a CNN for the segmentation of OARs and find that increasing the training set size beyond a limited number of images leads to sharply diminishing returns. Moreover, we find that by using multi-label segmentation, missing structures in the reference standard do not have a negative effect on overall segmentation accuracy. These results indicate that segmentations obtained in a clinical workflow can be used to train an accurate OAR segmentation model.
'Warriors of the Word' -- Deciphering Lyrical Topics in Music and Their Connection to Audio Feature Dimensions Based on a Corpus of Over 100,000 Metal Songs
Authors: Isabella Czedik-Eysenberg, Oliver Wieczorek, Christoph Reuter
Date: 2019-11-12 00:00:00

We look into the connection between the musical and lyrical content of metal music by combining automated extraction of high-level audio features and quantitative text analysis on a corpus of 124.288 song lyrics from this genre. Based on this text corpus, a topic model was first constructed using Latent Dirichlet Allocation (LDA). For a subsample of 503 songs, scores for predicting perceived musical hardness/heaviness and darkness/gloominess were extracted using audio feature models. By combining both audio feature and text analysis, we (1) offer a comprehensive overview of the lyrical topics present within the metal genre and (2) are able to establish whether or not levels of hardness and other music dimensions are associated with the occurrence of particularly harsh (and other) textual topics. Twenty typical topics were identified and projected into a topic space using multidimensional scaling (MDS). After Bonferroni correction, positive correlations were found between musical hardness and darkness and textual topics dealing with 'brutal death', 'dystopia', 'archaisms and occultism', 'religion and satanism', 'battle' and '(psychological) madness', while there is a negative associations with topics like 'personal life' and 'love and romance'.
Iteratively Training Look-Up Tables for Network Quantization
Authors: Fabien Cardinaux, Stefan Uhlich, Kazuki Yoshiyama, Javier Alonso Garcia, Lukas Mauch, Stephen Tiedemann, Thomas Kemp, Akira Nakamura
Date: 2019-11-12 00:00:00
Proceeding: NIPS Workshop CDNNRIA 2018
Tasks: Network Pruning, Quantization

Operating deep neural networks (DNNs) on devices with limited resources requires the reduction of their memory as well as computational footprint. Popular reduction methods are network quantization or pruning, which either reduce the word length of the network parameters or remove weights from the network if they are not needed. In this article we discuss a general framework for network reduction which we call `Look-Up Table Quantization` (LUT-Q). For each layer, we learn a value dictionary and an assignment matrix to represent the network weights. We propose a special solver which combines gradient descent and a one-step k-means update to learn both the value dictionaries and assignment matrices iteratively. This method is very flexible: by constraining the value dictionary, many different reduction problems such as non-uniform network quantization, training of multiplierless networks, network pruning or simultaneous quantization and pruning can be implemented without changing the solver. This flexibility of the LUT-Q method allows us to use the same method to train networks for different hardware capabilities.
Accelerating Training in Pommerman with Imitation and Reinforcement Learning
Authors: Hardik Meisheri, Omkar Shelke, Richa Verma, Harshad Khadilkar
Date: 2019-11-12 00:00:00
Tasks: Imitation Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

The Pommerman simulation was recently developed to mimic the classic Japanese game Bomberman, and focuses on competitive gameplay in a multi-agent setting. We focus on the 2$\times$2 team version of Pommerman, developed for a competition at NeurIPS 2018. Our methodology involves training an agent initially through imitation learning on a noisy expert policy, followed by a proximal-policy optimization (PPO) reinforcement learning algorithm. The basic PPO approach is modified for stable transition from the imitation learning phase through reward shaping, action filters based on heuristics, and curriculum learning. The proposed methodology is able to beat heuristic and pure reinforcement learning baselines with a combined 100,000 training games, significantly faster than other non-tree-search methods in literature. We present results against multiple agents provided by the developers of the simulation, including some that we have enhanced. We include a sensitivity analysis over different parameters, and highlight undesirable effects of some strategies that initially appear promising. Since Pommerman is a complex multi-agent competitive environment, the strategies developed here provide insights into several real-world problems with characteristics such as partial observability, decentralized execution (without communication), and very sparse and delayed rewards.
Combinatorial Optimization by Graph Pointer Networks and Hierarchical Reinforcement Learning
Authors: Qiang Ma, Suwen Ge, Danyang He, Darshan Thaker, Iddo Drori
Date: 2019-11-12 00:00:00
Tasks: Combinatorial Optimization, Graph Embedding, Hierarchical Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Traveling Salesman Problem

In this work, we introduce Graph Pointer Networks (GPNs) trained using reinforcement learning (RL) for tackling the traveling salesman problem (TSP). GPNs build upon Pointer Networks by introducing a graph embedding layer on the input, which captures relationships between nodes. Furthermore, to approximate solutions to constrained combinatorial optimization problems such as the TSP with time windows, we train hierarchical GPNs (HGPNs) using RL, which learns a hierarchical policy to find an optimal city permutation under constraints. Each layer of the hierarchy is designed with a separate reward function, resulting in stable training. Our results demonstrate that GPNs trained on small-scale TSP50/100 problems generalize well to larger-scale TSP500/1000 problems, with shorter tour lengths and faster computational times. We verify that for constrained TSP problems such as the TSP with time windows, the feasible solutions found via hierarchical RL training outperform previous baselines. In the spirit of reproducible research we make our data, models, and code publicly available.
Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks
Authors: Aditya Golatkar, Alessandro Achille, Stefano Soatto
Date: 2019-11-12 00:00:00
Proceeding: CVPR 2020 6

We explore the problem of selectively forgetting a particular subset of the data used for training a deep neural network. While the effects of the data to be forgotten can be hidden from the output of the network, insights may still be gleaned by probing deep into its weights. We propose a method for "scrubbing'" the weights clean of information about a particular set of training data. The method does not require retraining from scratch, nor access to the data originally used for training. Instead, the weights are modified so that any probing function of the weights is indistinguishable from the same function applied to the weights of a network trained without the data to be forgotten. This condition is a generalized and weaker form of Differential Privacy. Exploiting ideas related to the stability of stochastic gradient descent, we introduce an upper-bound on the amount of information remaining in the weights, which can be estimated efficiently even for deep neural networks.
Short-term forecasting of solar irradiance without local telemetry: a generalized model using satellite data
Authors: Jesus Lago, Karel De Brabandere, Fjo De Ridder, Bart De Schutter
Date: 2019-11-12 00:00:00

Due to the increasing integration of solar power into the electrical grid, forecasting short-term solar irradiance has become key for many applications, e.g.~operational planning, power purchases, reserve activation, etc. In this context, as solar generators are geographically dispersed and ground measurements are not always easy to obtain, it is very important to have general models that can predict solar irradiance without the need of local data. In this paper, a model that can perform short-term forecasting of solar irradiance in any general location without the need of ground measurements is proposed. To do so, the model considers satellite-based measurements and weather-based forecasts, and employs a deep neural network structure that is able to generalize across locations; particularly, the network is trained only using a small subset of sites where ground data is available, and the model is able to generalize to a much larger number of locations where ground data does not exist. As a case study, 25 locations in The Netherlands are considered and the proposed model is compared against four local models that are individually trained for each location using ground measurements. Despite the general nature of the model, it is shown show that the proposed model is equal or better than the local models: when comparing the average performance across all the locations and prediction horizons, the proposed model obtains a 31.31% rRMSE (relative root mean square error) while the best local model achieves a 32.01% rRMSE.
Efficient Fair Principal Component Analysis
Authors: Mohammad Mahdi Kamani, Farzin Haddadpour, Rana Forsati, Mehrdad Mahdavi
Date: 2019-11-12 00:00:00
Tasks: Dimensionality Reduction, Fairness

It has been shown that dimension reduction methods such as PCA may be inherently prone to unfairness and treat data from different sensitive groups such as race, color, sex, etc., unfairly. In pursuit of fairness-enhancing dimensionality reduction, using the notion of Pareto optimality, we propose an adaptive first-order algorithm to learn a subspace that preserves fairness, while slightly compromising the reconstruction loss. Theoretically, we provide sufficient conditions that the solution of the proposed algorithm belongs to the Pareto frontier for all sensitive groups; thereby, the optimal trade-off between overall reconstruction loss and fairness constraints is guaranteed. We also provide the convergence analysis of our algorithm and show its efficacy through empirical studies on different datasets, which demonstrates superior performance in comparison with state-of-the-art algorithms. The proposed fairness-aware PCA algorithm can be efficiently generalized to multiple group sensitive features and effectively reduce the unfairness decisions in downstream tasks such as classification.
Fairness-Aware Neural Réyni Minimization for Continuous Features
Authors: Vincent Grari, Boris Ruf, Sylvain Lamprier, Marcin Detyniecki
Date: 2019-11-12 00:00:00
Tasks: Fairness

The past few years have seen a dramatic rise of academic and societal interest in fair machine learning. While plenty of fair algorithms have been proposed recently to tackle this challenge for discrete variables, only a few ideas exist for continuous ones. The objective in this paper is to ensure some independence level between the outputs of regression models and any given continuous sensitive variables. For this purpose, we use the Hirschfeld-Gebelein-R\'enyi (HGR) maximal correlation coefficient as a fairness metric. We propose two approaches to minimize the HGR coefficient. First, by reducing an upper bound of the HGR with a neural network estimation of the $\chi^{2}$ divergence. Second, by minimizing the HGR directly with an adversarial neural network architecture. The idea is to predict the output Y while minimizing the ability of an adversarial neural network to find the estimated transformations which are required to predict the HGR coefficient. We empirically assess and compare our approaches and demonstrate significant improvements on previously presented work in the field.
Machine Intelligence at the Edge with Learning Centric Power Allocation
Authors: Shuai Wang, Yik-Chung Wu, Minghua Xia, Rui Wang, H. Vincent Poor
Date: 2019-11-12 00:00:00
Tasks: Fairness, Learning Theory

While machine-type communication (MTC) devices generate considerable amounts of data, they often cannot process the data due to limited energy and computational power. To empower MTC with intelligence, edge machine learning has been proposed. However, power allocation in this paradigm requires maximizing the learning performance instead of the communication throughput, for which the celebrated water-filling and max-min fairness algorithms become inefficient. To this end, this paper proposes learning centric power allocation (LCPA), which provides a new perspective on radio resource allocation in learning driven scenarios. By employing 1) an empirical classification error model that is supported by learning theory and 2) an uncertainty sampling method that accounts for different distributions at users, LCPA is formulated as a nonconvex nonsmooth optimization problem, and is solved using a majorization minimization (MM) framework. To get deeper insights into LCPA, asymptotic analysis shows that the transmit powers are inversely proportional to the channel gains, and scale exponentially with the learning parameters. This is in contrast to traditional power allocations where quality of wireless channels is the only consideration. Last but not least, a large-scale optimization algorithm termed mirror-prox LCPA is further proposed to enable LCPA in large-scale settings. Extensive numerical results demonstrate that the proposed LCPA algorithms outperform traditional power allocation algorithms, and the large-scale optimization algorithm reduces the computation time by orders of magnitude compared with MM-based LCPA but still achieves competing learning performance.
Morphological Segmentation Inside-Out
Authors: Ryan Cotterell, Arun Kumar, Hinrich Schütze
Date: 2019-11-12 00:00:00
Proceeding: EMNLP 2016 11
Tasks: Morphological Analysis, Segmentation

Morphological segmentation has traditionally been modeled with non-hierarchical models, which yield flat segmentations as output. In many cases, however, proper morphological analysis requires hierarchical structure -- especially in the case of derivational morphology. In this work, we introduce a discriminative, joint model of morphological segmentation along with the orthographic changes that occur during word formation. To the best of our knowledge, this is the first attempt to approach discriminative segmentation with a context-free model. Additionally, we release an annotated treebank of 7454 English words with constituency parses, encouraging future research in this area.
Privacy-Preserving Adversarial Representation Learning in ASR: Reality or Illusion?
Authors: Brij Mohan Lal Srivastava, Aurélien Bellet, Marc Tommasi, Emmanuel Vincent
Date: 2019-11-12 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Decoder, Privacy Preserving, Representation Learning, Speaker Identification, speech-recognition, Speech Recognition

Automatic speech recognition (ASR) is a key technology in many services and applications. This typically requires user devices to send their speech data to the cloud for ASR decoding. As the speech signal carries a lot of information about the speaker, this raises serious privacy concerns. As a solution, an encoder may reside on each user device which performs local computations to anonymize the representation. In this paper, we focus on the protection of speaker identity and study the extent to which users can be recognized based on the encoded representation of their speech as obtained by a deep encoder-decoder architecture trained for ASR. Through speaker identification and verification experiments on the Librispeech corpus with open and closed sets of speakers, we show that the representations obtained from a standard architecture still carry a lot of information about speaker identity. We then propose to use adversarial training to learn representations that perform well in ASR while hiding speaker identity. Our results demonstrate that adversarial training dramatically reduces the closed-set classification accuracy, but this does not translate into increased open-set verification error hence into increased protection of the speaker identity in practice. We suggest several possible reasons behind this negative result.
Generating an Explainable ECG Beat Space With Variational Auto-Encoders
Authors: Tom Van Steenkiste, Dirk Deschrijver, Tom Dhaene
Date: 2019-11-12 00:00:00
Tasks: Classification, General Classification, Time Series, Time Series Analysis

Electrocardiogram signals are omnipresent in medicine. A vital aspect in the analysis of this data is the identification and classification of heart beat types which is often done through automated algorithms. Advancements in neural networks and deep learning have led to a high classification accuracy. However, the final adoption of these models into clinical practice is limited due to the black-box nature of the methods. In this work, we explore the use of variational auto-encoders based on linear dense networks to learn human interpretable beat embeddings in time-series data. We demonstrate that using this method, an interpretable and explainable ECG beat space can be generated, set up by characteristic base beats.
A Survey on Why-Type Question Answering Systems
Authors: Manvi Breja, Sanjay Kumar Jain
Date: 2019-11-12 00:00:00
Tasks: Question Answering, Survey, Vocal Bursts Type Prediction

Search engines such as Google, Yahoo and Baidu yield information in the form of a relevant set of web pages according to the need of the user. Question Answering Systems reduce the time taken to get an answer, to a query asked in natural language by providing the one most relevant answer. To the best of our knowledge, major research in Why-type questions began in early 2000's and our work on Why-type questions can help explore newer avenues for fact-finding and analysis. The paper presents a survey on Why-type Question Answering System, details the architecture, the processes involved in the system and suggests further areas of research.
Two Ridge Solutions for the Incremental Broad Learning System on Added Nodes
Authors: Hufei Zhu
Date: 2019-11-12 00:00:00

The original Broad Learning System (BLS) on new added nodes and its existing efficient implementation both assume the ridge parameter lambda -> 0 in the ridge inverse to approximate the generalized inverse, and compute the generalized inverse solution for the output weights. In this paper, we propose two ridge solutions for the output weights in the BLS on added nodes, where lambda -> 0 is no longer assumed, and lambda can be any positive real number. One of the proposed ridge solutions computes the output weights from the inverse Cholesky factor, which is updated efficiently by extending the existing inverse Cholesky factorization. The other proposed ridge solution computes the output weights from the ridge inverse, and updates the ridge inverse by extending the Greville's method that is a classical tool to compute the generalized inverse of partitioned matrices. For the proposed efficient ridge solution based on the inverse Cholesky factor, we also develop another implementation that is numerically more stable when the ridge parameter lambda is very small. The proposed ridge solution based on the ridge inverse and the numerically more stable implementation of the proposed efficient ridge solution require the same complexity as the original BLS and the existing efficient BLS, respectively. Moreover, the speedups of the proposed efficient ridge solution to the original BLS and the existing efficient BLS are 3 and more than 1.67 respectively, when the computational complexities for each update are compared, and the speedups are 1.99 - 2.52 and 1.31 - 1.58, respectively, when the total training time is compared by numerical experiments. On the other hand, our numerical experiments show that both the proposed ridge solutions for BLS achieve better testing accuracies than the original BLS and the existing efficient BLS.
Efficient Inverse-Free Algorithms for Extreme Learning Machine Based on the Recursive Matrix Inverse and the Inverse LDL' Factorization
Authors: Hufei Zhu, Chenghao Wei
Date: 2019-11-12 00:00:00

The inverse-free extreme learning machine (ELM) algorithm proposed in [4] was based on an inverse-free algorithm to compute the regularized pseudo-inverse, which was deduced from an inverse-free recursive algorithm to update the inverse of a Hermitian matrix. Before that recursive algorithm was applied in [4], its improved version had been utilized in previous literatures [9], [10]. Accordingly from the improved recursive algorithm [9], [10], we deduce a more efficient inverse-free algorithm to update the regularized pseudo-inverse, from which we develop the proposed inverse-free ELM algorithm 1. Moreover, the proposed ELM algorithm 2 further reduces the computational complexity, which computes the output weights directly from the updated inverse, and avoids computing the regularized pseudoinverse. Lastly, instead of updating the inverse, the proposed ELM algorithm 3 updates the LDLT factor of the inverse by the inverse LDLT factorization [11], to avoid numerical instabilities after a very large number of iterations [12]. With respect to the existing ELM algorithm, the proposed ELM algorithms 1, 2 and 3 are expected to require only (8+3)/M , (8+1)/M and (8+1)/M of complexities, respectively, where M is the output node number. In the numerical experiments, the standard ELM, the existing inverse-free ELM algorithm and the proposed ELM algorithms 1, 2 and 3 achieve the same performance in regression and classification, while all the 3 proposed algorithms significantly accelerate the existing inverse-free ELM algorithm