pwc-archive/papers-with-abstracts huggingface dataset:
An Improved Coarse-to-Fine Method for Solving Generation Tasks
Authors: Wenyv Guan, Qianying Liu, Guangzhi Han, Bin Wang, Sujian Li
Date: 2019-04-01 00:00:00
Proceeding: ALTA 2019 4
Tasks: Math, Math Word Problem Solving, Semantic Parsing

The coarse-to-fine (coarse2fine) methods have recently been widely used in the generation tasks. The methods first generate a rough sketch in the coarse stage and then use the sketch to get the final result in the fine stage. However, they usually lack the correction ability when getting a wrong sketch. To solve this problem, in this paper, we propose an improved coarse2fine model with a control mechanism, with which our method can control the influence of the sketch on the final results in the fine stage. Even if the sketch is wrong, our model still has the opportunity to get a correct result. We have experimented our model on the tasks of semantic parsing and math word problem solving. The results have shown the effectiveness of our proposed model.
Overview of the 2019 ALTA Shared Task: Sarcasm Target Identification
Authors: Diego Molla, Aditya Joshi
Date: 2019-04-01 00:00:00
Proceeding: ALTA 2019 4

We present an overview of the 2019 ALTA shared task. This is the 10th of the series of shared tasks organised by ALTA since 2010. The task was to detect the target of sarcastic comments posted on social media. We intro- duce the task, describe the data and present the results of baselines and participants. This year{'}s shared task was particularly challenging and no participating systems improved the re- sults of our baseline.
Detecting Target of Sarcasm using Ensemble Methods
Authors: Pradeesh Parameswaran, Andrew Trotman, Veronica Liesaputra, David Eyers
Date: 2019-04-01 00:00:00
Proceeding: ALTA 2019 4

We describe our methods in trying to detect the target of sarcasm as part of ALTA 2019 shared task. We use combination of ensemble of clas- sifiers and a rule-based system. Our team ob- tained a Dice-Sorensen Coefficient score of 0.37150, which placed 2nd in the public leader- board. Despite no team beating the baseline score for the private dataset, we present our findings and also some of the challenges and future improvements which can be used in or- der to tackle the problem.
Deep sparse multi-task learning for feature selection in Alzheimer’s disease diagnosis
Authors: Heung-Il Suk1 • Seong-Whan Lee1 • Dinggang Shen1, 2 •
Date: 2015-05-21 00:00:00
Proceeding: Scientific journal 2015 5
Tasks: Clustering, feature selection, Multi-class Classification, Multi-Task Learning, regression

Recently, neuroimaging-based Alzheimer’s
disease (AD) or mild cognitive impairment (MCI) diagnosis
has attracted researchers in the field, due to the increasing
prevalence of the diseases. Unfortunately, the unfavorable
high-dimensional nature of neuroimaging data, but a limited small number of samples available, makes it challenging
to build a robust computer-aided diagnosis system. Machine
learning techniques have been considered as a useful tool in
this respect and, among various methods, sparse regression
has shown its validity in the literature. However, to our best
knowledge, the existing sparse regression methods mostly
try to select features based on the optimal regression coefficients in one step. We argue that since the training feature
vectors are composed of both informative and uninformative or less informative features, the resulting optimal regression coefficients are inevidently affected by the
uninformative or less informative features. To this end, we
first propose a novel deep architecture to recursively discard
uninformative features by performing sparse multi-task
learning in a hierarchical fashion. We further hypothesize
that the optimal regression coefficients reflect the relative
importance of features in representing the target response
variables. In this regard, we use the optimal regression coefficients learned in one hierarchy as feature weighting
factors in the following hierarchy, and formulate a weighted
sparse multi-task learning method. Lastly, we also take into
account the distributional characteristics of samples per
class and use clustering-induced subclass label vectors as
target response values in our sparse regression model. In our
experiments on the ADNI cohort, we performed both binary
and multi-class classification tasks in AD/MCI diagnosis
and showed the superiority of the proposed method by
comparing with the state-of-the-art methods.
SynSin: End-to-end View Synthesis from a Single Image
Authors: Olivia Wiles, Georgia Gkioxari, Richard Szeliski, Justin Johnson
Date: 2019-12-18 00:00:00
Proceeding: CVPR 2020 6
Tasks: Novel View Synthesis

Single image view synthesis allows for the generation of new views of a scene given a single input image. This is challenging, as it requires comprehensively understanding the 3D scene from a single image. As a result, current methods typically use multiple images, train on ground-truth depth, or are limited to synthetic data. We propose a novel end-to-end model for this task; it is trained on real images without any ground-truth 3D information. To this end, we introduce a novel differentiable point cloud renderer that is used to transform a latent 3D point cloud of features into the target view. The projected features are decoded by our refinement network to inpaint missing regions and generate a realistic output image. The 3D component inside of our generative model allows for interpretable manipulation of the latent feature space at test time, e.g. we can animate trajectories from a single image. Unlike prior work, we can generate high resolution images and generalise to other input resolutions. We outperform baselines and prior work on the Matterport, Replica, and RealEstate10K datasets.
Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion
Authors: Hongxu Yin, Pavlo Molchanov, Zhizhong Li, Jose M. Alvarez, Arun Mallya, Derek Hoiem, Niraj K. Jha, Jan Kautz
Date: 2019-12-18 00:00:00
Proceeding: CVPR 2020 6
Tasks: Continual Learning, Diversity, Network Pruning, Transfer Learning

We introduce DeepInversion, a new method for synthesizing images from the image distribution used to train a deep neural network. We 'invert' a trained network (teacher) to synthesize class-conditional input images starting from random noise, without using any additional information about the training dataset. Keeping the teacher fixed, our method optimizes the input while regularizing the distribution of intermediate feature maps using information stored in the batch normalization layers of the teacher. Further, we improve the diversity of synthesized images using Adaptive DeepInversion, which maximizes the Jensen-Shannon divergence between the teacher and student network logits. The resulting synthesized images from networks trained on the CIFAR-10 and ImageNet datasets demonstrate high fidelity and degree of realism, and help enable a new breed of data-free applications - ones that do not require any real images or labeled data. We demonstrate the applicability of our proposed method to three tasks of immense practical importance -- (i) data-free network pruning, (ii) data-free knowledge transfer, and (iii) data-free continual learning. Code is available at https://github.com/NVlabs/DeepInversion
TOCO: A Framework for Compressing Neural Network Models Based on Tolerance Analysis
Authors: Soroosh Khoram, Jing Li
Date: 2019-12-18 00:00:00
Tasks: Active Learning, Neural Network Compression

Neural network compression methods have enabled deploying large models on emerging edge devices with little cost, by adapting already-trained models to the constraints of these devices. The rapid development of AI-capable edge devices with limited computation and storage requires streamlined methodologies that can efficiently satisfy the constraints of different devices. In contrast, existing methods often rely on heuristic and manual adjustments to maintain accuracy, support only coarse compression policies, or target specific device constraints that limit their applicability. We address these limitations by proposing the TOlerance-based COmpression (TOCO) framework. TOCO uses an in-depth analysis of the model, to maintain the accuracy, in an active learning system. The results of the analysis are tolerances that can be used to perform compression in a fine-grained manner. Finally, by decoupling compression from the tolerance analysis, TOCO allows flexibility to changes in the hardware.
Why we need an AI-resilient society
Authors: Thomas Bartz-Beielstein
Date: 2019-12-18 00:00:00

Artificial intelligence is considered as a key technology. It has a huge impact on our society. Besides many positive effects, there are also some negative effects or threats. Some of these threats to society are well-known, e.g., weapons or killer robots. But there are also threats that are ignored. These unknown-knowns or blind spots affect privacy, and facilitate manipulation and mistaken identities. We cannot trust data, audio, video, and identities any more. Democracies are able to cope with known threats, the known-knowns. Transforming unknown-knowns to known-knowns is one important cornerstone of resilient societies. An AI-resilient society is able to transform threats caused by new AI tecchnologies such as generative adversarial networks. Resilience can be seen as a positive adaptation of these threats. We propose three strategies how this adaptation can be achieved: awareness, agreements, and red flags. This article accompanies the TEDx talk "Why we urgently need an AI-resilient society", see https://youtu.be/f6c2ngp7rqY.
Unsupervised Anomaly Detection in Stream Data with Online Evolving Spiking Neural Networks
Authors: Piotr S. Maciąg, Marzena Kryszkiewicz, Robert Bembenik, Jesus L. Lobo, Javier Del Ser
Date: 2019-12-18 00:00:00
Tasks: Anomaly Detection, Time Series Analysis, Unsupervised Anomaly Detection

Unsupervised anomaly discovery in stream data is a research topic with many practical applications. However, in many cases, it is not easy to collect enough training data with labeled anomalies for supervised learning of an anomaly detector in order to deploy it later for identification of real anomalies in streaming data. It is thus important to design anomalies detectors that can correctly detect anomalies without access to labeled training data. Our idea is to adapt the Online evolving Spiking Neural Network (OeSNN) classifier to the anomaly detection task. As a result, we offer an Online evolving Spiking Neural Network for Unsupervised Anomaly Detection algorithm (OeSNN-UAD), which, unlike OeSNN, works in an unsupervised way and does not separate output neurons into disjoint decision classes. OeSNN-UAD uses our proposed new two-step anomaly detection method. Also, we derive new theoretical properties of neuronal model and input layer encoding of OeSNN, which enable more effective and efficient detection of anomalies in our OeSNN-UAD approach. The proposed OeSNN-UAD detector was experimentally compared with state-of-the-art unsupervised and semi-supervised detectors of anomalies in stream data from the Numenta Anomaly Benchmark and Yahoo Anomaly Datasets repositories. Our approach outperforms the other solutions provided in the literature in the case of data streams from the Numenta Anomaly Benchmark repository. Also, in the case of real data files of the Yahoo Anomaly Benchmark repository, OeSNN-UAD outperforms other selected algorithms, whereas in the case of Yahoo Anomaly Benchmark synthetic data files, it provides competitive results to the results recently reported in the literature.
PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization
Authors: Jingqing Zhang, Yao Zhao, Mohammad Saleh, Peter J. Liu
Date: 2019-12-18 00:00:00
Proceeding: ICML 2020 1
Tasks: Abstractive Text Summarization, Decoder, Text Summarization

Recent work pre-training Transformers with self-supervised objectives on large text corpora has shown great success when fine-tuned on downstream NLP tasks including text summarization. However, pre-training objectives tailored for abstractive text summarization have not been explored. Furthermore there is a lack of systematic evaluation across diverse domains. In this work, we propose pre-training large Transformer-based encoder-decoder models on massive text corpora with a new self-supervised objective. In PEGASUS, important sentences are removed/masked from an input document and are generated together as one output sequence from the remaining sentences, similar to an extractive summary. We evaluated our best PEGASUS model on 12 downstream summarization tasks spanning news, science, stories, instructions, emails, patents, and legislative bills. Experiments demonstrate it achieves state-of-the-art performance on all 12 downstream datasets measured by ROUGE scores. Our model also shows surprising performance on low-resource summarization, surpassing previous state-of-the-art results on 6 datasets with only 1000 examples. Finally we validated our results using human evaluation and show that our model summaries achieve human performance on multiple datasets.
Frequency-Aware Reconstruction of Fluid Simulations with Generative Networks
Authors: Simon Biland, Vinicius C. Azevedo, Byung-soo Kim, Barbara Solenthaler
Date: 2019-12-18 00:00:00

Convolutional neural networks were recently employed to fully reconstruct fluid simulation data from a set of reduced parameters. However, since (de-)convolutions traditionally trained with supervised L1-loss functions do not discriminate between low and high frequencies in the data, the error is not minimized efficiently for higher bands. This directly correlates with the quality of the perceived results, since missing high frequency details are easily noticeable. In this paper, we analyze the reconstruction quality of generative networks and present a frequency-aware loss function that is able to focus on specific bands of the dataset during training time. We show that our approach improves reconstruction quality of fluid simulation data in mid-frequency bands, yielding perceptually better results while requiring comparable training time.
MRI Pulse Sequence Integration for Deep-Learning Based Brain Metastasis Segmentation
Authors: Darvin Yi, Endre Grøvik, Michael Iv, Elizabeth Tong, Kyrre Eeg Emblem, Line Brennhaug Nilsen, Cathrine Saxhaug, Anna Latysheva, Kari Dolven Jacobsen, Åslaug Helland, Greg Zaharchuk, Daniel Rubin
Date: 2019-12-18 00:00:00
Tasks: Diagnostic, Small Data Image Classification

Magnetic resonance (MR) imaging is an essential diagnostic tool in clinical medicine. Recently, a variety of deep learning methods have been applied to segmentation tasks in medical images, with promising results for computer-aided diagnosis. For MR images, effectively integrating different pulse sequences is important to optimize performance. However, the best way to integrate different pulse sequences remains unclear. In this study, we evaluate multiple architectural features and characterize their effects in the task of metastasis segmentation. Specifically, we consider (1) different pulse sequence integration schemas, (2) different modes of weight sharing for parallel network branches, and (3) a new approach for enabling robustness to missing pulse sequences. We find that levels of integration and modes of weight sharing that favor low variance work best in our regime of small data (n = 100). By adding an input-level dropout layer, we could preserve the overall performance of these networks while allowing for inference on inputs with missing pulse sequence. We illustrate not only the generalizability of the network but also the utility of this robustness when applying the trained model to data from a different center, which does not use the same pulse sequences. Finally, we apply network visualization methods to better understand which input features are most important for network performance. Together, these results provide a framework for building networks with enhanced robustness to missing data while maintaining comparable performance in medical imaging applications.
Computationally Efficient Neural Image Compression
Authors: Nick Johnston, Elad Eban, Ariel Gordon, Johannes Ballé
Date: 2019-12-18 00:00:00
Tasks: Decoder, Image Compression

Image compression using neural networks have reached or exceeded non-neural methods (such as JPEG, WebP, BPG). While these networks are state of the art in ratedistortion performance, computational feasibility of these models remains a challenge. We apply automatic network optimization techniques to reduce the computational complexity of a popular architecture used in neural image compression, analyze the decoder complexity in execution runtime and explore the trade-offs between two distortion metrics, rate-distortion performance and run-time performance to design and research more computationally efficient neural image compression. We find that our method decreases the decoder run-time requirements by over 50% for a stateof-the-art neural architecture.
RealMix: Towards Realistic Semi-Supervised Deep Learning Algorithms
Authors: Varun Nair, Javier Fuentes Alonso, Tony Beltramelli
Date: 2019-12-18 00:00:00
Tasks: Deep Learning, Semi-Supervised Image Classification

Semi-Supervised Learning (SSL) algorithms have shown great potential in training regimes when access to labeled data is scarce but access to unlabeled data is plentiful. However, our experiments illustrate several shortcomings that prior SSL algorithms suffer from. In particular, poor performance when unlabeled and labeled data distributions differ. To address these observations, we develop RealMix, which achieves state-of-the-art results on standard benchmark datasets across different labeled and unlabeled set sizes while overcoming the aforementioned challenges. Notably, RealMix achieves an error rate of 9.79% on CIFAR10 with 250 labels and is the only SSL method tested able to surpass baseline performance when there is significant mismatch in the labeled and unlabeled data distributions. RealMix demonstrates how SSL can be used in real world situations with limited access to both data and compute and guides further research in SSL with practical applicability in mind.
Interleaved Composite Quantization for High-Dimensional Similarity Search
Authors: Soroosh Khoram, Stephen J. Wright, Jing Li
Date: 2019-12-18 00:00:00
Tasks: Quantization, Vocal Bursts Intensity Prediction

Similarity search retrieves the nearest neighbors of a query vector from a dataset of high-dimensional vectors. As the size of the dataset grows, the cost of performing the distance computations needed to implement a query can become prohibitive. A method often used to reduce this computational cost is quantization of the vector space and location-based encoding of the dataset vectors. These encodings can be used during query processing to find approximate nearest neighbors of the query point quickly. Search speed can be improved by using shorter codes, but shorter codes have higher quantization error, leading to degraded precision. In this work, we propose the Interleaved Composite Quantization (ICQ) which achieves fast similarity search without using shorter codes. In ICQ, a small subset of the code is used to approximate the distances, with complete codes being used only when necessary. Our method effectively reduces both code length and quantization error. Furthermore, ICQ is compatible with several recently proposed techniques for reducing quantization error and can be used in conjunction with these other techniques to improve results. We confirm these claims and show strong empirical performance of ICQ using several synthetic and real-word datasets.
Clusters in Explanation Space: Inferring disease subtypes from model explanations
Authors: Marc-Andre Schulz, Matt Chapman-Rounds, Manisha Verma, Danilo Bzdok, Konstantinos Georgatzis
Date: 2019-12-18 00:00:00
Tasks: Clustering, Diagnostic

Identification of disease subtypes and corresponding biomarkers can substantially improve clinical diagnosis and treatment selection. Discovering these subtypes in noisy, high dimensional biomedical data is often impossible for humans and challenging for machines. We introduce a new approach to facilitate the discovery of disease subtypes: Instead of analyzing the original data, we train a diagnostic classifier (healthy vs. diseased) and extract instance-wise explanations for the classifier's decisions. The distribution of instances in the explanation space of our diagnostic classifier amplifies the different reasons for belonging to the same class - resulting in a representation that is uniquely useful for discovering latent subtypes. We compare our ability to recover subtypes via cluster analysis on model explanations to classical cluster analysis on the original data. In multiple datasets with known ground-truth subclasses, most compellingly on UK Biobank brain imaging data and transcriptome data from the Cancer Genome Atlas, we show that cluster analysis on model explanations substantially outperforms the classical approach. While we believe clustering in explanation space to be particularly valuable for inferring disease subtypes, the method is more general and applicable to any kind of sub-type identification.
Towards Robust Learning with Different Label Noise Distributions
Authors: Diego Ortego, Eric Arazo, Paul Albert, Noel E. O'Connor, Kevin McGuinness
Date: 2019-12-18 00:00:00
Tasks: Memorization, Representation Learning

Noisy labels are an unavoidable consequence of labeling processes and detecting them is an important step towards preventing performance degradations in Convolutional Neural Networks. Discarding noisy labels avoids a harmful memorization, while the associated image content can still be exploited in a semi-supervised learning (SSL) setup. Clean samples are usually identified using the small loss trick, i.e. they exhibit a low loss. However, we show that different noise distributions make the application of this trick less straightforward and propose to continuously relabel all images to reveal a discriminative loss against multiple distributions. SSL is then applied twice, once to improve the clean-noisy detection and again for training the final model. We design an experimental setup based on ImageNet32/64 for better understanding the consequences of representation learning with differing label noise distributions and find that non-uniform out-of-distribution noise better resembles real-world noise and that in most cases intermediate features are not affected by label noise corruption. Experiments in CIFAR-10/100, ImageNet32/64 and WebVision (real-world noise) demonstrate that the proposed label noise Distribution Robust Pseudo-Labeling (DRPL) approach gives substantial improvements over recent state-of-the-art. Code is available at https://git.io/JJ0PV.
Effective Integration of Symbolic and Connectionist Approaches through a Hybrid Representation
Authors: Marcio Moreno, Daniel Civitarese, Rafael Brandao, Renato Cerqueira
Date: 2019-12-18 00:00:00
Tasks: Position

In this paper, we present our position for a neuralsymbolic integration strategy, arguing in favor of a hybrid representation to promote an effective integration. Such description differs from others fundamentally, since its entities aim at representing AI models in general, allowing to describe both nonsymbolic and symbolic knowledge, the integration between them and their corresponding processors. Moreover, the entities also support representing workflows, leveraging traceability to keep track of every change applied to models and their related entities (e.g., data or concepts) throughout the lifecycle of the models.