pwc-archive/papers-with-abstracts huggingface dataset:
Multi-defect microscopy image restoration under limited data conditions
Authors: Anastasia Razdaibiedina, Jeevaa Velayutham, Miti Modi
Date: 2019-10-31 00:00:00
Tasks: Data Augmentation, Generative Adversarial Network, Image Restoration

Deep learning methods are becoming widely used for restoration of defects associated with fluorescence microscopy imaging. One of the major challenges in application of such methods is the availability of training data. In this work, we propose a unified method for reconstruction of multi-defect fluorescence microscopy images when training data is limited. Our approach consists of two stages: first, we perform data augmentation using Generative Adversarial Network (GAN) with conditional instance normalization (CIN); second, we train a conditional GAN (cGAN) on paired ground-truth and defected images to perform restoration. The experiments on three common types of imaging defects with different amounts of training data show that the proposed method gives comparable results or outperforms CARE, deblurGAN and CycleGAN in restored image quality when available data is limited.
Automatic Cobb Angle Detection using Vertebra Detector and Vertebra Corners Regression
Authors: Bidur Khanal, Lavsen Dahal, Prashant Adhikari, Bishesh Khanal
Date: 2019-10-31 00:00:00
Tasks: regression

Correct evaluation and treatment of Scoliosis require accurate estimation of spinal curvature. Current gold standard is to manually estimate Cobb Angles in spinal X-ray images which is time consuming and has high inter-rater variability. We propose an automatic method with a novel framework that first detects vertebrae as objects followed by a landmark detector that estimates the 4 landmark corners of each vertebra separately. Cobb Angles are calculated using the slope of each vertebra obtained from the predicted landmarks. For inference on test data, we perform pre and post processings that include cropping, outlier rejection and smoothing of the predicted landmarks. The results were assessed in AASCE MICCAI challenge 2019 which showed a promise with a SMAPE score of 25.69 on the challenge test set.
Improving speech emotion recognition via Transformer-based Predictive Coding through transfer learning
Authors: Zheng Lian, Ya Li, Jian-Hua Tao, Jian Huang
Date: 2018-11-11 00:00:00
Tasks: Emotion Recognition, Speech Emotion Recognition, Transfer Learning

I have submitted a new version to arXiv:1910.13806. I forget to choose to replace the old version, but submitted a new one. It's my mistake.
Evaluation of Surrogate Models for Multi-fin Flapping Propulsion Systems
Authors: Kamal Viswanath, Alisha Sharma, Saketh Gabbita, Jason Geder, Ravi Ramamurti, Marius Pruessner
Date: 2019-10-31 00:00:00

The aim of this study is to develop surrogate models for quick, accurate prediction of thrust forces generated through flapping fin propulsion for given operating conditions and fin geometries. Different network architectures and configurations are explored to model the training data separately for the lead fin and rear fin of a tandem fin setup. We progressively improve the data representation of the input parameter space for model predictions. The models are tested on three unseen fin geometries and the predictions validated with computational fluid dynamics (CFD) data. Finally, the orders of magnitude gains in computational performance of these surrogate models, compared to experimental and CFD runs, vs their tradeoff with accuracy is discussed within the context of this tandem fin configuration.
Transferable End-to-End Aspect-based Sentiment Analysis with Selective Adversarial Learning
Authors: Zheng Li, Xin Li, Ying WEI, Lidong Bing, Yu Zhang, Qiang Yang
Date: 2019-10-31 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Aspect-Based Sentiment Analysis, Aspect-Based Sentiment Analysis (ABSA), Domain Adaptation, Sentiment Analysis, Unsupervised Domain Adaptation

Joint extraction of aspects and sentiments can be effectively formulated as a sequence labeling problem. However, such formulation hinders the effectiveness of supervised methods due to the lack of annotated sequence data in many domains. To address this issue, we firstly explore an unsupervised domain adaptation setting for this task. Prior work can only use common syntactic relations between aspect and opinion words to bridge the domain gaps, which highly relies on external linguistic resources. To resolve it, we propose a novel Selective Adversarial Learning (SAL) method to align the inferred correlation vectors that automatically capture their latent relations. The SAL method can dynamically learn an alignment weight for each word such that more important words can possess higher alignment weights to achieve fine-grained (word-level) adaptation. Empirically, extensive experiments demonstrate the effectiveness of the proposed SAL method.
On the Regularization Properties of Structured Dropout
Authors: Ambar Pal, Connor Lane, René Vidal, Benjamin D. Haeffele
Date: 2019-10-30 00:00:00
Proceeding: CVPR 2020 6

Dropout and its extensions (eg. DropBlock and DropConnect) are popular heuristics for training neural networks, which have been shown to improve generalization performance in practice. However, a theoretical understanding of their optimization and regularization properties remains elusive. Recent work shows that in the case of single hidden-layer linear networks, Dropout is a stochastic gradient descent method for minimizing a regularized loss, and that the regularizer induces solutions that are low-rank and balanced. In this work we show that for single hidden-layer linear networks, DropBlock induces spectral k-support norm regularization, and promotes solutions that are low-rank and have factors with equal norm. We also show that the global minimizer for DropBlock can be computed in closed form, and that DropConnect is equivalent to Dropout. We then show that some of these results can be extended to a general class of Dropout-strategies, and, with some assumptions, to deep non-linear networks when Dropout is applied to the last layer. We verify our theoretical claims and assumptions experimentally with commonly used network architectures.
Beyond Universal Person Re-ID Attack
Authors: Wenjie Ding, Xing Wei, Rongrong Ji, Xiaopeng Hong, Qi Tian, Yihong Gong
Date: 2019-10-30 00:00:00
Tasks: General Classification, Person Re-Identification

Deep learning-based person re-identification (Re-ID) has made great progress and achieved high performance recently. In this paper, we make the first attempt to examine the vulnerability of current person Re-ID models against a dangerous attack method, \ie, the universal adversarial perturbation (UAP) attack, which has been shown to fool classification models with a little overhead. We propose a \emph{more universal} adversarial perturbation (MUAP) method for both image-agnostic and model-insensitive person Re-ID attack. Firstly, we adopt a list-wise attack objective function to disrupt the similarity ranking list directly. Secondly, we propose a model-insensitive mechanism for cross-model attack. Extensive experiments show that the proposed attack approach achieves high attack performance and outperforms other state of the arts by large margin in cross-model scenario. The results also demonstrate the vulnerability of current Re-ID models to MUAP and further suggest the need of designing more robust Re-ID models.
Heteroscedastic Calibration of Uncertainty Estimators in Deep Learning
Authors: Bindya Venkatesh, Jayaraman J. Thiagarajan
Date: 2019-10-30 00:00:00
Tasks: Deep Learning, regression, Uncertainty Quantification

The role of uncertainty quantification (UQ) in deep learning has become crucial with growing use of predictive models in high-risk applications. Though a large class of methods exists for measuring deep uncertainties, in practice, the resulting estimates are found to be poorly calibrated, thus making it challenging to translate them into actionable insights. A common workaround is to utilize a separate recalibration step, which adjusts the estimates to compensate for the miscalibration. Instead, we propose to repurpose the heteroscedastic regression objective as a surrogate for calibration and enable any existing uncertainty estimator to be inherently calibrated. In addition to eliminating the need for recalibration, this also regularizes the training process. Using regression experiments, we demonstrate the effectiveness of the proposed heteroscedastic calibration with two popular uncertainty estimators.
Predicting Discourse Structure using Distant Supervision from Sentiment
Authors: Patrick Huber, Giuseppe Carenini
Date: 2019-10-30 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Discourse Parsing, Multiple Instance Learning, Prediction, Sentiment Analysis, Sentiment Classification

Discourse parsing could not yet take full advantage of the neural NLP revolution, mostly due to the lack of annotated datasets. We propose a novel approach that uses distant supervision on an auxiliary task (sentiment classification), to generate abundant data for RST-style discourse structure prediction. Our approach combines a neural variant of multiple-instance learning, using document-level supervision, with an optimal CKY-style tree generation algorithm. In a series of experiments, we train a discourse parser (for only structure prediction) on our automatically generated dataset and compare it with parsers trained on human-annotated corpora (news domain RST-DT and Instructional domain). Results indicate that while our parser does not yet match the performance of a parser trained and tested on the same dataset (intra-domain), it does perform remarkably well on the much more difficult and arguably more useful task of inter-domain discourse structure prediction, where the parser is trained on one domain and tested/applied on another one.
Learn-By-Calibrating: Using Calibration as a Training Objective
Authors: Jayaraman J. Thiagarajan, Bindya Venkatesh, Deepta Rajan
Date: 2019-10-30 00:00:00
Tasks: Prediction Intervals

Calibration error is commonly adopted for evaluating the quality of uncertainty estimators in deep neural networks. In this paper, we argue that such a metric is highly beneficial for training predictive models, even when we do not explicitly measure the uncertainties. This is conceptually similar to heteroscedastic neural networks that produce variance estimates for each prediction, with the key difference that we do not place a Gaussian prior on the predictions. We propose a novel algorithm that performs simultaneous interval estimation for different calibration levels and effectively leverages the intervals to refine the mean estimates. Our results show that, our approach is consistently superior to existing regularization strategies in deep regression models. Finally, we propose to augment partial dependence plots, a model-agnostic interpretability tool, with expected prediction intervals to reveal interesting dependencies between data and the target.
Iterative Hessian Sketch in Input Sparsity Time
Authors: Graham Cormode, Charlie Dickens
Date: 2019-10-30 00:00:00
Tasks: regression

Scalable algorithms to solve optimization and regression tasks even approximately, are needed to work with large datasets. In this paper we study efficient techniques from matrix sketching to solve a variety of convex constrained regression problems. We adopt "Iterative Hessian Sketching" (IHS) and show that the fast CountSketch and sparse Johnson-Lindenstrauss Transforms yield state-of-the-art accuracy guarantees under IHS, while drastically improving the time cost. As a result, we obtain significantly faster algorithms for constrained regression, for both sparse and dense inputs. Our empirical results show that we can summarize data roughly 100x faster for sparse data, and, surprisingly, 10x faster on dense data! Consequently, solutions accurate to within machine precision of the optimal solution can be found much faster than the previous state of the art.
Lexical Learning as an Online Optimal Experiment: Building Efficient Search Engines through Human-Machine Collaboration
Authors: Jacopo Tagliabue, Reuben Cohn-Gordon
Date: 2019-10-30 00:00:00
Tasks: Information Retrieval, Retrieval

Information retrieval (IR) systems need to constantly update their knowledge as target objects and user queries change over time. Due to the power-law nature of linguistic data, learning lexical concepts is a problem resisting standard machine learning approaches: while manual intervention is always possible, a more general and automated solution is desirable. In this work, we propose a novel end-to-end framework that models the interaction between a search engine and users as a virtuous human-in-the-loop inference. The proposed framework is the first to our knowledge combining ideas from psycholinguistics and experiment design to maximize efficiency in IR. We provide a brief overview of the main components and initial simulations in a toy world, showing how inference works end-to-end and discussing preliminary results and next steps.
Lsh-sampling Breaks the Computation Chicken-and-egg Loop in Adaptive Stochastic Gradient Estimation
Authors: Beidi Chen, Yingchen Xu, Anshumali Shrivastava
Date: 2019-10-30 00:00:00

Stochastic Gradient Descent or SGD is the most popular optimization algorithm for large-scale problems. SGD estimates the gradient by uniform sampling with sample size one. There have been several other works that suggest faster epoch-wise convergence by using weighted non-uniform sampling for better gradient estimates. Unfortunately, the per-iteration cost of maintaining this adaptive distribution for gradient estimation is more than calculating the full gradient itself, which we call the chicken-and-the-egg loop. As a result, the false impression of faster convergence in iterations, in reality, leads to slower convergence in time. In this paper, we break this barrier by providing the first demonstration of a scheme, Locality sensitive hashing (LSH) sampled Stochastic Gradient Descent (LGD), which leads to superior gradient estimation while keeping the sampling cost per iteration similar to that of the uniform sampling. Such an algorithm is possible due to the sampling view of LSH, which came to light recently. As a consequence of superior and fast estimation, we reduce the running time of all existing gradient descent algorithms, that relies on gradient estimates including Adam, Ada-grad, etc. We demonstrate the effectiveness of our proposal with experiments on linear models as well as the non-linear BERT, which is a recent popular deep learning based language representation model.
How does Grammatical Gender Affect Noun Representations in Gender-Marking Languages?
Authors: Hila Gonen, Yova Kementchedjhieva, Yoav Goldberg
Date: 2019-10-30 00:00:00
Proceeding: CONLL 2019 11
Tasks: Word Embeddings

Many natural languages assign grammatical gender also to inanimate nouns in the language. In such languages, words that relate to the gender-marked nouns are inflected to agree with the noun's gender. We show that this affects the word representations of inanimate nouns, resulting in nouns with the same gender being closer to each other than nouns with different gender. While "embedding debiasing" methods fail to remove the effect, we demonstrate that a careful application of methods that neutralize grammatical gender signals from the words' context when training word embeddings is effective in removing it. Fixing the grammatical gender bias yields a positive effect on the quality of the resulting word embeddings, both in monolingual and cross-lingual settings. We note that successfully removing gender signals, while achievable, is not trivial to do and that a language-specific morphological analyzer, together with careful usage of it, are essential for achieving good results.
A Unified Framework for Data Poisoning Attack to Graph-based Semi-supervised Learning
Authors: Xuanqing Liu, Si Si, Xiaojin Zhu, Yang Li, Cho-Jui Hsieh
Date: 2019-10-30 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Binary Classification, Data Poisoning, General Classification

In this paper, we proposed a general framework for data poisoning attacks to graph-based semi-supervised learning (G-SSL). In this framework, we first unify different tasks, goals, and constraints into a single formula for data poisoning attack in G-SSL, then we propose two specialized algorithms to efficiently solve two important cases --- poisoning regression tasks under $\ell_2$-norm constraint and classification tasks under $\ell_0$-norm constraint. In the former case, we transform it into a non-convex trust region problem and show that our gradient-based algorithm with delicate initialization and update scheme finds the (globally) optimal perturbation. For the latter case, although it is an NP-hard integer programming problem, we propose a probabilistic solver that works much better than the classical greedy method. Lastly, we test our framework on real datasets and evaluate the robustness of G-SSL algorithms. For instance, on the MNIST binary classification problem (50000 training data with 50 labeled), flipping two labeled data is enough to make the model perform like random guess (around 50\% error).
Parameter elimination in particle Gibbs sampling
Authors: Anna Wigren, Riccardo Sven Risuleo, Lawrence Murray, Fredrik Lindsten
Date: 2019-10-30 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Bayesian Inference, Epidemiology, Probabilistic Programming, State Space Models

Bayesian inference in state-space models is challenging due to high-dimensional state trajectories. A viable approach is particle Markov chain Monte Carlo, combining MCMC and sequential Monte Carlo to form "exact approximations" to otherwise intractable MCMC methods. The performance of the approximation is limited to that of the exact method. We focus on particle Gibbs and particle Gibbs with ancestor sampling, improving their performance beyond that of the underlying Gibbs sampler (which they approximate) by marginalizing out one or more parameters. This is possible when the parameter prior is conjugate to the complete data likelihood. Marginalization yields a non-Markovian model for inference, but we show that, in contrast to the general case, this method still scales linearly in time. While marginalization can be cumbersome to implement, recent advances in probabilistic programming have enabled its automation. We demonstrate how the marginalized methods are viable as efficient inference backends in probabilistic programming, and demonstrate with examples in ecology and epidemiology.
Discourse-Aware Neural Extractive Text Summarization
Authors: Jiacheng Xu, Zhe Gan, Yu Cheng, Jingjing Liu
Date: 2019-10-30 00:00:00
Proceeding: ACL 2020 6
Tasks: Extractive Text Summarization, Sentence, Text Summarization

Recently BERT has been adopted for document encoding in state-of-the-art text summarization models. However, sentence-based extractive models often result in redundant or uninformative phrases in the extracted summaries. Also, long-range dependencies throughout a document are not well captured by BERT, which is pre-trained on sentence pairs instead of documents. To address these issues, we present a discourse-aware neural summarization model - DiscoBert. DiscoBert extracts sub-sentential discourse units (instead of sentences) as candidates for extractive selection on a finer granularity. To capture the long-range dependencies among discourse units, structural discourse graphs are constructed based on RST trees and coreference mentions, encoded with Graph Convolutional Networks. Experiments show that the proposed model outperforms state-of-the-art methods by a significant margin on popular summarization benchmarks compared to other BERT-base models.
FutureMapping 2: Gaussian Belief Propagation for Spatial AI
Authors: Andrew J. Davison, Joseph Ortiz
Date: 2019-10-30 00:00:00

We argue the case for Gaussian Belief Propagation (GBP) as a strong algorithmic framework for the distributed, generic and incremental probabilistic estimation we need in Spatial AI as we aim at high performance smart robots and devices which operate within the constraints of real products. Processor hardware is changing rapidly, and GBP has the right character to take advantage of highly distributed processing and storage while estimating global quantities, as well as great flexibility. We present a detailed tutorial on GBP, relating to the standard factor graph formulation used in robotics and computer vision, and give several simulation examples with code which demonstrate its properties.
Belief revision and 3-valued logics: Characterization of 19,683 belief change operators
Authors: Nerio Borges, Ramón Pino Pérez
Date: 2019-10-30 00:00:00

In most classical models of belief change, epistemic states are represented by theories (AGM) or formulas (Katsuno-Mendelzon) and the new pieces of information by formulas. The Representation Theorem for revision operators says that operators are represented by total preorders. This important representation is exploited by Darwiche and Pearl to shift the notion of epistemic state to a more abstract one, where the paradigm of epistemic state is indeed that of a total preorder over interpretations. In this work, we introduce a 3-valued logic where the formulas can be identified with a generalisation of total preorders of three levels: a ranking function mapping interpretations into the truth values. Then we analyse some sort of changes in this kind of structures and give syntactical characterizations of them.
Investigating Under and Overfitting in Wasserstein Generative Adversarial Networks
Authors: Ben Adlam, Charles Weill, Amol Kapoor
Date: 2019-10-30 00:00:00

We investigate under and overfitting in Generative Adversarial Networks (GANs), using discriminators unseen by the generator to measure generalization. We find that the model capacity of the discriminator has a significant effect on the generator's model quality, and that the generator's poor performance coincides with the discriminator underfitting. Contrary to our expectations, we find that generators with large model capacities relative to the discriminator do not show evidence of overfitting on CIFAR10, CIFAR100, and CelebA.
Meta-Learning to Cluster
Authors: Yibo Jiang, Nakul Verma
Date: 2019-10-30 00:00:00
Tasks: Clustering, Meta-Learning

Clustering is one of the most fundamental and wide-spread techniques in exploratory data analysis. Yet, the basic approach to clustering has not really changed: a practitioner hand-picks a task-specific clustering loss to optimize and fit the given data to reveal the underlying cluster structure. Some types of losses---such as k-means, or its non-linear version: kernelized k-means (centroid based), and DBSCAN (density based)---are popular choices due to their good empirical performance on a range of applications. Although every so often the clustering output using these standard losses fails to reveal the underlying structure, and the practitioner has to custom-design their own variation. In this work we take an intrinsically different approach to clustering: rather than fitting a dataset to a specific clustering loss, we train a recurrent model that learns how to cluster. The model uses as training pairs examples of datasets (as input) and its corresponding cluster identities (as output). By providing multiple types of training datasets as inputs, our model has the ability to generalize well on unseen datasets (new clustering tasks). Our experiments reveal that by training on simple synthetically generated datasets or on existing real datasets, we can achieve better clustering performance on unseen real-world datasets when compared with standard benchmark clustering techniques. Our meta clustering model works well even for small datasets where the usual deep learning models tend to perform worse.