pwc-archive/papers-with-abstracts huggingface dataset:
Learning Generalizable Physical Dynamics of 3D Rigid Objects
Authors: Davis Rempe, Srinath Sridhar, He Wang, Leonidas J. Guibas
Date: 2019-01-02 00:00:00
Tasks: Autonomous Vehicles, Object, Position

Humans have a remarkable ability to predict the effect of physical
interactions on the dynamics of objects. Endowing machines with this ability
would allow important applications in areas like robotics and autonomous
vehicles. In this work, we focus on predicting the dynamics of 3D rigid
objects, in particular an object's final resting position and total rotation
when subjected to an impulsive force. Different from previous work, our
approach is capable of generalizing to unseen object shapes - an important
requirement for real-world applications. To achieve this, we represent object
shape as a 3D point cloud that is used as input to a neural network, making our
approach agnostic to appearance variation. The design of our network is
informed by an understanding of physical laws. We train our model with data
from a physics engine that simulates the dynamics of a large number of shapes.
Experiments show that we can accurately predict the resting position and total
rotation for unseen object geometries.
Improved Hyperspectral Unmixing With Endmember Variability Parametrized Using an Interpolated Scaling Tensor
Authors: Ricardo Augusto Borsoi, Tales Imbiriba, José Carlos Moreira Bermudez
Date: 2019-01-02 00:00:00
Tasks: Hyperspectral Unmixing

Endmember (EM) variability has an important impact on the performance of
hyperspectral image (HI) analysis algorithms. Recently, extended linear mixing
models have been proposed to account for EM variability in the spectral
unmixing (SU) problem. The direct use of these models has led to severely
ill-posed optimization problems. Different regularization strategies have been
considered to deal with this issue, but none so far has consistently exploited
the information provided by the existence of multiple pure pixels often present
in HIs. In this work, we propose to break the SU problem into a sequence of two
problems. First, we use pure pixel information to estimate an interpolated
tensor of scaling factors representing spectral variability. This is done by
considering the spectral variability to be a smooth function over the HI and
confining the energy of the scaling tensor to a low-rank structure. Afterwards,
we solve a matrix-factorization problem to estimate the fractional abundances
using the variability scaling factors estimated in the previous step, what
leads to a significantly more well-posed problem. Simulation swith synthetic
and real data attest the effectiveness of the proposed strategy.
A CNN adapted to time series for the classification of Supernovae
Authors: Anthony Brunel, Johanna Pasquet, Jérôme Pasquet, Nancy Rodriguez, Frédéric Comby, Dominique Fouchez, Marc Chaumont
Date: 2019-01-02 00:00:00
Tasks: Binary Classification, General Classification, Time Series, Time Series Analysis

Cosmologists are facing the problem of the analysis of a huge quantity of
data when observing the sky. The methods used in cosmology are, for the most of
them, relying on astrophysical models, and thus, for the classification, they
usually use a machine learning approach in two-steps, which consists in, first,
extracting features, and second, using a classifier. In this paper, we are
specifically studying the supernovae phenomenon and especially the binary
classification "I.a supernovae versus not-I.a supernovae". We present two
Convolutional Neural Networks (CNNs) defeating the current state-of-the-art.
The first one is adapted to time series and thus to the treatment of supernovae
light-curves. The second one is based on a Siamese CNN and is suited to the
nature of data, i.e. their sparsity and their weak quantity (small learning
database).
Cost-sensitive Selection of Variables by Ensemble of Model Sequences
Authors: Donghui Yan, Zhiwei Qin, Songxiang Gu, Haiping Xu, Ming Shao
Date: 2019-01-02 00:00:00

Many applications require the collection of data on different variables or measurements over many system performance metrics. We term those broadly as measures or variables. Often data collection along each measure incurs a cost, thus it is desirable to consider the cost of measures in modeling. This is a fairly new class of problems in the area of cost-sensitive learning. A few attempts have been made to incorporate costs in combining and selecting measures. However, existing studies either do not strictly enforce a budget constraint, or are not the `most' cost effective. With a focus on classification problem, we propose a computationally efficient approach that could find a near optimal model under a given budget by exploring the most `promising' part of the solution space. Instead of outputting a single model, we produce a model schedule -- a list of models, sorted by model costs and expected predictive accuracy. This could be used to choose the model with the best predictive accuracy under a given budget, or to trade off between the budget and the predictive accuracy. Experiments on some benchmark datasets show that our approach compares favorably to competing methods.
SGD Converges to Global Minimum in Deep Learning via Star-convex Path
Authors: Yi Zhou, Junjie Yang, Huishuai Zhang, Yingbin Liang, Vahid Tarokh
Date: 2019-01-02 00:00:00
Proceeding: ICLR 2019 5

Stochastic gradient descent (SGD) has been found to be surprisingly effective
in training a variety of deep neural networks. However, there is still a lack
of understanding on how and why SGD can train these complex networks towards a
global minimum. In this study, we establish the convergence of SGD to a global
minimum for nonconvex optimization problems that are commonly encountered in
neural network training. Our argument exploits the following two important
properties: 1) the training loss can achieve zero value (approximately), which
has been widely observed in deep learning; 2) SGD follows a star-convex path,
which is verified by various experiments in this paper. In such a context, our
analysis shows that SGD, although has long been considered as a randomized
algorithm, converges in an intrinsically deterministic manner to a global
minimum.
Game Semantics and Linear Logic in the Cognition Process
Authors: Dmitry Maximov
Date: 2018-12-27 00:00:00

A description of the environment cognition process by intelligent systems
with a fixed set of system goals is suggested. Such a system is represented by
the set of its goals only without any models of the system elements or the
environment. The set has a lattice structure and a monoid structure; thus, the
structure of linear logic is defined on the set. The cognition process of some
environment by the system is described on this basis. The environment is
represented as a configuration space of possible system positions which are
estimated by an information amount (by corresponding sets). This information is
supplied to the system by the environment. Thus, it is possible to define the
category of Conway games with a payoff on the configuration space and to choose
an optimal system's play (i.e., a trajectory). The choice is determined by the
requirement of maximal information increasing and takes into account the
structure of the system goal set: the linear logic on the set is used to
determine the priority of possible different parallel processes. The survey may
be useful to describe the behavior of robots and simple biological systems,
e.g., ants.
BMF: Block matrix approach to factorization of large scale data
Authors: Prasad G Bhavana, Vineet C Nair
Date: 2019-01-02 00:00:00
Tasks: CPU, GPU

Matrix Factorization (MF) on large scale matrices is computationally as well
as memory intensive task. Alternative convergence techniques are needed when
the size of the input matrix is higher than the available memory on a Central
Processing Unit (CPU) and Graphical Processing Unit (GPU). While alternating
least squares (ALS) convergence on CPU could take forever, loading all the
required matrices on to GPU memory may not be possible when the dimensions are
significantly higher. Hence we introduce a novel technique that is based on
considering the entire data into a block matrix and relies on factorization at
a block level.
The capacity of feedforward neural networks
Authors: Pierre Baldi, Roman Vershynin
Date: 2019-01-02 00:00:00

A long standing open problem in the theory of neural networks is the
development of quantitative methods to estimate and compare the capabilities of
different architectures. Here we define the capacity of an architecture by the
binary logarithm of the number of functions it can compute, as the synaptic
weights are varied. The capacity provides an upper bound on the number of bits
that can be extracted from the training data and stored in the architecture
during learning. We study the capacity of layered, fully-connected,
architectures of linear threshold neurons with $L$ layers of size $n_1,n_2,
\ldots, n_L$ and show that in essence the capacity is given by a cubic
polynomial in the layer sizes: $C(n_1,\ldots, n_L)=\sum_{k=1}^{L-1}
\min(n_1,\ldots,n_k)n_kn_{k+1}$, where layers that are smaller than all
previous layers act as bottlenecks. In proving the main result, we also develop
new techniques (multiplexing, enrichment, and stacking) as well as new bounds
on the capacity of finite sets. We use the main result to identify
architectures with maximal or minimal capacity under a number of natural
constraints. This leads to the notion of structural regularization for deep
architectures. While in general, everything else being equal, shallow networks
compute more functions than deep networks, the functions computed by deep
networks are more regular and "interesting".
Causal Calculus in the Presence of Cycles, Latent Confounders and Selection Bias
Authors: Patrick Forré, Joris M. Mooij
Date: 2019-01-02 00:00:00
Tasks: Selection bias

We prove the main rules of causal calculus (also called do-calculus) for i/o structural causal models (ioSCMs), a generalization of a recently proposed general class of non-/linear structural causal models that allow for cycles, latent confounders and arbitrary probability distributions. We also generalize adjustment criteria and formulas from the acyclic setting to the general one (i.e. ioSCMs). Such criteria then allow to estimate (conditional) causal effects from observational data that was (partially) gathered under selection bias and cycles. This generalizes the backdoor criterion, the selection-backdoor criterion and extensions of these to arbitrary ioSCMs. Together, our results thus enable causal reasoning in the presence of cycles, latent confounders and selection bias. Finally, we extend the ID algorithm for the identification of causal effects to ioSCMs.
Lipi Gnani - A Versatile OCR for Documents in any Language Printed in Kannada Script
Authors: Shiva Kumar H R, Ramakrishnan A G
Date: 2019-01-02 00:00:00
Tasks: Binarization, Optical Character Recognition (OCR)

A Kannada OCR, named Lipi Gnani, has been designed and developed from
scratch, with the motivation of it being able to convert printed text or poetry
in Kannada script, without any restriction on vocabulary. The training and test
sets have been collected from over 35 books published between the period 1970
to 2002, and this includes books written in Halegannada and pages containing
Sanskrit slokas written in Kannada script. The coverage of the OCR is nearly
complete in the sense that it recognizes all the punctuation marks, special
symbols, Indo-Arabic and Kannada numerals and also the interspersed English
words. Several minor and major original contributions have been done in
developing this OCR at the different processing stages such as binarization,
line and character segmentation, recognition and Unicode mapping. This has
created a Kannada OCR that performs as good as, and in some cases, better than
the Google's Tesseract OCR, as shown by the results. To the knowledge of the
authors, this is the maiden report of a complete Kannada OCR, handling all the
issues involved. Currently, there is no dictionary based postprocessing, and
the obtained results are due solely to the recognition process. Four benchmark
test databases containing scanned pages from books in Kannada, Sanskrit,
Konkani and Tulu languages, but all of them printed in Kannada script, have
been created. The word level recognition accuracy of Lipi Gnani is 4% higher on
the Kannada dataset than that of Google's Tesseract OCR, 8% higher on the
datasets of Tulu and Sanskrit, and 25% higher on the Konkani dataset.
Can You Trust This Prediction? Auditing Pointwise Reliability After Learning
Authors: Peter Schulam, Suchi Saria
Date: 2019-01-02 00:00:00
Tasks: Bayesian Inference

To use machine learning in high stakes applications (e.g. medicine), we need
tools for building confidence in the system and evaluating whether it is
reliable. Methods to improve model reliability often require new learning
algorithms (e.g. using Bayesian inference to obtain uncertainty estimates). An
alternative is to audit a model after it is trained. In this paper, we describe
resampling uncertainty estimation (RUE), an algorithm to audit the pointwise
reliability of predictions. Intuitively, RUE estimates the amount that a
prediction would change if the model had been fit on different training data.
The algorithm uses the gradient and Hessian of the model's loss function to
create an ensemble of predictions. Experimentally, we show that RUE more
effectively detects inaccurate predictions than existing tools for auditing
reliability subsequent to training. We also show that RUE can create predictive
distributions that are competitive with state-of-the-art methods like Monte
Carlo dropout, probabilistic backpropagation, and deep ensembles, but does not
depend on specific algorithms at train-time like these methods do.
Judge the Judges: A Large-Scale Evaluation Study of Neural Language Models for Online Review Generation
Authors: Cristina Garbacea, Samuel Carton, Shiyan Yan, Qiaozhu Mei
Date: 2019-01-02 00:00:00
Proceeding: IJCNLP 2019 11
Tasks: Diversity, Review Generation, Text Generation

We conduct a large-scale, systematic study to evaluate the existing evaluation methods for natural language generation in the context of generating online product reviews. We compare human-based evaluators with a variety of automated evaluation procedures, including discriminative evaluators that measure how well machine-generated text can be distinguished from human-written text, as well as word overlap metrics that assess how similar the generated text compares to human-written references. We determine to what extent these different evaluators agree on the ranking of a dozen of state-of-the-art generators for online product reviews. We find that human evaluators do not correlate well with discriminative evaluators, leaving a bigger question of whether adversarial accuracy is the correct objective for natural language generation. In general, distinguishing machine-generated text is challenging even for human evaluators, and human decisions correlate better with lexical overlaps. We find lexical diversity an intriguing metric that is indicative of the assessments of different evaluators. A post-experiment survey of participants provides insights into how to evaluate and improve the quality of natural language generation systems.
A Full Probabilistic Model for Yes/No Type Crowdsourcing in Multi-Class Classification
Authors: Belen Saldias, Pavlos Protopapas, Karim Pichara
Date: 2019-01-02 00:00:00
Tasks: General Classification, image-classification, Image Classification, Multi-class Classification, Time Series, Time Series Analysis, Variational Inference

Crowdsourcing has become widely used in supervised scenarios where training sets are scarce and difficult to obtain. Most crowdsourcing models in the literature assume labelers can provide answers to full questions. In classification contexts, full questions require a labeler to discern among all possible classes. Unfortunately, discernment is not always easy in realistic scenarios. Labelers may not be experts in differentiating all classes. In this work, we provide a full probabilistic model for a shorter type of queries. Our shorter queries only require "yes" or "no" responses. Our model estimates a joint posterior distribution of matrices related to labelers' confusions and the posterior probability of the class of every object. We developed an approximate inference approach, using Monte Carlo Sampling and Black Box Variational Inference, which provides the derivation of the necessary gradients. We built two realistic crowdsourcing scenarios to test our model. The first scenario queries for irregular astronomical time-series. The second scenario relies on the image classification of animals. We achieved results that are comparable with those of full query crowdsourcing. Furthermore, we show that modeling labelers' failures plays an important role in estimating true classes. Finally, we provide the community with two real datasets obtained from our crowdsourcing experiments. All our code is publicly available.
Attribute-Aware Attention Model for Fine-grained Representation Learning
Authors: Kai Han, Jianyuan Guo, Chao Zhang, Mingjian Zhu
Date: 2019-01-02 00:00:00
Tasks: Attribute, Fine-Grained Image Classification, Image Retrieval, Person Re-Identification, Representation Learning, Retrieval

How to learn a discriminative fine-grained representation is a key point in many computer vision applications, such as person re-identification, fine-grained classification, fine-grained image retrieval, etc. Most of the previous methods focus on learning metrics or ensemble to derive better global representation, which are usually lack of local information. Based on the considerations above, we propose a novel Attribute-Aware Attention Model ($A^3M$), which can learn local attribute representation and global category representation simultaneously in an end-to-end manner. The proposed model contains two attention models: attribute-guided attention module uses attribute information to help select category features in different regions, at the same time, category-guided attention module selects local features of different attributes with the help of category cues. Through this attribute-category reciprocal process, local and global features benefit from each other. Finally, the resulting feature contains more intrinsic information for image recognition instead of the noisy and irrelevant features. Extensive experiments conducted on Market-1501, CompCars, CUB-200-2011 and CARS196 demonstrate the effectiveness of our $A^3M$. Code is available at https://github.com/iamhankai/attribute-aware-attention.
Learning Efficient Detector with Semi-supervised Adaptive Distillation
Authors: Shitao Tang, Litong Feng, Wenqi Shao, Zhanghui Kuang, Wei zhang, Yimin Chen
Date: 2019-01-02 00:00:00
Tasks: image-classification, Image Classification, Knowledge Distillation, Model Compression

Knowledge Distillation (KD) has been used in image classification for model
compression. However, rare studies apply this technology on single-stage object
detectors. Focal loss shows that the accumulated errors of easily-classified
samples dominate the overall loss in the training process. This problem is also
encountered when applying KD in the detection task. For KD, the teacher-defined
hard samples are far more important than any others. We propose ADL to address
this issue by adaptively mimicking the teacher's logits, with more attention
paid on two types of hard samples: hard-to-learn samples predicted by teacher
with low certainty and hard-to-mimic samples with a large gap between the
teacher's and the student's prediction. ADL enlarges the distillation loss for
hard-to-learn and hard-to-mimic samples and reduces distillation loss for the
dominant easy samples, enabling distillation to work on the single-stage
detector first time, even if the student and the teacher are identical.
Besides, ADL is effective in both the supervised setting and the
semi-supervised setting, even when the labeled data and unlabeled data are from
different distributions. For distillation on unlabeled data, ADL achieves
better performance than existing data distillation which simply utilizes hard
targets, making the student detector surpass its teacher. On the COCO database,
semi-supervised adaptive distillation (SAD) makes a student detector with a
backbone of ResNet-50 surpasses its teacher with a backbone of ResNet-101,
while the student has half of the teacher's computation complexity. The code is
avaiable at https://github.com/Tangshitao/Semi-supervised-Adaptive-Distillation
Detecting Text in the Wild with Deep Character Embedding Network
Authors: Jiaming Liu, Chengquan Zhang, Yipeng Sun, Junyu Han, Errui Ding
Date: 2019-01-02 00:00:00
Tasks: Clustering, Text Detection

Most text detection methods hypothesize texts are horizontal or
multi-oriented and thus define quadrangles as the basic detection unit.
However, text in the wild is usually perspectively distorted or curved, which
can not be easily tackled by existing approaches. In this paper, we propose a
deep character embedding network (CENet) which simultaneously predicts the
bounding boxes of characters and their embedding vectors, thus making text
detection a simple clustering task in the character embedding space. The
proposed method does not require strong assumptions of forming a straight line
on general text detection, which provides flexibility on arbitrarily curved or
perspectively distorted text. For character detection task, a dense prediction
subnetwork is designed to obtain the confidence score and bounding boxes of
characters. For character embedding task, a subnet is trained with contrastive
loss to project detected characters into embedding space. The two tasks share a
backbone CNN from which the multi-scale feature maps are extracted. The final
text regions can be easily achieved by a thresholding process on character
confidence and embedding distance of character pairs. We evaluated our method
on ICDAR13, ICDAR15, MSRA-TD500, and Total-Text. The proposed method achieves
state-of-the-art or comparable performance on all these datasets, and shows
substantial improvement in the irregular-text datasets, i.e. Total-Text.
Optical Fringe Patterns Filtering Based on Multi-Stage Convolution Neural Network
Authors: Bowen Lin, Shujun Fu, Caiming Zhang, Fengling Wang, Yuliang Li
Date: 2019-01-02 00:00:00
Tasks: Denoising

Optical fringe patterns are often contaminated by speckle noise, making it difficult to accurately and robustly extract their phase fields. To deal with this problem, we propose a filtering method based on deep learning, called optical fringe patterns denoising convolutional neural network (FPD-CNN), for directly removing speckle from the input noisy fringe patterns. Regularization technology is integrated into the design of deep architecture. Specifically, the FPD-CNN method is divided into multiple stages, each stage consists of a set of convolutional layers along with batch normalization and leaky rectified linear unit (Leaky ReLU) activation function. The end-to-end joint training is carried out using the Euclidean loss. Extensive experiments on simulated and experimental optical fringe patterns,especially finer ones with high-density regions, show that the proposed method is competitive with some state-of-the-art denoising techniques in spatial or transform domains, efficiently preserving main features of fringe at a fairly fast speed.