pwc-archive/papers-with-abstracts huggingface dataset:
Several Experiments on Investigating Pretraining and Knowledge-Enhanced Models for Natural Language Inference
Authors: Tianda Li, Xiaodan Zhu, Quan Liu, Qian Chen, Zhigang Chen, Si Wei
Date: 2019-04-27 00:00:00
Tasks: Language Modeling, Language Modelling, Natural Language Inference, Natural Language Understanding, Sentence

Natural language inference (NLI) is among the most challenging tasks in
natural language understanding. Recent work on unsupervised pretraining that
leverages unsupervised signals such as language-model and sentence prediction
objectives has shown to be very effective on a wide range of NLP problems. It
would still be desirable to further understand how it helps NLI; e.g., if it
learns artifacts in data annotation or instead learn true inference knowledge.
In addition, external knowledge that does not exist in the limited amount of
NLI training data may be added to NLI models in two typical ways, e.g., from
human-created resources or an unsupervised pretraining paradigm. We runs
several experiments here to investigate whether they help NLI in the same way,
and if not,how?
Fast Infant MRI Skullstripping with Multiview 2D Convolutional Neural Networks
Authors: Amod Jog, P. Ellen Grant, Joseph L. Jacobson, Andre van der Kouwe, Ernesta M. Meintjes, Bruce Fischl, Lilla Zöllei
Date: 2019-04-27 00:00:00
Tasks: Brain Segmentation

Skullstripping is defined as the task of segmenting brain tissue from a full
head magnetic resonance image~(MRI). It is a critical component in neuroimage
processing pipelines. Downstream deformable registration and whole brain
segmentation performance is highly dependent on accurate skullstripping.
Skullstripping is an especially challenging task for infant~(age range 0--18
months) head MRI images due to the significant size and shape variability of
the head and the brain in that age range. Infant brain tissue development also
changes the $T_1$-weighted image contrast over time, making consistent
skullstripping a difficult task. Existing tools for adult brain MRI
skullstripping are ill equipped to handle these variations and a specialized
infant MRI skullstripping algorithm is necessary. In this paper, we describe a
supervised skullstripping algorithm that utilizes three trained fully
convolutional neural networks~(CNN), each of which segments 2D $T_1$-weighted
slices in axial, coronal, and sagittal views respectively. The three
probabilistic segmentations in the three views are linearly fused and
thresholded to produce a final brain mask. We compared our method to existing
adult and infant skullstripping algorithms and showed significant improvement
based on Dice overlap metric~(average Dice of 0.97) with a manually labeled
ground truth data set. Label fusion experiments on multiple, unlabeled data
sets show that our method is consistent and has fewer failure modes. In
addition, our method is computationally very fast with a run time of 30 seconds
per image on NVidia P40/P100/Quadro 4000 GPUs.
Learning to Fuse Local Geometric Features for 3D Rigid Data Matching
Authors: Jiaqi Yang, Chen Zhao, Ke Xian, Angfan Zhu, Zhiguo Cao
Date: 2019-04-27 00:00:00
Tasks: Triplet

This paper presents a simple yet very effective data-driven approach to fuse
both low-level and high-level local geometric features for 3D rigid data
matching. It is a common practice to generate distinctive geometric descriptors
by fusing low-level features from various viewpoints or subspaces, or enhance
geometric feature matching by leveraging multiple high-level features. In prior
works, they are typically performed via linear operations such as concatenation
and min pooling. We show that more compact and distinctive representations can
be achieved by optimizing a neural network (NN) model under the triplet
framework that non-linearly fuses local geometric features in Euclidean spaces.
The NN model is trained by an improved triplet loss function that fully
leverages all pairwise relationships within the triplet. Moreover, the fused
descriptor by our approach is also competitive to deep learned descriptors from
raw data while being more lightweight and rotational invariant. Experimental
results on four standard datasets with various data modalities and application
contexts confirm the advantages of our approach in terms of both feature
matching and geometric registration.
Enabling Open-World Specification Mining via Unsupervised Learning
Authors: Jordan Henkel, Shuvendu K. Lahiri, Ben Liblit, Thomas Reps
Date: 2019-04-27 00:00:00
Tasks: Word Embeddings

Many programming tasks require using both domain-specific code and
well-established patterns (such as routines concerned with file IO). Together,
several small patterns combine to create complex interactions. This compounding
effect, mixed with domain-specific idiosyncrasies, creates a challenging
environment for fully automatic specification inference. Mining specifications
in this environment, without the aid of rule templates, user-directed feedback,
or predefined API surfaces, is a major challenge. We call this challenge
Open-World Specification Mining.
  In this paper, we present a framework for mining specifications and usage
patterns in an Open-World setting. We design this framework to be
miner-agnostic and instead focus on disentangling complex and noisy API
interactions. To evaluate our framework, we introduce a benchmark of 71
clusters extracted from five open-source projects. Using this dataset, we show
that interesting clusters can be recovered, in a fully automatic way, by
leveraging unsupervised learning in the form of word embeddings. Once clusters
have been recovered, the challenge of Open-World Specification Mining is
simplified and any trace-based mining technique can be applied. In addition, we
provide a comprehensive evaluation of three word-vector learners to showcase
the value of sub-word information for embeddings learned in the
software-engineering domain.
Accelerating Proposal Generation Network for \\Fast Face Detection on Mobile Devices
Authors: Heming Zhang, Xiaolong Wang, Jingwen Zhu, C. -C. Jay Kuo
Date: 2019-04-27 00:00:00
Tasks: Face Detection

Face detection is a widely studied problem over the past few decades.
Recently, significant improvements have been achieved via the deep neural
network, however, it is still challenging to directly apply these techniques to
mobile devices for its limited computational power and memory. In this work, we
present a proposal generation acceleration framework for real-time face
detection. More specifically, we adopt a popular cascaded convolutional neural
network (CNN) as the basis, then apply our acceleration approach on the basic
framework to speed up the model inference time. We are motivated by the
observation that the computation bottleneck of this framework arises from the
proposal generation stage, where each level of the dense image pyramid has to
go through the network. In this work, we reduce the number of image pyramid
levels by utilizing both global and local facial characteristics (i.e., global
face and facial parts). Experimental results on public benchmarks WIDER-face
and FDDB demonstrate the satisfactory performance and faster speed compared to
the state-of-the-arts. %the comparable accuracy to state-of-the-arts with
faster speed.
Neural source-filter waveform models for statistical parametric speech synthesis
Authors: Xin Wang, Shinji Takaki, Junichi Yamagishi
Date: 2019-04-27 00:00:00
Tasks: Speech Synthesis

Neural waveform models such as WaveNet have demonstrated better performance than conventional vocoders for statistical parametric speech synthesis. As an autoregressive (AR) model, WaveNet is limited by a slow sequential waveform generation process. Some new models that use the inverse-autoregressive flow (IAF) can generate a whole waveform in a one-shot manner. However, these IAF-based models require sequential transformation during training, which severely slows down the training speed. Other models such as Parallel WaveNet and ClariNet bring together the benefits of AR and IAF-based models and train an IAF model by transferring the knowledge from a pre-trained AR teacher to an IAF student without any sequential transformation. However, both models require additional training criteria, and their implementation is prohibitively complicated. We propose a framework for neural source-filter (NSF) waveform modeling without AR nor IAF-based approaches. This framework requires only three components for waveform generation: a source module that generates a sine-based signal as excitation, a non-AR dilated-convolution-based filter module that transforms the excitation into a waveform, and a conditional module that pre-processes the acoustic features for the source and filer modules. This framework minimizes spectral-amplitude distances for model training, which can be efficiently implemented by using short-time Fourier transform routines. Under this framework, we designed three NSF models and compared them with WaveNet. It was demonstrated that the NSF models generated waveforms at least 100 times faster than WaveNet, and the quality of the synthetic speech from the best NSF model was better than or equally good as that from WaveNet.
Experiments in Cuneiform Language Identification
Authors: Gustavo Henrique Paetzold, Marcos Zampieri
Date: 2019-04-27 00:00:00
Proceeding: WS 2019 6
Tasks: Language Identification

This paper presents methods to discriminate between languages and dialects
written in Cuneiform script, one of the first writing systems in the world. We
report the results obtained by the PZ team in the Cuneiform Language
Identification (CLI) shared task organized within the scope of the VarDial
Evaluation Campaign 2019. The task included two languages, Sumerian and
Akkadian. The latter is divided into six dialects: Old Babylonian, Middle
Babylonian peripheral, Standard Babylonian, Neo Babylonian, Late Babylonian,
and Neo Assyrian. We approach the task using a meta-classifier trained on
various SVM models and we show the effectiveness of the system for this task.
Our submission achieved 0.738 F1 score in discriminating between the seven
languages and dialects and it was ranked fourth in the competition among eight
teams.
Graph Neural Reasoning for 2-Quantified Boolean Formula Solvers
Authors: Zhanfu Yang, Fei Wang, Ziliang Chen, Guannan Wei, Tiark Rompf
Date: 2019-04-27 00:00:00
Tasks: Graph Neural Network

In this paper, we investigate the feasibility of learning GNN (Graph Neural
Network) based solvers and GNN-based heuristics for specified QBF (Quantified
Boolean Formula) problems. We design and evaluate several GNN architectures for
2QBF formulae, and conjecture that GNN has limitations in learning 2QBF
solvers. Then we show how to learn a heuristic CEGAR 2QBF solver. We further
explore generalizing GNN-based heuristics to larger unseen instances, and
uncover some interesting challenges. In summary, this paper provides a
comprehensive surveying view of applying GNN-embeddings to specified QBF
solvers, and aims to offer guidance in applying ML to more complicated symbolic
reasoning problems.
Exponential Family Estimation via Adversarial Dynamics Embedding
Authors: Bo Dai, Zhen Liu, Hanjun Dai, Niao He, Arthur Gretton, Le Song, Dale Schuurmans
Date: 2019-04-27 00:00:00
Proceeding: NeurIPS 2019 12

We present an efficient algorithm for maximum likelihood estimation (MLE) of exponential family models, with a general parametrization of the energy function that includes neural networks. We exploit the primal-dual view of the MLE with a kinetics augmented model to obtain an estimate associated with an adversarial dual sampler. To represent this sampler, we introduce a novel neural architecture, dynamics embedding, that generalizes Hamiltonian Monte-Carlo (HMC). The proposed approach inherits the flexibility of HMC while enabling tractable entropy estimation for the augmented model. By learning both a dual sampler and the primal model simultaneously, and sharing parameters between them, we obviate the requirement to design a separate sampling procedure once the model has been trained, leading to more effective learning. We show that many existing estimators, such as contrastive divergence, pseudo/composite-likelihood, score matching, minimum Stein discrepancy estimator, non-local contrastive objectives, noise-contrastive estimation, and minimum probability flow, are special cases of the proposed approach, each expressed by a different (fixed) dual sampler. An empirical investigation shows that adapting the sampler during MLE can significantly improve on state-of-the-art estimators.
Flow-based generative models for Markov chain Monte Carlo in lattice field theory
Authors: M. S. Albergo, G. Kanwar, P. E. Shanahan
Date: 2019-04-26 00:00:00

A Markov chain update scheme using a machine-learned flow-based generative model is proposed for Monte Carlo sampling in lattice field theories. The generative model may be optimized (trained) to produce samples from a distribution approximating the desired Boltzmann distribution determined by the lattice action of the theory being studied. Training the model systematically improves autocorrelation times in the Markov chain, even in regions of parameter space where standard Markov chain Monte Carlo algorithms exhibit critical slowing down in producing decorrelated updates. Moreover, the model may be trained without existing samples from the desired distribution. The algorithm is compared with HMC and local Metropolis sampling for $\phi^4$ theory in two dimensions.
Smoothing and Interpolating Noisy GPS Data with Smoothing Splines
Authors: Jeffrey J. Early, Adam M. Sykulski
Date: 2019-04-26 00:00:00

A comprehensive methodology is provided for smoothing noisy, irregularly sampled data with non-Gaussian noise using smoothing splines. We demonstrate how the spline order and tension parameter can be chosen a priori from physical reasoning. We also show how to allow for non-Gaussian noise and outliers which are typical in GPS signals. We demonstrate the effectiveness of our methods on GPS trajectory data obtained from oceanographic floating instruments known as drifters.
ARCHANGEL: Tamper-proofing Video Archives using Temporal Content Hashes on the Blockchain
Authors: Tu Bui, Daniel Cooper, John Collomosse, Mark Bell, Alex Green, John Sheridan, Jez Higgins, Arindra Das, Jared Keller, Olivier Thereaux, Alan Brown
Date: 2019-04-26 00:00:00

We present ARCHANGEL; a novel distributed ledger based system for assuring
the long-term integrity of digital video archives. First, we describe a novel
deep network architecture for computing compact temporal content hashes (TCHs)
from audio-visual streams with durations of minutes or hours. Our TCHs are
sensitive to accidental or malicious content modification (tampering) but
invariant to the codec used to encode the video. This is necessary due to the
curatorial requirement for archives to format shift video over time to ensure
future accessibility. Second, we describe how the TCHs (and the models used to
derive them) are secured via a proof-of-authority blockchain distributed across
multiple independent archives. We report on the efficacy of ARCHANGEL within
the context of a trial deployment in which the national government archives of
the United Kingdom, Estonia and Norway participated.
Inductive Matrix Completion Based on Graph Neural Networks
Authors: Muhan Zhang, Yixin Chen
Date: 2019-04-26 00:00:00
Proceeding: ICLR 2020 1
Tasks: Graph Neural Network, Matrix Completion, Recommendation Systems, Transfer Learning

We propose an inductive matrix completion model without using side information. By factorizing the (rating) matrix into the product of low-dimensional latent embeddings of rows (users) and columns (items), a majority of existing matrix completion methods are transductive, since the learned embeddings cannot generalize to unseen rows/columns or to new matrices. To make matrix completion inductive, most previous works use content (side information), such as user's age or movie's genre, to make predictions. However, high-quality content is not always available, and can be hard to extract. Under the extreme setting where not any side information is available other than the matrix to complete, can we still learn an inductive matrix completion model? In this paper, we propose an Inductive Graph-based Matrix Completion (IGMC) model to address this problem. IGMC trains a graph neural network (GNN) based purely on 1-hop subgraphs around (user, item) pairs generated from the rating matrix and maps these subgraphs to their corresponding ratings. It achieves highly competitive performance with state-of-the-art transductive baselines. In addition, IGMC is inductive -- it can generalize to users/items unseen during the training (given that their interactions exist), and can even transfer to new tasks. Our transfer learning experiments show that a model trained out of the MovieLens dataset can be directly used to predict Douban movie ratings with surprisingly good performance. Our work demonstrates that: 1) it is possible to train inductive matrix completion models without using side information while achieving similar or better performances than state-of-the-art transductive methods; 2) local graph patterns around a (user, item) pair are effective predictors of the rating this user gives to the item; and 3) Long-range dependencies might not be necessary for modeling recommender systems.
Benchmark and Survey of Automated Machine Learning Frameworks
Authors: Marc-André Zöller, Marco F. Huber
Date: 2019-04-26 00:00:00
Tasks: AutoML, BIG-bench Machine Learning, Survey

Machine learning (ML) has become a vital part in many aspects of our daily life. However, building well performing machine learning applications requires highly specialized data scientists and domain experts. Automated machine learning (AutoML) aims to reduce the demand for data scientists by enabling domain experts to build machine learning applications automatically without extensive knowledge of statistics and machine learning. This paper is a combination of a survey on current AutoML methods and a benchmark of popular AutoML frameworks on real data sets. Driven by the selected frameworks for evaluation, we summarize and review important AutoML techniques and methods concerning every step in building an ML pipeline. The selected AutoML frameworks are evaluated on 137 data sets from established AutoML benchmark suits.
Sample Amplification: Increasing Dataset Size even when Learning is Impossible
Authors: Brian Axelrod, Shivam Garg, Vatsal Sharan, Gregory Valiant
Date: 2019-04-26 00:00:00
Proceeding: ICML 2020 1
Tasks: valid

Given data drawn from an unknown distribution, $D$, to what extent is it possible to ``amplify'' this dataset and output an even larger set of samples that appear to have been drawn from $D$? We formalize this question as follows: an $(n,m)$ $\text{amplification procedure}$ takes as input $n$ independent draws from an unknown distribution $D$, and outputs a set of $m > n$ ``samples''. An amplification procedure is valid if no algorithm can distinguish the set of $m$ samples produced by the amplifier from a set of $m$ independent draws from $D$, with probability greater than $2/3$. Perhaps surprisingly, in many settings, a valid amplification procedure exists, even when the size of the input dataset, $n$, is significantly less than what would be necessary to learn $D$ to non-trivial accuracy. Specifically we consider two fundamental settings: the case where $D$ is an arbitrary discrete distribution supported on $\le k$ elements, and the case where $D$ is a $d$-dimensional Gaussian with unknown mean, and fixed covariance. In the first case, we show that an $\left(n, n + \Theta(\frac{n}{\sqrt{k}})\right)$ amplifier exists. In particular, given $n=O(\sqrt{k})$ samples from $D$, one can output a set of $m=n+1$ datapoints, whose total variation distance from the distribution of $m$ i.i.d. draws from $D$ is a small constant, despite the fact that one would need quadratically more data, $n=\Theta(k)$, to learn $D$ up to small constant total variation distance. In the Gaussian case, we show that an $\left(n,n+\Theta(\frac{n}{\sqrt{d}} )\right)$ amplifier exists, even though learning the distribution to small constant total variation distance requires $\Theta(d)$ samples. In both the discrete and Gaussian settings, we show that these results are tight, to constant factors. Beyond these results, we formalize a number of curious directions for future research along this vein.
Data Poisoning Attack against Knowledge Graph Embedding
Authors: Hengtong Zhang, Tianhang Zheng, Jing Gao, Chenglin Miao, Lu Su, Yaliang Li, Kui Ren
Date: 2019-04-26 00:00:00
Tasks: Data Poisoning, Graph Embedding, Knowledge Graph Completion, Knowledge Graph Embedding, Question Answering

Knowledge graph embedding (KGE) is a technique for learning continuous embeddings for entities and relations in the knowledge graph.Due to its benefit to a variety of downstream tasks such as knowledge graph completion, question answering and recommendation, KGE has gained significant attention recently. Despite its effectiveness in a benign environment, KGE' robustness to adversarial attacks is not well-studied. Existing attack methods on graph data cannot be directly applied to attack the embeddings of knowledge graph due to its heterogeneity. To fill this gap, we propose a collection of data poisoning attack strategies, which can effectively manipulate the plausibility of arbitrary targeted facts in a knowledge graph by adding or deleting facts on the graph. The effectiveness and efficiency of the proposed attack strategies are verified by extensive evaluations on two widely-used benchmarks.
Dynamic Mini-batch SGD for Elastic Distributed Training: Learning in the Limbo of Resources
Authors: Haibin Lin, Hang Zhang, Yifei Ma, Tong He, Zhi Zhang, Sheng Zha, Mu Li
Date: 2019-04-26 00:00:00
Tasks: image-classification, Image Classification, object-detection, Object Detection, Scheduling, Semantic Segmentation

With an increasing demand for training powers for deep learning algorithms
and the rapid growth of computation resources in data centers, it is desirable
to dynamically schedule different distributed deep learning tasks to maximize
resource utilization and reduce cost. In this process, different tasks may
receive varying numbers of machines at different time, a setting we call
elastic distributed training. Despite the recent successes in large mini-batch
distributed training, these methods are rarely tested in elastic distributed
training environments and suffer degraded performance in our experiments, when
we adjust the learning rate linearly immediately with respect to the batch
size. One difficulty we observe is that the noise in the stochastic momentum
estimation is accumulated over time and will have delayed effects when the
batch size changes. We therefore propose to smoothly adjust the learning rate
over time to alleviate the influence of the noisy momentum estimation. Our
experiments on image classification, object detection and semantic segmentation
have demonstrated that our proposed Dynamic SGD method achieves stabilized
performance when varying the number of GPUs from 8 to 128. We also provide
theoretical understanding on the optimality of linear learning rate scheduling
and the effects of stochastic momentum.
Towards a Science of Mind
Authors: Jerome Feldman
Date: 2018-11-06 00:00:00

The ancient mind/body problem continues to be one of deepest mysteries of science and of the human spirit. Despite major advances in many fields, there is still no plausible link between subjective experience (qualia) and its realization in the body. This paper outlines some of the elements of a rigorous science of mind (SoM) - key ideas include scientific realism of mind, agnostic mysterianism, careful attention to language, and a focus on concrete (touchstone) questions and results. A core suggestion is to focus effort on the (still mysterious) mapping from neural activity to subjective experience.