pwc-archive/papers-with-abstracts huggingface dataset:
Tencent ML-Images: A Large-Scale Multi-Label Image Database for Visual Representation Learning
Authors: Baoyuan Wu, Weidong Chen, Yanbo Fan, Yong Zhang, Jinlong Hou, Jie Liu, Tong Zhang
Date: 2019-01-07 00:00:00
Tasks: image-classification, Image Classification, object-detection, Object Detection, Representation Learning, Semantic Segmentation, TAG, Transfer Learning

In existing visual representation learning tasks, deep convolutional neural networks (CNNs) are often trained on images annotated with single tags, such as ImageNet. However, a single tag cannot describe all important contents of one image, and some useful visual information may be wasted during training. In this work, we propose to train CNNs from images annotated with multiple tags, to enhance the quality of visual representation of the trained CNN model. To this end, we build a large-scale multi-label image database with 18M images and 11K categories, dubbed Tencent ML-Images. We efficiently train the ResNet-101 model with multi-label outputs on Tencent ML-Images, taking 90 hours for 60 epochs, based on a large-scale distributed deep learning framework,i.e.,TFplus. The good quality of the visual representation of the Tencent ML-Images checkpoint is verified through three transfer learning tasks, including single-label image classification on ImageNet and Caltech-256, object detection on PASCAL VOC 2007, and semantic segmentation on PASCAL VOC 2012. The Tencent ML-Images database, the checkpoints of ResNet-101, and all the training codehave been released at https://github.com/Tencent/tencent-ml-images. It is expected to promote other vision tasks in the research and industry community.
Semi-supervised learning in unbalanced and heterogeneous networks
Authors: Ting Li, Ningchen Ying, Xianshi Yu, Bin-Yi Jing
Date: 2019-01-07 00:00:00
Tasks: Clustering, Community Detection

Community detection was a hot topic on network analysis, where the main aim
is to perform unsupervised learning or clustering in networks. Recently,
semi-supervised learning has received increasing attention among researchers.
In this paper, we propose a new algorithm, called weighted inverse Laplacian
(WIL), for predicting labels in partially labeled networks. The idea comes from
the first hitting time in random walk, and it also has nice explanations both
in information propagation and the regularization framework. We propose a
partially labeled degree-corrected block model (pDCBM) to describe the
generation of partially labeled networks. We show that WIL ensures the
misclassification rate is of order $O(\frac{1}{d})$ for the pDCBM with average
degree $d=\Omega(\log n),$ and that it can handle situations with greater
unbalanced than traditional Laplacian methods. WIL outperforms other
state-of-the-art methods in most of our simulations and real datasets,
especially in unbalanced networks and heterogeneous networks.
Vector representations of text data in deep learning
Authors: Karol Grzegorczyk
Date: 2019-01-07 00:00:00
Tasks: Deep Learning, Information Retrieval, Part-Of-Speech Tagging, Retrieval, Transfer Learning, Word Embeddings, Word Sense Induction

In this dissertation we report results of our research on dense distributed
representations of text data. We propose two novel neural models for learning
such representations. The first model learns representations at the document
level, while the second model learns word-level representations.
  For document-level representations we propose Binary Paragraph Vector: a
neural network models for learning binary representations of text documents,
which can be used for fast document retrieval. We provide a thorough evaluation
of these models and demonstrate that they outperform the seminal method in the
field in the information retrieval task. We also report strong results in
transfer learning settings, where our models are trained on a generic text
corpus and then used to infer codes for documents from a domain-specific
dataset. In contrast to previously proposed approaches, Binary Paragraph Vector
models learn embeddings directly from raw text data.
  For word-level representations we propose Disambiguated Skip-gram: a neural
network model for learning multi-sense word embeddings. Representations learned
by this model can be used in downstream tasks, like part-of-speech tagging or
identification of semantic relations. In the word sense induction task
Disambiguated Skip-gram outperforms state-of-the-art models on three out of
four benchmarks datasets. Our model has an elegant probabilistic
interpretation. Furthermore, unlike previous models of this kind, it is
differentiable with respect to all its parameters and can be trained with
backpropagation. In addition to quantitative results, we present qualitative
evaluation of Disambiguated Skip-gram, including two-dimensional visualisations
of selected word-sense embeddings.
Ten ways to fool the masses with machine learning
Authors: Fayyaz Minhas, Amina Asif, Asa Ben-Hur
Date: 2019-01-07 00:00:00
Tasks: BIG-bench Machine Learning

If you want to tell people the truth, make them laugh, otherwise they'll kill
you. (source unclear)
  Machine learning and deep learning are the technologies of the day for
developing intelligent automatic systems. However, a key hurdle for progress in
the field is the literature itself: we often encounter papers that report
results that are difficult to reconstruct or reproduce, results that
mis-represent the performance of the system, or contain other biases that limit
their validity. In this semi-humorous article, we discuss issues that arise in
running and reporting results of machine learning experiments. The purpose of
the article is to provide a list of watch out points for researchers to be
aware of when developing machine learning models or writing and reviewing
machine learning papers.
Image Super-Resolution as a Defense Against Adversarial Attacks
Authors: Aamir Mustafa, Salman H. Khan, Munawar Hayat, Jianbing Shen, Ling Shao
Date: 2019-01-07 00:00:00
Tasks: Adversarial Defense, Image Enhancement, Image Restoration, Image Super-Resolution, Super-Resolution

Convolutional Neural Networks have achieved significant success across multiple computer vision tasks. However, they are vulnerable to carefully crafted, human-imperceptible adversarial noise patterns which constrain their deployment in critical security-sensitive systems. This paper proposes a computationally efficient image enhancement approach that provides a strong defense mechanism to effectively mitigate the effect of such adversarial perturbations. We show that deep image restoration networks learn mapping functions that can bring off-the-manifold adversarial samples onto the natural image manifold, thus restoring classification towards correct classes. A distinguishing feature of our approach is that, in addition to providing robustness against attacks, it simultaneously enhances image quality and retains models performance on clean images. Furthermore, the proposed method does not modify the classifier or requires a separate mechanism to detect adversarial images. The effectiveness of the scheme has been demonstrated through extensive experiments, where it has proven a strong defense in gray-box settings. The proposed scheme is simple and has the following advantages: (1) it does not require any model training or parameter optimization, (2) it complements other existing defense mechanisms, (3) it is agnostic to the attacked model and attack type and (4) it provides superior performance across all popular attack algorithms. Our codes are publicly available at https://github.com/aamir-mustafa/super-resolution-adversarial-defense.
Generalization in Deep Networks: The Role of Distance from Initialization
Authors: Vaishnavh Nagarajan, J. Zico Kolter
Date: 2019-01-07 00:00:00

Why does training deep neural networks using stochastic gradient descent
(SGD) result in a generalization error that does not worsen with the number of
parameters in the network? To answer this question, we advocate a notion of
effective model capacity that is dependent on {\em a given random
initialization of the network} and not just the training algorithm and the data
distribution. We provide empirical evidences that demonstrate that the model
capacity of SGD-trained deep networks is in fact restricted through implicit
regularization of {\em the $\ell_2$ distance from the initialization}. We also
provide theoretical arguments that further highlight the need for
initialization-dependent notions of model capacity. We leave as open questions
how and why distance from initialization is regularized, and whether it is
sufficient to explain generalization.
Deeper and Wider Siamese Networks for Real-Time Visual Tracking
Authors: Zhipeng Zhang, Houwen Peng
Date: 2019-01-07 00:00:00
Proceeding: CVPR 2019 6
Tasks: Real-Time Visual Tracking, Video Object Tracking, Visual Object Tracking, Visual Tracking

Siamese networks have drawn great attention in visual tracking because of
their balanced accuracy and speed. However, the backbone networks used in
Siamese trackers are relatively shallow, such as AlexNet [18], which does not
fully take advantage of the capability of modern deep neural networks. In this
paper, we investigate how to leverage deeper and wider convolutional neural
networks to enhance tracking robustness and accuracy. We observe that direct
replacement of backbones with existing powerful architectures, such as ResNet
[14] and Inception [33], does not bring improvements. The main reasons are that
1)large increases in the receptive field of neurons lead to reduced feature
discriminability and localization precision; and 2) the network padding for
convolutions induces a positional bias in learning. To address these issues, we
propose new residual modules to eliminate the negative impact of padding, and
further design new architectures using these modules with controlled receptive
field size and network stride. The designed architectures are lightweight and
guarantee real-time tracking speed when applied to SiamFC [2] and SiamRPN [20].
Experiments show that solely due to the proposed network architectures, our
SiamFC+ and SiamRPN+ obtain up to 9.8%/5.7% (AUC), 23.3%/8.8% (EAO) and
24.4%/25.0% (EAO) relative improvements over the original versions [2, 20] on
the OTB-15, VOT-16 and VOT-17 datasets, respectively.
Learning Nonlinear Input-Output Maps with Dissipative Quantum Systems
Authors: Jiayin Chen, Hendra I. Nurdin
Date: 2019-01-07 00:00:00

In this paper, we develop a theory of learning nonlinear input-output maps with fading memory by dissipative quantum systems, as a quantum counterpart of the theory of approximating such maps using classical dynamical systems. The theory identifies the properties required for a class of dissipative quantum systems to be {\em universal}, in that any input-output map with fading memory can be approximated arbitrarily closely by an element of this class. We then introduce an example class of dissipative quantum systems that is provably universal. Numerical experiments illustrate that with a small number of qubits, this class can achieve comparable performance to classical learning schemes with a large number of tunable parameters. Further numerical analysis suggests that the exponentially increasing Hilbert space presents a potential resource for dissipative quantum systems to surpass classical learning schemes for input-output maps.
Randomized Tensor Ring Decomposition and Its Application to Large-scale Data Reconstruction
Authors: Longhao Yuan, Chao Li, Jianting Cao, Qibin Zhao
Date: 2019-01-07 00:00:00
Tasks: Dimensionality Reduction, Image Reconstruction

Dimensionality reduction is an essential technique for multi-way large-scale
data, i.e., tensor. Tensor ring (TR) decomposition has become popular due to
its high representation ability and flexibility. However, the traditional TR
decomposition algorithms suffer from high computational cost when facing
large-scale data. In this paper, taking advantages of the recently proposed
tensor random projection method, we propose two TR decomposition algorithms. By
employing random projection on every mode of the large-scale tensor, the TR
decomposition can be processed at a much smaller scale. The simulation
experiment shows that the proposed algorithms are $4-25$ times faster than
traditional algorithms without loss of accuracy, and our algorithms show
superior performance in deep learning dataset compression and hyperspectral
image reconstruction experiments compared to other randomized algorithms.
Tooth morphometry using quasi-conformal theory
Authors: Gary P. T. Choi, Hei Long Chan, Robin Yong, Sarbin Ranjitkar, Alan Brook, Grant Townsend, Ke Chen, Lok Ming Lui
Date: 2019-01-07 00:00:00
Tasks: General Classification

Shape analysis is important in anthropology, bioarchaeology and forensic
science for interpreting useful information from human remains. In particular,
teeth are morphologically stable and hence well-suited for shape analysis. In
this work, we propose a framework for tooth morphometry using quasi-conformal
theory. Landmark-matching Teichm\"uller maps are used for establishing a 1-1
correspondence between tooth surfaces with prescribed anatomical landmarks.
Then, a quasi-conformal statistical shape analysis model based on the
Teichm\"uller mapping results is proposed for building a tooth classification
scheme. We deploy our framework on a dataset of human premolars to analyze the
tooth shape variation among genders and ancestries. Experimental results show
that our method achieves much higher classification accuracy with respect to
both gender and ancestry when compared to the existing methods. Furthermore,
our model reveals the underlying tooth shape difference between different
genders and ancestries in terms of the local geometric distortion and
curvatures.
Better Guider Predicts Future Better: Difference Guided Generative Adversarial Networks
Authors: Guohao Ying, Yingtian Zou, Lin Wan, Yiming Hu, Jiashi Feng
Date: 2019-01-07 00:00:00
Tasks: Video Prediction

Predicting the future is a fantasy but practicality work. It is the key
component to intelligent agents, such as self-driving vehicles, medical
monitoring devices and robotics. In this work, we consider generating unseen
future frames from previous obeservations, which is notoriously hard due to the
uncertainty in frame dynamics. While recent works based on generative
adversarial networks (GANs) made remarkable progress, there is still an
obstacle for making accurate and realistic predictions. In this paper, we
propose a novel GAN based on inter-frame difference to circumvent the
difficulties. More specifically, our model is a multi-stage generative network,
which is named the Difference Guided Generative Adversarial Netwok (DGGAN). The
DGGAN learns to explicitly enforce future-frame predictions that is guided by
synthetic inter-frame difference. Given a sequence of frames, DGGAN first uses
dual paths to generate meta information. One path, called Coarse Frame
Generator, predicts the coarse details about future frames, and the other path,
called Difference Guide Generator, generates the difference image which include
complementary fine details. Then our coarse details will then be refined via
guidance of difference image under the support of GANs. With this model and
novel architecture, we achieve state-of-the-art performance for future video
prediction on UCF-101, KITTI.
Text Mining Customer Reviews For Aspect-based Restaurant Rating
Authors: Jovelyn C. Cuizon, Jesserine Lopez, Danica Rose Jones
Date: 2019-01-07 00:00:00
Tasks: Sentence

This study applies text mining to analyze customer reviews and automatically
assign a collective restaurant star rating based on five predetermined aspects:
ambiance, cost, food, hygiene, and service. The application provides a web and
mobile crowd sourcing platform where users share dining experiences and get
insights about the strengths and weaknesses of a restaurant through user
contributed feedback. Text reviews are tokenized into sentences. Noun-adjective
pairs are extracted from each sentence using Stanford Core NLP library and are
associated to aspects based on the bag of associated words fed into the system.
The sentiment weight of the adjectives is determined through AFINN library. An
overall restaurant star rating is computed based on the individual aspect
rating. Further, a word cloud is generated to provide visual display of the
most frequently occurring terms in the reviews. The more feedbacks are added
the more reflective the sentiment score to the restaurants' performance.
Blind Motion Deblurring with Cycle Generative Adversarial Networks
Authors: Quan Yuan, Junxia Li, Lingwei Zhang, Zhefu Wu, Guangyu Liu
Date: 2019-01-07 00:00:00
Tasks: Deblurring, Decoder, Generative Adversarial Network, Image Deblurring

Blind motion deblurring is one of the most basic and challenging problems in
image processing and computer vision. It aims to recover a sharp image from its
blurred version knowing nothing about the blur process. Many existing methods
use Maximum A Posteriori (MAP) or Expectation Maximization (EM) frameworks to
deal with this kind of problems, but they cannot handle well the figh frequency
features of natural images. Most recently, deep neural networks have been
emerging as a powerful tool for image deblurring. In this paper, we prove that
encoder-decoder architecture gives better results for image deblurring tasks.
In addition, we propose a novel end-to-end learning model which refines
generative adversarial network by many novel training strategies so as to
tackle the problem of deblurring. Experimental results show that our model can
capture high frequency features well, and the results on benchmark dataset show
that proposed model achieves the competitive performance.
Sharp Restricted Isometry Bounds for the Inexistence of Spurious Local Minima in Nonconvex Matrix Recovery
Authors: Richard Y. Zhang, Somayeh Sojoudi, Javad Lavaei
Date: 2019-01-07 00:00:00

Nonconvex matrix recovery is known to contain no spurious local minima under a restricted isometry property (RIP) with a sufficiently small RIP constant $\delta$. If $\delta$ is too large, however, then counterexamples containing spurious local minima are known to exist. In this paper, we introduce a proof technique that is capable of establishing sharp thresholds on $\delta$ to guarantee the inexistence of spurious local minima. Using the technique, we prove that in the case of a rank-1 ground truth, an RIP constant of $\delta<1/2$ is both necessary and sufficient for exact recovery from any arbitrary initial point (such as a random point). We also prove a local recovery result: given an initial point $x_{0}$ satisfying $f(x_{0})\le(1-\delta)^{2}f(0)$, any descent algorithm that converges to second-order optimality guarantees exact recovery.
Composite optimization for robust blind deconvolution
Authors: Vasileios Charisopoulos, Damek Davis, Mateo Díaz, Dmitriy Drusvyatskiy
Date: 2019-01-06 00:00:00

The blind deconvolution problem seeks to recover a pair of vectors from a set
of rank one bilinear measurements. We consider a natural nonsmooth formulation
of the problem and show that under standard statistical assumptions, its moduli
of weak convexity, sharpness, and Lipschitz continuity are all dimension
independent. This phenomenon persists even when up to half of the measurements
are corrupted by noise. Consequently, standard algorithms, such as the
subgradient and prox-linear methods, converge at a rapid dimension-independent
rate when initialized within constant relative error of the solution. We then
complete the paper with a new initialization strategy, complementing the local
search algorithms. The initialization procedure is both provably efficient and
robust to outlying measurements. Numerical experiments, on both simulated and
real data, illustrate the developed theory and methods.
Healthy versus pathological learning transferability in shoulder muscle MRI segmentation using deep convolutional encoder-decoders
Authors: Pierre-Henri Conze, Sylvain Brochard, Valérie Burdin, Frances T. Sheehan, Christelle Pons
Date: 2019-01-06 00:00:00
Tasks: Management, MRI segmentation, Segmentation

Automatic segmentation of pathological shoulder muscles in patients with musculo-skeletal diseases is a challenging task due to the huge variability in muscle shape, size, location, texture and injury. A reliable fully-automated segmentation method from magnetic resonance images could greatly help clinicians to plan therapeutic interventions and predict interventional outcomes while eliminating time consuming manual segmentation efforts. The purpose of this work is three-fold. First, we investigate the feasibility of pathological shoulder muscle segmentation using deep learning techniques, given a very limited amount of available annotated pediatric data. Second, we address the learning transferability from healthy to pathological data by comparing different learning schemes in terms of model generalizability. Third, extended versions of deep convolutional encoder-decoder architectures using encoders pre-trained on non-medical data are proposed to improve the segmentation accuracy. Methodological aspects are evaluated in a leave-one-out fashion on a dataset of 24 shoulder examinations from patients with obstetrical brachial plexus palsy and focus on 4 different muscles including deltoid as well as infraspinatus, supraspinatus and subscapularis from the rotator cuff. The most relevant segmentation model is partially pre-trained on ImageNet and jointly exploits inter-patient healthy and pathological annotated data. Its performance reaches Dice scores of 82.4%, 82.0%, 71.0% and 82.8% for deltoid, infraspinatus, supraspinatus and subscapularis muscles. Absolute surface estimation errors are all below 83mm$^2$ except for supraspinatus with 134.6mm$^2$. These contributions offer new perspectives for force inference in the context of musculo-skeletal disorder management.
Scaling description of generalization with number of parameters in deep learning
Authors: Mario Geiger, Arthur Jacot, Stefano Spigler, Franck Gabriel, Levent Sagun, Stéphane d'Ascoli, Giulio Biroli, Clément Hongler, Matthieu Wyart
Date: 2019-01-06 00:00:00

Supervised deep learning involves the training of neural networks with a large number $N$ of parameters. For large enough $N$, in the so-called over-parametrized regime, one can essentially fit the training data points. Sparsity-based arguments would suggest that the generalization error increases as $N$ grows past a certain threshold $N^{*}$. Instead, empirical studies have shown that in the over-parametrized regime, generalization error keeps decreasing with $N$. We resolve this paradox through a new framework. We rely on the so-called Neural Tangent Kernel, which connects large neural nets to kernel methods, to show that the initialization causes finite-size random fluctuations $\|f_{N}-\bar{f}_{N}\|\sim N^{-1/4}$ of the neural net output function $f_{N}$ around its expectation $\bar{f}_{N}$. These affect the generalization error $\epsilon_{N}$ for classification: under natural assumptions, it decays to a plateau value $\epsilon_{\infty}$ in a power-law fashion $\sim N^{-1/2}$. This description breaks down at a so-called jamming transition $N=N^{*}$. At this threshold, we argue that $\|f_{N}\|$ diverges. This result leads to a plausible explanation for the cusp in test error known to occur at $N^{*}$. Our results are confirmed by extensive empirical observations on the MNIST and CIFAR image datasets. Our analysis finally suggests that, given a computational envelope, the smallest generalization error is obtained using several networks of intermediate sizes, just beyond $N^{*}$, and averaging their outputs.