pwc-archive/papers-with-abstracts huggingface dataset:
Towards Efficient Large-Scale Graph Neural Network Computing
Authors: Lingxiao Ma, Zhi Yang, Youshan Miao, Jilong Xue, Ming Wu, Lidong Zhou, Yafei Dai
Date: 2018-10-19 00:00:00
Tasks: GPU, Graph Neural Network, graph partitioning, Knowledge Graphs

Recent deep learning models have moved beyond low-dimensional regular grids
such as image, video, and speech, to high-dimensional graph-structured data,
such as social networks, brain connections, and knowledge graphs. This
evolution has led to large graph-based irregular and sparse models that go
beyond what existing deep learning frameworks are designed for. Further, these
models are not easily amenable to efficient, at scale, acceleration on parallel
hardwares (e.g. GPUs). We introduce NGra, the first parallel processing
framework for graph-based deep neural networks (GNNs). NGra presents a new
SAGA-NN model for expressing deep neural networks as vertex programs with each
layer in well-defined (Scatter, ApplyEdge, Gather, ApplyVertex) graph operation
stages. This model not only allows GNNs to be expressed intuitively, but also
facilitates the mapping to an efficient dataflow representation. NGra addresses
the scalability challenge transparently through automatic graph partitioning
and chunk-based stream processing out of GPU core or over multiple GPUs, which
carefully considers data locality, data movement, and overlapping of parallel
processing and data movement. NGra further achieves efficiency through highly
optimized Scatter/Gather operators on GPUs despite its sparsity. Our evaluation
shows that NGra scales to large real graphs that none of the existing
frameworks can handle directly, while achieving up to about 4 times speedup
even at small scales over the multiple-baseline design on TensorFlow.
STACL: Simultaneous Translation with Implicit Anticipation and Controllable Latency using Prefix-to-Prefix Framework
Authors: Mingbo Ma, Liang Huang, Hao Xiong, Renjie Zheng, Kaibo Liu, Baigong Zheng, Chuanqiang Zhang, Zhongjun He, Hairong Liu, Xing Li, Hua Wu, Haifeng Wang
Date: 2018-10-19 00:00:00
Proceeding: ACL 2019 7
Tasks: de-en, Sentence, Translation

Simultaneous translation, which translates sentences before they are finished, is useful in many scenarios but is notoriously difficult due to word-order differences. While the conventional seq-to-seq framework is only suitable for full-sentence translation, we propose a novel prefix-to-prefix framework for simultaneous translation that implicitly learns to anticipate in a single translation model. Within this framework, we present a very simple yet surprisingly effective wait-k policy trained to generate the target sentence concurrently with the source sentence, but always k words behind. Experiments show our strategy achieves low latency and reasonable quality (compared to full-sentence translation) on 4 directions: zh<->en and de<->en.
DGC-Net: Dense Geometric Correspondence Network
Authors: Iaroslav Melekhov, Aleksei Tiulpin, Torsten Sattler, Marc Pollefeys, Esa Rahtu, Juho Kannala
Date: 2018-10-19 00:00:00
Tasks: Camera Pose Estimation, Dense Pixel Correspondence Estimation, Optical Flow Estimation, Translation

This paper addresses the challenge of dense pixel correspondence estimation
between two images. This problem is closely related to optical flow estimation
task where ConvNets (CNNs) have recently achieved significant progress. While
optical flow methods produce very accurate results for the small pixel
translation and limited appearance variation scenarios, they hardly deal with
the strong geometric transformations that we consider in this work. In this
paper, we propose a coarse-to-fine CNN-based framework that can leverage the
advantages of optical flow approaches and extend them to the case of large
transformations providing dense and subpixel accurate estimates. It is trained
on synthetic transformations and demonstrates very good performance to unseen,
realistic, data. Further, we apply our method to the problem of relative camera
pose estimation and demonstrate that the model outperforms existing dense
approaches.
Impact of Corpora Quality on Neural Machine Translation
Authors: Matīss Rikters
Date: 2018-10-19 00:00:00
Tasks: Machine Translation, Translation

Large parallel corpora that are automatically obtained from the web,
documents or elsewhere often exhibit many corrupted parts that are bound to
negatively affect the quality of the systems and models that learn from these
corpora. This paper describes frequent problems found in data and such data
affects neural machine translation systems, as well as how to identify and deal
with them. The solutions are summarised in a set of scripts that remove
problematic sentences from input corpora.
Invocation-driven Neural Approximate Computing with a Multiclass-Classifier and Multiple Approximators
Authors: Haiyue Song, Chengwen Xu, Qiang Xu, Zhuoran Song, Naifeng Jing, Xiaoyao Liang, Li Jiang
Date: 2018-10-19 00:00:00

Neural approximate computing gains enormous energy-efficiency at the cost of
tolerable quality-loss. A neural approximator can map the input data to output
while a classifier determines whether the input data are safe to approximate
with quality guarantee. However, existing works cannot maximize the invocation
of the approximator, resulting in limited speedup and energy saving. By
exploring the mapping space of those target functions, in this paper, we
observe a nonuniform distribution of the approximation error incurred by the
same approximator. We thus propose a novel approximate computing architecture
with a Multiclass-Classifier and Multiple Approximators (MCMA). These
approximators have identical network topologies and thus can share the same
hardware resource in a neural processing unit(NPU) clip. In the runtime, MCMA
can swap in the invoked approximator by merely shipping the synapse weights
from the on-chip memory to the buffers near MAC within a cycle. We also propose
efficient co-training methods for such MCMA architecture. Experimental results
show a more substantial invocation of MCMA as well as the gain of
energy-efficiency.
Saliency guided deep network for weakly-supervised image segmentation
Authors: Fengdong Sun, Wenhui Li
Date: 2018-10-19 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

Weakly-supervised image segmentation is an important task in computer vision.
A key problem is how to obtain high quality objects location from image-level
category. Classification activation mapping is a common method which can be
used to generate high-precise object location cues. However these location cues
are generally very sparse and small such that they can not provide effective
information for image segmentation. In this paper, we propose a saliency guided
image segmentation network to resolve this problem. We employ a self-attention
saliency method to generate subtle saliency maps, and render the location cues
grow as seeds by seeded region growing method to expand pixel-level labels
extent. In the process of seeds growing, we use the saliency values to weight
the similarity between pixels to control the growing. Therefore saliency
information could help generate discriminative object regions, and the effects
of wrong salient pixels can be suppressed efficiently. Experimental results on
a common segmentation dataset PASCAL VOC2012 demonstrate the effectiveness of
our method.
Temporal Action Detection by Joint Identification-Verification
Authors: Wen Wang, Yongjian Wu, Haijun Liu, Shiguang Wang, Jian Cheng
Date: 2018-10-19 00:00:00
Tasks: Action Detection

Temporal action detection aims at not only recognizing action category but
also detecting start time and end time for each action instance in an untrimmed
video. The key challenge of this task is to accurately classify the action and
determine the temporal boundaries of each action instance. In temporal action
detection benchmark: THUMOS 2014, large variations exist in the same action
category while many similarities exist in different action categories, which
always limit the performance of temporal action detection. To address this
problem, we propose to use joint Identification-Verification network to reduce
the intra-action variations and enlarge inter-action differences. The joint
Identification-Verification network is a siamese network based on 3D ConvNets,
which can simultaneously predict the action categories and the similarity
scores for the input pairs of video proposal segments. Extensive experimental
results on the challenging THUMOS 2014 dataset demonstrate the effectiveness of
our proposed method compared to the existing state-of-art methods for temporal
action detection in untrimmed videos.
Generative Low-Shot Network Expansion
Authors: Adi Hayat, Mark Kliger, Shachar Fleishman, Daniel Cohen-Or
Date: 2018-10-19 00:00:00
Proceeding: ICLR 2018 1

Conventional deep learning classifiers are static in the sense that they are
trained on a predefined set of classes and learning to classify a novel class
typically requires re-training. In this work, we address the problem of
Low-Shot network expansion learning. We introduce a learning framework which
enables expanding a pre-trained (base) deep network to classify novel classes
when the number of examples for the novel classes is particularly small. We
present a simple yet powerful hard distillation method where the base network
is augmented with additional weights to classify the novel classes, while
keeping the weights of the base network unchanged. We show that since only a
small number of weights needs to be trained, the hard distillation excels in
low-shot training scenarios. Furthermore, hard distillation avoids detriment to
classification performance on the base classes. Finally, we show that low-shot
network expansion can be done with a very small memory footprint by using a
compact generative model of the base classes training data with only a
negligible degradation relative to learning with the full training set.
Malicious Web Domain Identification using Online Credibility and Performance Data by Considering the Class Imbalance Issue
Authors: Zhongyi Hu, Raymond Chiong, Ilung Pranata, Yukun Bao, Yuqing Lin
Date: 2018-10-19 00:00:00
Tasks: BIG-bench Machine Learning

Purpose: Malicious web domain identification is of significant importance to
the security protection of Internet users. With online credibility and
performance data, this paper aims to investigate the use of machine learning
tech-niques for malicious web domain identification by considering the class
imbalance issue (i.e., there are more benign web domains than malicious ones).
Design/methodology/approach: We propose an integrated resampling approach to
handle class imbalance by combining the Synthetic Minority Over-sampling
TEchnique (SMOTE) and Particle Swarm Optimisation (PSO), a population-based
meta-heuristic algorithm. We use the SMOTE for over-sampling and PSO for
under-sampling. Findings: By applying eight well-known machine learning
classifiers, the proposed integrated resampling approach is comprehensively
examined using several imbalanced web domain datasets with different imbalance
ratios. Com-pared to five other well-known resampling approaches, experimental
results confirm that the proposed approach is highly effective. Practical
implications: This study not only inspires the practical use of online
credibility and performance data for identifying malicious web domains, but
also provides an effective resampling approach for handling the class
imbal-ance issue in the area of malicious web domain identification.
Originality/value: Online credibility and performance data is applied to build
malicious web domain identification models using machine learning techniques.
An integrated resampling approach is proposed to address the class im-balance
issue. The performance of the proposed approach is confirmed based on
real-world datasets with different imbalance ratios.
Super-pixel cloud detection using Hierarchical Fusion CNN
Authors: Han Liu, Dan Zeng, Qi Tian
Date: 2018-10-19 00:00:00
Tasks: Binary Classification, Cloud Detection, Cultural Vocal Bursts Intensity Prediction, General Classification

Cloud detection plays a very important role in the process of remote sensing
images. This paper designs a super-pixel level cloud detection method based on
convolutional neural network (CNN) and deep forest. Firstly, remote sensing
images are segmented into super-pixels through the combination of SLIC and
SEEDS. Structured forests is carried out to compute edge probability of each
pixel, based on which super-pixels are segmented more precisely. Segmented
super-pixels compose a super-pixel level remote sensing database. Though cloud
detection is essentially a binary classification problem, our database is
labeled into four categories: thick cloud, cirrus cloud, building and other
culture, to improve the generalization ability of our proposed models.
Secondly, super-pixel level database is used to train our cloud detection
models based on CNN and deep forest. Considering super-pixel level remote
sensing images contain less semantic information compared with general object
classification database, we propose a Hierarchical Fusion CNN (HFCNN). It takes
full advantage of low-level features like color and texture information and is
more applicable to cloud detection task. In test phase, every super-pixel in
remote sensing images is classified by our proposed models and then combined to
recover final binary mask by our proposed distance metric, which is used to
determine ambiguous super-pixels. Experimental results show that, compared with
conventional methods, HFCNN can achieve better precision and recall.
Exchangeability and Kernel Invariance in Trained MLPs
Authors: Russell Tsuchida, Fred Roosta, Marcus Gallagher
Date: 2018-10-19 00:00:00
Tasks: BIG-bench Machine Learning

In the analysis of machine learning models, it is often convenient to assume
that the parameters are IID. This assumption is not satisfied when the
parameters are updated through training processes such as SGD. A relaxation of
the IID condition is a probabilistic symmetry known as exchangeability. We show
the sense in which the weights in MLPs are exchangeable. This yields the result
that in certain instances, the layer-wise kernel of fully-connected layers
remains approximately constant during training. We identify a sharp change in
the macroscopic behavior of networks as the covariance between weights changes
from zero.
Multi-Domain Pose Network for Multi-Person Pose Estimation and Tracking
Authors: Hengkai Guo, Tang Tang, Guozhong Luo, Riwei Chen, Yongchen Lu, Linfu Wen
Date: 2018-10-19 00:00:00
Tasks: Multi-Person Pose Estimation, Multi-Person Pose Estimation and Tracking, Pose Estimation, Pose Prediction

Multi-person human pose estimation and tracking in the wild is important and
challenging. For training a powerful model, large-scale training data are
crucial. While there are several datasets for human pose estimation, the best
practice for training on multi-dataset has not been investigated. In this
paper, we present a simple network called Multi-Domain Pose Network (MDPN) to
address this problem. By treating the task as multi-domain learning, our
methods can learn a better representation for pose prediction. Together with
prediction heads fine-tuning and multi-branch combination, it shows significant
improvement over baselines and achieves the best performance on PoseTrack ECCV
2018 Challenge without additional datasets other than MPII and COCO.
Zero and Few Shot Learning with Semantic Feature Synthesis and Competitive Learning
Authors: Zhiwu Lu, Jiechao Guan, Aoxue Li, Tao Xiang, An Zhao, Ji-Rong Wen
Date: 2018-10-19 00:00:00
Tasks: Attribute, Few-Shot Learning, Hallucination, Zero-Shot Learning

Zero-shot learning (ZSL) is made possible by learning a projection function
between a feature space and a semantic space (e.g.,~an attribute space). Key to
ZSL is thus to learn a projection that is robust against the often large domain
gap between the seen and unseen class domains. In this work, this is achieved
by unseen class data synthesis and robust projection function learning.
Specifically, a novel semantic data synthesis strategy is proposed, by which
semantic class prototypes (e.g., attribute vectors) are used to simply perturb
seen class data for generating unseen class ones. As in any data
synthesis/hallucination approach, there are ambiguities and uncertainties on
how well the synthesised data can capture the targeted unseen class data
distribution. To cope with this, the second contribution of this work is a
novel projection learning model termed competitive bidirectional projection
learning (BPL) designed to best utilise the ambiguous synthesised data.
Specifically, we assume that each synthesised data point can belong to any
unseen class; and the most likely two class candidates are exploited to learn a
robust projection function in a competitive fashion. As a third contribution,
we show that the proposed ZSL model can be easily extended to few-shot learning
(FSL) by again exploiting semantic (class prototype guided) feature synthesis
and competitive BPL. Extensive experiments show that our model achieves the
state-of-the-art results on both problems.
Transferrable Feature and Projection Learning with Class Hierarchy for Zero-Shot Learning
Authors: Aoxue Li, Zhiwu Lu, Jiechao Guan, Tao Xiang, Li-Wei Wang, Ji-Rong Wen
Date: 2018-10-19 00:00:00
Tasks: Attribute, Clustering, Few-Shot Learning, Zero-Shot Learning

Zero-shot learning (ZSL) aims to transfer knowledge from seen classes to
unseen ones so that the latter can be recognised without any training samples.
This is made possible by learning a projection function between a feature space
and a semantic space (e.g. attribute space). Considering the seen and unseen
classes as two domains, a big domain gap often exists which challenges ZSL.
Inspired by the fact that an unseen class is not exactly `unseen' if it belongs
to the same superclass as a seen class, we propose a novel inductive ZSL model
that leverages superclasses as the bridge between seen and unseen classes to
narrow the domain gap. Specifically, we first build a class hierarchy of
multiple superclass layers and a single class layer, where the superclasses are
automatically generated by data-driven clustering over the semantic
representations of all seen and unseen class names. We then exploit the
superclasses from the class hierarchy to tackle the domain gap challenge in two
aspects: deep feature learning and projection function learning. First, to
narrow the domain gap in the feature space, we integrate a recurrent neural
network (RNN) defined with the superclasses into a convolutional neural network
(CNN), in order to enforce the superclass hierarchy. Second, to further learn a
transferrable projection function for ZSL, a novel projection function learning
method is proposed by exploiting the superclasses to align the two domains.
Importantly, our transferrable feature and projection learning methods can be
easily extended to a closely related task -- few-shot learning (FSL). Extensive
experiments show that the proposed model significantly outperforms the
state-of-the-art alternatives in both ZSL and FSL tasks.
Domain-Invariant Projection Learning for Zero-Shot Recognition
Authors: An Zhao, Mingyu Ding, Jiechao Guan, Zhiwu Lu, Tao Xiang, Ji-Rong Wen
Date: 2018-10-19 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Attribute, Transfer Learning, Zero-Shot Learning

Zero-shot learning (ZSL) aims to recognize unseen object classes without any
training samples, which can be regarded as a form of transfer learning from
seen classes to unseen ones. This is made possible by learning a projection
between a feature space and a semantic space (e.g. attribute space). Key to ZSL
is thus to learn a projection function that is robust against the often large
domain gap between the seen and unseen classes. In this paper, we propose a
novel ZSL model termed domain-invariant projection learning (DIPL). Our model
has two novel components: (1) A domain-invariant feature self-reconstruction
task is introduced to the seen/unseen class data, resulting in a simple linear
formulation that casts ZSL into a min-min optimization problem. Solving the
problem is non-trivial, and a novel iterative algorithm is formulated as the
solver, with rigorous theoretic algorithm analysis provided. (2) To further
align the two domains via the learned projection, shared semantic structure
among seen and unseen classes is explored via forming superclasses in the
semantic space. Extensive experiments show that our model outperforms the
state-of-the-art alternatives by significant margins.
Learning Multi-Layer Transform Models
Authors: Saiprasad Ravishankar, Brendt Wohlberg
Date: 2018-10-19 00:00:00
Tasks: Denoising, Image Denoising

Learned data models based on sparsity are widely used in signal processing
and imaging applications. A variety of methods for learning synthesis
dictionaries, sparsifying transforms, etc., have been proposed in recent years,
often imposing useful structures or properties on the models. In this work, we
focus on sparsifying transform learning, which enjoys a number of advantages.
We consider multi-layer or nested extensions of the transform model, and
propose efficient learning algorithms. Numerical experiments with image data
illustrate the behavior of the multi-layer transform learning algorithm and its
usefulness for image denoising. Multi-layer models provide better denoising
quality than single layer schemes.
Sequenced-Replacement Sampling for Deep Learning
Authors: Chiu Man Ho, Dae Hoon Park, Wei Yang, Yi Chang
Date: 2018-10-19 00:00:00
Proceeding: ICLR 2019 5
Tasks: Deep Learning

We propose sequenced-replacement sampling (SRS) for training deep neural
networks. The basic idea is to assign a fixed sequence index to each sample in
the dataset. Once a mini-batch is randomly drawn in each training iteration, we
refill the original dataset by successively adding samples according to their
sequence index. Thus we carry out replacement sampling but in a batched and
sequenced way. In a sense, SRS could be viewed as a way of performing
"mini-batch augmentation". It is particularly useful for a task where we have a
relatively small images-per-class such as CIFAR-100. Together with a longer
period of initial large learning rate, it significantly improves the
classification accuracy in CIFAR-100 over the current state-of-the-art results.
Our experiments indicate that training deeper networks with SRS is less prone
to over-fitting. In the best case, we achieve an error rate as low as 10.10%.
Heteroskedastic PCA: Algorithm, Optimality, and Applications
Authors: Anru R. Zhang, T. Tony Cai, Yihong Wu
Date: 2018-10-19 00:00:00
Tasks: Denoising

A general framework for principal component analysis (PCA) in the presence of heteroskedastic noise is introduced. We propose an algorithm called HeteroPCA, which involves iteratively imputing the diagonal entries of the sample covariance matrix to remove estimation bias due to heteroskedasticity. This procedure is computationally efficient and provably optimal under the generalized spiked covariance model. A key technical step is a deterministic robust perturbation analysis on singular subspaces, which can be of independent interest. The effectiveness of the proposed algorithm is demonstrated in a suite of problems in high-dimensional statistics, including singular value decomposition (SVD) under heteroskedastic noise, Poisson PCA, and SVD for heteroskedastic and incomplete data.