pwc-archive/papers-with-abstracts huggingface dataset:
Exploring Hyper-Parameter Optimization for Neural Machine Translation on GPU Architectures
Authors: Robert Lim, Kenneth Heafield, Hieu Hoang, Mark Briers, Allen Malony
Date: 2018-05-05 00:00:00
Tasks: GPU, Language Modeling, Language Modelling, Machine Translation, NMT, Translation

Neural machine translation (NMT) has been accelerated by deep learning neural networks over statistical-based approaches, due to the plethora and programmability of commodity heterogeneous computing architectures such as FPGAs and GPUs and the massive amount of training corpuses generated from news outlets, government agencies and social media. Training a learning classifier for neural networks entails tuning hyper-parameters that would yield the best performance. Unfortunately, the number of parameters for machine translation include discrete categories as well as continuous options, which makes for a combinatorial explosive problem. This research explores optimizing hyper-parameters when training deep learning neural networks for machine translation. Specifically, our work investigates training a language model with Marian NMT. Results compare NMT under various hyper-parameter settings across a variety of modern GPU architecture generations in single node and multi-node settings, revealing insights on which hyper-parameters matter most in terms of performance, such as words processed per second, convergence rates, and translation accuracy, and provides insights on how to best achieve high-performing NMT systems.
RiFCN: Recurrent Network in Fully Convolutional Network for Semantic Segmentation of High Resolution Remote Sensing Images
Authors: Lichao Mou, Xiao Xiang Zhu
Date: 2018-05-05 00:00:00
Tasks: Segmentation, Semantic Segmentation

Semantic segmentation in high resolution remote sensing images is a
fundamental and challenging task. Convolutional neural networks (CNNs), such as
fully convolutional network (FCN) and SegNet, have shown outstanding
performance in many segmentation tasks. One key pillar of these successes is
mining useful information from features in convolutional layers for producing
high resolution segmentation maps. For example, FCN nonlinearly combines
high-level features extracted from last convolutional layers; whereas SegNet
utilizes a deconvolutional network which takes as input only coarse, high-level
feature maps of the last convolutional layer. However, how to better fuse
multi-level convolutional feature maps for semantic segmentation of remote
sensing images is underexplored. In this work, we propose a novel bidirectional
network called recurrent network in fully convolutional network (RiFCN), which
is end-to-end trainable. It has a forward stream and a backward stream. The
former is a classification CNN architecture for feature extraction, which takes
an input image and produces multi-level convolutional feature maps from shallow
to deep; while in the later, to achieve accurate boundary inference and
semantic segmentation, boundary-aware high resolution feature maps in shallower
layers and high-level but low-resolution features are recursively embedded into
the learning framework (from deep to shallow) to generate a fused feature
representation that draws a holistic picture of not only high-level semantic
information but also low-level fine-grained details. Experimental results on
two widely-used high resolution remote sensing data sets for semantic
segmentation tasks, ISPRS Potsdam and Inria Aerial Image Labeling Data Set,
demonstrate competitive performance obtained by the proposed methodology
compared to other studied approaches.
A Constraint-Based Algorithm For Causal Discovery with Cycles, Latent Variables and Selection Bias
Authors: Eric V. Strobl
Date: 2018-05-05 00:00:00
Tasks: Causal Discovery, Causal Inference, Selection bias

Causal processes in nature may contain cycles, and real datasets may violate
causal sufficiency as well as contain selection bias. No constraint-based
causal discovery algorithm can currently handle cycles, latent variables and
selection bias (CLS) simultaneously. I therefore introduce an algorithm called
Cyclic Causal Inference (CCI) that makes sound inferences with a conditional
independence oracle under CLS, provided that we can represent the cyclic causal
process as a non-recursive linear structural equation model with independent
errors. Empirical results show that CCI outperforms CCD in the cyclic case as
well as rivals FCI and RFCI in the acyclic case.
Deep Reinforcement Learning for Playing 2.5D Fighting Games
Authors: Yu-Jhe Li, Hsin-Yu Chang, Yu-Jing Lin, Po-Wei Wu, Yu-Chiang Frank Wang
Date: 2018-05-05 00:00:00
Tasks: Deep Reinforcement Learning, OpenAI Gym, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Deep reinforcement learning has shown its success in game playing. However,
2.5D fighting games would be a challenging task to handle due to ambiguity in
visual appearances like height or depth of the characters. Moreover, actions in
such games typically involve particular sequential action orders, which also
makes the network design very difficult. Based on the network of Asynchronous
Advantage Actor-Critic (A3C), we create an OpenAI-gym-like gaming environment
with the game of Little Fighter 2 (LF2), and present a novel A3C+ network for
learning RL agents. The introduced model includes a Recurrent Info network,
which utilizes game-related info features with recurrent layers to observe
combo skills for fighting. In the experiments, we consider LF2 in different
settings, which successfully demonstrates the use of our proposed model for
learning 2.5D fighting games.
Bone marrow cells detection: A technique for the microscopic image analysis
Authors: Haichao Cao, Hong Liu, Enmin Song
Date: 2018-05-05 00:00:00
Tasks: General Classification, Segmentation

In the detection of myeloproliferative, the number of cells in each type of
bone marrow cells (BMC) is an important parameter for the evaluation. In this
study, we propose a new counting method, which also consists of three modules
including localization, segmentation and classification. The localization of
BMC is achieved from a color transformation enhanced BMC sample image and
stepwise averaging method (SAM). In the nucleus segmentation, both SAM and
Otsu's method will be applied to obtain a weighted threshold for segmenting the
patch into nucleus and non-nucleus. In the cytoplasm segmentation, a color
weakening transformation, an improved region growing method and the K-Means
algorithm are used. The connected cells with BMC will be separated by the
marker-controlled watershed algorithm. The features will be extracted for the
classification after the segmentation. In this study, the BMC are classified
using the SVM, Random Forest, Artificial Neural Networks, Adaboost and Bayesian
Networks into five classes including one outlier, namely, neutrophilic split
granulocyte, neutrophilic stab granulocyte, metarubricyte, mature lymphocytes
and the outlier (all other cells not listed). Our experimental results show
that the best average recognition rate is 87.49% for the SVM.
Transfer Learning of Artist Group Factors to Musical Genre Classification
Authors: Jaehun Kim, Minz Won, Xavier Serra, Cynthia C. S. Liem
Date: 2018-05-05 00:00:00
Tasks: Classification, General Classification, Genre classification, Multi-Task Learning, Transfer Learning

The automated recognition of music genres from audio information is a
challenging problem, as genre labels are subjective and noisy. Artist labels
are less subjective and less noisy, while certain artists may relate more
strongly to certain genres. At the same time, at prediction time, it is not
guaranteed that artist labels are available for a given audio segment.
Therefore, in this work, we propose to apply the transfer learning framework,
learning artist-related information which will be used at inference time for
genre classification. We consider different types of artist-related
information, expressed through artist group factors, which will allow for more
efficient learning and stronger robustness to potential label noise.
Furthermore, we investigate how to achieve the highest validation accuracy on
the given FMA dataset, by experimenting with various kinds of transfer methods,
including single-task transfer, multi-task transfer and finally multi-task
learning.
Compositional Representation of Morphologically-Rich Input for Neural Machine Translation
Authors: Duygu Ataman, Marcello Federico
Date: 2018-05-05 00:00:00
Proceeding: ACL 2018 7
Tasks: Machine Translation, NMT, Translation

Neural machine translation (NMT) models are typically trained with fixed-size
input and output vocabularies, which creates an important bottleneck on their
accuracy and generalization capability. As a solution, various studies proposed
segmenting words into sub-word units and performing translation at the
sub-lexical level. However, statistical word segmentation methods have recently
shown to be prone to morphological errors, which can lead to inaccurate
translations. In this paper, we propose to overcome this problem by replacing
the source-language embedding layer of NMT with a bi-directional recurrent
neural network that generates compositional representations of the input at any
desired level of granularity. We test our approach in a low-resource setting
with five languages from different morphological typologies, and under
different composition assumptions. By training NMT to compose word
representations from character n-grams, our approach consistently outperforms
(from 1.71 to 2.48 BLEU points) NMT learning embeddings of statistically
generated sub-word units.
Weakly-supervised Visual Instrument-playing Action Detection in Videos
Authors: Jen-Yu Liu, Yi-Hsuan Yang, Shyh-Kang Jeng
Date: 2018-05-05 00:00:00
Tasks: Action Detection

Instrument playing is among the most common scenes in music-related videos,
which represent nowadays one of the largest sources of online videos. In order
to understand the instrument-playing scenes in the videos, it is important to
know what instruments are played, when they are played, and where the playing
actions occur in the scene. While audio-based recognition of instruments has
been widely studied, the visual aspect of the music instrument playing remains
largely unaddressed in the literature. One of the main obstacles is the
difficulty in collecting annotated data of the action locations for
training-based methods. To address this issue, we propose a weakly-supervised
framework to find when and where the instruments are played in the videos. We
propose to use two auxiliary models, a sound model and an object model, to
provide supervisions for training the instrument-playing action model. The
sound model provides temporal supervisions, while the object model provides
spatial supervisions. They together can simultaneously provide temporal and
spatial supervisions. The resulted model only needs to analyze the visual part
of a music video to deduce which, when and where instruments are played. We
found that the proposed method significantly improves the localization
accuracy. We evaluate the result of the proposed method temporally and
spatially on a small dataset (totally 5,400 frames) that we manually annotated.
Position Estimation of Camera Based on Unsupervised Learning
Authors: YanTong Wu, Yang Liu
Date: 2018-05-05 00:00:00
Tasks: Camera Pose Estimation, Depth Estimation, Pose Estimation, Position, Simultaneous Localization and Mapping

It is an exciting task to recover the scene's 3d-structure and camera pose
from the video sequence. Most of the current solutions divide it into two
parts, monocular depth recovery and camera pose estimation. The monocular depth
recovery is often studied as an independent part, and a better depth estimation
is used to solve the pose. While camera pose is still estimated by traditional
SLAM (Simultaneous Localization And Mapping) methods in most cases. The use of
unsupervised method for monocular depth recovery and pose estimation has
benefited from the study of [1] and achieved good results. In this paper, we
improve the method of [1]. Our emphasis is laid on the improvement of the idea
and related theory, introducing a more reasonable inter frame constraints and
finally synthesize the camera trajectory with inter frame pose estimation in
the unified world coordinate system. And our results get better performance.
A Cost-Sensitive Deep Belief Network for Imbalanced Classification
Authors: Chong Zhang, Kay Chen Tan, Haizhou Li, Geok Soon Hong
Date: 2018-04-28 00:00:00
Tasks: Classification, Fault Diagnosis, General Classification, imbalanced classification

Imbalanced data with a skewed class distribution are common in many
real-world applications. Deep Belief Network (DBN) is a machine learning
technique that is effective in classification tasks. However, conventional DBN
does not work well for imbalanced data classification because it assumes equal
costs for each class. To deal with this problem, cost-sensitive approaches
assign different misclassification costs for different classes without
disrupting the true data sample distributions. However, due to lack of prior
knowledge, the misclassification costs are usually unknown and hard to choose
in practice. Moreover, it has not been well studied as to how cost-sensitive
learning could improve DBN performance on imbalanced data problems. This paper
proposes an evolutionary cost-sensitive deep belief network (ECS-DBN) for
imbalanced classification. ECS-DBN uses adaptive differential evolution to
optimize the misclassification costs based on training data, that presents an
effective approach to incorporating the evaluation measure (i.e. G-mean) into
the objective function. We first optimize the misclassification costs, then
apply them to deep belief network. Adaptive differential evolution optimization
is implemented as the optimization algorithm that automatically updates its
corresponding parameters without the need of prior domain knowledge. The
experiments have shown that the proposed approach consistently outperforms the
state-of-the-art on both benchmark datasets and real-world dataset for fault
diagnosis in tool condition monitoring.
Various Approaches to Aspect-based Sentiment Analysis
Authors: Amlaan Bhoi, Sandeep Joshi
Date: 2018-05-05 00:00:00
Tasks: Aspect-Based Sentiment Analysis, Aspect-Based Sentiment Analysis (ABSA), General Classification, Sentence, Sentiment Analysis, Sentiment Classification

The problem of aspect-based sentiment analysis deals with classifying
sentiments (negative, neutral, positive) for a given aspect in a sentence. A
traditional sentiment classification task involves treating the entire sentence
as a text document and classifying sentiments based on all the words. Let us
assume, we have a sentence such as "the acceleration of this car is fast, but
the reliability is horrible". This can be a difficult sentence because it has
two aspects with conflicting sentiments about the same entity. Considering
machine learning techniques (or deep learning), how do we encode the
information that we are interested in one aspect and its sentiment but not the
other? Let us explore various pre-processing steps, features, and methods used
to facilitate in solving this task.
A Survey of Machine Learning for Big Code and Naturalness
Authors: Miltiadis Allamanis, Earl T. Barr, Premkumar Devanbu, Charles Sutton
Date: 2017-09-18 00:00:00
Tasks: BIG-bench Machine Learning, Navigate, Survey

Research at the intersection of machine learning, programming languages, and
software engineering has recently taken important steps in proposing learnable
probabilistic models of source code that exploit code's abundance of patterns.
In this article, we survey this work. We contrast programming languages against
natural languages and discuss how these similarities and differences drive the
design of probabilistic models. We present a taxonomy based on the underlying
design principles of each model and use it to navigate the literature. Then, we
review how researchers have adapted these models to application areas and
discuss cross-cutting and application-specific challenges and opportunities.
Iterative proportional scaling revisited: a modern optimization perspective
Authors: Yiyuan She, Shao Tang
Date: 2016-10-08 00:00:00
Tasks: feature selection

This paper revisits the classic iterative proportional scaling (IPS) from a
modern optimization perspective. In contrast to the criticisms made in the
literature, we show that based on a coordinate descent characterization, IPS
can be slightly modified to deliver coefficient estimates, and from a
majorization-minimization standpoint, IPS can be extended to handle log-affine
models with features not necessarily binary-valued or nonnegative. Furthermore,
some state-of-the-art optimization techniques such as block-wise computation,
randomization and momentum-based acceleration can be employed to provide more
scalable IPS algorithms, as well as some regularized variants of IPS for
concurrent feature selection.
Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination
Authors: Zhirong Wu, Yuanjun Xiong, Stella Yu, Dahua Lin
Date: 2018-05-05 00:00:00
Tasks: Contrastive Learning, General Classification, object-detection, Object Detection, Retrieval

Neural net classifiers trained on data with annotated class labels can also
capture apparent visual similarity among categories without being directed to
do so. We study whether this observation can be extended beyond the
conventional domain of supervised learning: Can we learn a good feature
representation that captures apparent similarity among instances, instead of
classes, by merely asking the feature to be discriminative of individual
instances? We formulate this intuition as a non-parametric classification
problem at the instance-level, and use noise-contrastive estimation to tackle
the computational challenges imposed by the large number of instance classes.
Our experimental results demonstrate that, under unsupervised learning
settings, our method surpasses the state-of-the-art on ImageNet classification
by a large margin. Our method is also remarkable for consistently improving
test performance with more training data and better network architectures. By
fine-tuning the learned feature, we further obtain competitive results for
semi-supervised learning and object detection tasks. Our non-parametric model
is highly compact: With 128 features per image, our method requires only 600MB
storage for a million images, enabling fast nearest neighbour retrieval at the
run time.
SdcNet: A Computation-Efficient CNN for Object Recognition
Authors: Yunlong Ma, Chunyan Wang
Date: 2018-05-03 00:00:00
Tasks: Management, Object, Object Recognition

Extracting features from a huge amount of data for object recognition is a
challenging task. Convolution neural network can be used to meet the challenge,
but it often requires a large number of computation resources. In this paper, a
computation-efficient convolutional module, named SdcBlock, is proposed and
based on it, the convolution network SdcNet is introduced for object
recognition tasks. In the proposed module, optimized successive depthwise
convolutions supported by appropriate data management is applied in order to
generate vectors containing high density and more varieties of feature
information. The hyperparameters can be easily adjusted to suit varieties of
tasks under different computation restrictions without significantly
jeopardizing the performance. The experiments have shown that SdcNet achieved
an error rate of 5.60% in CIFAR-10 with only 55M Flops and also reduced further
the error rate to 5.24% using a moderate volume of 103M Flops. The expected
computation efficiency of the SdcNet has been confirmed.
Fast-converging Conditional Generative Adversarial Networks for Image Synthesis
Authors: Chengcheng Li, Zi Wang, Hairong Qi
Date: 2018-05-05 00:00:00
Tasks: Image Generation

Building on top of the success of generative adversarial networks (GANs),
conditional GANs attempt to better direct the data generation process by
conditioning with certain additional information. Inspired by the most recent
AC-GAN, in this paper we propose a fast-converging conditional GAN (FC-GAN). In
addition to the real/fake classifier used in vanilla GANs, our discriminator
has an advanced auxiliary classifier which distinguishes each real class from
an extra `fake' class. The `fake' class avoids mixing generated data with real
data, which can potentially confuse the classification of real data as AC-GAN
does, and makes the advanced auxiliary classifier behave as another real/fake
classifier. As a result, FC-GAN can accelerate the process of differentiation
of all classes, thus boost the convergence speed. Experimental results on image
synthesis demonstrate our model is competitive in the quality of images
generated while achieving a faster convergence rate.
Socially Aware Motion Planning with Deep Reinforcement Learning
Authors: Yu Fan Chen, Michael Everett, Miao Liu, Jonathan P. How
Date: 2017-03-26 00:00:00
Tasks: Autonomous Navigation, Deep Reinforcement Learning, Motion Planning, Navigate, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

For robotic vehicles to navigate safely and efficiently in pedestrian-rich
environments, it is important to model subtle human behaviors and navigation
rules (e.g., passing on the right). However, while instinctive to humans,
socially compliant navigation is still difficult to quantify due to the
stochasticity in people's behaviors. Existing works are mostly focused on using
feature-matching techniques to describe and imitate human paths, but often do
not generalize well since the feature values can vary from person to person,
and even run to run. This work notes that while it is challenging to directly
specify the details of what to do (precise mechanisms of human navigation), it
is straightforward to specify what not to do (violations of social norms).
Specifically, using deep reinforcement learning, this work develops a
time-efficient navigation policy that respects common social norms. The
proposed method is shown to enable fully autonomous navigation of a robotic
vehicle moving at human walking speed in an environment with many pedestrians.
Contextual Outlier Interpretation
Authors: Ninghao Liu, Donghwa Shin, Xia Hu
Date: 2017-11-28 00:00:00
Tasks: feature selection, Outlier Detection, Outlier Interpretation

Outlier detection plays an essential role in many data-driven applications to
identify isolated instances that are different from the majority. While many
statistical learning and data mining techniques have been used for developing
more effective outlier detection algorithms, the interpretation of detected
outliers does not receive much attention. Interpretation is becoming
increasingly important to help people trust and evaluate the developed models
through providing intrinsic reasons why the certain outliers are chosen. It is
difficult, if not impossible, to simply apply feature selection for explaining
outliers due to the distinct characteristics of various detection models,
complicated structures of data in certain applications, and imbalanced
distribution of outliers and normal instances. In addition, the role of
contrastive contexts where outliers locate, as well as the relation between
outliers and contexts, are usually overlooked in interpretation. To tackle the
issues above, in this paper, we propose a novel Contextual Outlier
INterpretation (COIN) method to explain the abnormality of existing outliers
spotted by detectors. The interpretability for an outlier is achieved from
three aspects: outlierness score, attributes that contribute to the
abnormality, and contextual description of its neighborhoods. Experimental
results on various types of datasets demonstrate the flexibility and
effectiveness of the proposed framework compared with existing interpretation
approaches.