pwc-archive/papers-with-abstracts huggingface dataset:
Max-plus Operators Applied to Filter Selection and Model Pruning in Neural Networks
Authors: Yunxiang Zhang, Samy Blusseau, Santiago Velasco-Forero, Isabelle Bloch, Jesus Angulo
Date: 2019-03-19 00:00:00

Following recent advances in morphological neural networks, we propose to
study in more depth how Max-plus operators can be exploited to define
morphological units and how they behave when incorporated in layers of
conventional neural networks. Besides showing that they can be easily
implemented with modern machine learning frameworks , we confirm and extend the
observation that a Max-plus layer can be used to select important filters and
reduce redundancy in its previous layer, without incurring performance loss.
Experimental results demonstrate that the filter selection strategy enabled by
a Max-plus is highly efficient and robust, through which we successfully
performed model pruning on different neural network architectures. We also
point out that there is a close connection between Maxout networks and our
pruned Max-plus networks by comparing their respective characteristics. The
code for reproducing our experiments is available online.
Trained Quantization Thresholds for Accurate and Efficient Fixed-Point Inference of Deep Neural Networks
Authors: Sambhav R. Jain, Albert Gural, Michael Wu, Chris H. Dick
Date: 2019-03-19 00:00:00
Tasks: Quantization

We propose a method of training quantization thresholds (TQT) for uniform symmetric quantizers using standard backpropagation and gradient descent. Contrary to prior work, we show that a careful analysis of the straight-through estimator for threshold gradients allows for a natural range-precision trade-off leading to better optima. Our quantizers are constrained to use power-of-2 scale-factors and per-tensor scaling of weights and activations to make it amenable for hardware implementations. We present analytical support for the general robustness of our methods and empirically validate them on various CNNs for ImageNet classification. We are able to achieve near-floating-point accuracy on traditionally difficult networks such as MobileNets with less than 5 epochs of quantized (8-bit) retraining. Finally, we present Graffitist, a framework that enables automatic quantization of TensorFlow graphs for TQT (available at https://github.com/Xilinx/graffitist ).
Identity-Free Facial Expression Recognition using conditional Generative Adversarial Network
Authors: Jie Cai, Zibo Meng, Ahmed Shehab Khan, Zhiyuan Li, James O'Reilly, Shizhong Han, Yan Tong
Date: 2019-03-19 00:00:00
Tasks: Facial Expression Recognition, Facial Expression Recognition (FER), Generative Adversarial Network

A novel Identity-Free conditional Generative Adversarial Network (IF-GAN) was proposed for Facial Expression Recognition (FER) to explicitly reduce high inter-subject variations caused by identity-related facial attributes, e.g., age, race, and gender. As part of an end-to-end system, a cGAN was designed to transform a given input facial expression image to an "average" identity face with the same expression as the input. Then, identity-free FER is possible since the generated images have the same synthetic "average" identity and differ only in their displayed expressions. Experiments on four facial expression datasets, one with spontaneous expressions, show that IF-GAN outperforms the baseline CNN and achieves state-of-the-art performance for FER.
IndyLSTMs: Independently Recurrent LSTMs
Authors: Pedro Gonnet, Thomas Deselaers
Date: 2019-03-19 00:00:00
Tasks: Attribute

We introduce Independently Recurrent Long Short-term Memory cells: IndyLSTMs.
These differ from regular LSTM cells in that the recurrent weights are not
modeled as a full matrix, but as a diagonal matrix, i.e.\ the output and state
of each LSTM cell depends on the inputs and its own output/state, as opposed to
the input and the outputs/states of all the cells in the layer. The number of
parameters per IndyLSTM layer, and thus the number of FLOPS per evaluation, is
linear in the number of nodes in the layer, as opposed to quadratic for regular
LSTM layers, resulting in potentially both smaller and faster models. We
evaluate their performance experimentally by training several models on the
popular \iamondb and CASIA online handwriting datasets, as well as on several
of our in-house datasets. We show that IndyLSTMs, despite their smaller size,
consistently outperform regular LSTMs both in terms of accuracy per parameter,
and in best accuracy overall. We attribute this improved performance to the
IndyLSTMs being less prone to overfitting.
Ensemble Clustering for Graphs: Comparisons and Applications
Authors: Valérie Poulin, François Théberge
Date: 2019-03-19 00:00:00
Tasks: Anomaly Detection, Clustering, Community Detection, Graph Clustering

We recently proposed a new ensemble clustering algorithm for graphs (ECG)
based on the concept of consensus clustering. We validated our approach by
replicating a study comparing graph clustering algorithms over benchmark
graphs, showing that ECG outperforms the leading algorithms. In this paper, we
extend our comparison by considering a wider range of parameters for the
benchmark, generating graphs with different properties. We provide new
experimental results showing that the ECG algorithm alleviates the well-known
resolution limit issue, and that it leads to better stability of the
partitions. We also illustrate how the ensemble obtained with ECG can be used
to quantify the presence of community structure in the graph, and to zoom in on
the sub-graph most closely associated with seed vertices. Finally, we
illustrate further applications of ECG by comparing it to previous results for
community detection on weighted graphs, and community-aware anomaly detection.
Data-driven PDE discovery with evolutionary approach
Authors: Michail Maslyaev, Alexander Hvatov, Anna Kalyuzhnaya
Date: 2019-03-19 00:00:00
Tasks: Equation Discovery, Form, regression, Symbolic Regression

The data-driven models allow one to define the model structure in cases when
a priori information is not sufficient to build other types of models. The
possible way to obtain physical interpretation is the data-driven differential
equation discovery techniques. The existing methods of PDE (partial derivative
equations) discovery are bound with the sparse regression. However, sparse
regression is restricting the resulting model form, since the terms for PDE are
defined before regression. The evolutionary approach described in the article
has a symbolic regression as the background instead and thus has fewer
restrictions on the PDE form. The evolutionary method of PDE discovery (EPDE)
is described and tested on several canonical PDEs. The question of robustness
is examined on a noised data example.
An Evaluation of Bitcoin Address Classification based on Transaction History Summarization
Authors: Yu-Jing Lin, Po-Wei Wu, Cheng-Han Hsu, I-Ping Tu, Shih-wei Liao
Date: 2019-03-19 00:00:00
Tasks: General Classification

Bitcoin is a cryptocurrency that features a distributed, decentralized and
trustworthy mechanism, which has made Bitcoin a popular global transaction
platform. The transaction efficiency among nations and the privacy benefiting
from address anonymity of the Bitcoin network have attracted many activities
such as payments, investments, gambling, and even money laundering in the past
decade. Unfortunately, some criminal behaviors which took advantage of this
platform were not identified. This has discouraged many governments to support
cryptocurrency. Thus, the capability to identify criminal addresses becomes an
important issue in the cryptocurrency network. In this paper, we propose new
features in addition to those commonly used in the literature to build a
classification model for detecting abnormality of Bitcoin network addresses.
These features include various high orders of moments of transaction time
(represented by block height) which summarizes the transaction history in an
efficient way. The extracted features are trained by supervised machine
learning methods on a labeling category data set. The experimental evaluation
shows that these features have improved the performance of Bitcoin address
classification significantly. We evaluate the results under eight classifiers
and achieve the highest Micro-F1/Macro-F1 of 87%/86% with LightGBM.
Efficient Smoothing of Dilated Convolutions for Image Segmentation
Authors: Thomas Ziegler, Manuel Fritsche, Lorenz Kuhn, Konstantin Donhauser
Date: 2019-03-19 00:00:00
Tasks: Image Segmentation, Segmentation, Semantic Segmentation

Dilated Convolutions have been shown to be highly useful for the task of
image segmentation. By introducing gaps into convolutional filters, they enable
the use of larger receptive fields without increasing the original kernel size.
Even though this allows for the inexpensive capturing of features at different
scales, the structure of the dilated convolutional filter leads to a loss of
information. We hypothesise that inexpensive modifications to Dilated
Convolutional Neural Networks, such as additional averaging layers, could
overcome this limitation. In this project we test this hypothesis by evaluating
the effect of these modifications for a state-of-the art image segmentation
system and compare them to existing approaches with the same objective. Our
experiments show that our proposed methods improve the performance of dilated
convolutions for image segmentation. Crucially, our modifications achieve these
results at a much lower computational cost than previous smoothing approaches.
Deep Eikonal Solvers
Authors: Moshe Lichtenstein, Gautam Pai, Ron Kimmel
Date: 2019-03-19 00:00:00

A deep learning approach to numerically approximate the solution to the
Eikonal equation is introduced. The proposed method is built on the fast
marching scheme which comprises of two components: a local numerical solver and
an update scheme. We replace the formulaic local numerical solver with a
trained neural network to provide highly accurate estimates of local distances
for a variety of different geometries and sampling conditions. Our learning
approach generalizes not only to flat Euclidean domains but also to curved
surfaces enabled by the incorporation of certain invariant features in the
neural network architecture. We show a considerable gain in performance,
validated by smaller errors and higher orders of accuracy for the numerical
solutions of the Eikonal equation computed on different surfaces The proposed
approach leverages the approximation power of neural networks to enhance the
performance of numerical algorithms, thereby, connecting the somewhat disparate
themes of numerical geometry and learning.
Convergence Analysis of Inexact Randomized Iterative Methods
Authors: Nicolas Loizou, Peter Richtárik
Date: 2019-03-19 00:00:00

In this paper we present a convergence rate analysis of inexact variants of
several randomized iterative methods. Among the methods studied are: stochastic
gradient descent, stochastic Newton, stochastic proximal point and stochastic
subspace ascent. A common feature of these methods is that in their update rule
a certain sub-problem needs to be solved exactly. We relax this requirement by
allowing for the sub-problem to be solved inexactly. In particular, we propose
and analyze inexact randomized iterative methods for solving three closely
related problems: a convex stochastic quadratic optimization problem, a best
approximation problem and its dual, a concave quadratic maximization problem.
We provide iteration complexity results under several assumptions on the
inexactness error. Inexact variants of many popular and some more exotic
methods, including randomized block Kaczmarz, randomized Gaussian Kaczmarz and
randomized block coordinate descent, can be cast as special cases. Numerical
experiments demonstrate the benefits of allowing inexactness.
A Choquet Fuzzy Integral Vertical Bagging Classifier for Mobile Telematics Data Analysis
Authors: Mohammad Siami, Mohsen Naderpour, Jie Lu
Date: 2019-03-19 00:00:00

Mobile app development in recent years has resulted in new products and
features to improve human life. Mobile telematics is one such development that
encompasses multidisciplinary fields for transportation safety. The application
of mobile telematics has been explored in many areas, such as insurance and
road safety. However, to the best of our knowledge, its application in gender
detection has not been explored. This paper proposes a Choquet fuzzy integral
vertical bagging classifier that detects gender through mobile telematics. In
this model, different random forest classifiers are trained by randomly
generated features with rough set theory, and the top three classifiers are
fused using the Choquet fuzzy integral. The model is implemented and evaluated
on a real dataset. The empirical results indicate that the Choquet fuzzy
integral vertical bagging classifier outperforms other classifiers.
A Matrix-in-matrix Neural Network for Image Super Resolution
Authors: Hailong Ma, Xiangxiang Chu, Shaohua Wan, Bo Zhang
Date: 2019-03-19 00:00:00
Tasks: Image Super-Resolution, Super-Resolution

In recent years, deep learning methods have achieved impressive results with
higher peak signal-to-noise ratio in single image super-resolution (SISR) tasks
by utilizing deeper layers. However, their application is quite limited since
they require high computing power. In addition, most of the existing methods
rarely take full advantage of the intermediate features which are helpful for
restoration. To address these issues, we propose a moderate-size SISR net work
named matrixed channel attention network (MCAN) by constructing a matrix
ensemble of multi-connected channel attention blocks (MCAB). Several models of
different sizes are released to meet various practical requirements.
Conclusions can be drawn from our extensive benchmark experiments that the
proposed models achieve better performance with much fewer multiply-adds and
parameters. Our models will be made publicly available.
Truly Proximal Policy Optimization
Authors: Yuhui Wang, Hao He, Chao Wen, Xiaoyang Tan
Date: 2019-03-19 00:00:00
Tasks: Deep Reinforcement Learning, Reinforcement Learning

Proximal policy optimization (PPO) is one of the most successful deep reinforcement-learning methods, achieving state-of-the-art performance across a wide range of challenging tasks. However, its optimization behavior is still far from being fully understood. In this paper, we show that PPO could neither strictly restrict the likelihood ratio as it attempts to do nor enforce a well-defined trust region constraint, which means that it may still suffer from the risk of performance instability. To address this issue, we present an enhanced PPO method, named Truly PPO. Two critical improvements are made in our method: 1) it adopts a new clipping function to support a rollback behavior to restrict the difference between the new policy and the old one; 2) the triggering condition for clipping is replaced with a trust region-based one, such that optimizing the resulted surrogate objective function provides guaranteed monotonic improvement of the ultimate policy performance. It seems, by adhering more truly to making the algorithm proximal - confining the policy within the trust region, the new algorithm improves the original PPO on both sample efficiency and performance.
What the Constant Velocity Model Can Teach Us About Pedestrian Motion Prediction
Authors: Christoph Schöller, Vincent Aravantinos, Florian Lay, Alois Knoll
Date: 2019-03-19 00:00:00
Tasks: motion prediction, Prediction

Pedestrian motion prediction is a fundamental task for autonomous robots and vehicles to operate safely. In recent years many complex approaches based on neural networks have been proposed to address this problem. In this work we show that - surprisingly - a simple Constant Velocity Model can outperform even state-of-the-art neural models. This indicates that either neural networks are not able to make use of the additional information they are provided with, or that this information is not as relevant as commonly believed. Therefore, we analyze how neural networks process their input and how it impacts their predictions. Our analysis reveals pitfalls in training neural networks for pedestrian motion prediction and clarifies false assumptions about the problem itself. In particular, neural networks implicitly learn environmental priors that negatively impact their generalization capability, the motion history of pedestrians is irrelevant and interactions are too complex to predict. Our work shows how neural networks for pedestrian motion prediction can be thoroughly evaluated and our results indicate which research directions for neural motion prediction are promising in future.
compare-mt: A Tool for Holistic Comparison of Language Generation Systems
Authors: Graham Neubig, Zi-Yi Dou, Junjie Hu, Paul Michel, Danish Pruthi, Xinyi Wang, John Wieting
Date: 2019-03-19 00:00:00
Proceeding: NAACL 2019 6
Tasks: Machine Translation, Sentence, Text Generation, Translation

In this paper, we describe compare-mt, a tool for holistic analysis and comparison of the results of systems for language generation tasks such as machine translation. The main goal of the tool is to give the user a high-level and coherent view of the salient differences between systems that can then be used to guide further analysis or system improvement. It implements a number of tools to do so, such as analysis of accuracy of generation of particular types of words, bucketed histograms of sentence accuracies or counts based on salient characteristics, and extraction of characteristic $n$-grams for each system. It also has a number of advanced features such as use of linguistic labels, source side data, or comparison of log likelihoods for probabilistic models, and also aims to be easily extensible by users to new types of analysis. The code is available at https://github.com/neulab/compare-mt
CVIT-MT Systems for WAT-2018
Authors: Jerin Philip, Vinay P. Namboodiri, C. V. Jawahar
Date: 2019-03-19 00:00:00
Proceeding: PACLIC 2018 12
Tasks: Machine Translation, Translation

This document describes the machine translation system used in the
submissions of IIIT-Hyderabad CVIT-MT for the WAT-2018 English-Hindi
translation task. Performance is evaluated on the associated corpus provided by
the organizers. We experimented with convolutional sequence to sequence
architectures. We also train with additional data obtained through
backtranslation.
Geometry-constrained Car Recognition Using a 3D Perspective Network
Authors: Rui Zeng, ZongYuan Ge, Simon Denman, Sridha Sridharan, Clinton Fookes
Date: 2019-03-19 00:00:00

We present a novel learning framework for vehicle recognition from a single RGB image. Unlike existing methods which only use attention mechanisms to locate 2D discriminative information, our work learns a novel 3D perspective feature representation of a vehicle, which is then fused with 2D appearance feature to predict the category. The framework is composed of a global network (GN), a 3D perspective network (3DPN), and a fusion network. The GN is used to locate the region of interest (RoI) and generate the 2D global feature. With the assistance of the RoI, the 3DPN estimates the 3D bounding box under the guidance of the proposed vanishing point loss, which provides a perspective geometry constraint. Then the proposed 3D representation is generated by eliminating the viewpoint variance of the 3D bounding box using perspective transformation. Finally, the 3D and 2D feature are fused to predict the category of the vehicle. We present qualitative and quantitative results on the vehicle classification and verification tasks in the BoxCars dataset. The results demonstrate that, by learning such a concise 3D representation, we can achieve superior performance to methods that only use 2D information while retain 3D meaningful information without the challenge of requiring a 3D CAD model.
Improving Video Compression With Deep Visual-Attention Models
Authors: Vitaliy Lyudvichenko, Mikhail Erofeev, Alexander Ploshkin, Dmitriy Vatolin
Date: 2019-03-19 00:00:00
Tasks: Video Compression

Recent advances in deep learning have markedly improved the quality of
visual-attention modelling. In this work we apply these advances to video
compression.
  We propose a compression method that uses a saliency model to adaptively
compress frame areas in accordance with their predicted saliency. We selected
three state-of-the-art saliency models, adapted them for video compression and
analyzed their results. The analysis includes objective evaluation of the
models as well as objective and subjective evaluation of the compressed videos.
  Our method, which is based on the x264 video codec, can produce videos with
the same visual quality as regular x264, but it reduces the bitrate by 25%
according to the objective evaluation and by 17% according to the subjective
one. Also, both the subjective and objective evaluations demonstrate that
saliency models can compete with gaze maps for a single observer.
  Our method can extend to most video bitstream formats and can improve video
compression quality without requiring a switch to a new video encoding
standard.
NeuralHydrology -- Interpreting LSTMs in Hydrology
Authors: Frederik Kratzert, Mathew Herrnegger, Daniel Klotz, Sepp Hochreiter, Günter Klambauer
Date: 2019-03-19 00:00:00

Despite the huge success of Long Short-Term Memory networks, their applications in environmental sciences are scarce. We argue that one reason is the difficulty to interpret the internals of trained networks. In this study, we look at the application of LSTMs for rainfall-runoff forecasting, one of the central tasks in the field of hydrology, in which the river discharge has to be predicted from meteorological observations. LSTMs are particularly well-suited for this problem since memory cells can represent dynamic reservoirs and storages, which are essential components in state-space modelling approaches of the hydrological system. On basis of two different catchments, one with snow influence and one without, we demonstrate how the trained model can be analyzed and interpreted. In the process, we show that the network internally learns to represent patterns that are consistent with our qualitative understanding of the hydrological system.
A Comparative Study for Unsupervised Network Representation Learning
Authors: Megha Khosla, Vinay Setty, Avishek Anand
Date: 2019-03-19 00:00:00
Tasks: Experimental Design, Link Prediction, Node Classification, Representation Learning

There has been appreciable progress in unsupervised network representation learning (UNRL) approaches over graphs recently with flexible random-walk approaches, new optimization objectives and deep architectures. However, there is no common ground for systematic comparison of embeddings to understand their behavior for different graphs and tasks. In this paper we theoretically group different approaches under a unifying framework and empirically investigate the effectiveness of different network representation methods. In particular, we argue that most of the UNRL approaches either explicitly or implicit model and exploit context information of a node. Consequently, we propose a framework that casts a variety of approaches -- random walk based, matrix factorization and deep learning based -- into a unified context-based optimization function. We systematically group the methods based on their similarities and differences. We study the differences among these methods in detail which we later use to explain their performance differences (on downstream tasks). We conduct a large-scale empirical study considering 9 popular and recent UNRL techniques and 11 real-world datasets with varying structural properties and two common tasks -- node classification and link prediction. We find that there is no single method that is a clear winner and that the choice of a suitable method is dictated by certain properties of the embedding methods, task and structural properties of the underlying graph. In addition we also report the common pitfalls in evaluation of UNRL methods and come up with suggestions for experimental design and interpretation of results.