pwc-archive/papers-with-abstracts huggingface dataset:
Multi Objective Particle Swarm Optimization based Cooperative Agents with Automated Negotiation
Authors: Najwa Kouka, Raja Fdhila, Adel M. ALIMI
Date: 2019-01-27 00:00:00
Tasks: Diversity, Management

This paper investigates a new hybridization of multi-objective particle swarm
optimization (MOPSO) and cooperative agents (MOPSO-CA) to handle the problem of
stagnation encounters in MOPSO, which leads solutions to trap in local optima.
The proposed approach involves a new distribution strategy based on the idea of
having a set of a sub-population, each of which is processed by one agent. The
number of the sub-population and agents are adjusted dynamically through the
Pareto ranking. This method allocates a dynamic number of sub-population as
required to improve diversity in the search space. Additionally, agents are
used for better management for the exploitation within a sub-population, and
for exploration among sub-populations. Furthermore, we investigate the
automated negotiation within agents in order to share the best knowledge. To
validate our approach, several benchmarks are performed. The results show that
the introduced variant ensures the trade-off between the exploitation and
exploration with respect to the comparative algorithms
PruneTrain: Fast Neural Network Training by Dynamic Sparse Model Reconfiguration
Authors: Sangkug Lym, Esha Choukse, Siavash Zangeneh, Wei Wen, Sujay Sanghavi, Mattan Erez
Date: 2019-01-26 00:00:00
Tasks: GPU

State-of-the-art convolutional neural networks (CNNs) used in vision applications have large models with numerous weights. Training these models is very compute- and memory-resource intensive. Much research has been done on pruning or compressing these models to reduce the cost of inference, but little work has addressed the costs of training. We focus precisely on accelerating training. We propose PruneTrain, a cost-efficient mechanism that gradually reduces the training cost during training. PruneTrain uses a structured group-lasso regularization approach that drives the training optimization toward both high accuracy and small weight values. Small weights can then be periodically removed by reconfiguring the network model to a smaller one. By using a structured-pruning approach and additional reconfiguration techniques we introduce, the pruned model can still be efficiently processed on a GPU accelerator. Overall, PruneTrain achieves a reduction of 39% in the end-to-end training time of ResNet50 for ImageNet by reducing computation cost by 40% in FLOPs, memory accesses by 37% for memory bandwidth bound layers, and the inter-accelerator communication by 55%.
Real-time Video Summarization on Commodity Hardware
Authors: Wesley Taylor, Faisal Z. Qureshi
Date: 2019-01-26 00:00:00
Tasks: Video Summarization

We present a method for creating video summaries in real-time on commodity
hardware. Real-time here refers to the fact that the time required for video
summarization is less than the duration of the input video. First, low-level
features are use to discard undesirable frames. Next, video is divided into
segments, and segment-level features are extracted for each segment. Tree-based
models trained on widely available video summarization and computational
aesthetics datasets are then used to rank individual segments, and top-ranked
segments are selected to generate the final video summary. We evaluate the
proposed method on SUMME dataset and show that our method is able to achieve
summarization accuracy that is comparable to that of a current state-of-the-art
deep learning method, while posting significantly faster run-times. Our method
on average is able to generate a video summary in time that is shorter than the
duration of the video.
Money on the Table: Statistical information ignored by Softmax can improve classifier accuracy
Authors: Charles B. Delahunt, Courosh Mehanian, J. Nathan Kutz
Date: 2019-01-26 00:00:00

Softmax is a standard final layer used in Neural Nets (NNs) to summarize information encoded in the trained NN and return a prediction. However, Softmax leverages only a subset of the class-specific structure encoded in the trained model and ignores potentially valuable information: During training, models encode an array $D$ of class response distributions, where $D_{ij}$ is the distribution of the $j^{th}$ pre-Softmax readout neuron's responses to the $i^{th}$ class. Given a test sample, Softmax implicitly uses only the row of this array $D$ that corresponds to the readout neurons' responses to the sample's true class. Leveraging more of this array $D$ can improve classifier accuracy, because the likelihoods of two competing classes can be encoded in other rows of $D$. To explore this potential resource, we develop a hybrid classifier (Softmax-Pooling Hybrid, $SPH$) that uses Softmax on high-scoring samples, but on low-scoring samples uses a log-likelihood method that pools the information from the full array $D$. We apply $SPH$ to models trained on a vectorized MNIST dataset to varying levels of accuracy. $SPH$ replaces only the final Softmax layer in the trained NN, at test time only. All training is the same as for Softmax. Because the pooling classifier performs better than Softmax on low-scoring samples, $SPH$ reduces test set error by 6% to 23%, using the exact same trained model, whatever the baseline Softmax accuracy. This reduction in error reflects hidden capacity of the trained NN that is left unused by Softmax.
Points2Pix: 3D Point-Cloud to Image Translation using conditional Generative Adversarial Networks
Authors: Stefan Milz, Martin Simon, Kai Fischer, Maximillian Pöpperl
Date: 2019-01-26 00:00:00
Tasks: object-detection, Object Detection, Translation

We present the first approach for 3D point-cloud to image translation based on conditional Generative Adversarial Networks (cGAN). The model handles multi-modal information sources from different domains, i.e. raw point-sets and images. The generator is capable of processing three conditions, whereas the point-cloud is encoded as raw point-set and camera projection. An image background patch is used as constraint to bias environmental texturing. A global approximation function within the generator is directly applied on the point-cloud (Point-Net). Hence, the representative learning model incorporates global 3D characteristics directly at the latent feature space. Conditions are used to bias the background and the viewpoint of the generated image. This opens up new ways in augmenting or texturing 3D data to aim the generation of fully individual images. We successfully evaluated our method on the Kitti and SunRGBD dataset with an outstanding object detection inception score.
Towards Practical Lipschitz Bandits
Authors: Tianyu Wang, Weicheng Ye, Dawei Geng, Cynthia Rudin
Date: 2019-01-26 00:00:00
Tasks: Gaussian Processes

Stochastic Lipschitz bandit algorithms balance exploration and exploitation, and have been used for a variety of important task domains. In this paper, we present a framework for Lipschitz bandit methods that adaptively learns partitions of context- and arm-space. Due to this flexibility, the algorithm is able to efficiently optimize rewards and minimize regret, by focusing on the portions of the space that are most relevant. In our analysis, we link tree-based methods to Gaussian processes. In light of our analysis, we design a novel hierarchical Bayesian model for Lipschitz bandit problems. Our experiments show that our algorithms can achieve state-of-the-art performance in challenging real-world tasks such as neural network hyperparameter tuning.
Challenges in Designing Datasets and Validation for Autonomous Driving
Authors: Michal Uricar, David Hurych, Pavel Krizek, Senthil Yogamani
Date: 2019-01-26 00:00:00
Tasks: Autonomous Driving

Autonomous driving is getting a lot of attention in the last decade and will
be the hot topic at least until the first successful certification of a car
with Level 5 autonomy. There are many public datasets in the academic
community. However, they are far away from what a robust industrial production
system needs. There is a large gap between academic and industrial setting and
a substantial way from a research prototype, built on public datasets, to a
deployable solution which is a challenging task. In this paper, we focus on bad
practices that often happen in the autonomous driving from an industrial
deployment perspective. Data design deserves at least the same amount of
attention as the model design. There is very little attention paid to these
issues in the scientific community, and we hope this paper encourages better
formalization of dataset design. More specifically, we focus on the datasets
design and validation scheme for autonomous driving, where we would like to
highlight the common problems, wrong assumptions, and steps towards avoiding
them, as well as some open problems.
Distributed Learning with Compressed Gradient Differences
Authors: Konstantin Mishchenko, Eduard Gorbunov, Martin Takáč, Peter Richtárik
Date: 2019-01-26 00:00:00
Tasks: Distributed Computing, Quantization

Training large machine learning models requires a distributed computing approach, with communication of the model updates being the bottleneck. For this reason, several methods based on the compression (e.g., sparsification and/or quantization) of updates were recently proposed, including QSGD (Alistarh et al., 2017), TernGrad (Wen et al., 2017), SignSGD (Bernstein et al., 2018), and DQGD (Khirirat et al., 2018). However, none of these methods are able to learn the gradients, which renders them incapable of converging to the true optimum in the batch mode. In this work we propose a new distributed learning method -- DIANA -- which resolves this issue via compression of gradient differences. We perform a theoretical analysis in the strongly convex and nonconvex settings and show that our rates are superior to existing rates. We also provide theory to support non-smooth regularizers study the difference between quantization schemes. Our analysis of block-quantization and differences between $\ell_2$ and $\ell_{\infty}$ quantization closes the gaps in theory and practice. Finally, by applying our analysis technique to TernGrad, we establish the first convergence rate for this method.
Estimating multi-year 24/7 origin-destination demand using high-granular multi-source traffic data
Authors: Wei Ma, Zhen, Qian
Date: 2019-01-26 00:00:00
Tasks: GPU, Traffic Prediction

Dynamic origin-destination (OD) demand is central to transportation system
modeling and analysis. The dynamic OD demand estimation problem (DODE) has been
studied for decades, most of which solve the DODE problem on a typical day or
several typical hours. There is a lack of methods that estimate high-resolution
dynamic OD demand for a sequence of many consecutive days over several years
(referred to as 24/7 OD in this research). Having multi-year 24/7 OD demand
would allow a better understanding of characteristics of dynamic OD demands and
their evolution/trends over the past few years, a critical input for modeling
transportation system evolution and reliability. This paper presents a
data-driven framework that estimates day-to-day dynamic OD using high-granular
traffic counts and speed data collected over many years. The proposed framework
statistically clusters daily traffic data into typical traffic patterns using
t-Distributed Stochastic Neighbor Embedding (t-SNE) and k-means methods. A
GPU-based stochastic projected gradient descent method is proposed to
efficiently solve the multi-year 24/7 DODE problem. It is demonstrated that the
new method efficiently estimates the 5-minute dynamic OD demand for every
single day from 2014 to 2016 on I-5 and SR-99 in the Sacramento region. The
resultant multi-year 24/7 dynamic OD demand reveals the daily, weekly, monthly,
seasonal and yearly change in travel demand in a region, implying intriguing
demand characteristics over the years.
Soft labeling by Distilling Anatomical knowledge for Improved MS Lesion Segmentation
Authors: Eytan Kats, Jacob Goldberger, Hayit Greenspan
Date: 2019-01-26 00:00:00
Tasks: Lesion Segmentation, Segmentation

This paper explores the use of a soft ground-truth mask ("soft mask'') to
train a Fully Convolutional Neural Network (FCNN) for segmentation of Multiple
Sclerosis (MS) lesions. Detection and segmentation of MS lesions is a complex
task largely due to the extreme unbalanced data, with very small number of
lesion pixels that can be used for training. Utilizing the anatomical knowledge
that the lesion surrounding pixels may also include some lesion level
information, we suggest to increase the data set of the lesion class with
neighboring pixel data - with a reduced confidence weight. A soft mask is
constructed by morphological dilation of the binary segmentation mask provided
by a given expert, where expert-marked voxels receive label 1 and voxels of the
dilated region are assigned a soft label. In the methodology proposed, the FCNN
is trained using the soft mask. On the ISBI 2015 challenge dataset, this is
shown to provide a better precision-recall tradeoff and to achieve a higher
average Dice similarity coefficient. We also show that by using this soft mask
scheme we can improve the network segmentation performance when compared to a
second independent expert.
4D Generic Video Object Proposals
Authors: Aljosa Osep, Paul Voigtlaender, Mark Weber, Jonathon Luiten, Bastian Leibe
Date: 2019-01-26 00:00:00
Tasks: Instance Segmentation, Object, Semantic Segmentation, Video Understanding

Many high-level video understanding methods require input in the form of object proposals. Currently, such proposals are predominantly generated with the help of networks that were trained for detecting and segmenting a set of known object classes, which limits their applicability to cases where all objects of interest are represented in the training set. This is a restriction for automotive scenarios, where unknown objects can frequently occur. We propose an approach that can reliably extract spatio-temporal object proposals for both known and unknown object categories from stereo video. Our 4D Generic Video Tubes (4D-GVT) method leverages motion cues, stereo data, and object instance segmentation to compute a compact set of video-object proposals that precisely localizes object candidates and their contours in 3D space and time. We show that given only a small amount of labeled data, our 4D-GVT proposal generator generalizes well to real-world scenarios, in which unknown categories appear. It outperforms other approaches that try to detect as many objects as possible by increasing the number of classes in the training set to several thousand.
Clustering Discrete-Valued Time Series
Authors: Tyler Roick, Dimitris Karlis, Paul D. McNicholas
Date: 2019-01-26 00:00:00
Tasks: Clustering, Model Selection, Time Series, Time Series Analysis

There is a need for the development of models that are able to account for discreteness in data, along with its time series properties and correlation. Our focus falls on INteger-valued AutoRegressive (INAR) type models. The INAR type models can be used in conjunction with existing model-based clustering techniques to cluster discrete-valued time series data. With the use of a finite mixture model, several existing techniques such as the selection of the number of clusters, estimation using expectation-maximization and model selection are applicable. The proposed model is then demonstrated on real data to illustrate its clustering applications.
Spectrum Data Poisoning with Adversarial Deep Learning
Authors: Yi Shi, Tugba Erpek, Yalin E. Sagduyu, Jason H. Li
Date: 2019-01-26 00:00:00
Tasks: BIG-bench Machine Learning, Data Poisoning, Deep Learning

Machine learning has been widely applied in wireless communications. However,
the security aspects of machine learning in wireless applications have not been
well understood yet. We consider the case that a cognitive transmitter senses
the spectrum and transmits on idle channels determined by a machine learning
algorithm. We present an adversarial machine learning approach to launch a
spectrum data poisoning attack by inferring the transmitter's behavior and
attempting to falsify the spectrum sensing data over the air. For that purpose,
the adversary transmits for a short period of time when the channel is idle to
manipulate the input for the decision mechanism of the transmitter. The
cognitive engine at the transmitter is a deep neural network model that
predicts idle channels with minimum sensing error for data transmissions. The
transmitter collects spectrum sensing data and uses it as the input to its
machine learning algorithm. In the meantime, the adversary builds a cognitive
engine using another deep neural network model to predict when the transmitter
will have a successful transmission based on its spectrum sensing data. The
adversary then performs the over-the-air spectrum data poisoning attack, which
aims to change the channel occupancy status from idle to busy when the
transmitter is sensing, so that the transmitter is fooled into making incorrect
transmit decisions. This attack is more energy efficient and harder to detect
compared to jamming of data transmissions. We show that this attack is very
effective and reduces the throughput of the transmitter substantially.
DistInit: Learning Video Representations Without a Single Labeled Video
Authors: Rohit Girdhar, Du Tran, Lorenzo Torresani, Deva Ramanan
Date: 2019-01-26 00:00:00
Proceeding: ICCV 2019 10
Tasks: Action Recognition, Temporal Action Localization, Video Recognition

Video recognition models have progressed significantly over the past few years, evolving from shallow classifiers trained on hand-crafted features to deep spatiotemporal networks. However, labeled video data required to train such models have not been able to keep up with the ever-increasing depth and sophistication of these networks. In this work, we propose an alternative approach to learning video representations that require no semantically labeled videos and instead leverages the years of effort in collecting and labeling large and clean still-image datasets. We do so by using state-of-the-art models pre-trained on image datasets as "teachers" to train video models in a distillation framework. We demonstrate that our method learns truly spatiotemporal features, despite being trained only using supervision from still-image networks. Moreover, it learns good representations across different input modalities, using completely uncurated raw video data sources and with different 2D teacher models. Our method obtains strong transfer performance, outperforming standard techniques for bootstrapping video architectures with image-based models by 16%. We believe that our approach opens up new approaches for learning spatiotemporal representations from unlabeled video data.
Efficient Toxicity Prediction via Simple Features Using Shallow Neural Networks and Decision Trees
Authors: Abdul Karim, Avinash Mishra, M. A. Hakim Newton, Abdul Sattar
Date: 2019-01-26 00:00:00
Tasks: CPU, GPU

Toxicity prediction of chemical compounds is a grand challenge. Lately, it
achieved significant progress in accuracy but using a huge set of features,
implementing a complex blackbox technique such as a deep neural network, and
exploiting enormous computational resources. In this paper, we strongly argue
for the models and methods that are simple in machine learning characteristics,
efficient in computing resource usage, and powerful to achieve very high
accuracy levels. To demonstrate this, we develop a single task-based chemical
toxicity prediction framework using only 2D features that are less compute
intensive. We effectively use a decision tree to obtain an optimum number of
features from a collection of thousands of them. We use a shallow neural
network and jointly optimize it with decision tree taking both network
parameters and input features into account. Our model needs only a minute on a
single CPU for its training while existing methods using deep neural networks
need about 10 min on NVidia Tesla K40 GPU. However, we obtain similar or better
performance on several toxicity benchmark tasks. We also develop a cumulative
feature ranking method which enables us to identify features that can help
chemists perform prescreening of toxic compounds effectively.
On Detecting GANs and Retouching based Synthetic Alterations
Authors: Anubhav Jain, Richa Singh, Mayank Vatsa
Date: 2019-01-26 00:00:00
Tasks: Face Recognition

Digitally retouching images has become a popular trend, with people posting
altered images on social media and even magazines posting flawless facial
images of celebrities. Further, with advancements in Generative Adversarial
Networks (GANs), now changing attributes and retouching have become very easy.
Such synthetic alterations have adverse effect on face recognition algorithms.
While researchers have proposed to detect image tampering, detecting GANs
generated images has still not been explored. This paper proposes a supervised
deep learning algorithm using Convolutional Neural Networks (CNNs) to detect
synthetically altered images. The algorithm yields an accuracy of 99.65% on
detecting retouching on the ND-IIITD dataset. It outperforms the previous state
of the art which reported an accuracy of 87% on the database. For
distinguishing between real images and images generated using GANs, the
proposed algorithm yields an accuracy of 99.83%.
Distributed Convolutional Dictionary Learning (DiCoDiLe): Pattern Discovery in Large Images and Signals
Authors: Thomas Moreau, Alexandre Gramfort
Date: 2019-01-26 00:00:00
Tasks: Denoising, Dictionary Learning, Image Denoising

Convolutional dictionary learning (CDL) estimates shift invariant basis
adapted to multidimensional data. CDL has proven useful for image denoising or
inpainting, as well as for pattern discovery on multivariate signals. As
estimated patterns can be positioned anywhere in signals or images,
optimization techniques face the difficulty of working in extremely high
dimensions with millions of pixels or time samples, contrarily to standard
patch-based dictionary learning. To address this optimization problem, this
work proposes a distributed and asynchronous algorithm, employing locally
greedy coordinate descent and an asynchronous locking mechanism that does not
require a central server. This algorithm can be used to distribute the
computation on a number of workers which scales linearly with the encoded
signal's size. Experiments confirm the scaling properties which allows us to
learn patterns on large scales images from the Hubble Space Telescope.
DELTA: DEep Learning Transfer using Feature Map with Attention for Convolutional Networks
Authors: Xingjian Li, Haoyi Xiong, Hanchao Wang, Yuxuan Rao, Li-Ping Liu, Zeyu Chen, Jun Huan
Date: 2019-01-26 00:00:00
Proceeding: ICLR 2019 5
Tasks: Transfer Learning

Transfer learning through fine-tuning a pre-trained neural network with an extremely large dataset, such as ImageNet, can significantly accelerate training while the accuracy is frequently bottlenecked by the limited dataset size of the new target task. To solve the problem, some regularization methods, constraining the outer layer weights of the target network using the starting point as references (SPAR), have been studied. In this paper, we propose a novel regularized transfer learning framework DELTA, namely DEep Learning Transfer using Feature Map with Attention. Instead of constraining the weights of neural network, DELTA aims to preserve the outer layer outputs of the target network. Specifically, in addition to minimizing the empirical loss, DELTA intends to align the outer layer outputs of two networks, through constraining a subset of feature maps that are precisely selected by attention that has been learned in an supervised learning manner. We evaluate DELTA with the state-of-the-art algorithms, including L2 and L2-SP. The experiment results show that our proposed method outperforms these baselines with higher accuracy for new tasks.