pwc-archive/papers-with-abstracts huggingface dataset:
Latency and Throughput Characterization of Convolutional Neural Networks for Mobile Computer Vision
Authors: Jussi Hanhirova, Teemu Kämäräinen, Sipi Seppälä, Matti Siekkinen, Vesa Hirvisalo, Antti Ylä-Jääski
Date: 2018-03-26 00:00:00
Tasks: Object Recognition

We study performance characteristics of convolutional neural networks (CNN)
for mobile computer vision systems. CNNs have proven to be a powerful and
efficient approach to implement such systems. However, the system performance
depends largely on the utilization of hardware accelerators, which are able to
speed up the execution of the underlying mathematical operations tremendously
through massive parallelism. Our contribution is performance characterization
of multiple CNN-based models for object recognition and detection with several
different hardware platforms and software frameworks, using both local
(on-device) and remote (network-side server) computation. The measurements are
conducted using real workloads and real processing platforms. On the platform
side, we concentrate especially on TensorFlow and TensorRT. Our measurements
include embedded processors found on mobile devices and high-performance
processors that can be used on the network side of mobile systems. We show that
there exists significant latency--throughput trade-offs but the behavior is
very complex. We demonstrate and discuss several factors that affect the
performance and yield this complex behavior.
Unsupervised Learning and Segmentation of Complex Activities from Video
Authors: Fadime Sener, Angela Yao
Date: 2018-03-26 00:00:00
Proceeding: CVPR 2018 6

This paper presents a new method for unsupervised segmentation of complex
activities from video into multiple steps, or sub-activities, without any
textual input. We propose an iterative discriminative-generative approach which
alternates between discriminatively learning the appearance of sub-activities
from the videos' visual features to sub-activity labels and generatively
modelling the temporal structure of sub-activities using a Generalized Mallows
Model. In addition, we introduce a model for background to account for frames
unrelated to the actual activities. Our approach is validated on the
challenging Breakfast Actions and Inria Instructional Videos datasets and
outperforms both unsupervised and weakly-supervised state of the art.
Semantic See-Through Rendering on Light Fields
Authors: Huangjie Yu, Guli Zhang, Yuanxi Ma, Yingliang Zhang, Jingyi Yu
Date: 2018-03-26 00:00:00
Tasks: Stereo Matching, Stereo Matching Hand

We present a novel semantic light field (LF) refocusing technique that can
achieve unprecedented see-through quality. Different from prior art, our
semantic see-through (SST) differentiates rays in their semantic meaning and
depth. Specifically, we combine deep learning and stereo matching to provide
each ray a semantic label. We then design tailored weighting schemes for
blending the rays. Although simple, our solution can effectively remove
foreground residues when focusing on the background. At the same time, SST
maintains smooth transitions in varying focal depths. Comprehensive experiments
on synthetic and new real indoor and outdoor datasets demonstrate the
effectiveness and usefulness of our technique.
Near-optimal sample compression for nearest neighbors
Authors: Lee-Ad Gottlieb, Aryeh Kontorovich, Pinhas Nisnevitch
Date: 2014-04-13 00:00:00
Proceeding: NeurIPS 2014 12
Tasks: General Classification

We present the first sample compression algorithm for nearest neighbors with
non-trivial performance guarantees. We complement these guarantees by
demonstrating almost matching hardness lower bounds, which show that our bound
is nearly optimal. Our result yields new insight into margin-based nearest
neighbor classification in metric spaces and allows us to significantly sharpen
and simplify existing bounds. Some encouraging empirical results are also
presented.
Real Time Surveillance for Low Resolution and Limited-Data Scenarios: An Image Set Classification Approach
Authors: Uzair Nadeem, Syed Afaq Ali Shah, Mohammed Bennamoun, Roberto Togneri, Ferdous Sohel
Date: 2018-03-26 00:00:00
Tasks: Face Recognition, General Classification, Object Recognition, regression

This paper proposes a novel image set classification technique based on the
concept of linear regression. Unlike most other approaches, the proposed
technique does not involve any training or feature extraction. The gallery
image sets are represented as subspaces in a high dimensional space. Class
specific gallery subspaces are used to estimate regression models for each
image of the test image set. Images of the test set are then projected on the
gallery subspaces. Residuals, calculated using the Euclidean distance between
the original and the projected test images, are used as the distance metric.
Three different strategies are devised to decide on the final class of the test
image set. We performed extensive evaluations of the proposed technique under
the challenges of low resolution, noise and less gallery data for the tasks of
surveillance, video-based face recognition and object recognition. Experiments
show that the proposed technique achieves a better classification accuracy and
a faster execution time compared to existing techniques especially under the
challenging conditions of low resolution and small gallery and test data.
Decoupling "when to update" from "how to update"
Authors: Eran Malach, Shai Shalev-Shwartz
Date: 2017-06-08 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Face Recognition, Gender Classification, General Classification

Deep learning requires data. A useful approach to obtain data is to be
creative and mine data from various sources, that were created for different
purposes. Unfortunately, this approach often leads to noisy labels. In this
paper, we propose a meta algorithm for tackling the noisy labels problem. The
key idea is to decouple "when to update" from "how to update". We demonstrate
the effectiveness of our algorithm by mining data for gender classification by
combining the Labeled Faces in the Wild (LFW) face recognition dataset with a
textual genderizing service, which leads to a noisy dataset. While our approach
is very simple to implement, it leads to state-of-the-art results. We analyze
some convergence properties of the proposed algorithm.
Regularizing Deep Hashing Networks Using GAN Generated Fake Images
Authors: Libing Geng, Yan Pan, Jikai Chen, Hanjiang Lai
Date: 2018-03-26 00:00:00
Tasks: Deep Hashing, Generative Adversarial Network, Image Retrieval, Retrieval

Recently, deep-networks-based hashing (deep hashing) has become a leading
approach for large-scale image retrieval. It aims to learn a compact bitwise
representation for images via deep networks, so that similar images are mapped
to nearby hash codes. Since a deep network model usually has a large number of
parameters, it may probably be too complicated for the training data we have,
leading to model over-fitting. To address this issue, in this paper, we propose
a simple two-stage pipeline to learn deep hashing models, by regularizing the
deep hashing networks using fake images. The first stage is to generate fake
images from the original training set without extra data, via a generative
adversarial network (GAN). In the second stage, we propose a deep architec-
ture to learn hash functions, in which we use a maximum-entropy based loss to
incorporate the newly created fake images by the GAN. We show that this loss
acts as a strong regularizer of the deep architecture, by penalizing
low-entropy output hash codes. This loss can also be interpreted as a model
ensemble by simultaneously training many network models with massive weight
sharing but over different training sets. Empirical evaluation results on
several benchmark datasets show that the proposed method has superior
performance gains over state-of-the-art hashing methods.
Scalable inference for crossed random effects models
Authors: Omiros Papaspiliopoulos, Gareth O. Roberts, Giacomo Zanella
Date: 2018-03-26 00:00:00

We analyze the complexity of Gibbs samplers for inference in crossed random
effect models used in modern analysis of variance. We demonstrate that for
certain designs the plain vanilla Gibbs sampler is not scalable, in the sense
that its complexity is worse than proportional to the number of parameters and
data. We thus propose a simple modification leading to a collapsed Gibbs
sampler that is provably scalable. Although our theory requires some
balancedness assumptions on the data designs, we demonstrate in simulated and
real datasets that the rates it predicts match remarkably the correct rates in
cases where the assumptions are violated. We also show that the collapsed Gibbs
sampler, extended to sample further unknown hyperparameters, outperforms
significantly alternative state of the art algorithms.
Degeneration in VAE: in the Light of Fisher Information Loss
Authors: Huangjie Zheng, Jiangchao Yao, Ya zhang, Ivor W. Tsang
Date: 2018-02-19 00:00:00
Tasks: Representation Learning

While enormous progress has been made to Variational Autoencoder (VAE) in
recent years, similar to other deep networks, VAE with deep networks suffers
from the problem of degeneration, which seriously weakens the correlation
between the input and the corresponding latent codes, deviating from the goal
of the representation learning. To investigate how degeneration affects VAE
from a theoretical perspective, we illustrate the information transmission in
VAE and analyze the intermediate layers of the encoders/decoders. Specifically,
we propose a Fisher Information measure for the layer-wise analysis. With such
measure, we demonstrate that information loss is ineluctable in feed-forward
networks and causes the degeneration in VAE. We show that skip connections in
VAE enable the preservation of information without changing the model
architecture. We call this class of VAE equipped with skip connections as SCVAE
and perform a range of experiments to show its advantages in information
preservation and degeneration mitigation.
Fast and Accurate Single Image Super-Resolution via Information Distillation Network
Authors: Zheng Hui, Xiumei Wang, Xinbo Gao
Date: 2018-03-26 00:00:00
Proceeding: CVPR 2018 6
Tasks: Image Super-Resolution, Super-Resolution

Recently, deep convolutional neural networks (CNNs) have been demonstrated
remarkable progress on single image super-resolution. However, as the depth and
width of the networks increase, CNN-based super-resolution methods have been
faced with the challenges of computational complexity and memory consumption in
practice. In order to solve the above questions, we propose a deep but compact
convolutional network to directly reconstruct the high resolution image from
the original low resolution image. In general, the proposed model consists of
three parts, which are feature extraction block, stacked information
distillation blocks and reconstruction block respectively. By combining an
enhancement unit with a compression unit into a distillation block, the local
long and short-path features can be effectively extracted. Specifically, the
proposed enhancement unit mixes together two different types of features and
the compression unit distills more useful information for the sequential
blocks. In addition, the proposed network has the advantage of fast execution
due to the comparatively few numbers of filters per layer and the use of group
convolution. Experimental results demonstrate that the proposed method is
superior to the state-of-the-art methods, especially in terms of time
performance.
CNN in MRF: Video Object Segmentation via Inference in A CNN-Based Higher-Order Spatio-Temporal MRF
Authors: Linchao Bao, Baoyuan Wu, Wei Liu
Date: 2018-03-26 00:00:00
Proceeding: CVPR 2018 6
Tasks: Object, One-Shot Segmentation, Optical Flow Estimation, Segmentation, Semantic Segmentation, Semi-Supervised Video Object Segmentation, Video Object Segmentation, Video Semantic Segmentation

This paper addresses the problem of video object segmentation, where the
initial object mask is given in the first frame of an input video. We propose a
novel spatio-temporal Markov Random Field (MRF) model defined over pixels to
handle this problem. Unlike conventional MRF models, the spatial dependencies
among pixels in our model are encoded by a Convolutional Neural Network (CNN).
Specifically, for a given object, the probability of a labeling to a set of
spatially neighboring pixels can be predicted by a CNN trained for this
specific object. As a result, higher-order, richer dependencies among pixels in
the set can be implicitly modeled by the CNN. With temporal dependencies
established by optical flow, the resulting MRF model combines both spatial and
temporal cues for tackling video object segmentation. However, performing
inference in the MRF model is very difficult due to the very high-order
dependencies. To this end, we propose a novel CNN-embedded algorithm to perform
approximate inference in the MRF. This algorithm proceeds by alternating
between a temporal fusion step and a feed-forward CNN step. When initialized
with an appearance-based one-shot segmentation CNN, our model outperforms the
winning entries of the DAVIS 2017 Challenge, without resorting to model
ensembling or any dedicated detectors.
Variational Autoencoders for Learning Latent Representations of Speech Emotion: A Preliminary Study
Authors: Siddique Latif, Rajib Rana, Junaid Qadir, Julien Epps
Date: 2017-12-23 00:00:00
Tasks: Emotion Classification, Emotion Recognition, General Classification, Speech Emotion Recognition

Learning the latent representation of data in unsupervised fashion is a very interesting process that provides relevant features for enhancing the performance of a classifier. For speech emotion recognition tasks, generating effective features is crucial. Currently, handcrafted features are mostly used for speech emotion recognition, however, features learned automatically using deep learning have shown strong success in many problems, especially in image processing. In particular, deep generative models such as Variational Autoencoders (VAEs) have gained enormous success for generating features for natural images. Inspired by this, we propose VAEs for deriving the latent representation of speech signals and use this representation to classify emotions. To the best of our knowledge, we are the first to propose VAEs for speech emotion classification. Evaluations on the IEMOCAP dataset demonstrate that features learned by VAEs can produce state-of-the-art results for speech emotion classification.
REST: Real-to-Synthetic Transform for Illumination Invariant Camera Localization
Authors: Sota Shoman, Tomohiro Mashita, Alexander Plopski, Photchara Ratsamee, Yuki Uranishi, Haruo Takemura
Date: 2018-03-26 00:00:00
Tasks: Camera Localization

Accurate camera localization is an essential part of tracking systems.
However, localization results are greatly affected by illumination. Including
data collected under various lighting conditions can improve the robustness of
the localization algorithm to lighting variation. However, this is very tedious
and time consuming. By using synthesized images it is possible to easily
accumulate a large variety of views under varying illumination and weather
conditions. Despite continuously improving processing power and rendering
algorithms, synthesized images do not perfectly match real images of the same
scene, i.e. there exists a gap between real and synthesized images that also
affects the accuracy of camera localization. To reduce the impact of this gap,
we introduce "REal-to-Synthetic Transform (REST)." REST is an autoencoder-like
network that converts real features to their synthetic counterpart. The
converted features can then be matched against the accumulated database for
robust camera localization. In our experiments REST improved feature matching
accuracy under variable lighting conditions by approximately 30%. Moreover, our
system outperforms state of the art CNN-based camera localization methods
trained with synthetic images. We believe our method could be used to
initialize local tracking and to simplify data accumulation for lighting robust
localization.
Repulsion Loss: Detecting Pedestrians in a Crowd
Authors: Xinlong Wang, Tete Xiao, Yuning Jiang, Shuai Shao, Jian Sun, Chunhua Shen
Date: 2017-11-21 00:00:00
Proceeding: CVPR 2018 6
Tasks: Pedestrian Detection, regression

Detecting individual pedestrians in a crowd remains a challenging problem
since the pedestrians often gather together and occlude each other in
real-world scenarios. In this paper, we first explore how a state-of-the-art
pedestrian detector is harmed by crowd occlusion via experimentation, providing
insights into the crowd occlusion problem. Then, we propose a novel bounding
box regression loss specifically designed for crowd scenes, termed repulsion
loss. This loss is driven by two motivations: the attraction by target, and the
repulsion by other surrounding objects. The repulsion term prevents the
proposal from shifting to surrounding objects thus leading to more crowd-robust
localization. Our detector trained by repulsion loss outperforms all the
state-of-the-art methods with a significant improvement in occlusion cases.
PU-Net: Point Cloud Upsampling Network
Authors: Lequan Yu, Xianzhi Li, Chi-Wing Fu, Daniel Cohen-Or, Pheng-Ann Heng
Date: 2018-01-21 00:00:00
Proceeding: CVPR 2018 6
Tasks: Point Cloud Super Resolution, point cloud upsampling

Learning and analyzing 3D point clouds with deep networks is challenging due
to the sparseness and irregularity of the data. In this paper, we present a
data-driven point cloud upsampling technique. The key idea is to learn
multi-level features per point and expand the point set via a multi-branch
convolution unit implicitly in feature space. The expanded feature is then
split to a multitude of features, which are then reconstructed to an upsampled
point set. Our network is applied at a patch-level, with a joint loss function
that encourages the upsampled points to remain on the underlying surface with a
uniform distribution. We conduct various experiments using synthesis and scan
data to evaluate our method and demonstrate its superiority over some baseline
methods and an optimization-based method. Results show that our upsampled
points have better uniformity and are located closer to the underlying
surfaces.
Scalable photonic reinforcement learning by time-division multiplexing of laser chaos
Authors: Makoto Naruse, Takatomo Mihana, Hirokazu Hori, Hayato Saigo, Kazuya Okamura, Mikio Hasegawa, Atsushi Uchida
Date: 2018-03-26 00:00:00
Tasks: Decision Making, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Time Series, Time Series Analysis

Reinforcement learning involves decision making in dynamic and uncertain
environments and constitutes a crucial element of artificial intelligence. In
our previous work, we experimentally demonstrated that the ultrafast chaotic
oscillatory dynamics of lasers can be used to solve the two-armed bandit
problem efficiently, which requires decision making concerning a class of
difficult trade-offs called the exploration-exploitation dilemma. However, only
two selections were employed in that research; thus, the scalability of the
laser-chaos-based reinforcement learning should be clarified. In this study, we
demonstrated a scalable, pipelined principle of resolving the multi-armed
bandit problem by introducing time-division multiplexing of chaotically
oscillated ultrafast time-series. The experimental demonstrations in which
bandit problems with up to 64 arms were successfully solved are presented in
this report. Detailed analyses are also provided that include performance
comparisons among laser chaos signals generated in different physical
conditions, which coincide with the diffusivity inherent in the time series.
This study paves the way for ultrafast reinforcement learning by taking
advantage of the ultrahigh bandwidths of light wave and practical enabling
technologies.
Multi-scale Processing of Noisy Images using Edge Preservation Losses
Authors: Nati Ofir, Yosi Keller
Date: 2018-03-26 00:00:00
Tasks: Denoising, Edge Detection, General Classification, GPU, image-classification, Image Classification, Image Denoising

Noisy images processing is a fundamental task of computer vision. The first
example is the detection of faint edges in noisy images, a challenging problem
studied in the last decades. A recent study introduced a fast method to detect
faint edges in the highest accuracy among all the existing approaches. Their
complexity is nearly linear in the image's pixels and their runtime is seconds
for a noisy image. Their approach utilizes a multi-scale binary partitioning of
the image. By utilizing the multi-scale U-net architecture, we show in this
paper that their method can be dramatically improved in both aspects of run
time and accuracy. By training the network on a dataset of binary images, we
developed an approach for faint edge detection that works in a linear
complexity. Our runtime of a noisy image is milliseconds on a GPU. Even though
our method is orders of magnitude faster, we still achieve higher accuracy of
detection under many challenging scenarios. In addition, we show that our
approach to performing multi-scale preprocessing of noisy images using U-net
improves the ability to perform other vision tasks under the presence of noise.
We prove it on the problems of noisy objects classification and classical image
denoising. We show that multi-scale denoising can be carried out by a novel
edge preservation loss. As our experiments show, we achieve high-quality
results in the three aspects of faint edge detection, noisy image
classification and natural image denoising.
Object Detection for Comics using Manga109 Annotations
Authors: Toru Ogawa, Atsushi Otsubo, Rei Narita, Yusuke Matsui, Toshihiko Yamasaki, Kiyoharu Aizawa
Date: 2018-03-23 00:00:00
Tasks: Object, object-detection, Object Detection

With the growth of digitized comics, image understanding techniques are
becoming important. In this paper, we focus on object detection, which is a
fundamental task of image understanding. Although convolutional neural networks
(CNN)-based methods archived good performance in object detection for
naturalistic images, there are two problems in applying these methods to the
comic object detection task. First, there is no large-scale annotated comics
dataset. The CNN-based methods require large-scale annotations for training.
Secondly, the objects in comics are highly overlapped compared to naturalistic
images. This overlap causes the assignment problem in the existing CNN-based
methods. To solve these problems, we proposed a new annotation dataset and a
new CNN model. We annotated an existing image dataset of comics and created the
largest annotation dataset, named Manga109-annotations. For the assignment
problem, we proposed a new CNN-based detector, SSD300-fork. We compared
SSD300-fork with other detection methods using Manga109-annotations and
confirmed that our model outperformed them based on the mAP score.
Precision Sugarcane Monitoring Using SVM Classifier
Authors: Sachin Kumar, Sumita Mishra, Pooja Khanna, Pragya
Date: 2018-03-26 00:00:00
Tasks: Clustering

India is agriculture based economy and sugarcane is one of the major crops
produced in northern India. Productivity of sugarcane decreases due to
inappropriate soil conditions and infections caused by various types of
diseases , timely and accurate disease diagnosis, plays an important role
towards optimizing crop yield. This paper presents a system model for
monitoring of sugarcane crop, the proposed model continuously monitor
parameters (temperature, humidity and moisture) responsible for healthy growth
of the crop in addition KNN clustering along with SVM classifier is utilized
for infection identification if any through images obtained at regular
intervals. The data has been transmitted wirelessly from the site to the
control unit. Model achieves an accuracy of 96% on a sample of 200 images, the
model was tested at Lolai, near Malhaur, Gomti Nagar Extension.