pwc-archive/papers-with-abstracts huggingface dataset:
Occlusion-Aware Video Deblurring with a New Layered Blur Model
Authors: Byeongjoo Ahn, Tae Hyun Kim, Wonsik Kim, Kyoung Mu Lee
Date: 2016-11-29 00:00:00
Tasks: Deblurring, Object, Video Deblurring

We present a deblurring method for scenes with occluding objects using a
carefully designed layered blur model. Layered blur model is frequently used in
the motion deblurring problem to handle locally varying blurs, which is caused
by object motions or depth variations in a scene. However, conventional models
have a limitation in representing the layer interactions occurring at occlusion
boundaries. In this paper, we address this limitation in both theoretical and
experimental ways, and propose a new layered blur model reflecting actual blur
generation process. Based on this model, we develop an occlusion-aware
deblurring method that can estimate not only the clear foreground and
background, but also the object motion more accurately. We also provide a novel
analysis on the blur kernel at object boundaries, which shows the distinctive
characteristics of the blur kernel that cannot be captured by conventional blur
models. Experimental results on synthetic and real blurred videos demonstrate
that the proposed method yields superior results, especially at object
boundaries.
Is a picture worth a thousand words? A Deep Multi-Modal Fusion Architecture for Product Classification in e-commerce
Authors: Tom Zahavy, Alessandro Magnani, Abhinandan Krishnan, Shie Mannor
Date: 2016-11-29 00:00:00
Tasks: General Classification

Classifying products into categories precisely and efficiently is a major
challenge in modern e-commerce. The high traffic of new products uploaded daily
and the dynamic nature of the categories raise the need for machine learning
models that can reduce the cost and time of human editors. In this paper, we
propose a decision level fusion approach for multi-modal product classification
using text and image inputs. We train input specific state-of-the-art deep
neural networks for each input source, show the potential of forging them
together into a multi-modal architecture and train a novel policy network that
learns to choose between them. Finally, we demonstrate that our multi-modal
network improves the top-1 accuracy % over both networks on a real-world
large-scale product classification dataset that we collected fromWalmart.com.
While we focus on image-text fusion that characterizes e-commerce domains, our
algorithms can be easily applied to other modalities such as audio, video,
physical sensors, etc.
Learning Filter Banks Using Deep Learning For Acoustic Signals
Authors: Shuhui Qu, Juncheng Li, Wei Dai, Samarjit Das
Date: 2016-11-29 00:00:00
Tasks: Deep Learning

Designing appropriate features for acoustic event recognition tasks is an
active field of research. Expressive features should both improve the
performance of the tasks and also be interpret-able. Currently, heuristically
designed features based on the domain knowledge requires tremendous effort in
hand-crafting, while features extracted through deep network are difficult for
human to interpret. In this work, we explore the experience guided learning
method for designing acoustic features. This is a novel hybrid approach
combining both domain knowledge and purely data driven feature designing. Based
on the procedure of log Mel-filter banks, we design a filter bank learning
layer. We concatenate this layer with a convolutional neural network (CNN)
model. After training the network, the weight of the filter bank learning layer
is extracted to facilitate the design of acoustic features. We smooth the
trained weight of the learning layer and re-initialize it in filter bank
learning layer as audio feature extractor. For the environmental sound
recognition task based on the Urban- sound8K dataset, the experience guided
learning leads to a 2% accuracy improvement compared with the fixed feature
extractors (the log Mel-filter bank). The shape of the new filter banks are
visualized and explained to prove the effectiveness of the feature design
process.
FlyCap: Markerless Motion Capture Using Multiple Autonomous Flying Cameras
Authors: Lan Xu, Lu Fang, Wei Cheng, Kaiwen Guo, Guyue Zhou, Qionghai Dai, Yebin Liu
Date: 2016-10-29 00:00:00
Tasks: Markerless Motion Capture, Visual Odometry

Aiming at automatic, convenient and non-instrusive motion capture, this paper
presents a new generation markerless motion capture technique, the FlyCap
system, to capture surface motions of moving characters using multiple
autonomous flying cameras (autonomous unmanned aerial vehicles(UAV) each
integrated with an RGBD video camera). During data capture, three cooperative
flying cameras automatically track and follow the moving target who performs
large scale motions in a wide space. We propose a novel non-rigid surface
registration method to track and fuse the depth of the three flying cameras for
surface motion tracking of the moving target, and simultaneously calculate the
pose of each flying camera. We leverage the using of visual-odometry
information provided by the UAV platform, and formulate the surface tracking
problem in a non-linear objective function that can be linearized and
effectively minimized through a Gaussian-Newton method. Quantitative and
qualitative experimental results demonstrate the competent and plausible
surface and motion reconstruction results
Graph-Based Manifold Frequency Analysis for Denoising
Authors: Shay Deutsch, Antonio Ortega, Gerard Medioni
Date: 2016-11-29 00:00:00
Tasks: Denoising

We propose a new framework for manifold denoising based on processing in the
graph Fourier frequency domain, derived from the spectral decomposition of the
discrete graph Laplacian. Our approach uses the Spectral Graph Wavelet
transform in order to per- form non-iterative denoising directly in the graph
frequency domain, an approach inspired by conventional wavelet-based signal
denoising methods. We theoretically justify our approach, based on the fact
that for smooth manifolds the coordinate information energy is localized in the
low spectral graph wavelet sub-bands, while the noise affects all frequency
bands in a similar way. Experimental results show that our proposed manifold
frequency denoising (MFD) approach significantly outperforms the state of the
art denoising meth- ods, and is robust to a wide range of parameter selections,
e.g., the choice of k nearest neighbor connectivity of the graph.
Active learning with version spaces for object detection
Authors: Soumya Roy, Vinay P. Namboodiri, Arijit Biswas
Date: 2016-11-22 00:00:00
Tasks: Active Learning, Object, object-detection, Object Detection, Structured Prediction

Given an image, we would like to learn to detect objects belonging to
particular object categories. Common object detection methods train on large
annotated datasets which are annotated in terms of bounding boxes that contain
the object of interest. Previous works on object detection model the problem as
a structured regression problem which ranks the correct bounding boxes more
than the background ones. In this paper we develop algorithms which actively
obtain annotations from human annotators for a small set of images, instead of
all images, thereby reducing the annotation effort. Towards this goal, we make
the following contributions: 1. We develop a principled version space based
active learning method that solves for object detection as a structured
prediction problem in a weakly supervised setting 2. We also propose two
variants of the margin sampling strategy 3. We analyse the results on standard
object detection benchmarks that show that with only 20% of the data we can
obtain more than 95% of the localization accuracy of full supervision. Our
methods outperform random sampling and the classical uncertainty-based active
learning algorithms like entropy
Exploiting Unlabeled Data for Neural Grammatical Error Detection
Authors: Zhuoran Liu, Yang Liu
Date: 2016-11-28 00:00:00
Tasks: Binary Classification, General Classification, Grammatical Error Detection

Identifying and correcting grammatical errors in the text written by
non-native writers has received increasing attention in recent years. Although
a number of annotated corpora have been established to facilitate data-driven
grammatical error detection and correction approaches, they are still limited
in terms of quantity and coverage because human annotation is labor-intensive,
time-consuming, and expensive. In this work, we propose to utilize unlabeled
data to train neural network based grammatical error detection models. The
basic idea is to cast error detection as a binary classification problem and
derive positive and negative training examples from unlabeled data. We
introduce an attention-based neural network to capture long-distance
dependencies that influence the word being detected. Experiments show that the
proposed approach significantly outperforms SVMs and convolutional networks
with fixed-size context window.
Deep Quantization: Encoding Convolutional Activations with Deep Generative Model
Authors: Zhaofan Qiu, Ting Yao, Tao Mei
Date: 2016-11-29 00:00:00
Proceeding: CVPR 2017 7
Tasks: Action Recognition, Fine-Grained Image Classification, image-classification, Image Classification, Quantization, Temporal Action Localization, Variational Inference

Deep convolutional neural networks (CNNs) have proven highly effective for
visual recognition, where learning a universal representation from activations
of convolutional layer plays a fundamental problem. In this paper, we present
Fisher Vector encoding with Variational Auto-Encoder (FV-VAE), a novel deep
architecture that quantizes the local activations of convolutional layer in a
deep generative model, by training them in an end-to-end manner. To incorporate
FV encoding strategy into deep generative models, we introduce Variational
Auto-Encoder model, which steers a variational inference and learning in a
neural network which can be straightforwardly optimized using standard
stochastic gradient method. Different from the FV characterized by conventional
generative models (e.g., Gaussian Mixture Model) which parsimoniously fit a
discrete mixture model to data distribution, the proposed FV-VAE is more
flexible to represent the natural property of data for better generalization.
Extensive experiments are conducted on three public datasets, i.e., UCF101,
ActivityNet, and CUB-200-2011 in the context of video action recognition and
fine-grained image classification, respectively. Superior results are reported
when compared to state-of-the-art representations. Most remarkably, our
proposed FV-VAE achieves to-date the best published accuracy of 94.2% on
UCF101.
Object Detection Free Instance Segmentation With Labeling Transformations
Authors: Long Jin, Zeyu Chen, Zhuowen Tu
Date: 2016-11-28 00:00:00
Tasks: Instance Segmentation, Object, object-detection, Object Detection, Segmentation, Semantic Segmentation

Instance segmentation has attracted recent attention in computer vision and
existing methods in this domain mostly have an object detection stage. In this
paper, we study the intrinsic challenge of the instance segmentation problem,
the presence of a quotient space (swapping the labels of different instances
leads to the same result), and propose new methods that are object proposal-
and object detection- free. We propose three alternative methods, namely
pixel-based affinity mapping, superpixel-based affinity learning, and
boundary-based component segmentation, all focusing on performing labeling
transformations to cope with the quotient space problem. By adopting fully
convolutional neural networks (FCN) like models, our framework attains
competitive results on both the PASCAL dataset (object-centric) and the Gland
dataset (texture-centric), which the existing methods are not able to do. Our
work also has the advantages in its transparency, simplicity, and being all
segmentation based.
Fast Wavenet Generation Algorithm
Authors: Tom Le Paine, Pooya Khorrami, Shiyu Chang, Yang Zhang, Prajit Ramachandran, Mark A. Hasegawa-Johnson, Thomas S. Huang
Date: 2016-11-29 00:00:00

This paper presents an efficient implementation of the Wavenet generation
process called Fast Wavenet. Compared to a naive implementation that has
complexity O(2^L) (L denotes the number of layers in the network), our proposed
approach removes redundant convolution operations by caching previous
calculations, thereby reducing the complexity to O(L) time. Timing experiments
show significant advantages of our fast implementation over a naive one. While
this method is presented for Wavenet, the same scheme can be applied anytime
one wants to perform autoregressive generation or online prediction using a
model with dilated convolution layers. The code for our method is publicly
available.
Design of Kernels in Convolutional Neural Networks for Image Classification
Authors: Zhun Sun, Mete Ozay, Takayuki Okatani
Date: 2015-11-30 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification

Despite the effectiveness of Convolutional Neural Networks (CNNs) for image
classification, our understanding of the relationship between shape of
convolution kernels and learned representations is limited. In this work, we
explore and employ the relationship between shape of kernels which define
Receptive Fields (RFs) in CNNs for learning of feature representations and
image classification. For this purpose, we first propose a feature
visualization method for visualization of pixel-wise classification score maps
of learned features. Motivated by our experimental results, and observations
reported in the literature for modeling of visual systems, we propose a novel
design of shape of kernels for learning of representations in CNNs. In the
experimental results, we achieved a state-of-the-art classification performance
compared to a base CNN model [28] by reducing the number of parameters and
computational time of the model using the ILSVRC-2012 dataset [24]. The
proposed models also outperform the state-of-the-art models employed on the
CIFAR-10/100 datasets [12] for image classification. Additionally, we analyzed
the robustness of the proposed method to occlusion for classification of
partially occluded images compared with the state-of-the-art methods. Our
results indicate the effectiveness of the proposed approach. The code is
available in github.com/minogame/caffe-qhconv.
Column Networks for Collective Classification
Authors: Trang Pham, Truyen Tran, Dinh Phung, Svetha Venkatesh
Date: 2016-09-15 00:00:00
Tasks: Classification, General Classification, Genre classification, Relational Reasoning

Relational learning deals with data that are characterized by relational
structures. An important task is collective classification, which is to jointly
classify networked objects. While it holds a great promise to produce a better
accuracy than non-collective classifiers, collective classification is
computational challenging and has not leveraged on the recent breakthroughs of
deep learning. We present Column Network (CLN), a novel deep learning model for
collective classification in multi-relational domains. CLN has many desirable
theoretical properties: (i) it encodes multi-relations between any two
instances; (ii) it is deep and compact, allowing complex functions to be
approximated at the network level with a small set of free parameters; (iii)
local and relational features are learned simultaneously; (iv) long-range,
higher-order dependencies between instances are supported naturally; and (v)
crucially, learning and inference are efficient, linear in the size of the
network and the number of relations. We evaluate CLN on multiple real-world
applications: (a) delay prediction in software projects, (b) PubMed Diabetes
publication classification and (c) film genre classification. In all
applications, CLN demonstrates a higher accuracy than state-of-the-art rivals.
Social Behavior Prediction from First Person Videos
Authors: Shan Su, Jung Pyo Hong, Jianbo Shi, Hyun Soo Park
Date: 2016-11-29 00:00:00
Tasks: 3D Reconstruction, Prediction

This paper presents a method to predict the future movements (location and
gaze direction) of basketball players as a whole from their first person
videos. The predicted behaviors reflect an individual physical space that
affords to take the next actions while conforming to social behaviors by
engaging to joint attention. Our key innovation is to use the 3D reconstruction
of multiple first person cameras to automatically annotate each other's the
visual semantics of social configurations.
  We leverage two learning signals uniquely embedded in first person videos.
Individually, a first person video records the visual semantics of a spatial
and social layout around a person that allows associating with past similar
situations. Collectively, first person videos follow joint attention that can
link the individuals to a group. We learn the egocentric visual semantics of
group movements using a Siamese neural network to retrieve future trajectories.
We consolidate the retrieved trajectories from all players by maximizing a
measure of social compatibility---the gaze alignment towards joint attention
predicted by their social formation, where the dynamics of joint attention is
learned by a long-term recurrent convolutional network. This allows us to
characterize which social configuration is more plausible and predict future
group trajectories.
MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
Authors: Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, Tong Wang
Date: 2016-11-28 00:00:00
Tasks: Benchmarking, Machine Reading Comprehension, Question Answering, Reading Comprehension

We introduce a large scale MAchine Reading COmprehension dataset, which we
name MS MARCO. The dataset comprises of 1,010,916 anonymized
questions---sampled from Bing's search query logs---each with a human generated
answer and 182,669 completely human rewritten generated answers. In addition,
the dataset contains 8,841,823 passages---extracted from 3,563,535 web
documents retrieved by Bing---that provide the information necessary for
curating the natural language answers. A question in the MS MARCO dataset may
have multiple answers or no answers at all. Using this dataset, we propose
three different tasks with varying levels of difficulty: (i) predict if a
question is answerable given a set of context passages, and extract and
synthesize the answer as a human would (ii) generate a well-formed answer (if
possible) based on the context passages that can be understood with the
question and passage context, and finally (iii) rank a set of retrieved
passages given a question. The size of the dataset and the fact that the
questions are derived from real user search queries distinguishes MS MARCO from
other well-known publicly available datasets for machine reading comprehension
and question-answering. We believe that the scale and the real-world nature of
this dataset makes it attractive for benchmarking machine reading comprehension
and question-answering models.
The empirical size of trained neural networks
Authors: Kevin K. Chen, Anthony Gamst, Alden Walker
Date: 2016-11-29 00:00:00

ReLU neural networks define piecewise linear functions of their inputs.
However, initializing and training a neural network is very different from
fitting a linear spline. In this paper, we expand empirically upon previous
theoretical work to demonstrate features of trained neural networks. Standard
network initialization and training produce networks vastly simpler than a
naive parameter count would suggest and can impart odd features to the trained
network. However, we also show the forced simplicity is beneficial and, indeed,
critical for the wide success of these networks.
Sentiment Analysis for Twitter : Going Beyond Tweet Text
Authors: Lahari Poddar, Kishaloy Halder, Xianyan Jia
Date: 2016-11-29 00:00:00
Tasks: Sentiment Analysis

Analysing sentiment of tweets is important as it helps to determine the
users' opinion. Knowing people's opinion is crucial for several purposes
starting from gathering knowledge about customer base, e-governance,
campaigning and many more. In this report, we aim to develop a system to detect
the sentiment from tweets. We employ several linguistic features along with
some other external sources of information to detect the sentiment of a tweet.
We show that augmenting the 140 character-long tweet with information harvested
from external urls shared in the tweet as well as Social Media features
enhances the sentiment prediction accuracy significantly.
Easy-setup eye movement recording system for human-computer interaction
Authors: Manh Duong Phung, Quang Vinh Tran, Kenji Hara, Hirohito Inagaki, Masanobu Abe
Date: 2016-11-28 00:00:00
Tasks: Position

Tracking the movement of human eyes is expected to yield natural and
convenient applications based on human-computer interaction (HCI). To implement
an effective eye-tracking system, eye movements must be recorded without
placing any restriction on the user's behavior or user discomfort. This paper
describes an eye movement recording system that offers free-head, simple
configuration. It does not require the user to wear anything on her head, and
she can move her head freely. Instead of using a computer, the system uses a
visual digital signal processor (DSP) camera to detect the position of eye
corner, the center of pupil and then calculate the eye movement. Evaluation
tests show that the sampling rate of the system can be 300 Hz and the accuracy
is about 1.8 degree/s.
An End-to-End Architecture for Keyword Spotting and Voice Activity Detection
Authors: Chris Lengerich, Awni Hannun
Date: 2016-11-28 00:00:00
Tasks: Action Detection, Activity Detection, General Classification, Keyword Spotting

We propose a single neural network architecture for two tasks: on-line
keyword spotting and voice activity detection. We develop novel inference
algorithms for an end-to-end Recurrent Neural Network trained with the
Connectionist Temporal Classification loss function which allow our model to
achieve high accuracy on both keyword spotting and voice activity detection
without retraining. In contrast to prior voice activity detection models, our
architecture does not require aligned training data and uses the same
parameters as the keyword spotting model. This allows us to deploy a high
quality voice activity detector with no additional memory or maintenance
requirements.
Open-Vocabulary Semantic Parsing with both Distributional Statistics and Formal Knowledge
Authors: Matt Gardner, Jayant Krishnamurthy
Date: 2016-07-12 00:00:00
Tasks: Question Answering, Semantic Parsing

Traditional semantic parsers map language onto compositional, executable
queries in a fixed schema. This mapping allows them to effectively leverage the
information contained in large, formal knowledge bases (KBs, e.g., Freebase) to
answer questions, but it is also fundamentally limiting---these semantic
parsers can only assign meaning to language that falls within the KB's
manually-produced schema. Recently proposed methods for open vocabulary
semantic parsing overcome this limitation by learning execution models for
arbitrary language, essentially using a text corpus as a kind of knowledge
base. However, all prior approaches to open vocabulary semantic parsing replace
a formal KB with textual information, making no use of the KB in their models.
We show how to combine the disparate representations used by these two
approaches, presenting for the first time a semantic parser that (1) produces
compositional, executable representations of language, (2) can successfully
leverage the information contained in both a formal KB and a large corpus, and
(3) is not limited to the schema of the underlying KB. We demonstrate
significantly improved performance over state-of-the-art baselines on an
open-domain natural language question answering task.