pwc-archive/papers-with-abstracts huggingface dataset:
FastDeRain: A Novel Video Rain Streak Removal Method Using Directional Gradient Priors
Authors: Tai-Xiang Jiang, Ting-Zhu Huang, Xi-Le Zhao, Liang-Jian Deng, Yao Wang
Date: 2018-03-20 00:00:00

Rain streak removal is an important issue in outdoor vision systems and has
recently been investigated extensively. In this paper, we propose a novel video
rain streak removal approach FastDeRain, which fully considers the
discriminative characteristics of rain streaks and the clean video in the
gradient domain. Specifically, on the one hand, rain streaks are sparse and
smooth along the direction of the raindrops, whereas on the other hand, clean
videos exhibit piecewise smoothness along the rain-perpendicular direction and
continuity along the temporal direction. Theses smoothness and continuity
results in the sparse distribution in the different directional gradient
domain, respectively. Thus, we minimize 1) the $\ell_1$ norm to enhance the
sparsity of the underlying rain streaks, 2) two $\ell_1$ norm of unidirectional
Total Variation (TV) regularizers to guarantee the anisotropic spatial
smoothness, and 3) an $\ell_1$ norm of the time-directional difference operator
to characterize the temporal continuity. A split augmented Lagrangian shrinkage
algorithm (SALSA) based algorithm is designed to solve the proposed
minimization model. Experiments conducted on synthetic and real data
demonstrate the effectiveness and efficiency of the proposed method. According
to comprehensive quantitative performance measures, our approach outperforms
other state-of-the-art methods especially on account of the running time. The
code of FastDeRain can be downloaded at
https://github.com/TaiXiangJiang/FastDeRain.
Visual Data Synthesis via GAN for Zero-Shot Video Classification
Authors: Chenrui Zhang, Yuxin Peng
Date: 2018-04-26 00:00:00
Tasks: Classification, General Classification, Video Classification, Zero-Shot Learning

Zero-Shot Learning (ZSL) in video classification is a promising research
direction, which aims to tackle the challenge from explosive growth of video
categories. Most existing methods exploit seen-to-unseen correlation via
learning a projection between visual and semantic spaces. However, such
projection-based paradigms cannot fully utilize the discriminative information
implied in data distribution, and commonly suffer from the information
degradation issue caused by "heterogeneity gap". In this paper, we propose a
visual data synthesis framework via GAN to address these problems.
Specifically, both semantic knowledge and visual distribution are leveraged to
synthesize video feature of unseen categories, and ZSL can be turned into
typical supervised problem with the synthetic features. First, we propose
multi-level semantic inference to boost video feature synthesis, which captures
the discriminative information implied in joint visual-semantic distribution
via feature-level and label-level semantic inference. Second, we propose
Matching-aware Mutual Information Correlation to overcome information
degradation issue, which captures seen-to-unseen correlation in matched and
mismatched visual-semantic pairs by mutual information, providing the zero-shot
synthesis procedure with robust guidance signals. Experimental results on four
video datasets demonstrate that our approach can improve the zero-shot video
classification performance significantly.
Temporally Identity-Aware SSD with Attentional LSTM
Authors: Xingyu Chen, Junzhi Yu, Zhengxing Wu
Date: 2018-03-01 00:00:00
Tasks: object-detection, Object Detection

Temporal object detection has attracted significant attention, but most popular detection methods cannot leverage rich temporal information in videos. Very recently, many algorithms have been developed for video detection task, yet very few approaches can achieve \emph{real-time online} object detection in videos. In this paper, based on attention mechanism and convolutional long short-term memory (ConvLSTM), we propose a temporal single-shot detector (TSSD) for real-world detection. Distinct from previous methods, we take aim at temporally integrating pyramidal feature hierarchy using ConvLSTM, and design a novel structure including a low-level temporal unit as well as a high-level one (LH-TU) for multi-scale feature maps. Moreover, we develop a creative temporal analysis unit, namely, attentional ConvLSTM (AC-LSTM), in which a temporal attention mechanism is specially tailored for background suppression and scale suppression while a ConvLSTM integrates attention-aware features across time. An association loss and a multi-step training are designed for temporal coherence. Besides, an online tubelet analysis (OTA) is exploited for identification. Our framework is evaluated on ImageNet VID dataset and 2DMOT15 dataset. Extensive comparisons on the detection and tracking capability validate the superiority of the proposed approach. Consequently, the developed TSSD-OTA achieves a fast speed and an overall competitive performance in terms of detection and tracking. Finally, a real-world maneuver is conducted for underwater object grasping. The source code is publicly available at https://github.com/SeanChenxy/TSSD-OTA.
Adaptive pooling operators for weakly labeled sound event detection
Authors: Brian McFee, Justin Salamon, Juan Pablo Bello
Date: 2018-04-26 00:00:00
Tasks: Event Detection, Multiple Instance Learning, Sound Event Detection, Time Series Analysis

Sound event detection (SED) methods are tasked with labeling segments of
audio recordings by the presence of active sound sources. SED is typically
posed as a supervised machine learning problem, requiring strong annotations
for the presence or absence of each sound source at every time instant within
the recording. However, strong annotations of this type are both labor- and
cost-intensive for human annotators to produce, which limits the practical
scalability of SED methods.
  In this work, we treat SED as a multiple instance learning (MIL) problem,
where training labels are static over a short excerpt, indicating the presence
or absence of sound sources but not their temporal locality. The models,
however, must still produce temporally dynamic predictions, which must be
aggregated (pooled) when comparing against static labels during training. To
facilitate this aggregation, we develop a family of adaptive pooling
operators---referred to as auto-pool---which smoothly interpolate between
common pooling operators, such as min-, max-, or average-pooling, and
automatically adapt to the characteristics of the sound sources in question. We
evaluate the proposed pooling operators on three datasets, and demonstrate that
in each case, the proposed methods outperform non-adaptive pooling operators
for static prediction, and nearly match the performance of models trained with
strong, dynamic annotations. The proposed method is evaluated in conjunction
with convolutional neural networks, but can be readily applied to any
differentiable model for time-series label prediction.
Better and Faster: Knowledge Transfer from Multiple Self-supervised Learning Tasks via Graph Distillation for Video Classification
Authors: Chenrui Zhang, Yuxin Peng
Date: 2018-04-26 00:00:00
Tasks: Classification, General Classification, Representation Learning, Self-Supervised Learning, Transfer Learning, Video Classification

Video representation learning is a vital problem for classification task.
Recently, a promising unsupervised paradigm termed self-supervised learning has
emerged, which explores inherent supervisory signals implied in massive data
for feature learning via solving auxiliary tasks. However, existing methods in
this regard suffer from two limitations when extended to video classification.
First, they focus only on a single task, whereas ignoring complementarity among
different task-specific features and thus resulting in suboptimal video
representation. Second, high computational and memory cost hinders their
application in real-world scenarios. In this paper, we propose a graph-based
distillation framework to address these problems: (1) We propose logits graph
and representation graph to transfer knowledge from multiple self-supervised
tasks, where the former distills classifier-level knowledge by solving a
multi-distribution joint matching problem, and the latter distills internal
feature knowledge from pairwise ensembled representations with tackling the
challenge of heterogeneity among different features; (2) The proposal that
adopts a teacher-student framework can reduce the redundancy of knowledge
learnt from teachers dramatically, leading to a lighter student model that
solves classification task more efficiently. Experimental results on 3 video
datasets validate that our proposal not only helps learn better video
representation but also compress model for faster inference.
End-to-End Speech Separation with Unfolded Iterative Phase Reconstruction
Authors: Zhong-Qiu Wang, Jonathan Le Roux, DeLiang Wang, John R. Hershey
Date: 2018-04-26 00:00:00
Tasks: Speech Separation

This paper proposes an end-to-end approach for single-channel
speaker-independent multi-speaker speech separation, where time-frequency (T-F)
masking, the short-time Fourier transform (STFT), and its inverse are
represented as layers within a deep network. Previous approaches, rather than
computing a loss on the reconstructed signal, used a surrogate loss based on
the target STFT magnitudes. This ignores reconstruction error introduced by
phase inconsistency. In our approach, the loss function is directly defined on
the reconstructed signals, which are optimized for best separation. In
addition, we train through unfolded iterations of a phase reconstruction
algorithm, represented as a series of STFT and inverse STFT layers. While mask
values are typically limited to lie between zero and one for approaches using
the mixture phase for reconstruction, this limitation is less relevant if the
estimated magnitudes are to be used together with phase reconstruction. We thus
propose several novel activation functions for the output layer of the T-F
masking, to allow mask values beyond one. On the publicly-available wsj0-2mix
dataset, our approach achieves state-of-the-art 12.6 dB scale-invariant
signal-to-distortion ratio (SI-SDR) and 13.1 dB SDR, revealing new
possibilities for deep learning based phase reconstruction and representing a
fundamental progress towards solving the notoriously-hard cocktail party
problem.
Using Machine Learning to Improve Cylindrical Algebraic Decomposition
Authors: Zongyan Huang, Matthew England, David Wilson, James H. Davenport, Lawrence C. Paulson
Date: 2018-04-26 00:00:00
Tasks: BIG-bench Machine Learning

Cylindrical Algebraic Decomposition (CAD) is a key tool in computational
algebraic geometry, best known as a procedure to enable Quantifier Elimination
over real-closed fields. However, it has a worst case complexity doubly
exponential in the size of the input, which is often encountered in practice.
It has been observed that for many problems a change in algorithm settings or
problem formulation can cause huge differences in runtime costs, changing
problem instances from intractable to easy. A number of heuristics have been
developed to help with such choices, but the complicated nature of the
geometric relationships involved means these are imperfect and can sometimes
make poor choices. We investigate the use of machine learning (specifically
support vector machines) to make such choices instead.
  Machine learning is the process of fitting a computer model to a complex
function based on properties learned from measured data. In this paper we apply
it in two case studies: the first to select between heuristics for choosing a
CAD variable ordering; the second to identify when a CAD problem instance would
benefit from Groebner Basis preconditioning. These appear to be the first such
applications of machine learning to Symbolic Computation. We demonstrate in
both cases that the machine learned choice outperforms human developed
heuristics.
Decentralized learning with budgeted network load using Gaussian copulas and classifier ensembles
Authors: John Klein, Mahmoud Albardan, Benjamin Guedj, Olivier Colot
Date: 2018-04-26 00:00:00
Tasks: Ensemble Learning, General Classification

We examine a network of learners which address the same classification task but must learn from different data sets. The learners cannot share data but instead share their models. Models are shared only one time so as to preserve the network load. We introduce DELCO (standing for Decentralized Ensemble Learning with COpulas), a new approach allowing to aggregate the predictions of the classifiers trained by each learner. The proposed method aggregates the base classifiers using a probabilistic model relying on Gaussian copulas. Experiments on logistic regressor ensembles demonstrate competing accuracy and increased robustness in case of dependent classifiers. A companion python implementation can be downloaded at https://github.com/john-klein/DELCO
Extended Vertical Lists for Temporal Pattern Mining from Multivariate Time Series
Authors: Anton Kocheturov, Petar Momcilovic, Azra Bihorac, Panos M. Pardalos
Date: 2018-04-26 00:00:00
Tasks: Time Series, Time Series Analysis

Temporal Pattern Mining (TPM) is the problem of mining predictive complex
temporal patterns from multivariate time series in a supervised setting. We
develop a new method called the Fast Temporal Pattern Mining with Extended
Vertical Lists. This method utilizes an extension of the Apriori property which
requires a more complex pattern to appear within records only at places where
all of its subpatterns are detected as well. The approach is based on a novel
data structure called the Extended Vertical List that tracks positions of the
first state of the pattern inside records. Extensive computational results
indicate that the new method performs significantly faster than the previous
version of the algorithm for TMP. However, the speed-up comes at the expense of
memory usage.
Candidate Labeling for Crowd Learning
Authors: Iker Beñaran-Muñoz, Jerónimo Hernández-González, Aritz Pérez
Date: 2018-04-26 00:00:00

Crowdsourcing has become very popular among the machine learning community as
a way to obtain labels that allow a ground truth to be estimated for a given
dataset. In most of the approaches that use crowdsourced labels, annotators are
asked to provide, for each presented instance, a single class label. Such a
request could be inefficient, that is, considering that the labelers may not be
experts, that way to proceed could fail to take real advantage of the knowledge
of the labelers. In this paper, the use of candidate labeling for crowd
learning is proposed, where the annotators may provide more than a single label
per instance to try not to miss the real label. The main hypothesis is that, by
allowing candidate labeling, knowledge can be extracted from the labelers more
efficiently by than in the standard crowd learning scenario. Empirical evidence
which supports that hypothesis is presented.
Deep Keyframe Detection in Human Action Videos
Authors: Xiang Yan, Syed Zulqarnain Gilani, Hanlin Qin, Mingtao Feng, Liang Zhang, Ajmal Mian
Date: 2018-04-26 00:00:00

Detecting representative frames in videos based on human actions is quite
challenging because of the combined factors of human pose in action and the
background. This paper addresses this problem and formulates the key frame
detection as one of finding the video frames that optimally maximally
contribute to differentiating the underlying action category from all other
categories. To this end, we introduce a deep two-stream ConvNet for key frame
detection in videos that learns to directly predict the location of key frames.
Our key idea is to automatically generate labeled data for the CNN learning
using a supervised linear discriminant method. While the training data is
generated taking many different human action videos into account, the trained
CNN can predict the importance of frames from a single video. We specify a new
ConvNet framework, consisting of a summarizer and discriminator. The summarizer
is a two-stream ConvNet aimed at, first, capturing the appearance and motion
features of video frames, and then encoding the obtained appearance and motion
features for video representation. The discriminator is a fitting function
aimed at distinguishing between the key frames and others in the video. We
conduct experiments on a challenging human action dataset UCF101 and show that
our method can detect key frames with high accuracy.
Joint Deformable Registration of Large EM Image Volumes: A Matrix Solver Approach
Authors: Khaled Khairy, Gennady Denisov, Stephan Saalfeld
Date: 2018-04-26 00:00:00
Tasks: Image Registration

Large electron microscopy image datasets for connectomics are typically
composed of thousands to millions of partially overlapping two-dimensional
images (tiles), which must be registered into a coherent volume prior to
further analysis. A common registration strategy is to find matching features
between neighboring and overlapping image pairs, followed by a numerical
estimation of optimal image deformation using a so-called solver program.
  Existing solvers are inadequate for large data volumes, and inefficient for
small-scale image registration.
  In this work, an efficient and accurate matrix-based solver method is
presented. A linear system is constructed that combines minimization of
feature-pair square distances with explicit constraints in a regularization
term. In absence of reliable priors for regularization, we show how to
construct a rigid-model approximation to use as prior. The linear system is
solved using available computer programs, whose performance on typical
registration tasks we briefly compare, and to which future scale-up is
delegated. Our method is applied to the joint alignment of 2.67 million images,
with more than 200 million point-pairs and has been used for successfully
aligning the first full adult fruit fly brain.
Fine-grained Visual-textual Representation Learning
Authors: Xiangteng He, Yuxin Peng
Date: 2017-08-31 00:00:00
Tasks: Fine-Grained Visual Categorization, Representation Learning

Fine-grained visual categorization is to recognize hundreds of subcategories
belonging to the same basic-level category, which is a highly challenging task
due to the quite subtle and local visual distinctions among similar
subcategories. Most existing methods generally learn part detectors to discover
discriminative regions for better categorization performance. However, not all
parts are beneficial and indispensable for visual categorization, and the
setting of part detector number heavily relies on prior knowledge as well as
experimental validation. As is known to all, when we describe the object of an
image via textual descriptions, we mainly focus on the pivotal characteristics,
and rarely pay attention to common characteristics as well as the background
areas. This is an involuntary transfer from human visual attention to textual
attention, which leads to the fact that textual attention tells us how many and
which parts are discriminative and significant to categorization. So textual
attention could help us to discover visual attention in image. Inspired by
this, we propose a fine-grained visual-textual representation learning (VTRL)
approach, and its main contributions are: (1) Fine-grained visual-textual
pattern mining devotes to discovering discriminative visual-textual pairwise
information for boosting categorization performance through jointly modeling
vision and text with generative adversarial networks (GANs), which
automatically and adaptively discovers discriminative parts. (2) Visual-textual
representation learning jointly combines visual and textual information, which
preserves the intra-modality and inter-modality information to generate
complementary fine-grained representation, as well as further improves
categorization performance.
Force Estimation from OCT Volumes using 3D CNNs
Authors: Nils Gessert, Jens Beringhoff, Christoph Otte, Alexander Schlaefer
Date: 2018-04-26 00:00:00
Tasks: Friction

\textit{Purpose} Estimating the interaction forces of instruments and tissue
is of interest, particularly to provide haptic feedback during robot assisted
minimally invasive interventions. Different approaches based on external and
integrated force sensors have been proposed. These are hampered by friction,
sensor size, and sterilizability. We investigate a novel approach to estimate
the force vector directly from optical coherence tomography image volumes.
  \textit{Methods} We introduce a novel Siamese 3D CNN architecture. The
network takes an undeformed reference volume and a deformed sample volume as an
input and outputs the three components of the force vector. We employ a deep
residual architecture with bottlenecks for increased efficiency. We compare the
Siamese approach to methods using difference volumes and two-dimensional
projections. Data was generated using a robotic setup to obtain ground truth
force vectors for silicon tissue phantoms as well as porcine tissue.
  \textit{Results} Our method achieves a mean average error of 7.7 +- 4.3 mN
when estimating the force vector. Our novel Siamese 3D CNN architecture
outperforms single-path methods that achieve a mean average error of 11.59 +-
6.7 mN. Moreover, the use of volume data leads to significantly higher
performance compared to processing only surface information which achieves a
mean average error of 24.38 +- 22.0 mN. Based on the tissue dataset, our
methods shows good generalization in between different subjects.
  \textit{Conclusions} We propose a novel image-based force estimation method
using optical coherence tomography. We illustrate that capturing the
deformation of subsurface structures substantially improves force estimation.
Our approach can provide accurate force estimates in surgical setups when using
intraoperative optical coherence tomography.
Profile-guided memory optimization for deep neural networks
Authors: Taro Sekiyama, Takashi Imamichi, Haruki Imai, Rudy Raymond
Date: 2018-04-26 00:00:00

Recent years have seen deep neural networks (DNNs) becoming wider and deeper
to achieve better performance in many applications of AI. Such DNNs however
require huge amounts of memory to store weights and intermediate results (e.g.,
activations, feature maps, etc.) in propagation. This requirement makes it
difficult to run the DNNs on devices with limited, hard-to-extend memory,
degrades the running time performance, and restricts the design of network
models. We address this challenge by developing a novel profile-guided memory
optimization to efficiently and quickly allocate memory blocks during the
propagation in DNNs. The optimization utilizes a simple and fast heuristic
algorithm based on the two-dimensional rectangle packing problem. Experimenting
with well-known neural network models, we confirm that our method not only
reduces the memory consumption by up to $49.5\%$ but also accelerates training
and inference by up to a factor of four thanks to the rapidity of the memory
allocation and the ability to use larger mini-batch sizes.