pwc-archive/papers-with-abstracts huggingface dataset:
Deep Fluids: A Generative Network for Parameterized Fluid Simulations
Authors: Byung-soo Kim, Vinicius C. Azevedo, Nils Thuerey, Theodore Kim, Markus Gross, Barbara Solenthaler
Date: 2018-06-06 00:00:00
Tasks: CPU

This paper presents a novel generative model to synthesize fluid simulations
from a set of reduced parameters. A convolutional neural network is trained on
a collection of discrete, parameterizable fluid simulation velocity fields. Due
to the capability of deep learning architectures to learn representative
features of the data, our generative model is able to accurately approximate
the training data set, while providing plausible interpolated in-betweens. The
proposed generative model is optimized for fluids by a novel loss function that
guarantees divergence-free velocity fields at all times. In addition, we
demonstrate that we can handle complex parameterizations in reduced spaces, and
advance simulations in time by integrating in the latent space with a second
network. Our method models a wide variety of fluid behaviors, thus enabling
applications such as fast construction of simulations, interpolation of fluids
with different parameters, time re-sampling, latent space simulations, and
compression of fluid simulation data. Reconstructed velocity fields are
generated up to 700x faster than re-simulating the data with the underlying CPU
solver, while achieving compression rates of up to 1300x.
Predicting the Driver's Focus of Attention: the DR(eye)VE Project
Authors: Andrea Palazzi, Davide Abati, Simone Calderara, Francesco Solera, Rita Cucchiara
Date: 2017-05-10 00:00:00

In this work we aim to predict the driver's focus of attention. The goal is
to estimate what a person would pay attention to while driving, and which part
of the scene around the vehicle is more critical for the task. To this end we
propose a new computer vision model based on a multi-branch deep architecture
that integrates three sources of information: raw video, motion and scene
semantics. We also introduce DR(eye)VE, the largest dataset of driving scenes
for which eye-tracking annotations are available. This dataset features more
than 500,000 registered frames, matching ego-centric views (from glasses worn
by drivers) and car-centric views (from roof-mounted camera), further enriched
by other sensors measurements. Results highlight that several attention
patterns are shared across drivers and can be reproduced to some extent. The
indication of which elements in the scene are likely to capture the driver's
attention may benefit several applications in the context of human-vehicle
interaction and driver attention analysis.
PieAPP: Perceptual Image-Error Assessment through Pairwise Preference
Authors: Ekta Prashnani, Hong Cai, Yasamin Mostofi, Pradeep Sen
Date: 2018-06-06 00:00:00
Proceeding: CVPR 2018 6
Tasks: Video Quality Assessment

The ability to estimate the perceptual error between images is an important
problem in computer vision with many applications. Although it has been studied
extensively, however, no method currently exists that can robustly predict
visual differences like humans. Some previous approaches used hand-coded
models, but they fail to model the complexity of the human visual system.
Others used machine learning to train models on human-labeled datasets, but
creating large, high-quality datasets is difficult because people are unable to
assign consistent error labels to distorted images. In this paper, we present a
new learning-based method that is the first to predict perceptual image error
like human observers. Since it is much easier for people to compare two given
images and identify the one more similar to a reference than to assign quality
scores to each, we propose a new, large-scale dataset labeled with the
probability that humans will prefer one image over another. We then train a
deep-learning model using a novel, pairwise-learning framework to predict the
preference of one distorted image over the other. Our key observation is that
our trained network can then be used separately with only one distorted image
and a reference to predict its perceptual error, without ever being trained on
explicit human perceptual-error labels. The perceptual error estimated by our
new metric, PieAPP, is well-correlated with human opinion. Furthermore, it
significantly outperforms existing algorithms, beating the state-of-the-art by
almost 3x on our test set in terms of binary error rate, while also
generalizing to new kinds of distortions, unlike previous learning-based
methods.
Explicit Inductive Bias for Transfer Learning with Convolutional Networks
Authors: Xuhong Li, Yves GRANDVALET, Franck DAVOINE
Date: 2018-02-05 00:00:00
Proceeding: ICML 2018 7
Tasks: Inductive Bias, Transfer Learning

In inductive transfer learning, fine-tuning pre-trained convolutional
networks substantially outperforms training from scratch. When using
fine-tuning, the underlying assumption is that the pre-trained model extracts
generic features, which are at least partially relevant for solving the target
task, but would be difficult to extract from the limited amount of data
available on the target task. However, besides the initialization with the
pre-trained model and the early stopping, there is no mechanism in fine-tuning
for retaining the features learned on the source task. In this paper, we
investigate several regularization schemes that explicitly promote the
similarity of the final solution with the initial model. We show the benefit of
having an explicit inductive bias towards the initial model, and we eventually
recommend a simple $L^2$ penalty with the pre-trained model being a reference
as the baseline of penalty for transfer learning tasks.
Training Generative Reversible Networks
Authors: Robin Tibor Schirrmeister, Patryk Chrabąszcz, Frank Hutter, Tonio Ball
Date: 2018-06-05 00:00:00
Tasks: Decoder

Generative models with an encoding component such as autoencoders currently
receive great interest. However, training of autoencoders is typically
complicated by the need to train a separate encoder and decoder model that have
to be enforced to be reciprocal to each other. To overcome this problem,
by-design reversible neural networks (RevNets) had been previously used as
generative models either directly optimizing the likelihood of the data under
the model or using an adversarial approach on the generated data. Here, we
instead investigate their performance using an adversary on the latent space in
the adversarial autoencoder framework. We investigate the generative
performance of RevNets on the CelebA dataset, showing that generative RevNets
can generate coherent faces with similar quality as Variational Autoencoders.
This first attempt to use RevNets inside the adversarial autoencoder framework
slightly underperformed relative to recent advanced generative models using an
autoencoder component on CelebA, but this gap may diminish with further
optimization of the training setup of generative RevNets. In addition to the
experiments on CelebA, we show a proof-of-principle experiment on the MNIST
dataset suggesting that adversary-free trained RevNets can discover meaningful
latent dimensions without pre-specifying the number of dimensions of the latent
sampling distribution. In summary, this study shows that RevNets can be
employed in different generative training settings.
  Source code for this study is at
https://github.com/robintibor/generative-reversible
Spatial Frequency Loss for Learning Convolutional Autoencoders
Authors: Naoyuki Ichimura
Date: 2018-06-06 00:00:00
Tasks: object-detection, Object Detection

This paper presents a learning method for convolutional autoencoders (CAEs)
for extracting features from images. CAEs can be obtained by utilizing
convolutional neural networks to learn an approximation to the identity
function in an unsupervised manner. The loss function based on the pixel loss
(PL) that is the mean squared error between the pixel values of original and
reconstructed images is the common choice for learning. However, using the loss
function leads to blurred reconstructed images. A method for learning CAEs
using a loss function computed from features reflecting spatial frequencies is
proposed to mitigate the problem. The blurs in reconstructed images show lack
of high spatial frequency components mainly constituting edges and detailed
textures that are important features for tasks such as object detection and
spatial matching. In order to evaluate the lack of components, a convolutional
layer with a Laplacian filter bank as weights is added to CAEs and the mean
squared error of features in a subband, called the spatial frequency loss
(SFL), is computed from the outputs of each filter. The learning is performed
using a loss function based on the SFL. Empirical evaluation demonstrates that
using the SFL reduces the blurs in reconstructed images.
Learning Hierarchical Item Categories from Implicit Feedback Data for Efficient Recommendations and Browsing
Authors: Farhan Khawar, Nevin L. Zhang
Date: 2018-06-06 00:00:00
Tasks: Diversity

Searching, browsing, and recommendations are common ways in which the "choice overload" faced by users in the online marketplace can be mitigated. In this paper we propose the use of hierarchical item categories, obtained from implicit feedback data, to enable efficient browsing and recommendations. We present a method of creating hierarchical item categories from implicit feedback data only i.e., without any other information on the items like name, genre etc. Categories created in this fashion are based on users' co-consumption of items. Thus, they can be more useful for users in finding interesting and relevant items while they are browsing through the hierarchy. We also show that this item hierarchy can be useful in making category based recommendations, which makes the recommendations more explainable and increases the diversity of the recommendations without compromising much on the accuracy. Item hierarchy can also be useful in the creation of an automatic item taxonomy skeleton by bypassing manual labeling and annotation. This can especially be useful for small vendors. Our data-driven hierarchical categories are based on hierarchical latent tree analysis (HLTA) which has been previously used for text analysis. We present a scaled up learning algorithm \emph{HLTA-Forest} so that HLTA can be applied to implicit feedback data.
Why rankings of biomedical image analysis competitions should be interpreted with care
Authors: Lena Maier-Hein, Matthias Eisenmann, Annika Reinke, Sinan Onogur, Marko Stankovic, Patrick Scholz, Tal Arbel, Hrvoje Bogunovic, Andrew P. Bradley, Aaron Carass, Carolin Feldmann, Alejandro F. Frangi, Peter M. Full, Bram van Ginneken, Allan Hanbury, Katrin Honauer, Michal Kozubek, Bennett A. Landman, Keno März, Oskar Maier, Klaus Maier-Hein, Bjoern H. Menze, Henning Müller, Peter F. Neher, Wiro Niessen, Nasir Rajpoot, Gregory C. Sharp, Korsuk Sirinukunwattana, Stefanie Speidel, Christian Stock, Danail Stoyanov, Abdel Aziz Taha, Fons van der Sommen, Ching-Wei Wang, Marc-André Weber, Guoyan Zheng, Pierre Jannin, Annette Kopp-Schneider
Date: 2018-06-06 00:00:00

International challenges have become the standard for validation of biomedical image analysis methods. Given their scientific impact, it is surprising that a critical analysis of common practices related to the organization of challenges has not yet been performed. In this paper, we present a comprehensive analysis of biomedical image analysis challenges conducted up to now. We demonstrate the importance of challenges and show that the lack of quality control has critical consequences. First, reproducibility and interpretation of the results is often hampered as only a fraction of relevant information is typically provided. Second, the rank of an algorithm is generally not robust to a number of variables such as the test data used for validation, the ranking scheme applied and the observers that make the reference annotations. To overcome these problems, we recommend best practice guidelines and define open research questions to be addressed in the future.
Implicit regularization and solution uniqueness in over-parameterized matrix sensing
Authors: Kelly Geyer, Anastasios Kyrillidis, Amir Kalev
Date: 2018-06-06 00:00:00

We consider whether algorithmic choices in over-parameterized linear matrix factorization introduce implicit regularization. We focus on noiseless matrix sensing over rank-$r$ positive semi-definite (PSD) matrices in $\mathbb{R}^{n \times n}$, with a sensing mechanism that satisfies restricted isometry properties (RIP). The algorithm we study is \emph{factored gradient descent}, where we model the low-rankness and PSD constraints with the factorization $UU^\top$, for $U \in \mathbb{R}^{n \times r}$. Surprisingly, recent work argues that the choice of $r \leq n$ is not pivotal: even setting $U \in \mathbb{R}^{n \times n}$ is sufficient for factored gradient descent to find the rank-$r$ solution, which suggests that operating over the factors leads to an implicit regularization. In this contribution, we provide a different perspective to the problem of implicit regularization. We show that under certain conditions, the PSD constraint by itself is sufficient to lead to a unique rank-$r$ matrix recovery, without implicit or explicit low-rank regularization. \emph{I.e.}, under assumptions, the set of PSD matrices, that are consistent with the observed data, is a singleton, regardless of the algorithm used.
Medical Concept Embedding with Time-Aware Attention
Authors: Xiangrui Cai, Jinyang Gao, Kee Yuan Ngiam, Beng Chin Ooi, Ying Zhang, Xiaojie Yuan
Date: 2018-06-06 00:00:00
Tasks: Clustering

Embeddings of medical concepts such as medication, procedure and diagnosis
codes in Electronic Medical Records (EMRs) are central to healthcare analytics.
Previous work on medical concept embedding takes medical concepts and EMRs as
words and documents respectively. Nevertheless, such models miss out the
temporal nature of EMR data. On the one hand, two consecutive medical concepts
do not indicate they are temporally close, but the correlations between them
can be revealed by the time gap. On the other hand, the temporal scopes of
medical concepts often vary greatly (e.g., \textit{common cold} and
\textit{diabetes}). In this paper, we propose to incorporate the temporal
information to embed medical codes. Based on the Continuous Bag-of-Words model,
we employ the attention mechanism to learn a "soft" time-aware context window
for each medical concept. Experiments on public and proprietary datasets
through clustering and nearest neighbour search tasks demonstrate the
effectiveness of our model, showing that it outperforms five state-of-the-art
baselines.
Simplex Search Based Brain Storm Optimization
Authors: Wei Chen, YingYing Cao, Shi Cheng, Yifei Sun, Qunfeng Liu, Yun Li
Date: 2017-10-24 00:00:00
Tasks: global-optimization

Through modeling human's brainstorming process, the brain storm optimization
(BSO) algorithm has become a promising population-based evolutionary algorithm.
However, BSO is pointed out that it possesses a degenerated L-curve phenomenon,
i.e., it often gets near optimum quickly but needs much more cost to improve
the accuracy. To overcome this question in this paper, an excellent direct
search based local solver, the Nelder-Mead Simplex (NMS) method is adopted in
BSO. Through combining BSO's exploration ability and NMS's exploitation ability
together, a simplex search based BSO (Simplex-BSO) is developed via a better
balance between global exploration and local exploitation. Simplex-BSO is shown
to be able to eliminate the degenerated L-curve phenomenon on unimodal
functions, and alleviate significantly this phenomenon on multimodal functions.
Large number of experimental results show that Simplex-BSO is a promising
algorithm for global optimization problems.
Degrees of Freedom and Model Selection for k-means Clustering
Authors: David P. Hofmeyr
Date: 2018-06-06 00:00:00
Tasks: Clustering, LEMMA, Model Selection

This paper investigates the model degrees of freedom in k-means clustering. An extension of Stein's lemma provides an expression for the effective degrees of freedom in the k-means model. Approximating the degrees of freedom in practice requires simplifications of this expression, however empirical studies evince the appropriateness of our proposed approach. The practical relevance of this new degrees of freedom formulation for k-means is demonstrated through model selection using the Bayesian Information Criterion. The reliability of this method is validated through experiments on simulated data as well as on a large collection of publicly available benchmark data sets from diverse application areas. Comparisons with popular existing techniques indicate that this approach is extremely competitive for selecting high quality clustering solutions. Code to implement the proposed approach is available in the form of an R package from https://github.com/DavidHofmeyr/edfkmeans.
Killing four birds with one Gaussian process: the relation between different test-time attacks
Authors: Kathrin Grosse, Michael T. Smith, Michael Backes
Date: 2018-06-06 00:00:00
Tasks: General Classification, Relation

In machine learning (ML) security, attacks like evasion, model stealing or membership inference are generally studied in individually. Previous work has also shown a relationship between some attacks and decision function curvature of the targeted model. Consequently, we study an ML model allowing direct control over the decision surface curvature: Gaussian Process classifiers (GPCs). For evasion, we find that changing GPC's curvature to be robust against one attack algorithm boils down to enabling a different norm or attack algorithm to succeed. This is backed up by our formal analysis showing that static security guarantees are opposed to learning. Concerning intellectual property, we show formally that lazy learning does not necessarily leak all information when applied. In practice, often a seemingly secure curvature can be found. For example, we are able to secure GPC against empirical membership inference by proper configuration. In this configuration, however, the GPC's hyper-parameters are leaked, e.g. model reverse engineering succeeds. We conclude that attacks on classification should not be studied in isolation, but in relation to each other.
Real-time Surgical Tools Recognition in Total Knee Arthroplasty Using Deep Neural Networks
Authors: Moazzem Hossain, Soichi Nishio, Takafumi Hiranaka, Syoji Kobashi
Date: 2018-06-06 00:00:00

Total knee arthroplasty (TKA) is a commonly performed surgical procedure to
mitigate knee pain and improve functions for people with knee arthritis. The
procedure is complicated due to the different surgical tools used in the stages
of surgery. The recognition of surgical tools in real-time can be a solution to
simplify surgical procedures for the surgeon. Also, the presence and movement
of tools in surgery are crucial information for the recognition of the
operational phase and to identify the surgical workflow. Therefore, this
research proposes the development of a real-time system for the recognition of
surgical tools during surgery using a convolutional neural network (CNN).
Surgeons wearing smart glasses can see essential information about tools during
surgery that may reduce the complication of the procedures. To evaluate the
performance of the proposed method, we calculated and compared the Mean Average
Precision (MAP) with state-of-the-art methods which are fast R-CNN and
deformable part models (DPM). We achieved 87.6% mAP which is better in
comparison to the existing methods. With the additional improvements of our
proposed method, it can be a future point of reference, also the baseline for
operational phase recognition.
ScreenerNet: Learning Self-Paced Curriculum for Deep Neural Networks
Authors: Tae-hoon Kim, Jonghyun Choi
Date: 2018-01-03 00:00:00
Tasks: Deep Reinforcement Learning, Q-Learning, Reinforcement Learning

We propose to learn a curriculum or a syllabus for supervised learning and
deep reinforcement learning with deep neural networks by an attachable deep
neural network, called ScreenerNet. Specifically, we learn a weight for each
sample by jointly training the ScreenerNet and the main network in an
end-to-end self-paced fashion. The ScreenerNet neither has sampling bias nor
requires to remember the past learning history. We show the networks augmented
with the ScreenerNet achieve early convergence with better accuracy than the
state-of-the-art curricular learning methods in extensive experiments using
three popular vision datasets such as MNIST, CIFAR10 and Pascal VOC2012, and a
Cart-pole task using Deep Q-learning. Moreover, the ScreenerNet can extend
other curriculum learning methods such as Prioritized Experience Replay (PER)
for further accuracy improvement.
Joint Estimation of Age and Gender from Unconstrained Face Images using Lightweight Multi-task CNN for Mobile Applications
Authors: Jia-Hong Lee, Yi-Ming Chan, Ting-Yen Chen, Chu-Song Chen
Date: 2018-06-06 00:00:00
Tasks: Age And Gender Classification, Classification, Gender Classification, General Classification

Automatic age and gender classification based on unconstrained images has
become essential techniques on mobile devices. With limited computing power,
how to develop a robust system becomes a challenging task. In this paper, we
present an efficient convolutional neural network (CNN) called lightweight
multi-task CNN for simultaneous age and gender classification. Lightweight
multi-task CNN uses depthwise separable convolution to reduce the model size
and save the inference time. On the public challenging Adience dataset, the
accuracy of age and gender classification is better than baseline multi-task
CNN methods.
A Peek Into the Hidden Layers of a Convolutional Neural Network Through a Factorization Lens
Authors: Uday Singh Saini, Evangelos E. Papalexakis
Date: 2018-06-06 00:00:00

Despite their increasing popularity and success in a variety of supervised
learning problems, deep neural networks are extremely hard to interpret and
debug: Given and already trained Deep Neural Net, and a set of test inputs, how
can we gain insight into how those inputs interact with different layers of the
neural network? Furthermore, can we characterize a given deep neural network
based on it's observed behavior on different inputs? In this paper we propose a
novel factorization based approach on understanding how different deep neural
networks operate. In our preliminary results, we identify fascinating patterns
that link the factorization rank (typically used as a measure of
interestingness in unsupervised data analysis) with how well or poorly the deep
network has been trained. Finally, our proposed approach can help provide
visual insights on how high-level. interpretable patterns of the network's
input behave inside the hidden layers of the deep network.
Deep Algorithms: designs for networks
Authors: Abhejit Rajagopal, Shivkumar Chandrasekaran, Hrushikesh N. Mhaskar
Date: 2018-06-06 00:00:00

A new design methodology for neural networks that is guided by traditional
algorithm design is presented. To prove our point, we present two heuristics
and demonstrate an algorithmic technique for incorporating additional weights
in their signal-flow graphs. We show that with training the performance of
these networks can not only exceed the performance of the initial network, but
can match the performance of more-traditional neural network architectures. A
key feature of our approach is that these networks are initialized with
parameters that provide a known performance threshold for the architecture on a
given task.