pwc-archive/papers-with-abstracts huggingface dataset:
Learning with Collaborative Neural Network Group by Reflection
Authors: Liyao Gao, Zehua Cheng
Date: 2019-01-08 00:00:00

For the present engineering of neural systems, the preparing of extensive
scale learning undertakings generally not just requires a huge neural system
with a mind boggling preparing process yet additionally troublesome discover a
clarification for genuine applications. In this paper, we might want to present
the Collaborative Neural Network Group (CNNG). CNNG is a progression of neural
systems that work cooperatively to deal with various errands independently in a
similar learning framework. It is advanced from a solitary neural system by
reflection. Along these lines, in light of various circumstances removed by the
calculation, the CNNG can perform diverse techniques when handling the
information. The examples of chose methodology can be seen by human to make
profound adapting more reasonable. In our execution, the CNNG is joined by a
few moderately little neural systems. We give a progression of examinations to
assess the execution of CNNG contrasted with other learning strategies. The
CNNG is able to get a higher accuracy with a much lower training cost. We can
reduce the error rate by 74.5% and reached the accuracy of 99.45% in MNIST with
three feedforward networks (4 layers) in one training epoch.
Adaptive Activity Monitoring with Uncertainty Quantification in Switching Gaussian Process Models
Authors: Randy Ardywibowo, Guang Zhao, Zhangyang Wang, Bobak Mortazavi, Shuai Huang, Xiaoning Qian
Date: 2019-01-08 00:00:00
Tasks: Activity Recognition, Gaussian Processes, Human Activity Recognition, Uncertainty Quantification

Emerging wearable sensors have enabled the unprecedented ability to
continuously monitor human activities for healthcare purposes. However, with so
many ambient sensors collecting different measurements, it becomes important
not only to maintain good monitoring accuracy, but also low power consumption
to ensure sustainable monitoring. This power-efficient sensing scheme can be
achieved by deciding which group of sensors to use at a given time, requiring
an accurate characterization of the trade-off between sensor energy usage and
the uncertainty in ignoring certain sensor signals while monitoring. To address
this challenge in the context of activity monitoring, we have designed an
adaptive activity monitoring framework. We first propose a switching Gaussian
process to model the observed sensor signals emitting from the underlying
activity states. To efficiently compute the Gaussian process model likelihood
and quantify the context prediction uncertainty, we propose a block circulant
embedding technique and use Fast Fourier Transforms (FFT) for inference. By
computing the Bayesian loss function tailored to switching Gaussian processes,
an adaptive monitoring procedure is developed to select features from available
sensors that optimize the trade-off between sensor power consumption and the
prediction performance quantified by state prediction entropy. We demonstrate
the effectiveness of our framework on the popular benchmark of UCI Human
Activity Recognition using Smartphones.
Richer and Deeper Supervision Network for Salient Object Detection
Authors: Sen Jia, Neil D. B. Bruce
Date: 2019-01-08 00:00:00
Tasks: Object, object-detection, Object Detection, RGB Salient Object Detection, Salient Object Detection

Recent Salient Object Detection (SOD) systems are mostly based on
Convolutional Neural Networks (CNNs). Specifically, Deeply Supervised Saliency
(DSS) system has shown it is very useful to add short connections to the
network and supervising on the side output. In this work, we propose a new SOD
system which aims at designing a more efficient and effective way to pass back
global information. Richer and Deeper Supervision (RDS) is applied to better
combine features from each side output without demanding much extra
computational space. Meanwhile, the backbone network used for SOD is normally
pre-trained on the object classification dataset, ImageNet. But the pre-trained
model has been trained on cropped images in order to only focus on
distinguishing features within the region of the object. But the ignored
background information is also significant in the task of SOD. We try to solve
this problem by introducing the training data designed for object detection. A
coarse global information is learned based on an entire image with its bounding
box before training on the SOD dataset. The large-scale of object images can
slightly improve the performance of SOD. Our experiment shows the proposed RDS
network achieves the state-of-the-art results on five public SOD datasets.
SNRA: A Spintronic Neuromorphic Reconfigurable Array for In-Circuit Training and Evaluation of Deep Belief Networks
Authors: Ramtin Zand, Ronald F. DeMara
Date: 2019-01-08 00:00:00

In this paper, a spintronic neuromorphic reconfigurable Array (SNRA) is
developed to fuse together power-efficient probabilistic and in-field
programmable deterministic computing during both training and evaluation phases
of restricted Boltzmann machines (RBMs). First, probabilistic spin logic
devices are used to develop an RBM realization which is adapted to construct
deep belief networks (DBNs) having one to three hidden layers of size 10 to 800
neurons each. Second, we design a hardware implementation for the contrastive
divergence (CD) algorithm using a four-state finite state machine capable of
unsupervised training in N+3 clocks where N denotes the number of neurons in
each RBM. The functionality of our proposed CD hardware implementation is
validated using ModelSim simulations. We synthesize the developed Verilog HDL
implementation of our proposed test/train control circuitry for various DBN
topologies where the maximal RBM dimensions yield resource utilization ranging
from 51 to 2,421 lookup tables (LUTs). Next, we leverage spin Hall effect
(SHE)-magnetic tunnel junction (MTJ) based non-volatile LUTs circuits as an
alternative for static random access memory (SRAM)-based LUTs storing the
deterministic logic configuration to form a reconfigurable fabric. Finally, we
compare the performance of our proposed SNRA with SRAM-based configurable
fabrics focusing on the area and power consumption induced by the LUTs used to
implement both CD and evaluation modes. The results obtained indicate more than
80% reduction in combined dynamic and static power dissipation, while achieving
at least 50% reduction in device count.
Interpretable CNNs for Object Classification
Authors: Quanshi Zhang, Xin Wang, Ying Nian Wu, Huilin Zhou, Song-Chun Zhu
Date: 2019-01-08 00:00:00
Tasks: Classification, General Classification, Object

This paper proposes a generic method to learn interpretable convolutional filters in a deep convolutional neural network (CNN) for object classification, where each interpretable filter encodes features of a specific object part. Our method does not require additional annotations of object parts or textures for supervision. Instead, we use the same training data as traditional CNNs. Our method automatically assigns each interpretable filter in a high conv-layer with an object part of a certain category during the learning process. Such explicit knowledge representations in conv-layers of CNN help people clarify the logic encoded in the CNN, i.e., answering what patterns the CNN extracts from an input image and uses for prediction. We have tested our method using different benchmark CNNs with various structures to demonstrate the broad applicability of our method. Experiments have shown that our interpretable filters are much more semantically meaningful than traditional filters.
Forecasting Granular Audience Size for Online Advertising
Authors: Ritwik Sinha, Dhruv Singal, Pranav Maneriker, Kushal Chawla, Yash Shrivastava, Deepak Pai, Atanu R. Sinha
Date: 2019-01-08 00:00:00
Tasks: Attribute, Time Series, Time Series Analysis

Orchestration of campaigns for online display advertising requires marketers
to forecast audience size at the granularity of specific attributes of web
traffic, characterized by the categorical nature of all attributes (e.g. {US,
Chrome, Mobile}). With each attribute taking many values, the very large
attribute combination set makes estimating audience size for any specific
attribute combination challenging. We modify Eclat, a frequent itemset mining
(FIM) algorithm, to accommodate categorical variables. For consequent frequent
and infrequent itemsets, we then provide forecasts using time series analysis
with conditional probabilities to aid approximation. An extensive simulation,
based on typical characteristics of audience data, is built to stress test our
modified-FIM approach. In two real datasets, comparison with baselines
including neural network models, shows that our method lowers computation time
of FIM for categorical data. On hold out samples we show that the proposed
forecasting method outperforms these baselines.
Morphological Networks for Image De-raining
Authors: Ranjan Mondal, Pulak Purkait, Sanchayan Santra, Bhabatosh Chanda
Date: 2019-01-08 00:00:00
Tasks: SSIM

Mathematical morphological methods have successfully been applied to filter
out (emphasize or remove) different structures of an image. However, it is
argued that these methods could be suitable for the task only if the type and
order of the filter(s) as well as the shape and size of operator kernel are
designed properly. Thus the existing filtering operators are problem (instance)
specific and are designed by the domain experts. In this work we propose a
morphological network that emulates classical morphological filtering
consisting of a series of erosion and dilation operators with trainable
structuring elements. We evaluate the proposed network for image de-raining
task where the SSIM and mean absolute error (MAE) loss corresponding to
predicted and ground-truth clean image is back-propagated through the network
to train the structuring elements. We observe that a single morphological
network can de-rain an image with any arbitrary shaped rain-droplets and
achieves similar performance with the contemporary CNNs for this task with a
fraction of trainable parameters (network size). The proposed morphological
network(MorphoN) is not designed specifically for de-raining and can readily be
applied to similar filtering / noise cleaning tasks. The source code can be
found here https://github.com/ranjanZ/2D-Morphological-Network
GILT: Generating Images from Long Text
Authors: Ori Bar El, Ori Licht, Netanel Yosephian
Date: 2019-01-08 00:00:00

Creating an image reflecting the content of a long text is a complex process
that requires a sense of creativity. For example, creating a book cover or a
movie poster based on their summary or a food image based on its recipe. In
this paper we present the new task of generating images from long text that
does not describe the visual content of the image directly. For this, we build
a system for generating high-resolution 256 $\times$ 256 images of food
conditioned on their recipes. The relation between the recipe text (without its
title) to the visual content of the image is vague, and the textual structure
of recipes is complex, consisting of two sections (ingredients and
instructions) both containing multiple sentences.
  We used the recipe1M dataset to train and evaluate our model that is based on
a the StackGAN-v2 architecture.
Contamination Attacks and Mitigation in Multi-Party Machine Learning
Authors: Jamie Hayes, Olga Ohrimenko
Date: 2019-01-08 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: BIG-bench Machine Learning

Machine learning is data hungry; the more data a model has access to in
training, the more likely it is to perform well at inference time. Distinct
parties may want to combine their local data to gain the benefits of a model
trained on a large corpus of data. We consider such a case: parties get access
to the model trained on their joint data but do not see each others individual
datasets. We show that one needs to be careful when using this multi-party
model since a potentially malicious party can taint the model by providing
contaminated data. We then show how adversarial training can defend against
such attacks by preventing the model from learning trends specific to
individual parties data, thereby also guaranteeing party-level membership
privacy.
Fair Algorithms for Clustering
Authors: Suman K. Bera, Deeparnab Chakrabarty, Nicolas J. Flores, Maryam Negahbani
Date: 2019-01-08 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Clustering, Fairness

We study the problem of finding low-cost Fair Clusterings in data where each data point may belong to many protected groups. Our work significantly generalizes the seminal work of Chierichetti et.al. (NIPS 2017) as follows. - We allow the user to specify the parameters that define fair representation. More precisely, these parameters define the maximum over- and minimum under-representation of any group in any cluster. - Our clustering algorithm works on any $\ell_p$-norm objective (e.g. $k$-means, $k$-median, and $k$-center). Indeed, our algorithm transforms any vanilla clustering solution into a fair one incurring only a slight loss in quality. - Our algorithm also allows individuals to lie in multiple protected groups. In other words, we do not need the protected groups to partition the data and we can maintain fairness across different groups simultaneously. Our experiments show that on established data sets, our algorithm performs much better in practice than what our theoretical results suggest.
Graphical model inference: Sequential Monte Carlo meets deterministic approximations
Authors: Fredrik Lindsten, Jouni Helske, Matti Vihola
Date: 2019-01-08 00:00:00
Proceeding: NeurIPS 2018 12

Approximate inference in probabilistic graphical models (PGMs) can be grouped
into deterministic methods and Monte-Carlo-based methods. The former can often
provide accurate and rapid inferences, but are typically associated with biases
that are hard to quantify. The latter enjoy asymptotic consistency, but can
suffer from high computational costs. In this paper we present a way of
bridging the gap between deterministic and stochastic inference. Specifically,
we suggest an efficient sequential Monte Carlo (SMC) algorithm for PGMs which
can leverage the output from deterministic inference methods. While generally
applicable, we show explicitly how this can be done with loopy belief
propagation, expectation propagation, and Laplace approximations. The resulting
algorithm can be viewed as a post-correction of the biases associated with
these methods and, indeed, numerical results show clear improvements over the
baseline deterministic methods as well as over "plain" SMC.
Genetic Algorithm based Multi-Objective Optimization of Solidification in Die Casting using Deep Neural Network as Surrogate Model
Authors: Shantanu Shahane, Narayana Aluru, Placid Ferreira, Shiv G Kapoor, Surya Pratap Vanka
Date: 2019-01-08 00:00:00

In this paper, a novel strategy of multi-objective optimization of die
casting is presented. The cooling of molten metal inside the mold is achieved
by passing a coolant, typically water through the cooling lines in the die.
Depending on the cooling line location, coolant flow rate and die geometry,
nonuniform temperatures are imposed on the molten metal at the mold wall. This
boundary condition along with the initial molten metal temperature affect the
product quality quantified in terms of micro-structure parameters and yield
strength. A finite volume based numerical solver is used to correlate the
inputs to outputs. The objective of this research is to estimate the initial
and wall temperatures so as to optimize the product quality. The non-dominated
sorting genetic algorithm (NSGA--II) which is popular for solving
multi-objective optimization problems is used. The number of function
evaluations required for NSGA--II can be of the order of millions and hence,
the finite volume solver cannot be used directly for optimization. Thus, a
neural network trained using the results from the numerical solver is used as a
surrogate model. Simplified versions of the actual problem are designed to
verify results of the genetic algorithm. An innovative local sensitivity based
approach is used to rank the final Pareto optimal solutions and choose a single
best design.
FastGRNN: A Fast, Accurate, Stable and Tiny Kilobyte Sized Gated Recurrent Neural Network
Authors: Aditya Kusupati, Manish Singh, Kush Bhatia, Ashish Kumar, Prateek Jain, Manik Varma
Date: 2019-01-08 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Action Classification, Language Modelling, Speech Recognition, Time Series Analysis, Time Series Classification

This paper develops the FastRNN and FastGRNN algorithms to address the twin
RNN limitations of inaccurate training and inefficient prediction. Previous
approaches have improved accuracy at the expense of prediction costs making
them infeasible for resource-constrained and real-time applications. Unitary
RNNs have increased accuracy somewhat by restricting the range of the state
transition matrix's singular values but have also increased the model size as
they require a larger number of hidden units to make up for the loss in
expressive power. Gated RNNs have obtained state-of-the-art accuracies by
adding extra parameters thereby resulting in even larger models. FastRNN
addresses these limitations by adding a residual connection that does not
constrain the range of the singular values explicitly and has only two extra
scalar parameters. FastGRNN then extends the residual connection to a gate by
reusing the RNN matrices to match state-of-the-art gated RNN accuracies but
with a 2-4x smaller model. Enforcing FastGRNN's matrices to be low-rank, sparse
and quantized resulted in accurate models that could be up to 35x smaller than
leading gated and unitary RNNs. This allowed FastGRNN to accurately recognize
the "Hey Cortana" wakeword with a 1 KB model and to be deployed on severely
resource-constrained IoT microcontrollers too tiny to store other RNN models.
FastGRNN's code is available at https://github.com/Microsoft/EdgeML/.
Comparing Sample-wise Learnability Across Deep Neural Network Models
Authors: Seung-Geon Lee, Jaedeok Kim, Hyun-Joo Jung, Yoonsuck Choe
Date: 2019-01-08 00:00:00

Estimating the relative importance of each sample in a training set has
important practical and theoretical value, such as in importance sampling or
curriculum learning. This kind of focus on individual samples invokes the
concept of sample-wise learnability: How easy is it to correctly learn each
sample (cf. PAC learnability)? In this paper, we approach the sample-wise
learnability problem within a deep learning context. We propose a measure of
the learnability of a sample with a given deep neural network (DNN) model. The
basic idea is to train the given model on the training set, and for each
sample, aggregate the hits and misses over the entire training epochs. Our
experiments show that the sample-wise learnability measure collected this way
is highly linearly correlated across different DNN models (ResNet-20, VGG-16,
and MobileNet), suggesting that such a measure can provide deep general
insights on the data's properties. We expect our method to help develop better
curricula for training, and help us better understand the data itself.
Learning with Fenchel-Young Losses
Authors: Mathieu Blondel, André F. T. Martins, Vlad Niculae
Date: 2019-01-08 00:00:00
Tasks: Structured Prediction

Over the past decades, numerous loss functions have been been proposed for a variety of supervised learning tasks, including regression, classification, ranking, and more generally structured prediction. Understanding the core principles and theoretical properties underpinning these losses is key to choose the right loss for the right problem, as well as to create new losses which combine their strengths. In this paper, we introduce Fenchel-Young losses, a generic way to construct a convex loss function for a regularized prediction function. We provide an in-depth study of their properties in a very broad setting, covering all the aforementioned supervised learning tasks, and revealing new connections between sparsity, generalized entropies, and separation margins. We show that Fenchel-Young losses unify many well-known loss functions and allow to create useful new ones easily. Finally, we derive efficient predictive and training algorithms, making Fenchel-Young losses appealing both in theory and practice.
Fusion Strategies for Learning User Embeddings with Neural Networks
Authors: Philipp Blandfort, Tushar Karayil, Federico Raue, Jörn Hees, Andreas Dengel
Date: 2019-01-08 00:00:00
Tasks: Prediction

Growing amounts of online user data motivate the need for automated
processing techniques. In case of user ratings, one interesting option is to
use neural networks for learning to predict ratings given an item and a user.
While training for prediction, such an approach at the same time learns to map
each user to a vector, a so-called user embedding. Such embeddings can for
example be valuable for estimating user similarity. However, there are various
ways how item and user information can be combined in neural networks, and it
is unclear how the way of combining affects the resulting embeddings. In this
paper, we run an experiment on movie ratings data, where we analyze the effect
on embedding quality caused by several fusion strategies in neural networks.
For evaluating embedding quality, we propose a novel measure, Pair-Distance
Correlation, which quantifies the condition that similar users should have
similar embedding vectors. We find that the fusion strategy affects results in
terms of both prediction performance and embedding quality. Surprisingly, we
find that prediction performance not necessarily reflects embedding quality.
This suggests that if embeddings are of interest, the common tendency to select
models based on their prediction ability should be reconsidered.
Complexity Bounds for the Controllability of Temporal Networks with Conditions, Disjunctions, and Uncertainty
Authors: Nikhil Bhargava, Brian Williams
Date: 2019-01-08 00:00:00

In temporal planning, many different temporal network formalisms are used to
model real world situations. Each of these formalisms has different features
which affect how easy it is to determine whether the underlying network of
temporal constraints is consistent. While many of the simpler models have been
well-studied from a computational complexity perspective, the algorithms
developed for advanced models which combine features have very loose complexity
bounds. In this paper, we provide tight completeness bounds for strong, weak,
and dynamic controllability checking of temporal networks that have conditions,
disjunctions, and temporal uncertainty. Our work exposes some of the subtle
differences between these different structures and, remarkably, establishes a
guarantee that all of these problems are computable in PSPACE.