pwc-archive/papers-with-abstracts huggingface dataset:
Is Ordered Weighted $\ell_1$ Regularized Regression Robust to Adversarial Perturbation? A Case Study on OSCAR
Authors: Pin-Yu Chen, Bhanukiran Vinzamuri, Sijia Liu
Date: 2018-09-24 00:00:00
Tasks: BIG-bench Machine Learning, Clustering, regression

Many state-of-the-art machine learning models such as deep neural networks
have recently shown to be vulnerable to adversarial perturbations, especially
in classification tasks. Motivated by adversarial machine learning, in this
paper we investigate the robustness of sparse regression models with strongly
correlated covariates to adversarially designed measurement noises.
Specifically, we consider the family of ordered weighted $\ell_1$ (OWL)
regularized regression methods and study the case of OSCAR (octagonal shrinkage
clustering algorithm for regression) in the adversarial setting. Under a
norm-bounded threat model, we formulate the process of finding a maximally
disruptive noise for OWL-regularized regression as an optimization problem and
illustrate the steps towards finding such a noise in the case of OSCAR.
Experimental results demonstrate that the regression performance of grouping
strongly correlated features can be severely degraded under our adversarial
setting, even when the noise budget is significantly smaller than the
ground-truth signals.
On the Behavior of the Expectation-Maximization Algorithm for Mixture Models
Authors: Babak Barazandeh, Meisam Razaviyayn
Date: 2018-09-24 00:00:00

Finite mixture models are among the most popular statistical models used in
different data science disciplines. Despite their broad applicability,
inference under these models typically leads to computationally challenging
non-convex problems. While the Expectation-Maximization (EM) algorithm is the
most popular approach for solving these non-convex problems, the behavior of
this algorithm is not well understood. In this work, we focus on the case of
mixture of Laplacian (or Gaussian) distribution. We start by analyzing a simple
equally weighted mixture of two single dimensional Laplacian distributions and
show that every local optimum of the population maximum likelihood estimation
problem is globally optimal. Then, we prove that the EM algorithm converges to
the ground truth parameters almost surely with random initialization. Our
result extends the existing results for Gaussian distribution to Laplacian
distribution. Then we numerically study the behavior of mixture models with
more than two components. Motivated by our extensive numerical experiments, we
propose a novel stochastic method for estimating the mean of components of a
mixture model. Our numerical experiments show that our algorithm outperforms
the Naive EM algorithm in almost all scenarios.
Envy-Free Classification
Authors: Maria-Florina Balcan, Travis Dick, Ritesh Noothigattu, Ariel D. Procaccia
Date: 2018-09-23 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Classification, Fairness, General Classification

In classic fair division problems such as cake cutting and rent division, envy-freeness requires that each individual (weakly) prefer his allocation to anyone else's. On a conceptual level, we argue that envy-freeness also provides a compelling notion of fairness for classification tasks. Our technical focus is the generalizability of envy-free classification, i.e., understanding whether a classifier that is envy free on a sample would be almost envy free with respect to the underlying distribution with high probability. Our main result establishes that a small sample is sufficient to achieve such guarantees, when the classifier in question is a mixture of deterministic classifiers that belong to a family of low Natarajan dimension.
Textually Enriched Neural Module Networks for Visual Question Answering
Authors: Khyathi Raghavi Chandu, Mary Arpita Pyreddy, Matthieu Felix, Narendra Nath Joshi
Date: 2018-09-23 00:00:00
Tasks: Image Captioning, Question Answering, Visual Question Answering, Visual Question Answering (VQA)

Problems at the intersection of language and vision, like visual question
answering, have recently been gaining a lot of attention in the field of
multi-modal machine learning as computer vision research moves beyond
traditional recognition tasks. There has been recent success in visual question
answering using deep neural network models which use the linguistic structure
of the questions to dynamically instantiate network layouts. In the process of
converting the question to a network layout, the question is simplified, which
results in loss of information in the model. In this paper, we enrich the image
information with textual data using image captions and external knowledge bases
to generate more coherent answers. We achieve 57.1% overall accuracy on the
test-dev open-ended questions from the visual question answering (VQA 1.0) real
image dataset.
A Hybrid Variational Autoencoder for Collaborative Filtering
Authors: Kilol Gupta, Mukund Yelahanka Raghuprasad, Pankhuri Kumar
Date: 2018-07-14 00:00:00
Tasks: Clustering, Collaborative Filtering, Movie Recommendation, Recommendation Systems

In today's day and age when almost every industry has an online presence with
users interacting in online marketplaces, personalized recommendations have
become quite important. Traditionally, the problem of collaborative filtering
has been tackled using Matrix Factorization which is linear in nature. We
extend the work of [11] on using variational autoencoders (VAEs) for
collaborative filtering with implicit feedback by proposing a hybrid,
multi-modal approach. Our approach combines movie embeddings (learned from a
sibling VAE network) with user ratings from the Movielens 20M dataset and
applies it to the task of movie recommendation. We empirically show how the VAE
network is empowered by incorporating movie embeddings. We also visualize movie
and user embeddings by clustering their latent representations obtained from a
VAE.
Unsupervised parameter selection for denoising with the elastic net
Authors: Ernesto de Vito, Zeljko Kereta, Valeria Naumova
Date: 2018-09-23 00:00:00
Tasks: Denoising, Learning Theory

Despite recent advances in regularisation theory, the issue of parameter selection still remains a challenge for most applications. In a recent work the framework of statistical learning was used to approximate the optimal Tikhonov regularisation parameter from noisy data. In this work, we improve their results and extend the analysis to the elastic net regularisation, providing explicit error bounds on the accuracy of the approximated parameter and the corresponding regularisation solution in a simplified case. Furthermore, in the general case we design a data-driven, automated algorithm for the computation of an approximate regularisation parameter. Our analysis combines statistical learning theory with insights from regularisation theory. We compare our approach with state-of-the-art parameter selection criteria and illustrate its superiority in terms of accuracy and computational time on simulated and real data sets.
Curvilinear Structure Enhancement by Multiscale Top-Hat Tensor in 2D/3D Images
Authors: Shuaa S. Alharbi, Cigdem Sazak, Carl J. Nelson, Boguslaw Obara
Date: 2018-09-23 00:00:00

A wide range of biomedical applications requires enhancement, detection,
quantification and modelling of curvilinear structures in 2D and 3D images.
Curvilinear structure enhancement is a crucial step for further analysis, but
many of the enhancement approaches still suffer from contrast variations and
noise. This can be addressed using a multiscale approach that produces a better
quality enhancement for low contrast and noisy images compared with a
single-scale approach in a wide range of biomedical images. Here, we propose
the Multiscale Top-Hat Tensor (MTHT) approach, which combines multiscale
morphological filtering with a local tensor representation of curvilinear
structures in 2D and 3D images. The proposed approach is validated on synthetic
and real data and is also compared to the state-of-the-art approaches. Our
results show that the proposed approach achieves high-quality curvilinear
structure enhancement in synthetic examples and in a wide range of 2D and 3D
images.
Learning to Read by Spelling: Towards Unsupervised Text Recognition
Authors: Ankush Gupta, Andrea Vedaldi, Andrew Zisserman
Date: 2018-09-23 00:00:00
Tasks: Unsupervised Text Recognition, valid

This work presents a method for visual text recognition without using any
paired supervisory data. We formulate the text recognition task as one of
aligning the conditional distribution of strings predicted from given text
images, with lexically valid strings sampled from target corpora. This enables
fully automated, and unsupervised learning from just line-level text-images,
and unpaired text-string samples, obviating the need for large aligned
datasets. We present detailed analysis for various aspects of the proposed
method, namely - (1) impact of the length of training sequences on convergence,
(2) relation between character frequencies and the order in which they are
learnt, (3) generalisation ability of our recognition network to inputs of
arbitrary lengths, and (4) impact of varying the text corpus on recognition
accuracy. Finally, we demonstrate excellent text recognition accuracy on both
synthetically generated text images, and scanned images of real printed books,
using no labelled training examples.
Data-Driven Design: Exploring new Structural Forms using Machine Learning and Graphic Statics
Authors: Lukas Fuhrimann, Vahid Moosavi, Patrick Ole Ohlbrock, Pierluigi Dacunto
Date: 2018-09-23 00:00:00
Tasks: BIG-bench Machine Learning, Form

The aim of this research is to introduce a novel structural design process
that allows architects and engineers to extend their typical design space
horizon and thereby promoting the idea of creativity in structural design. The
theoretical base of this work builds on the combination of structural
form-finding and state-of-the-art machine learning algorithms. In the first
step of the process, Combinatorial Equilibrium Modelling (CEM) is used to
generate a large variety of spatial networks in equilibrium for given input
parameters. In the second step, these networks are clustered and represented in
a form-map through the implementation of a Self Organizing Map (SOM) algorithm.
In the third step, the solution space is interpreted with the help of a Uniform
Manifold Approximation and Projection algorithm (UMAP). This allows gaining
important insights in the structure of the solution space. A specific case
study is used to illustrate how the infinite equilibrium states of a given
topology can be defined and represented by clusters. Furthermore, three
classes, related to the non-linear interaction between the input parameters and
the form space, are verified and a statement about the entire manifold of the
solution space of the case study is made. To conclude, this work presents an
innovative approach on how the manifold of a solution space can be grasped with
a minimum amount of data and how to operate within the manifold in order to
increase the diversity of solutions.
Accelerated Gossip via Stochastic Heavy Ball Method
Authors: Nicolas Loizou, Peter Richtárik
Date: 2018-09-23 00:00:00

In this paper we show how the stochastic heavy ball method (SHB) -- a popular
method for solving stochastic convex and non-convex optimization problems
--operates as a randomized gossip algorithm. In particular, we focus on two
special cases of SHB: the Randomized Kaczmarz method with momentum and its
block variant. Building upon a recent framework for the design and analysis of
randomized gossip algorithms, [Loizou Richtarik, 2016] we interpret the
distributed nature of the proposed methods. We present novel protocols for
solving the average consensus problem where in each step all nodes of the
network update their values but only a subset of them exchange their private
values. Numerical experiments on popular wireless sensor networks showing the
benefits of our protocols are also presented.
Mind Your Language: Abuse and Offense Detection for Code-Switched Languages
Authors: Raghav Kapoor, Yaman Kumar, Kshitij Rajput, Rajiv Ratn Shah, Ponnurangam Kumaraguru, Roger Zimmermann
Date: 2018-09-23 00:00:00
Tasks: Abuse Detection, General Classification, Transfer Learning

In multilingual societies like the Indian subcontinent, use of code-switched
languages is much popular and convenient for the users. In this paper, we study
offense and abuse detection in the code-switched pair of Hindi and English
(i.e. Hinglish), the pair that is the most spoken. The task is made difficult
due to non-fixed grammar, vocabulary, semantics and spellings of Hinglish
language. We apply transfer learning and make a LSTM based model for hate
speech classification. This model surpasses the performance shown by the
current best models to establish itself as the state-of-the-art in the
unexplored domain of Hinglish offensive text classification.We also release our
model and the embeddings trained for research purposes
Detecting Hate Speech and Offensive Language on Twitter using Machine Learning: An N-gram and TFIDF based Approach
Authors: Aditya Gaydhani, Vikrant Doma, Shrikant Kendre, Laxmi Bhagwat
Date: 2018-09-23 00:00:00
Tasks: Hate Speech Detection

Toxic online content has become a major issue in today's world due to an
exponential increase in the use of internet by people of different cultures and
educational background. Differentiating hate speech and offensive language is a
key challenge in automatic detection of toxic text content. In this paper, we
propose an approach to automatically classify tweets on Twitter into three
classes: hateful, offensive and clean. Using Twitter dataset, we perform
experiments considering n-grams as features and passing their term
frequency-inverse document frequency (TFIDF) values to multiple machine
learning models. We perform comparative analysis of the models considering
several values of n in n-grams and TFIDF normalization methods. After tuning
the model giving the best results, we achieve 95.6% accuracy upon evaluating it
on test data. We also create a module which serves as an intermediate between
user and Twitter.
Domain Adaptation for Robot Predictive Maintenance Systems
Authors: Arash Golibagh Mahyari, Thomas Locker
Date: 2018-09-23 00:00:00
Tasks: Domain Adaptation, Industrial Robots, Transfer Learning

Industrial robots play an increasingly important role in a growing number of fields. For example, robotics is used to increase productivity while reducing costs in various aspects of manufacturing. Since robots are often set up in production lines, the breakdown of a single robot has a negative impact on the entire process, in the worst case bringing the whole line to a halt until the issue is resolved, leading to substantial financial losses due to the unforeseen downtime. Therefore, predictive maintenance systems based on the internal signals of robots have gained attention as an essential component of robotics service offerings. The main shortcoming of existing predictive maintenance algorithms is that the extracted features typically differ significantly from the learnt model when the operation of the robot changes, incurring false alarms. In order to mitigate this problem, predictive maintenance algorithms require the model to be retrained with normal data of the new operation. In this paper, we propose a novel solution based on transfer learning to pass the knowledge of the trained model from one operation to another in order to prevent the need for retraining and to eliminate such false alarms. The deployment of the proposed unsupervised transfer learning algorithm on real-world datasets demonstrates that the algorithm can not only distinguish between operation and mechanical condition change, it further yields a sharper deviation from the trained model in case of a mechanical condition change and thus detects mechanical issues with higher confidence.
Unsupervised Learning of Dense Optical Flow, Depth and Egomotion from Sparse Event Data
Authors: Chengxi Ye, Anton Mitrokhin, Cornelia Fermüller, James A. Yorke, Yiannis Aloimonos
Date: 2018-09-23 00:00:00
Tasks: Decoder, GPU, Optical Flow Estimation

In this work we present a lightweight, unsupervised learning pipeline for
\textit{dense} depth, optical flow and egomotion estimation from sparse event
output of the Dynamic Vision Sensor (DVS). To tackle this low level vision
task, we use a novel encoder-decoder neural network architecture - ECN.
  Our work is the first monocular pipeline that generates dense depth and
optical flow from sparse event data only. The network works in self-supervised
mode and has just 150k parameters. We evaluate our pipeline on the MVSEC self
driving dataset and present results for depth, optical flow and and egomotion
estimation. Due to the lightweight design, the inference part of the network
runs at 250 FPS on a single GPU, making the pipeline ready for realtime
robotics applications. Our experiments demonstrate significant improvements
upon previous works that used deep learning on event data, as well as the
ability of our pipeline to perform well during both day and night.
Learning and Evaluating Sparse Interpretable Sentence Embeddings
Authors: Valentin Trifonov, Octavian-Eugen Ganea, Anna Potapenko, Thomas Hofmann
Date: 2018-09-23 00:00:00
Proceeding: WS 2018 11
Tasks: Sentence, Sentence Embedding, Sentence-Embedding, Sentence Embeddings, Word Embeddings

Previous research on word embeddings has shown that sparse representations,
which can be either learned on top of existing dense embeddings or obtained
through model constraints during training time, have the benefit of increased
interpretability properties: to some degree, each dimension can be understood
by a human and associated with a recognizable feature in the data. In this
paper, we transfer this idea to sentence embeddings and explore several
approaches to obtain a sparse representation. We further introduce a novel,
quantitative and automated evaluation metric for sentence embedding
interpretability, based on topic coherence methods. We observe an increase in
interpretability compared to dense models, on a dataset of movie dialogs and on
the scene descriptions from the MS COCO dataset.
Accelerate CU Partition in HEVC using Large-Scale Convolutional Neural Network
Authors: Chenying Wang, Li Yu, Shengwei Wang
Date: 2018-09-23 00:00:00

High efficiency video coding (HEVC) suffers high encoding computational
complexity, partly attributed to the rate-distortion optimization quad-tree
search in CU partition decision. Therefore, we propose a novel two-stage CU
partition decision approach in HEVC intra-mode. In the proposed approach,
CNN-based algorithm is designed to decide CU partition mode precisely in three
depths. In order to alleviate computational complexity further, an auxiliary
earl-termination mechanism is also proposed to filter obvious homogeneous CUs
out of the subsequent CNN-based algorithm. Experimental results show that the
proposed approach achieves about 37% encoding time saving on average and
insignificant BD-Bitrate rise compared with the original HEVC encoder.
Detecting Features of Tools, Objects, and Actions from Effects in a Robot using Deep Learning
Authors: Namiko Saito, Kitae Kim, Shingo Murata, Tetsuya OGATA, Shigeki SUGANO
Date: 2018-09-23 00:00:00

We propose a tool-use model that can detect the features of tools, target
objects, and actions from the provided effects of object manipulation. We
construct a model that enables robots to manipulate objects with tools, using
infant learning as a concept. To realize this, we train sensory-motor data
recorded during a tool-use task performed by a robot with deep learning.
Experiments include four factors: (1) tools, (2) objects, (3) actions, and (4)
effects, which the model considers simultaneously. For evaluation, the robot
generates predicted images and motions given information of the effects of
using unknown tools and objects. We confirm that the robot is capable of
detecting features of tools, objects, and actions by learning the effects and
executing the task.
Neural Arithmetic Expression Calculator
Authors: Kaiyu Chen, Yihan Dong, Xipeng Qiu, Zitian Chen
Date: 2018-09-23 00:00:00
Tasks: Hierarchical Reinforcement Learning, Reinforcement Learning

This paper presents a pure neural solver for arithmetic expression
calculation (AEC) problem. Previous work utilizes the powerful capabilities of
deep neural networks and attempts to build an end-to-end model to solve this
problem. However, most of these methods can only deal with the additive
operations. It is still a challenging problem to solve the complex expression
calculation problem, which includes the adding, subtracting, multiplying,
dividing and bracketing operations. In this work, we regard the arithmetic
expression calculation as a hierarchical reinforcement learning problem. An
arithmetic operation is decomposed into a series of sub-tasks, and each
sub-task is dealt with by a skill module. The skill module could be a basic
module performing elementary operations, or interactive module performing
complex operations by invoking other skill models. With curriculum learning,
our model can deal with a complex arithmetic expression calculation with the
deep hierarchical structure of skill models. Experiments show that our model
significantly outperforms the previous models for arithmetic expression
calculation.
Exponential Convergence Time of Gradient Descent for One-Dimensional Deep Linear Neural Networks
Authors: Ohad Shamir
Date: 2018-09-23 00:00:00

We study the dynamics of gradient descent on objective functions of the form $f(\prod_{i=1}^{k} w_i)$ (with respect to scalar parameters $w_1,\ldots,w_k$), which arise in the context of training depth-$k$ linear neural networks. We prove that for standard random initializations, and under mild assumptions on $f$, the number of iterations required for convergence scales exponentially with the depth $k$. We also show empirically that this phenomenon can occur in higher dimensions, where each $w_i$ is a matrix. This highlights a potential obstacle in understanding the convergence of gradient-based methods for deep linear neural networks, where $k$ is large.
The use of Virtual Reality in Enhancing Interdisciplinary Research and Education
Authors: Tiffany Leung, Farhana Zulkernine, Haruna Isah
Date: 2018-09-23 00:00:00

Virtual Reality (VR) is increasingly being recognized for its educational
potential and as an effective way to convey new knowledge to people, it
supports interactive and collaborative activities. Affordable VR powered by
mobile technologies is opening a new world of opportunities that can transform
the ways in which we learn and engage with others. This paper reports our study
regarding the application of VR in stimulating interdisciplinary communication.
It investigates the promises of VR in interdisciplinary education and research.
The main contributions of this study are (i) literature review of theories of
learning underlying the justification of the use of VR systems in education,
(ii) taxonomy of the various types and implementations of VR systems and their
application in supporting education and research (iii) evaluation of
educational applications of VR from a broad range of disciplines, (iv)
investigation of how the learning process and learning outcomes are affected by
VR systems, and (v) comparative analysis of VR and traditional methods of
teaching in terms of quality of learning. This study seeks to inspire and
inform interdisciplinary researchers and learners about the ways in which VR
might support them and also VR software developers to push the limits of their
craft.