pwc-archive/papers-with-abstracts huggingface dataset:
Document Decomposition of Bangla Printed Text
Authors: Md. Fahad Hasan, Tasmin Afroz, Sabir Ismail, Md. Saiful Islam
Date: 2017-01-27 00:00:00
Tasks: Optical Character Recognition, Optical Character Recognition (OCR)

Today all kind of information is getting digitized and along with all this
digitization, the huge archive of various kinds of documents is being digitized
too. We know that, Optical Character Recognition is the method through which,
newspapers and other paper documents convert into digital resources. But, it is
a fact that this method works on texts only. As a result, if we try to process
any document which contains non-textual zones, then we will get garbage texts
as output. That is why; in order to digitize documents properly they should be
prepossessed carefully. And while preprocessing, segmenting document in
different regions according to the category properly is most important. But,
the Optical Character Recognition processes available for Bangla language have
no such algorithm that can categorize a newspaper/book page fully. So we worked
to decompose a document into its several parts like headlines, sub headlines,
columns, images etc. And if the input is skewed and rotated, then the input was
also deskewed and de-rotated. To decompose any Bangla document we found out the
edges of the input image. Then we find out the horizontal and vertical area of
every pixel where it lies in. Later on the input image was cut according to
these areas. Then we pick each and every sub image and found out their
height-width ratio, line height. Then according to these values the sub images
were categorized. To deskew the image we found out the skew angle and de skewed
the image according to this angle. To de-rotate the image we used the line
height, matra line, pixel ratio of matra line.
Very Deep Convolutional Networks for Text Classification
Authors: Alexis Conneau, Holger Schwenk, Loïc Barrault, Yann Lecun
Date: 2016-06-06 00:00:00
Proceeding: EACL 2017 4
Tasks: General Classification, Text Classification

The dominant approach for many NLP tasks are recurrent neural networks, in
particular LSTMs, and convolutional neural networks. However, these
architectures are rather shallow in comparison to the deep convolutional
networks which have pushed the state-of-the-art in computer vision. We present
a new architecture (VDCNN) for text processing which operates directly at the
character level and uses only small convolutions and pooling operations. We are
able to show that the performance of this model increases with depth: using up
to 29 convolutional layers, we report improvements over the state-of-the-art on
several public text classification tasks. To the best of our knowledge, this is
the first time that very deep convolutional nets have been applied to text
processing.
LocDyn: Robust Distributed Localization for Mobile Underwater Networks
Authors: Cláudia Soares, João Gomes, Beatriz Ferreira, João Paulo Costeira
Date: 2017-01-27 00:00:00

How to self-localize large teams of underwater nodes using only noisy range
measurements? How to do it in a distributed way, and incorporating dynamics
into the problem? How to reject outliers and produce trustworthy position
estimates? The stringent acoustic communication channel and the accuracy needs
of our geophysical survey application demand faster and more accurate
localization methods. We approach dynamic localization as a MAP estimation
problem where the prior encodes dynamics, and we devise a convex relaxation
method that takes advantage of previous estimates at each measurement
acquisition step; The algorithm converges at an optimal rate for first order
methods. LocDyn is distributed: there is no fusion center responsible for
processing acquired data and the same simple computations are performed for
each node. LocDyn is accurate: experiments attest to a smaller positioning
error than a comparable Kalman filter. LocDyn is robust: it rejects outlier
noise, while the comparing methods succumb in terms of positioning error.
Redefinition of the concept of fuzzy set based on vague partition from the perspective of axiomatization
Authors: Xiaodong Pan, Yang Xu
Date: 2017-01-27 00:00:00
Tasks: Attribute

Based on the in-depth analysis of the essence and features of vague
phenomena, this paper focuses on establishing the axiomatical foundation of
membership degree theory for vague phenomena, presents an axiomatic system to
govern membership degrees and their interconnections. On this basis, the
concept of vague partition is introduced, further, the concept of fuzzy set
introduced by Zadeh in 1965 is redefined based on vague partition from the
perspective of axiomatization. The thesis defended in this paper is that the
relationship among vague attribute values should be the starting point to
recognize and model vague phenomena from a quantitative view.
VIME: Variational Information Maximizing Exploration
Authors: Rein Houthooft, Xi Chen, Yan Duan, John Schulman, Filip De Turck, Pieter Abbeel
Date: 2016-05-31 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: continuous-control, Continuous Control, Reinforcement Learning, Reinforcement Learning (RL), Variational Inference

Scalable and effective exploration remains a key challenge in reinforcement
learning (RL). While there are methods with optimality guarantees in the
setting of discrete state and action spaces, these methods cannot be applied in
high-dimensional deep RL scenarios. As such, most contemporary RL relies on
simple heuristics such as epsilon-greedy exploration or adding Gaussian noise
to the controls. This paper introduces Variational Information Maximizing
Exploration (VIME), an exploration strategy based on maximization of
information gain about the agent's belief of environment dynamics. We propose a
practical implementation, using variational inference in Bayesian neural
networks which efficiently handles continuous state and action spaces. VIME
modifies the MDP reward function, and can be applied with several different
underlying RL algorithms. We demonstrate that VIME achieves significantly
better performance compared to heuristic exploration methods across a variety
of continuous control tasks and algorithms, including tasks with very sparse
rewards.
Scaling Binarized Neural Networks on Reconfigurable Logic
Authors: Nicholas J. Fraser, Yaman Umuroglu, Giulio Gambardella, Michaela Blott, Philip Leong, Magnus Jahre, Kees Vissers
Date: 2017-01-12 00:00:00
Tasks: General Classification

Binarized neural networks (BNNs) are gaining interest in the deep learning
community due to their significantly lower computational and memory cost. They
are particularly well suited to reconfigurable logic devices, which contain an
abundance of fine-grained compute resources and can result in smaller, lower
power implementations, or conversely in higher classification rates. Towards
this end, the Finn framework was recently proposed for building fast and
flexible field programmable gate array (FPGA) accelerators for BNNs. Finn
utilized a novel set of optimizations that enable efficient mapping of BNNs to
hardware and implemented fully connected, non-padded convolutional and pooling
layers, with per-layer compute resources being tailored to user-provided
throughput requirements. However, FINN was not evaluated on larger topologies
due to the size of the chosen FPGA, and exhibited decreased accuracy due to
lack of padding. In this paper, we improve upon Finn to show how padding can be
employed on BNNs while still maintaining a 1-bit datapath and high accuracy.
Based on this technique, we demonstrate numerous experiments to illustrate
flexibility and scalability of the approach. In particular, we show that a
large BNN requiring 1.2 billion operations per frame running on an ADM-PCIE-8K5
platform can classify images at 12 kFPS with 671 us latency while drawing less
than 41 W board power and classifying CIFAR-10 images at 88.7% accuracy. Our
implementation of this network achieves 14.8 trillion operations per second. We
believe this is the fastest classification rate reported to date on this
benchmark at this level of accuracy.
Breeding electric zebras in the fields of Medicine
Authors: Federico Cabitza
Date: 2017-01-15 00:00:00
Tasks: BIG-bench Machine Learning

A few notes on the use of machine learning in medicine and the related
unintended consequences.
Statistical Analysis on Bangla Newspaper Data to Extract Trending Topic and Visualize Its Change Over Time
Authors: Syed Mehedi Hasan Nirob, Md. Kazi Nayeem, Md. Saiful Islam
Date: 2017-01-27 00:00:00

Trending topic of newspapers is an indicator to understand the situation of a
country and also a way to evaluate the particular newspaper. This paper
represents a model describing few techniques to select trending topics from
Bangla Newspaper. Topics that are discussed more frequently than other in
Bangla newspaper will be marked and how a very famous topic loses its
importance with the change of time and another topic takes its place will be
demonstrated. Data from two popular Bangla Newspaper with date and time were
collected. Statistical analysis was performed after on these data after
preprocessing. Popular and most used keywords were extracted from the stream of
Bangla keyword with this analysis. This model can also cluster category wise
news trend or a list of news trend in daily or weekly basis with enough data. A
pattern can be found on their news trend too. Comparison among past news trend
of Bangla newspapers will give a visualization of the situation of Bangladesh.
This visualization will be helpful to predict future trending topics of Bangla
Newspaper.
A Neural Architecture Mimicking Humans End-to-End for Natural Language Inference
Authors: Biswajit Paria, K. M. Annervaz, Ambedkar Dukkipati, Ankush Chatterjee, Sanjay Podder
Date: 2016-11-15 00:00:00
Tasks: Natural Language Inference, Representation Learning

In this work we use the recent advances in representation learning to propose
a neural architecture for the problem of natural language inference. Our
approach is aligned to mimic how a human does the natural language inference
process given two statements. The model uses variants of Long Short Term Memory
(LSTM), attention mechanism and composable neural networks, to carry out the
task. Each part of our model can be mapped to a clear functionality humans do
for carrying out the overall task of natural language inference. The model is
end-to-end differentiable enabling training by stochastic gradient descent. On
Stanford Natural Language Inference(SNLI) dataset, the proposed model achieves
better accuracy numbers than all published models in literature.
An Online Convex Optimization Approach to Dynamic Network Resource Allocation
Authors: Tianyi Chen, Qing Ling, Georgios B. Giannakis
Date: 2017-01-14 00:00:00

Existing approaches to online convex optimization (OCO) make sequential
one-slot-ahead decisions, which lead to (possibly adversarial) losses that
drive subsequent decision iterates. Their performance is evaluated by the
so-called regret that measures the difference of losses between the online
solution and the best yet fixed overall solution in hindsight. The present
paper deals with online convex optimization involving adversarial loss
functions and adversarial constraints, where the constraints are revealed after
making decisions, and can be tolerable to instantaneous violations but must be
satisfied in the long term. Performance of an online algorithm in this setting
is assessed by: i) the difference of its losses relative to the best dynamic
solution with one-slot-ahead information of the loss function and the
constraint (that is here termed dynamic regret); and, ii) the accumulated
amount of constraint violations (that is here termed dynamic fit). In this
context, a modified online saddle-point (MOSP) scheme is developed, and proved
to simultaneously yield sub-linear dynamic regret and fit, provided that the
accumulated variations of per-slot minimizers and constraints are sub-linearly
growing with time. MOSP is also applied to the dynamic network resource
allocation task, and it is compared with the well-known stochastic dual
gradient method. Under various scenarios, numerical experiments demonstrate the
performance gain of MOSP relative to the state-of-the-art.
Subset Selection for Multiple Linear Regression via Optimization
Authors: Young Woong Park, Diego Klabjan
Date: 2017-01-27 00:00:00
Tasks: regression, valid

Subset selection in multiple linear regression aims to choose a subset of candidate explanatory variables that tradeoff fitting error (explanatory power) and model complexity (number of variables selected). We build mathematical programming models for regression subset selection based on mean square and absolute errors, and minimal-redundancy-maximal-relevance criteria. The proposed models are tested using a linear-program-based branch-and-bound algorithm with tailored valid inequalities and big M values and are compared against the algorithms in the literature. For high dimensional cases, an iterative heuristic algorithm is proposed based on the mathematical programming models and a core set concept, and a randomized version of the algorithm is derived to guarantee convergence to the global optimum. From the computational experiments, we find that our models quickly find a quality solution while the rest of the time is spent to prove optimality; the iterative algorithms find solutions in a relatively short time and are competitive compared to state-of-the-art algorithms; using ad-hoc big M values is not recommended.
Improving Variational Auto-Encoders using Householder Flow
Authors: Jakub M. Tomczak, Max Welling
Date: 2016-11-29 00:00:00
Tasks: Computational Efficiency

Variational auto-encoders (VAE) are scalable and powerful generative models.
However, the choice of the variational posterior determines tractability and
flexibility of the VAE. Commonly, latent variables are modeled using the normal
distribution with a diagonal covariance matrix. This results in computational
efficiency but typically it is not flexible enough to match the true posterior
distribution. One fashion of enriching the variational posterior distribution
is application of normalizing flows, i.e., a series of invertible
transformations to latent variables with a simple posterior. In this paper, we
follow this line of thinking and propose a volume-preserving flow that uses a
series of Householder transformations. We show empirically on MNIST dataset and
histopathology data that the proposed flow allows to obtain more flexible
variational posterior and competitive results comparing to other normalizing
flows.
A Data-Oriented Model of Literary Language
Authors: Andreas van Cranenburgh, Rens Bod
Date: 2017-01-12 00:00:00
Proceeding: EACL 2017 4
Tasks: model

We consider the task of predicting how literary a text is, with a gold
standard from human ratings. Aside from a standard bigram baseline, we apply
rich syntactic tree fragments, mined from the training set, and a series of
hand-picked features. Our model is the first to distinguish degrees of highly
and less literary novels using a variety of lexical and syntactic features, and
explains 76.0 % of the variation in literary ratings.
Deep Region Hashing for Efficient Large-scale Instance Search from Images
Authors: Jingkuan Song, Tao He, Lianli Gao, Xing Xu, Heng Tao Shen
Date: 2017-01-26 00:00:00
Tasks: Code Generation, Image Retrieval, Instance Search, Object, Region Proposal

Instance Search (INS) is a fundamental problem for many applications, while
it is more challenging comparing to traditional image search since the
relevancy is defined at the instance level.
  Existing works have demonstrated the success of many complex ensemble systems
that are typically conducted by firstly generating object proposals, and then
extracting handcrafted and/or CNN features of each proposal for matching.
However, object bounding box proposals and feature extraction are often
conducted in two separated steps, thus the effectiveness of these methods
collapses. Also, due to the large amount of generated proposals, matching speed
becomes the bottleneck that limits its application to large-scale datasets. To
tackle these issues, in this paper we propose an effective and efficient Deep
Region Hashing (DRH) approach for large-scale INS using an image patch as the
query. Specifically, DRH is an end-to-end deep neural network which consists of
object proposal, feature extraction, and hash code generation. DRH shares
full-image convolutional feature map with the region proposal network, thus
enabling nearly cost-free region proposals. Also, each high-dimensional,
real-valued region features are mapped onto a low-dimensional, compact binary
codes for the efficient object region level matching on large-scale dataset.
Experimental results on four datasets show that our DRH can achieve even better
performance than the state-of-the-arts in terms of MAP, while the efficiency is
improved by nearly 100 times.
emLam -- a Hungarian Language Modeling baseline
Authors: Dávid Márk Nemeskey
Date: 2017-01-26 00:00:00
Tasks: Language Modeling, Language Modelling

This paper aims to make up for the lack of documented baselines for Hungarian
language modeling. Various approaches are evaluated on three publicly available
Hungarian corpora. Perplexity values comparable to models of similar-sized
English corpora are reported. A new, freely downloadable Hungar- ian benchmark
corpus is introduced.
A Compressed Sensing Based Decomposition of Electrodermal Activity Signals
Authors: Swayambhoo Jain, Urvashi Oswal, Kevin S. Xu, Brian Eriksson, Jarvis Haupt
Date: 2016-02-24 00:00:00
Tasks: compressed sensing, Seizure Detection

The measurement and analysis of Electrodermal Activity (EDA) offers
applications in diverse areas ranging from market research, to seizure
detection, to human stress analysis. Unfortunately, the analysis of EDA signals
is made difficult by the superposition of numerous components which can obscure
the signal information related to a user's response to a stimulus. We show how
simple pre-processing followed by a novel compressed sensing based
decomposition can mitigate the effects of the undesired noise components and
help reveal the underlying physiological signal. The proposed framework allows
for decomposition of EDA signals with provable bounds on the recovery of user
responses. We test our procedure on both synthetic and real-world EDA signals
from wearable sensors and demonstrate that our approach allows for more
accurate recovery of user responses as compared to the existing techniques.
DeepMath - Deep Sequence Models for Premise Selection
Authors: Alex A. Alemi, Francois Chollet, Niklas Een, Geoffrey Irving, Christian Szegedy, Josef Urban
Date: 2016-06-14 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Automated Theorem Proving

We study the effectiveness of neural sequence models for premise selection in
automated theorem proving, one of the main bottlenecks in the formalization of
mathematics. We propose a two stage approach for this task that yields good
results for the premise selection task on the Mizar corpus while avoiding the
hand-engineered features of existing state-of-the-art models. To our knowledge,
this is the first time deep learning has been applied to theorem proving on a
large scale.
An Empirical Analysis of Feature Engineering for Predictive Modeling
Authors: Jeff Heaton
Date: 2017-01-26 00:00:00
Tasks: BIG-bench Machine Learning, Feature Engineering

Machine learning models, such as neural networks, decision trees, random forests, and gradient boosting machines, accept a feature vector, and provide a prediction. These models learn in a supervised fashion where we provide feature vectors mapped to the expected output. It is common practice to engineer new features from the provided feature set. Such engineered features will either augment or replace portions of the existing feature vector. These engineered features are essentially calculated fields based on the values of the other features. Engineering such features is primarily a manual, time-consuming task. Additionally, each type of model will respond differently to different kinds of engineered features. This paper reports empirical research to demonstrate what kinds of engineered features are best suited to various machine learning model types. We provide this recommendation by generating several datasets that we designed to benefit from a particular type of engineered feature. The experiment demonstrates to what degree the machine learning model can synthesize the needed feature on its own. If a model can synthesize a planned feature, it is not necessary to provide that feature. The research demonstrated that the studied models do indeed perform differently with various types of engineered features.
Reinforcement Learning based Embodied Agents Modelling Human Users Through Interaction and Multi-Sensory Perception
Authors: Kory W. Mathewson, Patrick M. Pilarski
Date: 2017-01-09 00:00:00
Tasks: Reinforcement Learning, Reinforcement Learning (RL)

This paper extends recent work in interactive machine learning (IML) focused
on effectively incorporating human feedback. We show how control and feedback
signals complement each other in systems which model human reward. We
demonstrate that simultaneously incorporating human control and feedback
signals can improve interactive robotic systems' performance on a self-mirrored
movement control task where an RL-agent controlled right arm attempts to match
the preprogrammed movement pattern of the left arm. We illustrate the impact of
varying human feedback parameters on task performance by investigating the
probability of giving feedback on each time step and the likelihood of given
feedback being correct. We further illustrate that varying the temporal decay
with which the agent incorporates human feedback has a significant impact on
task performance. We found that smearing human feedback over time steps
improves performance and we show varying the probability of feedback at each
time step, and an increased likelihood of those feedbacks being 'correct' can
impact agent performance. We conclude that understanding latent variables in
human feedback is crucial for learning algorithms acting in human-machine
interaction domains.
Match-Tensor: a Deep Relevance Model for Search
Authors: Aaron Jaech, Hetunandan Kamisetty, Eric Ringger, Charlie Clarke
Date: 2017-01-26 00:00:00
Tasks: Feature Engineering, Learning-To-Rank, model

The application of Deep Neural Networks for ranking in search engines may
obviate the need for the extensive feature engineering common to current
learning-to-rank methods. However, we show that combining simple relevance
matching features like BM25 with existing Deep Neural Net models often
substantially improves the accuracy of these models, indicating that they do
not capture essential local relevance matching signals. We describe a novel
deep Recurrent Neural Net-based model that we call Match-Tensor. The
architecture of the Match-Tensor model simultaneously accounts for both local
relevance matching and global topicality signals allowing for a rich interplay
between them when computing the relevance of a document to a query. On a large
held-out test set consisting of social media documents, we demonstrate not only
that Match-Tensor outperforms BM25 and other classes of DNNs but also that it
largely subsumes signals present in these models.