pwc-archive/papers-with-abstracts huggingface dataset:
A Hardware-Friendly Algorithm for Scalable Training and Deployment of Dimensionality Reduction Models on FPGA
Authors: Mahdi Nazemi, Amir Erfan Eshratifar, Massoud Pedram
Date: 2018-01-11 00:00:00
Tasks: BIG-bench Machine Learning, Dimensionality Reduction, General Classification, image-classification, Image Classification, speech-recognition, Speech Recognition

With ever-increasing application of machine learning models in various
domains such as image classification, speech recognition and synthesis, and
health care, designing efficient hardware for these models has gained a lot of
popularity. While the majority of researches in this area focus on efficient
deployment of machine learning models (a.k.a inference), this work concentrates
on challenges of training these models in hardware. In particular, this paper
presents a high-performance, scalable, reconfigurable solution for both
training and deployment of different dimensionality reduction models in
hardware by introducing a hardware-friendly algorithm. Compared to
state-of-the-art implementations, our proposed algorithm and its hardware
realization decrease resource consumption by 50\% without any degradation in
accuracy.
Global overview of Imitation Learning
Authors: Alexandre Attia, Sharone Dayan
Date: 2018-01-19 00:00:00
Tasks: Imitation Learning

Imitation Learning is a sequential task where the learner tries to mimic an
expert's action in order to achieve the best performance. Several algorithms
have been proposed recently for this task. In this project, we aim at proposing
a wide review of these algorithms, presenting their main features and comparing
them on their performance and their regret bounds.
Dimensionality Reduction in Deep Learning for Chest X-Ray Analysis of Lung Cancer
Authors: Yu. Gordienko, Yu. Kochura, O. Alienin, O. Rokovyi, S. Stirenko, Peng Gang, Jiang Hui, Wei Zeng
Date: 2018-01-19 00:00:00
Tasks: Dimensionality Reduction, Segmentation

Efficiency of some dimensionality reduction techniques, like lung
segmentation, bone shadow exclusion, and t-distributed stochastic neighbor
embedding (t-SNE) for exclusion of outliers, is estimated for analysis of chest
X-ray (CXR) 2D images by deep learning approach to help radiologists identify
marks of lung cancer in CXR. Training and validation of the simple
convolutional neural network (CNN) was performed on the open JSRT dataset
(dataset #01), the JSRT after bone shadow exclusion - BSE-JSRT (dataset #02),
JSRT after lung segmentation (dataset #03), BSE-JSRT after lung segmentation
(dataset #04), and segmented BSE-JSRT after exclusion of outliers by t-SNE
method (dataset #05). The results demonstrate that the pre-processed dataset
obtained after lung segmentation, bone shadow exclusion, and filtering out the
outliers by t-SNE (dataset #05) demonstrates the highest training rate and best
accuracy in comparison to the other pre-processed datasets.
Large-scale Cloze Test Dataset Created by Teachers
Authors: Qizhe Xie, Guokun Lai, Zihang Dai, Eduard Hovy
Date: 2017-11-09 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Cloze Test, Language Modeling, Language Modelling

Cloze tests are widely adopted in language exams to evaluate students'
language proficiency. In this paper, we propose the first large-scale
human-created cloze test dataset CLOTH, containing questions used in
middle-school and high-school language exams. With missing blanks carefully
created by teachers and candidate choices purposely designed to be nuanced,
CLOTH requires a deeper language understanding and a wider attention span than
previously automatically-generated cloze datasets. We test the performance of
dedicatedly designed baseline models including a language model trained on the
One Billion Word Corpus and show humans outperform them by a significant
margin. We investigate the source of the performance gap, trace model
deficiencies to some distinct properties of CLOTH, and identify the limited
ability of comprehending the long-term context to be the key bottleneck.
Fast structure learning with modular regularization
Authors: Greg Ver Steeg, Hrayr Harutyunyan, Daniel Moyer, Aram Galstyan
Date: 2017-06-11 00:00:00
Proceeding: NeurIPS 2019 12

Estimating graphical model structure from high-dimensional and undersampled data is a fundamental problem in many scientific fields. Existing approaches, such as GLASSO, latent variable GLASSO, and latent tree models, suffer from high computational complexity and may impose unrealistic sparsity priors in some cases. We introduce a novel method that leverages a newly discovered connection between information-theoretic measures and structured latent factor models to derive an optimization objective which encourages modular structures where each observed variable has a single latent parent. The proposed method has linear stepwise computational complexity w.r.t. the number of observed variables. Our experiments on synthetic data demonstrate that our approach is the only method that recovers modular structure better as the dimensionality increases. We also use our approach for estimating covariance structure for a number of real-world datasets and show that it consistently outperforms state-of-the-art estimators at a fraction of the computational cost. Finally, we apply the proposed method to high-resolution fMRI data (with more than 10^5 voxels) and show that it is capable of extracting meaningful patterns.
Big Data Analytics for Wireless and Wired Network Design: A Survey
Authors: Mohammed S. Hadi, Ahmed Q. Lawey, Taisir E. H. El-Gorashi, Jaafar M. H. Elmirghani
Date: 2018-01-19 00:00:00
Tasks: Survey

Currently, the world is witnessing a mounting avalanche of data due to the
increasing number of mobile network subscribers, Internet websites, and online
services. This trend is continuing to develop in a quick and diverse manner in
the form of big data. Big data analytics can process large amounts of raw data
and extract useful, smaller-sized information, which can be used by different
parties to make reliable decisions. In this paper, we conduct a survey on the
role that big data analytics can play in the design of data communication
networks. Integrating the latest advances that employ big data analytics with
the networks control/traffic layers might be the best way to build robust data
communication networks with refined performance and intelligent features.
First, the survey starts with the introduction of the big data basic concepts,
framework, and characteristics. Second, we illustrate the main network design
cycle employing big data analytics. This cycle represents the umbrella concept
that unifies the surveyed topics. Third, there is a detailed review of the
current academic and industrial efforts toward network design using big data
analytics. Forth, we identify the challenges confronting the utilization of big
data analytics in network design. Finally, we highlight several future research
directions. To the best of our knowledge, this is the first survey that
addresses the use of big data analytics techniques for the design of a broad
range of networks.
Deep Learning for Detecting Cyberbullying Across Multiple Social Media Platforms
Authors: Sweta Agrawal, Amit Awekar
Date: 2018-01-19 00:00:00
Tasks: 16k, Deep Learning, Transfer Learning

Harassment by cyberbullies is a significant phenomenon on the social media.
Existing works for cyberbullying detection have at least one of the following
three bottlenecks. First, they target only one particular social media platform
(SMP). Second, they address just one topic of cyberbullying. Third, they rely
on carefully handcrafted features of the data. We show that deep learning based
models can overcome all three bottlenecks. Knowledge learned by these models on
one dataset can be transferred to other datasets. We performed extensive
experiments using three real-world datasets: Formspring (12k posts), Twitter
(16k posts), and Wikipedia(100k posts). Our experiments provide several useful
insights about cyberbullying detection. To the best of our knowledge, this is
the first work that systematically analyzes cyberbullying detection on various
topics across multiple SMPs using deep learning based models and transfer
learning.
Active Learning of Strict Partial Orders: A Case Study on Concept Prerequisite Relations
Authors: Chen Liang, Jianbo Ye, Han Zhao, Bart Pursel, C. Lee Giles
Date: 2018-01-19 00:00:00
Tasks: Active Learning, General Classification, Relational Reasoning

Strict partial order is a mathematical structure commonly seen in relational
data. One obstacle to extracting such type of relations at scale is the lack of
large-scale labels for building effective data-driven solutions. We develop an
active learning framework for mining such relations subject to a strict order.
Our approach incorporates relational reasoning not only in finding new
unlabeled pairs whose labels can be deduced from an existing label set, but
also in devising new query strategies that consider the relational structure of
labels. Our experiments on concept prerequisite relations show our proposed
framework can substantially improve the classification performance with the
same query budget compared to other baseline approaches.
A Practitioners' Guide to Transfer Learning for Text Classification using Convolutional Neural Networks
Authors: Tushar Semwal, Gaurav Mathur, Promod Yenigalla, Shivashankar B. Nair
Date: 2018-01-19 00:00:00
Tasks: General Classification, text-classification, Text Classification, Transfer Learning

Transfer Learning (TL) plays a crucial role when a given dataset has
insufficient labeled examples to train an accurate model. In such scenarios,
the knowledge accumulated within a model pre-trained on a source dataset can be
transferred to a target dataset, resulting in the improvement of the target
model. Though TL is found to be successful in the realm of image-based
applications, its impact and practical use in Natural Language Processing (NLP)
applications is still a subject of research. Due to their hierarchical
architecture, Deep Neural Networks (DNN) provide flexibility and customization
in adjusting their parameters and depth of layers, thereby forming an apt area
for exploiting the use of TL. In this paper, we report the results and
conclusions obtained from extensive empirical experiments using a Convolutional
Neural Network (CNN) and try to uncover thumb rules to ensure a successful
positive transfer. In addition, we also highlight the flawed means that could
lead to a negative transfer. We explore the transferability of various layers
and describe the effect of varying hyper-parameters on the transfer
performance. Also, we present a comparison of accuracy value and model size
against state-of-the-art methods. Finally, we derive inferences from the
empirical results and provide best practices to achieve a successful positive
transfer.
Quality Classified Image Analysis with Application to Face Detection and Recognition
Authors: Fei Yang, Qian Zhang, Miaohui Wang, Guoping Qiu
Date: 2018-01-19 00:00:00
Tasks: Face Detection

Motion blur, out of focus, insufficient spatial resolution, lossy compression
and many other factors can all cause an image to have poor quality. However,
image quality is a largely ignored issue in traditional pattern recognition
literature. In this paper, we use face detection and recognition as case
studies to show that image quality is an essential factor which will affect the
performances of traditional algorithms. We demonstrated that it is not the
image quality itself that is the most important, but rather the quality of the
images in the training set should have similar quality as those in the testing
set. To handle real-world application scenarios where images with different
kinds and severities of degradation can be presented to the system, we have
developed a quality classified image analysis framework to deal with images of
mixed qualities adaptively. We use deep neural networks first to classify
images based on their quality classes and then design a separate face detector
and recognizer for images in each quality class. We will present experimental
results to show that our quality classified framework can accurately classify
images based on the type and severity of image degradations and can
significantly boost the performances of state-of-the-art face detector and
recognizer in dealing with image datasets containing mixed quality images.
A Resource-Light Method for Cross-Lingual Semantic Textual Similarity
Authors: Goran Glavaš, Marc Franco-Salvador, Simone Paolo Ponzetto, Paolo Rosso
Date: 2018-01-19 00:00:00
Tasks: Cross-Lingual Information Retrieval, Cross-Lingual Semantic Textual Similarity, Information Retrieval, Machine Translation, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Retrieval, Semantic Similarity, Semantic Textual Similarity, Translation, Word Embeddings, Word Translation

Recognizing semantically similar sentences or paragraphs across languages is
beneficial for many tasks, ranging from cross-lingual information retrieval and
plagiarism detection to machine translation. Recently proposed methods for
predicting cross-lingual semantic similarity of short texts, however, make use
of tools and resources (e.g., machine translation systems, syntactic parsers or
named entity recognition) that for many languages (or language pairs) do not
exist. In contrast, we propose an unsupervised and a very resource-light
approach for measuring semantic similarity between texts in different
languages. To operate in the bilingual (or multilingual) space, we project
continuous word vectors (i.e., word embeddings) from one language to the vector
space of the other language via the linear translation model. We then align
words according to the similarity of their vectors in the bilingual embedding
space and investigate different unsupervised measures of semantic similarity
exploiting bilingual embeddings and word alignments. Requiring only a
limited-size set of word translation pairs between the languages, the proposed
approach is applicable to virtually any pair of languages for which there
exists a sufficiently large corpus, required to learn monolingual word
embeddings. Experimental results on three different datasets for measuring
semantic textual similarity show that our simple resource-light approach
reaches performance close to that of supervised and resource intensive methods,
displaying stability across different language pairs. Furthermore, we evaluate
the proposed method on two extrinsic tasks, namely extraction of parallel
sentences from comparable corpora and cross lingual plagiarism detection, and
show that it yields performance comparable to those of complex
resource-intensive state-of-the-art models for the respective tasks.
Minimum spanning tree release under differential privacy constraints
Authors: Rafael Pinot
Date: 2018-01-19 00:00:00
Tasks: Clustering

We investigate the problem of nodes clustering under privacy constraints when
representing a dataset as a graph. Our contribution is threefold. First we
formally define the concept of differential privacy for structured databases
such as graphs, and give an alternative definition based on a new neighborhood
notion between graphs. This definition is adapted to particular frameworks that
can be met in various application fields such as genomics, world wide web,
population survey, etc. Second, we introduce a new algorithm to tackle the
issue of privately releasing an approximated minimum spanning tree topology for
a simple-undirected-weighted graph. It provides a simple way of producing the
topology of a private almost minimum spanning tree which outperforms, in most
cases, the state of the art "Laplace mechanism" in terms of
weight-approximation error.
  Finally, we propose a theoretically motivated method combining a sanitizing
mechanism (such as Laplace or our new algorithm) with a Minimum Spanning Tree
(MST)-based clustering algorithm. It provides an accurate method for nodes
clustering in a graph while keeping the sensitive information contained in the
edges weights of the private graph. We provide some theoretical results on the
robustness of an almost minimum spanning tree construction for Laplace
sanitizing mechanisms. These results exhibit which conditions the graph weights
should respect in order to consider that the nodes form well separated clusters
both for Laplace and our algorithm as sanitizing mechanism. The method has been
experimentally evaluated on simulated data, and preliminary results show the
good behavior of the algorithm while identifying well separated clusters.
Size vs. Structure in Training Corpora for Word Embedding Models: Araneum Russicum Maximum and Russian National Corpus
Authors: Andrey Kutuzov, Maria Kunilovskaya
Date: 2018-01-19 00:00:00
Tasks: Semantic Similarity, Semantic Textual Similarity

In this paper, we present a distributional word embedding model trained on
one of the largest available Russian corpora: Araneum Russicum Maximum (over 10
billion words crawled from the web). We compare this model to the model trained
on the Russian National Corpus (RNC). The two corpora are much different in
their size and compilation procedures. We test these differences by evaluating
the trained models against the Russian part of the Multilingual SimLex999
semantic similarity dataset. We detect and describe numerous issues in this
dataset and publish a new corrected version. Aside from the already known fact
that the RNC is generally a better training corpus than web corpora, we
enumerate and explain fine differences in how the models process semantic
similarity task, what parts of the evaluation set are difficult for particular
models and why. Additionally, the learning curves for both models are
described, showing that the RNC is generally more robust as training material
for this task.
Non-myopic learning in repeated stochastic games
Authors: Jacob W. Crandall
Date: 2014-09-30 00:00:00

In repeated stochastic games (RSGs), an agent must quickly adapt to the
behavior of previously unknown associates, who may themselves be learning. This
machine-learning problem is particularly challenging due, in part, to the
presence of multiple (even infinite) equilibria and inherently large strategy
spaces. In this paper, we introduce a method to reduce the strategy space of
two-player general-sum RSGs to a handful of expert strategies. This process,
called Mega, effectually reduces an RSG to a bandit problem. We show that the
resulting strategy space preserves several important properties of the original
RSG, thus enabling a learner to produce robust strategies within a reasonably
small number of interactions. To better establish strengths and weaknesses of
this approach, we empirically evaluate the resulting learning system against
other algorithms in three different RSGs.
The BioDynaMo Project: a platform for computer simulations of biological dynamics
Authors: Leonard Johard, Lukas Breitwieser, Alberto Di Meglio, Marco Manca, Manuel Mazzara, Max Talanov
Date: 2016-08-05 00:00:00

This paper is a brief update on developments in the BioDynaMo project, a new
platform for computer simulations for biological research. We will discuss the
new capabilities of the simulator, important new concepts simulation
methodology as well as its numerous applications to the computational biology
and nanoscience communities.
Learning from Synthetic Humans
Authors: Gül Varol, Javier Romero, Xavier Martin, Naureen Mahmood, Michael J. Black, Ivan Laptev, Cordelia Schmid
Date: 2017-01-05 00:00:00
Proceeding: CVPR 2017 7
Tasks: 2D Human Pose Estimation, 3D Human Pose Estimation, Human Part Segmentation, Pose Estimation, Segmentation

Estimating human pose, shape, and motion from images and videos are
fundamental challenges with many applications. Recent advances in 2D human pose
estimation use large amounts of manually-labeled training data for learning
convolutional neural networks (CNNs). Such data is time consuming to acquire
and difficult to extend. Moreover, manual labeling of 3D pose, depth and motion
is impractical. In this work we present SURREAL (Synthetic hUmans foR REAL
tasks): a new large-scale dataset with synthetically-generated but realistic
images of people rendered from 3D sequences of human motion capture data. We
generate more than 6 million frames together with ground truth pose, depth
maps, and segmentation masks. We show that CNNs trained on our synthetic
dataset allow for accurate human depth estimation and human part segmentation
in real RGB images. Our results and the new dataset open up new possibilities
for advancing person analysis using cheap and large-scale synthetic data.
Introducing ReQuEST: an Open Platform for Reproducible and Quality-Efficient Systems-ML Tournaments
Authors: Thierry Moreau, Anton Lokhmotov, Grigori Fursin
Date: 2018-01-19 00:00:00
Tasks: BIG-bench Machine Learning

Co-designing efficient machine learning based systems across the whole
hardware/software stack to trade off speed, accuracy, energy and costs is
becoming extremely complex and time consuming. Researchers often struggle to
evaluate and compare different published works across rapidly evolving software
frameworks, heterogeneous hardware platforms, compilers, libraries, algorithms,
data sets, models, and environments.
  We present our community effort to develop an open co-design tournament
platform with an online public scoreboard. It will gradually incorporate best
research practices while providing a common way for multidisciplinary
researchers to optimize and compare the quality vs. efficiency Pareto
optimality of various workloads on diverse and complete hardware/software
systems. We want to leverage the open-source Collective Knowledge framework and
the ACM artifact evaluation methodology to validate and share the complete
machine learning system implementations in a standardized, portable, and
reproducible fashion. We plan to hold regular multi-objective optimization and
co-design tournaments for emerging workloads such as deep learning, starting
with ASPLOS'18 (ACM conference on Architectural Support for Programming
Languages and Operating Systems - the premier forum for multidisciplinary
systems research spanning computer architecture and hardware, programming
languages and compilers, operating systems and networking) to build a public
repository of the most efficient machine learning algorithms and systems which
can be easily customized, reused and built upon.
Verification of Binarized Neural Networks via Inter-Neuron Factoring
Authors: Chih-Hong Cheng, Georg Nührenberg, Chung-Hao Huang, Harald Ruess
Date: 2017-10-09 00:00:00

We study the problem of formal verification of Binarized Neural Networks
(BNN), which have recently been proposed as a energy-efficient alternative to
traditional learning networks. The verification of BNNs, using the reduction to
hardware verification, can be even more scalable by factoring computations
among neurons within the same layer. By proving the NP-hardness of finding
optimal factoring as well as the hardness of PTAS approximability, we design
polynomial-time search heuristics to generate factoring solutions. The overall
framework allows applying verification techniques to moderately-sized BNNs for
embedded devices with thousands of neurons and inputs.