pwc-archive/papers-with-abstracts huggingface dataset:
Assessment Formats and Student Learning Performance: What is the Relation?
Authors: Khondkar Islam, Pouyan Ahmadi, Salman Yousaf
Date: 2017-11-15 00:00:00
Tasks: Management, Relation

Although compelling assessments have been examined in recent years, more
studies are required to yield a better understanding of the several methods
where assessment techniques significantly affect student learning process. Most
of the educational research in this area does not consider demographics data,
differing methodologies, and notable sample size. To address these drawbacks,
the objective of our study is to analyse student learning outcomes of multiple
assessment formats for a web-facilitated in-class section with an asynchronous
online class of a core data communications course in the Undergraduate IT
program of the Information Sciences and Technology (IST) Department at George
Mason University (GMU). In this study, students were evaluated based on course
assessments such as home and lab assignments, skill-based assessments, and
traditional midterm and final exams across all four sections of the course. All
sections have equivalent content, assessments, and teaching methodologies.
Student demographics such as exam type and location preferences are considered
in our study to determine whether they have any impact on their learning
approach. Large amount of data from the learning management system (LMS),
Blackboard (BB) Learn, had to be examined to compare the results of several
assessment outcomes for all students within their respective section and
amongst students of other sections. To investigate the effect of dissimilar
assessment formats on student performance, we had to correlate individual
question formats with the overall course grade. The results show that
collective assessment formats allow students to be effective in demonstrating
their knowledge.
Variational Bi-LSTMs
Authors: Samira Shabanian, Devansh Arpit, Adam Trischler, Yoshua Bengio
Date: 2017-11-15 00:00:00
Proceeding: ICLR 2018 1

Recurrent neural networks like long short-term memory (LSTM) are important
architectures for sequential prediction tasks. LSTMs (and RNNs in general)
model sequences along the forward time direction. Bidirectional LSTMs
(Bi-LSTMs) on the other hand model sequences along both forward and backward
directions and are generally known to perform better at such tasks because they
capture a richer representation of the data. In the training of Bi-LSTMs, the
forward and backward paths are learned independently. We propose a variant of
the Bi-LSTM architecture, which we call Variational Bi-LSTM, that creates a
channel between the two paths (during training, but which may be omitted during
inference); thus optimizing the two paths jointly. We arrive at this joint
objective for our model by minimizing a variational lower bound of the joint
likelihood of the data sequence. Our model acts as a regularizer and encourages
the two networks to inform each other in making their respective predictions
using distinct information. We perform ablation studies to better understand
the different components of our model and evaluate the method on various
benchmarks, showing state-of-the-art performance.
The Neural Network Pushdown Automaton: Model, Stack and Learning Simulations
Authors: G. Z. Sun, C. L. Giles, H. H. Chen, Y. C. Lee
Date: 2017-11-15 00:00:00
Tasks: Quantization

In order for neural networks to learn complex languages or grammars, they
must have sufficient computational power or resources to recognize or generate
such languages. Though many approaches have been discussed, one ob- vious
approach to enhancing the processing power of a recurrent neural network is to
couple it with an external stack memory - in effect creating a neural network
pushdown automata (NNPDA). This paper discusses in detail this NNPDA - its
construction, how it can be trained and how useful symbolic information can be
extracted from the trained network.
  In order to couple the external stack to the neural network, an optimization
method is developed which uses an error function that connects the learning of
the state automaton of the neural network to the learning of the operation of
the external stack. To minimize the error function using gradient descent
learning, an analog stack is designed such that the action and storage of
information in the stack are continuous. One interpretation of a continuous
stack is the probabilistic storage of and action on data. After training on
sample strings of an unknown source grammar, a quantization procedure extracts
from the analog stack and neural network a discrete pushdown automata (PDA).
Simulations show that in learning deterministic context-free grammars - the
balanced parenthesis language, 1*n0*n, and the deterministic Palindrome - the
extracted PDA is correct in the sense that it can correctly recognize unseen
strings of arbitrary length. In addition, the extracted PDAs can be shown to be
identical or equivalent to the PDAs of the source grammars which were used to
generate the training strings.
CSWA: Aggregation-Free Spatial-Temporal Community Sensing
Authors: Jiang Bian, Haoyi Xiong, Yanjie Fu, Sajal K. Das
Date: 2017-11-15 00:00:00
Tasks: Compressive Sensing, Distributed Optimization

In this paper, we present a novel community sensing paradigm -- {C}ommunity
{S}ensing {W}ithout {A}ggregation}. CSWA is designed to obtain the environment
information (e.g., air pollution or temperature) in each subarea of the target
area, without aggregating sensor and location data collected by community
members. CSWA operates on top of a secured peer-to-peer network over the
community members and proposes a novel \emph{Decentralized Spatial-Temporal
Compressive Sensing} framework based on \emph{Parallelized Stochastic Gradient
Descent}. Through learning the \emph{low-rank structure} via distributed
optimization, CSWA approximates the value of the sensor data in each subarea
(both covered and uncovered) for each sensing cycle using the sensor data
locally stored in each member's mobile device. Simulation experiments based on
real-world datasets demonstrate that CSWA exhibits low approximation error
(i.e., less than $0.2 ^\circ$C in city-wide temperature sensing task and $10$
units of PM2.5 index in urban air pollution sensing) and performs comparably to
(sometimes better than) state-of-the-art algorithms based on the data
aggregation and centralized computation.
Modeling the Temporal Nature of Human Behavior for Demographics Prediction
Authors: Bjarke Felbo, Pål Sundsøy, Alex 'Sandy' Pentland, Sune Lehmann, Yves-Alexandre de Montjoye
Date: 2015-11-20 00:00:00
Tasks: Gender Prediction, Humanitarian

Mobile phone metadata is increasingly used for humanitarian purposes in
developing countries as traditional data is scarce. Basic demographic
information is however often absent from mobile phone datasets, limiting the
operational impact of the datasets. For these reasons, there has been a growing
interest in predicting demographic information from mobile phone metadata.
Previous work focused on creating increasingly advanced features to be modeled
with standard machine learning algorithms. We here instead model the raw mobile
phone metadata directly using deep learning, exploiting the temporal nature of
the patterns in the data. From high-level assumptions we design a data
representation and convolutional network architecture for modeling patterns
within a week. We then examine three strategies for aggregating patterns across
weeks and show that our method reaches state-of-the-art accuracy on both age
and gender prediction using only the temporal modality in mobile metadata. We
finally validate our method on low activity users and evaluate the modeling
assumptions.
Unsupervised Morphological Expansion of Small Datasets for Improving Word Embeddings
Authors: Syed Sarfaraz Akhtar, Arihant Gupta, Avijit Vajpayee, Arjit Srivastava, Manish Shrivastava
Date: 2017-11-15 00:00:00
Tasks: Word Embeddings, Word Similarity

We present a language independent, unsupervised method for building word
embeddings using morphological expansion of text. Our model handles the problem
of data sparsity and yields improved word embeddings by relying on training
word embeddings on artificially generated sentences. We evaluate our method
using small sized training sets on eleven test sets for the word similarity
task across seven languages. Further, for English, we evaluated the impacts of
our approach using a large training set on three standard test sets. Our method
improved results across all languages.
Introduction to intelligent computing unit 1
Authors: Isa Inuwa-Dutse
Date: 2017-11-15 00:00:00
Tasks: BIG-bench Machine Learning, Deep Learning

This brief note highlights some basic concepts required toward understanding
the evolution of machine learning and deep learning models. The note starts
with an overview of artificial intelligence and its relationship to biological
neuron that ultimately led to the evolution of todays intelligent models.
S$^3$FD: Single Shot Scale-invariant Face Detector
Authors: Shifeng Zhang, Xiangyu Zhu, Zhen Lei, Hailin Shi, Xiaobo Wang, Stan Z. Li
Date: 2017-08-17 00:00:00
Tasks: Face Detection

This paper presents a real-time face detector, named Single Shot
Scale-invariant Face Detector (S$^3$FD), which performs superiorly on various
scales of faces with a single deep neural network, especially for small faces.
Specifically, we try to solve the common problem that anchor-based detectors
deteriorate dramatically as the objects become smaller. We make contributions
in the following three aspects: 1) proposing a scale-equitable face detection
framework to handle different scales of faces well. We tile anchors on a wide
range of layers to ensure that all scales of faces have enough features for
detection. Besides, we design anchor scales based on the effective receptive
field and a proposed equal proportion interval principle; 2) improving the
recall rate of small faces by a scale compensation anchor matching strategy; 3)
reducing the false positive rate of small faces via a max-out background label.
As a consequence, our method achieves state-of-the-art detection performance on
all the common face detection benchmarks, including the AFW, PASCAL face, FDDB
and WIDER FACE datasets, and can run at 36 FPS on a Nvidia Titan X (Pascal) for
VGA-resolution images.
On the Convergence of Asynchronous Parallel Iteration with Unbounded Delays
Authors: Zhimin Peng, Yangyang Xu, Ming Yan, Wotao Yin
Date: 2016-12-13 00:00:00

Recent years have witnessed the surge of asynchronous parallel
(async-parallel) iterative algorithms due to problems involving very
large-scale data and a large number of decision variables. Because of
asynchrony, the iterates are computed with outdated information, and the age of
the outdated information, which we call delay, is the number of times it has
been updated since its creation. Almost all recent works prove convergence
under the assumption of a finite maximum delay and set their stepsize
parameters accordingly. However, the maximum delay is practically unknown.
  This paper presents convergence analysis of an async-parallel method from a
probabilistic viewpoint, and it allows for large unbounded delays. An explicit
formula of stepsize that guarantees convergence is given depending on delays'
statistics. With $p+1$ identical processors, we empirically measured that
delays closely follow the Poisson distribution with parameter $p$, matching our
theoretical model, and thus the stepsize can be set accordingly. Simulations on
both convex and nonconvex optimization problems demonstrate the validness of
our analysis and also show that the existing maximum-delay induced stepsize is
too conservative, often slowing down the convergence of the algorithm.
Spatial Mapping with Gaussian Processes and Nonstationary Fourier Features
Authors: Jean-Francois Ton, Seth Flaxman, Dino Sejdinovic, Samir Bhatt
Date: 2017-11-15 00:00:00
Tasks: Gaussian Processes, Time Series, Time Series Analysis

The use of covariance kernels is ubiquitous in the field of spatial
statistics. Kernels allow data to be mapped into high-dimensional feature
spaces and can thus extend simple linear additive methods to nonlinear methods
with higher order interactions. However, until recently, there has been a
strong reliance on a limited class of stationary kernels such as the Matern or
squared exponential, limiting the expressiveness of these modelling approaches.
Recent machine learning research has focused on spectral representations to
model arbitrary stationary kernels and introduced more general representations
that include classes of nonstationary kernels. In this paper, we exploit the
connections between Fourier feature representations, Gaussian processes and
neural networks to generalise previous approaches and develop a simple and
efficient framework to learn arbitrarily complex nonstationary kernel functions
directly from the data, while taking care to avoid overfitting using
state-of-the-art methods from deep learning. We highlight the very broad array
of kernel classes that could be created within this framework. We apply this to
a time series dataset and a remote sensing problem involving land surface
temperature in Eastern Africa. We show that without increasing the
computational or storage complexity, nonstationary kernels can be used to
improve generalisation performance and provide more interpretable results.
Interpreting Deep Visual Representations via Network Dissection
Authors: Bolei Zhou, David Bau, Aude Oliva, Antonio Torralba
Date: 2017-11-15 00:00:00

The success of recent deep convolutional neural networks (CNNs) depends on
learning hidden representations that can summarize the important factors of
variation behind the data. However, CNNs often criticized as being black boxes
that lack interpretability, since they have millions of unexplained model
parameters. In this work, we describe Network Dissection, a method that
interprets networks by providing labels for the units of their deep visual
representations. The proposed method quantifies the interpretability of CNN
representations by evaluating the alignment between individual hidden units and
a set of visual semantic concepts. By identifying the best alignments, units
are given human interpretable labels across a range of objects, parts, scenes,
textures, materials, and colors. The method reveals that deep representations
are more transparent and interpretable than expected: we find that
representations are significantly more interpretable than they would be under a
random equivalently powerful basis. We apply the method to interpret and
compare the latent representations of various network architectures trained to
solve different supervised and self-supervised training tasks. We then examine
factors affecting the network interpretability such as the number of the
training iterations, regularizations, different initializations, and the
network depth and width. Finally we show that the interpreted units can be used
to provide explicit explanations of a prediction given by a CNN for an image.
Our results highlight that interpretability is an important property of deep
neural networks that provides new insights into their hierarchical structure.
Globally Optimal Symbolic Regression
Authors: Vernon Austel, Sanjeeb Dash, Oktay Gunluk, Lior Horesh, Leo Liberti, Giacomo Nannicini, Baruch Schieber
Date: 2017-10-29 00:00:00
Tasks: regression, Symbolic Regression

In this study we introduce a new technique for symbolic regression that
guarantees global optimality. This is achieved by formulating a mixed integer
non-linear program (MINLP) whose solution is a symbolic mathematical expression
of minimum complexity that explains the observations. We demonstrate our
approach by rediscovering Kepler's law on planetary motion using exoplanet data
and Galileo's pendulum periodicity equation using experimental data.
Words are Malleable: Computing Semantic Shifts in Political and Media Discourse
Authors: Hosein Azarbonyad, Mostafa Dehghani, Kaspar Beelen, Alexandra Arkut, Maarten Marx, Jaap Kamps
Date: 2017-11-15 00:00:00

Recently, researchers started to pay attention to the detection of temporal
shifts in the meaning of words. However, most (if not all) of these approaches
restricted their efforts to uncovering change over time, thus neglecting other
valuable dimensions such as social or political variability. We propose an
approach for detecting semantic shifts between different viewpoints--broadly
defined as a set of texts that share a specific metadata feature, which can be
a time-period, but also a social entity such as a political party. For each
viewpoint, we learn a semantic space in which each word is represented as a low
dimensional neural embedded vector. The challenge is to compare the meaning of
a word in one space to its meaning in another space and measure the size of the
semantic shifts. We compare the effectiveness of a measure based on optimal
transformations between the two spaces with a measure based on the similarity
of the neighbors of the word in the respective spaces. Our experiments
demonstrate that the combination of these two performs best. We show that the
semantic shifts not only occur over time, but also along different viewpoints
in a short period of time. For evaluation, we demonstrate how this approach
captures meaningful semantic shifts and can help improve other tasks such as
the contrastive viewpoint summarization and ideology detection (measured as
classification accuracy) in political texts. We also show that the two laws of
semantic change which were empirically shown to hold for temporal shifts also
hold for shifts across viewpoints. These laws state that frequent words are
less likely to shift meaning while words with many senses are more likely to do
so.
Advances in Variational Inference
Authors: Cheng Zhang, Judith Butepage, Hedvig Kjellstrom, Stephan Mandt
Date: 2017-11-15 00:00:00
Tasks: Variational Inference

Many modern unsupervised or semi-supervised machine learning algorithms rely
on Bayesian probabilistic models. These models are usually intractable and thus
require approximate inference. Variational inference (VI) lets us approximate a
high-dimensional Bayesian posterior with a simpler variational distribution by
solving an optimization problem. This approach has been successfully used in
various models and large-scale applications. In this review, we give an
overview of recent trends in variational inference. We first introduce standard
mean field variational inference, then review recent advances focusing on the
following aspects: (a) scalable VI, which includes stochastic approximations,
(b) generic VI, which extends the applicability of VI to a large class of
otherwise intractable models, such as non-conjugate models, (c) accurate VI,
which includes variational models beyond the mean field approximation or with
atypical divergences, and (d) amortized VI, which implements the inference over
local latent variables with inference networks. Finally, we provide a summary
of promising future research directions.
People, Penguins and Petri Dishes: Adapting Object Counting Models To New Visual Domains And Object Types Without Forgetting
Authors: Mark Marsden, Kevin McGuinness, Suzanne Little, Ciara E. Keogh, Noel E. O'Connor
Date: 2017-11-15 00:00:00
Proceeding: CVPR 2018 6
Tasks: Cultural Vocal Bursts Intensity Prediction, Object, Object Counting

In this paper we propose a technique to adapt a convolutional neural network
(CNN) based object counter to additional visual domains and object types while
still preserving the original counting function. Domain-specific normalisation
and scaling operators are trained to allow the model to adjust to the
statistical distributions of the various visual domains. The developed
adaptation technique is used to produce a singular patch-based counting
regressor capable of counting various object types including people, vehicles,
cell nuclei and wildlife. As part of this study a challenging new cell counting
dataset in the context of tissue culture and patient diagnosis is constructed.
This new collection, referred to as the Dublin Cell Counting (DCC) dataset, is
the first of its kind to be made available to the wider computer vision
community. State-of-the-art object counting performance is achieved in both the
Shanghaitech (parts A and B) and Penguins datasets while competitive
performance is observed on the TRANCOS and Modified Bone Marrow (MBM) datasets,
all using a shared counting model.
Stochastic Optimization with Variance Reduction for Infinite Datasets with Finite-Sum Structure
Authors: Alberto Bietti, Julien Mairal
Date: 2016-10-04 00:00:00
Proceeding: NeurIPS 2017
Tasks: Data Augmentation, Stochastic Optimization

Stochastic optimization algorithms with variance reduction have proven
successful for minimizing large finite sums of functions. Unfortunately, these
techniques are unable to deal with stochastic perturbations of input data,
induced for example by data augmentation. In such cases, the objective is no
longer a finite sum, and the main candidate for optimization is the stochastic
gradient descent method (SGD). In this paper, we introduce a variance reduction
approach for these settings when the objective is composite and strongly
convex. The convergence rate outperforms SGD with a typically much smaller
constant factor, which depends on the variance of gradient estimates only due
to perturbations on a single example.
Dialogue Act Recognition via CRF-Attentive Structured Network
Authors: Zheqian Chen, Rongqin Yang, Zhou Zhao, Deng Cai, Xiaofei He
Date: 2017-11-15 00:00:00
Proceeding: SIGIR 2018 7
Tasks: Dialogue Act Classification, Dialogue Interpretation, Structured Prediction

Dialogue Act Recognition (DAR) is a challenging problem in dialogue
interpretation, which aims to attach semantic labels to utterances and
characterize the speaker's intention. Currently, many existing approaches
formulate the DAR problem ranging from multi-classification to structured
prediction, which suffer from handcrafted feature extensions and attentive
contextual structural dependencies. In this paper, we consider the problem of
DAR from the viewpoint of extending richer Conditional Random Field (CRF)
structural dependencies without abandoning end-to-end training. We incorporate
hierarchical semantic inference with memory mechanism on the utterance
modeling. We then extend structured attention network to the linear-chain
conditional random field layer which takes into account both contextual
utterances and corresponding dialogue acts. The extensive experiments on two
major benchmark datasets Switchboard Dialogue Act (SWDA) and Meeting Recorder
Dialogue Act (MRDA) datasets show that our method achieves better performance
than other state-of-the-art solutions to the problem. It is a remarkable fact
that our method is nearly close to the human annotator's performance on SWDA
within 2% gap.
Variational Adaptive-Newton Method for Explorative Learning
Authors: Mohammad Emtiyaz Khan, Wu Lin, Voot Tangkaratt, Zuozhu Liu, Didrik Nielsen
Date: 2017-11-15 00:00:00
Tasks: Active Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Variational Inference

We present the Variational Adaptive Newton (VAN) method which is a black-box
optimization method especially suitable for explorative-learning tasks such as
active learning and reinforcement learning. Similar to Bayesian methods, VAN
estimates a distribution that can be used for exploration, but requires
computations that are similar to continuous optimization methods. Our
theoretical contribution reveals that VAN is a second-order method that unifies
existing methods in distinct fields of continuous optimization, variational
inference, and evolution strategies. Our experimental results show that VAN
performs well on a wide-variety of learning tasks. This work presents a
general-purpose explorative-learning method that has the potential to improve
learning in areas such as active learning and reinforcement learning.