pwc-archive/papers-with-abstracts huggingface dataset:
A Universal Part-of-Speech Tagset
Authors: Slav Petrov, Dipanjan Das, Ryan Mcdonald
Date: 2011-04-11 00:00:00
Proceeding: LREC 2012 5

To facilitate future research in unsupervised induction of syntactic
structure and to standardize best-practices, we propose a tagset that consists
of twelve universal part-of-speech categories. In addition to the tagset, we
develop a mapping from 25 different treebank tagsets to this universal set. As
a result, when combined with the original treebank data, this universal tagset
and mapping produce a dataset consisting of common parts-of-speech for 22
different languages. We highlight the use of this resource via two experiments,
including one that reports competitive accuracies for unsupervised grammar
induction without gold standard part-of-speech tags.
Maximal Divergence Sequential Autoencoder for Binary Software Vulnerability Detection
Authors: Tue Le, Tuan Nguyen, Trung Le, Dinh Phung, Paul Montague, Olivier De Vel, Lizhen Qu
Date: 2019-05-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Computer Security, Vulnerability Detection

Due to the sharp increase in the severity of the threat imposed by software vulnerabilities, the detection of vulnerabilities in binary code has become an important concern in the software industry, such as the embedded systems industry, and in the field of computer security. However, most of the work in binary code vulnerability detection has relied on handcrafted features which are manually chosen by a select few, knowledgeable domain experts. In this paper, we attempt to alleviate this severe binary vulnerability detection bottleneck by leveraging recent advances in deep learning representations and propose the Maximal Divergence Sequential Auto-Encoder. In particular, latent codes representing vulnerable and non-vulnerable binaries are encouraged to be maximally divergent, while still being able to maintain crucial information from the original binaries. We conducted extensive experiments to compare and contrast our proposed methods with the baselines, and the results show that our proposed methods outperform the baselines in all performance measures of interest.
End-to-end speech recognition using lattice-free MMI
Authors: Hossein Hadian, Hossein Sameti, Daniel Povey, Sanjeev Khudanpur
Date: 2018-09-06 00:00:00
Proceeding: Interspeech 2018 2018 9
Tasks: speech-recognition, Speech Recognition

We present our work on end-to-end training of acoustic models
using the lattice-free maximum mutual information (LF-MMI)
objective function in the context of hidden Markov models.
By end-to-end training, we mean flat-start training of a single
DNN in one stage without using any previously trained models,
forced alignments, or building state-tying decision trees. We
use full biphones to enable context-dependent modeling without trees, and show that our end-to-end LF-MMI approach can
achieve comparable results to regular LF-MMI on well-known
large vocabulary tasks. We also compare with other end-to-end
methods such as CTC in character-based and lexicon-free settings and show 5 to 25 percent relative reduction in word error rates on different large vocabulary tasks while using significantly smaller models.
Inferring hybrid transportation modes from sparse GPS data using a moving window SVM classification
Authors: Adel Bolbol, Tao Cheng, Ioannis Tsapakis, James Haworth
Date: 2012-11-01 00:00:00
Proceeding: Computers, Environment and Urban Systems 2012 11
Tasks: Classification, General Classification, Trajectory Prediction

Understanding travel behaviour and travel demand is of constant importance to transportation communities and agencies in every country. Nowadays, attempts have been made to automatically infer transportation modes from positional data, such as the data collected by using GPS devices so that the cost in time and budget of conventional travel diary survey could be significantly reduced. Some limitations, however, exist in the literature, in aspects of data collection (sample size selected, duration of study, granularity of data), selection of variables (or combination of variables), and method of inference (the number of transportation modes to be used in the learning). This paper therefore, attempts to fully understand these aspects in the process of inference. We aim to solve a classification problem of GPS data into different transportation modes (car, walk, cycle, underground, train and bus). We first study the variables that could contribute positively to this classification, and statistically quantify their discriminatory power. We then introduce a novel approach to carry out this inference using a framework based on Support Vector Machines (SVMs) classification. The framework was tested using coarse-grained GPS data, which has been avoided in previous studies, achieving a promising accuracy of 88% with a Kappa statistic reflecting almost perfect agreement.
Impact of weather conditions on macroscopic urban travel times
Authors: Ioannis Tsapakis, Tao Cheng, Adel Bolbol
Date: 2013-04-01 00:00:00
Proceeding: Journal of Transport Geography 2013 4
Tasks: Management, Trajectory Prediction

Weather conditions may significantly impact a series of everyday human decisions and activities. As a result, engineers seek to integrate weather-related data into traffic operations in order to improve the current state of practice. Travel times and speeds are two of the elements of a transportation system that may be greatly affected by the weather resulting in deterioration of roadway network performance. This study aims to investigate the impact of different intensities of rain, snow and temperature levels on macroscopic travel times in the Greater London area (UK) during the period 1 October–10 December 2009. The analysis was carried out for three 2-h periods on weekdays during the morning, afternoon and evening periods. Automatic Number Plate Recognition (ANPR) data obtained from more than 380 travel links are used in the analysis. The main finding is that the impact of rain and snow is a function of their intensity. Specifically, the ranges of the total travel time increase due to light, moderate and heavy rain are: 0.1–2.1%, 1.5–3.8%, and 4.0–6.0% respectively. Light snow results in travel time increases of 5.5–7.6%, whilst heavy snow causes the highest percentage delays spanning from 7.4% to 11.4%. Temperature has nearly negligible effects on travel times. It was also found that the longer links within outer London generally yield greater travel time decreases than those in inner London, and even higher decreases than the shortest links in central London. This research provides planners with additional information that can be used in traffic management to modify planning decisions and improve the transportation system control on a network scale under different weather conditions. In order to determine whether the weather effects are region-specific, continued research is needed to replicate this study in other areas that exhibit different characteristics.
RNN-SM: Fast Steganalysis of VoIP Streams Using Recurrent Neural Network
Authors: Zinan Lin, Yongfeng Huang, Jilong Wang
Date: 2018-02-15 00:00:00
Proceeding: IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY 2018 2
Tasks: Quantization, Steganalysis

Quantization index modulation (QIM) steganography makes it possible to hide secret information in voice-over IP (VoIP) streams, which could be utilized by unauthorized entities to set up covert channels for malicious purposes. Detecting short QIM steganography samples, as is required by real circumstances, remains an unsolved challenge. In this paper, we propose an effective online steganalysis method to detect QIM steganography. We find four strong codeword correlation patterns in VoIP streams, which will be distorted after embedding with hidden data. To extract those correlation features, we propose the codeword correlation model, which is based on recurrent neural network (RNN). Furthermore, we propose the feature classification model to classify those correlation features into cover speech and stego speech categories. The whole RNN-based steganalysis model (RNN-SM) is trained in a supervised learning framework. Experiments show that on full embedding rate samples, RNN-SM is of high detection accuracy, which remains over 90% even when the sample is as short as 0.1 s, and is significantly higher than other state-of-the-art methods. For the challenging task of conducting steganalysis towards low embedding rate samples, RNN-SM also achieves a high accuracy. The average testing time for each sample is below 0.15% of sample length. These clues show that RNN-SM meets the short sample detection demand and is a state-of-the-art algorithm for online VoIP steganalysis.
Vector space explorations of literary language
Authors: Andreas van Cranenburgh, Karina van Dalen-Oskam, Joris van Zundert
Date: 2019-02-09 00:00:00
Proceeding: Language Resources and Evaluation 2019 2
Tasks: Topic Models

Literary novels are said to distinguish themselves from other novels through conventions associated with literariness. We investigate the task of predicting the literariness of novels as perceived by readers, based on a large reader survey of contemporary Dutch novels. Previous research showed that ratings of literariness are predictable from texts to a substantial extent using machine learning, suggesting that it may be possible to explain the consensus among readers on which novels are literary as a consensus on the kind of writing style that characterizes literature. Although we have not yet collected human judgments to establish the influence of writing style directly (we use a survey with judgments based on the titles of novels), we can try to analyze the behavior of machine learning models on particular text fragments as a proxy for human judgments. In order to explore aspects of the texts associated with literariness, we divide the texts of the novels in chunks of 2--3 pages and create vector space representations using topic models (Latent Dirichlet Allocation) and neural document embeddings (Distributed Bag-of-Words Paragraph Vectors). We analyze the semantic complexity of the novels using distance measures, supporting the notion that literariness can be partly explained as a deviation from the norm. Furthermore, we build predictive models and identify specific keywords and stylistic markers related to literariness. While genre plays a role, we find that the greater part of factors affecting judgments of literariness are explicable in bag-of-words terms,even in short text fragments and among novels with higher literary ratings. The code and notebook used to produce the results in this paper are available at https://github.com/andreasvc/litvecspace.
Wasserstein Barycenter Model Ensembling
Authors: Pierre Dognin*, Igor Melnyk*, Youssef Mroueh*, Jarret Ross*, Cicero Dos Santos*, Tom Sercu*
Date: 2019-05-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Attribute, General Classification, Image Captioning, model, Word Embeddings

In this paper we propose to perform model ensembling in a multiclass or a multilabel learning setting using Wasserstein (W.) barycenters. Optimal transport metrics, such as the Wasserstein distance, allow incorporating semantic side information such as word embeddings. Using W. barycenters to find the consensus between models allows us to balance confidence and semantics in finding the agreement between the models. We show applications of Wasserstein ensembling in attribute-based classification, multilabel learning and image captioning generation. These results show that the W. ensembling is a viable alternative to the basic geometric or arithmetic mean ensembling.
Understanding Fashionability: What drives sales of a style?
Authors: Aniket Jain, Yadunath Gupta, Pawan Kumar Singh, Aruna Rajan
Date: 2018-06-28 00:00:00
Tasks: Articles

We use customer demand data for fashion articles on Myntra, and derive a
fashionability or style quotient, which represents customer demand for the
stylistic content of a fashion article, decoupled with its commercials (price,
offers, etc.). We demonstrate learning for assortment planning in fashion that
would aim to keep a healthy mix of breadth and depth across various styles, and
we show the relationship between a customer's perception of a style vs a
merchandiser's catalogue of styles. We also backtest our method to calculate
prediction errors in our style quotient and customer demand, and discuss
various implications and findings.
Bag of Freebies for Training Object Detection Neural Networks
Authors: Zhi Zhang, Tong He, Hang Zhang, Zhongyue Zhang, Junyuan Xie, Mu Li
Date: 2019-02-11 00:00:00
Tasks: General Classification, image-classification, Image Classification, object-detection, Object Detection

Training heuristics greatly improve various image classification model
accuracies~\cite{he2018bag}. Object detection models, however, have more
complex neural network structures and optimization targets. The training
strategies and pipelines dramatically vary among different models. In this
works, we explore training tweaks that apply to various models including Faster
R-CNN and YOLOv3. These tweaks do not change the model architectures,
therefore, the inference costs remain the same. Our empirical results
demonstrate that, however, these freebies can improve up to 5% absolute
precision compared to state-of-the-art baselines.
You Only Look & Listen Once: Towards Fast and Accurate Visual Grounding
Authors: Chaorui Deng, Qi Wu, Guanghui Xu, ZhuLiang Yu, Yanwu Xu, Kui Jia, Mingkui Tan
Date: 2019-02-12 00:00:00
Tasks: object-detection, Object Detection, Region Proposal, Visual Grounding

Visual Grounding (VG) aims to locate the most relevant region in an image,
based on a flexible natural language query but not a pre-defined label, thus it
can be a more useful technique than object detection in practice. Most
state-of-the-art methods in VG operate in a two-stage manner, wherein the first
stage an object detector is adopted to generate a set of object proposals from
the input image and the second stage is simply formulated as a cross-modal
matching problem that finds the best match between the language query and all
region proposals. This is rather inefficient because there might be hundreds of
proposals produced in the first stage that need to be compared in the second
stage, not to mention this strategy performs inaccurately. In this paper, we
propose an simple, intuitive and much more elegant one-stage detection based
method that joints the region proposal and matching stage as a single detection
network. The detection is conditioned on the input query with a stack of novel
Relation-to-Attention modules that transform the image-to-query relationship to
an relation map, which is used to predict the bounding box directly without
proposing large numbers of useless region proposals. During the inference, our
approach is about 20x ~ 30x faster than previous methods and, remarkably, it
achieves 18% ~ 41% absolute performance improvement on top of the
state-of-the-art results on several benchmark datasets. We release our code and
all the pre-trained models at https://github.com/openblack/rvg.
Net2Vis -- A Visual Grammar for Automatically Generating Publication-Tailored CNN Architecture Visualizations
Authors: Alex Bäuerle, Christian van Onzenoodt, Timo Ropinski
Date: 2019-02-11 00:00:00

To convey neural network architectures in publications, appropriate visualizations are of great importance. While most current deep learning papers contain such visualizations, these are usually handcrafted just before publication, which results in a lack of a common visual grammar, significant time investment, errors, and ambiguities. Current automatic network visualization tools focus on debugging the network itself and are not ideal for generating publication visualizations. Therefore, we present an approach to automate this process by translating network architectures specified in Keras into visualizations that can directly be embedded into any publication. To do so, we propose a visual grammar for convolutional neural networks (CNNs), which has been derived from an analysis of such figures extracted from all ICCV and CVPR papers published between 2013 and 2019. The proposed grammar incorporates visual encoding, network layout, layer aggregation, and legend generation. We have further realized our approach in an online system available to the community, which we have evaluated through expert feedback, and a quantitative study. It not only reduces the time needed to generate network visualizations for publications, but also enables a unified and unambiguous visualization design.
MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation
Authors: Chen Liu, Yasutaka Furukawa
Date: 2019-02-12 00:00:00
Tasks: 3D Instance Segmentation, Clustering, Instance Segmentation, Segmentation, Semantic Segmentation

We propose a new approach for 3D instance segmentation based on sparse
convolution and point affinity prediction, which indicates the likelihood of
two points belonging to the same instance. The proposed network, built upon
submanifold sparse convolution [3], processes a voxelized point cloud and
predicts semantic scores for each occupied voxel as well as the affinity
between neighboring voxels at different scales. A simple yet effective
clustering algorithm segments points into instances based on the predicted
affinity and the mesh topology. The semantic for each instance is determined by
the semantic prediction. Experiments show that our method outperforms the
state-of-the-art instance segmentation methods by a large margin on the widely
used ScanNet benchmark [2]. We share our code publicly at
https://github.com/art-programmer/MASC.
Capacity allocation analysis of neural networks: A tool for principled architecture design
Authors: Jonathan Donier
Date: 2019-02-12 00:00:00

Designing neural network architectures is a task that lies somewhere between
science and art. For a given task, some architectures are eventually preferred
over others, based on a mix of intuition, experience, experimentation and luck.
For many tasks, the final word is attributed to the loss function, while for
some others a further perceptual evaluation is necessary to assess and compare
performance across models. In this paper, we introduce the concept of capacity
allocation analysis, with the aim of shedding some light on what network
architectures focus their modelling capacity on, when used on a given task. We
focus more particularly on spatial capacity allocation, which analyzes a
posteriori the effective number of parameters that a given model has allocated
for modelling dependencies on a given point or region in the input space, in
linear settings. We use this framework to perform a quantitative comparison
between some classical architectures on various synthetic tasks. Finally, we
consider how capacity allocation might translate in non-linear settings.
ELF OpenGo: An Analysis and Open Reimplementation of AlphaZero
Authors: Yuandong Tian, Jerry Ma, Qucheng Gong, Shubho Sengupta, Zhuoyuan Chen, James Pinkerton, C. Lawrence Zitnick
Date: 2019-02-12 00:00:00
Tasks: Game of Go

The AlphaGo, AlphaGo Zero, and AlphaZero series of algorithms are remarkable demonstrations of deep reinforcement learning's capabilities, achieving superhuman performance in the complex game of Go with progressively increasing autonomy. However, many obstacles remain in the understanding of and usability of these promising approaches by the research community. Toward elucidating unresolved mysteries and facilitating future research, we propose ELF OpenGo, an open-source reimplementation of the AlphaZero algorithm. ELF OpenGo is the first open-source Go AI to convincingly demonstrate superhuman performance with a perfect (20:0) record against global top professionals. We apply ELF OpenGo to conduct extensive ablation studies, and to identify and analyze numerous interesting phenomena in both the model training and in the gameplay inference procedures. Our code, models, selfplay datasets, and auxiliary data are publicly available at https://ai.facebook.com/tools/elf-opengo/.
Fast-SCNN: Fast Semantic Segmentation Network
Authors: Rudra P. K. Poudel, Stephan Liwicki, Roberto Cipolla
Date: 2019-02-12 00:00:00
Tasks: Decoder, Image Segmentation, Real-Time Semantic Segmentation, Segmentation, Semantic Segmentation, Thermal Image Segmentation

The encoder-decoder framework is state-of-the-art for offline semantic image
segmentation. Since the rise in autonomous systems, real-time computation is
increasingly desirable. In this paper, we introduce fast segmentation
convolutional neural network (Fast-SCNN), an above real-time semantic
segmentation model on high resolution image data (1024x2048px) suited to
efficient computation on embedded devices with low memory. Building on existing
two-branch methods for fast segmentation, we introduce our `learning to
downsample' module which computes low-level features for multiple resolution
branches simultaneously. Our network combines spatial detail at high resolution
with deep features extracted at lower resolution, yielding an accuracy of 68.0%
mean intersection over union at 123.5 frames per second on Cityscapes. We also
show that large scale pre-training is unnecessary. We thoroughly validate our
metric in experiments with ImageNet pre-training and the coarse labeled data of
Cityscapes. Finally, we show even faster computation with competitive results
on subsampled inputs, without any network modifications.
Maximum Likelihood Estimation for Learning Populations of Parameters
Authors: Ramya Korlakai Vinayak, Weihao Kong, Gregory Valiant, Sham M. Kakade
Date: 2019-02-12 00:00:00

Consider a setting with $N$ independent individuals, each with an unknown
parameter, $p_i \in [0, 1]$ drawn from some unknown distribution $P^\star$.
After observing the outcomes of $t$ independent Bernoulli trials, i.e., $X_i
\sim \text{Binomial}(t, p_i)$ per individual, our objective is to accurately
estimate $P^\star$. This problem arises in numerous domains, including the
social sciences, psychology, health-care, and biology, where the size of the
population under study is usually large while the number of observations per
individual is often limited. Our main result shows that, in the regime where $t
\ll N$, the maximum likelihood estimator (MLE) is both statistically minimax
optimal and efficiently computable. Precisely, for sufficiently large $N$, the
MLE achieves the information theoretic optimal error bound of
$\mathcal{O}(\frac{1}{t})$ for $t < c\log{N}$, with regards to the earth
mover's distance (between the estimated and true distributions). More
generally, in an exponentially large interval of $t$ beyond $c \log{N}$, the
MLE achieves the minimax error bound of $\mathcal{O}(\frac{1}{\sqrt{t\log
N}})$. In contrast, regardless of how large $N$ is, the naive "plug-in"
estimator for this problem only achieves the sub-optimal error of
$\Theta(\frac{1}{\sqrt{t}})$.
Infinite Mixture Prototypes for Few-Shot Learning
Authors: Kelsey R. Allen, Evan Shelhamer, Hanul Shin, Joshua B. Tenenbaum
Date: 2019-02-12 00:00:00
Tasks: Clustering, Few-Shot Learning

We propose infinite mixture prototypes to adaptively represent both simple
and complex data distributions for few-shot learning. Our infinite mixture
prototypes represent each class by a set of clusters, unlike existing
prototypical methods that represent each class by a single cluster. By
inferring the number of clusters, infinite mixture prototypes interpolate
between nearest neighbor and prototypical representations, which improves
accuracy and robustness in the few-shot regime. We show the importance of
adaptive capacity for capturing complex data distributions such as alphabets,
with 25% absolute accuracy improvements over prototypical networks, while still
maintaining or improving accuracy on the standard Omniglot and mini-ImageNet
benchmarks. In clustering labeled and unlabeled data by the same clustering
rule, infinite mixture prototypes achieves state-of-the-art semi-supervised
accuracy. As a further capability, we show that infinite mixture prototypes can
perform purely unsupervised clustering, unlike existing prototypical methods.