pwc-archive/papers-with-abstracts huggingface dataset:
DSFD: Dual Shot Face Detector
Authors: Jian Li, Yabiao Wang, Changan Wang, Ying Tai, Jianjun Qian, Jian Yang, Chengjie Wang, Jilin Li, Feiyue Huang
Date: 2018-10-24 00:00:00
Proceeding: CVPR 2019 6
Tasks: Data Augmentation, Face Detection, Occluded Face Detection

In this paper, we propose a novel face detection network with three novel
contributions that address three key aspects of face detection, including
better feature learning, progressive loss design and anchor assign based data
augmentation, respectively. First, we propose a Feature Enhance Module (FEM)
for enhancing the original feature maps to extend the single shot detector to
dual shot detector. Second, we adopt Progressive Anchor Loss (PAL) computed by
two different sets of anchors to effectively facilitate the features. Third, we
use an Improved Anchor Matching (IAM) by integrating novel anchor assign
strategy into data augmentation to provide better initialization for the
regressor. Since these techniques are all related to the two-stream design, we
name the proposed network as Dual Shot Face Detector (DSFD). Extensive
experiments on popular benchmarks, WIDER FACE and FDDB, demonstrate the
superiority of DSFD over the state-of-the-art face detectors.
First-order Convergence Theory for Weakly-Convex-Weakly-Concave Min-max Problems
Authors: Mingrui Liu, Hassan Rafique, Qihang Lin, Tianbao Yang
Date: 2018-10-24 00:00:00

In this paper, we consider first-order convergence theory and algorithms for solving a class of non-convex non-concave min-max saddle-point problems, whose objective function is weakly convex in the variables of minimization and weakly concave in the variables of maximization. It has many important applications in machine learning including training Generative Adversarial Nets (GANs). We propose an algorithmic framework motivated by the inexact proximal point method, where the weakly monotone variational inequality (VI) corresponding to the original min-max problem is solved through approximately solving a sequence of strongly monotone VIs constructed by adding a strongly monotone mapping to the original gradient mapping. We prove first-order convergence to a nearly stationary solution of the original min-max problem of the generic algorithmic framework and establish different rates by employing different algorithms for solving each strongly monotone VI. Experiments verify the convergence theory and also demonstrate the effectiveness of the proposed methods on training GANs.
Automated Evaluation of Semantic Segmentation Robustness for Autonomous Driving
Authors: Wei Zhou, Julie Stephany Berrio, Stewart Worrall, Eduardo Nebot
Date: 2018-10-24 00:00:00
Tasks: Autonomous Driving, Autonomous Vehicles, General Classification, Segmentation, Semantic Segmentation

One of the fundamental challenges in the design of perception systems for
autonomous vehicles is validating the performance of each algorithm under a
comprehensive variety of operating conditions. In the case of vision-based
semantic segmentation, there are known issues when encountering new scenarios
that are sufficiently different to the training data. In addition, even small
variations in environmental conditions such as illumination and precipitation
can affect the classification performance of the segmentation model. Given the
reliance on visual information, these effects often translate into poor
semantic pixel classification which can potentially lead to catastrophic
consequences when driving autonomously. This paper presents a novel method for
analysing the robustness of semantic segmentation models and provides a number
of metrics to evaluate the classification performance over a variety of
environmental conditions. The process incorporates an additional sensor (lidar)
to automate the process, eliminating the need for labour-intensive hand
labelling of validation data. The system integrity can be monitored as the
performance of the vision sensors are validated against a different sensor
modality. This is necessary for detecting failures that are inherent to vision
technology. Experimental results are presented based on multiple datasets
collected at different times of the year with different environmental
conditions. These results show that the semantic segmentation performance
varies depending on the weather, camera parameters, existence of shadows, etc..
The results also demonstrate how the metrics can be used to compare and
validate the performance after making improvements to a model, and compare the
performance of different networks.
Solving Poisson's Equation using Deep Learning in Particle Simulation of PN Junction
Authors: Zhongyang Zhang, Ling Zhang, Ze Sun, Nicholas Erickson, Ryan From, Jun Fan
Date: 2018-10-24 00:00:00
Tasks: Deep Learning

Simulating the dynamic characteristics of a PN junction at the microscopic
level requires solving the Poisson's equation at every time step. Solving at
every time step is a necessary but time-consuming process when using the
traditional finite difference (FDM) approach. Deep learning is a powerful
technique to fit complex functions. In this work, deep learning is utilized to
accelerate solving Poisson's equation in a PN junction. The role of the
boundary condition is emphasized in the loss function to ensure a better
fitting. The resulting I-V curve for the PN junction, using the deep learning
solver presented in this work, shows a perfect match to the I-V curve obtained
using the finite difference method, with the advantage of being 10 times faster
at every time step.
Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks
Authors: Michelle A. Lee, Yuke Zhu, Krishnan Srinivasan, Parth Shah, Silvio Savarese, Li Fei-Fei, Animesh Garg, Jeannette Bohg
Date: 2018-10-24 00:00:00
Tasks: Contact-rich Manipulation, Deep Reinforcement Learning, Reinforcement Learning, Reinforcement Learning (RL), Self-Supervised Learning

Contact-rich manipulation tasks in unstructured environments often require
both haptic and visual feedback. However, it is non-trivial to manually design
a robot controller that combines modalities with very different
characteristics. While deep reinforcement learning has shown success in
learning control policies for high-dimensional inputs, these algorithms are
generally intractable to deploy on real robots due to sample complexity. We use
self-supervision to learn a compact and multimodal representation of our
sensory inputs, which can then be used to improve the sample efficiency of our
policy learning. We evaluate our method on a peg insertion task, generalizing
over different geometry, configurations, and clearances, while being robust to
external perturbations. Results for simulated and real robot experiments are
presented.
Fault Area Detection in Leaf Diseases using k-means Clustering
Authors: Subhajit Maity, Sujan Sarkar, Avinaba Tapadar, Ayan Dutta, Sanket Biswas, Sayon Nayek, Pritam Saha
Date: 2018-10-24 00:00:00
Tasks: Clustering

With increasing population the crisis of food is getting bigger day by day.In
this time of crisis,the leaf disease of crops is the biggest problem in the
food industry.In this paper, we have addressed that problem and proposed an
efficient method to detect leaf disease.Leaf diseases can be detected from
sample images of the leaf with the help of image processing and
segmentation.Using k-means clustering and Otsu's method the faulty region in a
leaf is detected which helps to determine proper course of action to be
taken.Further the ratio of normal and faulty region if calculated would be able
to predict if the leaf can be cured at all.
Multi-Head Attention with Disagreement Regularization
Authors: Jian Li, Zhaopeng Tu, Baosong Yang, Michael R. Lyu, Tong Zhang
Date: 2018-10-24 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Diversity, Translation

Multi-head attention is appealing for the ability to jointly attend to
information from different representation subspaces at different positions. In
this work, we introduce a disagreement regularization to explicitly encourage
the diversity among multiple attention heads. Specifically, we propose three
types of disagreement regularization, which respectively encourage the
subspace, the attended positions, and the output representation associated with
each attention head to be different from other heads. Experimental results on
widely-used WMT14 English-German and WMT17 Chinese-English translation tasks
demonstrate the effectiveness and universality of the proposed approach.
Modeling Localness for Self-Attention Networks
Authors: Baosong Yang, Zhaopeng Tu, Derek F. Wong, Fandong Meng, Lidia S. Chao, Tong Zhang
Date: 2018-10-24 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Machine Translation, Translation

Self-attention networks have proven to be of profound value for its strength
of capturing global dependencies. In this work, we propose to model localness
for self-attention networks, which enhances the ability of capturing useful
local context. We cast localness modeling as a learnable Gaussian bias, which
indicates the central and scope of the local region to be paid more attention.
The bias is then incorporated into the original attention distribution to form
a revised distribution. To maintain the strength of capturing long distance
dependencies and enhance the ability of capturing short-range dependencies, we
only apply localness modeling to lower layers of self-attention networks.
Quantitative and qualitative analyses on Chinese-English and English-German
translation tasks demonstrate the effectiveness and universality of the
proposed approach.
Exploiting Deep Representations for Neural Machine Translation
Authors: Zi-Yi Dou, Zhaopeng Tu, Xing Wang, Shuming Shi, Tong Zhang
Date: 2018-10-24 00:00:00
Proceeding: EMNLP 2018 10
Tasks: Decoder, Machine Translation, NMT, Translation

Advanced neural machine translation (NMT) models generally implement encoder
and decoder as multiple layers, which allows systems to model complex functions
and capture complicated linguistic structures. However, only the top layers of
encoder and decoder are leveraged in the subsequent process, which misses the
opportunity to exploit the useful information embedded in other layers. In this
work, we propose to simultaneously expose all of these signals with layer
aggregation and multi-layer attention mechanisms. In addition, we introduce an
auxiliary regularization term to encourage different layers to capture diverse
information. Experimental results on widely-used WMT14 English-German and WMT17
Chinese-English translation data demonstrate the effectiveness and universality
of the proposed approach.
Understanding and correcting pathologies in the training of learned optimizers
Authors: Luke Metz, Niru Maheswaranathan, Jeremy Nixon, C. Daniel Freeman, Jascha Sohl-Dickstein
Date: 2018-10-24 00:00:00

Deep learning has shown that learned functions can dramatically outperform hand-designed functions on perceptual tasks. Analogously, this suggests that learned optimizers may similarly outperform current hand-designed optimizers, especially for specific problems. However, learned optimizers are notoriously difficult to train and have yet to demonstrate wall-clock speedups over hand-designed optimizers, and thus are rarely used in practice. Typically, learned optimizers are trained by truncated backpropagation through an unrolled optimization process resulting in gradients that are either strongly biased (for short truncations) or have exploding norm (for long truncations). In this work we propose a training scheme which overcomes both of these difficulties, by dynamically weighting two unbiased gradient estimators for a variational loss on optimizer performance, allowing us to train neural networks to perform optimization of a specific task faster than tuned first-order methods. We demonstrate these results on problems where our learned optimizer trains convolutional networks faster in wall-clock time compared to tuned first-order methods and with an improvement in test loss.
Text Embeddings for Retrieval From a Large Knowledge Base
Authors: Tolgahan Cakaloglu, Christian Szegedy, Xiaowei Xu
Date: 2018-10-24 00:00:00
Proceeding: ICLR 2019 5
Tasks: Open-Domain Question Answering, Question Answering, Retrieval

Text embedding representing natural language documents in a semantic vector space can be used for document retrieval using nearest neighbor lookup. In order to study the feasibility of neural models specialized for retrieval in a semantically meaningful way, we suggest the use of the Stanford Question Answering Dataset (SQuAD) in an open-domain question answering context, where the first task is to find paragraphs useful for answering a given question. First, we compare the quality of various text-embedding methods on the performance of retrieval and give an extensive empirical comparison on the performance of various non-augmented base embedding with, and without IDF weighting. Our main results are that by training deep residual neural models, specifically for retrieval purposes, can yield significant gains when it is used to augment existing embeddings. We also establish that deeper models are superior to this task. The best base baseline embeddings augmented by our learned neural approach improves the top-1 paragraph recall of the system by 14%.
Data-driven Blockbuster Planning on Online Movie Knowledge Library
Authors: Ye Liu, Jiawei Zhang, Chenwei Zhang, Philip S. Yu
Date: 2018-10-24 00:00:00

In the era of big data, logistic planning can be made data-driven to take
advantage of accumulated knowledge in the past. While in the movie industry,
movie planning can also exploit the existing online movie knowledge library to
achieve better results. However, it is ineffective to solely rely on
conventional heuristics for movie planning, due to a large number of existing
movies and various real-world factors that contribute to the success of each
movie, such as the movie genre, available budget, production team (involving
actor, actress, director, and writer), etc. In this paper, we study a
"Blockbuster Planning" (BP) problem to learn from previous movies and plan for
low budget yet high return new movies in a totally data-driven fashion. After a
thorough investigation of an online movie knowledge library, a novel movie
planning framework "Blockbuster Planning with Maximized Movie Configuration
Acquaintance" (BigMovie) is introduced in this paper. From the investment
perspective, BigMovie maximizes the estimated gross of the planned movies with
a given budget. It is able to accurately estimate the movie gross with a 0.26
mean absolute percentage error (and 0.16 for budget). Meanwhile, from the
production team's perspective, BigMovie is able to formulate an optimized team
with people/movie genres that team members are acquainted with. Historical
collaboration records are utilized to estimate acquaintance scores of movie
configuration factors via an acquaintance tensor. We formulate the BP problem
as a non-linear binary programming problem and prove its NP-hardness. To solve
it in polynomial time, BigMovie relaxes the hard binary constraints and
addresses the BP problem as a cubic programming problem. Extensive experiments
conducted on IMDB movie database demonstrate the capability of BigMovie for an
effective data-driven blockbuster planning.
Modified Multidimensional Scaling and High Dimensional Clustering
Authors: Xiucai Ding, Qiang Sun
Date: 2018-10-24 00:00:00
Tasks: Clustering, Dimensionality Reduction, Vocal Bursts Intensity Prediction

Multidimensional scaling is an important dimension reduction tool in statistics and machine learning. Yet few theoretical results characterizing its statistical performance exist, not to mention any in high dimensions. By considering a unified framework that includes low, moderate and high dimensions, we study multidimensional scaling in the setting of clustering noisy data. Our results suggest that, the classical multidimensional scaling can be modified to further improve the quality of embedded samples, especially when the noise level increases. To this end, we propose {\it modified multidimensional scaling} which applies a nonlinear transformation to the sample eigenvalues. The nonlinear transformation depends on the dimensionality, sample size and moment of noise. We show that modified multidimensional scaling followed by various clustering algorithms can achieve exact recovery, i.e., all the cluster labels can be recovered correctly with probability tending to one. Numerical simulations and two real data applications lend strong support to our proposed methodology.
Nonconvex and Nonsmooth Sparse Optimization via Adaptively Iterative Reweighted Methods
Authors: Hao Wang, Fan Zhang, Yuanming Shi, Yaohua Hu
Date: 2018-10-24 00:00:00

We propose a general formulation of nonconvex and nonsmooth sparse optimization problems with convex set constraint, which can take into account most existing types of nonconvex sparsity-inducing terms, bringing strong applicability to a wide range of applications. We design a general algorithmic framework of iteratively reweighted algorithms for solving the proposed nonconvex and nonsmooth sparse optimization problems, which solves a sequence of weighted convex regularization problems with adaptively updated weights. First-order optimality condition is derived and global convergence results are provided under loose assumptions, making our theoretical results a practical tool for analyzing a family of various reweighted algorithms. The effectiveness and efficiency of our proposed formulation and the algorithms are demonstrated in numerical experiments on various sparse optimization problems.
Resolving Referring Expressions in Images With Labeled Elements
Authors: Nevan Wichers, Dilek Hakkani-Tur, Jindong Chen
Date: 2018-10-24 00:00:00
Tasks: Optical Character Recognition, Optical Character Recognition (OCR), Referring Expression

Images may have elements containing text and a bounding box associated with
them, for example, text identified via optical character recognition on a
computer screen image, or a natural image with labeled objects. We present an
end-to-end trainable architecture to incorporate the information from these
elements and the image to segment/identify the part of the image a natural
language expression is referring to. We calculate an embedding for each element
and then project it onto the corresponding location (i.e., the associated
bounding box) of the image feature map. We show that this architecture gives an
improvement in resolving referring expressions, over only using the image, and
other methods that incorporate the element information. We demonstrate
experimental results on the referring expression datasets based on COCO, and on
a webpage image referring expression dataset that we developed.
Deep Learning with Long Short-Term Memory for Time Series Prediction
Authors: Yuxiu Hua, Zhifeng Zhao, Rongpeng Li, Xianfu Chen, Zhiming Liu, Honggang Zhang
Date: 2018-10-24 00:00:00
Tasks: Time Series, Time Series Analysis, Time Series Prediction

Time series prediction can be generalized as a process that extracts useful
information from historical records and then determines future values. Learning
long-range dependencies that are embedded in time series is often an obstacle
for most algorithms, whereas Long Short-Term Memory (LSTM) solutions, as a
specific kind of scheme in deep learning, promise to effectively overcome the
problem. In this article, we first give a brief introduction to the structure
and forward propagation mechanism of the LSTM model. Then, aiming at reducing
the considerable computing cost of LSTM, we put forward the Random Connectivity
LSTM (RCLSTM) model and test it by predicting traffic and user mobility in
telecommunication networks. Compared to LSTM, RCLSTM is formed via stochastic
connectivity between neurons, which achieves a significant breakthrough in the
architecture formation of neural networks. In this way, the RCLSTM model
exhibits a certain level of sparsity, which leads to an appealing decrease in
the computational complexity and makes the RCLSTM model become more applicable
in latency-stringent application scenarios. In the field of telecommunication
networks, the prediction of traffic series and mobility traces could directly
benefit from this improvement as we further demonstrate that the prediction
accuracy of RCLSTM is comparable to that of the conventional LSTM no matter how
we change the number of training samples or the length of input sequences.
Randomized Gradient Boosting Machine
Authors: Haihao Lu, Rahul Mazumder
Date: 2018-10-24 00:00:00

Gradient Boosting Machine (GBM) introduced by Friedman is a powerful supervised learning algorithm that is very widely used in practice---it routinely features as a leading algorithm in machine learning competitions such as Kaggle and the KDDCup. In spite of the usefulness of GBM in practice, our current theoretical understanding of this method is rather limited. In this work, we propose Randomized Gradient Boosting Machine (RGBM) which leads to substantial computational gains compared to GBM, by using a randomization scheme to reduce search in the space of weak-learners. We derive novel computational guarantees for RGBM. We also provide a principled guideline towards better step-size selection in RGBM that does not require a line search. Our proposed framework is inspired by a special variant of coordinate descent that combines the benefits of randomized coordinate descent and greedy coordinate descent; and may be of independent interest as an optimization algorithm. As a special case, our results for RGBM lead to superior computational guarantees for GBM. Our computational guarantees depend upon a curious geometric quantity that we call Minimal Cosine Angle, which relates to the density of weak-learners in the prediction space. On a series of numerical experiments on real datasets, we demonstrate the effectiveness of RGBM over GBM in terms of obtaining a model with good training and/or testing data fidelity with a fraction of the computational cost.
Automatic Identification of Indicators of Compromise using Neural-Based Sequence Labelling
Authors: Shengping Zhou, Zi Long, Lianzhi Tan, Hao Guo
Date: 2018-10-24 00:00:00
Proceeding: PACLIC 2018 12

Indicators of Compromise (IOCs) are artifacts observed on a network or in an
operating system that can be utilized to indicate a computer intrusion and
detect cyber-attacks in an early stage. Thus, they exert an important role in
the field of cybersecurity. However, state-of-the-art IOCs detection systems
rely heavily on hand-crafted features with expert knowledge of cybersecurity,
and require a large amount of supervised training corpora to train an IOC
classifier. In this paper, we propose using a neural-based sequence labelling
model to identify IOCs automatically from reports on cybersecurity without
expert knowledge of cybersecurity. Our work is the first to apply an end-to-end
sequence labelling to the task in IOCs identification. By using an attention
mechanism and several token spelling features, we find that the proposed model
is capable of identifying the low frequency IOCs from long sentences contained
in cybersecurity reports. Experiments show that the proposed model outperforms
other sequence labelling models, achieving over 88% average F1-score.