pwc-archive/papers-with-abstracts huggingface dataset:
When Semi-Supervised Learning Meets Transfer Learning: Training Strategies, Models and Datasets
Authors: Hong-Yu Zhou, Avital Oliver, Jianxin Wu, Yefeng Zheng
Date: 2018-12-13 00:00:00
Tasks: Pseudo Label, Transfer Learning

Semi-Supervised Learning (SSL) has been proved to be an effective way to
leverage both labeled and unlabeled data at the same time. Recent
semi-supervised approaches focus on deep neural networks and have achieved
promising results on several benchmarks: CIFAR10, CIFAR100 and SVHN. However,
most of their experiments are based on models trained from scratch instead of
pre-trained models. On the other hand, transfer learning has demonstrated its
value when the target domain has limited labeled data. Here comes the intuitive
question: is it possible to incorporate SSL when fine-tuning a pre-trained
model? We comprehensively study how SSL methods starting from pretrained models
perform under varying conditions, including training strategies, architecture
choice and datasets. From this study, we obtain several interesting and useful
observations.
  While practitioners have had an intuitive understanding of these
observations, we do a comprehensive emperical analysis and demonstrate that:
(1) the gains from SSL techniques over a fully-supervised baseline are smaller
when trained from a pre-trained model than when trained from random
initialization, (2) when the domain of the source data used to train the
pre-trained model differs significantly from the domain of the target task, the
gains from SSL are significantly higher and (3) some SSL methods are able to
advance fully-supervised baselines (like Pseudo-Label).
  We hope our studies can deepen the understanding of SSL research and
facilitate the process of developing more effective SSL methods to utilize
pre-trained models. Code is now available at github.
FDFNet : A Secure Cancelable Deep Finger Dorsal Template Generation Network Secured via. Bio-Hashing
Authors: Avantika Singh, Ashish Arora, Shreya Hasmukh Patel, Gaurav Jaswal, Aditya Nigam
Date: 2018-12-13 00:00:00
Tasks: Management

Present world has already been consistently exploring the fine edges of
online and digital world by imposing multiple challenging problems/scenarios.
Similar to physical world, personal identity management is very crucial
in-order to provide any secure online system. Last decade has seen a lot of
work in this area using biometrics such as face, fingerprint, iris etc. Still
there exist several vulnerabilities and one should have to address the problem
of compromised biometrics much more seriously, since they cannot be modified
easily once compromised. In this work, we have proposed a secure cancelable
finger dorsal template generation network (learning domain specific features)
secured via. Bio-Hashing. Proposed system effectively protects the original
finger dorsal images by withdrawing compromised template and reassigning the
new one. A novel Finger-Dorsal Feature Extraction Net (FDFNet) has been
proposed for extracting the discriminative features. This network is
exclusively trained on trait specific features without using any kind of
pre-trained architecture. Later Bio-Hashing, a technique based on assigning a
tokenized random number to each user, has been used to hash the features
extracted from FDFNet. To test the performance of the proposed architecture, we
have tested it over two benchmark public finger knuckle datasets: PolyU FKP and
PolyU Contactless FKI. The experimental results shows the effectiveness of the
proposed system in terms of security and accuracy.
Cyber-Physical Security and Safety of Autonomous Connected Vehicles: Optimal Control Meets Multi-Armed Bandit Learning
Authors: Aidin Ferdowsi, Samad Ali, Walid Saad, Narayan B. Mandayam
Date: 2018-12-13 00:00:00

Autonomous connected vehicles (ACVs) rely on intra-vehicle sensors such as
camera and radar as well as inter-vehicle communication to operate effectively.
This reliance on cyber components exposes ACVs to cyber and physical attacks in
which an adversary can manipulate sensor readings and physically take control
of an ACV. In this paper, a comprehensive framework is proposed to thwart cyber
and physical attacks on ACV networks. First, an optimal safe controller for
ACVs is derived to maximize the street traffic flow while minimizing the risk
of accidents by optimizing ACV speed and inter-ACV spacing. It is proven that
the proposed controller is robust to physical attacks which aim at making ACV
systems instable. To improve the cyber-physical security of ACV systems, next,
data injection attack (DIA) detection approaches are proposed to address cyber
attacks on sensors and their physical impact on the ACV system. To
comprehensively design the DIA detection approaches, ACV sensors are
characterized in two subsets based on the availability of a-priori information
about their data. For sensors having a prior information, a DIA detection
approach is proposed and an optimal threshold level is derived for the
difference between the actual and estimated values of sensors data which
enables ACV to stay robust against cyber attacks. For sensors having no prior
information, a novel multi-armed bandit (MAB) algorithm is proposed to enable
ACV to securely control its motion. Simulation results show that the proposed
optimal safe controller outperforms current state of the art controllers by
maximizing the robustness of ACVs to physical attacks. The results also show
that the proposed DIA detection approaches, compared to Kalman filtering, can
improve the security of ACV sensors against cyber attacks and ultimately
improve the physical robustness of an ACV system.
Dynamic Transfer Learning for Named Entity Recognition
Authors: Parminder Bhatia, Kristjan Arumae, Busra Celikkaya
Date: 2018-12-13 00:00:00
Tasks: Model Optimization, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Transfer Learning

State-of-the-art named entity recognition (NER) systems have been improving continuously using neural architectures over the past several years. However, many tasks including NER require large sets of annotated data to achieve such performance. In particular, we focus on NER from clinical notes, which is one of the most fundamental and critical problems for medical text analysis. Our work centers on effectively adapting these neural architectures towards low-resource settings using parameter transfer methods. We complement a standard hierarchical NER model with a general transfer learning framework consisting of parameter sharing between the source and target tasks, and showcase scores significantly above the baseline architecture. These sharing schemes require an exponential search over tied parameter sets to generate an optimal configuration. To mitigate the problem of exhaustively searching for model optimization, we propose the Dynamic Transfer Networks (DTN), a gated architecture which learns the appropriate parameter sharing scheme between source and target datasets. DTN achieves the improvements of the optimized transfer learning framework with just a single training setting, effectively removing the need for exponential search.
IRLAS: Inverse Reinforcement Learning for Architecture Search
Authors: Minghao Guo, Zhao Zhong, Wei Wu, Dahua Lin, Junjie Yan
Date: 2018-12-13 00:00:00
Proceeding: CVPR 2019 6
Tasks: Neural Architecture Search, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In this paper, we propose an inverse reinforcement learning method for architecture search (IRLAS), which trains an agent to learn to search network structures that are topologically inspired by human-designed network. Most existing architecture search approaches totally neglect the topological characteristics of architectures, which results in complicated architecture with a high inference latency. Motivated by the fact that human-designed networks are elegant in topology with a fast inference speed, we propose a mirror stimuli function inspired by biological cognition theory to extract the abstract topological knowledge of an expert human-design network (ResNeXt). To avoid raising a too strong prior over the search space, we introduce inverse reinforcement learning to train the mirror stimuli function and exploit it as a heuristic guidance for architecture search, easily generalized to different architecture search algorithms. On CIFAR-10, the best architecture searched by our proposed IRLAS achieves 2.60% error rate. For ImageNet mobile setting, our model achieves a state-of-the-art top-1 accuracy 75.28%, while being 2~4x faster than most auto-generated architectures. A fast version of this model achieves 10% faster than MobileNetV2, while maintaining a higher accuracy.
IPOD: Intensive Point-based Object Detector for Point Cloud
Authors: Zetong Yang, Yanan sun, Shu Liu, Xiaoyong Shen, Jiaya Jia
Date: 2018-12-13 00:00:00
Tasks: 2D Object Detection, 3D Object Detection, Object, object-detection, Object Detection

We present a novel 3D object detection framework, named IPOD, based on raw
point cloud. It seeds object proposal for each point, which is the basic
element. This paradigm provides us with high recall and high fidelity of
information, leading to a suitable way to process point cloud data. We design
an end-to-end trainable architecture, where features of all points within a
proposal are extracted from the backbone network and achieve a proposal feature
for final bounding inference. These features with both context information and
precise point cloud coordinates yield improved performance. We conduct
experiments on KITTI dataset, evaluating our performance in terms of 3D object
detection, Bird's Eye View (BEV) detection and 2D object detection. Our method
accomplishes new state-of-the-art , showing great advantage on the hard set.
Towards a General-Purpose Linguistic Annotation Backend
Authors: Graham Neubig, Patrick Littell, Chian-Yu Chen, Jean Lee, Zirui Li, Yu-Hsiang Lin, Yuyan Zhang
Date: 2018-12-13 00:00:00
Tasks: Management

Language documentation is inherently a time-intensive process; transcription,
glossing, and corpus management consume a significant portion of documentary
linguists' work. Advances in natural language processing can help to accelerate
this work, using the linguists' past decisions as training material, but
questions remain about how to prioritize human involvement. In this extended
abstract, we describe the beginnings of a new project that will attempt to ease
this language documentation process through the use of natural language
processing (NLP) technology. It is based on (1) methods to adapt NLP tools to
new languages, based on recent advances in massively multilingual neural
networks, and (2) backend APIs and interfaces that allow linguists to upload
their data. We then describe our current progress on two fronts: automatic
phoneme transcription, and glossing. Finally, we briefly describe our future
directions.
TextBugger: Generating Adversarial Text Against Real-world Applications
Authors: Jinfeng Li, Shouling Ji, Tianyu Du, Bo Li, Ting Wang
Date: 2018-12-13 00:00:00
Tasks: Adversarial Text, Machine Translation, Question Answering, Semantic Similarity, Semantic Textual Similarity, Sentiment Analysis, text-classification, Text Classification

Deep Learning-based Text Understanding (DLTU) is the backbone technique
behind various applications, including question answering, machine translation,
and text classification. Despite its tremendous popularity, the security
vulnerabilities of DLTU are still largely unknown, which is highly concerning
given its increasing use in security-sensitive applications such as sentiment
analysis and toxic content detection. In this paper, we show that DLTU is
inherently vulnerable to adversarial text attacks, in which maliciously crafted
texts trigger target DLTU systems and services to misbehave. Specifically, we
present TextBugger, a general attack framework for generating adversarial
texts. In contrast to prior works, TextBugger differs in significant ways: (i)
effective -- it outperforms state-of-the-art attacks in terms of attack success
rate; (ii) evasive -- it preserves the utility of benign text, with 94.9\% of
the adversarial text correctly recognized by human readers; and (iii) efficient
-- it generates adversarial text with computational complexity sub-linear to
the text length. We empirically evaluate TextBugger on a set of real-world DLTU
systems and services used for sentiment analysis and toxic content detection,
demonstrating its effectiveness, evasiveness, and efficiency. For instance,
TextBugger achieves 100\% success rate on the IMDB dataset based on Amazon AWS
Comprehend within 4.61 seconds and preserves 97\% semantic similarity. We
further discuss possible defense mechanisms to mitigate such attack and the
adversary's potential countermeasures, which leads to promising directions for
further research.
Joint Entity Extraction and Assertion Detection for Clinical Text
Authors: Parminder Bhatia, Busra Celikkaya, Mohammed Khalilia
Date: 2018-12-13 00:00:00
Proceeding: ACL 2019 7
Tasks: Decoder, Entity Extraction using GAN, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Negation, Negation Detection, NER

Negative medical findings are prevalent in clinical reports, yet discriminating them from positive findings remains a challenging task for information extraction. Most of the existing systems treat this task as a pipeline of two separate tasks, i.e., named entity recognition (NER) and rule-based negation detection. We consider this as a multi-task problem and present a novel end-to-end neural model to jointly extract entities and negations. We extend a standard hierarchical encoder-decoder NER model and first adopt a shared encoder followed by separate decoders for the two tasks. This architecture performs considerably better than the previous rule-based and machine learning-based systems. To overcome the problem of increased parameter size especially for low-resource settings, we propose the Conditional Softmax Shared Decoder architecture which achieves state-of-art results for NER and negation detection on the 2010 i2b2/VA challenge dataset and a proprietary de-identified clinical dataset.
ELASTIC: Improving CNNs with Dynamic Scaling Policies
Authors: Huiyu Wang, Aniruddha Kembhavi, Ali Farhadi, Alan Yuille, Mohammad Rastegari
Date: 2018-12-13 00:00:00
Proceeding: CVPR 2019 6
Tasks: General Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Semantic Segmentation

Scale variation has been a challenge from traditional to modern approaches in
computer vision. Most solutions to scale issues have a similar theme: a set of
intuitive and manually designed policies that are generic and fixed (e.g. SIFT
or feature pyramid). We argue that the scaling policy should be learned from
data. In this paper, we introduce ELASTIC, a simple, efficient and yet very
effective approach to learn a dynamic scale policy from data. We formulate the
scaling policy as a non-linear function inside the network's structure that (a)
is learned from data, (b) is instance specific, (c) does not add extra
computation, and (d) can be applied on any network architecture. We applied
ELASTIC to several state-of-the-art network architectures and showed consistent
improvement without extra (sometimes even lower) computation on ImageNet
classification, MSCOCO multi-label classification, and PASCAL VOC semantic
segmentation. Our results show major improvement for images with scale
challenges. Our code is available here: https://github.com/allenai/elastic
Learning to Communicate: A Machine Learning Framework for Heterogeneous Multi-Agent Robotic Systems
Authors: Hyung-Jin Yoon, Huaiyu Chen, Kehan Long, Heling Zhang, Aditya Gahlawat, Donghwan Lee, Naira Hovakimyan
Date: 2018-12-13 00:00:00
Tasks: BIG-bench Machine Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We present a machine learning framework for multi-agent systems to learn both
the optimal policy for maximizing the rewards and the encoding of the high
dimensional visual observation. The encoding is useful for sharing local visual
observations with other agents under communication resource constraints. The
actor-encoder encodes the raw images and chooses an action based on local
observations and messages sent by the other agents. The machine learning agent
generates not only an actuator command to the physical device, but also a
communication message to the other agents. We formulate a reinforcement
learning problem, which extends the action space to consider the communication
action as well. The feasibility of the reinforcement learning framework is
demonstrated using a 3D simulation environment with two collaborating agents.
The environment provides realistic visual observations to be used and shared
between the two agents.
Modeling Multi-speaker Latent Space to Improve Neural TTS: Quick Enrolling New Speaker and Enhancing Premium Voice
Authors: Yan Deng, Lei He, Frank Soong
Date: 2018-12-13 00:00:00

Neural TTS has shown it can generate high quality synthesized speech. In this paper, we investigate the multi-speaker latent space to improve neural TTS for adapting the system to new speakers with only several minutes of speech or enhancing a premium voice by utilizing the data from other speakers for richer contextual coverage and better generalization. A multi-speaker neural TTS model is built with the embedded speaker information in both spectral and speaker latent space. The experimental results show that, with less than 5 minutes of training data from a new speaker, the new model can achieve an MOS score of 4.16 in naturalness and 4.64 in speaker similarity close to human recordings (4.74). For a well-trained premium voice, we can achieve an MOS score of 4.5 for out-of-domain texts, which is comparable to an MOS of 4.58 for professional recordings, and significantly outperforms single speaker result of 4.28.
Dynamic Fusion with Intra- and Inter- Modality Attention Flow for Visual Question Answering
Authors: Gao Peng, Zhengkai Jiang, Haoxuan You, Pan Lu, Steven Hoi, Xiaogang Wang, Hongsheng Li
Date: 2018-12-13 00:00:00
Tasks: Question Answering, Visual Question Answering, Visual Question Answering (VQA)

Learning effective fusion of multi-modality features is at the heart of visual question answering. We propose a novel method of dynamically fusing multi-modal features with intra- and inter-modality information flow, which alternatively pass dynamic information between and across the visual and language modalities. It can robustly capture the high-level interactions between language and vision domains, thus significantly improves the performance of visual question answering. We also show that the proposed dynamic intra-modality attention flow conditioned on the other modality can dynamically modulate the intra-modality attention of the target modality, which is vital for multimodality feature fusion. Experimental evaluations on the VQA 2.0 dataset show that the proposed method achieves state-of-the-art VQA performance. Extensive ablation studies are carried out for the comprehensive analysis of the proposed method.
Shortcut Matrix Product States and its applications
Authors: Zhuan Li, Pan Zhang
Date: 2018-12-13 00:00:00
Tasks: Computational Efficiency

Matrix Product States (MPS), also known as Tensor Train (TT) decomposition in
mathematics, has been proposed originally for describing an (especially
one-dimensional) quantum system, and recently has found applications in various
applications such as compressing high-dimensional data, supervised kernel
linear classifier, and unsupervised generative modeling. However, when applied
to systems which are not defined on one-dimensional lattices, a serious
drawback of the MPS is the exponential decay of the correlations, which limits
its power in capturing long-range dependences among variables in the system. To
alleviate this problem, we propose to introduce long-range interactions, which
act as shortcuts, to MPS, resulting in a new model \textit{ Shortcut Matrix
Product States} (SMPS). When chosen properly, the shortcuts can decrease
significantly the correlation length of the MPS, while preserving the
computational efficiency. We develop efficient training methods of SMPS for
various tasks, establish some of their mathematical properties, and show how to
find a good location to add shortcuts. Finally, using extensive numerical
experiments we evaluate its performance in a variety of applications, including
function fitting, partition function calculation of $2-$d Ising model, and
unsupervised generative modeling of handwritten digits, to illustrate its
advantages over vanilla matrix product states.
Improving fairness in machine learning systems: What do industry practitioners need?
Authors: Kenneth Holstein, Jennifer Wortman Vaughan, Hal Daumé III, Miro Dudík, Hanna Wallach
Date: 2018-12-13 00:00:00
Tasks: BIG-bench Machine Learning, Fairness

The potential for machine learning (ML) systems to amplify social inequities
and unfairness is receiving increasing popular and academic attention. A surge
of recent work has focused on the development of algorithmic tools to assess
and mitigate such unfairness. If these tools are to have a positive impact on
industry practice, however, it is crucial that their design be informed by an
understanding of real-world needs. Through 35 semi-structured interviews and an
anonymous survey of 267 ML practitioners, we conduct the first systematic
investigation of commercial product teams' challenges and needs for support in
developing fairer ML systems. We identify areas of alignment and disconnect
between the challenges faced by industry practitioners and solutions proposed
in the fair ML research literature. Based on these findings, we highlight
directions for future ML and HCI research that will better address industry
practitioners' needs.
Code Failure Prediction and Pattern Extraction using LSTM Networks
Authors: Mahdi Hajiaghayi, Ehsan Vahedi
Date: 2018-12-13 00:00:00
Tasks: Bayesian Optimization, Sentence, Sentiment Analysis

In this paper, we use a well-known Deep Learning technique called Long Short
Term Memory (LSTM) recurrent neural networks to find sessions that are prone to
code failure in applications that rely on telemetry data for system health
monitoring. We also use LSTM networks to extract telemetry patterns that lead
to a specific code failure. For code failure prediction, we treat the telemetry
events, sequence of telemetry events and the outcome of each sequence as words,
sentence and sentiment in the context of sentiment analysis, respectively. Our
proposed method is able to process a large set of data and can automatically
handle edge cases in code failure prediction. We take advantage of Bayesian
optimization technique to find the optimal hyper parameters as well as the type
of LSTM cells that leads to the best prediction performance. We then introduce
the Contributors and Blockers concepts. In this paper, contributors are the set
of events that cause a code failure, while blockers are the set of events that
each of them individually prevents a code failure from happening, even in
presence of one or multiple contributor(s). Once the proposed LSTM model is
trained, we use a greedy approach to find the contributors and blockers. To
develop and test our proposed method, we use synthetic (simulated) data in the
first step. The synthetic data is generated using a number of rules for code
failures, as well as a number of rules for preventing a code failure from
happening. The trained LSTM model shows over 99% accuracy for detecting code
failures in the synthetic data. The results from the proposed method outperform
the classical learning models such as Decision Tree and Random Forest. Using
the proposed greedy method, we are able to find the contributors and blockers
in the synthetic data in more than 90% of the cases, with a performance better
than sequential rule and pattern mining algorithms.