pwc-archive/papers-with-abstracts huggingface dataset:
Semi-supervised Acoustic Event Detection based on tri-training
Authors: Bowen Shi, Ming Sun, Chieh-Chi Kao, Viktor Rozgic, Spyros Matsoukas, Chao Wang
Date: 2019-04-29 00:00:00
Tasks: Event Detection, Knowledge Distillation

This paper presents our work of training acoustic event detection (AED)
models using unlabeled dataset. Recent acoustic event detectors are based on
large-scale neural networks, which are typically trained with huge amounts of
labeled data. Labels for acoustic events are expensive to obtain, and relevant
acoustic event audios can be limited, especially for rare events. In this paper
we leverage an Internet-scale unlabeled dataset with potential domain shift to
improve the detection of acoustic events. Based on the classic tri-training
approach, our proposed method shows accuracy improvement over both the
supervised training baseline, and semisupervised self-training set-up, in all
pre-defined acoustic event detection tasks. As our approach relies on ensemble
models, we further show the improvements can be distilled to a single model via
knowledge distillation, with the resulting single student model maintaining
high accuracy of teacher ensemble models.
Enabling Robots to Understand Incomplete Natural Language Instructions Using Commonsense Reasoning
Authors: Haonan Chen, Hao Tan, Alan Kuntz, Mohit Bansal, Ron Alterovitz
Date: 2019-04-29 00:00:00
Tasks: Common Sense Reasoning, Language Modeling, Language Modelling

Enabling robots to understand instructions provided via spoken natural language would facilitate interaction between robots and people in a variety of settings in homes and workplaces. However, natural language instructions are often missing information that would be obvious to a human based on environmental context and common sense, and hence does not need to be explicitly stated. In this paper, we introduce Language-Model-based Commonsense Reasoning (LMCR), a new method which enables a robot to listen to a natural language instruction from a human, observe the environment around it, and automatically fill in information missing from the instruction using environmental context and a new commonsense reasoning approach. Our approach first converts an instruction provided as unconstrained natural language into a form that a robot can understand by parsing it into verb frames. Our approach then fills in missing information in the instruction by observing objects in its vicinity and leveraging commonsense reasoning. To learn commonsense reasoning automatically, our approach distills knowledge from large unstructured textual corpora by training a language model. Our results show the feasibility of a robot learning commonsense knowledge automatically from web-based textual corpora, and the power of learned commonsense reasoning models in enabling a robot to autonomously perform tasks based on incomplete natural language instructions.
Neuromorphic Acceleration for Approximate Bayesian Inference on Neural Networks via Permanent Dropout
Authors: Nathan Wycoff, Prasanna Balaprakash, Fangfang Xia
Date: 2019-04-29 00:00:00
Tasks: Bayesian Inference, Uncertainty Quantification

As neural networks have begun performing increasingly critical tasks for
society, ranging from driving cars to identifying candidates for drug
development, the value of their ability to perform uncertainty quantification
(UQ) in their predictions has risen commensurately. Permanent dropout, a
popular method for neural network UQ, involves injecting stochasticity into the
inference phase of the model and creating many predictions for each of the test
data. This shifts the computational and energy burden of deep neural networks
from the training phase to the inference phase. Recent work has demonstrated
near-lossless conversion of classical deep neural networks to their spiking
counterparts. We use these results to demonstrate the feasibility of conducting
the inference phase with permanent dropout on spiking neural networks,
mitigating the technique's computational and energy burden, which is essential
for its use at scale or on edge platforms. We demonstrate the proposed approach
via the Nengo spiking neural simulator on a combination drug therapy dataset
for cancer treatment, where UQ is critical. Our results indicate that the
spiking approximation gives a predictive distribution practically
indistinguishable from that given by the classical network.
Challenges of Real-World Reinforcement Learning
Authors: Gabriel Dulac-Arnold, Daniel Mankowitz, Todd Hester
Date: 2019-04-29 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Reinforcement learning (RL) has proven its worth in a series of artificial
domains, and is beginning to show some successes in real-world scenarios.
However, much of the research advances in RL are often hard to leverage in
real-world systems due to a series of assumptions that are rarely satisfied in
practice. We present a set of nine unique challenges that must be addressed to
productionize RL to real world problems. For each of these challenges, we
specify the exact meaning of the challenge, present some approaches from the
literature, and specify some metrics for evaluating that challenge. An approach
that addresses all nine challenges would be applicable to a large number of
real world problems. We also present an example domain that has been modified
to present these challenges as a testbed for practical RL research.
DiamondGAN: Unified Multi-Modal Generative Adversarial Networks for MRI Sequences Synthesis
Authors: Hongwei Li, Johannes C. Paetzold, Anjany Sekuboyina, Florian Kofler, Jian-Guo Zhang, Jan S. Kirschke, Benedikt Wiestler, Bjoern Menze
Date: 2019-04-29 00:00:00
Tasks: Image Generation

Synthesizing MR imaging sequences is highly relevant in clinical practice, as single sequences are often missing or are of poor quality (e.g. due to motion). Naturally, the idea arises that a target modality would benefit from multi-modal input, as proprietary information of individual modalities can be synergistic. However, existing methods fail to scale up to multiple non-aligned imaging modalities, facing common drawbacks of complex imaging sequences. We propose a novel, scalable and multi-modal approach called DiamondGAN. Our model is capable of performing exible non-aligned cross-modality synthesis and data infill, when given multiple modalities or any of their arbitrary subsets, learning structured information in an end-to-end fashion. We synthesize two MRI sequences with clinical relevance (i.e., double inversion recovery (DIR) and contrast-enhanced T1 (T1-c)), reconstructed from three common sequences. In addition, we perform a multi-rater visual evaluation experiment and find that trained radiologists are unable to distinguish synthetic DIR images from real ones.
Curriculum Learning in Deep Neural Networks for Financial Forecasting
Authors: Allison Koenecke, Amita Gajewar
Date: 2019-04-29 00:00:00
Tasks: Decoder, Time Series, Time Series Analysis, Time Series Forecasting, Transfer Learning

For any financial organization, computing accurate quarterly forecasts for various products is one of the most critical operations. As the granularity at which forecasts are needed increases, traditional statistical time series models may not scale well. We apply deep neural networks in the forecasting domain by experimenting with techniques from Natural Language Processing (Encoder-Decoder LSTMs) and Computer Vision (Dilated CNNs), as well as incorporating transfer learning. A novel contribution of this paper is the application of curriculum learning to neural network models built for time series forecasting. We illustrate the performance of our models using Microsoft's revenue data corresponding to Enterprise, and Small, Medium & Corporate products, spanning approximately 60 regions across the globe for 8 different business segments, and totaling in the order of tens of billions of USD. We compare our models' performance to the ensemble model of traditional statistics and machine learning techniques currently used by Microsoft Finance. With this in-production model as a baseline, our experiments yield an approximately 30% improvement in overall accuracy on test data. We find that our curriculum learning LSTM-based model performs best, showing that it is reasonable to implement our proposed methods without overfitting on medium-sized data.
Property Inference for Deep Neural Networks
Authors: Divya Gopinath, Hayes Converse, Corina S. Pasareanu, Ankur Taly
Date: 2019-04-29 00:00:00

We present techniques for automatically inferring formal properties of feed-forward neural networks. We observe that a significant part (if not all) of the logic of feed forward networks is captured in the activation status ('on' or 'off') of its neurons. We propose to extract patterns based on neuron decisions as preconditions that imply certain desirable output property e.g., the prediction being a certain class. We present techniques to extract input properties, encoding convex predicates on the input space that imply given output properties and layer properties, representing network properties captured in the hidden layers that imply the desired output behavior. We apply our techniques on networks for the MNIST and ACASXU applications. Our experiments highlight the use of the inferred properties in a variety of tasks, such as explaining predictions, providing robustness guarantees, simplifying proofs, and network distillation.
From Digitalization to Data-Driven Decision Making in Container Terminals
Authors: Leonard Heilig, Robert Stahlbock, Stefan Voß
Date: 2019-04-29 00:00:00
Tasks: Decision Making, Management

With the new opportunities emerging from the current wave of digitalization,
terminal planning and management need to be revisited by taking a data-driven
perspective. Business analytics, as a practice of extracting insights from
operational data, assists in reducing uncertainties using predictions and helps
to identify and understand causes of inefficiencies, disruptions, and anomalies
in intra- and inter-organizational terminal operations. Despite the growing
complexity of data within and around container terminals, a lack of data-driven
approaches in the context of container terminals can be identified. In this
chapter, the concept of business analytics for supporting terminal planning and
management is introduced. The chapter specifically focuses on data mining
approaches and provides a comprehensive overview on applications in container
terminals and related research. As such, we aim to establish a data-driven
perspective on terminal planning and management, complementing the traditional
optimization perspective.
AutoCross: Automatic Feature Crossing for Tabular Data in Real-World Applications
Authors: Yuanfei Luo, Mengshuo Wang, Hao Zhou, Quanming Yao, Wei-Wei Tu, Yuqiang Chen, Qiang Yang, Wenyuan Dai
Date: 2019-04-29 00:00:00
Tasks: Distributed Computing

Feature crossing captures interactions among categorical features and is useful to enhance learning from tabular data in real-world businesses. In this paper, we present AutoCross, an automatic feature crossing tool provided by 4Paradigm to its customers, ranging from banks, hospitals, to Internet corporations. By performing beam search in a tree-structured space, AutoCross enables efficient generation of high-order cross features, which is not yet visited by existing works. Additionally, we propose successive mini-batch gradient descent and multi-granularity discretization to further improve efficiency and effectiveness, while ensuring simplicity so that no machine learning expertise or tedious hyper-parameter tuning is required. Furthermore, the algorithms are designed to reduce the computational, transmitting, and storage costs involved in distributed computing. Experimental results on both benchmark and real-world business datasets demonstrate the effectiveness and efficiency of AutoCross. It is shown that AutoCross can significantly enhance the performance of both linear and deep models.
A supervised-learning-based strategy for optimal demand response of an HVAC System
Authors: Youngjin Kim
Date: 2019-04-29 00:00:00
Tasks: Scheduling

The large thermal capacity of buildings enables heating, ventilating, and
air-conditioning (HVAC) systems to be exploited as demand response (DR)
resources. Optimal DR of HVAC units is challenging, particularly for multi-zone
buildings, because this requires detailed physics-based models of zonal
temperature variations for HVAC system operation and building thermal
conditions. This paper proposes a new strategy for optimal DR of an HVAC system
in a multi-zone building, based on supervised learning (SL). Artificial neural
networks (ANNs) are trained with data obtained under normal building operating
conditions. The ANNs are replicated using piecewise linear equations, which are
explicitly integrated into an optimal scheduling problem for price-based DR.
The optimization problem is solved for various electricity prices and building
thermal conditions. The solutions are further used to train a deep neural
network (DNN) to directly determine the optimal DR schedule, referred to here
as supervised-learning-aided meta-prediction (SLAMP). Case studies are
performed using three different methods: explicit ANN replication (EAR), SLAMP,
and physics-based modeling. The case study results verify the effectiveness of
the proposed SL-based strategy, in terms of both practical applicability and
computational time, while also ensuring the thermal comfort of occupants and
cost-effective operation of the HVAC system.
Fourier Transform Approach to Machine Learning II: Fourier Clustering
Authors: Soheil Mehrabkhani
Date: 2019-04-29 00:00:00
Tasks: BIG-bench Machine Learning, Clustering, global-optimization

We propose a Fourier-based approach for optimization of several clustering algorithms. Mathematically, clusters data can be described by a density function represented by the Dirac mixture distribution. The density function can be smoothed by applying the Fourier transform and a Gaussian filter. The determination of the optimal standard deviation of the Gaussian filter will be accomplished by the use of a convergence criterion related to the correlation between the smoothed and the original density functions. In principle, the optimal smoothed density function exhibits local maxima, which correspond to the cluster centroids. Thus, the complex task of finding the centroids of the clusters is simplified by the detection of the peaks of the smoothed density function. A multiple sliding windows procedure is used to detect the peaks. The remarkable accuracy of the proposed algorithm demonstrates its capability as a reliable general method for enhancement of the clustering performance, its global optimization and also removing the initialization problem in many clustering methods.
A General Spatio-Temporal Clustering-Based Non-local Formulation for Multiscale Modeling of Compartmentalized Reservoirs
Authors: Soheil Esmaeilzadeh, Amir Salehi, Gill Hetz, Feyisayo Olalotiti-lawal, Hamed Darabi, David Castineira
Date: 2019-04-28 00:00:00
Tasks: Clustering

Representing the reservoir as a network of discrete compartments with
neighbor and non-neighbor connections is a fast, yet accurate method for
analyzing oil and gas reservoirs. Automatic and rapid detection of coarse-scale
compartments with distinct static and dynamic properties is an integral part of
such high-level reservoir analysis. In this work, we present a hybrid framework
specific to reservoir analysis for an automatic detection of clusters in space
using spatial and temporal field data, coupled with a physics-based multiscale
modeling approach. In this work a novel hybrid approach is presented in which
we couple a physics-based non-local modeling framework with data-driven
clustering techniques to provide a fast and accurate multiscale modeling of
compartmentalized reservoirs. This research also adds to the literature by
presenting a comprehensive work on spatio-temporal clustering for reservoir
studies applications that well considers the clustering complexities, the
intrinsic sparse and noisy nature of the data, and the interpretability of the
outcome.
  Keywords: Artificial Intelligence; Machine Learning; Spatio-Temporal
Clustering; Physics-Based Data-Driven Formulation; Multiscale Modeling
Dynamic Environment Prediction in Urban Scenes using Recurrent Representation Learning
Authors: Masha Itkina, Katherine Driggs-Campbell, Mykel J. Kochenderfer
Date: 2019-04-28 00:00:00
Tasks: Autonomous Driving, Representation Learning

A key challenge for autonomous driving is safe trajectory planning in cluttered, urban environments with dynamic obstacles, such as pedestrians, bicyclists, and other vehicles. A reliable prediction of the future environment, including the behavior of dynamic agents, would allow planning algorithms to proactively generate a trajectory in response to a rapidly changing environment. We present a novel framework that predicts the future occupancy state of the local environment surrounding an autonomous agent by learning a motion model from occupancy grid data using a neural network. We take advantage of the temporal structure of the grid data by utilizing a convolutional long-short term memory network in the form of the PredNet architecture. This method is validated on the KITTI dataset and demonstrates higher accuracy and better predictive power than baseline methods.
Video Analytics with Zero-streaming Cameras
Authors: Mengwei Xu, Tiantu Xu, Yunxin Liu, Felix Xiaozhu Lin
Date: 2019-04-28 00:00:00

Low-cost cameras enable powerful analytics. An unexploited opportunity is that most captured videos remain "cold" without being queried. For efficiency, we advocate for these cameras to be zero streaming: capturing videos to local storage and communicating with the cloud only when analytics is requested. How to query zero-streaming cameras efficiently? Our response is a camera/cloud runtime system called DIVA. It addresses two key challenges: to best use limited camera resource during video capture; to rapidly explore massive videos during query execution. DIVA contributes two unconventional techniques. (1) When capturing videos, a camera builds sparse yet accurate landmark frames, from which it learns reliable knowledge for accelerating future queries. (2) When executing a query, a camera processes frames in multiple passes with increasingly more expensive operators. As such, DIVA presents and keeps refining inexact query results throughout the query's execution. On diverse queries over 15 videos lasting 720 hours in total, DIVA runs at more than 100x video realtime and outperforms competitive alternative designs. To our knowledge, DIVA is the first system for querying large videos stored on low-cost remote cameras.
Generating protein sequences from antibiotic resistance genes data using Generative Adversarial Networks
Authors: Prabal Chhibbar, Arpit Joshi
Date: 2019-04-28 00:00:00
Tasks: Generative Adversarial Network

We introduce a method to generate synthetic protein sequences which are
predicted to be resistant to certain antibiotics. We did this using 6,023 genes
that were predicted to be resistant to antibiotics in the intestinal region of
the human gut and were fed as input to a Wasserstein generative adversarial
network (W-GAN) model a variant to the original generative adversarial model
which has been known to perform efficiently when it comes to mimicking the
distribution of the real data in order to generate new data which is similar in
style to the original data which was fed as the training data
Unsupervised Representation Learning with Minimax Distance Measures
Authors: Morteza Haghir Chehreghani
Date: 2019-04-27 00:00:00
Tasks: Representation Learning

We investigate the use of Minimax distances to extract in a nonparametric way the features that capture the unknown underlying patterns and structures in the data. We develop a general-purpose and computationally efficient framework to employ Minimax distances with many machine learning methods that perform on numerical data. We study both computing the pairwise Minimax distances for all pairs of objects and as well as computing the Minimax distances of all the objects to/from a fixed (test) object. We first efficiently compute the pairwise Minimax distances between the objects, using the equivalence of Minimax distances over a graph and over a minimum spanning tree constructed on that. Then, we perform an embedding of the pairwise Minimax distances into a new vector space, such that their squared Euclidean distances in the new space equal to the pairwise Minimax distances in the original space. We also study the case of having multiple pairwise Minimax matrices, instead of a single one. Thereby, we propose an embedding via first summing up the centered matrices and then performing an eigenvalue decomposition to obtain the relevant features. In the following, we study computing Minimax distances from a fixed (test) object which can be used for instance in K-nearest neighbor search. Similar to the case of all-pair pairwise Minimax distances, we develop an efficient and general-purpose algorithm that is applicable with any arbitrary base distance measure. Moreover, we investigate in detail the edges selected by the Minimax distances and thereby explore the ability of Minimax distances in detecting outlier objects. Finally, for each setting, we perform several experiments to demonstrate the effectiveness of our framework.
Sentiment Classification using N-gram IDF and Automated Machine Learning
Authors: Rungroj Maipradit, Hideaki Hata, Kenichi Matsumoto
Date: 2019-04-27 00:00:00
Tasks: BIG-bench Machine Learning, Classification, General Classification, Sentiment Analysis, Sentiment Classification

We propose a sentiment classification method with a general machine learning framework. For feature representation, n-gram IDF is used to extract software-engineering-related, dataset-specific, positive, neutral, and negative n-gram expressions. For classifiers, an automated machine learning tool is used. In the comparison using publicly available datasets, our method achieved the highest F1 values in positive and negative sentences on all datasets.
Synthetic Ground Truth Generation for Evaluating Generative Policy Models
Authors: Daniel Cunnington, Graham White, Geeth de Mel
Date: 2019-04-26 00:00:00

Generative Policy-based Models aim to enable a coalition of systems, be they
devices or services to adapt according to contextual changes such as
environmental factors, user preferences and different tasks whilst adhering to
various constraints and regulations as directed by a managing party or the
collective vision of the coalition. Recent developments have proposed new
architectures to realize the potential of GPMs but as the complexity of systems
and their associated requirements increases, there is an emerging requirement
to have scenarios and associated datasets to realistically evaluate GPMs with
respect to the properties of the operating environment, be it the future
battlespace or an autonomous organization. In order to address this
requirement, in this paper, we present a method of applying an agile knowledge
representation framework to model requirements, both individualistic and
collective that enables synthetic generation of ground truth data such that
advanced GPMs can be evaluated robustly in complex environments. We also
release conceptual models, annotated datasets, as well as means to extend the
data generation approach so that similar datasets can be developed for varying
complexities and different situations.
Survey of Computer Vision and Machine Learning in Gastrointestinal Endoscopy
Authors: Anant S. Vemuri
Date: 2019-04-26 00:00:00
Tasks: BIG-bench Machine Learning, Deep Learning, Survey

This paper attempts to provide the reader a place to begin studying the
application of computer vision and machine learning to gastrointestinal (GI)
endoscopy. They have been classified into 18 categories. It should be be noted
by the reader that this is a review from pre-deep learning era. A lot of deep
learning based applications have not been covered in this thesis.