pwc-archive/papers-with-abstracts huggingface dataset:
Time-aware Test Case Execution Scheduling for Cyber-Physical Systems
Authors: Morten Mossige, Arnaud Gotlieb, Helge Spieker, Hein Meling, Mats Carlsson
Date: 2019-02-12 00:00:00
Tasks: Industrial Robots, Scheduling

Testing cyber-physical systems involves the execution of test cases on
target-machines equipped with the latest release of a software control system.
When testing industrial robots, it is common that the target machines need to
share some common resources, e.g., costly hardware devices, and so there is a
need to schedule test case execution on the target machines, accounting for
these shared resources. With a large number of such tests executed on a regular
basis, this scheduling becomes difficult to manage manually. In fact, with
manual test execution planning and scheduling, some robots may remain
unoccupied for long periods of time and some test cases may not be executed.
This paper introduces TC-Sched, a time-aware method for automated test case
execution scheduling. TC-Sched uses Constraint Programming to schedule tests to
run on multiple machines constrained by the tests' access to shared resources,
such as measurement or networking devices. The CP model is written in SICStus
Prolog and uses the Cumulatives global constraint. Given a set of test cases, a
set of machines, and a set of shared resources, TC-Sched produces an execution
schedule where each test is executed once with minimal time between when a
source code change is committed and the test results are reported to the
developer. Experiments reveal that TC-Sched can schedule 500 test cases over
100 machines in less than 4 minutes for 99.5% of the instances. In addition,
TC-Sched largely outperforms simpler methods based on a greedy algorithm and is
suitable for deployment on industrial robot testing.
Learning Theory and Support Vector Machines - a primer
Authors: Michael Banf
Date: 2019-02-12 00:00:00
Tasks: Decision Making, Learning Theory

The main goal of statistical learning theory is to provide a fundamental
framework for the problem of decision making and model construction based on
sets of data. Here, we present a brief introduction to the fundamentals of
statistical learning theory, in particular the difference between empirical and
structural risk minimization, including one of its most prominent
implementations, i.e. the Support Vector Machine.
Extreme Tensoring for Low-Memory Preconditioning
Authors: Xinyi Chen, Naman Agarwal, Elad Hazan, Cyril Zhang, Yi Zhang
Date: 2019-02-12 00:00:00
Proceeding: ICLR 2020 1
Tasks: Stochastic Optimization

State-of-the-art models are now trained with billions of parameters, reaching
hardware limits in terms of memory consumption. This has created a recent
demand for memory-efficient optimizers. To this end, we investigate the limits
and performance tradeoffs of memory-efficient adaptively preconditioned
gradient methods. We propose extreme tensoring for high-dimensional stochastic
optimization, showing that an optimizer needs very little memory to benefit
from adaptive preconditioning. Our technique applies to arbitrary models (not
necessarily with tensor-shaped parameters), and is accompanied by regret and
convergence guarantees, which shed light on the tradeoffs between
preconditioner quality and expressivity. On a large-scale NLP model, we reduce
the optimizer memory overhead by three orders of magnitude, without degrading
performance.
Salus: Fine-Grained GPU Sharing Primitives for Deep Learning Applications
Authors: Peifeng Yu, Mosharaf Chowdhury
Date: 2019-02-12 00:00:00
Tasks: CPU, Deep Learning, Fairness, GPU, Management, Scheduling

GPU computing is becoming increasingly more popular with the proliferation of
deep learning (DL) applications. However, unlike traditional resources such as
CPU or the network, modern GPUs do not natively support fine-grained sharing
primitives. Consequently, implementing common policies such as time sharing and
preemption are expensive. Worse, when a DL application cannot completely use a
GPU's resources, the GPU cannot be efficiently shared between multiple
applications, leading to GPU underutilization.
  We present Salus to enable two GPU sharing primitives: fast job switching and
memory sharing, in order to achieve fine-grained GPU sharing among multiple DL
applications. Salus implements an efficient, consolidated execution service
that exposes the GPU to different DL applications, and enforces fine-grained
sharing by performing iteration scheduling and addressing associated memory
management issues. We show that these primitives can then be used to implement
flexible sharing policies such as fairness, prioritization, and packing for
various use cases. Our integration of Salus with TensorFlow and evaluation on
popular DL jobs show that Salus can improve the average completion time of DL
training jobs by $3.19\times$, GPU utilization for hyper-parameter tuning by
$2.38\times$, and GPU utilization of DL inference applications by $42\times$
over not sharing the GPU and $7\times$ over NVIDIA MPS with small overhead.
Progressively Growing Generative Adversarial Networks for High Resolution Semantic Segmentation of Satellite Images
Authors: Edward Collier, Kate Duffy, Sangram Ganguly, Geri Madanguit, Subodh Kalia, Gayaka Shreekant, Ramakrishna Nemani, Andrew Michaelis, Shuang Li, Auroop Ganguly, Supratik Mukhopadhyay
Date: 2019-02-12 00:00:00
Tasks: BIG-bench Machine Learning, General Classification, Image Generation, Segmentation, Semantic Segmentation

Machine learning has proven to be useful in classification and segmentation
of images. In this paper, we evaluate a training methodology for pixel-wise
segmentation on high resolution satellite images using progressive growing of
generative adversarial networks. We apply our model to segmenting building
rooftops and compare these results to conventional methods for rooftop
segmentation. We present our findings using the SpaceNet version 2 dataset.
Progressive GAN training achieved a test accuracy of 93% compared to 89% for
traditional GAN training.
Contrastive Variational Autoencoder Enhances Salient Features
Authors: Abubakar Abid, James Zou
Date: 2019-02-12 00:00:00
Tasks: Contrastive Learning

Variational autoencoders are powerful algorithms for identifying dominant
latent structure in a single dataset. In many applications, however, we are
interested in modeling latent structure and variation that are enriched in a
target dataset compared to some background---e.g. enriched in patients compared
to the general population. Contrastive learning is a principled framework to
capture such enriched variation between the target and background, but
state-of-the-art contrastive methods are limited to linear models. In this
paper, we introduce the contrastive variational autoencoder (cVAE), which
combines the benefits of contrastive learning with the power of deep generative
models. The cVAE is designed to identify and enhance salient latent features.
The cVAE is trained on two related but unpaired datasets, one of which has
minimal contribution from the salient latent features. The cVAE explicitly
models latent features that are shared between the datasets, as well as those
that are enriched in one dataset relative to the other, which allows the
algorithm to isolate and enhance the salient latent features. The algorithm is
straightforward to implement, has a similar run-time to the standard VAE, and
is robust to noise and dataset purity. We conduct experiments across diverse
types of data, including gene expression and facial images, showing that the
cVAE effectively uncovers latent structure that is salient in a particular
analysis.
Machine Reading Comprehension for Answer Re-Ranking in Customer Support Chatbots
Authors: Momchil Hardalov, Ivan Koychev, Preslav Nakov
Date: 2019-02-12 00:00:00
Tasks: Information Retrieval, Language Modeling, Language Modelling, Machine Reading Comprehension, Reading Comprehension, Re-Ranking, Retrieval, Text Generation

Recent advances in deep neural networks, language modeling and language
generation have introduced new ideas to the field of conversational agents. As
a result, deep neural models such as sequence-to-sequence, Memory Networks, and
the Transformer have become key ingredients of state-of-the-art dialog systems.
While those models are able to generate meaningful responses even in unseen
situation, they need a lot of training data to build a reliable model. Thus,
most real-world systems stuck to traditional approaches based on information
retrieval and even hand-crafted rules, due to their robustness and
effectiveness, especially for narrow-focused conversations. Here, we present a
method that adapts a deep neural architecture from the domain of machine
reading comprehension to re-rank the suggested answers from different models
using the question as context. We train our model using negative sampling based
on question-answer pairs from the Twitter Customer Support Dataset.The
experimental results show that our re-ranking framework can improve the
performance in terms of word overlap and semantics both for individual models
as well as for model combinations.
Real-time tracker with fast recovery from target loss
Authors: Alessandro Bay, Panagiotis Sidiropoulos, Eduard Vazquez, Michele Sasdelli
Date: 2019-02-12 00:00:00
Tasks: Position

In this paper, we introduce a variation of a state-of-the-art real-time
tracker (CFNet), which adds to the original algorithm robustness to target loss
without a significant computational overhead. The new method is based on the
assumption that the feature map can be used to estimate the tracking confidence
more accurately. When the confidence is low, we avoid updating the object's
position through the feature map; instead, the tracker passes to a single-frame
failure mode, during which the patch's low-level visual content is used to
swiftly update the object's position, before recovering from the target loss in
the next frame. The experimental evidence provided by evaluating the method on
several tracking datasets validates both the theoretical assumption that the
feature map is associated to tracking confidence, and that the proposed
implementation can achieve target recovery in multiple scenarios, without
compromising the real-time performance.
Gauge Equivariant Convolutional Networks and the Icosahedral CNN
Authors: Taco S. Cohen, Maurice Weiler, Berkay Kicanaoglu, Max Welling
Date: 2019-02-11 00:00:00
Tasks: Semantic Segmentation

The principle of equivariance to symmetry transformations enables a theoretically grounded approach to neural network architecture design. Equivariant networks have shown excellent performance and data efficiency on vision and medical imaging problems that exhibit symmetries. Here we show how this principle can be extended beyond global symmetries to local gauge transformations. This enables the development of a very general class of convolutional neural networks on manifolds that depend only on the intrinsic geometry, and which includes many popular methods from equivariant and geometric deep learning. We implement gauge equivariant CNNs for signals defined on the surface of the icosahedron, which provides a reasonable approximation of the sphere. By choosing to work with this very regular manifold, we are able to implement the gauge equivariant convolution using a single conv2d call, making it a highly scalable and practical alternative to Spherical CNNs. Using this method, we demonstrate substantial improvements over previous methods on the task of segmenting omnidirectional images and global climate patterns.
Subspace metrics for multivariate dictionaries and application to EEG
Authors: Sylvain Chevallier, Quentin Barthélemy, Jamal Atif
Date: 2014-07-14 00:00:00
Proceeding: ICASSP 2014 7
Tasks: Clustering, Dictionary Learning, EEG, Electroencephalogram (EEG)

Overcomplete representations and dictionary learning algorithms are attracting a growing interest in the machine learning community. This paper addresses the emerging problem of comparing multivari-ate overcomplete dictionaries. Despite a recurrent need to rely on a distance for learning or assessing multivariate overcomplete dictionaries , no metrics in their underlying spaces have yet been proposed. Henceforth we propose to study overcomplete representations from the perspective of matrix manifolds. We consider distances between multivariate dictionaries as distances between their spans which reveal to be elements of a Grassmannian manifold. We introduce set-metrics defined on Grassmannian spaces and study their properties both theoretically and numerically. Thanks to the introduced met-rics, experimental convergences of dictionary learning algorithms are assessed on synthetic datasets. Set-metrics are embedded in a clustering algorithm for a qualitative analysis of real EEG signals for Brain-Computer Interfaces (BCI). The obtained clusters of subjects are associated with subject performances. This is a major method-ological advance to understand the BCI-inefficiency phenomenon and to predict the ability of a user to interact with a BCI.
On the need for metrics in dictionary learning assessment
Authors: Sylvain Chevallier, Quentin Barthélemy, Jamal Atif
Date: 2014-09-08 00:00:00
Proceeding: EUSIPCO 2014 9
Tasks: Dictionary Learning

Dictionary-based approaches are the focus of a growing attention in the signal processing community, often achieving state of the art results in several application fields. Albeit their success , the criteria introduced so far for the assessment of their performances suffer from several shortcomings. The scope of this paper is to conduct a thorough analysis of these criteria and to highlight the need for principled criteria, enjoying the properties of metrics. Henceforth we introduce new criteria based on transportation like metrics and discuss their behaviors w.r.t the literature.
Online SSVEP-based BCI using Riemannian geometry
Authors: Emmanuel Kalunga, Sylvain Chevallier, Quentin Barthélemy, Karim Djouani, Eric Monacelli, Yskandar Hamam
Date: 2016-05-26 00:00:00
Proceeding: Neurocomputing 2016 5
Tasks: EEG, Electroencephalogram (EEG), General Classification, SSVEP

Challenges for the next generation of Brain Computer Interfaces (BCI) are to mitigate the common sources of variability (electronic, electrical, biological) and to develop online and adaptive systems following the evolution of the subject׳s brain waves. Studying electroencephalographic (EEG) signals from their associated covariance matrices allows the construction of a representation which is invariant to extrinsic perturbations. As covariance matrices should be estimated, this paper first presents a thorough study of all estimators conducted on real EEG recording. Working in Euclidean space with covariance matrices is known to be error-prone, one might take advantage of algorithmic advances in Riemannian geometry and matrix manifold to implement methods for Symmetric Positive-Definite (SPD) matrices. Nonetheless, existing classification algorithms in Riemannian spaces are designed for offline analysis. We propose a novel algorithm for online and asynchronous processing of brain signals, borrowing principles from semi-unsupervised approaches and following a dynamic stopping scheme to provide a prediction as soon as possible. The assessment is conducted on real EEG recording: this is the first study on Steady-State Visually Evoked Potential (SSVEP) experimentations to exploit online classification based on Riemannian geometry. The proposed online algorithm is evaluated and compared with state-of-the-art SSVEP methods, which are based on Canonical Correlation Analysis (CCA). It is shown to improve both the classification accuracy and the information transfer rate in the online and asynchronous setup.
From Euclidean to Riemannian Means: Information Geometry for SSVEP Classification
Authors: Emmanuel Kalunga, Sylvain Chevallier, Quentin Barthélemy, Karim Djouani, Yskandar Hamam, Eric Monacelli
Date: 2016-04-03 00:00:00
Proceeding: Geometric Science of Information 2016 4
Tasks: Classification, EEG, Electroencephalogram (EEG), General Classification, SSVEP

Brain Computer Interfaces (BCI) based on electroencephalog-raphy (EEG) rely on multichannel brain signal processing. Most of the state-of-the-art approaches deal with covariance matrices , and indeed Riemannian geometry has provided a substantial framework for developing new algorithms. Most notably , a straightforward algorithm such as Minimum Distance to Mean yields competitive results when applied with a Riemannian distance. This applicative contribution aims at assessing the impact of several distances on real EEG dataset , as the invariances embedded in those distances have an influence on the classification accuracy . Euclidean and Riemannian distances and means are compared both in term of quality of results and of computational load .
Support Vector Machines with Time Series Distance Kernels for Action Classification
Authors: Mohammad Ali Bagheri, Qigang Gao, Sergio Escalera
Date: 2016-03-07 00:00:00
Proceeding: IEEE Winter Conference on Applications of Computer Vision (WACV) 2016 3
Tasks: Action Classification, Action Recognition, Classification, Dynamic Time Warping, General Classification, Temporal Action Localization, Time Series, Time Series Analysis, Time Series Classification

Despite the outperformance of Support Vector Machine (SVM) on many practical classification problems, the algorithm is not directly applicable to multi-dimensional trajectories having different lengths. In this paper, a new class of SVM that is applicable to trajectory classification, such as action recognition, is developed by incorporating two efficient time-series distances measures into the kernel function. Dynamic Time Warping and Longest Common Subsequence distance measures along with their derivatives are employed as the SVM kernel. In addition, the pairwise proximity learning strategy is utilized in order to make use of non-positive semi-definite kernels in the SVM formulation. The proposed method is employed for a challenging classification problem: action recognition by depth cameras using only skeleton data; and evaluated on three benchmark action datasets. Experimental results demonstrate the outperformance of our methodology compared to the state-of-the-art on the considered datasets.
[1]	M .A. Bagheri, Q. Gao, and S. Escalera, "Support Vector Machines with Time Series Distance Kernels for Action Classification", in Proc. IEEE Winter Conference on Applications of Computer Vision, New York, 2016.
Language Models are Unsupervised Multitask Learners
Authors: Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever
Date: 2019-02-14 00:00:00
Proceeding: Preprint 2019 2
Tasks: Common Sense Reasoning, Coreference Resolution, Data-to-Text Generation, Dialogue State Tracking, Document Summarization, Language Modeling, Language Modelling, Machine Translation, Multi-task Language Understanding, Multi-Task Learning, Question Answering, Reading Comprehension, Response Generation, Sentiment Analysis, Text Generation, Translation

Natural language processing tasks, such as question answering, machine translation, reading comprehension, and summarization, are typically
approached with supervised learning on taskspecific datasets. We demonstrate that language
models begin to learn these tasks without any explicit supervision when trained on a new dataset
of millions of webpages called WebText. When
conditioned on a document plus questions, the answers generated by the language model reach 55
F1 on the CoQA dataset - matching or exceeding
the performance of 3 out of 4 baseline systems
without using the 127,000+ training examples.
The capacity of the language model is essential
to the success of zero-shot task transfer and increasing it improves performance in a log-linear
fashion across tasks. Our largest model, GPT-2,
is a 1.5B parameter Transformer that achieves
state of the art results on 7 out of 8 tested language modeling datasets in a zero-shot setting
but still underfits WebText. Samples from the
model reflect these improvements and contain coherent paragraphs of text. These findings suggest
a promising path towards building language processing systems which learn to perform tasks from
their naturally occurring demonstrations.
One Kernel to Solve Nearly Everything: Unified 3D Binary Convolutions for Image Analysis
Authors: MP Heinrich, O Oktay, N Bouteldja
Date: 2018-04-11 00:00:00
Proceeding: MIDL 2018 4
Tasks: Data Augmentation, Medical Image Analysis, Organ Segmentation, Semantic Segmentation

Deep networks have set the state-of-the-art in most image analysis tasks by replacing handcrafted features with learned convolution filters within end-to-end trainable architectures. Still, the specifications of a convolutional network are subject to much manual design - the shape and size of the receptive field for convolutional operations is a very sensitive part that has to be tuned for different image analysis applications. 3D fully-convolutional multi-scale architectures with skip-connection that excel at semantic segmentation and landmark localisation have huge memory requirements and rely on large annotated datasets - an important limitation for wider adaptation in medical image analysis. 

We propose a novel and effective method based on a single trainable 3D convolution kernel that addresses these issues and enables high quality results with a compact four-layer architecture and without sensitive hyperparameters for convolutions and architectural design. Instead of a manual choice of filter size, dilation of weights, and number of scales, our one binary extremely large and inflecting sparse kernel (OBELISK) automatically learns filter offsets in a differentiable continuous space together with weight coefficients. Geometric data augmentation can be directly incorporated into the training by simple coordinate transforms. This powerful new architecture has less than 130'000 parameters, can be trained in few minutes with only 700 MBytes of memory and achieves an increase of Dice overlap of +5.5\% compared to the U-Net for CT multi-organ segmentation.
Overcoming the Disentanglement vs Reconstruction Trade-off via Jacobian Supervision
Authors: José Lezama
Date: 2019-05-01 00:00:00
Proceeding: ICLR 2019 5
Tasks: Attribute, Decoder, Disentanglement, Image Generation, Image Reconstruction

A major challenge in learning image representations is the disentangling of the factors of variation underlying the image formation.  This is typically achieved with an autoencoder architecture where a subset of the latent variables is constrained to correspond to specific factors, and the rest of them are considered nuisance variables. This approach has an important drawback: as the dimension of the nuisance variables is increased, image reconstruction is improved, but the decoder has the flexibility to ignore the specified factors, thus losing the ability to condition the output on them.  In this work, we propose to overcome this trade-off by progressively growing the dimension of the latent code, while constraining the Jacobian of the output image with respect to the disentangled variables to remain the same.  As a result, the obtained models are effective at both disentangling and reconstruction.  We demonstrate the applicability of this method in both unsupervised and supervised scenarios for learning disentangled representations. In a facial attribute manipulation task, we obtain high quality image generation while smoothly controlling dozens of attributes with a single model. This is an order of magnitude more disentangled factors than state-of-the-art methods, while obtaining visually similar or superior results, and avoiding adversarial training.