pwc-archive/papers-with-abstracts huggingface dataset:
Learning Constrained Dynamics with Gauss' Principle adhering Gaussian Processes
Authors: Andreas Geist, Sebastian Trimpe
Date: 2020-06-08 00:00:00
Proceeding: L4DC 2020 6
Tasks: Gaussian Processes, regression, Transfer Learning

The identification of the constrained dynamics of mechanical systems is often challenging. Learning methods promise to ease an analytical analysis, but require considerable amounts of data for training. We propose to combine insights from analytical mechanics with Gaussian process regression to improve the model's data efficiency and constraint integrity. The result is a Gaussian process model that incorporates a priori constraint knowledge such that its predictions adhere Gauss' principle of least constraint. In return, predictions of the system's acceleration naturally respect potentially non-ideal (non-)holonomic equality constraints. As corollary results, our model enables to infer the acceleration of the unconstrained system from data of the constrained system and enables knowledge transfer between differing constraint configurations.
Online Data Poisoning Attacks
Authors: Xuezhou Zhang, Xiaojin Zhu, Laurent Lessard
Date: 2020-06-08 00:00:00
Proceeding: L4DC 2020 6
Tasks: Data Poisoning, Deep Reinforcement Learning, Model Predictive Control, reinforcement-learning, Reinforcement Learning (RL)

We study data poisoning attacks in the online learning setting, where training data arrive sequentially, and the attacker is eavesdropping the data stream and has the ability to contaminate the current data point to affect the online learning process. We formulate the optimal online attack problem as a stochastic optimal control problem, and provide a systematic solution using tools from model predictive control and deep reinforcement learning. We further provide theoretical analysis on the regret suffered by the attacker for not knowing the true data sequence. Experiments validate our control approach in generating near-optimal attacks on both supervised and unsupervised learning tasks
Policy Optimization for $\mathcal{H}_{2}$ Linear Control with $\mathcal{H}_{\infty}$ Robustness Guarantee: Implicit Regularization and Global Convergence
Authors: Kaiqing Zhang, Bin Hu, Tamer Basar
Date: 2020-06-08 00:00:00
Proceeding: L4DC 2020 6
Tasks: Reinforcement Learning (RL)

Policy optimization (PO) is a key ingredient for modern reinforcement learning (RL). For control design, certain constraints are usually enforced on the policies to optimize, accounting for either the stability, robustness, or safety concerns on the system. Hence, PO is by nature a constrained (nonconvex) optimization in most cases, whose global convergence is challenging to analyze in general. More importantly, some constraints that are safety-critical, e.g., the closed-loop stability, or the $\mathcal{H}_{\infty}$-norm constraint that guarantees the system robustness, can be difficult to enforce on the controller being learned as the PO methods proceed. In this paper, we study the convergence theory of PO for $\mathcal{H}_{2}$ linear control with $\mathcal{H}_{\infty}$ robustness guarantee. This general framework includes risk-sensitive linear control as a special case. One significant new feature of this problem, in contrast to the standard $\mathcal{H}_{2}$ linear control, namely, linear quadratic regulator (LQR) problems, is the lack of coercivity of the cost function. This makes it challenging to guarantee the feasibility, namely, the $\mathcal{H}_{\infty}$ robustness, of the iterates. Interestingly, we propose two PO algorithms that enjoy the implicit regularization property, i.e., the iterates preserve the $\mathcal{H}_{\infty}$ robustness, as if they are regularized by the algorithms. Furthermore, convergence to the globally optimal policies with globally sublinear and locally (super-)linear rates are provided under certain conditions, despite the nonconvexity of the problem. To the best of our knowledge, our work offers the first results on the implicit regularization property and global convergence of PO methods for robust/risk-sensitive control.
Linear Antisymmetric Recurrent Neural Networks
Authors: Signe Moe, Filippo Remonato, Esten Ingar Grøtli, Jan Tommy Gravdahl
Date: 2020-06-08 00:00:00
Proceeding: L4DC 2020 6

Recurrent Neural Networks (RNNs) have a form of memory where the output from a node at one timestep is fed back as input the next timestep in addition to data from the previous layer. This makes them highly suitable for timeseries analysis. However, standard RNNs have known weaknesses such as exploding/vanishing gradient and thereby struggle with a long-term memory. In this paper, we suggest a new recurrent network structure called Linear Antisymmetric RNN (LARNN). This structure is based on the numerical solution to an Ordinary Differential Equation (ODE) with stability properties resulting in a stable solution, which corresponds to long-term memory and trainability. Three different numerical methods are suggested to solve the ODE: Forward and Backward Euler and the midpoint method. The suggested structure has been implemented in Keras and several simulated datasets have been used to evaluate the performance. In the investigated cases, the LARNN performs better or similar to the Long Short Term Memory (LSTM) network which is the current state of the art for RNNs.
Smart Forgetting for Safe Online Learning with Gaussian Processes
Authors: Jonas Umlauft, Thomas Beckers, Alexandre Capone, Armin Lederer, Sandra Hirche
Date: 2020-06-08 00:00:00
Proceeding: L4DC 2020 6
Tasks: Computational Efficiency, Gaussian Processes

The identification of unknown dynamical systems using supervised learning enables model-based control of systems that cannot be modeled based on first principles. While most control literature focuses on the analysis of a static dataset, online learning control, where data points are added while the controller is running, has rarely been studied in depth. In this paper, we present a data-efficient approach for online learning control based on Gaussian process models. To enable real-time capability despite high computational loads with growing datasets, we propose a safe forgetting mechanism. Using an entropy criterion, data points are selected based on their utility for the future trajectory under consideration of the stability of the closed-loop system. The approach is evaluated in a simulation and in a robotic experiment to demonstrate its computational efficiency.
Regret Bound for Safe Gaussian Process Bandit Optimization
Authors: Sanae Amani, Mahnoosh Alizadeh, Christos Thrampoulidis
Date: 2020-06-08 00:00:00
Proceeding: L4DC 2020 6
Tasks: Gaussian Processes

Many applications require a learner to make sequential decisions given uncertainty regarding both the system’s payoff function and safety constraints. When learning algorithms are used in safety-critical systems, it is paramount that the learner’s actions do not violate the safety constraints at any stage of the learning process. In this paper, we study a stochastic bandit optimization problem where the system’s unknown payoff and constraint functions are sampled from Gaussian Processes (GPs). We develop a safe variant of the proposed algorithm by Srinivas et al. (2010), GP-UCB, called SGP-UCB, with necessary modifications to respect safety constraints at every round. Our most important contribution is to derive the first sub-linear regret bounds for this problem.
Structured Variational Inference in Partially Observable Unstable Gaussian Process State Space Models
Authors: Sebastian Curi, Silvan Melchior, Felix Berkenkamp, Andreas Krause
Date: 2020-06-08 00:00:00
Proceeding: L4DC 2020 6
Tasks: State Space Models, Variational Inference

We propose a new variational inference algorithm for learning in Gaussian Process State-Space Models (GPSSMs). Our algorithm enables learning of unstable and partially observable systems, where previous algorithms fail. Our main algorithmic contribution is a novel approximate posterior that can be calculated efficiently using a single forward and backward pass along the training trajectories. The forward-backward pass is inspired on Kalman smoothing for linear dynamical systems but generalizes to GPSSMs. Our second contribution is a modification of the conditioning step that effectively lowers the Kalman gain. This modification is crucial to attaining good test performance where no measurements are available. Finally, we show experimentally that our learning algorithm performs well in stable and unstable real systems with hidden states.
Feed-forward Neural Networks with Trainable Delay
Authors: Xunbi Ji, Sergei Avedisov, Tamas Molnar, Gabor Orosz
Date: 2020-06-08 00:00:00
Proceeding: L4DC 2020 6

In this paper we build a bridge between feed-forward neural networks and delayed dynamical systems. As an initial demonstration, we capture the car-following behavior of a connected automated vehicle including time delay using both simulation data and experimental data. We construct a delayed feed-forward neural network (DFNN) and introduce a training algorithm in order to learn the delay. We demonstrate that this algorithm works well on the proposed structures.
Learning solutions to hybrid control problems using Benders cuts
Authors: Sandeep Menta, Joseph Warrington, John Lygeros, Manfred Morari
Date: 2020-06-08 00:00:00
Proceeding: L4DC 2020 6
Tasks: Model Predictive Control

Hybrid control problems are complicated by the need to make a suitable sequence of discrete decisions related to future modes of operation of the system. Model predictive control (MPC) encodes a finite-horizon truncation of such problems as a mixed-integer program, and then imposes a cost and/or constraints on the terminal state intended to reflect all post-horizon behaviour. However, these are often ad hoc choices tuned by hand after empirically observing performance. We present a learning method that sidesteps this problem, in which the so-called N-step Q-function of the problem is approximated from below, using Benders’ decomposition. The function takes a state and a sequence of N control decisions as arguments, and therefore extends the traditional notion of a Q-function from reinforcement learning. After learning it from a training process exploring the state-input space, we use it in place of the usual MPC objective. We take an example hybrid control task and show that it can be completed successfully with a shorter planning horizon than conventional hybrid MPC thanks to our proposed method. Furthermore, we report that Q-functions trained with long horizons can be truncated to a shorter horizon for online use, yielding simpler control laws with apparently little loss of performance.
Bayesian joint state and parameter tracking in autoregressive models
Authors: Ismail Senoz, Albert Podusenko, Wouter M. Kouw, Bert de Vries
Date: 2020-06-08 00:00:00
Proceeding: L4DC 2020 6

We address the problem of online Bayesian state and parameter tracking in autoregressive (AR) models with time-varying process noise variance. The involved marginalization and expectation integrals cannot be analytically solved. Moreover, the online tracking constraint makes sampling and batch learning methods unsuitable for this problem. We propose a hybrid variational message passing algorithm that robustly tracks the time-varying dynamics of the latent states, AR coefficients and process noise variance. Since message passing in a factor graph is a highly modular inference approach, the proposed methods easily extend to other non-stationary dynamic modeling problems.
Estimating Reachable Sets with Scenario Optimization
Authors: Alex Devonport, Murat Arcak
Date: 2020-06-08 00:00:00
Proceeding: L4DC 2020 6

Many practical systems are not amenable to the reachability methods that give guarantees of correctness, since they have dynamics that are strongly nonlinear, uncertain, and possibly unknown. While reachable sets for these kinds of systems can still be estimated in a data-driven way, datadriven methods typically do not guarantee the validity of their results. However, certain data-driven approaches may be given a probabilistic guarantee of correctness, by reframing the problem as a chance-constrained optimization problem that is solved with scenario optimization. We apply this approach to the problem of approximating a reachable set by a norm ball from data. The method requires only O(n^2 ) sample trajectories and the solution of a convex problem. A variant of the method restricted to axis-aligned norm balls requires only O(n) samples.
Direct data-driven control with embedded anti-windup compensation
Authors: Valentina Breschi, Simone Formentin
Date: 2020-06-08 00:00:00
Proceeding: L4DC 2020 6

Input saturation is an ubiquitous nonlinearity in control systems and arises from the fact that all actuators are subject to a maximum power, thereby resulting in a hard limitation on the allowable magnitude of the input effort. In the scientific literature, anti-windup augmentation has been proposed to recover the desired linear closed-loop dynamics during transients, but the effectiveness of such a compensation is strongly linked to the accuracy of the mathematical model of the plant. In this work, it is shown that a feedback controller with embedded anti-windup compensator can be directly identified from data, by suitably extending the existing data-driven design theory. The effectiveness of the resulting method is illustrated on a benchmark simulation example.
Virtual Reference Feedback Tuning with data-driven reference model selection
Authors: Valentina Breschi, Simone Formentin
Date: 2020-06-08 00:00:00
Proceeding: L4DC 2020 6
Tasks: Model Selection, valid

In control applications where finding a model of the plant is the most costly and time consuming task, Virtual Reference Feedback Tuning (VRFT) represents a valid - purely data-driven - alternative for the design of model reference controllers. However, the selection of a proper reference model within a model-free setting is known to be a critical task, with this model typically playing the role of a hyper-parameter. In this work, we extend the VRFT methodology to compute both a proper reference model and the corresponding optimal controller parameters from data by means of Particle Swarm optimization. The effectiveness of the proposed approach is illustrated on a benchmark simulation example.
Finite Sample System Identification: Improved Rates and the Role of Regularization
Authors: Yue Sun, Samet Oymak, Maryam Fazel
Date: 2020-06-08 00:00:00
Proceeding: L4DC 2020 6

This paper studies low-order linear system identification via regularized regression. The nuclear norm of the system’s Hankel matrix is added as a regularizer to the least-squares cost function due to the following advantages. The regularized problem is (1) often easier to tune, (2) has lower sample complexity, and (3) returns a Hankel matrix with a clear singular value gap, which robustly recovers a low-order linear system from noisy output observations. Recently, the performance of the unregularized least-squares formulations have been studied statistically in terms of finite sample complexity and recovery errors; however, no results are known for the regularized approach. In this work, we provide a novel statistical analysis of the regularized algorithm. Our analysis leads to new bounds on estimating the impulse response and the Hankel matrix associated with the system while using smaller number of observations than the least-squares estimator.
Attention-based Multiple Instance Learning with Mixed Supervision on the Camelyon16 Dataset
Authors: Paul Tourniaire, Marius Ilie, Paul Hofman, Nicholas Ayache, Hervé Delingette
Date: 2021-07-20 00:00:00
Proceeding: MICCAI Workshop COMPAY 2021 9
Tasks: Multiple Instance Learning, whole slide images

Since the standardization of Whole Slide Images (WSIs) digitization, the use of deep learning methods for the analysis of histological images has shown much potential. However, the sheer size of WSIs is a real challenge, as they are often up to 100,000 pixels wide and high at the highest resolution, and therefore cannot be processed directly by any model. Moreover, as the manual delineation of structures within WSIs is tedious, histological datasets often only contain slide-level labels, or a limited amount of delineated slides. In this context, multiple-instance learning (MIL) approaches have been proposed, considering WSIs as bags of smaller images, designated as tiles or patches. Among these methods, the attention-based MIL aims at learning the importance of each tile for the slide final classification while at the same time performing a clustering of those tiles. In this paper, we introduce the concept of  mixed supervision within this framework, by exploiting tile-level labels in addition to slide-level labels to improve the classification of slides. More precisely, we show on the  Camelyon16 dataset that even a small proportion  of slides with pixel-wise annotations can improve their classification  but also the localization of tumorous regions. This improves the consistency of the results between the tile and slide levels and the interpretability of the algorithm.
Improving Mask R-CNN for Nuclei Instance Segmentation in Hematoxylin & Eosin-Stained Histological Images
Authors: Benjamin Bancher, Amirreza Mahbod, Isabella Ellinger, Rupert Ecker, Georg Dorffner
Date: 2021-07-20 00:00:00
Proceeding: MICCAI Workshop COMPAY 2021 9
Tasks: Instance Segmentation, Segmentation, Semantic Segmentation, Transfer Learning, whole slide images

Digital pathology is an emerging topic in the analysis of pathologic tissue samples. It includes providing the tools towards more automated workflows to derive clinically relevant information. Digitization and storage of whole slide images have become commonplace and allow modern image analysis methods to be used. In recent years, computer-based segmentation of cell nuclei has gathered considerable attention in the development of highly specialized algorithms. Currently, most of these algorithms are based on performing semantic segmentation of all cell nuclei and separating overlapping instances in a post-processing step. Recently, instance-aware segmentation methods such as Mask R-CNN have been proposed to enable unified instance detection and segmentation, even in overlapping cases. In this work, we propose a modified Mask R-CNN-based approach by incorporating distance maps of instances and hematoxylin-stain intensities as extra input channels to the model. Moreover, we explore the impact of three well-known inference strategies, namely test-time augmentation, ensembling, and knowledge transfer through pre-training on the segmentation performance. We perform extensive ablation experiments across multiple runs to quantitatively define the most optimal inference strategy in the proposed Mask R-CNN algorithm. Our results show that average instance segmentation improvements of up to 3.5% and 4.1% based on Aggregate Jaccard Index and Panoptic Quality score can be obtained, respectively, using the proposed techniques in comparison to a standard Mask R-CNN model. Our findings confirm the effectiveness of aggregating information at the network input stage and optimizing inference workflows using minimal effort. Implemented modifications and codes are publicly available through a GitHub repository under: https://github.com/bbanc/Improved-Mask-R-CNN-for-nuclei-segmentation.
Unsupervised Domain Adaptation for the Histopathological Cell Segmentation through Self-Ensembling
Authors: CHAOQUN LI, Yitian Zhou, TangQi Shi, Yenan Wu, Meng Yang, Zhongyu Li
Date: 2021-07-20 00:00:00
Proceeding: MICCAI Workshop COMPAY 2021 9
Tasks: Cell Segmentation, Domain Adaptation, Segmentation, Unsupervised Domain Adaptation

Histopathological images are generally considered as the golden standard for clinical diagnosis and cancer grading. Accurate segmentation of cells/nuclei from histopathological images is a critical step to obtain reliable morphological information for quantitative analysis. However, cell/nuclei segmentation relies heavily on well-annotated datasets, which are extremely labor-intensive, time-consuming, and expensive in practical applications. Meanwhile, one might want to fine-tune pretrained models on certain target datasets. But it is always difficult to collect enough target training images for proper fine-tuning. Therefore, there is a need for methods that can transfer learned information from one domain to another without additional target annotations. In this paper, we propose a novel framework for cell segmentation on the unlabeled images through the unsupervised domain adaptation with self-ensembling. It is achieved by applying generative adversarial networks (GANs) for the unsupervised domain adaptation of cell segmentation crossing different tissues. Images in the source and target domain can be differentiated through the learned discriminator. Meanwhile, we present a self-ensembling model to consider the source and the target domain together as a semi-supervised segmentation task to reduce the differences of outputs. Additionally, we introduce conditional random field (CRF) as post-processing to preserve the local consistency on the outputs. We validate our framework with unsupervised domain adaptation on three public cell segmentation datasets captured from different types of tissues, which achieved superior performance in comparison with state-of-the-art.
End-to-end Multiple Instance Learning for Whole-Slide Cytopathology of Urothelial Carcinoma
Authors: Joshua Butke, Tatjana Frick, Florian Roghmann, Samir El-Mashtoly, Klaus Gerwert, Axel Mosig
Date: 2021-07-20 00:00:00
Proceeding: MICCAI Workshop COMPAY 2021 9
Tasks: Multiple Instance Learning, whole slide images

As a non-invasive approach, cytopathology of urine sediment is a highly promising approach to diagnosing urothelial carcinoma. However, computational assessment of the cytopathological status of a sample raises the challenge of identifying few cancerous cells among thousands of cells in a microscopic whole-slide image.   
To address this challenge, we propose an end-to-end trainable multiple instance learning approach that combines the attention mechanism and hard negative mining to classify hematoxylin and eosin stained patient-level whole-slide images of urine sediment cells. 
The singular cells are extracted by a simple foreground detection algorithm. With feature embeddings computed for each image patch in a bag by a convolutional neural network, the attention mechanism serves as the pooling operator, enabling a bag-level prediction while still giving an interpretable score for each image patch. 
This enables the identification of key instances and potential regions of interest that trigger a patient-level decision. Our results show that the proposed system can differentiate between normal and cancerous urothelial cells, thus enabling the non-invasive diagnosis of urothelial carcinoma in patients using urine sediment analysis.