title: Robustness verification and novelty detection of deep learning-based raman spectral Analysis
Creators: name: Lei, Zixin
Publisher: name: The University of Edinburgh
Year: 2022
Subjects: subject: neural networks
subject: neural network-based Sparse Coding
subject: multi-label classification
subject: robustness verification
subject: Learned Iterative Shrinkage and Thresholding Algorithm (LISTA)
subject: non-negative counterpart Non-Negative LISTA (NNLISTA)
subject: Mixed Integer Linear Programming
subject: Generative Adversarial Networks (GAN)
subject: Multi-NDGan

description: Sparse coding aims at forming a linear combination of a few basis atoms out of a large set, to reconstruct input vectors. Conventional model-based methods often tend to be slow and unsuccessful in exact sparse recovery when the dictionary is coherent. A good amount of research have recently been devoted to address this problem using neural networks, such as deep unfolding approaches and fully learned approaches. Although neural networks have achieved success in both accuracy and prediction speed, there is lack of robustness verification methods for neural network-based Sparse Coding (NNSC) and multi-label classification approaches to quantify the output error due to the input perturbations. This thesis formulates the problem of NNSC robustness verification and develops three robustness verification methods for Learned Iterative Shrinkage and Thresholding Algorithm (LISTA), and its non-negative counterpart Non-Negative LISTA (NNLISTA), which are prominent data adaptive sparse coding algorithms, including a dual-network based method, the tightest Linear Programming (LP) relaxed method and a Mixed Integer Linear Programming (MILP)-based method. Another potential issue in conventional machine learning approach is that solutions are incapable of identifying the existence of inputs which are different from those observed during the training (called novelties) and fail silently. Some works have proposed algorithms for novelty detection and some of them jointly consider the tasks of reliable classification and novelty detection. However, there is a lack of deep-learning based methods that consider sparse decomposition and novelty detection jointly. In this thesis, a Generative Adversarial Networks (GAN)-based method called Multi-NDGan is proposed for simultaneous sparse decomposition and novelty detection. On the application side, Raman spectral mixture analysis was selected which aims at separating the input spectrum to the elementary components to be able to identify materials contained in the mixture. It can be formulated as a sparse coding problem and is the application considered in this project. Experiments show that fully-trained NNLISTA is more robust on l-infinity bounded perturbations than fully-trained standard LISTA for non-negative sparse coding, and the proposed robustness verification methods can be used as a guide for the training process of NNSC. Experiments for novelty detection show that the proposed Multi-NDGan can obtain state-of-the-art novelty detection accuracy and operate sparse decomposition for mixtures containing novel components with significantly higher accuracy than existing methods.