pwc-archive/papers-with-abstracts huggingface dataset:
Generalizing Complex/Hyper-complex Convolutions to Vector Map Convolutions
Authors: Chase J Gaudet, Anthony S. Maida
Date: 2020-09-09 00:00:00

We show that the core reasons that complex and hypercomplex valued neural networks offer improvements over their real-valued counterparts is the weight sharing mechanism and treating multidimensional data as a single entity. Their algebra linearly combines the dimensions, making each dimension related to the others. However, both are constrained to a set number of dimensions, two for complex and four for quaternions. Here we introduce novel vector map convolutions which capture both of these properties provided by complex/hypercomplex convolutions, while dropping the unnatural dimensionality constraints they impose. This is achieved by introducing a system that mimics the unique linear combination of input dimensions, such as the Hamilton product for quaternions. We perform three experiments to show that these novel vector map convolutions seem to capture all the benefits of complex and hyper-complex networks, such as their ability to capture internal latent relations, while avoiding the dimensionality restriction.
Method for classifying a noisy Raman spectrum based on a wavelet transform and a deep neural network
Authors: Liangrui Pan, Pronthep Pipitsunthonsan, Chalongrat Daengngam, Sittiporn Channumsin, Suwat Sreesawet, Mitchai Chongcheawchamnan
Date: 2020-09-09 00:00:00

This paper proposes a new framework based on a wavelet transform and deep neural network for identifying noisy Raman spectrum since, in practice, it is relatively difficult to classify the spectrum under baseline noise and additive white Gaussian noise environments. The framework consists of two main engines. Wavelet transform is proposed as the framework front-end for transforming 1-D noise Raman spectrum to two-dimensional data. This two-dimensional data will be fed to the framework back-end which is a classifier. The optimum classifier is chosen by implementing several traditional machine learning (ML) and deep learning (DL) algorithms, and then we investigated their classification accuracy and robustness performances. The four MLs we choose included a Naive Bayes (NB), a Support Vector Machine (SVM), a Random Forest (RF) and a K-Nearest Neighbor (KNN) where a deep convolution neural network (DCNN) was chosen for a DL classifier. Noise-free, Gaussian noise, baseline noise, and mixed-noise Raman spectrums were applied to train and validate the ML and DCNN models. The optimum back-end classifier was obtained by testing the ML and DCNN models with several noisy Raman spectrums (10-30 dB noise power). Based on the simulation, the accuracy of the DCNN classifier is 9% higher than the NB classifier, 3.5% higher than the RF classifier, 1% higher than the KNN classifier, and 0.5% higher than the SVM classifier. In terms of robustness to the mixed noise scenarios, the framework with DCNN back-end showed superior performance than the other ML back-ends. The DCNN back-end achieved 90% accuracy at 3 dB SNR while NB, SVM, RF, and K-NN back-ends required 27 dB, 22 dB, 27 dB, and 23 dB SNR, respectively. In addition, in the low-noise test data set, the F-measure score of the DCNN back-end exceeded 99.1% while the F-measure scores of the other ML engines were below 98.7%.
1-Dimensional polynomial neural networks for audio signal related problems
Authors: Habib Ben Abdallah, Christopher J. Henry, Sheela Ramanna
Date: 2020-09-09 00:00:00
Tasks: Polynomial Neural Networks

In addition to being extremely non-linear, modern problems require millions if not billions of parameters to solve or at least to get a good approximation of the solution, and neural networks are known to assimilate that complexity by deepening and widening their topology in order to increase the level of non-linearity needed for a better approximation. However, compact topologies are always preferred to deeper ones as they offer the advantage of using less computational units and less parameters. This compacity comes at the price of reduced non-linearity and thus, of limited solution search space. We propose the 1-Dimensional Polynomial Neural Network (1DPNN) model that uses automatic polynomial kernel estimation for 1-Dimensional Convolutional Neural Networks (1DCNNs) and that introduces a high degree of non-linearity from the first layer which can compensate the need for deep and/or wide topologies. We show that this non-linearity enables the model to yield better results with less computational and spatial complexity than a regular 1DCNN on various classification and regression problems related to audio signals, even though it introduces more computational and spatial complexity on a neuronal level. The experiments were conducted on three publicly available datasets and demonstrate that, on the problems that were tackled, the proposed model can extract more relevant information from the data than a 1DCNN in less time and with less memory.
Investigation of REFINED CNN ensemble learning for anti-cancer drug sensitivity prediction
Authors: Omid Bazgir, Souparno Ghosh, Ranadip Pal
Date: 2020-09-09 00:00:00
Tasks: Ensemble Learning, Prediction, Sensitivity

Anti-cancer drug sensitivity prediction using deep learning models for individual cell line is a significant challenge in personalized medicine. REFINED (REpresentation of Features as Images with NEighborhood Dependencies) CNN (Convolutional Neural Network) based models have shown promising results in drug sensitivity prediction. The primary idea behind REFINED CNN is representing high dimensional vectors as compact images with spatial correlations that can benefit from convolutional neural network architectures. However, the mapping from a vector to a compact 2D image is not unique due to variations in considered distance measures and neighborhoods. In this article, we consider predictions based on ensembles built from such mappings that can improve upon the best single REFINED CNN model prediction. Results illustrated using NCI60 and NCIALMANAC databases shows that the ensemble approaches can provide significant performance improvement as compared to individual models. We further illustrate that a single mapping created from the amalgamation of the different mappings can provide performance similar to stacking ensemble but with significantly lower computational complexity.
Multilinear Latent Conditioning for Generating Unseen Attribute Combinations
Authors: Markos Georgopoulos, Grigorios Chrysos, Maja Pantic, Yannis Panagakis
Date: 2020-09-09 00:00:00
Proceeding: ICML 2020 1
Tasks: Attribute, Inductive Bias

Deep generative models rely on their inductive bias to facilitate generalization, especially for problems with high dimensional data, like images. However, empirical studies have shown that variational autoencoders (VAE) and generative adversarial networks (GAN) lack the generalization ability that occurs naturally in human perception. For example, humans can visualize a woman smiling after only seeing a smiling man. On the contrary, the standard conditional VAE (cVAE) is unable to generate unseen attribute combinations. To this end, we extend cVAE by introducing a multilinear latent conditioning framework that captures the multiplicative interactions between the attributes. We implement two variants of our model and demonstrate their efficacy on MNIST, Fashion-MNIST and CelebA. Altogether, we design a novel conditioning framework that can be used with any architecture to synthesize unseen attribute combinations.
Exploiting Multi-Modal Features From Pre-trained Networks for Alzheimer's Dementia Recognition
Authors: Junghyun Koo, Jie Hwan Lee, Jaewoo Pyo, Yujin Jo, Kyogu Lee
Date: 2020-09-09 00:00:00
Tasks: regression

Collecting and accessing a large amount of medical data is very time-consuming and laborious, not only because it is difficult to find specific patients but also because it is required to resolve the confidentiality of a patient's medical records. On the other hand, there are deep learning models, trained on easily collectible, large scale datasets such as Youtube or Wikipedia, offering useful representations. It could therefore be very advantageous to utilize the features from these pre-trained networks for handling a small amount of data at hand. In this work, we exploit various multi-modal features extracted from pre-trained networks to recognize Alzheimer's Dementia using a neural network, with a small dataset provided by the ADReSS Challenge at INTERSPEECH 2020. The challenge regards to discern patients suspicious of Alzheimer's Dementia by providing acoustic and textual data. With the multi-modal features, we modify a Convolutional Recurrent Neural Network based structure to perform classification and regression tasks simultaneously and is capable of computing conversations with variable lengths. Our test results surpass baseline's accuracy by 18.75%, and our validation result for the regression task shows the possibility of classifying 4 classes of cognitive impairment with an accuracy of 78.70%.
Noise Reduction Technique for Raman Spectrum using Deep Learning Network
Authors: Liangrui Pan, Pronthep Pipitsunthonsan, Peng Zhang, Chalongrat Daengngam, Apidach Booranawong, Mitcham Chongcheawchamnan
Date: 2020-09-09 00:00:00
Tasks: Deep Learning

In a normal indoor environment, Raman spectrum encounters noise often conceal spectrum peak, leading to difficulty in spectrum interpretation. This paper proposes deep learning (DL) based noise reduction technique for Raman spectroscopy. The proposed DL network is developed with several training and test sets of noisy Raman spectrum. The proposed technique is applied to denoise and compare the performance with different wavelet noise reduction methods. Output signal-to-noise ratio (SNR), root-mean-square error (RMSE) and mean absolute percentage error (MAPE) are the performance evaluation index. It is shown that output SNR of the proposed noise reduction technology is 10.24 dB greater than that of the wavelet noise reduction method while the RMSE and the MAPE are 292.63 and 10.09, which are much better than the proposed technique.
View-consistent 4D Light Field Depth Estimation
Authors: Numair Khan, Min H. Kim, James Tompkin
Date: 2020-09-09 00:00:00
Tasks: Depth Estimation

We propose a method to compute depth maps for every sub-aperture image in a light field in a view consistent way. Previous light field depth estimation methods typically estimate a depth map only for the central sub-aperture view, and struggle with view consistent estimation. Our method precisely defines depth edges via EPIs, then we diffuse these edges spatially within the central view. These depth estimates are then propagated to all other views in an occlusion-aware way. Finally, disoccluded regions are completed by diffusion in EPI space. Our method runs efficiently with respect to both other classical and deep learning-based approaches, and achieves competitive quantitative metrics and qualitative performance on both synthetic and real-world light fields
Going Deep: Using deep learning techniques with simplified mathematical models against XOR BR and TBR PUFs (Attacks and Countermeasures)
Authors: Mahmoud Khalafalla, Mahmoud A. Elmohr, Catherine Gebotys
Date: 2020-09-09 00:00:00

This paper contributes to the study of PUFs vulnerability against modeling attacks by evaluating the security of XOR BR PUFs, XOR TBR PUFs, and obfuscated architectures of XOR BR PUF using a simplified mathematical model and deep learning (DL) techniques. Obtained results show that DL modeling attacks could easily break the security of 4-input XOR BR PUFs and 4-input XOR TBR PUFs with modeling accuracy $\sim$ 99%. Similar attacks were executed using single-layer neural networks (NN) and support vector machines (SVM) with polynomial kernel and the obtained results showed that single NNs failed to break the PUF security. Furthermore, SVM results confirmed the same modeling accuracy reported in previous research ($\sim$ 50%). For the first time, this research empirically shows that DL networks can be used as powerful modeling techniques against these complex PUF architectures for which previous conventional machine learning techniques had failed. Furthermore, a detailed scalability analysis is conducted on the DL networks with respect to PUFs' stage size and complexity. The analysis shows that the number of layers and hidden neurons inside every layer has a linear relationship with PUFs' stage size, which agrees with the theoretical findings in deep learning. Consequently, A new obfuscated architecture is introduced as a first step to counter DL modeling attacks and it showed significant resistance against such attacks (16% - 40% less accuracy). This research provides an important step towards prioritizing the efforts to introduce new PUF architectures that are more secure and invulnerable to modeling attacks. Moreover, it triggers future discussions on the removal of influential bits and the level of obfuscation needed to confirm that a specific PUF architecture is resistant against powerful DL modeling attacks.
Improved Trainable Calibration Method for Neural Networks on Medical Imaging Classification
Authors: Gongbo Liang, Yu Zhang, Xiaoqin Wang, Nathan Jacobs
Date: 2020-09-09 00:00:00
Tasks: Classification, Decision Making, General Classification, image-classification, Image Classification, Uncertainty Quantification

Recent works have shown that deep neural networks can achieve super-human performance in a wide range of image classification tasks in the medical imaging domain. However, these works have primarily focused on classification accuracy, ignoring the important role of uncertainty quantification. Empirically, neural networks are often miscalibrated and overconfident in their predictions. This miscalibration could be problematic in any automatic decision-making system, but we focus on the medical field in which neural network miscalibration has the potential to lead to significant treatment errors. We propose a novel calibration approach that maintains the overall classification accuracy while significantly improving model calibration. The proposed approach is based on expected calibration error, which is a common metric for quantifying miscalibration. Our approach can be easily integrated into any classification task as an auxiliary loss term, thus not requiring an explicit training round for calibration. We show that our approach reduces calibration error significantly across various architectures and datasets.
Tunable Subnetwork Splitting for Model-parallelism of Neural Network Training
Authors: Junxiang Wang, Zheng Chai, Yue Cheng, Liang Zhao
Date: 2020-09-09 00:00:00

Alternating minimization methods have recently been proposed as alternatives to the gradient descent for deep neural network optimization. Alternating minimization methods can typically decompose a deep neural network into layerwise subproblems, which can then be optimized in parallel. Despite the significant parallelism, alternating minimization methods are rarely explored in training deep neural networks because of the severe accuracy degradation. In this paper, we analyze the reason and propose to achieve a compelling trade-off between parallelism and accuracy by a reformulation called Tunable Subnetwork Splitting Method (TSSM), which can tune the decomposition granularity of deep neural networks. Two methods gradient splitting Alternating Direction Method of Multipliers (gsADMM) and gradient splitting Alternating Minimization (gsAM) are proposed to solve the TSSM formulation. Experiments on five benchmark datasets show that our proposed TSSM can achieve significant speedup without observable loss of training accuracy. The code has been released at https://github.com/xianggebenben/TSSM.
An Experimentally Driven Automated Machine Learned lnter-Atomic Potential for a Refractory Oxide
Authors: Ganesh Sivaraman, Leighanne Gallington, Anand Narayanan Krishnamoorthy, Marius Stan, Gabor Csanyi, Alvaro Vazquez-Mayagoitia, Chris J. Benmore
Date: 2020-09-09 00:00:00
Tasks: BIG-bench Machine Learning

Understanding the structure and properties of refractory oxides are critical for high temperature applications. In this work, a combined experimental and simulation approach uses an automated closed loop via an active-learner, which is initialized by X-ray and neutron diffraction measurements, and sequentially improves a machine-learning model until the experimentally predetermined phase space is covered. A multi-phase potential is generated for a canonical example of the archetypal refractory oxide, HfO2, by drawing a minimum number of training configurations from room temperature to the liquid state at ~2900oC. The method significantly reduces model development time and human effort.
Unconstrained Text Detection in Manga: a New Dataset and Baseline
Authors: Julián Del Gobbo, Rosana Matuk Herrera
Date: 2020-09-09 00:00:00
Tasks: Binarization, Text Detection

The detection and recognition of unconstrained text is an open problem in research. Text in comic books has unusual styles that raise many challenges for text detection. This work aims to binarize text in a comic genre with highly sophisticated text styles: Japanese manga. To overcome the lack of a manga dataset with text annotations at a pixel level, we create our own. To improve the evaluation and search of an optimal model, in addition to standard metrics in binarization, we implement other special metrics. Using these resources, we designed and evaluated a deep network model, outperforming current methods for text binarization in manga in most metrics.
Data-assisted combustion simulations with dynamic submodel assignment using random forests
Authors: Wai Tong Chung, Aashwin Ananda Mishra, Nikolaos Perakis, Matthias Ihme
Date: 2020-09-08 00:00:00

In this investigation, we outline a data-assisted approach that employs random forest classifiers for local and dynamic combustion submodel assignment in turbulent-combustion simulations. This method is applied in simulations of a single-element GOX/GCH4 rocket combustor; a priori as well as a posteriori assessments are conducted to (i) evaluate the accuracy and adjustability of the classifier for targeting different quantities-of-interest (QoIs), and (ii) assess improvements, resulting from the data-assisted combustion model assignment, in predicting target QoIs during simulation runtime. Results from the a priori study show that random forests, trained with local flow properties as input variables and combustion model errors as training labels, assign three different combustion models - finite-rate chemistry (FRC), flamelet progress variable (FPV) model, and inert mixing (IM) - with reasonable classification performance even when targeting multiple QoIs. Applications in a posteriori studies demonstrate improved predictions from data-assisted simulations, in temperature and CO mass fraction, when compared with monolithic FPV calculations. These results demonstrate that this data-driven framework holds promise for the dynamic combustion submodel assignment in reacting flow simulations.
Brown University at TREC Deep Learning 2019
Authors: George Zerveas, Ruochen Zhang, Leila Kim, Carsten Eickhoff
Date: 2020-09-08 00:00:00
Tasks: Deep Learning, Information Retrieval, Language Modeling, Language Modelling, Passage Retrieval, Re-Ranking, Retrieval

This paper describes Brown University's submission to the TREC 2019 Deep Learning track. We followed a 2-phase method for producing a ranking of passages for a given input query: In the the first phase, the user's query is expanded by appending 3 queries generated by a transformer model which was trained to rephrase an input query into semantically similar queries. The expanded query can exhibit greater similarity in surface form and vocabulary overlap with the passages of interest and can therefore serve as enriched input to any downstream information retrieval method. In the second phase, we use a BERT-based model pre-trained for language modeling but fine-tuned for query - document relevance prediction to compute relevance scores for a set of 1000 candidate passages per query and subsequently obtain a ranking of passages by sorting them based on the predicted relevance scores. According to the results published in the official Overview of the TREC Deep Learning Track 2019, our team ranked 3rd in the passage retrieval task (including full ranking and re-ranking), and 2nd when considering only re-ranking submissions.
Attribute Privacy: Framework and Mechanisms
Authors: Wanrong Zhang, Olga Ohrimenko, Rachel Cummings
Date: 2020-09-08 00:00:00
Tasks: Attribute

Ensuring the privacy of training data is a growing concern since many machine learning models are trained on confidential and potentially sensitive data. Much attention has been devoted to methods for protecting individual privacy during analyses of large datasets. However in many settings, global properties of the dataset may also be sensitive (e.g., mortality rate in a hospital rather than presence of a particular patient in the dataset). In this work, we depart from individual privacy to initiate the study of attribute privacy, where a data owner is concerned about revealing sensitive properties of a whole dataset during analysis. We propose definitions to capture \emph{attribute privacy} in two relevant cases where global attributes may need to be protected: (1) properties of a specific dataset and (2) parameters of the underlying distribution from which dataset is sampled. We also provide two efficient mechanisms and one inefficient mechanism that satisfy attribute privacy for these settings. We base our results on a novel use of the Pufferfish framework to account for correlations across attributes in the data, thus addressing "the challenging problem of developing Pufferfish instantiations and algorithms for general aggregate secrets" that was left open by \cite{kifer2014pufferfish}.
Learning joint segmentation of tissues and brain lesions from task-specific hetero-modal domain-shifted datasets
Authors: Reuben Dorent, Thomas Booth, Wenqi Li, Carole H. Sudre, Sina Kafiabadi, Jorge Cardoso, Sebastien Ourselin, Tom Vercauteren
Date: 2020-09-08 00:00:00
Tasks: Data Augmentation, Lesion Segmentation, Segmentation

Brain tissue segmentation from multimodal MRI is a key building block of many neuroimaging analysis pipelines. Established tissue segmentation approaches have, however, not been developed to cope with large anatomical changes resulting from pathology, such as white matter lesions or tumours, and often fail in these cases. In the meantime, with the advent of deep neural networks (DNNs), segmentation of brain lesions has matured significantly. However, few existing approaches allow for the joint segmentation of normal tissue and brain lesions. Developing a DNN for such a joint task is currently hampered by the fact that annotated datasets typically address only one specific task and rely on task-specific imaging protocols including a task-specific set of imaging modalities. In this work, we propose a novel approach to build a joint tissue and lesion segmentation model from aggregated task-specific hetero-modal domain-shifted and partially-annotated datasets. Starting from a variational formulation of the joint problem, we show how the expected risk can be decomposed and optimised empirically. We exploit an upper bound of the risk to deal with heterogeneous imaging modalities across datasets. To deal with potential domain shift, we integrated and tested three conventional techniques based on data augmentation, adversarial learning and pseudo-healthy generation. For each individual task, our joint approach reaches comparable performance to task-specific and fully-supervised models. The proposed framework is assessed on two different types of brain lesions: White matter lesions and gliomas. In the latter case, lacking a joint ground-truth for quantitative assessment purposes, we propose and use a novel clinically-relevant qualitative assessment methodology.