pwc-archive/papers-with-abstracts huggingface dataset:
A Deep Generative Model Imitating Predictive Coding in Human Brain
Authors: Anonymous
Date: 2020-10-09 00:00:00
Tasks: Prediction, Prediction Intervals

In recent years, the development of deep generative models for prediction has been attracting attention. In our study, we focus on predictive coding, a concept from the neuroscience literature that hypothesizes the brain is constantly making predictions of sensory input, and attempt to develop a human brain-like prediction model. PredNet is a deep learning model based on the concept of predictive coding, however, it can predict the next image with the same prediction interval, but not with any intervals. On the other hand, Temporal Differential Variational Auto-Encoder (TD-VAE) can predict the next images with any prediction intervals, although it is not a model reflecting human brain function. In this work, we develop a new human brain-like prediction model by unifying PredNet and TD- VAE, combining both predictive coding and flexible interval prediction abilities in one single model. Through experiments on the KITTI Vision Benchmark, we confirmed that our proposed model can predict the next images correctly with flexible prediction intervals. We also investigated the correlation between the feature values of representation layers in the model architecture and human brain activity data evoked under natural video stimulation.
Investigating Emotion-Color Association in Deep Neural Networks
Authors: Anonymous
Date: 2020-10-09 00:00:00
Tasks: Emotion Classification

Recent research has shown that Deep Neural Networks (DNNs) correlate very well to neural responses, and are widely used by cognitive scientists as a proxy for human representation to model human behavior. But previously it has not been explored whether DNNs capture any aspects of stimuli association. In this study, we experimentally investigate if DNNs can learn implicit associations in stimuli, particularly, an emotion-color association between image stimuli. Our study was conducted in two parts. First, we collected human responses on a forced-choice decision task in which subjects were asked to select a color for a specified emotion-inducing image. Next, we modeled this decision task on neural networks using the similarity between deep representation (extracted using DNNs trained on object classification tasks) of the stimuli images and images of colors used in the task. We found that our model showed a fuzzy linear relationship between the two decision probabilities. This results in two interesting findings, 1. The representations learned by deep neural networks can indeed show an emotion-color association 2. The emotion-color association is not just random but involves some cognitive phenomena. Finally, we also show that this method can help us in the emotion classification task.
Probabilistic Information Processing in Humans and Recurrent Neural Networks
Authors: Anonymous
Date: 2020-10-09 00:00:00
Tasks: Decision Making

In nature, sensory inputs are often highly structured, and statistical regularities of these signals can be extracted to form expectation about future sensorimotor associations, thereby facilitating optimal behavior. To date, the circuit mechanisms that underlie these probabilistic computations are not well understood. Through a human electrophysiolgical experiment and a recurrent neural network (RNN) model, the present study investigates how the brain extracts, processes, and utilizes probabilistic structures of sensory signals to guide behavior. To achieve this goal, we first constructed and trained a biophysically constrained RNN model to perform a probabilistic decision making task similar to task paradigms designed for humans. Specifically, the training environment was probabilistic such that one stimulus was more probable than the others. We show that both humans and the RNN model successfully extract information about stimulus probability and integrate this knowledge into their decisions and task strategy in a new environment. Specifically, performance of both humans and the RNN model varied with the degree to which the stimulus probability of the new environment matched the formed expectation. In both humans and RNNs, this expectation effect was more prominent when the strength of sensory evidence was low. These findings suggest that both humans and our RNN model placed more emphasis on prior expectation (top-down signals) when the available sensory information (bottom-up signals) was limited. Finally, by dissecting the trained RNN model, we demonstrate how competitive inhibition and recurrent excitation form the basis for neural circuitry optimized to perform probabilistic information processing.
Quantifying Adversarial Sensitivity of a Model as a Function of the Image Distribution
Authors: Anonymous
Date: 2020-10-09 00:00:00
Tasks: Adversarial Robustness, Object Recognition, Sensitivity

In this paper, we propose an adaptation to the area under the curve (AUC) metric to measure the adversarial robustness of a model over a particular $\epsilon$-interval $[\epsilon_0, \epsilon_1]$ (interval of adversarial perturbation strengths) that facilitates comparisons across models when they have different initial $\epsilon_0$ performance. This can be used to determine how adversarially sensitive a model is to different image distributions; and/or to measure how robust a model is comparatively to other models for the same distribution. We used this adversarial robustness metric on MNIST, CIFAR-10, and a Fusion dataset (CIFAR-10 + MNIST) where trained models performed either a digit or object recognition task using a LeNet, ResNet50, or a fully connected network (FullyConnectedNet) architecture and found the following: 1) CIFAR-10 models are more adversarially sensitive than MNIST models; 2) Pretraining with another image distribution \textit{sometimes} carries over the adversarial sensitivity induced from the image distribution -- contingent on the pretrained image manifold; 3) Increasing the complexity of the image manifold increases the adversarial sensitivity of a model trained on that image manifold, but also shows that the task plays a role on the sensitivity. Collectively, our results imply non-trivial differences of the learned representation space of one perceptual system over another given its exposure to different image statistics (mainly objects vs digits). Moreover, these results hold even when model systems are equalized to have the same level of performance, or when exposed to matched image statistics of fusion images but with different tasks.
Brain-Inspired Architectures for Efficient and Meaningful Learning from Temporally Smooth Data
Authors: Anonymous
Date: 2020-10-09 00:00:00

How can learning systems exploit the temporal smoothness of real-world training data? We tested the learning of neural networks equipped with two architectural features inspired by the temporal properties of neural circuits. First, because brain dynamics are correlated over time, we implemented a leaky memory mechanism in the hidden representations of neural networks. Second, because cortical circuits can rapidly shift their internal state, “resetting” their local memory, we implemented a gating mechanism that could reset the leaky memory.  How do these architectural features affect learning efficiency and how do they affect the representations that are learned by neural networks? We found that networks equipped with leaky memory and gating could exploit the temporal smoothness in training data, surpassing the performance of conventional feedforward networks. Moreover, networks with multi-scale leaky memory and gating could learn internal representations that “un-mixed” data sources which vary on fast and slow timescales across training samples. Altogether, we showed that brain-inspired architectural mechanisms enabled neural networks to learn more efficiently from temporally smooth data, and to generate internal representations that separate timescales in the training signal.
Hypothesis classes with a unique persistence diagram are NOT nonuniformly learnable
Authors: Nicholas George Bishop, Thomas Davies, Long Tran-Thanh
Date: 2020-10-10 00:00:00
Proceeding: NeurIPS Workshop TDA_and_Beyond 2020 12

*We have since shown that these results are incorrect. Please see PDF for details* Persistence-based summaries are increasingly integrated into deep learning through topological loss functions or regularisers. The implicit role of a topological term in a loss function is to restrict the class of functions in which we are learning (the hypothesis class) to those with a specific topology. Although doing so has had empirical success, to the best of our knowledge there exists no result in the literature that theoretically justifies this restriction. Given a binary classifier in the plane with a Morse-like decision boundary, we prove that the hypothesis class defined by restricting the topology of the possible decision boundaries to those with a unique persistence diagram results in a nonuniformly learnable class of functions. In doing so, we provide a statistical learning theoretic justification for the use of persistence-based summaries in loss functions.
Passive Encrypted IoT Device Fingerprinting with Persistent Homology
Authors: Joe Collins, Michaela Iorga, Dmitry Cousin, David Chapman
Date: 2020-10-10 00:00:00
Proceeding: NeurIPS Workshop TDA_and_Beyond 2020 12
Tasks: Topological Data Analysis

Internet of things (IoT) devices are becoming increasingly prevalent. These devices can improve quality of life,  but often present significant security risks to end users. In this work we present a novel persistent homology based method for the fingerprinting of IoT traffic.  Traditional passive device fingerprinting methods directly inspect the packet attributes or contents within the captured traffic. Buttechniques to fingerprint devices based on inter-packet arrival time (IAT) are an important area of research, as this feature is available even in encrypted traffic.We demonstrate that Topological Data Analysis (TDA) using persistent homology over IAT packet windows is a viable approach to obtain discriminative features for device fingerprinting. The clique complex construction and weighting function we present are efficient to compute and robust to shifts of the packet window. The1-dimensional homology is calculated over the resulting filtered clique complex.We obtain competitive accuracy of 95.34% on the UNSW IoT dataset by using a convolutional neural network to classify over the corresponding persistence images.
Learning a manifold from a teacher’s demonstrations
Authors: Pei Wang, Arash Givchi, Patrick Shafto
Date: 2020-10-10 00:00:00
Proceeding: NeurIPS Workshop TDA_and_Beyond 2020 12

We consider the problem of learning a manifold from a teacher's demonstration. Extending existing approaches of learning from randomly sampled data points, we consider contexts where data may be chosen by a teacher. 
 We analyze learning from teachers who can provide structured data such as individual examples (isolated data points) and demonstrations (sequences of points). Our analysis shows that for the purpose of teaching the topology of a manifold, demonstrations can yield remarkable decreases in the amount of data points required in comparison to teaching with randomly sampled points.
Novel Topological Shapes of Model Interpretability
Authors: Hendrik Jacob van Veen
Date: 2020-10-10 00:00:00
Proceeding: NeurIPS Workshop TDA_and_Beyond 2020 12
Tasks: Interpretable Machine Learning, model

The most accurate models can be the most challenging to interpret. This paper advances interpretability analysis by combining insights from $\texttt{Mapper}$ with recent interpretable machine-learning research. Enforcing new visualization constraints on $\texttt{Mapper}$, we produce a globally - to locally interpretable visualization of the Explainable Boosting Machine. We demonstrate the usefulness of our approach to three data sets: cervical cancer risk, propaganda Tweets, and a loan default data set that was artificially hardened with severe concept drift.
Research Directions to Validate Topological Models of Multi-Dimensional Data
Authors: Nello Blaser, Michael Aupetit
Date: 2020-10-10 00:00:00
Proceeding: NeurIPS Workshop TDA_and_Beyond 2020 12

Various topological models of multi-dimensional data have been proposed for different applications. One of the main issues is to evaluate how correct these models are given the stochastic nature of the data source typical of exploratory data analysis and machine learning settings. We propose research directions to validate the quality of the Mapper and the Generative Simplicial Complex, two models that compute simplicial complexes from multi-dimensional data.
Challenging Euclidean Topological Autoencoders
Authors: Michael Moor, Max Horn, Karsten Borgwardt, Bastian Rieck
Date: 2020-10-10 00:00:00
Proceeding: NeurIPS Workshop TDA_and_Beyond 2020 12
Tasks: Dimensionality Reduction

 Topological autoencoders (TopoAE) have demonstrated their capabilities for performing dimensionality reduction while at the same time preserving topological information of the input space. In its original formulation, this method relies on a Vietoris--Rips filtration of the data space, using the Euclidean metric as the base distance. It is commonly assumed that this distance is not sufficiently powerful to capture salient features of image data sets. We therefore investigate alternative choices of distances in the data space, which are generally considered to be more faithful for image data in comparison to the pixel distance. In our experiments on real-world image datasets, we find that the Euclidean formulation of TopoAE is surprisingly competitive with more elaborate, perceptually-inspired image distances.
Understanding the Power of Persistence Pairing via Permutation Test
Authors: Anonymous
Date: 2020-10-10 00:00:00
Tasks: Graph Classification, Topological Data Analysis

  Recently many efforts have been made to incorporate persistence diagrams, one of major tools in topological data analysis (TDA), into machine learning pipelines. To better understand the power and limitation of persistence diagrams, we carry out a range of experiments on both graph data and shape data, aiming to decouple and inspect the effects of different factors involved. To this end, we also propose the so-called \emph{permutation test}\footnote{The term shall not be confused with the permutation test in the statistical hypothesis testing. } for persistence diagrams to delineate critical values and pairings of critical values. For graph classification tasks, we note that while persistence pairing yields consistent improvement over various benchmark datasets, it appears that for various filtration functions tested, most discriminative power comes from critical values. For shape segmentation and classification, however, we note that persistence pairing shows significant power on most of the benchmark datasets, and improves over both summaries based on merely critical values, and those based on permutation tests.  Our results help provide insights on when persistence diagram based summaries could be more suitable.
Functorial Clustering via Simplicial Complexes
Authors: Dan Shiebler
Date: 2020-10-10 00:00:00
Proceeding: NeurIPS Workshop TDA_and_Beyond 2020 12
Tasks: Clustering

We adapt previous research on topological unsupervised learning to characterize hierarchical overlapping clustering algorithms as functors that factor through a category of simplicial complexes. We first develop a pair of adjoint functors that map between simplicial complexes and the outputs of clustering algorithms. Next, we introduce the maximal and single linkage clustering algorithms as the respective composition of the flagification and connected components functors with McInnes et al's finite singular set functor. We then adapt a theorem by Culbertson et al to demonstrate that all other hierarchical overlapping clustering functors are refined by maximal linkage and refine single linkage.
Circular Coordinate Methods with Generalized Penalty Functions
Authors: Anonymous
Date: 2020-10-10 00:00:00
Tasks: Dimensionality Reduction

The circular coordinate representation performs dimension reduction and visualization for high-dimensional datasets on a torus using persistent cohomology. In this work, we propose a method to adapt the circular coordinate framework to take into account sparsity  in high-dimensional applications. We use a generalized penalty function instead of an $L_{2}$ penalty in the traditional circular coordinate algorithm.
Downsampling and geometric feature methods for EEG classification tasks with CNNs
Authors: Anonymous
Date: 2020-10-10 00:00:00
Tasks: Benchmarking, EEG, Electroencephalogram (EEG), Feature Engineering, Time Series, Time Series Analysis, Time Series Classification

We experimentally investigate a collection of feature engineering pipelines for use with a CNN for classifying electroencephalogram (EEG) time series from the Bonn dataset. We compare $\epsilon$-series of Betti-numbers and $\epsilon$-series of graph spectra (a novel construction)---two topological invariants of a latent geometry of the timeseries---to raw time series of the EEG to fill in a gap in the literature for benchmarking. Additionally, we test these feature pipelines' robustness to downsampling and data reduction. This paper seeks to establish clearer expectations for both time-series classification via geometric features, and how CNNs for time-series respond to data of degraded resolution.
Explainable Framework for Time-series Analysis via Topological Data Analysis
Authors: Anonymous
Date: 2020-10-10 00:00:00
Tasks: Time Series, Time Series Analysis, Topological Data Analysis

We propose an explainable framework for TDA-based time-series analysis, which characterizes time-series signals through time-delay embedding and persistent diagrams. Given the persistence diagram corresponding to a target class, our method continuously deforms an input signal into a signal whose diagram is close to the target diagram. We formulate this problem as a minimization of Wasserstein distance between persistence diagrams. The potential of this method is illustrated on some synthetic and real examples.
Simplicial Neural Networks:First Steps and Future Applications
Authors: Anonymous
Date: 2020-10-10 00:00:00

It is well-known that artificial neural networks are universal approximators. The Universal Approximation Theorem
proves that, given a continuous function on a compact set embedded in an $n$-dimensional space, there exists a one-hidden-layer feed-forward network that approximates the function; however, it does not provide a way of building such a network. In
previous work, the authors presented a constructive approach to tackle this problem for the case of a continuous function on triangulated spaces by connecting the Simplicial Approximation Theorem, a classical result from algebraic topology, and the Universal Approximation Theorem. In this paper, we revisit such a result and propose future applications.
Stability of Mapper Graph Invariants
Authors: Anonymous
Date: 2020-10-10 00:00:00

Mapper summarizes a large dataset with a smaller graph, which is useful for identifying patterns. However, the Mapper graph may vary widely over different datasets drawn from the same real-world data distribution. In order to use Mapper to make confident conclusions, it is important to have a strong intuition about the algorithm's stability under resampling.

In this paper we perform a case study to explore the empirical convergence properties of Mapper. We build bootstrap samples of different sizes from two real-world datasets, Fashion-MNIST and Wikipedia+Gigaword 5, and construct Mapper graphs from these samples. We then explore the relationship between the sample size and the distributions of structural invariants of these Mapper graphs.
Unsupervised local structure learning in elemental zirconium using Topological Data Analysis
Authors: Anonymous
Date: 2020-10-10 00:00:00
Tasks: Topological Data Analysis

We discuss in this extended abstract a new application in materials science where topological data analysis (TDA) has proven to be relevant. TDA is used to analyze local atomic structures during homogeneous nucleation phenomena of pure zirconium in the undercooled liquid namely below its melting temperature. Persistent diagrams (PDs) are considered as descriptors to represent configurations of} the physical system generated by large-scale molecular dynamics (MD) simulations up to one million atoms, and are evaluated by comparison to classical physical descriptors. This work is a first step towards a deeper understanding of the nucleation behavior of the system, where topological machine learning is a challenging but promising perspective.
The Intrinsic Dimension of Images and Its Impact on Learning
Authors: Anonymous
Date: 2020-10-10 00:00:00
Tasks: Adversarial Robustness

It is widely believed that natural image data exhibits low-dimensional structure despite being embedded in a high-dimensional pixel space. This idea underlies a common intuition for the success of deep learning and has been exploited for enhanced regularization and adversarial robustness. In this work, we apply dimension estimation tools to popular datasets and investigate the role of low dimensional structure in neural network learning. We find that common natural image datasets indeed have very low intrinsic dimension relative to the high number of pixels in the images. Additionally, we find that low dimensional datasets are easier for neural networks to learn.  We validate our findings by carefully-designed experiments to vary the intrinsic dimension of both synthetic and real data and evaluate its impact on sample complexity.
Topological Deep Learning
Authors: Anonymous
Date: 2020-10-10 00:00:00
Tasks: Classification, Deep Learning

Topological deep learning is a formalism that is aimed at introducing topological language to deep learning for the purpose of utilizing the minimal mathematical structures to formalize problems that arise in a generic deep learning problem. In this article, we define and study the classification problem in machine learning in a topological setting. Using this topological framework, we show when the classification problem is possible or not possible in the context of neural networks. Finally, we demonstrate how our topological setting immediately illuminates aspects of this problem that are not as readily apparent using traditional tools.
Zero-shot Active Learning with Topological Clustering for Multiclass Classification
Authors: Anonymous
Date: 2020-10-10 00:00:00
Tasks: Active Learning, Clustering, Multi-class Classification

We present a novel approach for zero-shot active learning for multi-class classification based on a clustering technique, called ToMATo, which is guided by topological persistence. Our objective is to identify effective regions in the feature space for label querying. The labeling of examples in these regions will allow the training of efficient multi-class classification prediction functions. We have adapted ToMATo with a density aware $\delta$-Rips graph in order to obtain homogeneous simplicial trees. From these trees, informative simplices are identified with respect to the annotation effort, or the budget. Representative examples from each of them are labeled by an oracle and these labels are then propagated through the trees. We adapt ToMATo by computing our persistence diagram (PD) from a $\delta$-Rips graph that is estimated using a $k$-nearest neighbor distance matrix. This allows the application of the method to large scale scenarios. From this perspective we also propose a local density estimator from the same distance matrix. Comparisons on different benchmarks show that the proposed approach greatly improves performance with respect to a random querying strategy for label assignment that has been found outperforming state-of-the art approaches in previous works.