pwc-archive/papers-with-abstracts huggingface dataset:
Testing Assumptions Underlying a Unified Theory for the Origin of Grid Cells
Authors: Rylan Schaeffer, Mikail Khona, Adrian Bertagnoli, Sanmi Koyejo, Ila Rani Fiete
Date: 2023-11-27 00:00:00

Representing and reasoning about physical space is fundamental to animal survival, and the mammalian lineage expresses a wealth of specialized neural representations that encode space. Grid cells, whose discovery earned a Nobel prize, are a striking example: a grid cell is a neuron that fires if and only if the animal is spatially located at the vertices of a regular triangular lattice that tiles all explored two-dimensional environments. Significant theoretical work has gone into understanding why mammals have learned these particular representations, and recent work has proposed a ``unified theory for the computational and mechanistic origin of grid cells," claiming to answer why the mammalian lineage has learned grid cells. However, the Unified Theory makes a series of highly specific assumptions about the target readouts of grid cells - putatively place cells. In this work, we explicitly identify what these mathematical assumptions are, then test two of the critical assumptions using biological place cell data. At both the population and single-cell levels, we find evidence suggesting that neither of the assumptions are likely true in biological neural representations. These results call the Unified Theory into question, suggesting that biological grid cells likely have a different origin than those obtained in trained artificial neural networks.
CORNN: Convex optimization of recurrent neural networks for rapid inference of neural dynamics
Authors: Fatih Dinc, Adam Shai, Mark Schnitzer, Hidenori Tanaka
Date: 2023-11-16 00:00:00
Proceeding: NeurIPS 2023 11

Advances in optical and electrophysiological recording technologies have made it possible to record the dynamics of thousands of neurons, opening up new possibilities for interpreting and controlling large neural populations in behaving animals. A promising way to extract computational principles from these large datasets is to train data-constrained recurrent neural networks (dRNNs). Performing this training in real-time could open doors for research techniques and medical applications to model and control interventions at single-cell resolution and drive desired forms of animal behavior. However, existing training algorithms for dRNNs are inefficient and have limited scalability, making it a challenge to analyze large neural recordings even in offline scenarios. To address these issues, we introduce a training method termed Convex Optimization of Recurrent Neural Networks (CORNN). In studies of simulated recordings, CORNN attained training speeds ~100-fold faster than traditional optimization approaches while maintaining or enhancing modeling accuracy. We further validated CORNN on simulations with thousands of cells that performed simple computations such as those of a 3-bit flip-flop or the execution of a timed response. Finally, we showed that CORNN can robustly reproduce network dynamics and underlying attractor structures despite mismatches between generator and inference models, severe subsampling of observed neurons, or mismatches in neural time-scales. Overall, by training dRNNs with millions of parameters in subminute processing times on a standard computer, CORNN constitutes a first step towards real-time network reproduction constrained on large-scale neural recordings and a powerful computational tool for advancing the understanding of neural computation.
Learning Time-Invariant Representations for Individual Neurons from Population Dynamics
Authors: Lu Mi, Trung Le, Tianxing He, Eli Shlizerman, Uygar Sümbül
Date: 2023-11-03 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Self-Supervised Learning

Neurons can display highly variable dynamics. While such variability presumably supports the wide range of behaviors generated by the organism, their gene expressions are relatively stable in the adult brain. This suggests that neuronal activity is a combination of its time-invariant identity and the inputs the neuron receives from the rest of the circuit. Here, we propose a self-supervised learning based method to assign time-invariant representations to individual neurons based on permutation-, and population size-invariant summary of population recordings. We fit dynamical models to neuronal activity to learn a representation by considering the activity of both the individual and the neighboring population. Our self-supervised approach and use of implicit representations enable robust inference against imperfections such as partial overlap of neurons across sessions, trial-to-trial variability, and limited availability of molecular (transcriptomic) labels for downstream supervised tasks. We demonstrate our method on a public multimodal dataset of mouse cortical neuronal activity and transcriptomic labels. We report > 35% improvement in predicting the transcriptomic subclass identity and > 20% improvement in predicting class identity with respect to the state-of-the-art.
Latent Exploration for Reinforcement Learning
Authors: Alberto Silvio Chiappa, Alessandro Marin Vargas, Ann Zixiang Huang, Alexander Mathis
Date: 2023-05-31 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: reinforcement-learning, Reinforcement Learning

In Reinforcement Learning, agents learn policies by exploring and interacting with the environment. Due to the curse of dimensionality, learning policies that map high-dimensional sensory input to motor output is particularly challenging. During training, state of the art methods (SAC, PPO, etc.) explore the environment by perturbing the actuation with independent Gaussian noise. While this unstructured exploration has proven successful in numerous tasks, it can be suboptimal for overactuated systems. When multiple actuators, such as motors or muscles, drive behavior, uncorrelated perturbations risk diminishing each other's effect, or modifying the behavior in a task-irrelevant way. While solutions to introduce time correlation across action perturbations exist, introducing correlation across actuators has been largely ignored. Here, we propose LATent TIme-Correlated Exploration (Lattice), a method to inject temporally-correlated noise into the latent state of the policy network, which can be seamlessly integrated with on- and off-policy algorithms. We demonstrate that the noisy actions generated by perturbing the network's activations can be modeled as a multivariate Gaussian distribution with a full covariance matrix. In the PyBullet locomotion tasks, Lattice-SAC achieves state of the art results, and reaches 18% higher reward than unstructured exploration in the Humanoid environment. In the musculoskeletal control environments of MyoSuite, Lattice-PPO achieves higher reward in most reaching and object manipulation tasks, while also finding more energy-efficient policies with reductions of 20-60%. Overall, we demonstrate the effectiveness of structured action noise in time and actuator space for complex motor control tasks. The code is available at: https://github.com/amathislab/lattice.
Data-Driven Network Neuroscience: On Data Collection and Benchmark
Authors: Jiaxing Xu, Yunhan Yang, David Tse Jung Huang, Sophi Shilpa Gururajapathy, Yiping Ke, Miao Qiao, Alan Wang, Haribalan Kumar, Josh McGeown, Eryn Kwon
Date: 2022-11-11 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Functional Connectivity

This paper presents a comprehensive and quality collection of functional human brain network data for potential research in the intersection of neuroscience, machine learning, and graph analytics. Anatomical and functional MRI images have been used to understand the functional connectivity of the human brain and are particularly important in identifying underlying neurodegenerative conditions such as Alzheimer's, Parkinson's, and Autism. Recently, the study of the brain in the form of brain networks using machine learning and graph analytics has become increasingly popular, especially to predict the early onset of these conditions. A brain network, represented as a graph, retains rich structural and positional information that traditional examination methods are unable to capture. However, the lack of publicly accessible brain network data prevents researchers from data-driven explorations. One of the main difficulties lies in the complicated domain-specific preprocessing steps and the exhaustive computation required to convert the data from MRI images into brain networks. We bridge this gap by collecting a large amount of MRI images from public databases and a private source, working with domain experts to make sensible design choices, and preprocessing the MRI images to produce a collection of brain network datasets. The datasets originate from 6 different sources, cover 4 brain conditions, and consist of a total of 2,702 subjects. We test our graph datasets on 12 machine learning models to provide baselines and validate the data quality on a recent graph analysis model. To lower the barrier to entry and promote the research in this interdisciplinary field, we release our brain network data and complete preprocessing details including codes at https://doi.org/10.17608/k6.auckland.21397377 and https://github.com/brainnetuoa/data_driven_network_neuroscience.
Sequential Memory with Temporal Predictive Coding
Authors: Mufeng Tang, Helen Barron, Rafal Bogacz
Date: 2023-05-19 00:00:00
Proceeding: NeurIPS 2023 11

Forming accurate memory of sequential stimuli is a fundamental function of biological agents. However, the computational mechanism underlying sequential memory in the brain remains unclear. Inspired by neuroscience theories and recent successes in applying predictive coding (PC) to \emph{static} memory tasks, in this work we propose a novel PC-based model for \emph{sequential} memory, called \emph{temporal predictive coding} (tPC). We show that our tPC models can memorize and retrieve sequential inputs accurately with a biologically plausible neural implementation. Importantly, our analytical study reveals that tPC can be viewed as a classical Asymmetric Hopfield Network (AHN) with an implicit statistical whitening process, which leads to more stable performance in sequential memory tasks of structured inputs. Moreover, we find that tPC exhibits properties consistent with behavioral observations and theories in neuroscience, thereby strengthening its biological relevance. Our work establishes a possible computational mechanism underlying sequential memory in the brain that can also be theoretically interpreted using existing memory model frameworks.
L-C2ST: Local Diagnostics for Posterior Approximations in Simulation-Based Inference
Authors: Julia Linhart, Alexandre Gramfort, Pedro L. C. Rodrigues
Date: 2023-06-06 00:00:00
Proceeding: NeurIPS 2023 11

Many recent works in simulation-based inference (SBI) rely on deep generative models to approximate complex, high-dimensional posterior distributions. However, evaluating whether or not these approximations can be trusted remains a challenge. Most approaches evaluate the posterior estimator only in expectation over the observation space. This limits their interpretability and is not sufficient to identify for which observations the approximation can be trusted or should be improved. Building upon the well-known classifier two-sample test (C2ST), we introduce L-C2ST, a new method that allows for a local evaluation of the posterior estimator at any given observation. It offers theoretically grounded and easy to interpret -- e.g. graphical -- diagnostics, and unlike C2ST, does not require access to samples from the true posterior. In the case of normalizing flow-based posterior estimators, L-C2ST can be specialized to offer better statistical power, while being computationally more efficient. On standard SBI benchmarks, L-C2ST provides comparable results to C2ST and outperforms alternative local approaches such as coverage tests based on highest predictive density (HPD). We further highlight the importance of local evaluation and the benefit of interpretability of L-C2ST on a challenging application from computational neuroscience.
Neural Crystals
Authors: Sofia Karamintziou, Thanassis Mavropoulos, Dimos Ntioudis, Georgios Meditskos, Stefanos Vrochidis, Ioannis, Kompatsiaris
Date: 2023-09-05 00:00:00

We face up to the challenge of explainability in Multimodal Artificial Intelligence (MMAI). At the nexus of neuroscience-inspired and quantum computing, interpretable and transparent spin-geometrical neural architectures for early fusion of large-scale, heterogeneous, graph-structured data are envisioned, harnessing recent evidence for relativistic quantum neural coding of (co-)behavioral states in the self-organizing brain, under competitive, multidimensional dynamics. The designs draw on a self-dual classical description - via special Clifford-Lipschitz operations - of spinorial quantum states within registers of at most 16 qubits for efficient encoding of exponentially large neural structures. Formally 'trained', Lorentz neural architectures with precisely one lateral layer of exclusively inhibitory interneurons accounting for anti-modalities, as well as their co-architectures with intra-layer connections are highlighted. The approach accommodates the fusion of up to 16 time-invariant interconnected (anti-)modalities and the crystallization of latent multidimensional patterns. Comprehensive insights are expected to be gained through applications to Multimodal Big Data, under diverse real-world scenarios.
Multiallelic Walsh transforms
Authors: Devin Greene
Date: 2023-11-28 00:00:00

A closed formula multiallelic Walsh (or Hadamard) transform is introduced. Basic results are derived, and a statistical interpretation of some of the resulting linear forms is discussed.
Modelling Cellular Perturbations with the Sparse Additive Mechanism Shift Variational Autoencoder
Authors: Michael Bereket, Theofanis Karaletsos
Date: 2023-11-05 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Disentanglement, Drug Discovery, scientific discovery

Generative models of observations under interventions have been a vibrant topic of interest across machine learning and the sciences in recent years. For example, in drug discovery, there is a need to model the effects of diverse interventions on cells in order to characterize unknown biological mechanisms of action. We propose the Sparse Additive Mechanism Shift Variational Autoencoder, SAMS-VAE, to combine compositionality, disentanglement, and interpretability for perturbation models. SAMS-VAE models the latent state of a perturbed sample as the sum of a local latent variable capturing sample-specific variation and sparse global variables of latent intervention effects. Crucially, SAMS-VAE sparsifies these global latent variables for individual perturbations to identify disentangled, perturbation-specific latent subspaces that are flexibly composable. We evaluate SAMS-VAE both quantitatively and qualitatively on a range of tasks using two popular single cell sequencing datasets. In order to measure perturbation-specific model-properties, we also introduce a framework for evaluation of perturbation models based on average treatment effects with links to posterior predictive checks. SAMS-VAE outperforms comparable models in terms of generalization across in-distribution and out-of-distribution tasks, including a combinatorial reasoning task under resource paucity, and yields interpretable latent structures which correlate strongly to known biological mechanisms. Our results suggest SAMS-VAE is an interesting addition to the modeling toolkit for machine learning-driven scientific discovery.
PoET: A generative model of protein families as sequences-of-sequences
Authors: Timothy F. Truong Jr, Tristan Bepler
Date: 2023-06-09 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Language Modelling, Multiple Sequence Alignment, Transfer Learning

Generative protein language models are a natural way to design new proteins with desired functions. However, current models are either difficult to direct to produce a protein from a specific family of interest, or must be trained on a large multiple sequence alignment (MSA) from the specific family of interest, making them unable to benefit from transfer learning across families. To address this, we propose $\textbf{P}$r$\textbf{o}$tein $\textbf{E}$volutionary $\textbf{T}$ransformer (PoET), an autoregressive generative model of whole protein families that learns to generate sets of related proteins as sequences-of-sequences across tens of millions of natural protein sequence clusters. PoET can be used as a retrieval-augmented language model to generate and score arbitrary modifications conditioned on any protein family of interest, and can extrapolate from short context lengths to generalize well even for small families. This is enabled by a unique Transformer layer; we model tokens sequentially within sequences while attending between sequences order invariantly, allowing PoET to scale to context lengths beyond those used during training. In extensive experiments on deep mutational scanning datasets, we show that PoET outperforms existing protein language models and evolutionary sequence models for variant function prediction across proteins of all MSA depths. We also demonstrate PoET's ability to controllably generate new protein sequences.
Evaluating Self-Supervised Learning for Molecular Graph Embeddings
Authors: Hanchen Wang, Jean Kaddour, Shengchao Liu, Jian Tang, Joan Lasenby, Qi Liu
Date: 2022-06-16 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Self-Supervised Learning

Graph Self-Supervised Learning (GSSL) provides a robust pathway for acquiring embeddings without expert labelling, a capability that carries profound implications for molecular graphs due to the staggering number of potential molecules and the high cost of obtaining labels. However, GSSL methods are designed not for optimisation within a specific domain but rather for transferability across a variety of downstream tasks. This broad applicability complicates their evaluation. Addressing this challenge, we present "Molecular Graph Representation Evaluation" (MOLGRAPHEVAL), generating detailed profiles of molecular graph embeddings with interpretable and diversified attributes. MOLGRAPHEVAL offers a suite of probing tasks grouped into three categories: (i) generic graph, (ii) molecular substructure, and (iii) embedding space properties. By leveraging MOLGRAPHEVAL to benchmark existing GSSL methods against both current downstream datasets and our suite of tasks, we uncover significant inconsistencies between inferences drawn solely from existing datasets and those derived from more nuanced probing. These findings suggest that current evaluation methodologies fail to capture the entirety of the landscape.
Infection-responsivity of Commercial Dressings Through Halochromic Drop-casting
Authors: Charles Brooker, Giuseppe Tronci
Date: 2023-11-28 00:00:00
Tasks: Decision Making, Diagnostic

Infection control remains one of the most challenging tasks in wound care, due to growing antimicrobial resistance and ineffective infection diagnostic tools at the point-of-care. To integrate therapeutic wound dressings with wound monitoring capability at the point-of-care to enable informed clinical decision-making, we investigate the encapsulation of a halochromic dye, i.e. bromothymol blue (BTB), onto two commercial dressings, i.e. Aquacel Extra and Promogran, through a simple drop-casting method. Our concept leverages the infection-associated rise in wound pH, on the one hand, and BTB's colour change capability in the pH range of healing (pH: 5-6) and infected wounds (pH > 7), on the other hand. BTB-encapsulated samples show a prompt colour switch (yellow/orange --> blue) following 1-hour incubation at pH 8. The effect of swelling ratio, chemical composition and microstructure is then explored to draw relationships between colour change capability and dressing dye retention.
Machine learning methods for American-style path-dependent contracts
Authors: Matteo Gambara, Giulia Livieri, Andrea Pallavicini
Date: 2023-11-28 00:00:00

In the present work, we introduce and compare state-of-the-art algorithms, that are now classified under the name of machine learning, to price Asian and look-back products with early-termination features. These include randomized feed-forward neural networks, randomized recurrent neural networks, and a novel method based on signatures of the underlying price process. Additionally, we explore potential applications on callable certificates. Furthermore, we present an innovative approach for calculating sensitivities, specifically Delta and Gamma, leveraging Chebyshev interpolation techniques.
Development of a Bankruptcy Prediction Model for the Banking Sector in Mozambique Using Linear Discriminant Analysis
Authors: Reis Castigo Intupo
Date: 2023-11-28 00:00:00
Tasks: Prediction

In Mozambique there is no evidence of a bankruptcy prediction model developed in the national economic context, yet, back in 2016, the national banking sector suffered a financial shock that resulted in Mozambique Central Bank intervention in two banks (Moza Banco, S.A. and Nosso Banco, S.A.). This was a result of the deterioration of their financial and prudential indicators, although Mozambique had been adhering to the Basel Accords since 1994. The Basel Accords provides recommendations on banking sector supervision worldwide with the aim to enhance financial system stability. While it does not predict bankruptcy, the prediction model can be used as an auxiliary tool to manage that risk, but this has to be built in the national economic context. This paper develops for Mozambique banking sector a bankruptcy prediction model in the Mozambican context through the linear discriminant analyses method, following two assumptions: (i) composition of the sample and (ii) robustness of the financial prediction indicators (the capital structure, profitability asset concentration and asset quality) from 2012 to 2020. The developed model attained an accuracy level of 84% one year before Central Bank intervention (2015) with the entire population of 19 banks of the sector, which makes it recommendable as a risk management tool for this sector.
Structure Learning with Adaptive Random Neighborhood Informed MCMC
Authors: Alberto Caron, Xitong Liang, Samuel Livingstone, Jim Griffin
Date: 2023-11-01 00:00:00
Proceeding: NeurIPS 2023 11

In this paper, we introduce a novel MCMC sampler, PARNI-DAG, for a fully-Bayesian approach to the problem of structure learning under observational data. Under the assumption of causal sufficiency, the algorithm allows for approximate sampling directly from the posterior distribution on Directed Acyclic Graphs (DAGs). PARNI-DAG performs efficient sampling of DAGs via locally informed, adaptive random neighborhood proposal that results in better mixing properties. In addition, to ensure better scalability with the number of nodes, we couple PARNI-DAG with a pre-tuning procedure of the sampler's parameters that exploits a skeleton graph derived through some constraint-based or scoring-based algorithms. Thanks to these novel features, PARNI-DAG quickly converges to high-probability regions and is less likely to get stuck in local modes in the presence of high correlation between nodes in high-dimensional settings. After introducing the technical novelties in PARNI-DAG, we empirically demonstrate its mixing efficiency and accuracy in learning DAG structures on a variety of experiments.