pwc-archive/papers-with-abstracts huggingface dataset:
On the feasibility of small-data learning in simulation-driven engineering tasks with known mechanisms and effective data representations
Authors: Haosu Zhou, Hamid Reza Attar, Yue Pan, Xuetao Li, Peter R N Childs, Nan Li
Date: 2021-09-24 00:00:00
Proceeding: NeurIPS Workshop AI4Scien 2021 12

The application of machine learning (ML) in scientific tasks is increasing, especially ML in simulation-driven engineering tasks. While previous studies were mostly model-centric and required large-data learning, recent studies start to pay attention to data-centric AI and are investigating small-data learning with effective structured representations, which is significant for industrial application. This article provides a theoretical discussion for the feasibility of small-data learning with structured representations, which is then verified through the surrogate modelling of hot stamping simulations. Future directions are also discussed.
Drug Re-positioning via Text Augmented Knowledge Graph Embeddings
Authors: Mian Zhong, Tiancheng Hu, Ying Jiao, Shehzaad Zuzar Dhuliawala, Bipin Singh
Date: 2021-10-20 00:00:00
Proceeding: NeurIPS Workshop AI4Scien 2021 12
Tasks: Knowledge Graph Embeddings, Knowledge Graphs, Link Prediction, Text Augmentation

Drug re-positioning, modeled as a link prediction problem over medical knowledge graphs (KG), has great potential in finding new usage or targets for approved medicine with relatively low cost. However, the semantic information in medical KGs is rarely utilized, let alone the external medical databases curated by domain experts. This work attempts to integrate textual descriptions of biomedical KG entities in training knowledge graph embeddings (KGE) and evaluates its effectiveness for drug re-positioning. We implement multiple text augmentation methods on TransE as a case study and further apply the best method on other embedding models. Both qualitative and quantitative error analyses with two novel metrics are conducted to shed light on the effects of adding textual information in our model. We conclude that textual information is generally useful, but it may also backfire.
Novel fuzzy approach to Antimicrobial Peptide Activity Prediction: A tale of limited and imbalanced data that models won’t hear
Authors: Aviral Chharia, Rahul Upadhyay, Vinay Kumar
Date: 2021-09-24 00:00:00
Proceeding: NeurIPS Workshop AI4Scien 2021 12
Tasks: Activity Prediction

Antimicrobial peptides have gained immense attention in recent years due to their potential for developing novel antibacterial medicines, next-generation anti-cancer treatment regimes, etc. Owing to the significant cost and time required for wet lab-based AMP screening, researchers have framed the task as an ML problem. However, traditional models rely on the unrealistic premise of large medical data availability to achieve significant performance levels; otherwise, they overfit, decreasing model precision. The collection of such labeled medical data is a challenging and expensive task in itself. The current study is the first to examine models in a real-world setting, training them on restricted and highly imbalanced data. A Fuzzy Intelligence based model is proposed for short (<30 aa) AMP activity prediction, and its ability to learn on limited and severely skewed high-dimensional space mapping is demonstrated over a set of experiments. The proposed model significantly outperforms state-of-the-art ML models trained on the same data. The findings demonstrate the model's efficacy as a potential method for in silico AMP activity prediction.
GraphGT: Machine Learning Datasets for Graph Generation and Transformation
Authors: Yuanqi Du, Shiyu Wang, Xiaojie Guo, Hengning Cao, Shujie Hu, Junji Jiang, Aishwarya Varala, Abhinav Angirekula, Liang Zhao
Date: 2021-09-24 00:00:00
Proceeding: NeurIPS Workshop AI4Scien 2021 12
Tasks: BIG-bench Machine Learning, Drug Design, Graph Generation, Graph Representation Learning, Representation Learning

Graph generation, which learns from known graphs and discovers novel graphs, has great potential in numerous research topics like drug design and mobility synthesis and is one of the fastest-growing domains recently due to its promise for discovering new knowledge. Though many benchmark datasets have emerged in the domain of graph representation learning, the real-world datasets for graph generation problem are much fewer and limited to a small number of areas such as molecules and citation networks. To fill the gap, we introduce GraphGT, a large dataset collection for graph generation problem in machine learning, which contains 36 datasets from 9 domains across 6 subjects. To assist the researchers with better explorations of the datasets, we provide a systemic review and classification of the datasets from various views including research tasks, graph types, and application domains.  In addition, GraphGT provides an easy-to-use graph generation pipeline that simplifies the process for graph data loading, experimental setup, model evaluation. The community can query and access datasets of interest according to a specific domain, task, or type of graph. GraphGT will be regularly updated and welcome inputs from the community. GraphGT is publicly available at \url{https://graphgt.github.io/} and can also be accessed via an open Python library.
Neuroprospecting with DeepRL agents
Authors: Satpreet Harcharan Singh
Date: 2021-09-24 00:00:00
Proceeding: NeurIPS Workshop AI4Scien 2021 12
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

A virtuous cycle between neuroscience and deep reinforcement learning is emerging, and the AI community can do much to enable and accelerate it.
Linear Transformations in Autoencoder Latent Space Predict Time Translations in Active Matter System
Authors: Enrique Amaya, Shahriar Shadkhoo, Dominik Schildknecht, Matt Thomson
Date: 2021-09-24 00:00:00
Proceeding: NeurIPS Workshop AI4Scien 2021 12

Machine Learning (ML) approaches are promising for deriving dynamical predictions of physical systems from data. ML approaches are relevant in active matter, a field that spans scales and studies dynamics of far-from-equilibrium systems where there are significant challenges in predicting macroscopic behavior from microscopic interactions of active particles. A major challenge in applying ML to active systems is encoding a continuous representation of time within a neural network. In this work, we develop a framework for predicting the dynamics of active networks of protein filaments and motors by combining a low-dimensional latent representation inferred through an autoencoder with a linear shift neural network that encodes time translation as a linear transformation within the latent space. Our method enables predicting the contraction and boundary deformations of active networks with various geometries. Although our method is trained to predict 20 time steps into the future, it can generalize to periods of 60 time steps and recapitulate the past 30 frames of a single given observation with less than 10\% error. Finally, we derive an approximate analytic expression for the linear transformation in the latent space that captures the dynamics. Broadly, our study reveals that neural networks are powerful for forecasting the behavior of active matter systems in the complete absence of knowledge of the microscopic dynamics.
Generative Neural Network Based Non-Convex Optimization Using Policy Gradients with an Application to Electromagnetic Design
Authors: Sean Hooten, Sri Krishna Vadlamani, Raymond G. Beausoleil, Thomas Van Vaerenbergh
Date: 2021-09-24 00:00:00
Proceeding: NeurIPS Workshop AI4Scien 2021 12

A generative neural network based non-convex optimization algorithm using a one-step implementation of the policy gradient method is introduced and applied to electromagnetic design. We demonstrate state-of-the-art performance of electromagnetic devices called grating couplers, with key advantages over local gradient-based optimization via the adjoint method.
Discovering Dynamical Parameters by Interpreting Echo State Networks
Authors: Oreoluwa Alao, Peter Y Lu, Marin Soljacic
Date: 2021-09-24 00:00:00
Proceeding: NeurIPS Workshop AI4Scien 2021 12

Reservoir computing architectures known as echo state networks (ESNs) have been shown to have exceptional predictive capabilities when trained on chaotic systems. However, ESN models are often seen as black-box predictors that lack interpretability. We show that the parameters governing the dynamics of a complex nonlinear system can be encoded in the learned readout layer of an ESN. We can extract these dynamical parameters by examining the geometry of the readout layer weights through principal component analysis. We demonstrate this approach by extracting the values of three dynamical parameters ($\sigma$, $\rho$, $\beta$) from a dataset of Lorenz systems where all three parameters are varying among different trajectories. Our proposed method not only demonstrates the interpretability of the ESN readout layer but also provides a computationally inexpensive, unsupervised data-driven approach for identifying uncontrolled variables affecting real-world data from nonlinear dynamical systems.
Improving Hit-finding: Multilabel Neural Architecture with DEL
Authors: Kehang Han, Steven Kearnes, Jin Xu, Wen Torng, JW Feng
Date: 2021-09-24 00:00:00
Proceeding: NeurIPS Workshop AI4Scien 2021 12
Tasks: Drug Discovery


DNA-Encoded Libraries (DEL thereafter) data, often with millions of data points, enables large deep learning models to make real contributions in the drug discovery process (e.g., hit-finding). The current state-of-the-art method of modeling DEL data, GCNN multiclass model, requires domain experts to create mutually exclusive classification labels from multiple selection readouts of DEL data, which is not always an ideal assumption to formulate the problem. In this work, we designed a GCNN multilabel architecture that directly models each selection data to eliminate the corresponding dependency on human expertise. We selected effective choices for key modeling components such as label reduction scheme from in silico evaluation.To assess its performance in real-world drug discovery settings, we further carried out prospective wet-lab testing where the multilabel model shows consistent improvement in hit-rate (percentage of hits in a proposed molecule list) over the current state-of-the-art multiclass model.
Distributed Deep Learning for Persistent Monitoring of agricultural Fields
Authors: Yasaman Esfandiari, Koushik Nagasubramanian, Fateme Fotouhi, Patrick S. Schnable, Baskar Ganapathysubramanian, Soumik Sarkar
Date: 2021-09-24 00:00:00
Proceeding: NeurIPS Workshop AI4Scien 2021 12
Tasks: Anomaly Detection, Deep Learning, Image Retrieval, Management, Retrieval

Distributed deep learning algorithms have shown eminent performance in learning from data that are privately allocated between several agents. Recent advances in sensor technology have enabled the cheap collection of spatial and temporal high-resolution data for agriculture across a wide geographical area. This continuous increase in the amount of data collected has created both the opportunity for, as well as the need to deploy distributed deep learning algorithms for a wide variety of decision support tasks in agriculture. Distributed deep learning algorithms are typically divided into two major categories: centralized vs decentralized learning algorithms, depending on whether a central parameter server exists for gathering information from participating agents. In the case of rural agriculture applications, transferring a large amount of high-resolution data (e.g., images, videos) collected with IoT devices to a central server/cloud could be very expensive especially with limited communication infrastructure. This suggests the need for decentralized learning approaches, which also naturally provide some measure of privacy. Here, autoencoders are trained using a decentralized optimization algorithm to create a latent representation of growing maize plants in a large-scale field experiment involving several hundred cameras deployed in a maize genome diversity growth experiment. We trained the autoencoders for different communication network topologies of the field-deployed cameras. The feature representations from these autoencoders are then utilized to solve downstream tasks such as anomaly detection and image retrieval. Experimental results show that distributed deep learning is effective in learning from large datasets distributed among several learning agents associated with different cameras. Anomaly detection in particular was useful to make course corrections in imaging protocol and identify localized crop management.
Uncovering motif interactions from convolutional-attention networks for genomics
Authors: Rohan Singh Ghotra, Nicholas Keone Lee, Peter K Koo
Date: 2021-09-24 00:00:00
Proceeding: NeurIPS Workshop AI4Scien 2021 12
Tasks: Inductive Bias

A major goal of computational genomics is to understand how sequence patterns, called motifs, interact to regulate gene expression.  In principle, convolution-attention networks (CANs) should provide an inductive bias to infer motif interactions; convolutions can capture motifs while self-attention learns their interactions. However, it is unclear the extent to which this is true in practice. Here we perform an empirical study on synthetic data to test the efficacy of uncovering motif interactions in CANs. We find that irrespective of design choice, interpreting local attention (i.e. on an individual sequence basis) is noisy, leading to many false positive motif interactions. To address this issue, we propose Global Interactions via Filter Activity Correlations (GLIFAC). GLIFAC robustly uncovers motif interactions across a wide spectrum of model choices. This work provides guidance on design choices for CANs that lead to better interpretability for regulatory genomics without sacrificing generalization performance.
Towards trustworthy explanations with gradient-based attribution methods
Authors: Ethan Louis Labelson, Rohit Tripathy, Peter K Koo
Date: 2021-09-24 00:00:00
Proceeding: NeurIPS Workshop AI4Scien 2021 12
Tasks: Feature Importance, Model Selection, scientific discovery

The low interpretability of deep neural networks (DNNs) remains a key barrier to their wide-spread adoption in the sciences. Attribution methods offer a promising solution, providing feature importance scores that serve as first-order model explanations for a given input. In practice, gradient-based attribution methods, such as saliency maps, can yield noisy importance scores depending on model architecture and training procedure. Here we explore how various regularization techniques affect model explanations with saliency maps using synthetic regulatory genomic data, which allows us to quantitatively assess the efficacy of attribution maps. Strikingly, we find that generalization performance does not imply better saliency explanations; though unlike before, we do not observe a clear tradeoff. Interestingly, we find that conventional regularization strategies, when tuned appropriately, can yield high generalization and interpretability performance, similar to what can be achieved with more sophisticated techniques, such as manifold mixup. Our work challenges the conventional knowledge that model selection should be based on test performance; another criterion is needed to sub-select models ideally suited for downstream post hoc interpretability for scientific discovery.
AI as statistical methods for imperfect theories
Authors: Gael Varoquaux
Date: 2021-09-24 00:00:00
Proceeding: NeurIPS Workshop AI4Scien 2021 12
Tasks: BIG-bench Machine Learning

Science has progressed by reasoning on what models could not predict because they were missing important ingredients. And yet without correct models, standard statistical methods for scientific evidence are not sound. Here I argue that machine-learning methodology provides solutions to ground reasoning about empirically evidence more on models’ predictions, and less on their ingredients.
A Fresh Look at De Novo Molecular Design Benchmarks
Authors: Austin Tripp, Gregor N. C. Simm, José Miguel Hernández-Lobato
Date: 2021-09-24 00:00:00
Proceeding: NeurIPS Workshop AI4Scien 2021 12

De novo molecular design is a thriving research area in machine learning (ML) that lacks ubiquitous, high-quality, standardized benchmark tasks. Many existing benchmark tasks do not precisely specify a training dataset or an evaluation budget, which is problematic as they can significantly affect the performance of ML algorithms. This work elucidates the effect of dataset sizes and experimental budgets on established molecular optimization methods through a comprehensive evaluation with 11 selected benchmark tasks. We observe that the dataset size and budget significantly impact all methods' performance and relative ranking, suggesting that a meaningful comparison requires more than a single benchmark setup. Our results also highlight the relative difficulty of benchmarks, implying in particular that logP and QED are poor objectives. We end by offering guidance to researchers on their choice of experiments.
Semi-supervised Graph Neural Network for Particle-level Noise Removal
Authors: Tianchun Li, Shikun Liu, Yongbin Feng, Nhan Tran, Miaoyuan Liu, Pan Li
Date: 2021-09-24 00:00:00
Proceeding: NeurIPS Workshop AI4Scien 2021 12
Tasks: Graph Neural Network

The high instantaneous luminosity of the CERN Large Hadron Collider leads to multiple proton-proton interactions in the same or nearby bunch crossings (pileup). Advanced pileup mitigation algorithms are designed to remove this pileup particle noise and improve the performance of physics observables crucial to the science goals. This study applies the semi-supervised graph neural network to particle-level pileup noise removal, by identifying the particles produced from pileup. The graph neural network is trained on charged particles with well-known labels, which can be obtained from simulation truth information or measurements from data, and inferred on neutral particles of which such labeling is missing. This semi-supervised approach does not depend on the ground truth information from simulation and thus allows us to perform training directly on real data. The performance with this approach is found to be consistently better than widely-used domain algorithms and comparable to a fully supervised training approach. The study serves as the first attempt at applying semi-supervised learning on pileup mitigation, and opens up a new direction of fully data-driven pileup mitigation techniques.
Towards Brain-to-Text Generation: Neural Decoding with Pre-trained Encoder-Decoder Models
Authors: Shuxian Zou, Shaonan Wang, Jiajun Zhang, Chengqing Zong
Date: 2021-09-24 00:00:00
Proceeding: NeurIPS Workshop AI4Scien 2021 12
Tasks: Decoder, Text Generation

Decoding language from non-invasive brain signals is crucial in building widely applicable brain-computer interfaces (BCIs). However, most of the existing studies have focused on discriminating which one in two stimuli corresponds to the given brain image, which is far from directly generating text from neural activities. To move towards this, we first propose two neural decoding tasks with incremental difficulty. The first and simpler task is to predict a word given a brain image and a context, which is the first step towards text generation. And the second and more difficult one is to directly generate text from a given brain image and a prefix. Furthermore, to address the two tasks, we propose a general approach that leverages the powerful pre-trained encoder-decoder model to predict a word or generate a text fragment. Our model achieves 18.20% and 7.95% top-1 accuracy in a vocabulary of more than 2,000 words on average across all participants on the two tasks respectively, significantly outperforming their strong baselines. These results demonstrate the feasibility to directly generate text from neural activities in a non-invasive way. Hopefully, our work can promote practical non-invasive neural language decoders a step further.
AI Methods for Designing Energy-Efficient Buildings in Urban Environments
Authors: Sirine Taleb, Aram Yeretzian, Rabih A. Jabr, Hazem Hajj
Date: 2021-09-24 00:00:00
Proceeding: NeurIPS Workshop AI4Scien 2021 12

Designing energy-efficient buildings is an essential necessity since buildings are responsible for a significant proportion of energy consumption globally.  This concern is even more critical in urban environments where it is harder to understand and model energy use.  Recently, Artificial Intelligence (AI) and Machine Learning (ML) have been explored for improving the energy consumption in buildings.  However, the advances in AI and ML have not been fully exploited in the building design process. This paper aims to highlight the gap between the advancements in AI and its applications for energy-efficient buildings in urban environments. The article discusses opportunities in this direction and suggests future research to have buildings adapt to the ever-changing situations.
Traversing Geodesics to Grow Biological Structures
Authors: Pranav Bhamidipati, Guruprasad Raghavan, Matt Thomson
Date: 2021-09-24 00:00:00
Proceeding: NeurIPS Workshop AI4Scien 2021 12

Biological tissues reliably grow into precise, functional structures from simple starting states during development. Throughout the developmental process, the energy of a tissue changes depending on its natural resistance to deformations such as stretching, bending, shearing, and torsion. In this paper, we represent tissue structures as shapes and develop a mathematical framework to discover paths on the tissue shape manifold to minimize the total energy during development. We find that paths discovered by gradient descent and the geodesic algorithm outperform naive shape interpolation in energetic terms and resemble strategies observed in development. Broadly, these tools can be used to understand and compare shape transformations in biology and propose optimal strategies for synthetic tissue engineering.
$\textit{Ab Initio}$ Discovery of Biological Knowledge from scRNA-Seq Data Using Machine Learning
Authors: Najeebullah Shah, Jiaqi Li, Fanhong Li, Wenchang Chen, Haoxiang Gao, Sijie Chen, Kui Hua, Xuegong Zhang
Date: 2021-09-24 00:00:00
Proceeding: NeurIPS Workshop AI4Scien 2021 12

Expectations of machine learning (ML) are high for discovering new patterns in high-throughput biological data, but most such practices are accustomed to relying on existing knowledge conditions to design experiments. Investigations of the power and limitation of ML in revealing complex patterns from data without the guide of existing knowledge have been lacking. In this study, we conducted systematic experiments on such $\textit{ab initio}$ knowledge discovery with ML methods on single-cell RNA-sequencing data of early embryonic development. Results showed that a strategy combining unsupervised and supervised ML can reveal major cell lineages with minimum involvement of prior knowledge or manual intervention, and the $\textit{ab initio}$ mining enabled a new discovery of human early embryonic cell differentiation. The study illustrated the feasibility, significance, and limitation of $\textit{ab initio}$ ML knowledge discovery on complex biological problems.