pwc-archive/papers-with-abstracts huggingface dataset:
Antifragility Predicts the Robustness and Evolvability of Biological Networks through Multi-class Classification with a Convolutional Neural Network
Authors: Hyobin Kim, Stalin Muñoz, Pamela Osuna, Carlos Gershenson
Date: 2020-02-04 00:00:00
Tasks: Multi-class Classification

Robustness and evolvability are essential properties to the evolution of biological networks. To determine if a biological network is robust and/or evolvable, it is required to compare its functions before and after mutations. However, this sometimes takes a high computational cost as the network size grows. Here we develop a predictive method to estimate the robustness and evolvability of biological networks without an explicit comparison of functions. We measure antifragility in Boolean network models of biological systems and use this as the predictor. Antifragility occurs when a system benefits from external perturbations. By means of the differences of antifragility between the original and mutated biological networks, we train a convolutional neural network (CNN) and test it to classify the properties of robustness and evolvability. We found that our CNN model successfully classified the properties. Thus, we conclude that our antifragility measure can be used as a predictor of the robustness and evolvability of biological networks.
A Precise High-Dimensional Asymptotic Theory for Boosting and Minimum-$\ell_1$-Norm Interpolated Classifiers
Authors: Tengyuan Liang, Pragya Sur
Date: 2020-02-05 00:00:00

This paper establishes a precise high-dimensional asymptotic theory for boosting on separable data, taking statistical and computational perspectives. We consider a high-dimensional setting where the number of features (weak learners) $p$ scales with the sample size $n$, in an overparametrized regime. Under a class of statistical models, we provide an exact analysis of the generalization error of boosting when the algorithm interpolates the training data and maximizes the empirical $\ell_1$-margin. Further, we explicitly pin down the relation between the boosting test error and the optimal Bayes error, as well as the proportion of active features at interpolation (with zero initialization). In turn, these precise characterizations answer certain questions raised in \cite{breiman1999prediction, schapire1998boosting} surrounding boosting, under assumed data generating processes. At the heart of our theory lies an in-depth study of the maximum-$\ell_1$-margin, which can be accurately described by a new system of non-linear equations; to analyze this margin, we rely on Gaussian comparison techniques and develop a novel uniform deviation argument. Our statistical and computational arguments can handle (1) any finite-rank spiked covariance model for the feature distribution and (2) variants of boosting corresponding to general $\ell_q$-geometry, $q \in [1, 2]$. As a final component, via the Lindeberg principle, we establish a universality result showcasing that the scaled $\ell_1$-margin (asymptotically) remains the same, whether the covariates used for boosting arise from a non-linear random feature model or an appropriately linearized model with matching moments.
Deep Learning Tubes for Tube MPC
Authors: David D. Fan, Ali-akbar Agha-mohammadi, Evangelos A. Theodorou
Date: 2020-02-05 00:00:00
Tasks: Deep Learning, Model-based Reinforcement Learning, Model Predictive Control, quantile regression, Reinforcement Learning

Learning-based control aims to construct models of a system to use for planning or trajectory optimization, e.g. in model-based reinforcement learning. In order to obtain guarantees of safety in this context, uncertainty must be accurately quantified. This uncertainty may come from errors in learning (due to a lack of data, for example), or may be inherent to the system. Propagating uncertainty forward in learned dynamics models is a difficult problem. In this work we use deep learning to obtain expressive and flexible models of how distributions of trajectories behave, which we then use for nonlinear Model Predictive Control (MPC). We introduce a deep quantile regression framework for control that enforces probabilistic quantile bounds and quantifies epistemic uncertainty. Using our method we explore three different approaches for learning tubes that contain the possible trajectories of the system, and demonstrate how to use each of them in a Tube MPC scheme. We prove these schemes are recursively feasible and satisfy constraints with a desired margin of probability. We present experiments in simulation on a nonlinear quadrotor system, demonstrating the practical efficacy of these ideas.
Automated Deep Abstractions for Stochastic Chemical Reaction Networks
Authors: Tatjana Petrov, Denis Repin
Date: 2020-01-30 00:00:00
Tasks: Time Series Analysis

Predicting stochastic cellular dynamics as emerging from the mechanistic models of molecular interactions is a long-standing challenge in systems biology: low-level chemical reaction network (CRN) models give raise to a highly-dimensional continuous-time Markov chain (CTMC) which is computationally demanding and often prohibitive to analyse in practice. A recently proposed abstraction method uses deep learning to replace this CTMC with a discrete-time continuous-space process, by training a mixture density deep neural network with traces sampled at regular time intervals (which can obtained either by simulating a given CRN or as time-series data from experiment). The major advantage of such abstraction is that it produces a computational model that is dramatically cheaper to execute, while preserving the statistical features of the training data. In general, the abstraction accuracy improves with the amount of training data. However, depending on a CRN, the overall quality of the method -- the efficiency gain and abstraction accuracy -- will also depend on the choice of neural network architecture given by hyper-parameters such as the layer types and connections between them. As a consequence, in practice, the modeller would have to take care of finding the suitable architecture manually, for each given CRN, through a tedious and time-consuming trial-and-error cycle. In this paper, we propose to further automatise deep abstractions for stochastic CRNs, through learning the optimal neural network architecture along with learning the transition kernel of the abstract process. Automated search of the architecture makes the method applicable directly to any given CRN, which is time-saving for deep learning experts and crucial for non-specialists. We implement the method and demonstrate its performance on a number of representative CRNs with multi-modal emergent phenotypes.
Dynamic clustering of time series data
Authors: Victhor S. Sartório, Thaís C. O. Fonseca
Date: 2020-01-28 00:00:00
Tasks: Clustering, Clustering Multivariate Time Series, Time Series, Time Series Analysis, Time Series Clustering

We propose a new method for clustering multivariate time-series data based on Dynamic Linear Models. Whereas usual time-series clustering methods obtain static membership parameters, our proposal allows each time-series to dynamically change their cluster memberships over time. In this context, a mixture model is assumed for the time series and a flexible Dirichlet evolution for mixture weights allows for smooth membership changes over time. Posterior estimates and predictions can be obtained through Gibbs sampling, but a more efficient method for obtaining point estimates is presented, based on Stochastic Expectation-Maximization and Gradient Descent. Finally, two applications illustrate the usefulness of our proposed model to model both univariate and multivariate time-series: World Bank indicators for the renewable energy consumption of EU nations and the famous Gapminder dataset containing life-expectancy and GDP per capita for various countries.
A neural network model that learns differences in diagnosis strategies among radiologists has an improved area under the curve for aneurysm status classification in magnetic resonance angiography image series
Authors: Yasuhiko Tachibana, Masataka Nishimori, Naoyuki Kitamura, Kensuke Umehara, Junko Ota, Takayuki Obata, Tatsuya Higashi
Date: 2020-02-03 00:00:00
Tasks: General Classification

Purpose: To construct a neural network model that can learn the different diagnosing strategies of radiologists to better classify aneurysm status in magnetic resonance angiography images. Materials and methods: This retrospective study included 3423 time-of-flight brain magnetic resonance angiography image series (subjects: male 1843 [mean age, 50.2 +/- 11.7 years], female 1580 [50.8 +/- 11.3 years]) recorded from November 2017 through January 2019. The image series were read independently for aneurysm status by one of four board-certified radiologists, who were assisted by an established deep learning-based computer-assisted diagnosis (CAD) system. The constructed neural networks were trained to classify the aneurysm status of zero to five aneurysm-suspicious areas suggested by the CAD system for each image series, and any additional aneurysm areas added by the radiologists, and this classification was compared with the judgment of the annotating radiologist. Image series were randomly allocated to training and testing data in an 8:2 ratio. The accuracy of the classification was compared by receiver operating characteristic analysis between the control model that accepted only image data as input and the proposed model that additionally accepted the information of who the annotating radiologist was. The DeLong test was used to compare areas under the curves (P < 0.05 was considered significant). Results: The area under the curve was larger in the proposed model (0.845) than in the control model (0.793), and the difference was significant (P < 0.0001). Conclusion: The proposed model improved classification accuracy by learning the diagnosis strategies of individual annotating radiologists.
FEA-Net: A Physics-guided Data-driven Model for Efficient Mechanical Response Prediction
Authors: Houpu Yao, Yi Gao, Yongming Liu
Date: 2020-01-31 00:00:00

An innovative physics-guided learning algorithm for predicting the mechanical response of materials and structures is proposed in this paper. The key concept of the proposed study is based on the fact that physics models are governed by Partial Differential Equation (PDE), and its loading/ response mapping can be solved using Finite Element Analysis (FEA). Based on this, a special type of deep convolutional neural network (DCNN) is proposed that takes advantage of our prior knowledge in physics to build data-driven models whose architectures are of physics meaning. This type of network is named as FEA-Net and is used to solve the mechanical response under external loading. Thus, the identification of a mechanical system parameters and the computation of its responses are treated as the learning and inference of FEA-Net, respectively. Case studies on multi-physics (e.g., coupled mechanical-thermal analysis) and multi-phase problems (e.g., composite materials with random micro-structures) are used to demonstrate and verify the theoretical and computational advantages of the proposed method.
Topology optimization of 2D structures with nonlinearities using deep learning
Authors: Diab W. Abueidda, Seid Koric, Nahil A. Sobh
Date: 2020-01-31 00:00:00
Tasks: Cloud Computing, Deep Learning

The field of optimal design of linear elastic structures has seen many exciting successes that resulted in new architected materials and structural designs. With the availability of cloud computing, including high-performance computing, machine learning, and simulation, searching for optimal nonlinear structures is now within reach. In this study, we develop convolutional neural network models to predict optimized designs for a given set of boundary conditions, loads, and optimization constraints. We have considered the case of materials with a linear elastic response with and without stress constraint. Also, we have considered the case of materials with a hyperelastic response, where material and geometric nonlinearities are involved. For the nonlinear elastic case, the neo-Hookean model is utilized. For this purpose, we generate datasets composed of the optimized designs paired with the corresponding boundary conditions, loads, and constraints, using a topology optimization framework to train and validate the neural network models. The developed models are capable of accurately predicting the optimized designs without requiring an iterative scheme and with negligible inference computational time. The suggested pipeline can be generalized to other nonlinear mechanics scenarios and design domains.
Uncertainty Quantification for Bayesian Optimization
Authors: Rui Tuo, Wenjia Wang
Date: 2020-02-04 00:00:00
Tasks: Bayesian Optimization, global-optimization, regression, Uncertainty Quantification

Bayesian optimization is a class of global optimization techniques. In Bayesian optimization, the underlying objective function is modeled as a realization of a Gaussian process. Although the Gaussian process assumption implies a random distribution of the Bayesian optimization outputs, quantification of this uncertainty is rarely studied in the literature. In this work, we propose a novel approach to assess the output uncertainty of Bayesian optimization algorithms, which proceeds by constructing confidence regions of the maximum point (or value) of the objective function. These regions can be computed efficiently, and their confidence levels are guaranteed by the uniform error bounds for sequential Gaussian process regression newly developed in the present work. Our theory provides a unified uncertainty quantification framework for all existing sequential sampling policies and stopping criteria.
Relaxed N-Pairs Loss for Context-Aware Recommendations of Television Content
Authors: Miklas S. Kristoffersen, Sven E. Shepstone, Zheng-Hua Tan
Date: 2020-02-04 00:00:00
Tasks: Metric Learning

This paper studies context-aware recommendations in the television domain by proposing a deep learning-based method for learning joint context-content embeddings (JCCE). The method builds on recent developments within recommendations using latent representations and deep metric learning, in order to effectively represent contextual settings of viewing situations as well as available content in a shared latent space. This embedding space is used for exploring relevant content in various viewing settings by applying an N-pairs loss objective as well as a relaxed variant proposed in this paper. Experiments confirm the recommendation ability of JCCE, achieving improvements when compared to state-of-the-art methods. Further experiments display useful structures in the learned embeddings that can be used for gaining valuable knowledge of underlying variables in the relationship between contextual settings and content properties.
Interpretable & Time-Budget-Constrained Contextualization for Re-Ranking
Authors: Sebastian Hofstätter, Markus Zlabinger, Allan Hanbury
Date: 2020-02-04 00:00:00
Tasks: Re-Ranking, Word Embeddings

Search engines operate under a strict time constraint as a fast response is paramount to user satisfaction. Thus, neural re-ranking models have a limited time-budget to re-rank documents. Given the same amount of time, a faster re-ranking model can incorporate more documents than a less efficient one, leading to a higher effectiveness. To utilize this property, we propose TK (Transformer-Kernel): a neural re-ranking model for ad-hoc search using an efficient contextualization mechanism. TK employs a very small number of Transformer layers (up to three) to contextualize query and document word embeddings. To score individual term interactions, we use a document-length enhanced kernel-pooling, which enables users to gain insight into the model. TK offers an optimal ratio between effectiveness and efficiency: under realistic time constraints (max. 200 ms per query) TK achieves the highest effectiveness in comparison to BERT and other re-ranking models. We demonstrate this on three large-scale ranking collections: MSMARCO-Passage, MSMARCO-Document, and TREC CAR. In addition, to gain insight into TK, we perform a clustered query analysis of TK's results, highlighting its strengths and weaknesses on queries with different types of information need and we show how to interpret the cause of ranking differences of two documents by comparing their internal scores.
Whose Side are Ethics Codes On? Power, Responsibility and the Social Good
Authors: Anne L. Washington, Rachel S. Kuo
Date: 2020-02-04 00:00:00
Tasks: Ethics

The moral authority of ethics codes stems from an assumption that they serve a unified society, yet this ignores the political aspects of any shared resource. The sociologist Howard S. Becker challenged researchers to clarify their power and responsibility in the classic essay: Whose Side Are We On. Building on Becker's hierarchy of credibility, we report on a critical discourse analysis of data ethics codes and emerging conceptualizations of beneficence, or the "social good", of data technology. The analysis revealed that ethics codes from corporations and professional associations conflated consumers with society and were largely silent on agency. Interviews with community organizers about social change in the digital era supplement the analysis, surfacing the limits of technical solutions to concerns of marginalized communities. Given evidence that highlights the gulf between the documents and lived experiences, we argue that ethics codes that elevate consumers may simultaneously subordinate the needs of vulnerable populations. Understanding contested digital resources is central to the emerging field of public interest technology. We introduce the concept of digital differential vulnerability to explain disproportionate exposures to harm within data technology and suggest recommendations for future ethics codes.
Generalizing meanings from partners to populations: Hierarchical inference supports convention formation on networks
Authors: Robert D. Hawkins, Noah D. Goodman, Adele E. Goldberg, Thomas L. Griffiths
Date: 2020-02-04 00:00:00
Tasks: Specificity

A key property of linguistic conventions is that they hold over an entire community of speakers, allowing us to communicate efficiently even with people we have never met before. At the same time, much of our language use is partner-specific: we know that words may be understood differently by different people based on our shared history. This poses a challenge for accounts of convention formation. Exactly how do agents make the inferential leap to community-wide expectations while maintaining partner-specific knowledge? We propose a hierarchical Bayesian model to explain how speakers and listeners solve this inductive problem. To evaluate our model's predictions, we conducted an experiment where participants played an extended natural-language communication game with different partners in a small community. We examine several measures of generalization and find key signatures of both partner-specificity and community convergence that distinguish our model from alternatives. These results suggest that partner-specificity is not only compatible with the formation of community-wide conventions, but may facilitate it when coupled with a powerful inductive mechanism.
TPPO: A Novel Trajectory Predictor with Pseudo Oracle
Authors: Biao Yang, Caizhen He, Pin Wang, Ching-Yao Chan, Xiaofeng Liu, Yang Chen
Date: 2020-02-04 00:00:00
Tasks: Autonomous Driving, Human-Object Interaction Detection

Forecasting pedestrian trajectories in dynamic scenes remains a critical problem in various applications, such as autonomous driving and socially aware robots. Such forecasting is challenging due to human-human and human-object interactions and future uncertainties caused by human randomness. Generative model-based methods handle future uncertainties by sampling a latent variable. However, few studies explored the generation of the latent variable. In this work, we propose the Trajectory Predictor with Pseudo Oracle (TPPO), which is a generative model-based trajectory predictor. The first pseudo oracle is pedestrians' moving directions, and the second one is the latent variable estimated from ground truth trajectories. A social attention module is used to aggregate neighbors' interactions based on the correlation between pedestrians' moving directions and future trajectories. This correlation is inspired by the fact that pedestrians' future trajectories are often influenced by pedestrians in front. A latent variable predictor is proposed to estimate latent variable distributions from observed and ground-truth trajectories. Moreover, the gap between these two distributions is minimized during training. Therefore, the latent variable predictor can estimate the latent variable from observed trajectories to approximate that estimated from ground-truth trajectories. We compare the performance of TPPO with related methods on several public datasets. Results demonstrate that TPPO outperforms state-of-the-art methods with low average and final displacement errors. The ablation study shows that the prediction performance will not dramatically decrease as sampling times decline during tests.
Navigable maps of structural brain networks across species
Authors: Antoine Allard, M. Ángeles Serrano
Date: 2018-01-18 00:00:00
Tasks: Anatomy

Connectomes are spatially embedded networks whose architecture has been shaped by physical constraints and communication needs throughout evolution. Using a decentralized navigation protocol, we investigate the relationship between the structure of the connectomes of different species and their spatial layout. As a navigation strategy, we use greedy routing where nearest neighbors, in terms of geometric distance, are visited. We measure the fraction of successful greedy paths and their length as compared to shortest paths in the topology of connectomes. In Euclidean space, we find a striking difference between the navigability properties of mammalian and non-mammalian species, which implies the inability of Euclidean distances to fully explain the structural organization of their connectomes. In contrast, we find that hyperbolic space, the effective geometry of complex networks, provides almost perfectly navigable maps of connectomes for all species, meaning that hyperbolic distances are exceptionally congruent with the structure of connectomes. Hyperbolic maps therefore offer a quantitative meaningful representation of connectomes that suggests a new cartography of the brain based on the combination of its connectivity with its effective geometry rather than on its anatomy only. Hyperbolic maps also provide a universal framework to study decentralized communication processes in connectomes of different species and at different scales on an equal footing.