pwc-archive/papers-with-abstracts huggingface dataset:
Human Cognition and Language Processing with Neural-Lexicon Hypothesis
Authors: Zang-Hee Cho, Sun-Ha Paek, Young-Bo Kim, Taigyoun Cho, Hyejin Jeong, Haigun Lee
Date: 2022-10-24 00:00:00
Tasks: Deep Learning

Cognition and language seem closely related to the human cognitive process, although they have not been studied and investigated in detail. Our brain is too complex to fully comprehend the structures and connectivity, as well as its functions, with the currently available technology such as electro-encephalography, positron emission tomography, or functional magnetic resonance imaging, and neurobiological data. Therefore, the exploration of neurobiological processes, such as cognition, requires substantially more related evidences, especially from in-vivo human experiments. Cognition and language are of inter-disciplinary nature and additional methodological support is needed from other disciplines, such as deep learning in the field of artificial intelligence, for example. In this paper, we have attempted to explain the neural mechanisms underlying "cognition and language processing" or "cognition or thinking" using a novel neural network model with several newly emerging developments such as neuronal resonance, in-vivo human fiber tractography or connectivity data, Engram and Hebbian hypothesis, human memory formation in the high brain areas, deep learning, and more recently developed neural memory concepts, the neural lexicon. The neural lexicon is developed via language by repeated exposure to the neural system, similar to multilayer signal processing in deep learning. We have derived a neural model to explain how human "cognition and language processing" or "cognition and thinking" works, with a focus on language, a universal medium of the human society. Although the proposed hypothesis is not fully based on experimental evidences, a substantial portion of the observations in this study is directly and indirectly supported by recent experimental findings and the theoretical bases of deep learning research.
Bayesian inference is facilitated by modular neural networks with different time scales
Authors: Kohei Ichikawa, Kunihiko Kaneko
Date: 2022-10-21 00:00:00
Tasks: Bayesian Inference

Various animals, including humans, have been suggested to perform Bayesian inferences to handle noisy, time-varying external information. In performing Bayesian inference, the prior distribution must be shaped by sampling noisy external inputs. However, the mechanism by which neural activities represent such distributions has not yet been elucidated. In this study, we demonstrated that the neural networks with modular structures including fast and slow modules effectively represented the prior distribution in performing accurate Bayesian inferences. Using a recurrent neural network consisting of a main module connected with input and output layers and a sub-module connected only with the main module and having slower neural activity, we demonstrated that the modular network with distinct time scales performed more accurate Bayesian inference compared with the neural networks with uniform time scales. Prior information was represented selectively by the slow sub-module, which could integrate observed signals over an appropriate period and represent input means and variances. Accordingly, the network could effectively predict the time-varying inputs. Furthermore, by training the time scales of neurons starting from networks with uniform time scales and without modular structure, the above slow-fast modular network structure spontaneously emerged as a result of learning wherein prior information was selectively represented in the slower sub-module. These results explain how the prior distribution for Bayesian inference is represented in the brain, provide insight into the relevance of modular structure with time scale hierarchy to information processing, and elucidate the significance of brain areas with slower time scales.
A Missing Hallmark of Cancer: Dysregulation of Differentiation
Authors: Zachary Compton, Kathleen Hanlon, Carolyn C. Compton, Athena Aktipis, Carlo C. Maley
Date: 2022-10-24 00:00:00
Tasks: Prognosis

Cancer cells possess a nearly universal set of characteristics termed the hallmarks of cancer, including replicative immortality and resisting cell death. Dysregulated differentiation is present in virtually all cancers yet has not yet been described as a cancer hallmark. Like other hallmarks, dysregulated differentiation involves a breakdown of the cellular cooperation that typically makes multicellularity possible - in this case disrupting the division of labor among the cells of a body. At the time that the original hallmarks of cancer were described, it was not known that dysregulated differentiation was mechanistically distinct from growth inhibition, but now that this is known, it is a further reason to consider dysregulated differentiation a hallmark of cancer. Dysregulated differentiation also has clinical utility, as it forms the basis of pathological grading, predicts clinical outcomes, and is a viable target for therapies aimed at inducing differentiation. Here we argue that hallmarks of cancer should be near universal, mechanistically distinct, and have clinical utility for prognosis and/or therapy. Dysregulated differentiation meets all of these criteria.
A Comparative Study of Compartmental Models for COVID-19 Transmission in Ontario, Canada
Authors: Yuxuan Zhao, Samuel W. K. Wong
Date: 2022-10-24 00:00:00

The number of confirmed COVID-19 cases reached over 1.3 million in Ontario, Canada by June 4, 2022. The continued spread of the virus underlying COVID-19 has been spurred by the emergence of variants since the initial outbreak in December, 2019. Much attention has thus been devoted to tracking and modelling the transmission of COVID-19. Compartmental models are commonly used to mimic epidemic transmission mechanisms and are easy to understand. Their performance in real-world settings, however, needs to be more thoroughly assessed. In this comparative study, we examine five compartmental models -- four existing ones and an extended model that we propose -- and analyze their ability to describe COVID-19 transmission in Ontario from January 2022 to June 2022.
On the number of genealogical ancestors tracing to the source groups of an admixed population
Authors: Jazlyn A. Mooney, Lily Agranat-Tamir, Jonathan K. Pritchard, Noah A. Rosenberg
Date: 2022-10-22 00:00:00

In genetically admixed populations, admixed individuals possess ancestry from multiple source groups. Studies of human genetic admixture frequently estimate ancestry components corresponding to fractions of individual genomes that trace to specific ancestral populations. However, the same numerical ancestry fraction can represent a wide array of admixture scenarios. Using a mechanistic model of admixture, we characterize admixture genealogically: how many distinct ancestors from the source populations does the admixture represent? We consider African Americans, for whom continent-level estimates produce a 75-85% value for African ancestry on average and 15-25% for European ancestry. Genetic studies together with key features of African-American demographic history suggest ranges for model parameters. Using the model, we infer that if genealogical lineages of a random African American born during 1960-1965 are traced back until they reach members of source populations, the expected number of genealogical lines terminating with African individuals is 314, and the expected number terminating in Europeans is 51. Across discrete generations, the peak number of African genealogical ancestors occurs for birth cohorts from the early 1700s. The probability exceeds 50% that at least one European ancestor was born more recently than 1835. Our genealogical perspective can contribute to further understanding the admixture processes that underlie admixed populations. For African Americans, the results provide insight both on how many of the ancestors of a typical African American might have been forcibly displaced in the Transatlantic Slave Trade and on how many separate European admixture events might exist in a typical African-American genealogy.
What cleaves? Is proteasomal cleavage prediction reaching a ceiling?
Authors: Ingo Ziegler, Bolei Ma, Ercong Nie, Bernd Bischl, David Rügamer, Benjamin Schubert, Emilio Dorigatti
Date: 2022-10-24 00:00:00
Tasks: Benchmarking, Denoising, Prediction

Epitope vaccines are a promising direction to enable precision treatment for cancer, autoimmune diseases, and allergies. Effectively designing such vaccines requires accurate prediction of proteasomal cleavage in order to ensure that the epitopes in the vaccine are presented to T cells by the major histocompatibility complex (MHC). While direct identification of proteasomal cleavage \emph{in vitro} is cumbersome and low throughput, it is possible to implicitly infer cleavage events from the termini of MHC-presented epitopes, which can be detected in large amounts thanks to recent advances in high-throughput MHC ligandomics. Inferring cleavage events in such a way provides an inherently noisy signal which can be tackled with new developments in the field of deep learning that supposedly make it possible to learn predictors from noisy labels. Inspired by such innovations, we sought to modernize proteasomal cleavage predictors by benchmarking a wide range of recent methods, including LSTMs, transformers, CNNs, and denoising methods, on a recently introduced cleavage dataset. We found that increasing model scale and complexity appeared to deliver limited performance gains, as several methods reached about 88.5% AUC on C-terminal and 79.5% AUC on N-terminal cleavage prediction. This suggests that the noise and/or complexity of proteasomal cleavage and the subsequent biological processes of the antigen processing pathway are the major limiting factors for predictive performance rather than the specific modeling approach used. While biological complexity can be tackled by more data and better models, noise and randomness inherently limit the maximum achievable predictive performance. All our datasets and experiments are available at https://github.com/ziegler-ingo/cleavage_prediction.
An Adaptive Strain Estimation Algorithm Using Short Term Cross Correlation Kernels and 1.5D Lateral Search
Authors: Shaiban Ahmed, Rasheed Abid, S. Kaisar Alam
Date: 2022-10-21 00:00:00

Adaptive stretching, where the post compression signal is iteratively stretched to maximize the correlation between the pre and post compression rf echo frames, has demonstrated superior performance compared to gradient based methods. At higher levels of applied strain however, adaptive stretching suffers from decorrelation noise and the image quality deteriorates significantly. Reducing the size of correlation windows have previously showed to enhance the performance in a speckle tracking algorithm but a correlation filter was required to prevent peak hopping errors. In this paper, we present a novel strain estimation algorithm which utilizes an array of overlapping short term cross correlation kernels which are about one-fourth the size of a typical large kernel, to implement an adaptive stretching algorithm. Our method does not require any supplementary correlation filter to prevent false peak errors. Additionally, a lateral search is incorporated using 1.5D algorithm to account for the mechanically induced lateral shift. To validate the efficacy of our proposed method we analyzed the results using simulation and in-vivo data of breast tumors. Our proposed method demonstrated a superior performance compared to classical adaptive stretching algorithm in both qualitative and quantitative assessment. Strain SNRe, CNRe and image resolution are found to improve significantly. Lesion's shape and boundary are more clearly depicted. The results of improvement are clearly evident at higher levels of applied strain.
An Experimental Study of Dimension Reduction Methods on Machine Learning Algorithms with Applications to Psychometrics
Authors: Sean H. Merritt, Alexander P. Christensen
Date: 2022-10-19 00:00:00
Tasks: Dimensionality Reduction, Interpretable Machine Learning

Developing interpretable machine learning models has become an increasingly important issue. One way in which data scientists have been able to develop interpretable models has been to use dimension reduction techniques. In this paper, we examine several dimension reduction techniques including two recent approaches developed in the network psychometrics literature called exploratory graph analysis (EGA) and unique variable analysis (UVA). We compared EGA and UVA with two other dimension reduction techniques common in the machine learning literature (principal component analysis and independent component analysis) as well as no reduction to the variables real data. We show that EGA and UVA perform as well as the other reduction techniques or no reduction. Consistent with previous literature, we show that dimension reduction can decrease, increase, or provide the same accuracy as no reduction of variables. Our tentative results find that dimension reduction tends to lead to better performance when used for classification tasks.
Conditionally Risk-Averse Contextual Bandits
Authors: Mónika Farsang, Paul Mineiro, Wangda Zhang
Date: 2022-10-24 00:00:00
Tasks: Management, Multi-Armed Bandits, regression

Contextual bandits with average-case statistical guarantees are inadequate in risk-averse situations because they might trade off degraded worst-case behaviour for better average performance. Designing a risk-averse contextual bandit is challenging because exploration is necessary but risk-aversion is sensitive to the entire distribution of rewards; nonetheless we exhibit the first risk-averse contextual bandit algorithm with an online regret guarantee. We conduct experiments from diverse scenarios where worst-case outcomes should be avoided, from dynamic pricing, inventory management, and self-tuning software; including a production exascale data processing system.
Sequential Decision Making on Unmatched Data using Bayesian Kernel Embeddings
Authors: Diego Martinez-Taboada, Dino Sejdinovic
Date: 2022-10-25 00:00:00
Tasks: Bayesian Optimization, Decision Making, Sequential Decision Making

The problem of sequentially maximizing the expectation of a function seeks to maximize the expected value of a function of interest without having direct control on its features. Instead, the distribution of such features depends on a given context and an action taken by an agent. In contrast to Bayesian optimization, the arguments of the function are not under agent's control, but are indirectly determined by the agent's action based on a given context. If the information of the features is to be included in the maximization problem, the full conditional distribution of such features, rather than its expectation only, needs to be accounted for. Furthermore, the function is itself unknown, only counting with noisy observations of such function, and potentially requiring the use of unmatched data sets. We propose a novel algorithm for the aforementioned problem which takes into consideration the uncertainty derived from the estimation of both the conditional distribution of the features and the unknown function, by modeling the former as a Bayesian conditional mean embedding and the latter as a Gaussian process. Our algorithm empirically outperforms the current state-of-the-art algorithm in the experiments conducted.
A Spectral Method for Assessing and Combining Multiple Data Visualizations
Authors: Rong Ma, Eric D. Sun, James Zou
Date: 2022-10-25 00:00:00
Tasks: Data Visualization, Dimensionality Reduction

Dimension reduction and data visualization aim to project a high-dimensional dataset to a low-dimensional space while capturing the intrinsic structures in the data. It is an indispensable part of modern data science, and many dimensional reduction and visualization algorithms have been developed. However, different algorithms have their own strengths and weaknesses, making it critically important to evaluate their relative performance for a given dataset, and to leverage and combine their individual strengths. In this paper, we propose an efficient spectral method for assessing and combining multiple visualizations of a given dataset produced by diverse algorithms. The proposed method provides a quantitative measure -- the visualization eigenscore -- of the relative performance of the visualizations for preserving the structure around each data point. Then it leverages the eigenscores to obtain a consensus visualization, which has much improved { quality over the individual visualizations in capturing the underlying true data structure.} Our approach is flexible and works as a wrapper around any visualizations. We analyze multiple simulated and real-world datasets from diverse applications to demonstrate the effectiveness of the eigenscores for evaluating visualizations and the superiority of the proposed consensus visualization. Furthermore, we establish rigorous theoretical justification of our method based on a general statistical framework, yielding fundamental principles behind the empirical success of consensus visualization along with practical guidance.
Analysis of Estimating the Bayes Rule for Gaussian Mixture Models with a Specified Missing-Data Mechanism
Authors: Ziyang Lyu
Date: 2022-10-25 00:00:00
Tasks: Missing Labels

Semi-supervised learning (SSL) approaches have been successfully applied in a wide range of engineering and scientific fields. This paper investigates the generative model framework with a missingness mechanism for unclassified observations, as introduced by Ahfock and McLachlan(2020). We show that in a partially classified sample, a classifier using Bayes rule of allocation with a missing-data mechanism can surpass a fully supervised classifier in a two-class normal homoscedastic model, especially with moderate to low overlap and proportion of missing class labels, or with large overlap but few missing labels. It also outperforms a classifier with no missing-data mechanism regardless of the overlap region or the proportion of missing class labels. Our exploration of two- and three-component normal mixture models with unequal covariances through simulations further corroborates our findings. Finally, we illustrate the use of the proposed classifier with a missing-data mechanism on interneuronal and skin lesion datasets.
Online model error correction with neural networks in the incremental 4D-Var framework
Authors: Alban Farchi, Marcin Chrust, Marc Bocquet, Patrick Laloyaux, Massimo Bonavita
Date: 2022-10-25 00:00:00

Recent studies have demonstrated that it is possible to combine machine learning with data assimilation to reconstruct the dynamics of a physical model partially and imperfectly observed. Data assimilation is used to estimate the system state from the observations, while machine learning computes a surrogate model of the dynamical system based on those estimated states. The surrogate model can be defined as an hybrid combination where a physical model based on prior knowledge is enhanced with a statistical model estimated by a neural network. The training of the neural network is typically done offline, once a large enough dataset of model state estimates is available. By contrast, with online approaches the surrogate model is improved each time a new system state estimate is computed. Online approaches naturally fit the sequential framework encountered in geosciences where new observations become available with time. In a recent methodology paper, we have developed a new weak-constraint 4D-Var formulation which can be used to train a neural network for online model error correction. In the present article, we develop a simplified version of that method, in the incremental 4D-Var framework adopted by most operational weather centres. The simplified method is implemented in the ECMWF Object-Oriented Prediction System, with the help of a newly developed Fortran neural network library, and tested with a two-layer two-dimensional quasi geostrophic model. The results confirm that online learning is effective and yields a more accurate model error correction than offline learning. Finally, the simplified method is compatible with future applications to state-of-the-art models such as the ECMWF Integrated Forecasting System.
Learning Ability of Interpolating Deep Convolutional Neural Networks
Authors: Tian-Yi Zhou, Xiaoming Huo
Date: 2022-10-25 00:00:00

It is frequently observed that overparameterized neural networks generalize well. Regarding such phenomena, existing theoretical work mainly devotes to linear settings or fully-connected neural networks. This paper studies the learning ability of an important family of deep neural networks, deep convolutional neural networks (DCNNs), under both underparameterized and overparameterized settings. We establish the first learning rates of underparameterized DCNNs without parameter or function variable structure restrictions presented in the literature. We also show that by adding well-defined layers to a non-interpolating DCNN, we can obtain some interpolating DCNNs that maintain the good learning rates of the non-interpolating DCNN. This result is achieved by a novel network deepening scheme designed for DCNNs. Our work provides theoretical verification of how overfitted DCNNs generalize well.
Subspace Recovery from Heterogeneous Data with Non-isotropic Noise
Authors: John Duchi, Vitaly Feldman, Lunjia Hu, Kunal Talwar
Date: 2022-10-24 00:00:00
Tasks: Federated Learning

Recovering linear subspaces from data is a fundamental and important task in statistics and machine learning. Motivated by heterogeneity in Federated Learning settings, we study a basic formulation of this problem: the principal component analysis (PCA), with a focus on dealing with irregular noise. Our data come from $n$ users with user $i$ contributing data samples from a $d$-dimensional distribution with mean $\mu_i$. Our goal is to recover the linear subspace shared by $\mu_1,\ldots,\mu_n$ using the data points from all users, where every data point from user $i$ is formed by adding an independent mean-zero noise vector to $\mu_i$. If we only have one data point from every user, subspace recovery is information-theoretically impossible when the covariance matrices of the noise vectors can be non-spherical, necessitating additional restrictive assumptions in previous work. We avoid these assumptions by leveraging at least two data points from each user, which allows us to design an efficiently-computable estimator under non-spherical and user-dependent noise. We prove an upper bound for the estimation error of our estimator in general scenarios where the number of data points and amount of noise can vary across users, and prove an information-theoretic error lower bound that not only matches the upper bound up to a constant factor, but also holds even for spherical Gaussian noise. This implies that our estimator does not introduce additional estimation error (up to a constant factor) due to irregularity in the noise. We show additional results for a linear regression problem in a similar setup.