pwc-archive/papers-with-abstracts huggingface dataset:
A novel multiobjective evolutionary algorithm based on decomposition and multi-reference points strategy
Authors: Wang Chen, Jian Chen, Weitian Wu, Xinmin Yang, Hui Li
Date: 2021-10-27 00:00:00
Tasks: Diversity, Evolutionary Algorithms, Multiobjective Optimization

Many real-world optimization problems such as engineering design can be eventually modeled as the corresponding multiobjective optimization problems (MOPs) which must be solved to obtain approximate Pareto optimal fronts. Multiobjective evolutionary algorithm based on decomposition (MOEA/D) has been regarded as a significantly promising approach for solving MOPs. Recent studies have shown that MOEA/D with uniform weight vectors is well-suited to MOPs with regular Pareto optimal fronts, but its performance in terms of diversity usually deteriorates when solving MOPs with irregular Pareto optimal fronts. In this way, the solution set obtained by the algorithm can not provide more reasonable choices for decision makers. In order to efficiently overcome this drawback, we propose an improved MOEA/D algorithm by virtue of the well-known Pascoletti-Serafini scalarization method and a new strategy of multi-reference points. Specifically, this strategy consists of the setting and adaptation of reference points generated by the techniques of equidistant partition and projection. For performance assessment, the proposed algorithm is compared with existing four state-of-the-art multiobjective evolutionary algorithms on benchmark test problems with various types of Pareto optimal fronts. According to the experimental results, the proposed algorithm exhibits better diversity performance than that of the other compared algorithms. Finally, our algorithm is applied to two real-world MOPs in engineering optimization successfully.
Eigencurve: Optimal Learning Rate Schedule for SGD on Quadratic Objectives with Skewed Hessian Spectrums
Authors: Rui Pan, Haishan Ye, Tong Zhang
Date: 2021-10-27 00:00:00
Proceeding: ICLR 2022 4
Tasks: image-classification, Image Classification

Learning rate schedulers have been widely adopted in training deep neural networks. Despite their practical importance, there is a discrepancy between its practice and its theoretical analysis. For instance, it is not known what schedules of SGD achieve best convergence, even for simple problems such as optimizing quadratic objectives. In this paper, we propose Eigencurve, the first family of learning rate schedules that can achieve minimax optimal convergence rates (up to a constant) for SGD on quadratic objectives when the eigenvalue distribution of the underlying Hessian matrix is skewed. The condition is quite common in practice. Experimental results show that Eigencurve can significantly outperform step decay in image classification tasks on CIFAR-10, especially when the number of epochs is small. Moreover, the theory inspires two simple learning rate schedulers for practical applications that can approximate eigencurve. For some problems, the optimal shape of the proposed schedulers resembles that of cosine decay, which sheds light to the success of cosine decay for such situations. For other situations, the proposed schedulers are superior to cosine decay.
Open Markets and Hybrid Jacobi Processes
Authors: David Itkin, Martin Larsson
Date: 2021-10-26 00:00:00

We propose a unified approach to several problems in Stochastic Portfolio Theory (SPT), which is a framework for equity markets with a large number $d$ of stocks. Our approach combines open markets, where trading is confined to the top $N$ capitalized stocks as well as the market portfolio consisting of all $d$ assets, with a parametric family of models which we call hybrid Jacobi processes. We provide a detailed analysis of ergodicity, particle collisions, and boundary attainment, and use these results to study the associated financial markets. Their properties include (1) stability of the capital distribution curve and (2) unleveraged and explicit growth optimal strategies. The sub-class of rank Jacobi models are additionally shown to (3) serve as the worst-case model for a robust asymptotic growth problem under model ambiguity and (4) exhibit stability in the large-$d$ limit. Our definition of an open market is a relaxation of existing definitions which is essential to make the analysis tractable.
Fast rates for prediction with limited expert advice
Authors: El Mehdi Saad, Gilles Blanchard
Date: 2021-10-27 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Prediction

We investigate the problem of minimizing the excess generalization error with respect to the best expert prediction in a finite family in the stochastic setting, under limited access to information. We assume that the learner only has access to a limited number of expert advices per training round, as well as for prediction. Assuming that the loss function is Lipschitz and strongly convex, we show that if we are allowed to see the advice of only one expert per round for T rounds in the training phase, or to use the advice of only one expert for prediction in the test phase, the worst-case excess risk is $\Omega$(1/ $\sqrt$ T) with probability lower bounded by a constant. However, if we are allowed to see at least two actively chosen expert advices per training round and use at least two experts for prediction, the fast rate O(1/T) can be achieved. We design novel algorithms achieving this rate in this setting, and in the setting where the learner has a budget constraint on the total number of observed expert advices, and give precise instance-dependent bounds on the number of training rounds and queries needed to achieve a given generalization error precision.
The ODE Method for Asymptotic Statistics in Stochastic Approximation and Reinforcement Learning
Authors: Vivek Borkar, Shuhang Chen, Adithya Devraj, Ioannis Kontoyiannis, Sean Meyn
Date: 2021-10-27 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning (RL)

The paper concerns the $d$-dimensional stochastic approximation recursion, $$ \theta_{n+1}= \theta_n + \alpha_{n + 1} f(\theta_n, \Phi_{n+1}) $$ where $ \{ \Phi_n \}$ is a stochastic process on a general state space, satisfying a conditional Markov property that allows for parameter-dependent noise. The main results are established under additional conditions on the mean flow and a version of the Donsker-Varadhan Lyapunov drift condition known as (DV3): (i) An appropriate Lyapunov function is constructed that implies convergence of the estimates in $L_4$. (ii) A functional central limit theorem (CLT) is established, as well as the usual one-dimensional CLT for the normalized error. Moment bounds combined with the CLT imply convergence of the normalized covariance $\textsf{E}[ z_n z_n^T ]$ to the asymptotic covariance in the CLT, where $z_n =: (\theta_n-\theta^*)/\sqrt{\alpha_n}$. (iii) The CLT holds for the normalized version $z^{\text{PR}}_n =: \sqrt{n} [\theta^{\text{PR}}_n -\theta^*]$, of the averaged parameters $\theta^{\text{PR}}_n =:n^{-1} \sum_{k=1}^n\theta_k$, subject to standard assumptions on the step-size. Moreover, the covariance in the CLT coincides with the minimal covariance of Polyak and Ruppert. (iv) An example is given where $f$ and $\bar{f}$ are linear in $\theta$, and $\Phi$ is a geometrically ergodic Markov chain but does not satisfy (DV3). While the algorithm is convergent, the second moment of $\theta_n$ is unbounded and in fact diverges. This arXiv version represents a major extension of the results in prior versions.The main results now allow for parameter-dependent noise, as is often the case in applications to reinforcement learning.
Uniform Concentration Bounds toward a Unified Framework for Robust Clustering
Authors: Debolina Paul, Saptarshi Chakraborty, Swagatam Das, Jason Xu
Date: 2021-10-27 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Clustering

Recent advances in center-based clustering continue to improve upon the drawbacks of Lloyd's celebrated $k$-means algorithm over $60$ years after its introduction. Various methods seek to address poor local minima, sensitivity to outliers, and data that are not well-suited to Euclidean measures of fit, but many are supported largely empirically. Moreover, combining such approaches in a piecemeal manner can result in ad hoc methods, and the limited theoretical results supporting each individual contribution may no longer hold. Toward addressing these issues in a principled way, this paper proposes a cohesive robust framework for center-based clustering under a general class of dissimilarity measures. In particular, we present a rigorous theoretical treatment within a Median-of-Means (MoM) estimation framework, showing that it subsumes several popular $k$-means variants. In addition to unifying existing methods, we derive uniform concentration bounds that complete their analyses, and bridge these results to the MoM framework via Dudley's chaining arguments. Importantly, we neither require any assumptions on the distribution of the outlying observations nor on the relative number of observations $n$ to features $p$. We establish strong consistency and an error rate of $O(n^{-1/2})$ under mild conditions, surpassing the best-known results in the literature. The methods are empirically validated thoroughly on real and synthetic datasets.
ezcox: An R/CRAN Package for Cox Model Batch Processing and Visualization
Authors: Shixiang Wang, Xue-Song Liu, Jianfeng Li, Qi Zhao
Date: 2021-10-27 00:00:00

Cox analysis is a common clinical data analysis technique to link valuable variables to clinical outcomes including dead and relapse. In the omics era, Cox model batch processing is a basic strategy for screening clinically relevant variables, biomarker discovery and gene signature identification. However, all such analyses have been implemented with homebrew code in research community, thus lack of transparency and reproducibility. Here, we present ezcox, the first R/CRAN package for Cox model batch processing and visualization. ezcox is an open source R package under GPL-3 license and it is free available at https://github.com/ShixiangWang/ezcox and https://cran.r-project.org/package=ezcox.
BioGrad: Biologically Plausible Gradient-Based Learning for Spiking Neural Networks
Authors: Guangzhi Tang, Neelesh Kumar, Ioannis Polykretis, Konstantinos P. Michmizos
Date: 2021-10-27 00:00:00
Tasks: GPU

Spiking neural networks (SNN) are delivering energy-efficient, massively parallel, and low-latency solutions to AI problems, facilitated by the emerging neuromorphic chips. To harness these computational benefits, SNN need to be trained by learning algorithms that adhere to brain-inspired neuromorphic principles, namely event-based, local, and online computations. Yet, the state-of-the-art SNN training algorithms are based on backprop that does not follow the above principles. Due to its limited biological plausibility, the application of backprop to SNN requires non-local feedback pathways for transmitting continuous-valued errors, and relies on gradients from future timesteps. The introduction of biologically plausible modifications to backprop has helped overcome several of its limitations, but limits the degree to which backprop is approximated, which hinders its performance. We propose a biologically plausible gradient-based learning algorithm for SNN that is functionally equivalent to backprop, while adhering to all three neuromorphic principles. We introduced multi-compartment spiking neurons with local eligibility traces to compute the gradients required for learning, and a periodic "sleep" phase to further improve the approximation to backprop during which a local Hebbian rule aligns the feedback and feedforward weights. Our method achieved the same level of performance as backprop with multi-layer fully connected SNN on MNIST (98.13%) and the event-based N-MNIST (97.59%) datasets. We deployed our learning algorithm on Intel's Loihi to train a 1-hidden-layer network for MNIST, and obtained 93.32% test accuracy while consuming 400 times less energy per training sample than BioGrad on GPU. Our work shows that optimal learning is feasible in neuromorphic computing, and further pursuing its biological plausibility can better capture the benefits of this emerging computing paradigm.
Biological learning in key-value memory networks
Authors: Danil Tyulmankov, Ching Fang, Annapurna Vadaparty, Guangyu Robert Yang
Date: 2021-10-26 00:00:00
Proceeding: NeurIPS 2021 12

In neuroscience, classical Hopfield networks are the standard biologically plausible model of long-term memory, relying on Hebbian plasticity for storage and attractor dynamics for recall. In contrast, memory-augmented neural networks in machine learning commonly use a key-value mechanism to store and read out memories in a single step. Such augmented networks achieve impressive feats of memory compared to traditional variants, yet their biological relevance is unclear. We propose an implementation of basic key-value memory that stores inputs using a combination of biologically plausible three-factor plasticity rules. The same rules are recovered when network parameters are meta-learned. Our network performs on par with classical Hopfield networks on autoassociative memory tasks and can be naturally extended to continual recall, heteroassociative memory, and sequence learning. Our results suggest a compelling alternative to the classical Hopfield network as a model of biological long-term memory.
Modeling Category-Selective Cortical Regions with Topographic Variational Autoencoders
Authors: T. Anderson Keller, Qinghe Gao, Max Welling
Date: 2021-10-25 00:00:00
Proceeding: NeurIPS Workshop SVRHM 2021 12

Category-selectivity in the brain describes the observation that certain spatially localized areas of the cerebral cortex tend to respond robustly and selectively to stimuli from specific limited categories. One of the most well known examples of category-selectivity is the Fusiform Face Area (FFA), an area of the inferior temporal cortex in primates which responds preferentially to images of faces when compared with objects or other generic stimuli. In this work, we leverage the newly introduced Topographic Variational Autoencoder to model the emergence of such localized category-selectivity in an unsupervised manner. Experimentally, we demonstrate our model yields spatially dense neural clusters selective to faces, bodies, and places through visualized maps of Cohen's d metric. We compare our model with related supervised approaches, namely the Topographic Deep Artificial Neural Network (TDANN) of Lee et al., and discuss both theoretical and empirical similarities. Finally, we show preliminary results suggesting that our model yields a nested spatial hierarchy of increasingly abstract categories, analogous to observations from the human ventral temporal cortex.
Quantifying the rationality of rhythmic signals
Authors: A. Guillet, A. Arneodo, P. Argoul, F. Argoul
Date: 2021-10-25 00:00:00

Rhythms and vibrations represent the quintessence of life, they are ubiquitous (systemic) in all living systems. Recognising, unfolding these rhythms is paramount in medicine, for example in the physiology of the heart, lung, hearing, speech, brain, the cellular and molecular processes involved in biological clocks. The importance of the commensurability of the frequencies in different rhythms has been thoroughly studied in music. We define a log-frequency correlation measure on spectral densities that gives the temporal evolution of the distribution of frequency ratios (rational or irrational) in between two signals, using analytic wavelets. We illustrate these concepts on numerical signals (sums of sine functions) and voice recordings from the Voice-Icar-Federico II database. Finally, with a second correlation operation from two of these ratio distributions (a reference one, the other from the voices) we introduce another quantity that we call \emph{sonance}, measuring the ``harmony'' (rationality) of two voices sung together as a function of a pitch transposition.
A logical and topological proof of the irreducibility of consciousness to physical data
Authors: Iegor Reznikoff
Date: 2021-10-16 00:00:00
Tasks: Philosophy

We show here that what we call visual space of consciousness, the space of what we see, is a specific space different from the purely physical one and that its properties imply that it cannot be reduced to or deduced from physical laws. Some biological points are also briefly considered. The arguments are of logical, mathematical and physical character, and although elementary they require a careful reading. A first shorter version of this paper appeared in a hardly accessible Journal [1], and presented at the International Congress of Logic, Methodology and Philosophy of Sciences, in Beijing, August 2007. There is no need to define consciousness; we only observe some of its properties, namely geometric and topological properties of visual consciousness, and show that these properties cannot be based on physics only. Now, if a part of consciousness cannot be grounded on physics only, it is the same for consciousness as a whole and we speak of the irreducibility of consciousness to physical data. We do not consider philosophical questions or issues; in a simple physical and mathematical frame we give a logical proof of this irreducibility. Elements for a formal mathematical, logical proof are mentioned at the end of the paper.
Hodge-Laplacian of Brain Networks
Authors: D. Vijay Anand, Moo K. Chung
Date: 2021-10-15 00:00:00

The closed loops or cycles in a brain network embeds higher order signal transmission paths, which provide fundamental insights into the functioning of the brain. In this work, we propose an efficient algorithm for systematic identification and modeling of cycles using persistent homology and the Hodge Laplacian. Various statistical inference procedures on cycles are developed. We validate the our methods on simulations and apply to brain networks obtained through the resting state functional magnetic resonance imaging. The computer codes for the Hodge Laplacian are given in https://github.com/laplcebeltrami/hodge.
Diachronic Text Mining Investigation of Therapeutic Candidates for COVID-19
Authors: James Powell, Kari Sentz
Date: 2021-10-26 00:00:00
Tasks: Blocking

Diachronic text mining has frequently been applied to long-term linguistic surveys of word meaning and usage shifts over time. In this paper we apply short-term diachronic text mining to a rapidly growing corpus of scientific publications on COVID-19 captured in the CORD-19 dataset in order to identify co-occurrences and analyze the behavior of potential candidate treatments. We used a data set associated with a COVID-19 drug re-purposing study from Oak Ridge National Laboratory. This study identified existing candidate coronavirus treatments, including drugs and approved compounds, which had been analyzed and ranked according to their potential for blocking the ability of the SARS-COV-2 virus to invade human cells. We investigated the occurrence of these candidates in temporal instances of the CORD-19 corpus. We found that at least 25% of the identified terms occurred in temporal instances of the corpus to the extent that their frequency and contextual dynamics could be evaluated. We identified three classes of behaviors: those where frequency and contextual shifts were small and positively correlated; those where there was no correlation between frequency and contextual changes; and those where there was a negative correlation between frequency and contextual shift. We speculate that the latter two patterns are indicative that a target candidate therapeutics is undergoing active evaluation. The patterns we detected demonstrate the potential benefits of using diachronic text mining techniques with a large dynamic text corpus to track drug-repurposing activities across international clinical and laboratory settings.
Open microscopy in the life sciences: Quo Vadis?
Authors: Johannes Hohlbein, Benedict Diederich, Barbora Marsikova, Emmanuel G. Reynaud, Seamus Holden, Wiebke Jahr, Robert Haase, Kirti Prakash
Date: 2021-10-26 00:00:00

Light microscopy allows observing cellular features and objects with sub-micrometer resolution. As such, light microscopy has been playing a fundamental role in the life sciences for more than a hundred years. Fueled by the availability of mass-produced electronics and hardware, publicly shared documentation and building instructions, open-source software, wide access to rapid prototyping and 3D printing, and the enthusiasm of contributors and users involved, the concept of open microscopy has been gaining incredible momentum, bringing new sophisticated tools to an expanding user base. Here, we will first discuss the ideas behind open science and open microscopy before highlighting recent projects and developments in open microscopy. We argue that the availability of well-designed open hardware and software solutions targeting broad user groups or even non-experts, will increasingly be relevant to cope with the increasing complexity of cutting-edge imaging technologies. We will then extensively discuss the current and future challenges of open microscopy.
Towards a Theory of Evolution as Multilevel Learning
Authors: Vitaly Vanchurin, Yuri I. Wolf, Mikhail I. Katsnelson, Eugene V. Koonin
Date: 2021-10-27 00:00:00

We apply the theory of learning to physically renormalizable systems in an attempt to develop a theory of biological evolution, including the origin of life, as multilevel learning. We formulate seven fundamental principles of evolution that appear to be necessary and sufficient to render a universe observable and show that they entail the major features of biological evolution, including replication and natural selection. These principles also follow naturally from the theory of learning. We formulate the theory of evolution using the mathematical framework of neural networks, which provides for detailed analysis of evolutionary phenomena. To demonstrate the potential of the proposed theoretical framework, we derive a generalized version of the Central Dogma of molecular biology by analyzing the flow of information during learning (back-propagation) and predicting (forward-propagation) the environment by evolving organisms. The more complex evolutionary phenomena, such as major transitions in evolution, in particular, the origin of life, have to be analyzed in the thermodynamic limit, which is described in detail in the accompanying paper.
Modelling of spatial infection spread through heterogeneous population: from lattice to PDE models
Authors: A. Vaziry, T. Kolokolnikov, P. G. Kevrekidis
Date: 2021-10-26 00:00:00

We present a simple model for the spread of an infection that incorporates spatial variability in population density. Starting from first principle considerations, we explore how a novel PDE with state-dependent diffusion can be obtained. This model exhibits higher infection rates in the areas of higher population density, a feature that we argue to be consistent with epidemiological observations. The model also exhibits an infection wave whose speed varies with population density. In addition, we demonstrate the possibility that an infection can ``jump'' (i.e., tunnel) across areas of low population density towards the areas of high population density. We briefly touch upon the data reported for coronavirus spread in the Canadian province of Nova Scotia as a case example with a number of qualitatively similar features as our model. Lastly, we propose a number of generalizations of the model towards future studies.