bluuebunny/crossref_metadata_2025_split huggingface dataset:
Magneto- and electrophosphene thresholds in the retina: a dosimetry modeling study
Authors: Nissi Janita , Laakso Ilkka 
Published: December 2021

Abstract

Objective. Sensations of flickering light produced by time-varying magnetic fields or electric currents are called magneto- or electrophosphenes. Phosphene thresholds have been used in international guidelines and standards as an estimate of the thresholds of exposure that produce effects in the central nervous system (CNS). However, the estimated threshold values have a large range of uncertainty. Approach. Phosphene thresholds were approximated by simulating five phosphene threshold experiments. Retinal electric fields and currents induced by electric and magnetic stimulation were calculated using the finite element method and 14 anatomically realistic computational models of human heads. Main results. The radial component of retinal current density was determined to be in the range of 6.0–20.6 mA m−2. This study produces more accurate estimates for threshold current density in the retina using detailed anatomical models and the estimates had a reduced range of uncertainty compared to earlier studies. Significance. The results are useful for studying the mechanisms of retinal phosphenes and for the development of exposure limits for the CNS.
Bayesian on-line anticipation of critical transitions
Authors: Heßler Martin , Kamps Oliver 
Published: December 2021

Abstract
The design of reliable indicators to anticipate critical transitions in complex systems is an important task in order to detect imminent regime shifts and to intervene at an early stage to either prevent them or mitigate their consequences. We present a data-driven method based on the estimation of a parameterized nonlinear stochastic differential equation that allows for a robust anticipation of critical transitions even in the presence of strong noise which is a characteristic of many real world systems. Since the parameter estimation is done by a Markov chain Monte Carlo approach, we have access to credibility bands allowing for a better interpretation of the reliability of the results. We also show that the method can yield meaningful results under correlated noise. By introducing a Bayesian linear segment fit it is possible to give an estimate for the time horizon in which the transition will probably occur based on the current state of information. This approach is also able to handle nonlinear time dependencies of the parameter that controls the transition. The method can be used as a tool for on-line analysis to detect changes in the resilience of the system and to provide information on the probability of the occurrence of critical transitions in future. Additionally, it can give valuable information about the possibility of noise induced transitions. The discussed methods are made easily accessible via a flexibly adaptable open source toolkit named ‘antiCPy’ which is implemented in the programming language Python.
Reflective and transmissive cross-polarization converter for terahertz wave in a switchable metamaterial
Authors: Jiang Yuanyuan , Zhang Man , Wang Weihua , Song Zhengyong 
Published: December 2021

Abstract
Utilizing the phase transition characteristic of vanadium dioxide, we present a metamaterial configuration to achieve both reflective and transmissive cross-polarization converters. When vanadium dioxide is metal, the design behaves as a reflective cross-polarization converter. It consists of metallic grating, topas spacer, and vanadium dioxide film. Polarization conversion ratio is more than 90% in the frequency range from 4.80 THz to 13.13 THz. When vanadium dioxide is insulator, the design behaves as a transmissive cross-polarization converter using cascaded metallic gratings with rotation angle 45°. High-efficiency broadband cross-polarization wave conversion is achieved in the frequency band of 0.50–4.75 THz. Effect of oblique incidence is studied on polarization conversion. Results tell that cross-polarization conversion is better when incident angle is in the range of 0°–40°. The designed metamaterial may have a certain inspiration for the research of terahertz multifunctional polarization converter.
First investigation of eclipsing binary KIC 9026766: analysis of light curve and periodic changes
Authors: Soomandar Somaye , Abedi Abbas 
Published: December 2021

Abstract
We investigate a short-period W UMa binary KIC 9026766 with an orbital period of 0.2721278d in the Kepler field of view. By applying an automated q-search for the folded light curve and producing a synthetic light curve for this object based on the PHOEBE code, we calculate the fundamental stellar parameters. We also analyze the O − C curve of the primary minima. The orbital period changes can be attributed to the combination of an upward quadratic function and light-travel time effect (LTTE) due to a possible third body with a minimum mass of 0.029 M
⊙ and an orbital period of 972.5866 ±0.0041d. The relative luminosity of the primary and secondary eclipses (Min I − Min II) is calculated. The periodogram of the residuals of the LTTE and Min I − Min II show peaks with the same period of 0.8566d. The background effect of two nearby stars on our target is the possible reason for this signal. By considering the amplitudes and periods of the remaining signals in the O − C curve of minima, spot motion is possible.
Investigating the energy distribution of the high-energy particles in the Crab nebula
Authors: Wen Lu , Wu Ke-Yao , Yu Huan , Fang Jun 
Published: December 2021

Abstract
The Crab nebula is a prominent pulsar wind nebula detected in multiband observations ranging from radio to very high-energy γ-rays. Recently, γ-rays with energies above 1 PeV have been detected by the Large High Altitude Air Shower Observatory, and the energy of the most energetic particles in the nebula can be constrained. In this paper, we investigate the broadest spectral energy distribution of the Crab nebula and the energy distribution of the electrons emitting the multiwavelength nonthermal emission based on a one-zone time-dependent model. The nebula is powered by the pulsar, and high-energy electrons/positrons with a broken power-law spectrum are continually injected in the nebula as the pulsar spins down. Multiwavelength nonthermal emission is generated by the leptons through synchrotron radiation and inverse Compton scattering. Using appropriate parameters, the detected fluxes for the nebula can be well reproduced, especially for the γ-rays from 102 MeV to 1 PeV. The results show that the detected γ-rays can be produced by the leptons via the inverse Compton scattering, and the lower limit of the Lorentz factor of the most energetic leptons is ∼ 8.5 × 109. It can be concluded that there exist electrons/positrons with energies higher than 4.3 PeV in the Crab nebula.
Active learning of continuous-time Bayesian networks through interventions*
Authors: Linzner Dominik , Koeppl Heinz 
Published: December 2021

Abstract
We consider the problem of learning structures and parameters of continuous-time Bayesian networks (CTBNs) from time-course data under minimal experimental resources. In practice, the cost of generating experimental data poses a bottleneck, especially in the natural and social sciences. A popular approach to overcome this is Bayesian optimal experimental design (BOED). However, BOED becomes infeasible in high-dimensional settings, as it involves integration over all possible experimental outcomes. We propose a novel criterion for experimental design based on a variational approximation of the expected information gain. We show that for CTBNs, a semi-analytical expression for this criterion can be calculated for structure and parameter learning. By doing so, we can replace sampling over experimental outcomes by solving the CTBNs master-equation, for which scalable approximations exist. This alleviates the computational burden of integrating over possible experimental outcomes in high-dimensions. We employ this framework in order to recommend interventional sequences. In this context, we extend the CTBN model to conditional CTBNs in order to incorporate interventions. We demonstrate the performance of our criterion on synthetic and real-world data.
Triple descent and the two kinds of overfitting: where and why do they appear?*
Authors: d’Ascoli Stéphane , Sagun Levent , Biroli Giulio 
Published: December 2021

Abstract
A recent line of research has highlighted the existence of a ‘double descent’ phenomenon in deep learning, whereby increasing the number of training examples N causes the generalization error of neural networks (NNs) to peak when N is of the same order as the number of parameters P. In earlier works, a similar phenomenon was shown to exist in simpler models such as linear regression, where the peak instead occurs when N is equal to the input dimension D. Since both peaks coincide with the interpolation threshold, they are often conflated in the literature. In this paper, we show that despite their apparent similarity, these two scenarios are inherently different. In fact, both peaks can co-exist when NNs are applied to noisy regression tasks. The relative size of the peaks is then governed by the degree of nonlinearity of the activation function. Building on recent developments in the analysis of random feature models, we provide a theoretical ground for this sample-wise triple descent. As shown previously, the nonlinear peak at N = P is a true divergence caused by the extreme sensitivity of the output function to both the noise corrupting the labels and the initialization of the random features (or the weights in NNs). This peak survives in the absence of noise, but can be suppressed by regularization. In contrast, the linear peak at N = D is solely due to overfitting the noise in the labels, and forms earlier during training. We show that this peak is implicitly regularized by the nonlinearity, which is why it only becomes salient at high noise and is weakly affected by explicit regularization. Throughout the paper, we compare analytical results obtained in the random feature model with the outcomes of numerical experiments involving deep NNs.
An analytic theory of shallow networks dynamics for hinge loss classification*
Authors: Pellegrini Franco , Biroli Giulio 
Published: December 2021

Abstract
Neural networks have been shown to perform incredibly well in classification tasks over structured high-dimensional datasets. However, the learning dynamics of such networks is still poorly understood. In this paper we study in detail the training dynamics of a simple type of neural network: a single hidden layer trained to perform a classification task. We show that in a suitable mean-field limit this case maps to a single-node learning problem with a time-dependent dataset determined self-consistently from the average nodes population. We specialize our theory to the prototypical case of a linearly separable data and a linear hinge loss, for which the dynamics can be explicitly solved in the infinite dataset limit. This allows us to address in a simple setting several phenomena appearing in modern networks such as slowing down of training dynamics, crossover between rich and lazy learning, and overfitting. Finally, we assess the limitations of mean-field theory by studying the case of large but finite number of nodes and of training samples.
A random matrix analysis of random Fourier features: beyond the Gaussian kernel, a precise phase transition, and the corresponding double descent*
Authors: Liao Zhenyu , Couillet Romain , Mahoney Michael W 
Published: December 2021

Abstract
This article characterizes the exact asymptotics of random Fourier feature (RFF) regression, in the realistic setting where the number of data samples n, their dimension p, and the dimension of feature space N are all large and comparable. In this regime, the random RFF Gram matrix no longer converges to the well-known limiting Gaussian kernel matrix (as it does when N → ∞ alone), but it still has a tractable behavior that is captured by our analysis. This analysis also provides accurate estimates of training and test regression errors for large n, p, N. Based on these estimates, a precise characterization of two qualitatively different phases of learning, including the phase transition between them, is provided; and the corresponding double descent test error curve is derived from this phase transition behavior. These results do not depend on strong assumptions on the data distribution, and they perfectly match empirical results on real-world data sets.
Reconstruction of pairwise interactions using energy-based models*
Authors: Feinauer Christoph , Lucibello Carlo 
Published: December 2021

Abstract
Pairwise models like the Ising model or the generalized Potts model have found many successful applications in fields like physics, biology, and economics. Closely connected is the problem of inverse statistical mechanics, where the goal is to infer the parameters of such models given observed data. An open problem in this field is the question of how to train these models in the case where the data contain additional higher-order interactions that are not present in the pairwise model. In this work, we propose an approach based on energy-based models and pseudolikelihood maximization to address these complications: we show that hybrid models, which combine a pairwise model and a neural network, can lead to significant improvements in the reconstruction of pairwise interactions. We show these improvements to hold consistently when compared to a standard approach using only the pairwise model and to an approach using only a neural network. This is in line with the general idea that simple interpretable models and complex black-box models are not necessarily a dichotomy: interpolating these two classes of models can allow to keep some advantages of both.
Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classification*
Authors: Mignacco Francesca , Krzakala Florent , Urbani Pierfrancesco , Zdeborová Lenka 
Published: December 2021

Abstract
We analyze in a closed form the learning dynamics of the stochastic gradient descent (SGD) for a single-layer neural network classifying a high-dimensional Gaussian mixture where each cluster is assigned one of two labels. This problem provides a prototype of a non-convex loss landscape with interpolating regimes and a large generalization gap. We define a particular stochastic process for which SGD can be extended to a continuous-time limit that we call stochastic gradient flow. In the full-batch limit, we recover the standard gradient flow. We apply dynamical mean-field theory from statistical physics to track the dynamics of the algorithm in the high-dimensional limit via a self-consistent stochastic process. We explore the performance of the algorithm as a function of the control parameters shedding light on how it navigates the loss landscape.
When do neural networks outperform kernel methods?*
Authors: Ghorbani Behrooz , Mei Song , Misiakiewicz Theodor , Montanari Andrea 
Published: December 2021

Abstract
For a certain scaling of the initialization of stochastic gradient descent (SGD), wide neural networks (NN) have been shown to be well approximated by reproducing kernel Hilbert space (RKHS) methods. Recent empirical work showed that, for some classification tasks, RKHS methods can replace NNs without a large loss in performance. On the other hand, two-layers NNs are known to encode richer smoothness classes than RKHS and we know of special examples for which SGD-trained NN provably outperform RKHS. This is true even in the wide network limit, for a different scaling of the initialization. How can we reconcile the above claims? For which tasks do NNs outperform RKHS? If covariates are nearly isotropic, RKHS methods suffer from the curse of dimensionality, while NNs can overcome it by learning the best low-dimensional representation. Here we show that this curse of dimensionality becomes milder if the covariates display the same low-dimensional structure as the target function, and we precisely characterize this tradeoff. Building on these results, we present the spiked covariates model that can capture in a unified framework both behaviors observed in earlier work. We hypothesize that such a latent low-dimensional structure is present in image classification. We test numerically this hypothesis by showing that specific perturbations of the training distribution degrade the performances of RKHS methods much more significantly than NNs.
Understanding approximate Fisher information for fast convergence of natural gradient descent in wide neural networks*
Authors: Karakida Ryo , Osawa Kazuki 
Published: December 2021

Abstract
Natural gradient descent (NGD) helps to accelerate the convergence of gradient descent dynamics, but it requires approximations in large-scale deep neural networks because of its high computational cost. Empirical studies have confirmed that some NGD methods with approximate Fisher information converge sufficiently fast in practice. Nevertheless, it remains unclear from the theoretical perspective why and under what conditions such heuristic approximations work well. In this work, we reveal that, under specific conditions, NGD with approximate Fisher information achieves the same fast convergence to global minima as exact NGD. We consider deep neural networks in the infinite-width limit, and analyze the asymptotic training dynamics of NGD in function space via the neural tangent kernel. In the function space, the training dynamics with the approximate Fisher information are identical to those with the exact Fisher information, and they converge quickly. The fast convergence holds in layer-wise approximations; for instance, in block diagonal approximation where each block corresponds to a layer as well as in block tri-diagonal and K-FAC approximations. We also find that a unit-wise approximation achieves the same fast convergence under some assumptions. All of these different approximations have an isotropic gradient in the function space, and this plays a fundamental role in achieving the same convergence properties in training. Thus, the current study gives a novel and unified theoretical foundation with which to understand NGD methods in deep learning.
Generalisation error in learning with random features and the hidden manifold model*
Authors: Gerace Federica , Loureiro Bruno , Krzakala Florent , Mézard Marc , Zdeborová Lenka 
Published: December 2021

Abstract
We study generalised linear regression and classification for a synthetically generated dataset encompassing different problems of interest, such as learning with random features, neural networks in the lazy training regime, and the hidden manifold model. We consider the high-dimensional regime and using the replica method from statistical physics, we provide a closed-form expression for the asymptotic generalisation performance in these problems, valid in both the under- and over-parametrised regimes and for a broad choice of generalised linear model loss functions. In particular, we show how to obtain analytically the so-called double descent behaviour for logistic regression with a peak at the interpolation threshold, we illustrate the superiority of orthogonal against random Gaussian projections in learning with random features, and discuss the role played by correlations in the data generated by the hidden manifold model. Beyond the interest in these particular problems, the theoretical formalism introduced in this manuscript provides a path to further extensions to more complex tasks.
Entropic gradient descent algorithms and wide flat minima*
Authors: Pittorino Fabrizio , Lucibello Carlo , Feinauer Christoph , Perugini Gabriele , Baldassi Carlo , Demyanenko Elizaveta , Zecchina Riccardo 
Published: December 2021

Abstract
The properties of flat minima in the empirical risk landscape of neural networks have been debated for some time. Increasing evidence suggests they possess better generalization capabilities with respect to sharp ones. In this work we first discuss the relationship between alternative measures of flatness: the local entropy, which is useful for analysis and algorithm development, and the local energy, which is easier to compute and was shown empirically in extensive tests on state-of-the-art networks to be the best predictor of generalization capabilities. We show semi-analytically in simple controlled scenarios that these two measures correlate strongly with each other and with generalization. Then, we extend the analysis to the deep learning scenario by extensive numerical validations. We study two algorithms, entropy-stochastic gradient descent and replicated-stochastic gradient descent, that explicitly include the local entropy in the optimization objective. We devise a training schedule by which we consistently find flatter minima (using both flatness measures), and improve the generalization error for common architectures (e.g. ResNet, EfficientNet).
On one method of numerical modeling of piezoconductive processes of a two-phase fluid system in a fractured-porous reservoir
Authors: Bobreneva Yu O , Rahimly P I , Poveshchenko Yu A , Podryga V O , Enikeeva L V 
Published: December 2021

Abstract
A method of numerical modeling based on splitting by physical processes of two-phase fluid transfer in a formation with fractured-porous reservoirs is described. Reservoirs of this type have a natural fracture system and are described by the dual porosity model. A four-block mathematical model of the fluid redistribution between a pore-type matrix and a natural fracturing pattern is proposed and studied. The resulting system is complex and entails a number of difficulties associated with a large number of variables and the absence of important properties of a linearized system of equations, such as self-adjointness and symmetry, which are present in the description of piezoconductive processes. The complete splitting by physical processes is carried out to solve this problem. The resulting split model is differentially equivalent to the discrete initial balance equations of the system (conservation of the mass components of the fluids and the total energy of the system), written in divergent form. This approach is associated with a nonlinear approximation of the grid functions in time, which depends on the fraction of the volume occupied by the fluids in the pores, and is easy to implement.
Numerical modeling of multiphase mass transfer processes in fractured-porous reservoirs
Authors: Bobreneva Yu O , Rahimly P I , Poveshchenko Yu A , Podryga V O , Enikeeva L V 
Published: December 2021

Abstract
The paper presents an algorithm for solving the problem of the process of mass transfer of a two-phase fluid in a fractured-porous reservoir in a one-dimensional formulation. The presence of natural fractures in such reservoirs impedes various types of exploration and field development. Fractured-porous reservoirs are characterized by intense exchange fluid flow between fractures and porous blocks. Each system has its own individual set of filtration-capacity parameters, and this fact complicates the problem under consideration. To study the mass transfer of a two-phase fluid in a medium with double porosity, a four-block mathematical model with splitting by physical processes is proposed. The model is described by a system of partial differential equations. The splitting method forms two functional blocks on the water saturation and the piezoconductivity. For the numerical solution of this system, an absolutely stable implicit finite-difference scheme is constructed in the one-dimensional case. On the basis of the proposed difference scheme, pressures and saturations in the fracture system and matrix are obtained.