nils-herrmann/clean_scirepeval huggingface dataset:
Inference of a Mesoscopic Population Model from Population Spike Trains

Understanding how rich dynamics emerge in neural populations requires models exhibiting a wide range of behaviors while remaining interpretable in terms of connectivity and single-neuron dynamics. However, it has been challenging to fit such mechanistic spiking networks at the single-neuron scale to empirical population data. To close this gap, we propose to fit such data at a mesoscale, using a mechanistic but low-dimensional and, hence, statistically tractable model. The mesoscopic representation is obtained by approximating a population of neurons as multiple homogeneous pools of neurons and modeling the dynamics of the aggregate population activity within each pool. We derive the likelihood of both single-neuron and connectivity parameters given this activity, which can then be used to optimize parameters by gradient ascent on the log likelihood or perform Bayesian inference using Markov chain Monte Carlo (MCMC) sampling. We illustrate this approach using a model of generalized integrate-and-fire neurons for which mesoscopic dynamics have been previously derived and show that both single-neuron and connectivity parameters can be recovered from simulated data. In particular, our inference method extracts posterior correlations between model parameters, which define parameter subsets able to reproduce the data. We compute the Bayesian posterior for combinations of parameters using MCMC sampling and investigate how the approximations inherent in a mesoscopic population model affect the accuracy of the inferred single-neuron parameters.
Verification of Neural Network Behaviour: Formal Guarantees for Power System Applications

This paper presents for the first time, to our knowledge, a framework for verifying neural network behavior in power system applications. Up to this moment, neural networks have been applied in power systems as a black box; this has presented a major barrier for their adoption in practice. Developing a rigorous framework based on mixed-integer linear programming, our methods can determine the range of inputs that neural networks classify as safe or unsafe, and are able to systematically identify adversarial examples. Such methods have the potential to build the missing trust of power system operators on neural networks, and unlock a series of new applications in power systems. This paper presents the framework, methods to assess and improve neural network robustness in power systems, and addresses concerns related to scalability and accuracy. We demonstrate our methods on the IEEE 9-bus, 14-bus, and 162-bus systems, treating both N-1 security and small-signal stability.
Minimax Bounds for Distributed Logistic Regression

We consider a distributed logistic regression problem where labeled data pairs $(X_i,Y_i)\in \mathbb{R}^d\times\{-1,1\}$ for $i=1,\ldots,n$ are distributed across multiple machines in a network and must be communicated to a centralized estimator using at most $k$ bits per labeled pair. We assume that the data $X_i$ come independently from some distribution $P_X$, and that the distribution of $Y_i$ conditioned on $X_i$ follows a logistic model with some parameter $\theta\in\mathbb{R}^d$. By using a Fisher information argument, we give minimax lower bounds for estimating $\theta$ under different assumptions on the tail of the distribution $P_X$. We consider both $\ell^2$ and logistic losses, and show that for the logistic loss our sub-Gaussian lower bound is order-optimal and cannot be improved.
Improving Limited Angle CT Reconstruction with a Robust GAN Prior

Limited angle CT reconstruction is an under-determined linear inverse problem that requires appropriate regularization techniques to be solved. In this work we study how pre-trained generative adversarial networks (GANs) can be used to clean noisy, highly artifact laden reconstructions from conventional techniques, by effectively projecting onto the inferred image manifold. In particular, we use a robust version of the popularly used GAN prior for inverse problems, based on a recent technique called corruption mimicking, that significantly improves the reconstruction quality. The proposed approach operates in the image space directly, as a result of which it does not need to be trained or require access to the measurement model, is scanner agnostic, and can work over a wide range of sensing scenarios.
Self-Supervised Learning for Autonomous Vehicles Perception: A Conciliation Between Analytical and Learning Methods

The interest in autonomous driving has continuously increased in the last two decades. However, to be adopted, such critical systems need to be safe. Concerning the perception of the ego-vehicle environment, the literature has investigated two different types of methods. On the one hand, traditional analytical methods generally rely on handcrafted designs and features while on the other hand, learning methods aim at designing their own appropriate representation of the observed scene.
Constraining Star Formation Histories of Blue Galaxies using the Scatter between Stellar Mass and Halo Mass

We present constraints on the timescale of star formation variability and the correlation between star formation and host halo accretion histories of star-forming (SF) central galaxies from the scatter of the stellar-to-halo mass relation (SHMR). SF galaxies are found to have a tight relationship between their star formation rates and stellar masses on the so-called "star-forming sequence" (SFS), which characterizes their SF histories and $M_*$ growths. Meanwhile, observed SHMR constraints connect $M_*$ growth to halo accretion history. Combining these observed trends with a cosmological $N$-body simulation, we present flexible models that track the SF, $M_*$, and halo accretion histories of SF central galaxies at $z 0.6$ for $t_{\rm duty}=0.1$ Gyr. For $r\sim0.6$, as found in the literature, $t_{\rm duty} 0.5$. Although, the lack of consensus on $\sigma_{M_*|M_h=10^{12}M_\odot}$ at $M_h=10^{12}M_\odot$ and at $z=1$ from observations and galaxy formation models remains the main bottleneck in precisely constraining $r$ and $t_{\rm duty}$, we demonstrate that SHMR can be used to constrain the SF and host halo accretion histories of SF central galaxies.
Refined bounds on MeV-scale thermal dark sectors from BBN and the CMB

New light states thermally coupled to the Standard Model plasma alter the expansion history of the Universe and impact the synthesis of the primordial light elements. In this work, we carry out an exhaustive and precise analysis of the implications of MeV-scale BSM particles in Big Bang Nucleosynthesis (BBN) and for Cosmic Microwave Background (CMB) observations. We find that BBN observations set a lower bound on the thermal dark matter mass of mχ > 0.4 MeV at 2σ. This bound is independent of the spin and number of internal degrees of freedom of the particle, of the annihilation being s-wave or p-wave, and of the annihilation final state. Furthermore, we show that current BBN plus CMB observations constrain purely electrophilic and neutrinophilic BSM species to have a mass, mχ > 3.7 MeV at 2σ. We explore the reach of future BBN measurements and show that upcoming CMB missions should improve the bounds on light BSM thermal states to mχ > (10−15) MeV. Finally, we demonstrate that very light BSM species thermally coupled to the SM plasma are highly disfavoured by current cosmological observations.
On the stability of bimetric structure formation

Bimetric gravity can reproduce the accelerated expansion of the Universe, without a cosmological constant. However, the stability of these solutions to linear perturbations has been questioned, suggesting exponential growth of structure in this approximation. We present a simple model of structure formation, for which an analytical solution is derived. The solution is well-behaved, showing that there is no physical instability with respect to these perturbations. The model can yield a growth of structure exhibiting measurable differences from ΛCDM.
First Star-Forming Structures in Fuzzy Cosmic Filaments.

In hierarchical models of structure formation, the first galaxies form in low-mass dark matter potential wells, probing the behavior of dark matter on kiloparsec scales. Even though these objects are below the detection threshold of current telescopes, future missions will open an observational window into this emergent world. In this Letter, we investigate how the first galaxies are assembled in a "fuzzy" dark matter (FDM) cosmology where dark matter is an ultralight ∼10^{-22}  eV boson and the primordial stars are expected to form along dense dark matter filaments. Using a first-of-its-kind cosmological hydrodynamical simulation, we explore the interplay between baryonic physics and unique wavelike features inherent to FDM. In our simulation, the dark matter filaments show coherent interference patterns on the boson de Broglie scale and develop cylindrical solitonlike cores, which are unstable under gravity and collapse into kiloparsec-scale spherical solitons. Features of the dark matter distribution are largely unaffected by the baryonic feedback. On the contrary, the distributions of gas and stars, which do form along the entire filament, exhibit central cores imprinted by dark matter-a smoking gun signature of FDM.
Harnessing the Power of Infinitely Wide Deep Nets on Small-data Tasks

Recent research shows that the following two models are equivalent: (a) infinitely wide neural networks (NNs) trained under l2 loss by gradient descent with infinitesimally small learning rate (b) kernel regression with respect to so-called Neural Tangent Kernels (NTKs) (Jacot et al., 2018). An efficient algorithm to compute the NTK, as well as its convolutional counterparts, appears in Arora et al. (2019a), which allowed studying performance of infinitely wide nets on datasets like CIFAR-10. However, super-quadratic running time of kernel methods makes them best suited for small-data tasks. We report results suggesting neural tangent kernels perform strongly on low-data tasks. 
1. On a standard testbed of classification/regression tasks from the UCI database, NTK SVM beats the previous gold standard, Random Forests (RF), and also the corresponding finite nets. 
2. On CIFAR-10 with 10 - 640 training samples, Convolutional NTK consistently beats ResNet-34 by 1% - 3%. 
3. On VOC07 testbed for few-shot image classification tasks on ImageNet with transfer learning (Goyal et al., 2019), replacing the linear SVM currently used with a Convolutional NTK SVM consistently improves performance. 
4. Comparing the performance of NTK with the finite-width net it was derived from, NTK behavior starts at lower net widths than suggested by theoretical analysis(Arora et al., 2019a). NTK's efficacy may trace to lower variance of output.
Information based Deep Clustering: An experimental study

Recently, two methods have shown outstanding performance for clustering images and jointly learning the feature representation. The first, called Information Maximiz-ing Self-Augmented Training (IMSAT), maximizes the mutual information between input and clusters while using a regularization term based on virtual adversarial examples. The second, named Invariant Information Clustering (IIC), maximizes the mutual information between the clustering of a sample and its geometrically transformed version. These methods use mutual information in distinct ways and leverage different kinds of transformations. This work proposes a comprehensive analysis of transformation and losses for deep clustering, where we compare numerous combinations of these two components and evaluate how they interact with one another. Results suggest that mutual information between a sample and its transformed representation leads to state-of-the-art performance for deep clustering, especially when used jointly with geometrical and adversarial transformations.
Exploring Generative Physics Models with Scientific Priors in Inertial Confinement Fusion

There is significant interest in using modern neural networks for scientific applications due to their effectiveness in modeling highly complex, non-linear problems in a data-driven fashion. However, a common challenge is to verify the scientific plausibility or validity of outputs predicted by a neural network. This work advocates the use of known scientific constraints as a lens into evaluating, exploring, and understanding such predictions for the problem of inertial confinement fusion.
Pure and Spurious Critical Points: a Geometric Study of Linear Networks

The critical locus of the loss function of a neural network is determined by the geometry of the functional space and by the parameterization of this space by the network's weights. We introduce a natural distinction between pure critical points, which only depend on the functional space, and spurious critical points, which arise from the parameterization. We apply this perspective to revisit and extend the literature on the loss function of linear neural networks. For this type of network, the functional space is either the set of all linear maps from input to output space, or a determinantal variety, i.e., a set of linear maps with bounded rank. We use geometric properties of determinantal varieties to derive new results on the landscape of linear networks with different loss functions and different parameterizations. Our analysis clearly illustrates that the absence of "bad" local minima in the loss landscape of linear networks is due to two distinct phenomena that apply in different settings: it is true for arbitrary smooth convex losses in the case of architectures that can express all linear maps ("filling architectures") but it holds only for the quadratic loss when the functional space is a determinantal variety ("non-filling architectures"). Without any assumption on the architecture, smooth convex losses may lead to landscapes with many bad minima.
Genus One Cobordisms Between Torus Knots

We determine the pairs of torus knots that have a genus one cobordism between them, with one notable exception. This is done by combining obstructions using $\nu^+$ from the Heegaard Floer knot complex and explicit constructions of cobordisms. As an application, we determine the pairs of torus knots related by a single crossing change. Also, we determine the pairs of Thurston-Bennequin number maximizing Legendrian torus knots that have a genus one exact Lagrangian cobordism, with one exception.
Planck-scale-deformed relativistic symmetries and diffeomorphisms in momentum space

We study the implications of a change of coordinatization of momentum space for theories with curved momentum space. We of course find that after a passive diffeomorphism the theory yields the same physical predictions, as one would expect considering that a simple reparametrization should not change physics. However, it appears that general momentum-space covariance (invariance under active diffeomorphisms of momentum space) cannot be enforced, and within a given set of prescriptions on how the theory should encode momentum-space metric and affine connection the physical predictions do depend on the momentum space background. These conclusions find support in some general arguments and in our quantitative analysis of a much-studied toy model with maximally-symmetric (curved) momentum space.
1-point RANSAC for circular motion estimation in computed tomography (CT)

This paper proposes a RANSAC-based algorithm for determining the axial rotation angle of an object from a pair of its tomographic projections. An equation is derived for calculating the rotation angle using one correct keypoints correspondence of two tomographic projections. The proposed algorithm consists of the following steps: keypoints detection and matching, rotation angle estimation for each point correspondence, outliers filtering with the RANSAC algorithm, finally, calculation of the desired angle by minimizing the re-projection error from the remaining correspondences. To validate the proposed method an experimental comparison against methods based on analysis of the distribution of the angles computed from all correspondences is conducted.
Time-Dependent Deep Image Prior for Dynamic MRI

We propose a novel unsupervised deep-learning-based algorithm for dynamic magnetic resonance imaging (MRI) reconstruction. Dynamic MRI requires rapid data acquisition for the study of moving organs such as the heart. We introduce a generalized version of the deep-image-prior approach, which optimizes the weights of a reconstruction network to fit a sequence of sparsely acquired dynamic MRI measurements. Our method needs neither prior training nor additional data. In particular, for cardiac images, it does not require the marking of heartbeats or the reordering of spokes. The key ingredients of our method are threefold: 1) a fixed low-dimensional manifold that encodes the temporal variations of images; 2) a network that maps the manifold into a more expressive latent space; and 3) a convolutional neural network that generates a dynamic series of MRI images from the latent variables and that favors their consistency with the measurements in ${k}$ -space. Our method outperforms the state-of-the-art methods quantitatively and qualitatively in both retrospective and real fetal cardiac datasets. To the best of our knowledge, this is the first unsupervised deep-learning-based method that can reconstruct the continuous variation of dynamic MRI sequences with high spatial resolution.
Spike-based causal inference for weight alignment

In artificial neural networks trained with gradient descent, the weights used for processing stimuli are also used during backward passes to calculate gradients. For the real brain to approximate gradients, gradient information would have to be propagated separately, such that one set of synaptic weights is used for processing and another set is used for backward passes. This produces the so-called "weight transport problem" for biological models of learning, where the backward weights used to calculate gradients need to mirror the forward weights used to process stimuli. This weight transport problem has been considered so hard that popular proposals for biological learning assume that the backward weights are simply random, as in the feedback alignment algorithm. However, such random weights do not appear to work well for large networks. Here we show how the discontinuity introduced in a spiking system can lead to a solution to this problem. The resulting algorithm is a special case of an estimator used for causal inference in econometrics, regression discontinuity design. We show empirically that this algorithm rapidly makes the backward weights approximate the forward weights. As the backward weights become correct, this improves learning performance over feedback alignment on tasks such as Fashion-MNIST, SVHN, CIFAR-10 and VOC. Our results demonstrate that a simple learning rule in a spiking network can allow neurons to produce the right backward connections and thus solve the weight transport problem.
Behavior Mixing with Minimum Global and Subgroup Connectivity Maintenance for Large-Scale Multi-Robot Systems

In many cases the multi-robot systems are desired to execute simultaneously multiple behaviors with different controllers, and sequences of behaviors in real time, which we call behavior mixing. Behavior mixing is accomplished when different subgroups of the overall robot team change their controllers to collectively achieve given tasks while maintaining connectivity within and across subgroups in one connected communication graph. In this paper, we present a provably minimum connectivity maintenance framework to ensure the subgroups and overall robot team stay connected at all times while providing the highest freedom for behavior mixing. In particular, we propose a real-time distributed Minimum Connectivity Constraint Spanning Tree (MCCST) algorithm to select the minimum inter-robot connectivity constraints preserving subgroup and global connectivity that are least likely to be violated by the original controllers. With the employed safety and connectivity barrier certificates for the activated connectivity constraints and collision avoidance, the behavior mixing controllers are thus minimally modified from the original controllers. We demonstrate the effectiveness and scalability of our approach via simulations of up to 100 robots with multiple behaviors.
Fluid Flow Mass Transport for Generative Networks

Generative Adversarial Networks have been shown to be powerful in generating content. To this end, they have been studied intensively in the last few years. Nonetheless, training these networks requires solving a saddle point problem that is difficult to solve and slowly converging. Motivated from techniques in the registration of point clouds and by the fluid flow formulation of mass transport, we investigate a new formulation that is based on strict minimization, without the need for the maximization. The formulation views the problem as a matching problem rather than an adversarial one and thus allows us to quickly converge and obtain meaningful metrics in the optimization path.
Deformed general relativity

In this thesis, I investigate how to construct a self-consistent model of deformed general relativity using canonical methods and metric variables. The specific deformation of general covariance is predicted by some studies into loop quantum cosmology. I firstly find the minimally-deformed model for a scalar-tensor theory, thereby establishing a classical reference point, and investigate the cosmological effects of a non-minimal coupled scalar field. By treating the deformation perturbatively, I derive the deformed gravitational action which includes the nearest order of curvature corrections. Then working more generally, I derive the deformed scalar-tensor constraint to all orders and I find that the momenta and spatial derivatives from gravity and matter must combine in a very specific form. It suggests that the deformation should be equally affected by matter field derivatives as it is by gravitational curvature. Finally, I derive the deformed gravitational action to all orders, and find how intrinsic and extrinsic curvatures differently affect the deformation. The deformation seems to be required to satisfy a non-linear equation usually found in fluid mechanics.
Low-cost LIDAR based Vehicle Pose Estimation and Tracking

Detecting surrounding vehicles by low-cost LIDAR has been drawing enormous attention. In low-cost LIDAR, vehicles present a multi-layer L-Shape. Based on our previous optimization/criteria-based L-Shape fitting algorithm, we here propose a data-driven and model-based method for robust vehicle segmentation and tracking. The new method uses T-linkage RANSAC to take a limited amount of noisy data and performs a robust segmentation for a moving car against noise. Compared with our previous method, T-Linkage RANSAC is more tolerant of observation uncertainties, i.e., the number of sides of the target being observed, and gets rid of the L-Shape assumption. In addition, a vehicle tracking system with Multi-Model Association (MMA) is built upon the segmentation result, which provides smooth trajectories of tracked objects. A manually labeled dataset from low-cost multi-layer LIDARs for validation will also be released with the paper. Experiments on the dataset show that the new approach outperforms previous ones based on multiple criteria. The new algorithm can also run in real-time.