Miracle-dz/UniFAR-FTUs huggingface dataset:
Don't Just Blame Over-parametrization for Over-confidence: Theoretical Analysis of Calibration in Binary Classification

Modern machine learning models with high accuracy are often miscalibrated -- the predicted top probability does not reflect the actual accuracy, and tends to be over-confident. It is commonly believed that such over-confidence is mainly due to over-parametrization, in particular when the model is large enough to memorize the training data and maximize the confidence. In this paper, we show theoretically that over-parametrization is not the only reason for over-confidence. We prove that logistic regression is inherently over-confident, in the realizable, under-parametrized setting where the data is generated from the logistic model, and the sample size is much larger than the number of parameters. Further, this over-confidence happens for general well-specified binary classification problems as long as the activation is symmetric and concave on the positive part. Perhaps surprisingly, we also show that over-confidence is not always the case -- there exists another activation function (and a suitable loss function) under which the learned classifier is under-confident at some probability values. Overall, our theory provides a precise characterization of calibration in realizable binary classification, which we verify on simulations and real data experiments.
From n-layer planar ordering to the monolayer homeotropic structure of confined hard rods: The effect of shape anisotropy and wall-to-wall separation.

Using the Parsons-Lee theory, we examined the effect of shape anisotropy and the wall-to-wall separation (H) on the phase behavior of the hard parallelepiped rods with dimensions L, D, and D (L>D) in such narrow slitlike pores which only one homeotropic layer can form. The phase structures, including biaxiality, planar nematic layering transition as well as planar to homeotropic, were studied for some separations in the range 2.5D≤H≤10.0D for H-D≤L
Local Hyper-flow Diffusion

Recently, hypergraphs have attracted a lot of attention due to their ability to capture complex relations among entities. The insurgence of hypergraphs has resulted in data of increasing size and complexity that exhibit interesting small-scale and local structure, e.g., small-scale communities and localized node-ranking around a given set of seed nodes. Popular and principled ways to capture the local structure are the local hypergraph clustering problem and related seed set expansion problem. In this work, we propose the first local diffusion method that achieves edge-size-independent Cheeger-type guarantee for the problem of local hypergraph clustering while applying to a rich class of higher-order relations that covers many previously studied special cases. Our method is based on a primal-dual optimization formulation where the primal problem has a natural network flow interpretation, and the dual problem has a cut-based interpretation using the $\ell_2$-norm penalty on associated cut-costs. We demonstrate the new technique is significantly better than state-of-the-art methods on both synthetic and real-world data.
Polar Codes for Automorphism Ensemble Decoding

In this paper we deal with polar code automorphisms that are beneficial under low-latency automorphism ensemble (AE) decoding, and we propose polar code designs that have such automorphisms. Successive-cancellation (SC) decoding and thus SC-based AE decoding are invariant with respect to the only known polar code automorphisms, namely those of the lower-triangular affine (LTA) group. To overcome this problem, we provide methods to determine whether a given polar code has non-LTA automorphisms and to identify such automorphisms. Building on this, we design specific polar codes that admit automorphisms in the upper-diagonal linear (UTL) group, and thus render SC-based AE decoding effective. Demonstrated by examples, these new polar codes under AE decoding outperform conventional polar codes under SC list decoding in terms of error rate, while keeping the latency comparable to SC decoding. Moreover, state-of-the-art BP-based permutation decoding for polar codes is beaten by BP-based AE thanks to this design.
Beyond Horndeski interactions induced by quantum effects

In this paper, an opportunity to generate beyond Horndeski interactions is addressed. An amplitude generating a certain beyond Horndeski coupling is explicitly found. The amplitude is free from ultraviolet divergences, so it is protected from ultraviolet contributions and can be considered as a universal prediction of effective field theory.
Offline Model-Based Optimization via Normalized Maximum Likelihood Estimation

In this work we consider data-driven optimization problems where one must maximize a function given only queries at a fixed set of points. This problem setting emerges in many domains where function evaluation is a complex and expensive process, such as in the design of materials, vehicles, or neural network architectures. Because the available data typically only covers a small manifold of the possible space of inputs, a principal challenge is to be able to construct algorithms that can reason about uncertainty and out-of-distribution values, since a naive optimizer can easily exploit an estimated model to return adversarial inputs. We propose to tackle this problem by leveraging the normalized maximum-likelihood (NML) estimator, which provides a principled approach to handling uncertainty and out-of-distribution inputs. While in the standard formulation NML is intractable, we propose a tractable approximation that allows us to scale our method to high-capacity neural network models. We demonstrate that our method can effectively optimize high-dimensional design problems in a variety of disciplines such as chemistry, biology, and materials engineering.
Comparison of cosmic string and superstring models to NANOGrav 12.5-year results

We compare the spectrum of the stochastic gravitational wave background produced in several models of cosmic strings with the common-spectrum process recently reported by NANOGrav. We discuss theoretical uncertainties in computing such a background, and show that despite such uncertainties, cosmic strings remain a good explanation for the potential signal, but the consequences for cosmic string parameters depend on the model. Superstrings could also explain the signal, but only in a restricted parameter space where their network behavior is eﬀectively identical to that of ordinary cosmic strings.
Integrating Floor Plans into Hedonic Models for Rent Price Appraisal

Online real estate platforms have become significant marketplaces facilitating users’ search for an apartment or a house. Yet it remains challenging to accurately appraise a property’s value. Prior works have primarily studied real estate valuation based on hedonic price models that take structured data into account while accompanying unstructured data is typically ignored. In this study, we investigate to what extent an automated visual analysis of apartment floor plans on online real estate platforms can enhance hedonic rent price appraisal. We propose a tailored two-staged deep learning approach to learn price-relevant designs of floor plans from historical price data. Subsequently, we integrate the floor plan predictions into hedonic rent price models that account for both structural and locational characteristics of an apartment. Our empirical analysis based on a unique dataset of 9,174 real estate listings suggests that current hedonic models underutilize the available data. We find that (1) the visual design of floor plans has significant explanatory power regarding rent prices – even after controlling for structural and locational apartment characteristics, and (2) harnessing floor plans results in an up to 10.56 % lower out-of-sample prediction error. We further find that floor plans yield a particularly high gain in prediction performance for older and smaller apartments. Altogether, our empirical findings contribute to the existing research body by establishing the link between the visual design of floor plans and real estate prices. Moreover, our approach has important implications for online real estate platforms, which can use our findings to enhance user experience in their real estate listings.
Dynamic Virtual Graph Significance Networks for Predicting Influenza

Graph-structured data and their related algorithms have attracted significant attention in many fields, such as influenza prediction in public health. However, the variable influenza seasonality, occasional pandemics, and domain knowledge pose great challenges to construct an appropriate graph, which could impair the strength of the current popular graph-based algorithms to perform data analysis. In this study, we develop a novel method, Dynamic Virtual Graph Significance Networks (DVGSN), which can supervisedly and dynamically learn from similar"infection situations"in historical timepoints. Representation learning on the dynamic virtual graph can tackle the varied seasonality and pandemics, and therefore improve the performance. The extensive experiments on real-world influenza data demonstrate that DVGSN significantly outperforms the current state-of-the-art methods. To the best of our knowledge, this is the first attempt to supervisedly learn a dynamic virtual graph for time-series prediction tasks. Moreover, the proposed method needs less domain knowledge to build a graph in advance and has rich interpretability, which makes the method more acceptable in the fields of public health, life sciences, and so on.
AlphaNet: Improved Training of Supernets with Alpha-Divergence

Weight-sharing neural architecture search (NAS) is an effective technique for automating efficient neural architecture design. Weight-sharing NAS builds a supernet that assembles all the architectures as its sub-networks and jointly trains the supernet with the sub-networks. The success of weight-sharing NAS heavily relies on distilling the knowledge of the supernet to the sub-networks. However, we find that the widely used distillation divergence, i.e., KL divergence, may lead to student sub-networks that over-estimate or under-estimate the uncertainty of the teacher supernet, leading to inferior performance of the sub-networks. In this work, we propose to improve the supernet training with a more generalized alpha-divergence. By adaptively selecting the alpha-divergence, we simultaneously prevent the over-estimation or under-estimation of the uncertainty of the teacher model. We apply the proposed alpha-divergence based supernets training to both slimmable neural networks and weight-sharing NAS, and demonstrate significant improvements. Specifically, our discovered model family, AlphaNet, outperforms prior-art models on a wide range of FLOPs regimes, including BigNAS, Once-for-All networks, and AttentiveNAS. We achieve ImageNet top-1 accuracy of 80.0% with only 444M FLOPs. Our code and pretrained models are available at https://github.com/facebookresearch/AlphaNet.
The Yin-Yang dataset

The Yin-Yang dataset was developed for research on biologically plausible error backpropagation and deep learning in spiking neural networks. It serves as an alternative to classic deep learning datasets, especially in early-stage prototyping scenarios for both network models and hardware platforms, for which it provides several advantages. First, it is smaller and therefore faster to learn, thereby being better suited for small-scale exploratory studies in both software simulations and hardware prototypes. Second, it exhibits a very clear gap between the accuracies achievable using shallow as compared to deep neural networks. Third, it is easily transferable between spatial and temporal input domains, making it interesting for different types of classification scenarios.
Distance to healthy cardiovascular dynamics from fetal heart rate scale-dependent features in pregnant sheep model of human labor predicts cardiovascular decompensation

The overarching goal of the present work is to contribute to the understanding of the relations between fetal heart rate (FHR) temporal dynamics and the wellbeing of the fetus, notably in terms of predicting cardiovascular decompensation (CVD). It makes uses of an established animal model of human labor, where fourteen near-term ovine fetuses subjected to umbilical cord occlusions (UCO) were instrumented to permit regular intermittent measurements of metabolites, pH, and continuous recording of electrocardiogram (ECG) and systemic arterial blood pressure (to identify CVD) during UCO. ECG-derived FHR was digitized at the sampling rate of 1000 Hz and resampled to 4Hz, as used in clinical routine. We focused on four FHR variability features which are tunable to temporal scales of FHR dynamics, robustly computable from FHR sampled at 4Hz and within short-time sliding windows, hence permitting a time-dependent, or local, analysis of FHR which helps dealing with signal noise. Results show the sensitivity of the proposed features for early detection of CVD, correlation to metabolites and pH, useful for early acidosis detection and the importance of coarse time scales (2.5 to 8 seconds) which are not disturbed by the low FHR sampling rate. Further, we introduce the performance of an individualized selfreferencing metric of the distance to healthy state, based on a combination of the four features. We demonstrate that this novel metric, applied to clinically available FHR temporal dynamics alone, accurately predicts the time occurrence of CVD which heralds a clinically significant degradation of the fetal health reserve to tolerate the trial of labor.
Momentum Residual Neural Networks

The training of deep residual neural networks (ResNets) with backpropagation has a memory cost that increases linearly with respect to the depth of the network. A way to circumvent this issue is to use reversible architectures. In this paper, we propose to change the forward rule of a ResNet by adding a momentum term. The resulting networks, momentum residual neural networks (Momentum ResNets), are invertible. Unlike previous invertible architectures, they can be used as a drop-in replacement for any existing ResNet block. We show that Momentum ResNets can be interpreted in the infinitesimal step size regime as second-order ordinary differential equations (ODEs) and exactly characterize how adding momentum progressively increases the representation capabilities of Momentum ResNets. Our analysis reveals that Momentum ResNets can learn any linear mapping up to a multiplicative factor, while ResNets cannot. In a learning to optimize setting, where convergence to a fixed point is required, we show theoretically and empirically that our method succeeds while existing invertible architectures fail. We show on CIFAR and ImageNet that Momentum ResNets have the same accuracy as ResNets, while having a much smaller memory footprint, and show that pre-trained Momentum ResNets are promising for fine-tuning models.
Exterior Point Method for Completely Positive Factorization

Completely positive factorization (CPF) is a critical task with applications in many fields. This paper proposes a novel method for the CPF. Based on the idea of exterior point iteration, an optimization model is given, which aims to orthogonally transform a symmetric lower rank factor to be nonnegative. The optimization problem can be solved via a modified nonlinear conjugate gradient method iteratively. The iteration points locate on the exterior of the orthonormal manifold and the closed set whose transformed matrices are nonnegative before convergence generally. Convergence analysis is given for the local or global optimum of the objective function, together with the iteration algorithm. Some potential issues that may affect the CPF are explored numerically. The exterior point method performs much better than other algorithms, not only in the efficiency of computational cost or accuracy, but also in the ability to address the CPF in some hard cases.
On regularity of solutions for certain linear Boltzmann transport equations

The paper considers a class of linear Boltzmann transport equations which models a charged particle transport. The equation is an approximation of the original exact transport equation which involves hyper-singular integrals in their collision terms. Hyper-singular integrals can be approximated by partial differential operators together with partial integral operators which leads to an approximation under consideration. This type of approximation is applied, for example in the dose calculation of radiation therapy. The related transport problem is an initial inflow boundary value problem. Regularity results of solutions are verified utilizing the scales of relevant anisotropic Sobolev spaces.
Inelastic dark matter scattering off Thallium cannot save DAMA

We study the compatibility of the observed DAMA modulation signal with inelastic scattering of dark matter (DM) off of the 0.1% Thallium (Tl) dopant in DAMA. In this work we test whether there exist regions of parameter space where the Tl interpretation gives a good fit to the most recent data from DAMA, and whether these regions are compatible with the latest constraints from other direct detection experiments. Previously, Chang et al. in 2010 [1], had proposed the Tl interpretation of the DAMA data, and more recently (in 2019) the DAMA/LIBRA collaboration [2] found regions in parameter space of Tl inelastic scattering that differ by more than 10σ from a no modulation hypothesis. We have expanded upon their work by testing whether the regions of parameter space where inelastic DM-Tl scattering gives a good fit to the most recent DAMA data survive the constraints placed by the lack of a DM signal in XENON1T and CRESST-II. In addition, we have tested how these regions change with the main sources of uncertainty: the Tl quenching factor, which has never been measured directly, and the astrophysical uncertainties in the DM distribution. We conclude that inelastic DM scattering off Tl cannot explain the DAMA data in light of null results from other experiments.
Structured Graph Learning for Scalable Subspace Clustering: From Single View to Multiview

Graph-based subspace clustering methods have exhibited promising performance. However, they still suffer some of these drawbacks: they encounter the expensive time overhead, they fail to explore the explicit clusters, and cannot generalize to unseen data points. In this work, we propose a scalable graph learning framework, seeking to address the above three challenges simultaneously. Specifically, it is based on the ideas of anchor points and bipartite graph. Rather than building an <inline-formula> <tex-math notation="LaTeX">$n\times n$ </tex-math></inline-formula> graph, where <inline-formula> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula> is the number of samples, we construct a bipartite graph to depict the relationship between samples and anchor points. Meanwhile, a connectivity constraint is employed to ensure that the connected components indicate clusters directly. We further establish the connection between our method and the <inline-formula> <tex-math notation="LaTeX">$K$ </tex-math></inline-formula>-means clustering. Moreover, a model to process multiview data is also proposed, which is linearly scaled with respect to <inline-formula> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula>. Extensive experiments demonstrate the efficiency and effectiveness of our approach with respect to many state-of-the-art clustering methods.
A world without pythons would be so simple

We show that bulk operators lying between the outermost extremal surface and the asymptotic boundary admit a simple boundary reconstruction in the classical limit. This is the converse of the Python’s lunch conjecture, which proposes that operators with support between the minimal and outermost (quantum) extremal surfaces—e.g. the interior Hawking partners—are highly complex. Our procedure for reconstructing this ‘simple wedge’ is based on the HKLL construction, but uses causal bulk propagation of perturbed boundary conditions on Lorentzian timefolds to expand the causal wedge as far as the outermost extremal surface. As a corollary, we establish the Simple Entropy proposal for the holographic dual of the area of a marginally trapped surface as well as a similar holographic dual for the outermost extremal surface. We find that the simple wedge is dual to a particular coarse-grained CFT state, obtained via averaging over all possible Python’s lunches. An efficient quantum circuit converts this coarse-grained state into a ‘simple state’ that is indistinguishable in finite time from a state with a local modular Hamiltonian. Under certain circumstances, the simple state modular Hamiltonian generates an exactly local flow; we interpret this result as a holographic dual of black hole uniqueness.
Consistency Analysis of a Dark Matter Velocity-dependent Force as an Alternative to the Cosmological Constant

A range of cosmological observations demonstrate an accelerated expansion of the universe, and the most likely explanation of this phenomenon is a cosmological constant. Given the importance of understanding the underlying physics, it is relevant to investigate alternative models. This article uses numerical simulations to test the consistency of one such alternative model. Specifically, this model has no cosmological constant; instead, the dark matter particles have an extra force proportional to the velocity squared, somewhat reminiscent of the magnetic force in electrodynamics. The constant strength of the force is the only free parameter. Because bottom-up structure formation creates cosmological structures whose internal velocity dispersions increase in time, this model may mimic the temporal evolution of the effect from a cosmological constant. It is shown that models with force linearly proportional to internal velocities, or models proportional to velocity to power 3 or more, cannot mimic the accelerated expansion induced by a cosmological constant. However, models proportional to velocity squared are still consistent with the temporal evolution of a universe with a cosmological model.
Witnessing entanglement in quantum magnets using neutron scattering

A. Scheie,1, ∗ Pontus Laurell,2, 3, † A. M. Samarakoon,1 B. Lake,4, 5 S. E. Nagler,1, 6 G. E. Granroth,1 S. Okamoto,7, 6 G. Alvarez,2, 3 and D. A. Tennant1, 6, 8, ‡ 1Neutron Scattering Division, Oak Ridge National Laboratory, Oak Ridge, TN 37831, USA 2Center for Nanophase Materials Sciences, Oak Ridge National Laboratory, Oak Ridge, TN 37831, USA 3Computational Science and Engineering Division, Oak Ridge National Laboratory, Oak Ridge, TN 37831, USA 4Helmholtz-Zentrum Berlin für Materialien und Energie GmbH, Hahn-Meitner Platz 1, D-14109 Berlin, Germany 5Institut für Festkörperphysik, Technische Universität Berlin, Hardenbergstraße 36, D-10623 Berlin, Germany 6Quantum Science Center, Oak Ridge National Laboratory, Tennessee 37831, USA 7Materials Science and Technology Division, Oak Ridge National Laboratory, Oak Ridge, TN 37831, USA. 8Shull Wollan Center A Joint Institute for Neutron Sciences, Oak Ridge National Laboratory, TN 37831. USA (Dated: February 17, 2021)
Cubic and quartic points on modular curves using generalised symmetric Chabauty

Answering a question of Zureick-Brown, we determine the cubic points on the modular curves $X_0(N)$ for $N \in \{53,57,61,65,67,73\}$ as well as the quartic points on $X_0(65)$. To do so, we develop a"partially relative"symmetric Chabauty method. Our results generalise current symmetric Chabauty theorems, and improve upon them by lowering the involved prime bound. For our curves a number of novelties occur. We prove a"higher order"Chabauty theorem to deal with these cases. Finally, to study the isolated quartic points on $X_0(65)$, we rigorously compute the full rational Mordell--Weil group of its Jacobian.