Miracle-dz/UniFAR-FTUs huggingface dataset:
Trouble beyond 
H0
 and the new cosmic triangles

José Luis Bernal, Licia Verde, 3 Raul Jimenez, 3 Marc Kamionkowski, David Valcin, and Benjamin D. Wandelt 5, 6 Department of Physics and Astronomy, Johns Hopkins University, 3400 North Charles Street, Baltimore, Maryland 21218, USA ICC, University of Barcelona, Mart́ı i Franquès, 1, E-08028 Barcelona, Spain ICREA, Pg. Lluis Companys 23, Barcelona, 08010, Spain. Sorbonne Université, CNRS, UMR 7095, Institut d’Astrophysique de Paris, 98 bis bd Arago, 75014 Paris, France. Sorbonne Université, Institut Lagrange de Paris (ILP), 98 bis bd Arago, 75014 Paris, France. Center for Computational Astrophysics, Flatiron Institute, 162 5th Avenue, 10010, New York, NY, USA.
Patterns, predictions, and actions: A story about machine learning

This graduate textbook on machine learning tells a story of how patterns in data support predictions and consequential actions. Starting with the foundations of decision making, we cover representation, optimization, and generalization as the constituents of supervised learning. A chapter on datasets as benchmarks examines their histories and scientific bases. Self-contained introductions to causality, the practice of causal inference, sequential decision making, and reinforcement learning equip the reader with concepts and tools to reason about actions and their consequences. Throughout, the text discusses historical context and societal impact. We invite readers from all backgrounds; some experience with probability, calculus, and linear algebra suffices.
There are natural scores: Full comment on Shafer,"Testing by betting: A strategy for statistical and scientific communication"

Shafer (2021) offers a betting perspective on statistical testing which may be useful for foundational debates, given that disputes over such testing continue to be intense. To be helpful for researchers, however, this perspective will need more elaboration using real examples in which (a) the betting score has a justification and interpretation in terms of study goals that distinguishes it from the uncountable mathematical possibilities, and (b) the assumptions in the sampling model are uncertain. On justification, Shafer says 'No one has made a convincing case for any particular choice' of a score derived from a P-value and then states that 'the choice is fundamentally arbitrary'. Yet some (but not most) scores can be motivated by study goals (e.g., information measurement; decision making). The one I have seen repeatedly in information statistics and data mining is the surprisal, logworth or S-value s = -log(p), where the log base determines the scale. The present comment explains the rationale for this choice.
Robust Federated Learning with Attack-Adaptive Aggregation

Federated learning is vulnerable to various attacks, such as model poisoning and backdoor attacks, even if some existing defense strategies are used. To address this challenge, we propose an attack-adaptive aggregation strategy to defend against various attacks for robust federated learning. The proposed approach is based on training a neural network with an attention mechanism that learns the vulnerability of federated learning models from a set of plausible attacks. To the best of our knowledge, our aggregation strategy is the first one that can be adapted to defend against various attacks in a data-driven fashion. Our approach has achieved competitive performance in defending model poisoning and backdoor attacks in federated learning tasks on image and text datasets.
Is Space-Time Attention All You Need for Video Understanding?

We present a convolution-free approach to video classification built exclusively on self-attention over space and time. Our method, named"TimeSformer,"adapts the standard Transformer architecture to video by enabling spatiotemporal feature learning directly from a sequence of frame-level patches. Our experimental study compares different self-attention schemes and suggests that"divided attention,"where temporal attention and spatial attention are separately applied within each block, leads to the best video classification accuracy among the design choices considered. Despite the radically new design, TimeSformer achieves state-of-the-art results on several action recognition benchmarks, including the best reported accuracy on Kinetics-400 and Kinetics-600. Finally, compared to 3D convolutional networks, our model is faster to train, it can achieve dramatically higher test efficiency (at a small drop in accuracy), and it can also be applied to much longer video clips (over one minute long). Code and models are available at: https://github.com/facebookresearch/TimeSformer.
Guidelines for checking performance and verifying accuracy of rotational rheometers: viscosity measurements in steady and oscillatory shear (IUPAC Technical Report)

Abstract The paper addresses techniques for checking the performance of rotational rheometers with cone–plate, plate–plate, or concentric cylinder geometry. We focus on the determination of the viscosity as a function of the shear rate and of the magnitude of the complex viscosity as a function of the angular frequency. After summarizing the relevant definitions and test modes, we show examples of measurements in the linear viscoelastic range, and applications of the Cox–Merz relationship. Sources of reference fluids with defined viscosities are presented, and their use in tests for verification of accuracy is demonstrated. Relevant issues, predominantly for Newtonian reference liquids, are the exploration of measurement limits, related either to the shear rate range or to reliably accessible viscosity levels. Viscoelastic reference samples are also discussed. Prerequisites for sample preparation and loading are addressed. In particular, we present recommendations based on experience from various laboratories. Finally, we discuss the problem of temperature calibration, presenting techniques that allow the determination of the true sample temperature for a given set temperature of the rheometer. This paper summarizes contributions from various industrial and academic laboratories.
The integrated angular bispectrum of weak lensing

We investigate three-point statistics in weak lensing convergence, through the integrated bispectrum. This statistic involves measuring power spectra in patches, and is thus easy to measure, and avoids the complexity of estimating the very large number of possible bispectrum configurations. The integrated bispectrum principally probes the squeezed limit of the bispectrum. To be useful as a set of summary statistics, accurate theoretical predictions of the signal are required, and, assuming Gaussian sampling distributions, the covariance matrix. In this paper, we investigate through simulations how accurate are theoretical formulae for both the integrated bispectrum and its covariance, finding that there a small inaccuracies in the theoretical signal, and more serious deviations in the covariance matrix, which may need to be estimated using simulations.
What would the rational Urysohn space and the random graph look like if they were uncountable?

Building on the work of Avraham, Rubin, and Shelah, we aim to build a variant of the Fra\"iss\'e theory for uncountable models built from finite submodels. With this aim, we generalize the notion of an increasing set of reals to other structures. As an application, we prove that the following is consistent: there exists an uncountable, separable metric space $X$ with rational distances, such that every uncountable partial 1-1 function from $X$ to $X$ is an isometry on an uncountable subset. We aim for a general theory of structures with this kind of properties. This includes results about the automorphism groups, and partial classification results.
Modeling and SAR Imaging of the Sea Surface: a Review of the State-of-the-Art with Simulations

Among other remote sensing technologies, synthetic aperture radar (SAR) has become firmly established in the practice of oceanographic research. Despite solid experience in this field, comprehensive knowledge and interpretation of ocean/sea and vessel wave signatures on radar images are still very challenging. Many technical parameters and scanning conditions vary for different SAR platforms, which also imposes some restrictions on the cross-analysis of their respective images. Numerical simulation of SAR images allows the analysis of many radar imaging parameters including environmental, ship, or platform related. In this paper, we present a universal simulation framework for SAR imagery of the sea surface, which includes the superposition of sea-ship waves. This paper is the first attempt to cover exhaustively all SAR imaging effects for the sea waves and ship wakes scene. The study is based on well proven concepts: the linear theory of sea surface modeling, Michell thin-ship theory for Kelvin wake modeling, and ocean SAR imaging theory. We demonstrate the role of two main factors that affect imaging of both types of waves: (i) SAR parameters and (ii) Hydrodynamic related parameters such as wind state and Froude number. The SAR parameters include frequency, signal polarization, mean incidence angle, image resolution, variation by scanning platform (airborne or spaceborne) of the range-to-velocity (R/V) ratio, and velocity bunching with associated shifting, smearing and azimuthal cutoff effects. We perform modeling for five wave frequency spectra and four ship models. We also compare spectra in two aspects: with Cox and Munk's probability density function (PDF), and with a novel proposed evaluation of ship wake detectability. The simulation results agree well with SAR imaging theory. The study gives a fuller understanding of radar imaging mechanisms for sea waves and ship wakes.
Partial transfusion: on the expressive influence of trainable batch norm parameters for transfer learning

Transfer learning from ImageNet is the go-to approach when applying deep learning to medical images. The approach is either to fine-tune a pre-trained model or use it as a feature extractor. Most modern architecture contain batch normalisation layers, and fine-tuning a model with such layers requires taking a few precautions as they consist of trainable and non-trainable weights and have two operating modes: training and inference. Attention is primarily given to the non-trainable weights used during inference, as they are the primary source of unexpected behaviour or degradation in performance during transfer learning. It is typically recommended to fine-tune the model with the batch normalisation layers kept in inference mode during both training and inference. In this paper, we pay closer attention instead to the trainable weights of the batch normalisation layers, and we explore their expressive influence in the context of transfer learning. We find that only fine-tuning the trainable weights (scale and centre) of the batch normalisation layers leads to similar performance as to fine-tuning all of the weights, with the added benefit of faster convergence. We demonstrate this on a variety of seven publicly available medical imaging datasets, using four different model architectures.
Negative Data Augmentation

Data augmentation is often used to enlarge datasets with synthetic samples generated in accordance with the underlying data distribution. To enable a wider range of augmentations, we explore negative data augmentation strategies (NDA)that intentionally create out-of-distribution samples. We show that such negative out-of-distribution samples provide information on the support of the data distribution, and can be leveraged for generative modeling and representation learning. We introduce a new GAN training objective where we use NDA as an additional source of synthetic data for the discriminator. We prove that under suitable conditions, optimizing the resulting objective still recovers the true data distribution but can directly bias the generator towards avoiding samples that lack the desired structure. Empirically, models trained with our method achieve improved conditional/unconditional image generation along with improved anomaly detection capabilities. Further, we incorporate the same negative data augmentation strategy in a contrastive learning framework for self-supervised representation learning on images and videos, achieving improved performance on downstream image classification, object detection, and action recognition tasks. These results suggest that prior knowledge on what does not constitute valid data is an effective form of weak supervision across a range of unsupervised learning tasks.
Quantum spin liquid with emergent chiral order in the triangular-lattice Hubbard model

The interplay between spin frustration and charge ﬂuctuation gives rise to an exotic quantum state in the intermediate-interaction regime of the half-ﬁlled triangular-lattice Hubbard model, while the nature of the state is under debate. Using the density matrix renormalization group with SU(2) spin ⊗ U(1) charge symmetries imple-mented, we study the triangular-lattice Hubbard model deﬁned on the long cylinder geometry up to circumference W = 6. A gapped quantum spin liquid, with on-site interaction 9 (cid:46) U / t (cid:46) 10 . 75, is identiﬁed between the metallic and the antiferromagnetic Mott insulating phases. In particular, we ﬁnd that this spin liquid develops a robust long-range spin scalar-chiral correlation as the system length L increases, which unambiguously unveils the spontaneous time-reversal symmetry breaking. In addition, the degeneracy of the entanglement spectrum supports symmetry fractionalization and spinon edge modes in the obtained ground state. The possible origin of chiral order in this intermediate spin liquid and its relation to the rotonlike excitations have also been discussed.
A Pursuit-Evasion Differential Game with Strategic Information Acquisition

—This paper studies a two-person linear-quadratic-Gaussian pursuit-evasion differential game with costly but con- trolled information. One player can decide when to observe the other player’s state. However, one observation of another player’s state comes with two costs: the direct cost of observing and the implicit cost of exposing his state. We call games of this type a Pursuit-Evasion-Exposure-Concealment (PEEC) game. The PEEC game constitutes two types of strategies: The control strategies and the observation strategies. We fully characterize the Nash control strategies of the PEEC game using techniques such as completing squares and the calculus of variations. We show that the derivation of the Nash observation strategies and the Nash control strategies can be decoupled. We develop a set of necessary conditions that facilitate the numerical computation of the Nash observation strategies. We show, in theory, that players with less maneuverability prefer concealment to exposure. We also show that when the game’s horizon goes to inﬁnity, the Nash observation strategy is to observe periodically, and the expected distance between the pursuer and the evader goes to zero with a bounded second moment. We conducted a series of numerical experiments to study the proposed PEEC game. We illustrate the numerical results using both ﬁgures and animation. Numerical results show that the pursuer can maintain high-grade performance even when the number of observations is limited. We also show that an evader with low maneuverability can still escape if the evader increases his stealthiness.
On the Regularity of Attention

Attention is a powerful component of modern neural networks across a wide variety of domains. In this paper, we seek to quantify the regularity (i.e. the amount of smoothness) of the attention operation. To accomplish this goal, we propose a new mathematical framework that uses measure theory and integral operators to model attention. We show that this framework is consistent with the usual definition, and that it captures the essential properties of attention. Then we use this framework to prove that, on compact domains, the attention operation is Lipschitz continuous and provide an estimate of its Lipschitz constant. Additionally, by focusing on a specific type of attention, we extend these Lipschitz continuity results to non-compact domains. We also discuss the effects regularity can have on NLP models, and applications to invertible and infinitely-deep networks.
Exploring Automatic COVID-19 Diagnosis via Voice and Symptoms from Crowdsourced Data

The development of fast and accurate screening tools, which could facilitate testing and prevent more costly clinical tests, is key to the current pandemic of COVID-19. In this context, some initial work shows promise in detecting diagnostic signals of COVID-19 from audio sounds. In this paper, we propose a voice-based framework to automatically detect individuals who have tested positive for COVID-19. We evaluate the performance of the proposed framework on a subset of data crowdsourced from our app, containing 828 samples from 343 participants. By combining voice signals and reported symptoms, an AUC of 0.79 has been attained, with a sensitivity of 0.68 and a specificity of 0.82. We hope that this study opens the door to rapid, low-cost, and convenient pre-screening tools to automatically detect the disease.
Dark matter microhalos from simplified models

We introduce simplified models for enhancements in the matter power spectrum at small scales and study their implications for dark matter substructure and gravitational observables. These models capture the salient aspects of a variety of early universe scenarios that predict enhanced small-scale structure, such as axion-like particle dark matter, light vector dark matter, and epochs of early matter domination. We use a model-independent, semi-analytic treatment to map bumps in the matter power spectrum to early-forming sub-solar mass dark matter halos and estimate their evolution, disruption, and contribution to substructure of clusters and galaxies at late times. We discuss the sensitivity of gravitational observables, including pulsar timing arrays and caustic microlensing, to both the presence of bumps in the power spectrum and variations in their basic properties. 1 ar X iv :2 10 2. 05 07 0v 1 [ as tr oph . C O ] 9 F eb 2 02 1
Clusterability as an Alternative to Anchor Points When Learning with Noisy Labels

The label noise transition matrix, characterizing the probabilities of a training instance being wrongly annotated, is crucial to designing popular solutions to learning with noisy labels. Existing works heavily rely on finding"anchor points"or their approximates, defined as instances belonging to a particular class almost surely. Nonetheless, finding anchor points remains a non-trivial task, and the estimation accuracy is also often throttled by the number of available anchor points. In this paper, we propose an alternative option to the above task. Our main contribution is the discovery of an efficient estimation procedure based on a clusterability condition. We prove that with clusterable representations of features, using up to third-order consensuses of noisy labels among neighbor representations is sufficient to estimate a unique transition matrix. Compared with methods using anchor points, our approach uses substantially more instances and benefits from a much better sample complexity. We demonstrate the estimation accuracy and advantages of our estimates using both synthetic noisy labels (on CIFAR-10/100) and real human-level noisy labels (on Clothing1M and our self-collected human-annotated CIFAR-10). Our code and human-level noisy CIFAR-10 labels are available at https://github.com/UCSC-REAL/HOC.
Systematic Generalization for Predictive Control in Multivariate Time Series

Prior work has focused on evaluating the ability of neural networks to reason about novel combinations from known components, an intrinsic property of human cognition. In this work, we aim to study systematic generalization in predicting future state trajectories of a dynamical system, conditioned on past states' trajectory (dependent variables), past and future actions (control variables). In our context, systematic generalization implies that a good model should perform well on all new combinations of future actions after being trained on all of them, but only on a limited set of their combinations. For models to generalize out-of-distribution to unseen action combinations, they should reason about the states and their dependency relation with the applied actions. We conduct a rigorous study of useful inductive biases that learn to predict the trajectories up to large horizons well, and capture true dependency relations between the states and the controls through our synthetic setup, and simulated data from electric motors.
Adversarial Perturbations Are Not So Weird: Entanglement of Robust and Non-Robust Features in Neural Network Classifiers

Neural networks trained on visual data are well-known to be vulnerable to often imperceptible adversarial perturbations. The reasons for this vulnerability are still being debated in the literature. Recently Ilyas et al. (2019) showed that this vulnerability arises, in part, because neural network classifiers rely on highly predictive but brittle"non-robust"features. In this paper we extend the work of Ilyas et al. by investigating the nature of the input patterns that give rise to these features. In particular, we hypothesize that in a neural network trained in a standard way, non-robust features respond to small,"non-semantic"patterns that are typically entangled with larger, robust patterns, known to be more human-interpretable, as opposed to solely responding to statistical artifacts in a dataset. Thus, adversarial examples can be formed via minimal perturbations to these small, entangled patterns. In addition, we demonstrate a corollary of our hypothesis: robust classifiers are more effective than standard (non-robust) ones as a source for generating transferable adversarial examples in both the untargeted and targeted settings. The results we present in this paper provide new insight into the nature of the non-robust features responsible for adversarial vulnerability of neural network classifiers.
Classification of Long Noncoding RNA Elements Using Deep Convolutional Neural Networks and Siamese Networks

In the last decade, the discovery of noncoding RNA(ncRNA) has exploded. Classifying these ncRNA is critical todetermining their function. This thesis proposes a new methodemploying deep convolutional neural networks (CNNs) to classifyncRNA sequences. To this end, this paper first proposes anefficient approach to convert the RNA sequences into imagescharacterizing their base-pairing probability. As a result, clas-sifying RNA sequences is converted to an image classificationproblem that can be efficiently solved by available CNN-basedclassification models. This research also considers the foldingpotential of the ncRNAs in addition to their primary sequence.Based on the proposed approach, a benchmark image classifi-cation dataset is generated from the RFAM database of ncRNAsequences. In addition, three classical CNN models and threeSiamese network models have been implemented and comparedto demonstrate the superior performance and efficiency of theproposed approach. Extensive experimental results show thegreat potential of using deep learning approaches for RNAclassification.