Miracle-dz/UniFAR-FTUs huggingface dataset:
Search for an anomalous excess of inclusive charged-current 
νe
 interactions in the MicroBooNE experiment using Wire-Cell reconstruction

We report a search for an anomalous excess of inclusive charged-current (CC) ν e interactions using the Wire-Cell event reconstruction package in the MicroBooNE experiment, which is motivated by the previous observation of a low-energy excess (LEE) of electromagnetic events from the MiniBooNE experiment. With a single liquid argon time projection chamber detector, the measurements of ν μ CC interactions as well as π 0 interactions are used to constrain signal and background predictions of ν e CC interactions. A data set collected from February 2016 to July 2018 corresponding to an exposure of 6 . 369 × 10 20 protons on target from the Booster Neutrino Beam at FNAL is analyzed. With x representing an overall normalization factor and referred to as the LEE strength parameter, we select 56 fully contained ν e CC candidates while expecting 69 . 6 (cid:1) 8 . 0 (stat.) (cid:1) 5 . 0 (sys.) and 103 . 8 (cid:1) 9 . 0 (stat.) (cid:1) 7 . 4 (sys.) candidates after constraints for the absence (eLEE x ¼ 0 ) of the median signal strength derived from the MiniBooNE observation and the presence (eLEE x ¼ 1 ) of that signal strength, respectively. Under a nested hypothesis test using both rate and shape information in all available channels, the best-fit x is determined to be 0 (eLEE x ¼ 0 ) with a 95.5% confidence level upper limit of x at 0.502. Under a simple-vs-simple hypotheses test, the eLEE x ¼ 1 hypothesis is rejected at 3 . 75 σ , while the eLEE x ¼ 0 hypothesis is shown to be consistent with the observation at 0 . 45 σ . In the context of the eLEE model, the estimated 68.3% confidence interval of the ν e CC hypothesis to explain the LEE observed in the MiniBooNE experiment is disfavored at a significance level of more than 2 . 6 σ ( 3 . 0 σ ) considering MiniBooNE ’ s full (statistical) uncertainties.
Revisiting Sanity Checks for Saliency Maps

Saliency methods are a popular approach for model debugging and explainability. However, in the absence of ground-truth data for what the correct maps should be, evaluating and comparing different approaches remains a long-standing challenge. The sanity checks methodology of Adebayo et al [Neurips 2018] has sought to address this challenge. They argue that some popular saliency methods should not be used for explainability purposes since the maps they produce are not sensitive to the underlying model that is to be explained. Through a causal re-framing of their objective, we argue that their empirical evaluation does not fully establish these conclusions, due to a form of confounding introduced by the tasks they evaluate on. Through various experiments on simple custom tasks we demonstrate that some of their conclusions may indeed be artifacts of the tasks more than a criticism of the saliency methods themselves. More broadly, our work challenges the utility of the sanity check methodology, and further highlights that saliency map evaluation beyond ad-hoc visual examination remains a fundamental challenge.
Similarity and Matching of Neural Network Representations

We employ a toolset -- dubbed Dr. Frankenstein -- to analyse the similarity of representations in deep neural networks. With this toolset, we aim to match the activations on given layers of two trained neural networks by joining them with a stitching layer. We demonstrate that the inner representations emerging in deep convolutional neural networks with the same architecture but different initializations can be matched with a surprisingly high degree of accuracy even with a single, affine stitching layer. We choose the stitching layer from several possible classes of linear transformations and investigate their performance and properties. The task of matching representations is closely related to notions of similarity. Using this toolset, we also provide a novel viewpoint on the current line of research regarding similarity indices of neural network representations: the perspective of the performance on a task.
A Roadmap to Gamma-Ray Bursts: New Developments and Applications to Cosmology

Gamma-ray bursts are the most powerful explosions in the universe and are mainly placed at very large redshifts, up to z≃9. In this short review, we first discuss gamma-ray burst classification and morphological properties. We then report the likely relations between gamma-ray bursts and other astronomical objects, such as black holes, supernovae, neutron stars, etc., discussing in detail gamma-ray burst progenitors. We classify long and short gamma-ray bursts, working out their timescales, and introduce the standard fireball model. Afterwards, we focus on direct applications of gamma-ray bursts to cosmology and underline under which conditions such sources would act as perfect standard candles if correlations between photometric and spectroscopic properties were not jeopardized by the circularity problem. In this respect, we underline how the shortage of low-z gamma-ray bursts prevents anchor gamma-ray bursts with primary distance indicators. Moreover, we analyze in detail the most adopted gamma-ray burst correlations, highlighting their main differences. We therefore show calibration techniques, comparing such treatments with non-calibration scenarios. For completeness, we discuss the physical properties of the correlation scatters and systematics occurring during experimental computations. Finally, we develop the most recent statistical methods, star formation rate, and high-redshift gamma-ray burst excess and show the most recent constraints obtained from experimental analyses.
How Important is Importance Sampling for Deep Budgeted Training?

Long iterative training processes for Deep Neural Networks (DNNs) are commonly required to achieve state-of-the-art performance in many computer vision tasks. Importance sampling approaches might play a key role in budgeted training regimes, i.e. when limiting the number of training iterations. These approaches aim at dynamically estimating the importance of each sample to focus on the most relevant and speed up convergence. This work explores this paradigm and how a budget constraint interacts with importance sampling approaches and data augmentation techniques. We show that under budget restrictions, importance sampling approaches do not provide a consistent improvement over uniform sampling. We suggest that, given a specific budget, the best course of action is to disregard the importance and introduce adequate data augmentation; e.g. when reducing the budget to a 30% in CIFAR-10/100, RICAP data augmentation maintains accuracy, while importance sampling does not. We conclude from our work that DNNs under budget restrictions benefit greatly from variety in the training set and that finding the right samples to train on is not the most effective strategy when balancing high performance with low computational requirements. Source code available at https://git.io/JKHa3 .
Beyond Classification: Knowledge Distillation using Multi-Object Impressions

Knowledge Distillation (KD) utilizes training data as a transfer set to transfer knowledge from a complex network (Teacher) to a smaller network (Student). Several works have recently identified many scenarios where the training data may not be available due to data privacy or sensitivity concerns and have proposed solutions under this restrictive constraint for the classification task. Unlike existing works, we, for the first time, solve a much more challenging problem, i.e.,"KD for object detection with zero knowledge about the training data and its statistics". Our proposed approach prepares pseudo-targets and synthesizes corresponding samples (termed as"Multi-Object Impressions"), using only the pretrained Faster RCNN Teacher network. We use this pseudo-dataset as a transfer set to conduct zero-shot KD for object detection. We demonstrate the efficacy of our proposed method through several ablations and extensive experiments on benchmark datasets like KITTI, Pascal and COCO. Our approach with no training samples, achieves a respectable mAP of 64.2% and 55.5% on the student with same and half capacity while performing distillation from a Resnet-18 Teacher of 73.3% mAP on KITTI.
Robust Contrastive Learning Using Negative Samples with Diminished Semantics

Unsupervised learning has recently made exceptional progress because of the development of more effective contrastive learning methods. However, CNNs are prone to depend on low-level features that humans deem non-semantic. This dependency has been conjectured to induce a lack of robustness to image perturbations or domain shift. In this paper, we show that by generating carefully designed negative samples, contrastive learning can learn more robust representations with less dependence on such features. Contrastive learning utilizes positive pairs that preserve semantic information while perturbing superficial features in the training images. Similarly, we propose to generate negative samples in a reversed way, where only the superfluous instead of the semantic features are preserved. We develop two methods, texture-based and patch-based augmentations, to generate negative samples. These samples achieve better generalization, especially under out-of-domain settings. We also analyze our method and the generated texture-based samples, showing that texture features are indispensable in classifying particular ImageNet classes and especially finer classes. We also show that model bias favors texture and shape features differently under different test settings. Our code, trained models, and ImageNet-Texture dataset can be found at https://github.com/SongweiGe/Contrastive-Learning-with-Non-Semantic-Negatives.
History Determinism vs. Good for Gameness in Quantitative Automata

Automata models between determinism and nondeterminism/alternations can retain some of the algorithmic properties of deterministic automata while enjoying some of the expressiveness and succinctness of nondeterminism. We study three closely related such models -- history determinism, good for gameness and determinisability by pruning -- on quantitative automata. While in the Boolean setting, history determinism and good for gameness coincide, we show that this is no longer the case in the quantitative setting: good for gameness is broader than history determinism, and coincides with a relaxed version of it, defined with respect to thresholds. We further identify criteria in which history determinism, which is generally broader than determinisability by pruning, coincides with it, which we then apply to typical quantitative automata types. As a key application of good for games and history deterministic automata is synthesis, we clarify the relationship between the two notions and various quantitative synthesis problems. We show that good-for-games automata are central for"global"(classical) synthesis, while"local"(good-enough) synthesis reduces to deciding whether a nondeterministic automaton is history deterministic.
On Compositions of Transformations in Contrastive Self-Supervised Learning

In the image domain, excellent representations can be learned by inducing invariance to content-preserving transformations via noise contrastive learning. In this paper, we generalize contrastive learning to a wider set of transformations, and their compositions, for which either invariance or distinctiveness is sought. We show that it is not immediately obvious how existing methods such as SimCLR can be extended to do so. Instead, we introduce a number of formal requirements that all contrastive formulations must satisfy, and propose a practical construction which satisfies these requirements. In order to maximise the reach of this analysis, we express all components of noise contrastive formulations as the choice of certain generalized transformations of the data (GDTs), including data sampling. We then consider videos as an example of data in which a large variety of transformations are applicable, accounting for the extra modalities – for which we analyze audio and text – and the dimension of time. We find that being invariant to certain transformations and distinctive to others is critical to learning effective video representations, improving the state-of-the-art for multiple benchmarks by a large margin, and even surpassing supervised pretraining. Code and pretrained models are available1.
Improved Analysis and Rates for Variance Reduction under Without-replacement Sampling Orders

When applying a stochastic algorithm, one must choose an order to draw samples. The practical choices are without-replacement sampling orders, which are empirically faster and more cache-friendly than uniform-iid-sampling but often have inferior theoretical guarantees. Without-replacement sampling is well understood only for SGD without variance reduction. In this paper, we will improve the convergence analysis and rates of variance reduction under without-replacement sampling orders for composite finite-sum minimization. Our results are in two-folds. First, we develop a damped variant of Finito called Prox-DFinito and establish its convergence rates with random reshuffling, cyclic sampling, and shuffling-once, under both convex and strongly convex scenarios. These rates match full-batch gradient descent and are state-of-the-art compared to the existing results for without-replacement sampling with variance-reduction. Second, our analysis can gauge how the cyclic order will influence the rate of cyclic sampling and, thus, allows us to derive the optimal fixed ordering. In the highly data-heterogeneous scenario, Prox-DFinito with optimal cyclic sampling can attain a sample-size-independent convergence rate, which, to our knowledge, is the first result that can match with uniform-iid-sampling with variance reduction. We also propose a practical method to discover the optimal cyclic ordering numerically.
Weakly-Supervised Surface Crack Segmentation by Generating Pseudo-Labels Using Localization With a Classifier and Thresholding

Surface cracks are a common sight on public infrastructure nowadays. Recent work has been addressing this problem by supporting structural maintenance measures using machine learning methods. Those methods are used to segment surface cracks from their background, making them easier to localize. However, a common issue is that to create a well-functioning algorithm, the training data needs to have detailed annotations of pixels that belong to cracks. Our work proposes a weakly supervised approach that leverages a CNN classifier in a novel way to create surface crack pseudo labels. First, we use the classifier to create a rough crack localization map by using its class activation maps and a patch based classification approach and fuse this with a thresholding based approach to segment the mostly darker crack pixels. The classifier assists in suppressing noise from the background regions, which commonly are incorrectly highlighted as cracks by standard thresholding methods. Then, the pseudo labels can be used in an end-to-end approach when training a standard CNN for surface crack segmentation. Our method is shown to yield sufficiently accurate pseudo labels. Those labels, incorporated into segmentation CNN training using multiple recent crack segmentation architectures, achieve comparable performance to fully supervised methods on four popular crack segmentation datasets.
A Robust Protocol for Entropy Measurement in Mesoscopic Circuits

Previous measurements utilizing Maxwell relations to measure change in entropy, S, demonstrated remarkable accuracy in measuring the spin-1/2 entropy of electrons in a weakly coupled quantum dot. However, these previous measurements relied upon prior knowledge of the charge transition lineshape. This had the benefit of making the quantitative determination of entropy independent of scale factors in the measurement itself but at the cost of limiting the applicability of the approach to simple systems. To measure the entropy of more exotic mesoscopic systems, a more flexible analysis technique may be employed; however, doing so requires a precise calibration of the measurement. Here, we give details on the necessary improvements made to the original experimental approach and highlight some of the common challenges (along with strategies to overcome them) that other groups may face when attempting this type of measurement.
V-Learning - A Simple, Efficient, Decentralized Algorithm for Multiagent RL

A major challenge of multiagent reinforcement learning (MARL) is the curse of multiagents, where the size of the joint action space scales exponentially with the number of agents. This remains to be a bottleneck for designing efficient MARL algorithms even in a basic scenario with finitely many states and actions. This paper resolves this challenge for the model of episodic Markov games. We design a new class of fully decentralized algorithms -- V-learning, which provably learns Nash equilibria (in the two-player zero-sum setting), correlated equilibria and coarse correlated equilibria (in the multiplayer general-sum setting) in a number of samples that only scales with $\max_{i\in[m]} A_i$, where $A_i$ is the number of actions for the $i^{\rm th}$ player. This is in sharp contrast to the size of the joint action space which is $\prod_{i=1}^m A_i$. V-learning (in its basic form) is a new class of single-agent RL algorithms that convert any adversarial bandit algorithm with suitable regret guarantees into a RL algorithm. Similar to the classical Q-learning algorithm, it performs incremental updates to the value functions. Different from Q-learning, it only maintains the estimates of V-values instead of Q-values. This key difference allows V-learning to achieve the claimed guarantees in the MARL setting by simply letting all agents run V-learning independently.
Labeling Trick: A Theory of Using Graph Neural Networks for Multi-Node Representation Learning

In this paper, we provide a theory of using graph neural networks (GNNs) for multi-node representation learning (where we are interested in learning a representation for a set of more than one node, such as link). We know that GNN is designed to learn single-node representations. When we want to learn a node set representation involving multiple nodes, a common practice in previous works is to directly aggregate the single-node representations obtained by a GNN into a joint node set representation. In this paper, we show a fundamental constraint of such an approach, namely the inability to capture the dependence between nodes in the node set, and argue that directly aggregating individual node representations does not lead to an effective joint representation for multiple nodes. Then, we notice that a few previous successful works for multi-node representation learning, including SEAL, Distance Encoding, and ID-GNN, all used node labeling. These methods first label nodes in the graph according to their relationships with the target node set before applying a GNN. Then, the node representations obtained in the labeled graph are aggregated into a node set representation. By investigating their inner mechanisms, we unify these node labeling techniques into a single and most general form -- labeling trick. We prove that with labeling trick a sufficiently expressive GNN learns the most expressive node set representations, thus in principle solves any joint learning tasks over node sets. Experiments on one important two-node representation learning task, link prediction, verified our theory. Our work explains the superior performance of previous node-labeling-based methods, and establishes a theoretical foundation of using GNNs for multi-node representation learning.
The Value of Information When Deciding What to Learn

All sequential decision-making agents explore so as to acquire knowledge about a particular target. It is often the responsibility of the agent designer to construct this target which, in rich and complex environments, constitutes a onerous burden; without full knowledge of the environment itself, a designer may forge a sub-optimal learning target that poorly balances the amount of information an agent must acquire to identify the target against the target's associated performance shortfall. While recent work has developed a connection between learning targets and rate-distortion theory to address this challenge and empower agents that decide what to learn in an automated fashion, the proposed algorithm does not optimally tackle the equally important challenge of efficient information acquisition. In this work, building upon the seminal design principle of information-directed sampling (Russo&Van Roy, 2014), we address this shortcoming directly to couple optimal information acquisition with the optimal design of learning targets. Along the way, we offer new insights into learning targets from the literature on rate-distortion theory before turning to empirical results that confirm the value of information when deciding what to learn.
SILVERRUSH. XI. Constraints on the Lyα Luminosity Function and Cosmic Reionization at z = 7.3 with Subaru/Hyper Suprime-Cam

The Lyα luminosity function (LF) of Lyα emitters (LAEs) has been used to constrain the neutral hydrogen fraction in the intergalactic medium (IGM) and thus the timeline of cosmic reionization. Here we present the results of a new narrowband imaging survey for z = 7.3 LAEs in a large area of ∼3 deg2 with Subaru/Hyper Suprime-Cam. No LAEs are detected down to L Lyα ≃ 1043.2 erg s−1 in an effective cosmic volume of ∼2 × 106 Mpc3, placing an upper limit on the bright part of the z = 7.3 Lyα LF for the first time and confirming a decrease in bright LAEs from z = 7.0. By comparing this upper limit with the Lyα LF in the case of fully ionized IGM, which is predicted using an observed z = 5.7 Lyα LF on the assumption that the intrinsic Lyα LF evolves in the same way as the UV LF, we obtain the relative IGM transmission TLyαIGM(7.3)/TLyαIGM(5.7)<0.77 and then the volume-averaged neutral fraction x H I(7.3) > 0.28. Cosmic reionization is thus still ongoing at z = 7.3, consistent with results from other x H I estimation methods. A similar analysis using literature Lyα LFs finds that at z = 6.6 and 7.0, the observed Lyα LF agrees with the predicted one, consistent with full ionization.
Streaming Generalized Canonical Polyadic Tensor Decompositions

In this paper, we develop a method which we call OnlineGCP for computing the Generalized Canonical Polyadic (GCP) tensor decomposition of streaming data. GCP differs from traditional canonical polyadic (CP) tensor decompositions as it allows for arbitrary objective functions which the CP model attempts to minimize. This approach can provide better fits and more interpretable models when the observed tensor data is strongly non-Gaussian. In the streaming case, tensor data is gradually observed over time and the algorithm must incrementally update a GCP factorization with limited access to prior data. In this work, we extend the GCP formalism to the streaming context by deriving a GCP optimization problem to be solved as new tensor data is observed, formulate a tunable history term to balance reconstruction of recently observed data with data observed in the past, develop a scalable solution strategy based on segregated solves using stochastic gradient descent methods, describe a software implementation that provides performance and portability to contemporary CPU and GPU architectures and demonstrate the utility and performance of the approach and software on several synthetic and real tensor data sets.