Miracle-dz/UniFAR-FTUs huggingface dataset:
Prospects for 21 cm Galaxy Cross-correlations with HERA and the Roman High-latitude Survey

The cross-correlation between the 21 cm field and the galaxy distribution is a potential probe of the Epoch of Reionization (EoR). The 21 cm signal traces neutral gas in the intergalactic medium and, on large spatial scales, this should be anticorrelated with the high-redshift galaxy distribution, which partly sources and tracks the ionized gas. In the near future, interferometers such as the Hydrogen Epoch of Reionization Array (HERA) are projected to provide extremely sensitive measurements of the 21 cm power spectrum. At the same time, the Nancy Grace Roman Space Telescope (Roman) will produce the most extensive catalog to date of bright galaxies from the EoR. Using seminumeric simulations of reionization, we explore the prospects for measuring the cross-power spectrum between the 21 cm and galaxy fields during the EoR. We forecast a 12σ detection between HERA and Roman, assuming an overlapping survey area of 500 deg2, redshift uncertainties of σ z = 0.01 (as expected for the high-latitude spectroscopic survey of Lyα-emitting galaxies), and an effective Lyα emitter duty cycle of f LAE = 0.1. Thus the HERA–Roman cross-power spectrum may be used to help verify 21 cm detections from HERA. We find that the shot-noise in the galaxy distribution is a limiting factor for detection, and so supplemental observations using Roman should prioritize deeper observations, rather than covering a wider field of view. We have made a public GitHub repository containing key parts of the calculation, which accompanies this paper: https://github.com/plaplant/21cm_gal_cross_correlation.
What's the Harm? Sharp Bounds on the Fraction Negatively Affected by Treatment

The fundamental problem of causal inference -- that we never observe counterfactuals -- prevents us from identifying how many might be negatively affected by a proposed intervention. If, in an A/B test, half of users click (or buy, or watch, or renew, etc.), whether exposed to the standard experience A or a new one B, hypothetically it could be because the change affects no one, because the change positively affects half the user population to go from no-click to click while negatively affecting the other half, or something in between. While unknowable, this impact is clearly of material importance to the decision to implement a change or not, whether due to fairness, long-term, systemic, or operational considerations. We therefore derive the tightest-possible (i.e., sharp) bounds on the fraction negatively affected (and other related estimands) given data with only factual observations, whether experimental or observational. Naturally, the more we can stratify individuals by observable covariates, the tighter the sharp bounds. Since these bounds involve unknown functions that must be learned from data, we develop a robust inference algorithm that is efficient almost regardless of how and how fast these functions are learned, remains consistent when some are mislearned, and still gives valid conservative bounds when most are mislearned. Our methodology altogether therefore strongly supports credible conclusions: it avoids spuriously point-identifying this unknowable impact, focusing on the best bounds instead, and it permits exceedingly robust inference on these. We demonstrate our method in simulation studies and in a case study of career counseling for the unemployed.
Automated Scoring for Reading Comprehension via In-context BERT Tuning

Automated scoring of open-ended student responses has the potential to significantly reduce human grader effort. Recent advances in automated scoring often leverage textual representations based on pre-trained language models such as BERT and GPT as input to scoring models. Most existing approaches train a separate model for each item/question, which is suitable for scenarios such as essay scoring where items can be quite different from one another. However, these approaches have two limitations: 1) they fail to leverage item linkage for scenarios such as reading comprehension where multiple items may share a reading passage; 2) they are not scalable since storing one model per item becomes difficult when models have a large number of parameters. In this paper, we report our (grand prize-winning) solution to the National Assessment of Education Progress (NAEP) automated scoring challenge for reading comprehension. Our approach, in-context BERT fine-tuning, produces a single shared scoring model for all items with a carefully-designed input structure to provide contextual information on each item. We demonstrate the effectiveness of our approach via local evaluations using the training dataset provided by the challenge. We also discuss the biases, common error types, and limitations of our approach.
Prototypical Calibration for Few-shot Learning of Language Models

In-context learning of GPT-like models has been recognized as fragile across different hand-crafted templates, and demonstration permutations. In this work, we propose prototypical calibration to adaptively learn a more robust decision boundary for zero- and few-shot classification, instead of greedy decoding. Concretely, our method first adopts Gaussian mixture distribution to estimate the prototypical clusters for all categories. Then we assign each cluster to the corresponding label by solving a weighted bipartite matching problem. Given an example, its prediction is calibrated by the likelihood of prototypical clusters. Experimental results show that prototypical calibration yields a substantial improvement on a diverse set of tasks. Extensive analysis across different scales also indicates that our method calibrates the decision boundary as expected, greatly improving the robustness of GPT to templates, permutations, and class imbalance.
Kernel Normalized Convolutional Networks

Existing convolutional neural network architectures frequently rely upon batch normalization (BatchNorm) to effectively train the model. BatchNorm, however, performs poorly with small batch sizes, and is inapplicable to differential privacy. To address these limitations, we propose the kernel normalization (KernelNorm) and kernel normalized convolutional layers, and incorporate them into kernel normalized convolutional networks (KNConvNets) as the main building blocks. We implement KNConvNets corresponding to the state-of-the-art ResNets while forgoing the BatchNorm layers. Through extensive experiments, we illustrate that KNConvNets achieve higher or competitive performance compared to the BatchNorm counterparts in image classification and semantic segmentation. They also significantly outperform their batch-independent competitors including those based on layer and group normalization in non-private and differentially private training. Given that, KernelNorm combines the batch-independence property of layer and group normalization with the performance advantage of BatchNorm.
Quantum and gradient corrections to false vacuum decay on a de Sitter background

We study the eﬀects of a ﬁxed de Sitter geometry background in scenarios of false vacuum decay. It is currently understood that bubble nucleation processes associated with ﬁrst order phase transitions are particularly important in cosmology. Considering the geometry of spacetime complicates the interpretation of the decay rate of a metastable vacuum. However, the eﬀects of curvature can still be studied in the particular case where backreaction is neglected. We compute the imaginary part of the action in de Sitter space, including the one-loop and the gradient corrections. We use two independent methodologies and quantify the size of the corrections without any assumptions on the thickness of the wall of the scalar background conﬁguration.
Neural-network-based mixed subgrid-scale model for turbulent flow

Abstract An artificial neural-network-based subgrid-scale (SGS) model, which is capable of predicting turbulent flows at untrained Reynolds numbers and on untrained grid resolution is developed. Providing the grid-scale strain-rate tensor alone as an input leads the model to predict a SGS stress tensor that aligns with the strain-rate tensor, and the model performs similarly to the dynamic Smagorinsky model. On the other hand, providing the resolved stress tensor as an input in addition to the strain-rate tensor is found to significantly improve the prediction of the SGS stress and dissipation, and thereby the accuracy and stability of the solution. In an attempt to apply the neural-network-based model trained for turbulent flows with a limited range of the Reynolds number and grid resolution to turbulent flows at untrained conditions on untrained grid resolution, special attention is given to the normalisation of the input and output tensors. It is found that the successful generalization of the model to turbulence for various untrained conditions and resolution is possible if distributions of the normalised inputs and outputs of the neural network remain unchanged as the Reynolds number and grid resolution vary. In a posteriori tests of the forced and the decaying homogeneous isotropic turbulence and turbulent channel flows, the developed neural-network model is found to predict turbulence statistics more accurately, maintain the numerical stability without ad hoc stabilisation such as clipping of the excessive backscatter, and to be computationally more efficient than the algebraic dynamic SGS models.
MaskGAE: Masked Graph Modeling Meets Graph Autoencoders

We present masked graph autoencoder (MaskGAE), a self-supervised learning framework for graph-structured data. Different from previous graph autoencoders (GAEs), MaskGAE adopts masked graph modeling (MGM) as a principled pre-text task: masking a portion of edges and attempting to reconstruct the missing part with partially visible, unmasked graph structure. To understand whether MGM can help GAEs learn better representations, we provide both theoretical and empirical evidence to justify the beneﬁts of this pretext task. Theoretically, we establish the connections between GAEs and contrastive learning, showing that MGM signiﬁcantly improves the self-supervised learning scheme of GAEs. Empirically, we conduct extensive experiments on a number of benchmark datasets, demonstrating the superiority of MaskGAE over several state-of-the-arts on both link prediction and node classiﬁcation tasks. Our code is publicly available at https://github.com/EdisonLeeeee/MaskGAE .
Local dynamic mode of Cognitive Behavioral Therapy

In order to increase mental health equity among the most vulnerable and marginalized communities, it is important to increase access to high-quality therapists. One facet of address these needs, is to provide timely feedback to clinicians as they interact with their clients, in a way that is also contextualized to speciﬁc clients and interactions they have had. Dynamical systems provide a framework through which to analyze interactions. The present work applies these methods to the domain of automated psychotherapist evaluation for Cognitive Behavioral Therapy (CBT). Our methods extract local dynamic modes from short windows of conversation and learns to correlate the observed dynamics to CBT competence. The results demonstrate the value of this paradigm and outlines the way in which these methods can be used to study and improve therapeutic strategies.
The price of ignorance: how much does it cost to forget noise structure in low-rank matrix estimation?

We consider the problem of estimating a rank-1 signal corrupted by structured rotationally invariant noise, and address the following question: how well do inference algorithms perform when the noise statistics is unknown and hence Gaussian noise is assumed? While the matched Bayes-optimal setting with unstructured noise is well understood, the analysis of this mismatched problem is only at its premises. In this paper, we make a step towards understanding the effect of the strong source of mismatch which is the noise statistics. Our main technical contribution is the rigorous analysis of a Bayes estimator and of an approximate message passing (AMP) algorithm, both of which incorrectly assume a Gaussian setup. The first result exploits the theory of spherical integrals and of low-rank matrix perturbations; the idea behind the second one is to design and analyze an artificial AMP which, by taking advantage of the flexibility in the denoisers, is able to"correct"the mismatch. Armed with these sharp asymptotic characterizations, we unveil a rich and often unexpected phenomenology. For example, despite AMP is in principle designed to efficiently compute the Bayes estimator, the former is outperformed by the latter in terms of mean-square error. We show that this performance gap is due to an incorrect estimation of the signal norm. In fact, when the SNR is large enough, the overlaps of the AMP and the Bayes estimator coincide, and they even match those of optimal estimators taking into account the structure of the noise.
Deep Learning Methods for Proximal Inference via Maximum Moment Restriction

The No Unmeasured Confounding Assumption is widely used to identify causal effects in observational studies. Recent work on proximal inference has provided alternative identification results that succeed even in the presence of unobserved confounders, provided that one has measured a sufficiently rich set of proxy variables, satisfying specific structural conditions. However, proximal inference requires solving an ill-posed integral equation. Previous approaches have used a variety of machine learning techniques to estimate a solution to this integral equation, commonly referred to as the bridge function. However, prior work has often been limited by relying on pre-specified kernel functions, which are not data adaptive and struggle to scale to large datasets. In this work, we introduce a flexible and scalable method based on a deep neural network to estimate causal effects in the presence of unmeasured confounding using proximal inference. Our method achieves state of the art performance on two well-established proximal inference benchmarks. Finally, we provide theoretical consistency guarantees for our method.
An Efficient Methodology to Identify Missing Tags in Large-Scale RFID Systems

Radio frequency identification (RFID) has been widely has broad applications. One such application is to use RFID to track inventory in warehouses and retail stores. In this application, timely identifying the missing items is an ongoing engineering problem. A feasible solution to this problem is to map each tag to a time slot and verify the presence of a tag by comparing the status of the predicted time slot and the actual time slot. However, existing works are time inefficient because they only verify tags one by one in singleton slots but ignore the collision slots mapped by multiple tags. To accelerate the identification process, we use bit tracking to verify tags in collision slots and design two protocols accordingly. We first propose the Sequential String based Missing Tag Identification (SSMTI) protocol, which converts all time slots to collision slots and enables tags in each slot to reply to a designed string simultaneously. By using bit tracking to decode the combined string, the reader can verify multiple tags together. To improve the performance of SSMTI when most tags are missing, we further propose the Interactive String based Missing Tag Identification (ISMTI) protocol. ISMTI improves the strategies of designing strings for each collided tag so that the reader can verify more tags using shorter strings than SSMTI.Besides, ISMTI can dynamically adjust the verification mechanism according to the proportion of missing tags to maintain time efficiency. We also provide theoretical analysis for proposed protocols to minimize execution time and evaluate their performance through extensive simulations. Compared with state-of-the-art solutions, the proposed SSMTI and ISMTI can reduce the time cost by as much as 39.74% and 68.87%.
Calibration Matters: Tackling Maximization Bias in Large-scale Advertising Recommendation Systems

Calibration is defined as the ratio of the average predicted click rate to the true click rate. The optimization of calibration is essential to many online advertising recommendation systems because it directly affects the downstream bids in ads auctions and the amount of money charged to advertisers. Despite its importance, calibration optimization often suffers from a problem called"maximization bias". Maximization bias refers to the phenomenon that the maximum of predicted values overestimates the true maximum. The problem is introduced because the calibration is computed on the set selected by the prediction model itself. It persists even if unbiased predictions can be achieved on every datapoint and worsens when covariate shifts exist between the training and test sets. To mitigate this problem, we theorize the quantification of maximization bias and propose a variance-adjusting debiasing (VAD) meta-algorithm in this paper. The algorithm is efficient, robust, and practical as it is able to mitigate maximization bias problems under covariate shifts, neither incurring additional online serving costs nor compromising the ranking performance. We demonstrate the effectiveness of the proposed algorithm using a state-of-the-art recommendation neural network model on a large-scale real-world dataset.
Conductivity of an electron coupled to anharmonic phonons: Quantum-classical simulations and comparison of approximations

We study the impact of phonon anharmonicity on the electronic dynamics of soft materials using a nonperturbative quantum-classical approach. The method is applied to a one-dimensional model of doped organic semiconductors with low-frequency intermolecular lattice phonons. We find that anharmonicity that leads to phonon hardening increases the mobility and anharmonicity that leads to phonon softening decreases the mobility. We also test various approximations, including the use of adiabatic phonon disorder, an effective harmonic model with temperature-dependent frequencies, and the Boltzmann transport equation with second-order perturbation theory scattering rates. Overall, we find surprisingly good agreement between all methods but that accounting for phonon anharmonicity is important for accurate prediction of electronic transport including both quantitative mobility values and their qualitative temperature dependence. For the model studied, phonon lifetime effects have relatively little impact on carrier transport, but the effective frequency shift due to anharmonicity is essential. In cases with highly asymmetric, non-Gaussian disorder, an effective harmonic model cannot quantitatively reproduce mobilities or finite-frequency conductivity, and this is especially true for acoustic phonons.
DDDM: a Brain-Inspired Framework for Robust Classification

Despite their outstanding performance in a broad spectrum of real-world tasks, deep artificial neural networks are sensitive to input noises, particularly adversarial perturbations. On the contrary, human and animal brains are much less vulnerable. In contrast to the one-shot inference performed by most deep neural networks, the brain often solves decision-making with an evidence accumulation mechanism that may trade time for accuracy when facing noisy inputs. The mechanism is well described by the Drift-Diffusion Model (DDM). In the DDM, decision-making is modeled as a process in which noisy evidence is accumulated toward a threshold. Drawing inspiration from the DDM, we propose the Dropout-based Drift-Diffusion Model (DDDM) that combines test-phase dropout and the DDM for improving the robustness for arbitrary neural networks. The dropouts create temporally uncorrelated noises in the network that counter perturbations, while the evidence accumulation mechanism guarantees a reasonable decision accuracy. Neural networks enhanced with the DDDM tested in image, speech, and text classification tasks all significantly outperform their native counterparts, demonstrating the DDDM as a task-agnostic defense against adversarial attacks.
The Proton Structure in and out of Muonic Hydrogen

Laser spectroscopy of muonic atoms has been recently used to probe properties of light nuclei with unprecedented precision. We introduce nuclear effects in hydrogen-like atoms, nucleon structure quantities (form factors, structure functions, polarizabilities), and their effects in the Lamb shift and hyperfine splitting (HFS) of muonic hydrogen (μH). Updated theory predictions for the Lamb shift and HFS in μH are presented. We review the challenges of the ongoing effort to measure the ground-state HFS in μH and its impact on our understanding of the nucleon spin structure. To narrow down this search, we present a novel theory prediction obtained by scaling the measured HFS in hydrogen while leveraging radiative corrections. We also summarize recent developments in the spectroscopy of simple atomic and molecular systems and emphasize how they allow for precise determinations of fundamental constants, bound-state QED tests, and New Physics searches.
Neural-Symbolic Models for Logical Queries on Knowledge Graphs

Answering complex first-order logic (FOL) queries on knowledge graphs is a fundamental task for multi-hop reasoning. Traditional symbolic methods traverse a complete knowledge graph to extract the answers, which provides good interpretation for each step. Recent neural methods learn geometric embeddings for complex queries. These methods can generalize to incomplete knowledge graphs, but their reasoning process is hard to interpret. In this paper, we propose Graph Neural Network Query Executor (GNN-QE), a neural-symbolic model that enjoys the advantages of both worlds. GNN-QE decomposes a complex FOL query into relation projections and logical operations over fuzzy sets, which provides interpretability for intermediate variables. To reason about the missing links, GNN-QE adapts a graph neural network from knowledge graph completion to execute the relation projections, and models the logical operations with product fuzzy logic. Experiments on 3 datasets show that GNN-QE significantly improves over previous state-of-the-art models in answering FOL queries. Meanwhile, GNN-QE can predict the number of answers without explicit supervision, and provide visualizations for intermediate variables.
Minimal Explanations for Neural Network Predictions

Explaining neural network predictions is known to be a challenging problem. In this paper, we propose a novel approach which can be effectively exploited, either in isolation or in combination with other methods, to enhance the interpretability of neural model predictions. For a given input to a trained neural model, our aim is to compute a smallest set of input features so that the model prediction changes when these features are disregarded by setting them to an uninformative baseline value. While computing such minimal explanations is computationally intractable in general for fully-connected neural networks, we show that the problem becomes solvable in polynomial time by a greedy algorithm under mild assumptions on the network’s activation functions. We then show that our tractability result extends seamlessly to more advanced neural architectures such as convolutional and graph neural networks. We conduct experiments to showcase the capability of our method for identifying the input features that are essential to the model’s prediction.