Miracle-dz/UniFAR-FTUs huggingface dataset:
GW190521: Search for echoes due to stimulated Hawking radiation from black holes

Being arguably the most massive binary black hole merger event observed to date, GW190521 deserves special attention. The exceptionally loud ringdown of this merger makes it an ideal candidate to search for gravitational wave echoes, a proposed smoking gun for the quantum structure of black hole horizons. We perform an unprecedented multi-pronged search for echoes via two well-established and independent pipelines: a template-based search for stimulated emission of Hawking radiation, or Boltzmann echoes, and the model-agnostic coherent WaveBurst (cWB) search. Stimulated Hawking radiation from the merger is expected to lead to post-merger echoes at horizon mode frequency of $\sim50$ Hz (for quadrupolar gravitational radiation), repeating at intervals of $\sim1$ second, due to partial reflection off Planckian quantum structure of the horizon. A careful analysis using dynamic nested sampling yields a Bayesian evidence of $8^{+4}_{-2}$ (90\% confidence level) for this signal following GW190521, carrying an excess of $6^{+10}_{-5}\%$ in gravitational wave energy, relative to the main event (consistent with the predicted amplitude of Boltzmann echoes). The"look-elsewhere"effect is estimated by using General Relativity (plus Boltzmann echoes) injections in real data, around the event, giving a false (true) positive detection probability for higher Bayes factors of $1.5^{+1.2}_{-0.9}\%$ ($35\pm7\%$). Similarly, the reconstructed waveform of the first echo in cWB carries an energy excess of $13^{+16}_{-7}\%$. While the current evidence for stimulated Hawking radiation does not reach the gold standard of $5\sigma$ (or p-value $<3\times10^{-7}$), our findings are in line with expectations for stimulated Hawking radiation at current detector sensitivities. The next generation of gravitational wave observatories can thus draw a definitive conclusion on the quantum nature of black hole horizons.
Improved belief propagation decoding algorithm based on decoupling representation of Pauli operators for quantum LDPC codes

We propose a new method called decoupling representation to represent Pauli operators as vectors over $GF(2)$, based on which we propose partially decoupled belief propagation and fully decoupled belief propagation decoding algorithm for quantum low density parity-check codes. These two algorithms have the capability to deal with the correlations between the $X$ part and the $Z$ part of the vectors in symplectic representation, which are introduced by Pauli $Y$ errors. Hence, they can not only apply to CSS codes, but also to non-CSS codes. Under the assumption that there is no measurement error, compared with traditional belief propagation algorithm in symplectic representation over $GF(2)$, within the same number of iterations, the decoding accuracy of partially decoupled belief propagation and fully decoupled belief propagation algorithm is significantly improved in pure $Y$ noise and depolarizing noise, which supports that decoding algorithms of quantum error correcting codes might have better performance in decoupling representation than in symplectic representation. The impressive performance of fully decoupled belief propagation algorithm might promote the realization of quantum error correcting codes in engineering.
OSPC: Online Sequential Photometric Calibration

Photometric calibration is essential to many computer vision applications. One of its key benefits is enhancing the performance of Visual SLAM, especially when it depends on a direct method for tracking, such as the standard KLT algorithm. Another advantage could be in retrieving the sensor irradiance values from measured intensities, as a pre-processing step for some vision algorithms, such as shape-from-shading. Current photometric calibration systems rely on a joint optimization problem and encounter an ambiguity in the estimates, which can only be resolved using ground truth information. We propose a novel method that solves for photometric parameters using a sequential estimation approach. Our proposed method achieves high accuracy in estimating all parameters; furthermore, the formulations are linear and convex, which makes the solution fast and suitable for online applications. Experiments on a Visual Odometry system validate the proposed method and demonstrate its advantages.
The Implicit Regularization of Dynamical Stability in Stochastic Gradient Descent

In this paper, we study the implicit regularization of stochastic gradient descent (SGD) through the lens of {\em dynamical stability} (Wu et al., 2018). We start by revising existing stability analyses of SGD, showing how the Frobenius norm and trace of Hessian relate to different notions of stability. Notably, if a global minimum is linearly stable for SGD, then the trace of Hessian must be less than or equal to $2/\eta$, where $\eta$ denotes the learning rate. By contrast, for gradient descent (GD), the stability imposes a similar constraint but only on the largest eigenvalue of Hessian. We then turn to analyze the generalization properties of these stable minima, focusing specifically on two-layer ReLU networks and diagonal linear networks. Notably, we establish the {\em equivalence} between these metrics of sharpness and certain parameter norms for the two models, which allows us to show that the stable minima of SGD provably generalize well. By contrast, the stability-induced regularization of GD is provably too weak to ensure satisfactory generalization. This discrepancy provides an explanation of why SGD often generalizes better than GD. Note that the learning rate (LR) plays a pivotal role in the strength of stability-induced regularization. As the LR increases, the regularization effect becomes more pronounced, elucidating why SGD with a larger LR consistently demonstrates superior generalization capabilities. Additionally, numerical experiments are provided to support our theoretical findings.
Fine-Tuning Language Models with Just Forward Passes

Fine-tuning language models (LMs) has yielded success on diverse downstream tasks, but as LMs grow in size, backpropagation requires a prohibitively large amount of memory. Zeroth-order (ZO) methods can in principle estimate gradients using only two forward passes but are theorized to be catastrophically slow for optimizing large models. In this work, we propose a memory-efficient zerothorder optimizer (MeZO), adapting the classical ZO-SGD method to operate in-place, thereby fine-tuning LMs with the same memory footprint as inference. For example, with a single A100 80GB GPU, MeZO can train a 30-billion parameter model, whereas fine-tuning with backpropagation can train only a 2.7B LM with the same budget. We conduct comprehensive experiments across model types (masked and autoregressive LMs), model scales (up to 66B), and downstream tasks (classification, multiple-choice, and generation). Our results demonstrate that (1) MeZO significantly outperforms in-context learning and linear probing; (2) MeZO achieves comparable performance to fine-tuning with backpropagation across multiple tasks, with up to 12x memory reduction and up to 2x GPU-hour reduction in our implementation; (3) MeZO is compatible with both full-parameter and parameter-efficient tuning techniques such as LoRA and prefix tuning; (4) MeZO can effectively optimize non-differentiable objectives (e.g., maximizing accuracy or F1). We support our empirical findings with theoretical insights, highlighting how adequate pre-training and task prompts enable MeZO to fine-tune huge models, despite classical ZO analyses suggesting otherwise.
Understanding Predictive Coding as an Adaptive Trust-Region Method

Predictive coding (PC) is a brain-inspired local learning algorithm that has recently been suggested to provide advantages over backpropagation (BP) in biologically relevant scenarios. While theoretical work has mainly focused on showing how PC can approximate BP in various limits, the putative benefits of"natural"PC are less understood. Here we develop a theory of PC as an adaptive trust-region (TR) algorithm that uses second-order information. We show that the learning dynamics of PC can be interpreted as interpolating between BP's loss gradient direction and a TR direction found by the PC inference dynamics. Our theory suggests that PC should escape saddle points faster than BP, a prediction which we prove in a shallow linear model and support with experiments on deeper networks. This work lays a foundation for understanding PC in deep and wide networks.
Improved Sales Forecasting using Trend and Seasonality Decomposition with LightGBM

Retail sales forecasting presents a significant challenge for large retailers such as Walmart and Amazon, due to the vast assortment of products, geographical location heterogeneity, seasonality, and external factors including weather, local economic conditions, and geopolitical events. Various methods have been employed to tackle this challenge, including traditional time series models, machine learning models, and neural network mechanisms, but the difficulty persists. Categorizing data into relevant groups has been shown to improve sales forecast accuracy as time series from different categories may exhibit distinct patterns. In this paper, we propose a new measure to indicate the unique impacts of the trend and seasonality components on a time series and suggest grouping time series based on this measure. We apply this approach to Walmart sales data from 01/29/2011 to 05/22/2016 and generate sales forecasts from 05/23/2016 to 06/19/2016. Our experiments show that the proposed strategy can achieve improved accuracy. Furthermore, we present a robust pipeline for conducting retail sales forecasting.
Tri-level Joint Natural Language Understanding for Multi-turn Conversational Datasets

Natural language understanding typically maps single utterances to a dual level semantic frame, sentence level intent and slot labels at the word level. The best performing models force explicit interaction between intent detection and slot filling. We present a novel tri-level joint natural language understanding approach, adding domain, and explicitly exchange semantic information between all levels. This approach enables the use of multi-turn datasets which are a more natural conversational environment than single utterance. We evaluate our model on two multi-turn datasets for which we are the first to conduct joint slot-filling and intent detection. Our model outperforms state-of-the-art joint models in slot filling and intent detection on multi-turn data sets. We provide an analysis of explicit interaction locations between the layers. We conclude that including domain information improves model performance.
Generalizable Pose Estimation Using Implicit Scene Representations

6-DoF pose estimation is an essential component of robotic manipulation pipelines. However, it usually suffers from a lack of generalization to new instances and object types. Most widely used methods learn to infer the object pose in a discriminative setup where the model filters useful information to infer the exact pose of the object. While such methods offer accurate poses, the model does not store enough information to generalize to new objects. In this work, we address the generalization capability of pose estimation using models that contain enough information about the object to render it in different poses. We follow the line of work that inverts neural renderers to infer the pose. We propose i-σSRN to maximize the information flowing from the input pose to the rendered scene and invert them to infer the pose given an input image. Specifically, we extend Scene Representation Networks (SRNs) by incorporating a separate network for density estimation and introduce a new way of obtaining a weighted scene representation. We investigate several ways of initial pose estimates and losses for the neural renderer. Our final evaluation shows a significant improvement in inference performance and speed compared to existing approaches.
Counterfactual Evaluation of Peer-Review Assignment Policies

Peer review assignment algorithms aim to match research papers to suitable expert reviewers, working to maximize the quality of the resulting reviews. A key challenge in designing effective assignment policies is evaluating how changes to the assignment algorithm map to changes in review quality. In this work, we leverage recently proposed policies that introduce randomness in peer-review assignment--in order to mitigate fraud--as a valuable opportunity to evaluate counterfactual assignment policies. Specifically, we exploit how such randomized assignments provide a positive probability of observing the reviews of many assignment policies of interest. To address challenges in applying standard off-policy evaluation methods, such as violations of positivity, we introduce novel methods for partial identification based on monotonicity and Lipschitz smoothness assumptions for the mapping between reviewer-paper covariates and outcomes. We apply our methods to peer-review data from two computer science venues: the TPDP'21 workshop (95 papers and 35 reviewers) and the AAAI'22 conference (8,450 papers and 3,145 reviewers). We consider estimates of (i) the effect on review quality when changing weights in the assignment algorithm, e.g., weighting reviewers' bids vs. textual similarity (between the review's past papers and the submission), and (ii) the"cost of randomization", capturing the difference in expected quality between the perturbed and unperturbed optimal match. We find that placing higher weight on text similarity results in higher review quality and that introducing randomization in the reviewer-paper assignment only marginally reduces the review quality. Our methods for partial identification may be of independent interest, while our off-policy approach can likely find use evaluating a broad class of algorithmic matching systems.
Byte-Level Grammatical Error Correction Using Synthetic and Curated Corpora

Grammatical error correction (GEC) is the task of correcting typos, spelling, punctuation and grammatical issues in text. Approaching the problem as a sequence-to-sequence task, we compare the use of a common subword unit vocabulary and byte-level encoding. Initial synthetic training data is created using an error-generating pipeline, and used for finetuning two subword-level models and one byte-level model. Models are then finetuned further on hand-corrected error corpora, including texts written by children, university students, dyslexic and second-language writers, and evaluated over different error types and error origins. We show that a byte-level model enables higher correction quality than a subword approach, not only for simple spelling errors, but also for more complex semantic, stylistic and grammatical issues. In particular, initial training on synthetic corpora followed by finetuning on a relatively small parallel corpus of real-world errors helps the byte-level model correct a wide range of commonly occurring errors. Our experiments are run for the Icelandic language but should hold for other similar languages, and in particular to morphologically rich ones.
A reduced conjugate gradient basis method for fractional diffusion

This work is on a fast and accurate reduced basis method for solving discretized fractional elliptic partial differential equations (PDEs) of the form $\mathcal{A}^su=f$ by rational approximation. A direct computation of the action of such an approximation would require solving multiple (20$\sim$30) large-scale sparse linear systems. Our method constructs the reduced basis using the first few directions obtained from the preconditioned conjugate gradient method applied to one of the linear systems. As shown in the theory and experiments, only a small number of directions (5$\sim$10) are needed to approximately solve all large-scale systems on the reduced basis subspace. This reduces the computational cost dramatically because: (1) We only use one of the large-scale problems to construct the basis; and (2) all large-scale problems restricted to the subspace have much smaller sizes. We test our algorithms for fractional PDEs on a 3d Euclidean domain, a 2d surface, and random combinatorial graphs. We also use a novel approach to construct the rational approximation for the fractional power function by the orthogonal greedy algorithm (OGA).
Spectral-Domain Method of Moments for the Modal Analysis of Line Waveguides

A rigorous full-wave modal analysis based on the method of moments (MoM) in the spectral domain is presented for line waveguides constituted by two-part impedance planes with arbitrary anisotropic surface impedances. An integral equation is formulated by introducing an auxiliary current sheet on one of the two half-planes and extending the impedance boundary condition of the complementary half-plane to hold on the entire plane. The equation is then discretized with the MoM in the spectral domain, by employing exponentially weighted Laguerre polynomials as entire-domain basis functions and performing a Galerkin testing. Numerical results for both bound and leaky line waves are presented and validated against independent results, obtained for isotropic surface impedances with the analytical Sommerfeld–Maliuzhinets method and for the general anisotropic case with a commercial electromagnetic simulator. The proposed approach is computationally efficient, can accommodate the presence of spatial dispersion, and offers physical insight into the modal propagation regimes.
Structural, Optical and Single-domain Magnetic Features of the Noncollinear Ferrimagnetic Nano-spinel Chromites ACr$_2$O$_4$ (A = Ni, Co, and Mn)

Spinel chromites ACr$_2$O$_4$ with inherent magnetic geometrical frustration usually exhibit a noncollinear ferrimagnetic ground state when A are magnetic ions, with possibly crystallite-size dependent intriguing magnetic features. Here, we report single-domain magnetic properties of ACr$_2$O$_4$ (A = Ni, Co, and Mn) nanocrystals, with an average crystallite size of 18, 15 and 10 nm, exhibiting an optical energy gap of 2.87, 3.05 and 2.9 eV, respectively. The temperature dependence of magnetization indicates the main bulk magnetic transitions with a commonly coexisting spin-glass-like state and finite-size effects on the noncolinear ferrimagnetic transitions. An anomaly observed at Ts = 15, 24 and 10 K is attributed to the bulk magnetic transition to a canted antiferromagnetic state in NiCr$_2$O$_4$ and incommensurate spiral orders in CoCr$_2$O$_4$ and MnCr$_2$O$_4$ NCs, respectively. A further bulk magnetic transition to a commensurate spiral order is observed for CoCr$_2$O$_4$ NCs at a lock-in temperature Tl = 5 K much lower than that reported using bulk samples, while it is completely suppressed in the MnCr$_2$O$_4$ NCs. Finite-size effects and single-domain magnetic behaviors indicated by anomalous temperature-dependences of the coercive field and the hysteresis-loop squareness, mainly driven by a magnetocrystalline anisotropy, are discussed in comparison to results reported using bulk counterparts.
Closing the Gap in High-Risk Pregnancy Care Using Machine Learning and Human-AI Collaboration

High-risk pregnancy (HRP) is a pregnancy complicated by factors that can adversely affect outcomes of the mother or the infant. Health insurers use algorithms to identify members who would benefit from additional clinical support. We aimed to build machine learning algorithms to identify pregnant patients and triage them by risk of complication to assist care management. In this retrospective study, we trained a hybrid Lasso regularized classifier to predict whether a patient is currently pregnant using claims data from 36735 insured members of Independence Blue Cross (IBC), a health insurer in Philadelphia. We then train a linear classifier on a subset of 12,243 members to predict whether a patient will develop gestational diabetes or gestational hypertension. These algorithms were developed in cooperation with the care management team at IBC and integrated into the dashboard. In small user studies with the nurses, we evaluated the impact of integrating our algorithms into their workflow. We find that the proposed model predicts an earlier pregnancy start date for 3.54% (95% CI 3.05-4.00) for patients with complications compared to only using a set of pre-defined codes that indicate the start of pregnancy and never later at the expense of a 5.58% (95% CI 4.05-6.40) false positive rate. The classifier for predicting complications has an AUC of 0.754 (95% CI 0.764-0.788) using data up to the patient's first trimester. Nurses from the care management program expressed a preference for the proposed models over existing approaches. The proposed model outperformed commonly used claim codes for the identification of pregnant patients at the expense of a manageable false positive rate. Our risk complication classifier shows that we can accurately triage patients by risk of complication.
Provably Fast Finite Particle Variants of SVGD via Virtual Particle Stochastic Approximation

Stein Variational Gradient Descent (SVGD) is a popular variational inference algorithm which simulates an interacting particle system to approximately sample from a target distribution, with impressive empirical performance across various domains. Theoretically, its population (i.e, infinite-particle) limit dynamics is well studied but the behavior of SVGD in the finite-particle regime is much less understood. In this work, we design two computationally efficient variants of SVGD, namely VP-SVGD and GB-SVGD, with provably fast finite-particle convergence rates. We introduce the notion of virtual particles and develop novel stochastic approximations of population-limit SVGD dynamics in the space of probability measures, which are exactly implementable using a finite number of particles. Our algorithms can be viewed as specific random-batch approximations of SVGD, which are computationally more efficient than ordinary SVGD. We show that the $n$ particles output by VP-SVGD and GB-SVGD, run for $T$ steps with batch-size $K$, are at-least as good as i.i. d samples from a distribution whose Kernel Stein Discrepancy to the target is at most $O\left(\tfrac{d^{1/3}}{(KT)^{1/6}}\right)$ under standard assumptions. Our results also hold under a mild growth condition on the potential function, which is much weaker than the isoperimetric (e.g. Poincare Inequality) or information-transport conditions (e.g. Talagrand's Inequality $\mathsf{T}_1$) generally considered in prior works. As a corollary, we consider the convergence of the empirical measure (of the particles output by VP-SVGD and GB-SVGD) to the target distribution and demonstrate a double exponential improvement over the best known finite-particle analysis of SVGD. Beyond this, our results present the first known oracle complexities for this setting with polynomial dimension dependence.
Novel machine learning and differentiable programming techniques applied to the VIP-2 underground experiment

In this work, we present novel machine learning and differentiable programming enhanced calibration techniques used to improve the energy resolution of the Silicon Drift Detectors (SDDs) of the VIP-2 underground experiment at the Gran Sasso National Laboratory. We achieve for the first time a full width at half maximum in VIP-2 below 180 eV at 8 keV, improving around 10 eV on the previous state-of-the-art. SDDs energy resolution is a key parameter in the VIP-2 experiment, which is dedicated to searches for physics beyond the standard quantum theory, targeting Pauli exclusion principle violating atomic transitions. Additionally, we show that this method can correct for potential miscalibrations, requiring less fine-tuning with respect to standard methods.