Miracle-dz/UniFAR-FTUs huggingface dataset:
Ultra‐high dimensional variable selection for doubly robust causal inference

Causal inference has been increasingly reliant on observational studies with rich covariate information. To build tractable causal procedures, such as the doubly robust estimators, it is imperative to first extract important features from high or even ultra‐high dimensional data. In this paper, we propose causal ball screening for confounder selection from modern ultra‐high dimensional data sets. Unlike the familiar task of variable selection for prediction modeling, our confounder selection procedure aims to control for confounding while improving efficiency in the resulting causal effect estimate. Previous empirical and theoretical studies suggest excluding causes of the treatment that are not confounders. Motivated by these results, our goal is to keep all the predictors of the outcome in both the propensity score and outcome regression models. A distinctive feature of our proposal is that we use an outcome model‐free procedure for propensity score model selection, thereby maintaining double robustness in the resulting causal effect estimator. Our theoretical analyses show that the proposed procedure enjoys a number of properties, including model selection consistency and pointwise normality. Synthetic and real data analysis show that our proposal performs favorably with existing methods in a range of realistic settings. Data used in preparation of this paper were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database.
Motion Deblurring with an Adaptive Network

In this paper, we address the problem of dynamic scene deblurring in the presence of motion blur. Restoration of images affected by severe blur necessitates a network design with a large receptive field, which existing networks attempt to achieve through simple increment in the number of generic convolution layers, kernel-size, or the scales at which the image is processed. However, increasing the network capacity in this manner comes at the expense of increase in model size and inference speed, and ignoring the non-uniform nature of blur. We present a new architecture composed of spatially adaptive residual learning modules that implicitly discover the spatially varying shifts responsible for non-uniform blur in the input image and learn to modulate the filters. This capability is complemented by a self-attentive module which captures non-local relationships among the intermediate features and enhances the receptive field. We then incorporate a spatiotemporal recurrent module in the design to also facilitate efficient video deblurring. Our networks can implicitly model the spatially-varying deblurring process, while dispensing with multi-scale processing and large filters entirely. Extensive qualitative and quantitative comparisons with prior art on benchmark dynamic scene deblurring datasets clearly demonstrate the superiority of the proposed networks via reduction in model-size and significant improvements in accuracy and speed, enabling almost real-time deblurring.
Robust Multi-Robot Trajectory Optimization Using Alternating Direction Method of Multiplier

Wepropose a variant of alternating direction method of multiplier (ADMM) to solve constrained trajectory optimization problems. Our ADMM framework breaks a joint optimization into small sub-problems, leading to a low iteration cost and decentralized parameter updates. Starting from a collision-free initial trajectory, our method inherits the theoretical properties of primal interior point method (P-IPM), i.e., guaranteed collision avoidance and homotopy preservation throughout optimization, while being orders of magnitude faster. We have analyzed the convergence and evaluated our method for time-optimal multi-UAV trajectory optimizations and simultaneous goal-reaching of multiple robot arms, where we take into consider kinematics-, dynamics-limits, and homotopy-preserving collision constraints. Our method highlights an order of magnitude’s speedup, while generating trajectories of comparable qualities as state-of-the-art P-IPM solver.
Extended field theories are local and have classifying spaces

We show that all extended functorial field theories, both topological and nontopological, are local. We define the smooth (infinity,d)-category of bordisms with geometric data, such as Riemannian metrics or geometric string structures, and prove that it satisfies codescent with respect to the target S, which implies the locality property. We apply this result to construct a classifying space for concordance classes of functorial field theories with geometric data, solving a conjecture of Stolz and Teichner about the existence of such a space. We use our classifying space construction to develop a geometric theory of power operations, following the recent work of Barthel, Berwick-Evans, and Stapleton.
Principal Components Bias in Over-parameterized Linear Models, and its Manifestation in Deep Neural Networks

Recent work suggests that convolutional neural networks of different architectures learn to classify images in the same order. To understand this phenomenon, we revisit the over-parametrized deep linear network model. Our analysis reveals that, when the hidden layers are wide enough, the convergence rate of this model's parameters is exponentially faster along the directions of the larger principal components of the data, at a rate governed by the corresponding singular values. We term this convergence pattern the Principal Components bias (PC-bias). Empirically, we show how the PC-bias streamlines the order of learning of both linear and non-linear networks, more prominently at earlier stages of learning. We then compare our results to the simplicity bias, showing that both biases can be seen independently, and affect the order of learning in different ways. Finally, we discuss how the PC-bias may explain some benefits of early stopping and its connection to PCA, and why deep networks converge more slowly with random labels.
Deep networks on toroids: removing symmetries reveals the structure of flat regions in the landscape geometry

We systematize the approach to the investigation of deep neural network landscapes by basing it on the geometry of the space of implemented functions rather than the space of parameters. Grouping classifiers into equivalence classes, we develop a standardized parameterization in which all symmetries are removed, resulting in a toroidal topology. On this space, we explore the error landscape rather than the loss. This lets us derive a meaningful notion of the flatness of minimizers and of the geodesic paths connecting them. Using different optimization algorithms that sample minimizers with different flatness we study the mode connectivity and relative distances. Testing a variety of state-of-the-art architectures and benchmark datasets, we confirm the correlation between flatness and generalization performance; we further show that in function space flatter minima are closer to each other and that the barriers along the geodesics connecting them are small. We also find that minimizers found by variants of gradient descent can be connected by zero-error paths composed of two straight lines in parameter space, i.e. polygonal chains with a single bend. We observe similar qualitative results in neural networks with binary weights and activations, providing one of the first results concerning the connectivity in this setting. Our results hinge on symmetry removal, and are in remarkable agreement with the rich phenomenology described by some recent analytical studies performed on simple shallow models.
A proof of the Erdős primitive set conjecture

Abstract A set of integers greater than 1 is primitive if no member in the set divides another. Erdős proved in 1935 that the series $f(A) = \sum _{a\in A}1/(a \log a)$ is uniformly bounded over all choices of primitive sets A. In 1986, he asked if this bound is attained for the set of prime numbers. In this article, we answer in the affirmative. As further applications of the method, we make progress towards a question of Erdős, Sárközy and Szemerédi from 1968. We also refine the classical Davenport–Erdős theorem on infinite divisibility chains, and extend a result of Erdős, Sárközy and Szemerédi from 1966.
Simulating Hydrodynamics in Cosmology with CRK-HACC

We introduce CRK-HACC, an extension of the Hardware/Hybrid Accelerated Cosmology Code (HACC), to resolve gas hydrodynamics in large-scale structure formation simulations of the universe. The new framework couples the HACC gravitational N-body solver with a modern smoothed-particle hydrodynamics (SPH) approach called conservative reproducing kernel SPH (CRKSPH). CRKSPH utilizes smoothing functions that exactly interpolate linear fields while manifestly preserving conservation laws (momentum, mass, and energy). The CRKSPH method has been incorporated to accurately model baryonic effects in cosmology simulations—an important addition targeting the generation of precise synthetic sky predictions for upcoming observational surveys. CRK-HACC inherits the codesign strategies of the HACC solver and is built to run on modern GPU-accelerated supercomputers. In this work, we summarize the primary solver components and present a number of standard validation tests to demonstrate code accuracy, including idealized hydrodynamic and cosmological setups, as well as self-similarity measurements.
Pre-integration via Active Subspaces

Pre-integration is an extension of conditional Monte Carlo to quasi-Monte Carlo and randomized quasi-Monte Carlo. It can reduce but not increase the variance in Monte Carlo. For quasi-Monte Carlo it can bring about improved regularity of the integrand with potentially greatly improved accuracy. Pre-integration is ordinarily done by integrating out one of $d$ input variables to a function. In the common case of a Gaussian integral one can also pre-integrate over any linear combination of variables. We propose to do that and we choose the first eigenvector in an active subspace decomposition to be the pre-integrated linear combination. We find in numerical examples that this active subspace pre-integration strategy is competitive with pre-integrating the first variable in the principal components construction on the Asian option where principal components are known to be very effective. It outperforms other pre-integration methods on some basket options where there is no well established default. We show theoretically that, just as in Monte Carlo, pre-integration can reduce but not increase the variance when one uses scrambled net integration. We show that the lead eigenvector in an active subspace decomposition is closely related to the vector that maximizes a less computationally tractable criterion using a Sobol' index to find the most important linear combination of Gaussian variables. They optimize similar expectations involving the gradient. We show that the Sobol' index criterion for the leading eigenvector is invariant to the way that one chooses the remaining $d-1$ eigenvectors with which to sample the Gaussian vector.
Block shuffling learning for Deepfake Detection

Deepfake detection methods based on convolutional neural networks (CNN) have demonstrated high accuracy. \textcolor{black}{However, these methods often suffer from decreased performance when faced with unknown forgery methods and common transformations such as resizing and blurring, resulting in deviations between training and testing domains. } This phenomenon, known as overfitting, poses a significant challenge. To address this issue, we propose a novel block shuffling regularization method. Firstly, our approach involves dividing the images into blocks and applying both intra-block and inter-block shuffling techniques. This process indirectly achieves weight-sharing across different dimensions. Secondly, we introduce an adversarial loss algorithm to mitigate the overfitting problem induced by the shuffling noise. Finally, we restore the spatial layout of the blocks to capture the semantic associations among them. Extensive experiments validate the effectiveness of our proposed method, which surpasses existing approaches in forgery face detection. Notably, our method exhibits excellent generalization capabilities, demonstrating robustness against cross-dataset evaluations and common image transformations. Especially our method can be easily integrated with various CNN models. Source code is available at \href{https://github.com/NoWindButRain/BlockShuffleLearning}{Github}.
Transformers and the Representation of Biomedical Background Knowledge

Specialized transformers-based models (such as BioBERT and BioMegatron) are adapted for the biomedical domain based on publicly available biomedical corpora. As such, they have the potential to encode large-scale biological knowledge. We investigate the encoding and representation of biological knowledge in these models, and its potential utility to support inference in cancer precision medicine—namely, the interpretation of the clinical significance of genomic alterations. We compare the performance of different transformer baselines; we use probing to determine the consistency of encodings for distinct entities; and we use clustering methods to compare and contrast the internal properties of the embeddings for genes, variants, drugs, and diseases. We show that these models do indeed encode biological knowledge, although some of this is lost in fine-tuning for specific tasks. Finally, we analyze how the models behave with regard to biases and imbalances in the dataset.
Policy Optimization for Stochastic Shortest Path

Policy optimization is among the most popular and successful reinforcement learning algorithms, and there is increasing interest in understanding its theoretical guarantees. In this work, we initiate the study of policy optimization for the stochastic shortest path (SSP) problem, a goal-oriented reinforcement learning model that strictly generalizes the finite-horizon model and better captures many applications. We consider a wide range of settings, including stochastic and adversarial environments under full information or bandit feedback, and propose a policy optimization algorithm for each setting that makes use of novel correction terms and/or variants of dilated bonuses (Luo et al., 2021). For most settings, our algorithm is shown to achieve a near-optimal regret bound. One key technical contribution of this work is a new approximation scheme to tackle SSP problems that we call stacked discounted approximation and use in all our proposed algorithms. Unlike the finite-horizon approximation that is heavily used in recent SSP algorithms, our new approximation enables us to learn a near-stationary policy with only logarithmic changes during an episode and could lead to an exponential improvement in space complexity.
High phase-space density gas of NaCs Feshbach molecules

We report on the creation of ultracold gases of bosonic Feshbach molecules of NaCs. The molecules are associated from overlapping gases of Na and Cs using a Feshbach resonance at 864.12(5)G. We characterize the Feshbach resonance using bound state spectroscopy, in conjunction with a coupled-channel calculation. By varying the temperature and atom numbers of the initial atomic mixtures, we demonstrate the association of NaCs gases over a wide dynamic range of molecule numbers and temperatures, reaching 70 nK for our coldest systems and a phase-space density (PSD) near 0.1. This is an important stepping-stone for the creation of degenerate gases of strongly dipolar NaCs molecules in their absolute ground state.
Difference in Differences with Time-Varying Covariates

This paper considers identification and estimation of causal effect parameters from participating in a binary treatment in a difference in differences (DID) setup when the parallel trends assumption holds after conditioning on observed covariates. Relative to existing work in the econometrics literature, we consider the case where the value of covariates can change over time and, potentially, where participating in the treatment can affect the covariates themselves. We propose new empirical strategies in both cases. We also consider two-way fixed effects (TWFE) regressions that include time-varying regressors, which is the most common way that DID identification strategies are implemented under conditional parallel trends. We show that, even in the case with only two time periods, these TWFE regressions are not generally robust to (i) time-varying covariates being affected by the treatment, (ii) treatment effects and/or paths of untreated potential outcomes depending on the level of time-varying covariates in addition to only the change in the covariates over time, (iii) treatment effects and/or paths of untreated potential outcomes depending on time-invariant covariates, (iv) treatment effect heterogeneity with respect to observed covariates, and (v) violations of strong functional form assumptions, both for outcomes over time and the propensity score, that are unlikely to be plausible in most DID applications. Thus, TWFE regressions can deliver misleading estimates of causal effect parameters in a number of empirically relevant cases. We propose both doubly robust estimands and regression adjustment/imputation strategies that are robust to these issues while not being substantially more challenging to implement. JEL Codes: C14, C21, C23
Hierarchies of compatible maps and integrable difference systems

We present two non-equivalent hierarchies of non-Abelian 3D−compatible maps and we provide their Lax pair formulation. These hierarchies are naturally associated with integrable difference systems with variables defined on edges of an elementary cell of the Z graph, that in turn lead to hierarchies of difference systems with variables defined on vertices of the same cell.
Periapsis shifts in dark matter distribution with a dense core

We consider the periapsis shifts in dark matter distribution with a dense core. We model the dark matter distribution as an isotropic gas sphere, the Emden polytropic sphere of index 5 in general relativity. This model has a parameter range where all the energy conditions are satisfied in the entire region. Within the parameter range, the asymptotic analysis for stellar motion allows us to identify two competing effects on the bounded motion: the general-relativistic effect and a local-density effect of matter. Furthermore, using nearly circular bound orbits, we demonstrate that retrograde periapsis shifts occur near the center, where the local-density effect dominates over the general-relativistic effect, whereas prograde periapsis shifts occur in the far region, where the general-relativistic effect dominates over the local-density effect. This result means that a natural explanation for the retrograde periapsis shifts is not the existence of exotic objects (e.g., naked singularities or wormholes) but the local distribution of physically reasonable matter on the stellar orbit. Furthermore, it also implies that the periapsis shift plays a crucial role in distinguishing black hole alternatives, such as dark matter cores, from a pure black hole.
Benchmarking Deep Models for Salient Object Detection

In recent years, deep network-based methods have continuously refreshed state-of-the-art performance on Salient Object Detection (SOD) task. However, the performance discrepancy caused by different implementation details may conceal the real progress in this task. Making an impartial comparison is required for future researches. To meet this need, we construct a general SALient Object Detection (SALOD) benchmark to conduct a comprehensive comparison among several representative SOD methods. Specifically, we re-implement 14 representative SOD methods by using consistent settings for training. Moreover, two additional protocols are set up in our benchmark to investigate the robustness of existing methods in some limited conditions. In the first protocol, we enlarge the difference between objectness distributions of train and test sets to evaluate the robustness of these SOD methods. In the second protocol, we build multiple train subsets with different scales to validate whether these methods can extract discriminative features from only a few samples. In the above experiments, we find that existing loss functions usually specialized in some metrics but reported inferior results on the others. Therefore, we propose a novel Edge-Aware (EA) loss that promotes deep networks to learn more discriminative features by integrating both pixel- and image-level supervision signals. Experiments prove that our EA loss reports more robust performances compared to existing losses.
Gradient boosting machines and careful pre-processing work best: ASHRAE Great Energy Predictor III lessons learned

The ASHRAE Great Energy Predictor III (GEPIII) competition was held in late 2019 as one of the largest machine learning competitions ever held focused on building performance. It was hosted on the Kaggle platform and resulted in 39,402 prediction submissions, with the top five teams splitting $25,000 in prize money. This paper outlines lessons learned from participants, mainly from teams who scored in the top 5% of the competition. Various insights were gained from their experience through an online survey, analysis of publicly shared submissions and notebooks, and the documentation of the winning teams. The top-performing solutions mostly used ensembles of Gradient Boosting Machine (GBM) tree-based models, with the LightGBM package being the most popular. The survey participants indicated that the preprocessing and feature extraction phases were the most important aspects of creating the best modeling approach. All the survey respondents used Python as their primary modeling tool, and it was common to use Jupyter-style Notebooks as development environments. These conclusions are essential to help steer the research and practical implementation of building energy meter prediction in the future.
Preliminary Tests of High Speed PIV Measurements in a Ludwieg-Tube Supersonic Wind Tunnel

Preliminary Experimental results are presented on the application of Particle image velocimetry (PIV) in a Mach 3 flow produced in a Ludwieg tube facility. The transitory nature of the flow places high demands on the required hardware performance and presents a challenge to the seeding process. A Nd:YLF laser with a pulse rate of 3 KHz and a CMOS camera with interframe time of 2 µs along with TiO2 nanoparticles were selected, allowing the capture of the nozzle starting process. Best seeding results were achieved by filling the entire charge tube with a mixture of seeding particles and charge gas prior to tunnel startup. The starting shocks unsteady motion and the large scale structure were well resolved, and agreement was found between the velocity history from PIV and the wall pressure signal. The starting normal shock wave was used to determine the seeding particle performance parameters. In addition, an evaluation of the boundary layer profile developed over an axisymmetric body at Reynlods number of 2×10 7 was obtained and compared favorably to CFD results.
A search for technosignatures toward the Galactic Centre at 150 MHz

Abstract This paper is the fourth in a series of low-frequency searches for technosignatures. Using the Murchison Widefield Array over two nights, we integrate 7 h of data toward the Galactic Centre (centred on the position of Sagittarius $\mathrm{A}^{*}$ ) with a total field-of-view of $200\,\mathrm{deg}^{2}$ . We present a targeted search toward 144 exoplanetary systems, at our best yet angular resolution (75 arcsec). This is the first technosignature search at a central frequency of 155 MHz toward the Galactic Centre (our previous central frequencies have been lower). A blind search toward in excess of 3 million stars toward the Galactic Centre and Galactic bulge is also completed, placing an equivalent isotropic power limit $<\! 1.1\times10^{19}\,\mathrm{W}$ at the distance to the Galactic Centre. No plausible technosignatures are detected.