Miracle-dz/UniFAR-FTUs huggingface dataset:
PIGEON: Optimizing CUDA Code Generator for End-to-End Training and Inference of Relational Graph Neural Networks

Relational graph neural networks (RGNNs) are graph neural networks (GNNs) with dedicated structures for modeling the different types of nodes and/or edges in heterogeneous graphs. While RGNNs have been increasingly adopted in many real-world applications due to their versatility and accuracy, they pose performance and system design challenges due to their inherent computation patterns, gap between the programming interface and kernel APIs, and heavy programming efforts in optimizing kernels caused by their coupling with data layout and heterogeneity. To systematically address these challenges, we propose Pigeon, a novel two-level intermediate representation (IR) and its code generator framework, that (a) represents the key properties of the RGNN models to bridge the gap between the programming interface and kernel APIs, (b) decouples model semantics, data layout, and operators-specific optimization from each other to reduce programming efforts, (c) expresses and leverages optimization opportunities in inter-operator transforms, data layout, and operator-specific schedules. By building on one general matrix multiply (GEMM) template and a node/edge traversal template, Pigeon achieves up to 7.8x speed-up in inference and 5.6x speed-up in training compared with the state-of-the-art public systems in select models, i.e., RGCN, RGAT, HGT, when running heterogeneous graphs provided by Deep Graph Library (DGL) and Open Graph Benchmark (OGB). Pigeon also triggers fewer out-of-memory (OOM) errors. In addition, we propose linear operator fusion and compact materialization to further accelerate the system by up to 2.2x.
Student resources for learning introductory physics

With good reason, physics education research has focused almost exclusively on student difficulties and misconceptions. This work has been productive for curriculum development as well as in motivating the physics teaching community to examine and reconsider methods and assumptions, but it is limited in what it can tell us about student knowledge and learning. This article reviews perspectives on student resources for learning, with an emphasis on the practical benefits to be gained for instruction.
Block belief propagation algorithm for two-dimensional tensor networks

Belief propagation is a well-studied algorithm for approximating local marginals of multivariate probability distribution over complex networks, while tensor network states are powerful tools for quantum and classical many-body problems. Building on a recent connection between the belief propagation algorithm and the problem of tensor network contraction, we propose a block belief propagation algorithm for contracting two-dimensional tensor networks and approximating the ground state of $2D$ systems. The advantages of our method are three-fold: 1) the same algorithm works for both finite and infinite systems; 2) it allows natural and efficient parallelization; 3) given its flexibility it would allow to deal with different unit cells. As applications, we use our algorithm to study the $2D$ Heisenberg and transverse Ising models, and show that the accuracy of the method is on par with state-of-the-art results.
Multiple imputation for propensity score analysis with covariates missing at random: some clarity on within and across methods

In epidemiology and social sciences, propensity score methods are popular for estimating treatment effects using observational data, and multiple imputation is popular for handling covariate missingness. However, how to appropriately use multiple imputation for propensity score analysis is not completely clear. This paper aims to bring clarity on the consistency (or lack thereof) of methods that have been proposed, focusing on the within approach (where the effect is estimated separately in each imputed dataset and then the multiple estimates are combined) and the across approach (where typically propensity scores are averaged across imputed datasets before being used for effect estimation). We show that the within method is valid and can be used with any causal effect estimator that is consistent in the full-data setting. Existing across methods are inconsistent, but a different across method that averages the inverse probability weights across imputed datasets is consistent for propensity score weighting. We also comment on methods that rely on imputing a function of the missing covariate rather than the covariate itself, including imputation of the propensity score and of the probability weight. Based on consistency results and practical flexibility, we recommend generally using the standard within method. Throughout, we provide intuition to make the results meaningful to the broad audience of applied researchers.
Expected Gradients of Maxout Networks and Consequences to Parameter Initialization

We study the gradients of a maxout network with respect to inputs and parameters and obtain bounds for the moments depending on the architecture and the parameter distribution. We observe that the distribution of the input-output Jacobian depends on the input, which complicates a stable parameter initialization. Based on the moments of the gradients, we formulate parameter initialization strategies that avoid vanishing and exploding gradients in wide networks. Experiments with deep fully-connected and convolutional networks show that this strategy improves SGD and Adam training of deep maxout networks. In addition, we obtain refined bounds on the expected number of linear regions, results on the expected curve length distortion, and results on the NTK.
Cooperation Learning Enhanced Colonic Polyp Segmentation Based on Transformer-CNN Fusion

— T raditional segmentation methods for colonic polyps are mainly designed based on low-level features. They could not accurately extract the location of small colonic polyps. Although the existing deep learning methods can improve the segmentation accuracy, their effects are still unsatisfied. To meet the above challenges, we propose a hybrid network called Fusion-Transformer-HardNetMSEG (i.e., Fu-TransHNet) in this study. Fu-TransHNet uses deep learning of different mechanisms to fuse each other and is enhanced with multi-view collaborative learning techniques. Firstly, the Fu-TransHNet utilizes the Transformer branch and the CNN branch to realize the global feature learning and local feature learning, respectively. Secondly, a fusion module is designed to integrate the features from two branches. The fusion module consists of two parts: 1) the Global-Local Feature Fusion (GLFF) part and 2) the Dense Fusion of Multi-scale features (DFM) part. The former is built to compensate the feature information mission from two branches at the same scale; the latter is constructed to enhance the feature representation. Thirdly, the above two branches and fusion modules utilize multi-view cooperative learning techniques to obtain their respective weights that denote their importance and then make a final decision comprehensively. Experimental results showed that the Fu-TransHNet network was superior to the existing methods on five widely used benchmark datasets. In particular, on the ETIS-LaribPolypDB dataset containing many small-target colonic polyps, the mDice obtained by Fu-TransHNet were 12.4% and 6.2% higher than the state-of-the-art methods HardNet-MSEG and TransFuse-s, respectively.
Ranking with submodular functions on the fly

Maximizing submodular functions have been studied extensively for a wide range of subset-selection problems. However, much less attention has been given to the role of submodularity in sequence-selection and ranking problems. A recently-introduced framework, named \emph{maximum submodular ranking} (MSR), tackles a family of ranking problems that arise naturally when resources are shared among multiple demands with different budgets. For example, the MSR framework can be used to rank web pages for multiple user intents. In this paper, we extend the MSR framework in the streaming setting. In particular, we consider two different streaming models and we propose practical approximation algorithms. In the first streaming model, called \emph{function arriving}, we assume that submodular functions (demands) arrive continuously in a stream, while in the second model, called \emph{item arriving}, we assume that items (resources) arrive continuously. Furthermore, we study the MSR problem with additional constraints on the output sequence, such as a matroid constraint that can ensure fair exposure among items from different groups. These extensions significantly broaden the range of problems that can be captured by the MSR framework. On the practical side, we develop several novel applications based on the MSR formulation, and empirically evaluate the performance of the proposed~methods.
Generalized Zurek's bound on the cost of an individual classical or quantum computation

We consider the minimal thermodynamic cost of an individual computation, where a single input x is mapped to a single output y. In prior work, Zurek proposed that this cost was given by K(x|y), the conditional Kolmogorov complexity of x given y (up to an additive constant that does not depend on x or y). However, this result was derived from an informal argument, applied only to deterministic computations, and had an arbitrary dependence on the choice of protocol (via the additive constant). Here we use stochastic thermodynamics to derive a generalized version of Zurek's bound from a rigorous Hamiltonian formulation. Our bound applies to all quantum and classical processes, whether noisy or deterministic, and it explicitly captures the dependence on the protocol. We show that K(x|y) is a minimal cost of mapping x to y that must be paid using some combination of heat, noise, and protocol complexity, implying a trade-off between these three resources. Our result is a kind of "algorithmic fluctuation theorem" with implications for the relationship between the second law and the Physical Church-Turing thesis.
Quantifying the dynamics of peak innovation in scientific careers

We examine the innovation of researchers with long-lived careers in Computer Science and Physics. Despite the epistemological differences between such disciplines, we consistently ﬁnd that a researcher’s most innovative publication occurs earlier than expected if innovation were distributed at random across the sequence of publications in their career, and is accompanied by a peak year in which researchers publish other work which is more innovative than average. Through a series of linear models, we show that the innovation achieved by a researcher during their peak year is higher when it is preceded by a long period of low productivity. These ﬁndings are in stark contrast with the dynamics of academic impact, which researchers are incentivised to pursue through high productivity and incremental – less innovative – work by the currently prevalent paradigms of scientiﬁc evaluation.
Computational Assessment of Hyperpartisanship in News Titles

We ﬁrst adopt a human-guided machine learning framework to develop a new dataset for hyperpartisan news title detection with 2,200 manually labeled and 1.8 million machine- labeled titles that were posted from 2014 to the present by nine representative media organizations across three media bias groups - Left , Central , and Right in an active learning manner. The ﬁne-tuned transformer-based language model achieves an overall accuracy of 0.84 and an F 1 score of 0.78 on an external validation set. Next, we conduct a computational analysis to quantify the extent and dynamics of partisanship in news titles. While some aspects are as expected, our study reveals new or nuanced differences between the three media groups. We ﬁnd that overall the Right media tends to use proportionally more hyperpartisan titles. Roughly around the 2016 Presidential Election, the proportions of hyperpartisan titles increased in all media bias groups where the relative increase in the proportion of hyperpartisan titles of the Left media was the most. We identify three major topics including foreign issues , political systems , and so- cietal issues that are suggestive of hyperpartisanship in news titles using logistic regression models and the Shapley val- ues. Through an analysis of the topic distribution, we ﬁnd that societal issues gradually receive more attention from all me- dia groups. We further apply a lexicon-based language analysis tool to the titles of each topic and quantify the linguistic distance between any pairs of the three media groups. Three distinct patterns are discovered. The Left media is linguistically more different from Central and Right in terms of foreign issues. For political systems, the linguistic distance between the three media groups becomes smaller over recent years. In addition, a seasonal pattern where linguistic differ- ence is associated with elections is observed for societal issues.
Mobility Data in Operations: The Facility Location Problem

The recent large scale availability of mobility data, which captures individual mobility patterns, poses novel operational problems that are exciting and challenging. Motivated by this, we introduce and study a variant of the (cost-minimization) facility location problem where each individual is endowed with two locations (hereafter, her home and work locations), and the connection cost is the minimum distance between any of her locations and its closest facility. We design a polynomial-time algorithm whose approximation ratio is at most 2.497. We complement this positive result by showing that the proposed algorithm is at least a 2.428-approximation, and there exists no polynomial-time algorithm with approximation ratio $2-\epsilon$ under UG-hardness. We further extend our results and analysis to the model where each individual is endowed with K locations. Finally, we conduct numerical experiments over both synthetic data and US census data (for NYC, greater LA, greater DC, Research Triangle) and evaluate the performance of our algorithms.
Anisotropic Comptonization in thermal plasmas : spectral distribution in plane-parallel geometry

We present a semi-analytical formulation of the problem of inverse Compton scattering of low-frequency radiation in a hot thermal plasma. The scattering medium is assumed to have a plan-parallel geometry, and an input radiation field is localized on one side of the slab. The angular and energy distributions of the emitted radiation are computed taking into account the effects due to the anisotropy of the input photons. We show how the Comptonized spectrum depends on three parameters, namely the optical depth of the medium, the temperature of thermal electrons, and the viewing angle. The spectral shape is approximately, but not exactly, a broken power law
Black Hole Ultracompact X-Ray Binaries: Galactic Low-frequency Gravitational Wave Sources

In the Galaxy, close binaries with compact objects are important low-frequency gravitational wave (GW) sources. As potential low-frequency GW sources, neutron star/white dwarf (WD) ultracompact X-ray binaries (UCXBs) have been investigated extensively. Using the Modules for Experiments in Stellar Astrophysics code, we systematically explored the evolution of black hole (BH)-main-sequence star (MS) binaries to determine whether their descendants can be detected by space-borne GW detectors. Our simulations showed that BH-MS binaries with an initial orbital period less than the bifurcation period can evolve into BH UCXBs that can be detected by LISA. Such an evolutionary channel would form compact mass-transferring BH-WD systems rather than detached BH-WD systems. The calculated X-ray luminosities of BH UCXBs that can be detected by LISA at a distance d = 1 kpc are ∼1033–1035 erg s−1 (∼1034–1035 erg s−1 for d = 10 kpc); hence, it is possible to detect their electromagnetic counterparts. It is worth emphasizing that only some BH-MS systems with an initial orbital period very close to the bifurcation period can evolve toward low-frequency GW sources whose chirp masses can be measured. The maximum GW frequency of BH UCXBs forming via the BH-MS pathway is about 3 mHz, which is smaller than the minimum GW frequency (6.4 mHz) of mass-transferring BH-WDs originating from a dynamic process. Furthermore, we obtain an initial parameter space (donor-star masses and orbital periods) of progenitors of BH UCXB-GW sources, which can be applied to future population synthesis simulations. By a rough estimation, we predict that LISA would only be able to detect a few BH UCXB-GW sources formed by the BH-MS channel.
Robust Max Entrywise Error Bounds for Tensor Estimation From Sparse Observations via Similarity-Based Collaborative Filtering

Consider the task of estimating a 3-order <inline-formula> <tex-math notation="LaTeX">$n \times n \times n$ </tex-math></inline-formula> tensor from noisy observations of randomly chosen entries in the sparse regime. We introduce a similarity based collaborative filtering algorithm for estimating a tensor from sparse observations and argue that it achieves sample complexity that nearly matches the conjectured computationally efficient lower bound on the sample complexity for the setting of low-rank tensors. Our algorithm uses the matrix obtained from the flattened tensor to compute similarity, and estimates the tensor entries using a nearest neighbor estimator. We prove that the algorithm recovers a finite rank tensor with maximum entry-wise error (MEE) and mean-squared-error (MSE) decaying to 0 as long as each entry is observed independently with probability <inline-formula> <tex-math notation="LaTeX">$p = \Omega (n^{-3/2 + \kappa })$ </tex-math></inline-formula> for any arbitrarily small <inline-formula> <tex-math notation="LaTeX">$ \kappa > 0$ </tex-math></inline-formula>. More generally, we establish robustness of the estimator, showing that when arbitrary noise bounded by <inline-formula> <tex-math notation="LaTeX">$ \boldsymbol { \varepsilon }\geq 0$ </tex-math></inline-formula> is added to each observation, the estimation error with respect to MEE and MSE degrades by <inline-formula> <tex-math notation="LaTeX">${\sf poly}(\boldsymbol { \varepsilon })$ </tex-math></inline-formula>. Consequently, even if the tensor may not have finite rank but can be approximated within <inline-formula> <tex-math notation="LaTeX">$ \boldsymbol { \varepsilon }\geq 0$ </tex-math></inline-formula> by a finite rank tensor, then the estimation error converges to <inline-formula> <tex-math notation="LaTeX">${\sf poly}(\boldsymbol { \varepsilon })$ </tex-math></inline-formula>. Our analysis sheds insight into the conjectured sample complexity lower bound, showing that it matches the connectivity threshold of the graph used by our algorithm for estimating similarity between coordinates.
Deep nonparametric regression on approximate manifolds: Nonasymptotic error bounds with polynomial prefactors

We study the properties of nonparametric least squares regression using deep neural networks. We derive non-asymptotic upper bounds for the prediction error of the empirical risk minimizer of feedforward deep neural regression. Our error bounds achieve minimax optimal rate and significantly improve over the existing ones in the sense that they depend polynomially on the dimension of the predictor, instead of exponentially on dimension. We show that the neural regression estimator can circumvent the curse of dimensionality under the assumption that the predictor is supported on an approximate low-dimensional manifold or a set with low Minkowski dimension. We also establish the optimal convergence rate under the exact manifold support assumption. We investigate how the prediction error of the neural regression estimator depends on the structure of neural networks and propose a notion of network relative efficiency between two types of neural networks, which provides a quantitative measure for evaluating the relative merits of different network structures. To establish these results, we derive a novel approximation error bound for the H\"older smooth functions with a positive smoothness index using ReLU activated neural networks, which may be of independent interest. Our results are derived under weaker assumptions on the data distribution and the neural network structure than those in the existing literature.
Similar Signatures of Coplanar Gas Inflow and Disk Warps in Galactic Gas Kinematic Maps

Hydrodynamic simulations suggest that galactic gas disks form when coplanar gas spirals into the inner regions of the disk. We recently presented a simple modified accretion disk model of viscous galactic disks in which star formation is fed by a radial flow of gas. However, little observational evidence has been presented for such inflows, which are expected to be only a few kilometers per second in the central regions of the disk, i.e., within four disk scale lengths, but could reach of order 50–100 km s−1 in the very outer disk. The effects of systematic inflow on the 2D velocity field are examined and it is shown that these are quite similar to those produced by geometric warps of the disks, with twist distortions of both the kinematic major and minor axes. This makes it potentially difficult to distinguish between these in practice. By comparing the handedness of the observed twisting of the kinematic axes and of the spiral arms for a sample of nearby galaxies, we find (assuming that the spiral arms are generally trailing) that the effects of warps are in fact likely to dominate over the effects of radial inflows. However, the common practice of treating these twist distortions of the kinematic major and minor axes as being due only to warps can lead, for galaxies of low-to-intermediate inclinations, to substantial underestimates of any systematic inflow.
Counting graphic sequences via integrated random walks

Given an integer $n$, let $G(n)$ be the number of integer sequences $n-1\ge d_1\ge d_2\ge\dotsb\ge d_n\ge 0$ that are the degree sequence of some graph. We show that $G(n)=(c+o(1))4^n/n^{3/4}$ for some constant $c>0$, improving both the previously best upper and lower bounds by a factor of $n^{1/4}$ (up to polylog-factors). Additionally, we answer a question of Royle, extend the values of $n$ for which the exact value of $G(n)$ is known from $n\le290$ to $n\le 1651$ and determine the asymptotic probability that the integral of a (lazy) simple symmetric random walk bridge remains non-negative.
Steady free convection in a porous medium heated from below

This is an experimental and numerical study of steady free convection in a porous medium, a system dominated by a single non-linear process, the advection of heat. The paper presents results on three topics: (1) a system uniformly heated from below, for which the flow is cellular, as in the analogous Bénard-Rayleigh flows, (ii) the role of end-effects, and (iii) the role of mass discharge. Measurements of heat transfer are used to establish further the validity of the numerical scheme proposed by the author (1966a), while the other flows allow a more extensive study of the numerical scheme under various boundary conditions. The results are very satisfactory even though only moderately non-linear problems can be treated at present. The main new results are as follows. For the Rayleigh-type flow, above a critical Rayleigh number of about 40, the heat transferred across the layer is proportional to the square of the temperature difference across the layer and is independent of the thermal conductivity of the medium or the depth of the layer. This result is modified when the boundary-layer thickness is comparable to the grain size of the medium. The investigation of end-effects reveals variations in horizontal wave-number and a pronounced hysteresis and suggests an alternative explanation of some observations by Malkus (1954).