Miracle-dz/UniFAR-FTUs huggingface dataset:
Function Approximation via Sparse Random Features

Random feature methods have been successful in various machine learning tasks, are easy to compute, and come with theoretical accuracy bounds. They serve as an alternative approach to standard neural networks since they can represent similar function spaces without a costly training phase. However, for accuracy, random feature methods require more measurements than trainable parameters, limiting their use for data-scarce applications or problems in scientiﬁc machine learning. This paper introduces the sparse random feature method that learns parsimonious random feature models utilizing techniques from compressive sensing. We provide uniform bounds on the approximation error for functions in a reproducing kernel Hilbert space depending on the number of samples and the distribution of features. The error bounds improve with additional structural conditions, such as coordinate sparsity, compact clusters of the spectrum, or rapid spectral decay. We show that the sparse random feature method outperforms shallow networks for well-structured functions and applications to scientiﬁc machine learning tasks.
Wide Graph Neural Networks: Aggregation Provably Leads to Exponentially Trainability Loss

Graph convolutional networks (GCNs) and their variants have achieved great success in dealing with graph-structured data. However, it is well known that deep GCNs will suffer from over-smoothing problem, where node representations tend to be indistinguishable as we stack up more layers. Although extensive research has conﬁrmed this prevailing understanding, few theoretical analyses have been conducted to study the expressivity and trainability of deep GCNs. In this work, we demonstrate these characterizations by studying the Gaussian Process Kernel (GPK) and Graph Neural Tangent Kernel (GNTK) of an inﬁnitely-wide GCN, corresponding to the analysis on expressivity and trainability, respectively. We ﬁrst prove the expressivity of inﬁnitely-wide GCNs decaying at an exponential rate by applying the mean-ﬁeld theory on GPK. Besides, we formulate the asymptotic behaviors of GNTK in the large depth, which enables us to reveal the dropping trainability of wide and deep GCNs at an exponential rate. Additionally, we extend our theoretical framework to analyze residual connection-resemble techniques. We found that these techniques can mildly mitigate exponential decay, but they failed to overcome it fundamentally. Finally, all theoretical results in this work are corroborated experimentally on a variety of graph-structured datasets.
A Gaussian Process Regression Reveals No Evidence for Planets Orbiting Kapteyn’s Star

Radial–velocity (RV) planet searches are often polluted by signals caused by gas motion at the star’s surface. Stellar activity can mimic or mask changes in the RVs caused by orbiting planets, resulting in false positives or missed detections. Here we use Gaussian process regression to disentangle the contradictory reports of planets versus rotation artifacts from Kapteyn’s star. To model rotation, we use joint quasiperiodic kernels for the RV and Hα signals, requiring that their periods and correlation timescales be the same. We find that the rotation period of Kapteyn’s star is 125 days, while the characteristic active-region lifetime is 694 days. Adding a planet to the RV model produces a best-fit orbital period of 100 yr, or 10 times the observing time baseline, indicating that the observed RVs are best explained by star rotation only. We also find no significant periodic signals in residual RV data sets constructed by subtracting off realizations of the best-fit rotation model and conclude that both previously reported “planets” are artifacts of the star’s rotation and activity. Our results highlight the pitfalls of using sinusoids to model quasiperiodic rotation signals.
Deep Graph Structure Learning for Robust Representations: A Survey

Graph Neural Networks (GNNs) are widely used for analyzing graph-structured data. Most GNN methods are highly sensitive to the quality of graph structures and usually require a perfect graph structure for learning informative embeddings. However, the pervasiveness of noise in graphs necessitates learning robust representations for real-world problems. To improve the robustness of GNN models, many studies have been proposed around the central concept of Graph Structure Learning (GSL), which aims to jointly learn an optimized graph structure and corresponding representations. Towards this end, in the presented survey, we broadly review recent progress of GSL methods for learning robust representations. Speciﬁcally, we ﬁrst formulate a general paradigm of GSL, and then review state-of-the-art methods classiﬁed by how they model graph structures, followed by applications that incorporate the idea of GSL in other graph tasks. Finally, we point out some issues in current studies and discuss future directions.
Deciphering Star Cluster Evolution by Shape Morphology

We analyze the morphological evolution of open clusters and provide shape parameters for 265 open clusters. The results show that the overall shape of sample clusters becomes more elliptical as they grow older, while their core remains circular or slightly trending to circular. There is a negative correlation of the ellipticities with the number of members of the sample clusters. A significant negative correlation between the overall ellipticities and masses is also detected for the sample clusters with log(age/year) ≥8, suggesting that the overall shapes of the clusters are possibly influenced by the number of members and masses, in addition to the external forces and the surrounding environment. For most young sample clusters, the radial stratification degree of the short-axis direction is greater than that of the long, implying that the radial stratification degree in the two directions within the young sample cluster may be unevenly affected by an internal evolutionary process. Older sample clusters exhibit lower stratification in the tangential direction, which possibly means those clusters may continue to survive for a long time at a low level of stratification. Our analysis shows that the overall shape of the sample clusters may be more susceptible to the influence of Galactic tides toward the Galactic center than the shear forces embedded in Galactic differential rotation. By analyzing the distribution of the ages and number of members of star clusters, we suggest that NGC 6791 may originate from superclusters.
Preference-based Learning of Reward Function Features

Preference-based learning of reward functions, where the reward function is learned using comparison data, has been well studied for complex robotic tasks such as autonomous driving. Existing algorithms have focused on learning reward functions that are linear in a set of trajectory features. The features are typically hand-coded, and preference-based learning is used to determine a particular user's relative weighting for each feature. Designing a representative set of features to encode reward is challenging and can result in inaccurate models that fail to model the users' preferences or perform the task properly. In this paper, we present a method to learn both the relative weighting among features as well as additional features that help encode a user's reward function. The additional features are modeled as a neural network that is trained on the data from pairwise comparison queries. We apply our methods to a driving scenario used in previous work and compare the predictive power of our method to that of only hand-coded features. We perform additional analysis to interpret the learned features and examine the optimal trajectories. Our results show that adding an additional learned feature to the reward model enhances both its predictive power and expressiveness, producing unique results for each user.
First Detection of Hydroxyl Radical Emission from an Exoplanet Atmosphere: High-dispersion Characterization of WASP-33b Using Subaru/IRD

We report the first detection of a hydroxyl radical (OH) emission signature in the planetary atmosphere outside the solar system, in this case, in the dayside of WASP-33b. We analyze high-resolution near-infrared emission spectra of WASP-33b taken using the InfraRed Doppler spectrograph on the 8.2 m Subaru telescope. The telluric and stellar lines are removed using a detrending algorithm, SysRem. The residuals are then cross-correlated with OH and H2O planetary spectrum templates produced using several different line lists. We check and confirm the accuracy of OH line lists by cross-correlating with the spectrum of GJ 436. As a result, we detect the emission signature of OH at K p of km s−1 and v sys of −0.3 km s−1 with a signal-to-noise ratio (S/N) of 5.4 and a significance of 5. 5σ. Additionally, we marginally detect H2O emission in the H-band with an S/N of 4.0 and a significance of 5. 2σ using the POKAZATEL line list. However, no significant signal is detected using the HITEMP 2010, which might be due to differences in line positions and strengths, as well as the incompleteness of the line lists. Nonetheless, this marginal detection is consistent with the prediction that H2O is mostly thermally dissociated in the upper atmosphere of the ultra-hot Jupiters. Therefore, along with CO, OH is expected to be one of the most abundant O-bearing molecules in the dayside atmosphere of ultra-hot Jupiters and should be considered when studying their atmospheres.
Accounting for Variance in Machine Learning Benchmarks

Strong empirical evidence that one machine-learning algorithm A outperforms another one B ideally calls for multiple trials optimizing the learning pipeline over sources of variation such as data sampling, data augmentation, parameter initialization, and hyperparameters choices. This is prohibitively expensive, and corners are cut to reach conclusions. We model the whole benchmarking process, revealing that variance due to data sampling, parameter initialization and hyperparameter choice impact markedly the results. We analyze the predominant comparison methods used today in the light of this variance. We show a counter-intuitive result that adding more sources of variation to an imperfect estimator approaches better the ideal estimator at a 51 times reduction in compute cost. Building on these results, we study the error rate of detecting improvements, on five different deep-learning tasks/architectures. This study leads us to propose recommendations for performance comparisons.
Wireless Coded Caching Can Overcome the Worst-User Bottleneck by Exploiting Finite File Sizes

We address the worst-user bottleneck of wireless coded caching, which is known to severely diminish cache-aided multicasting gains due to the fundamental worst-channel limitation of multicasting. We consider the quasi-static Rayleigh fading Broadcast Channel, for which we first show that the effective coded caching gain of the standard XOR-based coded-caching scheme completely vanishes in the low signal-to-noise ratio (SNR) regime. Then, we reveal that this collapse is not intrinsic to coded caching. We do so by presenting a novel scheme that can fully recover the coded caching gains by capitalizing on one aspect that has remained unexploited to date: the shared side information brought about by the effectively unavoidable file-size constraint. As a consequence, the worst-user effect is dramatically ameliorated, as it is substituted by a much more subtle worst-group-of-users effect, where the suggested grouping is fixed, and it is decided before the channel or the demands are known. Furthermore, the theoretical gains are completely recovered as the number of users increases, and this is done without any user selection technique. We analyze the rate performance of the proposed scheme and derive approximations which prove to be very precise. Importantly, this novel approach can be translated to other coded caching schemes and scenarios, including decentralized scenarios.
Theory of Evolutionary Spectra for Heteroskedasticity and Autocorrelation Robust Inference in Possibly Misspecified and Nonstationary Models

We develop a theory of evolutionary spectra for heteroskedasticityand autocorrelationrobust (HAR) inference when the data may not satisfy second-order stationarity. Nonstationarity is a common feature of economic time series which may arise either from model misspecification or parameter variation. In such a context, the main theory that supports classical heteroskedasticity and autocorrelation consistent (HAC) estimators is not applicable. F and t-tests standardized by classical HAC estimators then may display size distortions and other undesirable properties such as little or no power. We introduce a class of nonstationary stochastic processes that have a time-varying spectral representation which evolves continuously except at a finite number of time points and presents a new positive semidefinite HAC estimator that is consistent for the covariance matrix of any time series within this class. The HAC estimator applies two smoothing procedures. One is over the autocovariance lag order—akin to classical HAC estimators—and the other is over time. The latter element is not shared by the classical HAC estimators but it is crucial because the covariance structure of economic time series can evolve over time. We show the consistency of the estimator and obtain an optimal HAC estimator under the mean-squared error (MSE) criterion. The optimal HAC estimator uses the Quadratic Spectral (QS) kernel for smoothing over autocovariance lag order and a quadratic-type kernel for smoothing over time. We propose a data-dependent procedure based on a “plug-in” approach that determines the bandwidth parameters for given kernels and a given sample size. Overall, F and t-tests standardized by the proposed HAC estimator control the size well and have good power. In those empirically relevant situations in which test statistics standardized by classical HAC estimators have little or no statistical power, the proposed HAC estimator leads to test statistics that have good power so that HAR inference is then meaningful. ∗I am grateful to Pierre Perron for his continuous support and advice. I wish to thank Jean-Jacques Forneron, Hiroaki Kaido and Zhongjun Qu for helpful comments and suggestions. I also thank Marine Carrasco, Giuseppe Cavaliere, Raffaella Giacomini, Simon Lee, Hashem Pesaran, Jing Cynthia Wu and Yohei Yamamoto for useful discussions, and seminar participants at the BC-BU Econometrics Workshop for comments. †Department of Economics, Boston University, 270 Bay State Rodad, Boston, MA 02215, US. Email: acasini@bu.edu. Web Homepage: http://alessandro-casini.com.
Barlow Twins: Self-Supervised Learning via Redundancy Reduction

Self-supervised learning (SSL) is rapidly closing the gap with supervised methods on large computer vision benchmarks. A successful approach to SSL is to learn embeddings which are invariant to distortions of the input sample. However, a recurring issue with this approach is the existence of trivial constant solutions. Most current methods avoid such solutions by careful implementation details. We propose an objective function that naturally avoids collapse by measuring the cross-correlation matrix between the outputs of two identical networks fed with distorted versions of a sample, and making it as close to the identity matrix as possible. This causes the embedding vectors of distorted versions of a sample to be similar, while minimizing the redundancy between the components of these vectors. The method is called Barlow Twins, owing to neuroscientist H. Barlow's redundancy-reduction principle applied to a pair of identical networks. Barlow Twins does not require large batches nor asymmetry between the network twins such as a predictor network, gradient stopping, or a moving average on the weight updates. Intriguingly it benefits from very high-dimensional output vectors. Barlow Twins outperforms previous methods on ImageNet for semi-supervised classification in the low-data regime, and is on par with current state of the art for ImageNet classification with a linear classifier head, and for transfer tasks of classification and object detection.
Impact of Massive Binary Star and Cosmic Evolution on Gravitational Wave Observations I: Black Hole – Neutron Star Mergers

Mergers of black hole-neutron star (BHNS) binaries have now been observed by GW detectors with the recent announcement of GW200105 and GW200115. Such observations not only provide confirmation that these systems exist, but will also give unique insights into the death of massive stars, the evolution of binary systems and their possible association with gamma-ray bursts, r-process enrichment and kilonovae. Here we perform binary population synthesis of isolated BHNS systems in order to present their merger rate and characteristics for ground-based GW observatories. We present the results for 420 different model permutations that explore key uncertainties in our assumptions about massive binary star evolution (e.g. mass transfer, common-envelope evolution, supernovae), and the metallicity-specific star formation rate density, and characterize their relative impacts on our predictions. We find intrinsic local BHNS merger rates spanning $\mathcal {R}_{\rm {m}}^0 \approx$ 4–830 $\, \rm {Gpc}^{-3}$ $\, \rm {yr}^{-1}$ for our full range of assumptions. This encompasses the rate inferred from recent BHNS GW detections, and would yield detection rates of $\mathcal {R}_{\rm {det}} \approx 1$–180$\, \rm {yr}^{-1}$ for a GW network consisting of LIGO, Virgo and KAGRA at design sensitivity. We find that the binary evolution and metallicity-specific star formation rate density each impact the predicted merger rates by order $\mathcal {O}(10)$. We also present predictions for the GW detected BHNS merger properties and find that all 420 model variations predict that $\lesssim 5{{\ \rm per\ cent}}$ of the BHNS mergers have BH masses $m_{\rm {BH}} \gtrsim 18\, \rm {M}_{\odot }$, total masses $m_{\rm {tot}} \gtrsim 20\, \rm {M}_{\odot }$, chirp masses ${\mathcal {M}}_{\rm {c}} \gtrsim 5.5\, \rm {M}_{\odot }$, mass ratios qf ≳ 12 or qf ≲ 2. Moreover, we find that massive NSs with $m_{\rm {NS}} > 2\, \rm {M}_{\odot }$ are expected to be commonly detected in BHNS mergers in almost all our model variations. Finally, a wide range of $\sim 0{{\ \rm per\ cent}}$–$70{{\ \rm per\ cent}}$ of the BHNS mergers are predicted to eject mass during the merger. Our results highlight the importance of considering variations in binary evolution and cosmological models when predicting, and eventually evaluating, populations of BHNS mergers.
Numerical investigation of some reductions for the Gatenby-Gawlinski model

The Gatenby-Gawlinski model for cancer invasion is object of analysis in order to investigate the mathematical framework behind the model working by means of suitable reductions. We perform numerical simulations to study the sharpness/smoothness of the traveling fronts starting from a brief overview about the full model and proceed by examining the case of a two-equations-based and one-equation-based reduction. We exploit a numerical strategy depending on a finite volume approximation and employ a space-averaged wave speed estimate to quantitatively approach the traveling waves phenomenon. Concerning the one equation-based model, we propose a reduction framed within the degenerate reaction-diffusion equations field, which proves to be effective in order to qualitatively recover the typical trends arising from the Gatenby-Gawlinski model. Finally, we carry out some numerical tests in a specific case where the analytical solution is available.
Topological magnetic textures in magnetic topological insulators

The surfaces of intrinsic magnetic topological insulators (TIs) host magnetic moments exchange-coupled to Dirac electrons. We study the magnetic phases arising from tuning the electron density using variational and exact diagonalization approaches. In the dilute limit, we find that magnetic skrymions are formed which bind to electrons leading to a skyrmion Wigner crystal phase while at higher densities spin spirals accompanied by chiral 1d channels of electrons are formed. The binding of electrons to textures raises the possibility of manipulating textures with electrostatic gating. We determine the phase diagram capturing the competition of intrinsic spin-spin interactions and carrier density and comment on the possible application to experiments in magnetic TIs and spintronic devices such as skyrmion-based memory.
Robustness Evaluation of Stacked Generative Adversarial Networks using Metamorphic Testing

Synthesising photo-realistic images from natural language is one of the challenging problems in computer vision. Over the past decade, a number of approaches have been proposed, of which the improved Stacked Generative Adversarial Network (StackGAN-v2) has proven capable of generating high resolution images that reflect the details specified in the input text descriptions. In this paper, we aim to assess the robustness and fault-tolerance capability of the StackGAN-v2 model by introducing variations in the training data. However, due to the working principle of Generative Adversarial Network (GAN), it is difficult to predict the output of the model when the training data are modified. Hence, in this work, we adopt Metamorphic Testing technique to evaluate the robustness of the model with a variety of unexpected training dataset. As such, we first implement StackGAN-v2 algorithm and test the pre-trained model provided by the original authors to establish a ground truth for our experiments. We then identify a metamorphic relation, from which test cases are generated. Further, metamorphic relations were derived successively based on the observations of prior test results. Finally, we synthesise the results from our experiment of all the metamorphic relations and found that StackGAN-v2 algorithm is susceptible to input images with obtrusive objects, even if it overlaps with the main object minimally, which was not reported by the authors and users of StackGAN-v2 model. The proposed metamorphic relations can be applied to other text-to-image synthesis models to not only verify the robustness but also to help researchers understand and interpret the results made by the machine learning models.
Nucleon mass radii and distribution: Holographic QCD, lattice QCD, and GlueX data

We briefly review and expand our recent analysis for all three invariant A,B,D gravitational form factors of the nucleon in holographic QCD. They compare well to the gluonic gravitational form factors recently measured using lattice QCD simulations. The holographic A-term is fixed by the tensor T = 2 (graviton) Regge trajectory, and the D-term by the difference between the tensor T = 2 (graviton) and scalar S = 0 (dilaton) Regge trajectories. The B-term is null in the absence of a tensor coupling to a Dirac fermion in bulk. A first measurement of the tensor form factor A-term is already accessible using the current GlueX data, and therefore the tensor gluonic mass radius, pressure and shear inside the proton, thanks to holography. The holographic A-term and D-term can be expressed exactly in terms of harmonic numbers. The tensor mass radius from the holographic threshold is found to be 〈r GT 〉 ≈ (0.57 − 0.60 fm), in agreement with 〈r GT 〉 ≈ (0.62 fm) as extracted from the overall numerical lattice data, and empirical GlueX data. The scalar mass radius is found to be slightly larger 〈r GS〉 ≈ (0.7 fm).