Miracle-dz/UniFAR-FTUs huggingface dataset:
A Hypernetwork Based Framework for Non-Stationary Channel Prediction

In order to break through the development bottleneck of modern wireless communication networks, a critical issue is the out-of-date channel state information (CSI) in high mobility scenarios. In general, non-stationary CSI has statistical properties which vary with time, implying that the data distribution changes continuously over time. This temporal distribution shift behavior undermines the accurate channel prediction and it is still an open problem in the related literature. In this paper, a hypernetwork based framework is proposed for non-stationary channel prediction. The framework aims to dynamically update the neural network (NN) parameters as the wireless channel changes to automatically adapt to various input CSI distributions. Based on this framework, we focus on low-complexity hypernetwork design and present a deep learning (DL) based channel prediction method, termed as LPCNet, which improves the CSI prediction accuracy with acceptable complexity. Moreover, to maximize the achievable downlink spectral efficiency (SE), a joint channel prediction and beamforming (BF) method is developed, termed as JLPCNet, which seeks to predict the BF vector. Our numerical results showcase the effectiveness and flexibility of the proposed framework, and demonstrate the superior performance of LPCNet and JLPCNet in various scenarios for fixed and varying user speeds.
Solving Continual Offline Reinforcement Learning with Decision Transformer

Continuous offline reinforcement learning (CORL) combines continuous and offline reinforcement learning, enabling agents to learn multiple tasks from static datasets without forgetting prior tasks. However, CORL faces challenges in balancing stability and plasticity. Existing methods, employing Actor-Critic structures and experience replay (ER), suffer from distribution shifts, low efficiency, and weak knowledge-sharing. We aim to investigate whether Decision Transformer (DT), another offline RL paradigm, can serve as a more suitable offline continuous learner to address these issues. We first compare AC-based offline algorithms with DT in the CORL framework. DT offers advantages in learning efficiency, distribution shift mitigation, and zero-shot generalization but exacerbates the forgetting problem during supervised parameter updates. We introduce multi-head DT (MH-DT) and low-rank adaptation DT (LoRA-DT) to mitigate DT's forgetting problem. MH-DT stores task-specific knowledge using multiple heads, facilitating knowledge sharing with common components. It employs distillation and selective rehearsal to enhance current task learning when a replay buffer is available. In buffer-unavailable scenarios, LoRA-DT merges less influential weights and fine-tunes DT's decisive MLP layer to adapt to the current task. Extensive experiments on MoJuCo and Meta-World benchmarks demonstrate that our methods outperform SOTA CORL baselines and showcase enhanced learning capabilities and superior memory efficiency.
Efficient and Mathematically Robust Operations for Certified Neural Networks Inference

In recent years, machine learning (ML) and neural networks (NNs) have gained widespread use and attention across various domains, particularly in transportation for achieving autonomy, including the emergence of flying taxis for urban air mobility (UAM). However, concerns about certification have come up, compelling the development of standardized processes encompassing the entire ML and NN pipeline. This paper delves into the inference stage and the requisite hardware, highlighting the challenges associated with IEEE 754 floating-point arithmetic and proposing alternative number representations. By evaluating diverse summation and dot product algorithms, we aim to mitigate issues related to non-associativity. Additionally, our exploration of fixed-point arithmetic reveals its advantages over floating-point methods, demonstrating significant hardware efficiencies. Employing an empirical approach, we ascertain the optimal bit-width necessary to attain an acceptable level of accuracy, considering the inherent complexity of bit-width optimization.
Deep Chandra Observation of the Remarkable Ionization Cones of NGC 5252

Seyfert galaxy NGC 5252 harbors enormously extended ionization cones that have been previously detected in the optical and X-ray band, offering a unique opportunity to investigate the interaction between the central active galactic nucleus (AGN) and the surrounding gas in the AGN host galaxy. We present deep Chandra imaging spectroscopy of NGC 5252 with a total exposure time of 230 ks. The morphology in the soft X-rays shows resolved extended structure from the nucleus to a large radial distance, and for the first time we detect the outermost X-ray arc at ∼20 kpc. The X-ray cone mostly follows the direction of the optical ionization cones in the southeast and northwest directions, about 20° misaligned with the major axis of the galactic disk of NGC 5252. Fitting the spectra extracted from radial sectors with photoionization models supports that extended emission is mainly photoionized by the central AGN. We also examine the variation of the photoionization parameter along the radial extension and infer a decreasing ionizing continuum of the central engine by a factor of ∼50 over the past 64,000 yr. These findings are consistent with previous suggestions that NGC 5252 resembles a quasar relic with an M ∼ 109 M ⊙ supermassive black hole that went through a minor merger event and switched to a low accretion rate state.
Stochastic Subnetwork Annealing: A Regularization Technique for Fine Tuning Pruned Subnetworks

Pruning methods have recently grown in popularity as an effective way to reduce the size and computational complexity of deep neural networks. Large numbers of parameters can be removed from trained models with little discernible loss in accuracy after a small number of continued training epochs. However, pruning too many parameters at once often causes an initial steep drop in accuracy which can undermine convergence quality. Iterative pruning approaches mitigate this by gradually removing a small number of parameters over multiple epochs. However, this can still lead to subnetworks that overfit local regions of the loss landscape. We introduce a novel and effective approach to tuning subnetworks through a regularization technique we call Stochastic Subnetwork Annealing. Instead of removing parameters in a discrete manner, we instead represent subnetworks with stochastic masks where each parameter has a probabilistic chance of being included or excluded on any given forward pass. We anneal these probabilities over time such that subnetwork structure slowly evolves as mask values become more deterministic, allowing for a smoother and more robust optimization of subnetworks at high levels of sparsity.
The Faiss library

Vector databases manage large collections of embedding vectors. As AI applications are growing rapidly, so are the number of embeddings that need to be stored and indexed. The Faiss library is dedicated to vector similarity search, a core functionality of vector databases. Faiss is a toolkit of indexing methods and related primitives used to search, cluster, compress and transform vectors. This paper first describes the tradeoff space of vector search, then the design principles of Faiss in terms of structure, approach to optimization and interfacing. We benchmark key features of the library and discuss a few selected applications to highlight its broad applicability.
Privacy Preserving Adaptive Experiment Design

Adaptive experiment is widely adopted to estimate conditional average treatment effect (CATE) in clinical trials and many other scenarios. While the primary goal in experiment is to maximize estimation accuracy, due to the imperative of social welfare, it's also crucial to provide treatment with superior outcomes to patients, which is measured by regret in contextual bandit framework. These two objectives often lead to contrast optimal allocation mechanism. Furthermore, privacy concerns arise in clinical scenarios containing sensitive data like patients health records. Therefore, it's essential for the treatment allocation mechanism to incorporate robust privacy protection measures. In this paper, we investigate the tradeoff between loss of social welfare and statistical power in contextual bandit experiment. We propose a matched upper and lower bound for the multi-objective optimization problem, and then adopt the concept of Pareto optimality to mathematically characterize the optimality condition. Furthermore, we propose differentially private algorithms which still matches the lower bound, showing that privacy is"almost free". Additionally, we derive the asymptotic normality of the estimator, which is essential in statistical inference and hypothesis testing.
The Effect of Intrinsic Dataset Properties on Generalization: Unraveling Learning Differences Between Natural and Medical Images

This paper investigates discrepancies in how neural networks learn from different imaging domains, which are commonly overlooked when adopting computer vision techniques from the domain of natural images to other specialized domains such as medical images. Recent works have found that the generalization error of a trained network typically increases with the intrinsic dimension ($d_{data}$) of its training set. Yet, the steepness of this relationship varies significantly between medical (radiological) and natural imaging domains, with no existing theoretical explanation. We address this gap in knowledge by establishing and empirically validating a generalization scaling law with respect to $d_{data}$, and propose that the substantial scaling discrepancy between the two considered domains may be at least partially attributed to the higher intrinsic ``label sharpness'' ($K_\mathcal{F}$) of medical imaging datasets, a metric which we propose. Next, we demonstrate an additional benefit of measuring the label sharpness of a training set: it is negatively correlated with the trained model's adversarial robustness, which notably leads to models for medical images having a substantially higher vulnerability to adversarial attack. Finally, we extend our $d_{data}$ formalism to the related metric of learned representation intrinsic dimension ($d_{repr}$), derive a generalization scaling law with respect to $d_{repr}$, and show that $d_{data}$ serves as an upper bound for $d_{repr}$. Our theoretical results are supported by thorough experiments with six models and eleven natural and medical imaging datasets over a range of training set sizes. Our findings offer insights into the influence of intrinsic dataset properties on generalization, representation learning, and robustness in deep neural networks. Code link: https://github.com/mazurowski-lab/intrinsic-properties
Tighter sum uncertainty relations via (α,β,γ) weighted Wigner-Yanase-Dyson skew information

We establish tighter uncertainty relations for arbitrary finite observables via (α,β,γ) weighted Wigner-Yanase-Dyson ((α,β,γ)WWYD) skew information. The results are also applicable to the (α,γ) weighted Wigner-Yanase-Dyson ((α,γ)WWYD) skew information and the weighted Wigner-Yanase-Dyson (WWYD) skew information. We also present tighter lower bounds for quantum channels and unitary channels via (α,β,γ) modified weighted Wigner-Yanase-Dyson ((α,β,γ)MWWYD) skew information. Detailed examples are provided to illustrate the tightness of our uncertainty relations.
Electrically charged black hole solutions in semiclassical gravity and dynamics of linear perturbations

We explore quantum corrections of electrically charged black holes subject to vacuum polarization effects of fermion fields in QED. Solving this problem exactly is challenging so we restrict to perturbative corrections that one can obtain using the heat kernel expansion in the one-loop effective action for electrons. Starting from the corrections originally computed by Drummond and Hathrell, we solve the full semiclassical Einstein-Maxwell system of coupled equations to leading order in Planck constant, and find a new electrically charged, static black hole solution. To probe these quantum corrections, we study electromagnetic and gravitational (axial) perturbations on this background, and derive the coupled system of Regge-Wheeler master equations that govern the propagation of these waves. In the classical limit our results agree with previous findings in the literature. We finally compare these results with those that one can obtain by working out the Euler-Heisenberg effective action. We find again a new electrically charged static black hole spacetime, and derive the coupled system of Regge-Wheeler equations governing the propagation of axial electromagnetic and gravitational perturbations. Results are qualitatively similar in both cases. We briefly discuss some challenges found in the numerical computation of the QNM frequency spectra when quantum corrections are included.
Micro-optics fabrication using blurred tomography

We demonstrate the fabrication of millimeter-sized optical components using tomographic volumetric additive manufacturing (VAM). By purposely blurring the writing beams through the use of a large etendue source, the layer-like artifacts called striations are eliminated enabling the rapid and direct fabrication of smooth surfaces. We call this method blurred tomography, and demonstrate its capability by printing a plano-convex optical lens with comparable imaging performance to that of a commercially-available glass lens. Furthermore, due to the intrinsic freeform design nature of VAM, we demonstrate the double-sided fabrication of a biconvex microlens array, and for the first time demonstrate overprinting of a lens onto an optical fiber using this printing modality. This approach to VAM will pave the way for low-cost, rapid-prototyping of freeform optical components.
EEG-based Cognitive Load Estimation of Acoustic Parameters for Data Sonification

Sonification is a data visualization technique which expresses data attributes via psychoacoustic parameters, which are non-speech audio signals used to convey information. This paper investigates the binary estimation of cognitive load induced by psychoacoustic parameters conveying the focus level of an astronomical image via Electroencephalogram (EEG) embeddings. Employing machine learning and deep learning methodologies, we demonstrate that EEG signals are reliable for (a) binary estimation of cognitive load, (b) isolating easy vs difficult visual-to-auditory perceptual mappings, and (c) capturing perceptual similarities among psychoacoustic parameters. Our key findings reveal that (1) EEG embeddings can reliably measure cognitive load, achieving a peak F1-score of 0.98; (2) Extreme focus levels are easier to detect via auditory mappings than intermediate ones, and (3) psychoacoustic parameters inducing comparable cognitive load levels tend to generate similar EEG encodings.
Measuring Hydrogen-to-Helium Ratio in Cool Stars

Conventionally, the helium-to-hydrogen ratio for the stars are adopted to be 0.1, as standard, unless, the stars are severely deficient in hydrogen like in RCB-class, or the stars' helium abundance is accurately measured using He I transitions in warm/hotter stars. In our study, the small change in helium-to-hydrogen ratio (from standard value, 0.1) in normal giants were detected from the large difference (>0.3 dex) in the Mg-abundance measured from Mg I lines and the subordinate lines of (0,0) MgH band. These are the stars that are mildly hydrogen-deficient/He-enhanced. Such stars were spectroscopically discovered for the first time among giants of the globular cluster Omega Centauri. The sample selection, observations, methodology and results are discussed in detail.
Space and Time Continuous Physics Simulation From Partial Observations

Modern techniques for physical simulations rely on numerical schemes and mesh-refinement methods to address trade-offs between precision and complexity, but these handcrafted solutions are tedious and require high computational power. Data-driven methods based on large-scale machine learning promise high adaptivity by integrating long-range dependencies more directly and efficiently. In this work, we focus on fluid dynamics and address the shortcomings of a large part of the literature, which are based on fixed support for computations and predictions in the form of regular or irregular grids. We propose a novel setup to perform predictions in a continuous spatial and temporal domain while being trained on sparse observations. We formulate the task as a double observation problem and propose a solution with two interlinked dynamical systems defined on, respectively, the sparse positions and the continuous domain, which allows to forecast and interpolate a solution from the initial condition. Our practical implementation involves recurrent GNNs and a spatio-temporal attention observer capable of interpolating the solution at arbitrary locations. Our model not only generalizes to new initial conditions (as standard auto-regressive models do) but also performs evaluation at arbitrary space and time locations. We evaluate on three standard datasets in fluid dynamics and compare to strong baselines, which are outperformed both in classical settings and in the extended new task requiring continuous predictions.
The Hidden Companion in J1527: A 0.69 Solar-mass White Dwarf?

Finding nearby neutron stars can probe the supernova and metal-enrichment histories near our solar system. Recently, Lin et al. reported an exciting neutron star candidate, Two Micron All Sky Survey J15274848+3536572 (hereafter J1527), with a small Gaia distance of 118 pc. They claim that J1527 harbors an unseen neutron star candidate with an unusually low mass of 0.98 ± 0.03 M ⊙. In this work, we use the Canada–France–Hawaii Telescope high-resolution spectrum to measure J1527's orbital inclination independently. Our spectral fitting suggests an orbital inclination of 63° ± 2°. Instead, by fitting a complex hybrid variability model consisting of the ellipsoidal-variation component and the starspot modulation to the observed light curve, Lin et al. obtain an orbital inclination of 45.2 − 0.20 + 0.13 degrees. We speculate that the orbital inclination obtained by the light-curve fitting is underestimated since J1527's light curves are obviously not pure ellipsoidal variations. According to our new inclination (i ∼ 63°), the mass of the unseen compact object is reduced to 0.69 ± 0.02 M ⊙, which is as massive as a typical white dwarf.
Classification and Reconstruction Processes in Deep Predictive Coding Networks: Antagonists or Allies?

Predictive coding-inspired deep networks for visual computing integrate classification and reconstruction processes in shared intermediate layers. Although synergy between these processes is commonly assumed, it has yet to be convincingly demonstrated. In this study, we take a critical look at how classifying and reconstructing interact in deep learning architectures. Our approach utilizes a purposefully designed family of model architectures reminiscent of autoencoders, each equipped with an encoder, a decoder, and a classification head featuring varying modules and complexities. We meticulously analyze the extent to which classification- and reconstruction-driven information can seamlessly coexist within the shared latent layer of the model architectures. Our findings underscore a significant challenge: Classification-driven information diminishes reconstruction-driven information in intermediate layers' shared representations and vice versa. While expanding the shared representation's dimensions or increasing the network's complexity can alleviate this trade-off effect, our results challenge prevailing assumptions in predictive coding and offer guidance for future iterations of predictive coding concepts in deep networks.
Revisiting Self-supervised Learning of Speech Representation from a Mutual Information Perspective

Existing studies on self-supervised speech representation learning have focused on developing new training methods and applying pre-trained models for different applications. However, the quality of these models is often measured by the performance of different downstream tasks. How well the representations access the information of interest is less studied. In this work, we take a closer look into existing self-supervised methods of speech from an information-theoretic perspective. We aim to develop metrics using mutual information to help practical problems such as model design and selection. We use linear probes to estimate the mutual information between the target information and learned representations, showing another insight into the accessibility to the target information from speech representations. Further, we explore the potential of evaluating representations in a self-supervised fashion, where we estimate the mutual information between different parts of the data without using any labels. Finally, we show that both supervised and unsupervised measures echo the performance of the models on layer-wise linear probing and speech recognition.
Arens regularity of weighted convolution algebras that arise from totally ordered semilattices

We study Arens regularity of weighted semigroup convolution algebras for the specific case of totally ordered semilattices. This paper is a natural continuation of that of Dales and Strauss (2022) [DS22], where they studied the unweighted case. We also study some other properties of these algebras, such as the existence of Banach-algebra preduals.
Local mean dimension theory for sofic group actions

Using a local perspective, we introduce mean dimension pairs and give sufficient conditions of when every non-trivial factor of a continuous group action of a sofic group $G$ has positive mean dimension. In addition, we prove that the mean dimension map is Borel, and that the set of subshifts with completely positive mean dimension of $[0,1]^G$, the full $G$-shift on the interval, is a complete coanalytic set in the set of all subshifts (hence not Borel). Our results are new even when the acting group is $\mathbb{Z}$.
ICON: Incremental CONfidence for Joint Pose and Radiance Field Optimization

Neural Radiance Fields (NeRF) exhibit remarkable performance for Novel View Synthesis (NVS) given a set of 2D images. However, NeRF training requires accurate camera pose for each input view, typically obtained by Structure-from-Motion (SfM) pipelines. Recent works have attempted to relax this constraint, but they still often rely on decent initial poses which they can refine. Here we aim at removing the requirement for pose initialization. We present Incremental CONfidence (ICON), an optimization procedure for training NeRFs from 2D video frames. ICON only assumes smooth camera motion to estimate initial guess for poses. Further, ICON introduces ``confidence": an adaptive measure of model quality used to dynamically reweight gradients. ICON relies on high-confidence poses to learn NeRF, and high-confidence 3D structure (as encoded by NeRF) to learn poses. We show that ICON, without prior pose initialization, achieves superior performance in both CO3D and HO3D versus methods which use SfM pose.