Miracle-dz/UniFAR-FTUs huggingface dataset:
An Analysis Tool for Push-Sum Based Distributed Optimization

The push-sum algorithm is probably the most important distributed averaging approach over directed graphs, which has been applied to various problems including distributed optimization. This paper establishes the explicit absolute probability sequence for the push-sum algorithm, and based on which, constructs quadratic Lyapunov functions for push-sum based distributed optimization algorithms. As illustrative examples, the proposed novel analysis tool can improve the convergence rates of the subgradient-push and stochastic gradient-push, two important algorithms for distributed convex optimization over unbalanced directed graphs. Specifically, the paper proves that the subgradient-push algorithm converges at a rate of $O(1/\sqrt{t})$ for general convex functions and stochastic gradient-push algorithm converges at a rate of $O(1/t)$ for strongly convex functions, over time-varying unbalanced directed graphs. Both rates are respectively the same as the state-of-the-art rates of their single-agent counterparts and thus optimal, which closes the theoretical gap between the centralized and push-sum based (sub)gradient methods. The paper further proposes a heterogeneous push-sum based subgradient algorithm in which each agent can arbitrarily switch between subgradient-push and push-subgradient. The heterogeneous algorithm thus subsumes both subgradient-push and push-subgradient as special cases, and still converges to an optimal point at an optimal rate. The proposed tool can also be extended to analyze distributed weighted averaging.
BCS in the Sky: Signatures of Inflationary Fermion Condensation

We consider a Bardeen-Cooper-Schrieffer (BCS)-like model in the inflationary background. We show that with an axial chemical potential, the attractive quartic fermion self-interaction can lead to a BCS-like condensation. In the de Sitter (dS) limit of inflation, we perform the first computation of the non-perturbative effective potential that includes the full spacetime curvature effects in the presence of the chemical potential. The corresponding BCS phase transition is always first-order, when the varying Hubble is interpreted as an effective Gibbons-Hawking temperature of dS spacetime. In the condensate phase, the theory can be understood from UV and IR sides as fermionic and bosonic, respectively. This leads to distinctive signatures in the primordial non-Gaussianity of curvature perturbations. Namely, the oscillatory cosmological collider signal is smoothly turned off at a finite momentum ratio, since different momentum ratios effectively probe different energy scales. In addition, such BCS phase transitions can also source stochastic gravitational waves, feasible for future experiments.
Decoupled Training for Long-Tailed Classification With Stochastic Representations

Decoupling representation learning and classifier learning has been shown to be effective in classification with long-tailed data. There are two main ingredients in constructing a decoupled learning scheme; 1) how to train the feature extractor for representation learning so that it provides generalizable representations and 2) how to re-train the classifier that constructs proper decision boundaries by handling class imbalances in long-tailed data. In this work, we first apply Stochastic Weight Averaging (SWA), an optimization technique for improving the generalization of deep neural networks, to obtain better generalizing feature extractors for long-tailed classification. We then propose a novel classifier re-training algorithm based on stochastic representation obtained from the SWA-Gaussian, a Gaussian perturbed SWA, and a self-distillation strategy that can harness the diverse stochastic representations based on uncertainty estimates to build more robust classifiers. Extensive experiments on CIFAR10/100-LT, ImageNet-LT, and iNaturalist-2018 benchmarks show that our proposed method improves upon previous methods both in terms of prediction accuracy and uncertainty estimation.
Sample-efficient Model-based Reinforcement Learning for Quantum Control

We propose a model-based reinforcement learning (RL) approach for noisy time-dependent gate optimization with improved sample complexity over model-free RL. Sample complexity is the number of controller interactions with the physical system. Leveraging an inductive bias, inspired by recent advances in neural ordinary differential equations (ODEs), we use an auto-differentiable ODE parametrised by a learnable Hamiltonian ansatz to represent the model approximating the environment whose time-dependent part, including the control, is fully known. Control alongside Hamiltonian learning of continuous time-independent parameters is addressed through interactions with the system. We demonstrate an order of magnitude advantage in the sample complexity of our method over standard model-free RL in preparing some standard unitary gates with closed and open system dynamics, in realistic numerical experiments incorporating single shot measurements, arbitrary Hilbert space truncations and uncertainty in Hamiltonian parameters. Also, the learned Hamiltonian can be leveraged by existing control methods like GRAPE for further gradient-based optimization with the controllers found by RL as initializations. Our algorithm that we apply on nitrogen vacancy (NV) centers and transmons in this paper is well suited for controlling partially characterised one and two qubit systems.
Coded Speech Quality Measurement by a Non-Intrusive PESQ-DNN

Wideband codecs such as AMR-WB or EVS are widely used in (mobile) speech communication. Evaluation of coded speech quality is often performed subjectively by an absolute category rating (ACR) listening test. However, the ACR test is impractical for online monitoring of speech communication networks. Perceptual evaluation of speech quality (PESQ) is one of the widely used metrics instrumentally predicting the results of an ACR test. However, the PESQ algorithm requires an original reference signal, which is usually unavailable in network monitoring, thus limiting its applicability. NISQA is a new non-intrusive neural-network-based speech quality measure, focusing on super-wideband speech signals. In this work, however, we aim at predicting the well-known PESQ metric using a non-intrusive PESQ-DNN model. We illustrate the potential of this model by predicting the PESQ scores of wideband-coded speech obtained from AMR-WB or EVS codecs operating at different bitrates in noisy, tandeming, and error-prone transmission conditions. We compare our methods with the state-of-the-art network topologies of QualityNet, WaweNet, and DNSMOS—all applied to PESQ prediction—by measuring the mean absolute error (MAE) and the linear correlation coefficient (LCC). The proposed PESQ-DNN offers the best total MAE and LCC of 0.11 and 0.92, respectively, in conditions without frame loss, and still is best when including frame loss. Note that our model could be similarly used to non-intrusively predict POLQA or other (intrusive) metrics.
Efficient computation of the sinc matrix function for the integration of second-order differential equations

This work deals with the numerical solution of systems of oscillatory second-order differential equations which often arise from the semi-discretization in space of partial differential equations. Since these differential equations exhibit (pronounced or highly) oscillatory behavior, standard numerical methods are known to perform poorly. Our approach consists in directly discretizing the problem by means of Gautschi-type integrators based on $\operatorname{sinc}$ matrix functions. The novelty contained here is that of using a suitable rational approximation formula for the $\operatorname{sinc}$ matrix function to apply a rational Krylov-like approximation method with suitable choices of poles. In particular, we discuss the application of the whole strategy to a finite element discretization of the wave equation.
Coarse race data conceals disparities in clinical risk score model performance

Healthcare data in the United States often records only a patient’s coarse race group: for example, both Indian and Chinese patients are typically coded as “Asian. ” It is unknown, however, whether this coarse coding conceals meaningful disparities in the performance of clinical risk scores across granular race groups. Here we show that it does. Using data from 418K emergency department visits, we assess clinical risk score performance disparities across granular race groups for three outcomes, ﬁve risk scores, and four performance metrics. Across outcomes and metrics, we show that there are signiﬁcant granular disparities in performance within coarse race categories. In fact, variation in performance metrics within coarse groups often exceeds the variation between coarse groups. We explore why these disparities arise, ﬁnding that outcome rates, feature distributions, and the relationships between features and outcomes all vary signiﬁcantly across granular race categories. Our results suggest that healthcare providers, hospital systems, and machine learning researchers should strive to collect, release, and use granular race data in place of coarse race data, and that existing analyses may signiﬁcantly underestimate racial disparities in performance.
To Compress or Not to Compress—Self-Supervised Learning and Information Theory: A Review

Deep neural networks excel in supervised learning tasks but are constrained by the need for extensive labeled data. Self-supervised learning emerges as a promising alternative, allowing models to learn without explicit labels. Information theory has shaped deep neural networks, particularly the information bottleneck principle. This principle optimizes the trade-off between compression and preserving relevant information, providing a foundation for efficient network design in supervised contexts. However, its precise role and adaptation in self-supervised learning remain unclear. In this work, we scrutinize various self-supervised learning approaches from an information-theoretic perspective, introducing a unified framework that encapsulates the self-supervised information-theoretic learning problem. This framework includes multiple encoders and decoders, suggesting that all existing work on self-supervised learning can be seen as specific instances. We aim to unify these approaches to understand their underlying principles better and address the main challenge: many works present different frameworks with differing theories that may seem contradictory. By weaving existing research into a cohesive narrative, we delve into contemporary self-supervised methodologies, spotlight potential research areas, and highlight inherent challenges. Moreover, we discuss how to estimate information-theoretic quantities and their associated empirical problems. Overall, this paper provides a comprehensive review of the intersection of information theory, self-supervised learning, and deep neural networks, aiming for a better understanding through our proposed unified approach.
Intensity effects of light coupling to one- or two-atom arrays of infinite extent

We theoretically and computationally investigate the behavior of infinite atom arrays when illuminated by nearly resonant light. We use higher order mean field equations to investigate the coherent reflection and transmission and incoherent scattering of photons from a single array and from a pair of arrays as a function of detuning for different values of the Rabi frequency. For the single array case, we show how increasing the light intensity changes the probabilities for these different processes. For example, the incoherent scattering probability initially increases with light intensity before decreasing at higher values. For a pair of parallel arrays at near resonant separation, the effects from increasing light intensity can become apparent with incredibly low intensity light. In addition, we derive the higher order mean field equations for these infinite arrays giving a representation that can be evaluated with a finite number of equations.
A new look on optimal foldover plans in terms of uniformity criteria

ABSTRACT In this paper, we develop a new mechanism for finding the optimal foldover plans (OFPs) which is based on the uniformity criteria measured by Lee discrepancy, wrap-around L2-discrepancy, and centered L2-discrepancy. For three-level fractional factorials as the original designs, general foldover plans and combined designs are defined, and lower bounds of these three discrepancies of combined designs under general foldover plans are also obtained, which can be used as benchmarks for searching OFPs. Illustrative examples with a comparison study between the foldover plans under these discrepancies are provided. Our results provide a theoretical justification for OFPs of three-level designs in terms of uniformity criteria.
Unpacking How Decentralized Autonomous Organizations (DAOs) Work in Practice

Decentralized Autonomous Organizations (DAOs) have emerged as a novel way to coordinate a group of (pseudonymous) entities towards a shared vision (e.g., promoting sustainability), utilizing self-executing smart contracts on blockchains to support decentralized governance and decision-making. In just a few years, over 4,000 DAOs have been launched in various domains, such as investment, education, health, and research. Despite such rapid growth and diversity, it is unclear how these DAOs actually work in practice and to what extent they are effective in achieving their goals. Given this, we aim to unpack how (well) DAOs work in practice. We conducted an in-depth analysis of a diverse set of 10 DAOs of various categories and smart contracts, leveraging on-chain (e.g., voting results) and off-chain data (e.g., community discussions) as well as our interviews with DAO organizers/members. Specifically, we defined metrics to characterize key aspects of DAOs, such as the degrees of decentralization and autonomy. We observed CompoundDAO, AssangeDAO, Bankless, and Krausehouse having poor decentralization in voting, while decentralization has improved over time for one-person-one-vote DAOs (e.g., Proof of Humanity). Moreover, the degree of autonomy varies among DAOs, with some (e.g., Compound and Krausehouse) relying more on third parties than others. Lastly, we offer a set of design implications for future DAO systems based on our findings.
Polytropic index of magnetosheath ions based on homogeneous MHD Bernoulli Integral

This paper uses Cluster data during the period from 2001 to 2010 to study the polytropic processes of magnetosheath ions. Utilizing the method of homogeneous magnetohydrodynamic (MHD) Bernoulli integral (MBI), we first identify streamflow tubes, then use the constant of polytropic relation to guarantee that the streamflow tube experiences an unchanged polytropic process, and finally determine the polytropic index of ions in these streamflow tubes. The statistical results show that the magnetosheath is a complicated system in which the polytropic index of ions ranges from −2 to 3. The polytropic index distribution of ions is dependent on the electromagnetic energy flux perpendicular to the streamline. The median polytropic index of ions in the magnetosheath is 0.960, 0.965, and 0.974 for perpendicular electromagnetic energy ratio δE × B < 5%, δE × B < 3%, and δE × B < 1%, respectively. There are two basic polytropic processes in the magnetosheath: the dominant isothermal process and the isobaric process. When there is no exchange of electromagnetic energy between neighboring streamflow tubes, the magnetosheath ions are isothermal. However, when the perpendicular electromagnetic energy ratio increases, the isobaric polytropic process starts to emerge. The magnetosheath ion flows are highly localized because most streamflow tubes experiencing same polytropic processes last less than 60 s. Thus, the polytropic index of magnetosheath ion flows is highly variable.
A pr 2 02 3 LIPSCHITZ TRUNCATION METHOD FOR THE PARABOLIC DOUBLE-PHASE SYSTEM AND APPLICATIONS

We prove energy estimates for a weak solution to the parabolic double-phase system, such as Caccioppoli type inequality and estimates related to the existence and uniqueness for the Dirichlet problem. The proof of Caccioppoli type inequality and uniqueness are based on the Lipschitz truncation method while the existence is proved by approximation from perturbed double-phase systems.
Using Offline Data to Speed-up Reinforcement Learning in Procedurally Generated Environments

One of the key challenges of Reinforcement Learning (RL) is the ability of agents to generalise their learned policy to unseen settings. Moreover, training RL agents requires large numbers of interactions with the environment. Motivated by the recent success of Offline RL and Imitation Learning (IL), we conduct a study to investigate whether agents can leverage offline data in the form of trajectories to improve the sample-efficiency in procedurally generated environments. We consider two settings of using IL from offline data for RL: (1) pre-training a policy before online RL training and (2) concurrently training a policy with online RL and IL from offline data. We analyse the impact of the quality (optimality of trajectories) and diversity (number of trajectories and covered level) of available offline trajectories on the effectiveness of both approaches. Across four well-known sparse reward tasks in the MiniGrid environment, we find that using IL for pre-training and concurrently during online RL training both consistently improve the sample-efficiency while converging to optimal policies. Furthermore, we show that pre-training a policy from as few as two trajectories can make the difference between learning an optimal policy at the end of online training and not learning at all. Our findings motivate the widespread adoption of IL for pre-training and concurrent IL in procedurally generated environments whenever offline trajectories are available or can be generated.
A Two-Stage Framework with Self-Supervised Distillation For Cross-Domain Text Classification

Cross-domain text classification aims to adapt models to a target domain that lacks labeled data. It leverages or reuses rich labeled data from the different but related source domain(s) and unlabeled data from the target domain. To this end, previous work focuses on either extracting domain-invariant features or task-agnostic features, ignoring domain-aware features that may be present in the target domain and could be useful for the downstream task. In this paper, we propose a two-stage framework for cross-domain text classification. In the first stage, we finetune the model with mask language modeling (MLM) and labeled data from the source domain. In the second stage, we further fine-tune the model with self-supervised distillation (SSD) and unlabeled data from the target domain. We evaluate its performance on a public cross-domain text classification benchmark and the experiment results show that our method achieves new state-of-the-art results for both single-source domain adaptations (94.17% $\uparrow$1.03%) and multi-source domain adaptations (95.09% $\uparrow$1.34%).
Optimum Output Long Short-Term Memory Cell for High-Frequency Trading Forecasting

High-frequency trading requires fast data processing without information lags for precise stock price forecasting. This high-paced stock price forecasting is usually based on vectors that need to be treated as sequential and time-independent signals due to the time irregularities that are inherent in high-frequency trading. A well-documented and tested method that considers these time-irregularities is a type of recurrent neural network, named long short-term memory neural network. This type of neural network is formed based on cells that perform sequential and stale calculations via gates and states without knowing whether their order, within the cell, is optimal. In this paper, we propose a revised and real-time adjusted long short-term memory cell that selects the best gate or state as its final output. Our cell is running under a shallow topology, has a minimal look-back period, and is trained online. This revised cell achieves lower forecasting error compared to other recurrent neural networks for online high-frequency trading forecasting tasks such as the limit order book mid-price prediction as it has been tested on two high-liquid US and two less-liquid Nordic stocks.
Approaches to Statistical Efficiency when comparing the embedded adaptive interventions in a SMART

Sequential, multiple assignment randomized trials (SMARTs), which assist in the optimization of adaptive interventions, are growing in popularity in education and behavioral sciences. This is unsurprising, as adaptive interventions reflect the sequential, tailored nature of learning in a classroom or school. Nonetheless, as is true elsewhere in education research, observed effect sizes in education-based SMARTs are frequently small. As a consequence, statistical efficiency is of paramount importance in their analysis. The contributions of this manuscript are two-fold. First, we provide an overview of adaptive interventions and SMART designs for researchers in education science. Second, we propose four techniques that have the potential to improve statistical efficiency in the analysis of SMARTs. We demonstrate the benefits of these techniques in SMART settings both through the analysis of a SMART designed to optimize an adaptive intervention for increasing cognitive behavioral therapy delivery in school settings and through a comprehensive simulation study. Each of the proposed techniques is easily implementable, either with over-the-counter statistical software or through R code provided in an online supplement.
The role of the background velocity model for the Marchenko focusing of reflected and refracted waves

Marchenko algorithms retrieve the wavefields excited by virtual sources in the subsurface, these are the Green's functions consisting of the primary and multiple reflected waves. The requirements for these algorithms are the same as for conventional imaging algorithms; they need an estimate of the velocity model and the recorded reflected waves. We investigate the dependence of the retrieved Green's functions using the Marchenko equation on the background velocity model and address the question: ``How well do we need to know the velocity model for accurate Marchenko focusing? ". We present three different background velocity models and compare the Green's functions retrieved using these models. We show that these retrieved Green's functions using the Marchenko equation give correlation coefficients with the exact Green's function larger than 90% on average except near the edges of the receiver aperture. We also examine the presence of refracted waves in the retrieved Green's function. We show with a numerical example that the Marchenko focusing algorithm produces refracted waves only if the initial velocity model used for the iterative scheme is sufficiently detailed to model the refracted waves.
CrossFusion: Interleaving Cross-modal Complementation for Noise-resistant 3D Object Detection

The combination of LiDAR and camera modalities is proven to be necessary and typical for 3D object detection according to recent studies. Existing fusion strategies tend to overly rely on the LiDAR modal in essence, which exploits the abundant semantics from the camera sensor insufficiently. However, existing methods cannot rely on information from other modalities because the corruption of LiDAR features results in a large domain gap. Following this, we propose CrossFusion, a more robust and noise-resistant scheme that makes full use of the camera and LiDAR features with the designed cross-modal complementation strategy. Extensive experiments we conducted show that our method not only outperforms the state-of-the-art methods under the setting without introducing an extra depth estimation network but also demonstrates our model's noise resistance without re-training for the specific malfunction scenarios by increasing 5.2\% mAP and 2.4\% NDS.