Miracle-dz/UniFAR-FTUs huggingface dataset:
Causal emergence is widespread across measures of causation

Causal emergence is the theory that macroscales can reduce the noise in causal relationships, leading to stronger causes at the macroscale. First identified using the effective information and later the integrated information in model systems, causal emergence has been analyzed in real data across the sciences since. But is it simply a quirk of these original measures? To answer this question we examined over a dozen popular measures of causation, all independently developed and widely used, and spanning different fields from philosophy to statistics to psychology to genetics. All showed cases of causal emergence. This is because, we prove, measures of causation are based on a small set of related"causal primitives. "This consilience of independently-developed measures of causation shows that macroscale causation is a general fact about causal relationships, is scientifically detectable, and is not a quirk of any particular measure of causation. This finding sets the science of emergence on firmer ground, opening the door for the detection of intrinsic scales of function in complex systems, as well as assisting with scientific modeling and experimental interventions.
A Comparison Between Quantile Regression and Linear Regression on Empirical Quantiles for Phenological Analysis in Migratory Response to Climate Change

It is well established that migratory birds in general have advanced their arrival times in spring, and in this paper we investigate potential ways of enhancing the level of detail in future phenological analyses. We perform single as well as multiple species analyses, using linear models on empirical quantiles, non-parametric quantile regression and likelihood-based parametric quantile regression with asymmetric Laplace distributed error terms. We conclude that non-parametric quantile regression appears most suited for single as well as multiple species analyses.
A Machine Learning Smartphone-based Sensing for Driver Behavior Classification

Driver behavior profiling is one of the main issues in the insurance industries and fleet management, thus being able to classify the driver behavior with low-cost mobile applications remains in the spotlight of autonomous driving. However, using mobile sensors may face the challenge of security, privacy, and trust issues. To overcome those challenges, we propose to collect data sensors using Carla Simulator available in smartphones (Accelerometer, Gyroscope, GPS) in order to classify the driver behavior using speed, acceleration, direction, the 3-axis rotation angles (Yaw, Pitch, Roll) taking into account the speed limit of the current road and weather conditions to better identify the risky behavior. Secondly, after fusing inter-axial data from multiple sensors into a single file, we explore different machine learning algorithms for time series classification to evaluate which algorithm results in the highest performance.
Demystify Optimization and Generalization of Over-parameterized PAC-Bayesian Learning

PAC-Bayesian is an analysis framework where the training error can be expressed as the weighted average of the hypotheses in the posterior distribution whilst incorporating the prior knowledge. In addition to being a pure generalization bound analysis tool, PAC-Bayesian bound can also be incorporated into an objective function to train a probabilistic neural network, making them a powerful and relevant framework that can numerically provide a tight generalization bound for supervised learning. For simplicity, we call probabilistic neural network learned using training objectives derived from PAC-Bayesian bounds as {\it PAC-Bayesian learning}. Despite their empirical success, the theoretical analysis of PAC-Bayesian learning for neural networks is rarely explored. This paper proposes a new class of convergence and generalization analysis for PAC-Bayes learning when it is used to train the over-parameterized neural networks by the gradient descent method. For a wide probabilistic neural network, we show that when PAC-Bayes learning is applied, the convergence result corresponds to solving a kernel ridge regression when the probabilistic neural tangent kernel (PNTK) is used as its kernel. Based on this finding, we further characterize the uniform PAC-Bayesian generalization bound which improves over the Rademacher complexity-based bound for non-probabilistic neural network. Finally, drawing the insight from our theoretical results, we propose a proxy measure for efficient hyperparameters selection, which is proven to be time-saving.
Ab initio circular dichroism with the Yambo code: applications to dipeptides

Circular dichroism (CD) spectroscopy is a useful technique for characterizing chiral molecules. It is more sensitive than total absorption to molecule conformation, and it is routinely used to identify enantiomers. We present here a first principles implementation of CD with application to three cyclo-dipeptides. Our CD approach for molecules has been integrated in the 5.0 release of the Yambo code, distributed under GPL.
AtmoDist: Self-supervised representation learning for atmospheric dynamics

Abstract Representation learning has proven to be a powerful methodology in a wide variety of machine-learning applications. For atmospheric dynamics, however, it has so far not been considered, arguably due to the lack of large-scale, labeled datasets that could be used for training. In this work, we show how to sidestep the difficulty and introduce a self-supervised learning task that is applicable to a wide variety of unlabeled atmospheric datasets. Specifically, we train a neural network on the simple yet intricate task of predicting the temporal distance between atmospheric fields from distinct but nearby times. We demonstrate that training with this task on the ERA5 reanalysis dataset leads to internal representations that capture intrinsic aspects of atmospheric dynamics. For example, when employed as a loss function in other machine-learning applications, the derived AtmoDist distance leads to improved results compared to the $ {\mathrm{\ell}}_2 $ -loss. For downscaling one obtains higher resolution fields that match the true statistics more closely than previous approaches and for the interpolation of missing or occluded data the AtmoDist distance leads to results that contain more realistic fine-scale features. Since it is obtained from observational data, AtmoDist also provides a novel perspective on atmospheric predictability.
A Note on "Assessing Generalization of SGD via Disagreement"

Several recent works find empirically that the average test error of deep neural networks can be estimated via the prediction disagreement of models, which does not require labels. In particular, Jiang et al. (2022) show for the disagreement between two separately trained networks that this `Generalization Disagreement Equality' follows from the well-calibrated nature of deep ensembles under the notion of a proposed `class-aggregated calibration. ' In this reproduction, we show that the suggested theory might be impractical because a deep ensemble's calibration can deteriorate as prediction disagreement increases, which is precisely when the coupling of test error and disagreement is of interest, while labels are needed to estimate the calibration on new datasets. Further, we simplify the theoretical statements and proofs, showing them to be straightforward within a probabilistic context, unlike the original hypothesis space view employed by Jiang et al. (2022).
To Impute or not to Impute? - Missing Data in Treatment Effect Estimation

Missing data is a systemic problem in practical scenarios that causes noise and bias when estimating treatment effects. This makes treatment effect estimation from data with missingness a particularly tricky endeavour. A key reason for this is that standard assumptions on missingness are rendered insufficient due to the presence of an additional variable, treatment, besides the input (e.g. an individual) and the label (e.g. an outcome). The treatment variable introduces additional complexity with respect to why some variables are missing that is not fully explored by previous work. In our work we introduce mixed confounded missingness (MCM), a new missingness mechanism where some missingness determines treatment selection and other missingness is determined by treatment selection. Given MCM, we show that naively imputing all data leads to poor performing treatment effects models, as the act of imputation effectively removes information necessary to provide unbiased estimates. However, no imputation at all also leads to biased estimates, as missingness determined by treatment introduces bias in covariates. Our solution is selective imputation, where we use insights from MCM to inform precisely which variables should be imputed and which should not. We empirically demonstrate how various learners benefit from selective imputation compared to other solutions for missing data. We highlight that our experiments encompass both average treatment effects and conditional average treatment effects.
Stochastic smoothing of the top-K calibrated hinge loss for deep imbalanced classification

In modern classification tasks, the number of labels is getting larger and larger, as is the size of the datasets encountered in practice. As the number of classes increases, class ambiguity and class imbalance become more and more problematic to achieve high top-1 accuracy. Meanwhile, Top-K metrics (metrics allowing K guesses) have become popular, especially for performance reporting. Yet, proposing top-K losses tailored for deep learning remains a challenge, both theoretically and practically. In this paper we introduce a stochastic top-K hinge loss inspired by recent developments on top-K calibrated losses. Our proposal is based on the smoothing of the top-K operator building on the flexible"perturbed optimizer"framework. We show that our loss function performs very well in the case of balanced datasets, while benefiting from a significantly lower computational time than the state-of-the-art top-K loss function. In addition, we propose a simple variant of our loss for the imbalanced case. Experiments on a heavy-tailed dataset show that our loss function significantly outperforms other baseline loss functions.
Active Visuo-Tactile Interactive Robotic Perception for Accurate Object Pose Estimation in Dense Clutter

This work presents a novel active visuo-tactile based framework for robotic systems to accurately estimate pose of objects in dense cluttered environments. The scene representation is derived using a novel declutter graph (DG) which describes the relationship among objects in the scene for decluttering by leveraging semantic segmentation and grasp affordances networks. The graph formulation allows robots to efficiently declutter the workspace by autonomously selecting the next best object to remove and the optimal action(prehensile or non-prehensile) to perform.Furthermore, we propose a novel translation-invariant Quaternion filter (TIQF) for active vision and active tactile based pose estimation. Both active visual and active tactile points are selected by maximising the expected information gain. We evaluate our proposed framework on a system with two robots coordinating on randomised scenes of dense cluttered objects and perform ablation studies with static vision and active vision based estimation prior and post decluttering a baselines. Our proposed active visuo-tactile interactive perception framework shows up to 36% improvement in pose accuracy compared to active vision base line and lowest average error (less than 1 cm) compared to other state-of-the-art approaches.
Privacy-aware Distributed Hypothesis Testing in Gray-Wyner Network with Side Information

The problem of distributed binary hypothesis testing in the Gray-Wyner network with side information is studied in this paper. An observer has access to a discrete memoryless and stationary source and describes its observation to two detectors via one common and two private channels. The channels are considered error-free but rate-limited. Each detector also has access to its own discrete memoryless and stationary source, i.e., the side information. The goal is to perform two distinct binary hypothesis testings on the joint distribution of observations at detectors. Additionally, the observer aims to keep a correlated latent source private against the detectors. Equivocation is used as the measure of the privacy preserved for the latent source. An achievable inner bound is derived for the general case by introducing a non-asymptotic account of the output statistics of the random binning.
Self-supervised Learning with Random-projection Quantizer for Speech Recognition

We present a simple and effective self-supervised learning approach for speech recognition. The approach learns a model to predict the masked speech signals, in the form of discrete labels generated with a random-projection quantizer. In particular the quantizer projects speech inputs with a randomly initialized matrix, and does a nearest-neighbor lookup in a randomly-initialized codebook. Neither the matrix nor the codebook is updated during self-supervised learning. Since the random-projection quantizer is not trained and is separated from the speech recognition model, the design makes the approach flexible and is compatible with universal speech recognition architecture. On LibriSpeech our approach achieves similar word-error-rates as previous work using self-supervised learning with non-streaming models, and provides lower word-error-rates and latency than wav2vec 2.0 and w2v-BERT with streaming models. On multilingual tasks the approach also provides significant improvement over wav2vec 2.0 and w2v-BERT.
Hyperboloidal method for frequency-domain self-force calculations

Gravitational self-force theory is the leading approach for modeling gravitational wave emission from small mass-ratio compact binaries. This method perturbatively expands the metric of the binary in powers of the mass ratio. The source for the perturbations depends on the orbital configuration, calculational approach, and the order of the perturbative expansion. These sources fall into three broad classes: (i) distributional, (ii) worldtube, and (iii) unbounded support. The latter, in particular, is important for emerging second-order (in the mass ratio) calculations. Traditional frequency domain approaches employ the variation of parameters method and compute the perturbation on standard time slices with numerical boundary conditions supplied at finite radius from series expansions of the asymptotic behavior. This approach has been very successful, but the boundary conditions calculations are tedious, and the approach is not well suited to unbounded sources where homogeneous solutions must be computed at all radii. This work develops an alternative approach where hyperboloidal slices foliate the spacetime, and compactifying coordinates simplify the boundary treatment. We implement this approach with a multi-domain spectral solver with analytic mesh refinement and use the scalar-field self-force on circular orbits around a Schwarzschild black hole as an example problem. The method works efficiently for all three source classes encountered in self-force calculations and has distinct advantages over the traditional approach. For example, our code efficiently computes the perturbation for orbits with extremely large orbital radii ($r_{p}>10^5M$) or modes with very high spherical harmonic mode index ($\ell \ge 100$). Our results indicate that hyperboloidal methods can play an essential role in self-force calculations.
Wideband Multi-User MIMO Communications with Frequency Selective RISs: Element Response Modeling and Sum-Rate Maximization

Reconfigurable Intelligent Surfaces (RISs) are an emerging technology for future wireless communication systems, enabling improved coverage in an energy efficient manner. RISs are usually metasurfaces, constituting of two-dimensional arrangements of metamaterial elements, whose individual re-sponse is commonly modeled in the literature as an adjustable phase shifter. However, this model holds only for narrowband communications, and when wideband transmissions are utilized, one has to account for the frequency selectivity of metamaterials, whose response usually follows a Lorentzian-like profile. In this paper, we consider the uplink of a wideband RIS-empowered multi-user Multiple-Input Multiple-Output (MIMO) wireless sys-tem with Orthogonal Frequency Division Multiplexing (OFDM) signaling, while accounting for the frequency selectivity of RISs. In particular, we focus on designing the controllable parameters dictating the Lorentzian response of each RIS metamaterial element, in order to maximize the achievable sum rate. We devise a scheme combining block coordinate descent with penalty dual decomposition to tackle the resulting challenging optimization framework. Our simulation results reveal the achievable rates one can achieve using realistically frequency selective RISs in wide band settings, and quantify the performance loss that occurs when using state-of-the-art methods which assume that the RIS elements behave as frequency-flat phase shifters.
The DESI PRObabilistic Value-added Bright Galaxy Survey (PROVABGS) Mock Challenge

The PRObabilistic Value-added Bright Galaxy Survey (PROVABGS) catalog will provide measurements of galaxy properties, such as stellar mass (M *), star formation rate (SFR), stellar metallicity (Z), and stellar age (t age), for >10 million galaxies of the Dark Energy Spectroscopic Instrument (DESI) Bright Galaxy Survey. Full posterior distributions of the galaxy properties will be inferred using state-of-the-art Bayesian spectral energy distribution (SED) modeling of DESI spectroscopy and Legacy Surveys photometry. In this work, we present the SED model, the neural emulator for the model, and the Bayesian inference framework of PROVABGS. Furthermore, we apply the PROVABGS SED modeling on realistic synthetic DESI spectra and photometry, constructed using the L-Galaxies semi-analytic model. We compare the inferred galaxy properties to the true values of the simulation using a hierarchical Bayesian framework to quantify accuracy and precision. Overall, we accurately infer the true M *, SFR, Z, and t age of the simulated galaxies. However, the priors on galaxy properties induced by the SED model have a significant impact on the posteriors, which we characterize in detail. This work also demonstrates that a joint analysis of spectra and photometry significantly improves the constraints on galaxy properties over photometry alone and is necessary to mitigate the impact of the priors. With the methodology presented and validated in this work, PROVABGS will maximize information extracted from DESI observations and extend current galaxy studies to new regimes and unlock cutting-edge probabilistic analyses. https://github.com/changhoonhahn/provabgs/
An exponentially convergent discretization for space-time fractional parabolic equations using hp-FEM

We consider a space-time fractional parabolic problem. Combining a sinc-quadrature based method for discretizing the Riesz-Dunford integral with $hp$-FEM in space yields an exponentially convergent scheme for the initial boundary value problem with homogeneous right-hand side. For the inhomogeneous problem, an $hp$-quadrature scheme is implemented. We rigorously prove exponential convergence with focus on small times $t$, proving robustness with respect to startup singularities due to data incompatibilities.
Data Scaling Laws in NMT: The Effect of Noise and Architecture

In this work, we study the effect of varying the architecture and training data quality on the data scaling properties of Neural Machine Translation (NMT). First, we establish that the test loss of encoder-decoder transformer models scales as a power law in the number of training samples, with a dependence on the model size. Then, we systematically vary aspects of the training setup to understand how they impact the data scaling laws. In particular, we change the following (1) Architecture and task setup: We compare to a transformer-LSTM hybrid, and a decoder-only transformer with a language modeling loss (2) Noise level in the training distribution: We experiment with filtering, and adding iid synthetic noise. In all the above cases, we find that the data scaling exponents are minimally impacted, suggesting that marginally worse architectures or training data can be compensated for by adding more data. Lastly, we find that using back-translated data instead of parallel data, can significantly degrade the scaling exponent.
Relativistic self-consistent 
GW
: Exact two-component formalism with one-electron approximation for solids

We present a formulation of relativistic self-consistent GW for solids based on the exact two-component formalism with one-electron approximation (X2C1e) and non-relativistic Coulomb interactions. Our theory allows us to study scalar relativistic eﬀects, spin-orbit coupling, and the interplay of relativistic eﬀects with electron correlation without adjustable parameters. Our all-electron implementation is fully ab initio and does not require a pseudopotential constructed from atomic calculations. We examine the eﬀect of the X2C1e approximation by comparison to the established four-component formalism and reach excellent agreement. The simplicity of X2C1e en-ables the construction of higher order theories, such as embedding theories, on top of perturbative calculations.
Static Solutions to the Spherically Symmetric Einstein-Vlasov System: A Particle Number-Casimir Approach

Existence of spherically symmetric solutions to the Einstein-Vlasov system is well-known. However, it is an open problem whether or not static solutions arise as minimizers of a variational problem. Apart from being of interest in its own right, it is the connection to non-linear stability that gives this topic its importance. This problem was considered in \cite{Wol}, but as has been pointed out in \cite{AK}, the paper \cite{Wol} contained serious flaws. In this work we construct static solutions by solving the Euler-Lagrange equation for the energy density $\rho$ as a fixed point problem. The Euler-Lagrange equation originates from the particle number-Casimir functional introduced in \cite{Wol}. We then define a density function $f$ on phase space which induces the energy density $\rho$ and we show that it constitutes a static solution of the Einstein-Vlasov system. Hence we settle rigorously parts of what the author of \cite{Wol} attempted to prove.
Industry Characteristics and Financial Risk Spillovers

This paper proposes a new measure of tail risk spillover, namely the conditional coexceedance which is the number of joint occurrences of extreme negative returns in an industry conditional on an extreme negative return in the financial sector. The empirical application provides evidence of significant volatility and tail risk spillovers from the financial sector to many real economy sectors in the U.S. economy in the period from 2001 to 2011. These spillovers increase in crisis periods. The conditional coexceedance in a given sector is positively related to its amount of debt financing, and negatively related to its relative valuation and investment. Therefore real economy sectors which require substantial external financing, and whose value and investment activity are relatively lower are prime candidates for depreciation in the wake of crisis in the financial sector. We also find some evidence suggesting that the higher the industry’s degree of competition the stronger the tail risk spillover from the financial sector.