Miracle-dz/UniFAR-FTUs huggingface dataset:
Do Input Gradients Highlight Discriminative Features?

Post-hoc gradient-based interpretability methods [Simonyan et al., 2013, Smilkov et al., 2017] that provide instance-specific explanations of model predictions are often based on assumption (A): magnitude of input gradients -- gradients of logits with respect to input -- noisily highlight discriminative task-relevant features. In this work, we test the validity of assumption (A) using a three-pronged approach. First, we develop an evaluation framework, DiffROAR, to test assumption (A) on four image classification benchmarks. Our results suggest that (i) input gradients of standard models (i.e., trained on original data) may grossly violate (A), whereas (ii) input gradients of adversarially robust models satisfy (A). Second, we introduce BlockMNIST, an MNIST-based semi-real dataset, that by design encodes a priori knowledge of discriminative features. Our analysis on BlockMNIST leverages this information to validate as well as characterize differences between input gradient attributions of standard and robust models. Finally, we theoretically prove that our empirical findings hold on a simplified version of the BlockMNIST dataset. Specifically, we prove that input gradients of standard one-hidden-layer MLPs trained on this dataset do not highlight instance-specific signal coordinates, thus grossly violating assumption (A). Our findings motivate the need to formalize and test common assumptions in interpretability in a falsifiable manner [Leavitt and Morcos, 2020]. We believe that the DiffROAR evaluation framework and BlockMNIST-based datasets can serve as sanity checks to audit instance-specific interpretability methods; code and data available at https://github.com/harshays/inputgradients.
Evidence for X(3872) in Pb-Pb Collisions and Studies of its Prompt Production at sqrt[s_{NN}]=5.02  TeV.

The first evidence for X(3872) production in relativistic heavy ion collisions is reported. The X(3872) production is studied in lead-lead (Pb-Pb) collisions at a center-of-mass energy of sqrt[s_{NN}]=5.02 TeV per nucleon pair, using the decay chain X(3872)→J/ψπ^{+}π^{-}→μ^{+}μ^{-}π^{+}π^{-}. The data were recorded with the CMS detector in 2018 and correspond to an integrated luminosity of 1.7 nb^{-1}. The measurement is performed in the rapidity and transverse momentum ranges |y|<1.6 and 15<p_{T}<50 GeV/c. The significance of the inclusive X(3872) signal is 4.2 standard deviations. The prompt X(3872) to ψ2S yield ratio is found to be ρ^{Pb-Pb}=1.08±0.49(stat)±0.52(syst), to be compared with typical values of 0.1 for pp collisions. This result provides a unique experimental input to theoretical models of the X(3872) production mechanism, and of the nature of this exotic state.
A Thin Self-Stabilizing Asynchronous Unison Algorithm with Applications to Fault Tolerant Biological Networks

Introduced by Emek and Wattenhofer (PODC 2013), the stone age (SA) model provides an abstraction for network algorithms distributed over randomized finite state machines. This model, designed to resemble the dynamics of biological processes in cellular networks, assumes a weak communication scheme that is built upon the nodes' ability to sense their vicinity in an asynchronous manner. Recent works demonstrate that the weak computation and communication capabilities of the SA model suffice for efficient solutions to some core tasks in distributed computing, but they do so under the (somewhat less realistic) assumption of fault free computations. In this paper, we initiate the study of self-stabilizing SA algorithms that are guaranteed to recover from any combination of transient faults. Specifically, we develop efficient self-stabilizing SA algorithms for the leader election and maximal independent set tasks in bounded diameter graphs subject to an asynchronous scheduler. These algorithms rely on a novel efficient self-stabilizing asynchronous unison (AU) algorithm that is "thin'' in terms of its state space: the number of states used by the AU algorithm is linear in the graph's diameter bound, irrespective of the number of nodes.
A mathematical modelling framework for the regulation of intra-cellular OCT4 in human pluripotent stem cells

Human pluripotent stem cells (hPSCs) have the potential to differentiate into all cell types, a property known as pluripotency. A deeper understanding of how pluripotency is regulated is required to assist in controlling pluripotency and differentiation trajectories experimentally. Mathematical modelling provides a non-invasive tool through which to explore, characterise and replicate the regulation of pluripotency and the consequences on cell fate. Here we use experimental data of the expression of the pluripotency transcription factor OCT4 in a growing hPSC colony to develop and evaluate mathematical models for temporal pluripotency regulation. We consider fractional Brownian motion and the stochastic logistic equation and explore the effects of both additive and multiplicative noise. We illustrate the use of time-dependent carrying capacities and the introduction of Allee effects to the stochastic logistic equation to describe cell differentiation. We conclude both methods adequately capture the decline in OCT4 upon differentiation, but the Allee effect model has the advantage of allowing differentiation to occur stochastically in a sub-set of cells. This mathematical framework for describing intra-cellular OCT4 regulation can be extended to other transcription factors and developed into predictive models.
Task-Agnostic Morphology Evolution

Deep reinforcement learning primarily focuses on learning behavior, usually overlooking the fact that an agent's function is largely determined by form. So, how should one go about finding a morphology fit for solving tasks in a given environment? Current approaches that co-adapt morphology and behavior use a specific task's reward as a signal for morphology optimization. However, this often requires expensive policy optimization and results in task-dependent morphologies that are not built to generalize. In this work, we propose a new approach, Task-Agnostic Morphology Evolution (TAME), to alleviate both of these issues. Without any task or reward specification, TAME evolves morphologies by only applying randomly sampled action primitives on a population of agents. This is accomplished using an information-theoretic objective that efficiently ranks agents by their ability to reach diverse states in the environment and the causality of their actions. Finally, we empirically demonstrate that across 2D, 3D, and manipulation environments TAME can evolve morphologies that match the multi-task performance of those learned with task supervised algorithms. Our code and videos can be found at https://sites.google. com/view/task-agnostic-evolution.
Estimation and Distributed Eradication of SIR Epidemics on Networks

This work examines the discrete-time networked SIR (susceptible-infected-recovered) epidemic model, where the infection and recovery parameters may be time-varying. We provide a sufficient condition for the SIR model to converge to the set of healthy states exponentially. We propose a stochastic framework to estimate the system states from observed testing data and provide an analytic expression for the error of the estimation algorithm. Employing the estimated and the true system states, we provide two novel eradication strategies that guarantee at least exponential convergence to the set of healthy states. We illustrate the results via simulations over northern Indiana, USA.
Clay-Organic Complexes as Adsorbents for Phenol and Chlorophenols

Several clay-organic complexes were synthesized by placing quaternary ammonium cations on smectite by cation exchange. They were then examined for their ability to adsorb phenol and several of its chlorinated congeners. The organic cations used were: hexadecylpyridinium (HDPY+), hexadecyltri-methyl ammonium (HDTMA+), trimethylphenyl ammonium (TMPA), and tetramethylammonium (TMA+). The complexes containing long-chain alkyl (hexadecyl) groups were the most hydrophobic and adsorbed the phenols from water in proportion to their hydrophobicities, which increase with chlorine addition (phenol < chlorophenol < dichloropohenol < trichlorophenol). With n-hexane as the solvent, different adsorption was found which depended on the type and degree of solvent interactions with the compound and the clay-organic complex. Thus, the amount of adsorption of these phenols on clay-organic complexes was dependent on the relative energies of adsorbent-adsorbate and adsorbate-solvent interactions.
Machine Learning-Based Optimal Mesh Generation in Computational Fluid Dynamics

Computational Fluid Dynamics (CFD) is a major sub-field of engineering. Corresponding flow simulations are typically characterized by heavy computational resource requirements. Often, very fine and complex meshes are required to resolve physical effects in an appropriate manner. Since all CFD algorithms scale at least linearly with the size of the underlying mesh discretization, finding an optimal mesh is key for computational efficiency. One methodology used to find optimal meshes is goal-oriented adaptive mesh refinement. However, this is typically computationally demanding and only available in a limited number of tools. Within this contribution, we adopt a machine learning approach to identify optimal mesh densities. We generate optimized meshes using classical methodologies and propose to train a convolutional network predicting optimal mesh densities given arbitrary geometries. The proposed concept is validated along 2d wind tunnel simulations with more than 60,000 simulations. Using a training set of 20,000 simulations we achieve accuracies of more than 98.7%. Corresponding predictions of optimal meshes can be used as input for any mesh generation and CFD tool. Thus without complex computations, any CFD engineer can start his predictions from a high quality mesh.
Investigating the Limitations of Transformers with Simple Arithmetic Tasks

The ability to perform arithmetic tasks is a remarkable trait of human intelligence and might form a critical component of more complex reasoning tasks. In this work, we investigate if the surface form of a number has any influence on how sequence-to-sequence language models learn simple arithmetic tasks such as addition and subtraction across a wide range of values. We find that how a number is represented in its surface form has a strong influence on the model's accuracy. In particular, the model fails to learn addition of five-digit numbers when using subwords (e.g.,"32"), and it struggles to learn with character-level representations (e.g.,"3 2"). By introducing position tokens (e.g.,"3 10e1 2"), the model learns to accurately add and subtract numbers up to 60 digits. We conclude that modern pretrained language models can easily learn arithmetic from very few examples, as long as we use the proper surface representation. This result bolsters evidence that subword tokenizers and positional encodings are components in current transformer designs that might need improvement. Moreover, we show that regardless of the number of parameters and training examples, models cannot learn addition rules that are independent of the length of the numbers seen during training. Code to reproduce our experiments is available at https://github.com/castorini/transformers-arithmetic
Federated Multi-armed Bandits with Personalization

A general framework of personalized federated multi-armed bandits (PF-MAB) is proposed, which is a new bandit paradigm analogous to the federated learning (FL) framework in supervised learning and enjoys the features of FL with personalization. Under the PF-MAB framework, a mixed bandit learning problem that flexibly balances generalization and personalization is studied. A lower bound analysis for the mixed model is presented. We then propose the Personalized Federated Upper Confidence Bound (PF-UCB) algorithm, where the exploration length is chosen carefully to achieve the desired balance of learning the local model and supplying global information for the mixed learning objective. Theoretical analysis proves that PF-UCB achieves an $O(\log(T))$ regret regardless of the degree of personalization, and has a similar instance dependency as the lower bound. Experiments using both synthetic and real-world datasets corroborate the theoretical analysis and demonstrate the effectiveness of the proposed algorithm.
Revisiting Optimal Resilience of Fast Byzantine Consensus

It is a common belief that Byzantine fault-tolerant solutions for consensus are significantly slower than their crash fault-tolerant counterparts. Indeed, in PBFT, the most widely known Byzantine fault-tolerant consensus protocol, it takes three message delays to decide a value, in contrast with just two in Paxos. This motivates the search for fast Byzantine consensus algorithms that can produce decisions after just two message delays in the common case, e.g., under the assumption that the current leader is correct and not suspected by correct processes. The (optimal) two-step latency comes with the cost of lower resilience: fast Byzantine consensus requires more processes to tolerate the same number of faults. In particular, 5f+1 processes were claimed to be necessary to tolerate f Byzantine failures. In this paper, we present a fast Byzantine consensus algorithm that relies on just 5f-1 processes. Moreover, we show that 5f-1 is the tight lower bound, correcting a mistake in the earlier work. While the difference of just 2 processes may appear insignificant for large values of f, it can be crucial for systems of a smaller scale. In particular, for f=1, our algorithm requires only 4 processes, which is optimal for any (not necessarily fast) partially synchronous Byzantine consensus algorithm.
Baryogenesis and gravity waves from a UV-completed electroweak phase transition

We study gravity wave production and baryogenesis at the electroweak phase transition, in a real singlet scalar extension of the Standard Model, including vector-like top partners to generate the CP violation needed for baryogenesis. The singlet makes the phase transition strongly firstorder through its coupling to the Higgs boson, and it spontaneously breaks CP invariance through a dimension-5 contribution to the top quark mass term, generated by integrating out the heavy top quark partners. We improve on previous studies by incorporating updated transport equations, compatible with large bubble wall velocities vw, to determine the friction on the wall, and thereby vw and the wall thickness, rather than treating these as free parameters. The baryon asymmetry is also computed with no assumptions, directly from the microphysical parameters. The size of the CPviolating dimension-5 operator is constrained by collider, electroweak precision, and renormalization group running constraints. We identify regions of parameter space that can produce the observed baryon asymmetry, and simultaneously produce gravitational waves that could be observed by future experiments. Contrary to standard lore, we find that for strong deflagrations, the efficiencies of large baryon asymmetry production and strong GW-signal can be positively correlated.
Upper Limit on Brackett-γ Emission from the Immediate Accretion Flow onto the Galactic Black Hole

We present the first observational constraint on the Brackett-γ (Brγ) recombination line emission associated with the supermassive black hole at the center of our Galaxy, known as Sgr A*. By combining 13 yr of data with the adaptive optics fed integral field spectrograph OSIRIS at the W. M. Keck Observatory obtained as part of the Galactic Center Orbits Initiative, we extract the near-infrared spectrum within ∼0.2″ of the black hole and we derive an upper limit on the Brγ flux. The aperture was set to match the size of the disk-like structure that was recently reported based on millimeter-wave Atacama Large Millimeter/submillimeter Array (ALMA) observations of the hydrogen recombination line, H30α. Our stringent upper limit is at least a factor of 80 (and up to a factor of 245) below what would be expected from the ALMA measurements and strongly constrains possible interpretation of emission from this highly underluminous supermassive black hole.
Dual MINE-based Neural Secure Communications under Gaussian Wiretap Channel

Recently, some researches are devoted to the topic of end-to-end learning a physical layer secure communication system based on autoencoder under Gaussian wiretap channel. However, in those works, the reliability and security of the encoder model were learned through necessary decoding outputs of not only legitimate receiver but also the eavesdropper. In fact, the assumption of known eavesdropper's decoder or its output is not practical. To address this issue, in this paper we propose a dual mutual information neural estimation (MINE) based neural secure communications model. The security constraints of this method is constructed only with the input and output signal samples of the legal and eavesdropper channels and benefit that training the encoder is completely independent of the decoder. Moreover, since the design of secure coding does not rely on the eavesdropper's decoding results, the security performance would not be affected by the eavesdropper's decoding means. Numerical results show that the performance of our model is guaranteed whether the eavesdropper learns the decoder himself or uses the legal decoder.
Adaptive Robust Large Volatility Matrix Estimation Based on High-Frequency Financial Data

Several novel statistical methods have been developed to estimate large integrated volatility matrices based on high-frequency financial data. To investigate their asymptotic behaviors, they require a sub-Gaussian or finite high-order moment assumption for observed log-returns, which cannot account for the heavy tail phenomenon of stock returns. Recently, a robust estimator was developed to handle heavy-tailed distributions with some bounded fourth-moment assumption. However, we often observe that log-returns have heavier tail distribution than the finite fourth-moment and that the degrees of heaviness of tails are heterogeneous over the asset and time period. In this paper, to deal with the heterogeneous heavy-tailed distributions, we develop an adaptive robust integrated volatility estimator that employs pre-averaging and truncation schemes based on jump-diffusion processes. We call this an adaptive robust pre-averaging realized volatility (ARP) estimator. We show that the ARP estimator has a sub-Weibull tail concentration with only finite 2α-th moments for any α &gt; 1. In addition, we establish matching upper and lower bounds to show that the ARP estimation procedure is optimal. To estimate large integrated volatility matrices using the approximate factor model, the ARP estimator is further regularized using the principal orthogonal complement thresholding (POET) method. The numerical study is conducted to check the finite sample performance of the ARP estimator.
Do Not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learning

The privacy leakage of the model about the training data can be bounded in the differential privacy mechanism. However, for meaningful privacy parameters, a differentially private model degrades the utility drastically when the model comprises a large number of trainable parameters. In this paper, we propose an algorithm \emph{Gradient Embedding Perturbation (GEP)} towards training differentially private deep models with decent accuracy. Specifically, in each gradient descent step, GEP first projects individual private gradient into a non-sensitive anchor subspace, producing a low-dimensional gradient embedding and a small-norm residual gradient. Then, GEP perturbs the low-dimensional embedding and the residual gradient separately according to the privacy budget. Such a decomposition permits a small perturbation variance, which greatly helps to break the dimensional barrier of private learning. With GEP, we achieve decent accuracy with reasonable computational cost and modest privacy guarantee for deep models. Especially, with privacy bound $\epsilon=8$, we achieve $74.9\%$ test accuracy on CIFAR10 and $95.1\%$ test accuracy on SVHN, significantly improving over existing results.
Multifidelity Ensemble Kalman Filtering using surrogate models defined by Physics-Informed Autoencoders

Data assimilation is a Bayesian inference process that obtains an enhanced understanding of a physical system of interest by fusing information from an inexact physics-based model, and from noisy sparse observations of reality. The multifidelity ensemble Kalman filter (MFEnKF) recently developed by the authors combines a full-order physical model and a hierarchy of reduced order surrogate models in order to increase the computational efficiency of data assimilation. The standard MFEnKF uses linear couplings between models, and is statistically optimal in case of Gaussian probability densities. This work extends MFEnKF to work with non-linear couplings between the models. Optimal nonlinear projection and interpolation operators are obtained by appropriately trained physics-informed autoencoders, and this approach allows to construct reduced order surrogate models with less error than conventional linear methods. Numerical experiments with the canonical Lorenz '96 model illustrate that nonlinear surrogates perform better than linear projection-based ones in the context of multifidelity filtering.
A study on the degradation mechanism of InGaZnO thin-film transistors under simultaneous gate and drain bias stresses based on the electronic trap characterization

We discuss the device degradation mechanism of amorphous indium–gallium–zinc oxide (a-IGZO) thin-film transistors (TFTs) under simultaneous gate and drain bias stresses based on the electronic trap characterization results. The transfer curve exhibits an apparent negative shift as the stress time increases, and a formation of hump is observed in the transfer curve after stresses. A notable increase of the frequency dispersion is observed after stresses in both gate-to-drain capacitance–voltage (CGD–VG) and gate-to-source capacitance–voltage (CGS–VG) curves, which implies that the subgap states are generated by simultaneous gate and drain bias stresses, and the damaged location is not limited to the drain side of TFTs. The larger frequency dispersion is observed in CGD–VG curves after stresses in a wider channel device, which implies that the heat is an important factor in the generation of the subgap states under simultaneous gate and drain bias stresses in a-IGZO TFTs. Based on the electronic trap characterization results, we conclude that the impact ionization near the drain side of the device is not a dominant mechanism causing the generation of subgap states and device degradation in a-IGZO TFTs under simultaneous gate and drain bias stresses. The generation of oxygen vacancy-related donor-like traps near the conduction band edge is considered as a possible mechanism causing the device degradation under simultaneous gate and drain bias stresses in a-IGZO TFTs.
High-throughput nanoindentation mapping of cast IN718 nickel-based superalloys: influence of the Nb concentration

A high-throughput correlative study of the local mechanical properties, chemical composition and crystallographic orientation has been carried out in selected areas of cast Inconel 718 specimens subjected to three different tempers. The specimens showed a strong Nb segregation at the scale of the dendrite arms, with local Nb contents that varied between 2 wt% in the core of the dendrite arms to 8 wt% in the interdendritic regions and 25 wt% within the second phase particles (MC carbides, Laves phases and δ phase needles). The nanohardness was found to correlate strongly with the local Nb content and the temper condition. On the contrary, the indentation elastic moduli was not influenced by the local chemical composition or temper condition, but directly correlated with the crystallographic grain orientation, due to the high elastic anisotropy of nickel alloys.