Miracle-dz/UniFAR-FTUs huggingface dataset:
A Survey on the Robustness of Feature Importance and Counterfactual Explanations

There exist several methods that aim to address the crucial task of understanding the behaviour of AI/ML models. Arguably, the most popular among them are local explanations that focus on investigating model behaviour for individual instances. Several methods have been proposed for local analysis, but relatively lesser effort has gone into understanding if the explanations are robust and accurately reflect the behaviour of underlying models. In this work, we present a survey of the works that analysed the robustness of two classes of local explanations (feature importance and counterfactual explanations) that are popularly used in analysing AI/ML models in finance. The survey aims to unify existing definitions of robustness, introduces a taxonomy to classify different robustness approaches, and discusses some interesting results. Finally, the survey introduces some pointers about extending current robustness analysis approaches so as to identify reliable explainability methods.
Low-symmetry nonlocal transport in microstructured squares of delafossite metals

Significance The strongly two-dimensional delafossite metals are among the highest-purity compounds known. Their huge low-temperature electron mean free paths of up to 20 µm allow fabrication of devices in the ballistic regime, in which the characteristic device dimension is shorter than the average distance between scattering events. Unlike any previous case in which this regime has been reached, the faceting of the nearly hexagonal Fermi surfaces leads to extremely strong directional effects. Remarkably, in the ballistic regime, this results in the transport response of squares sculpted from delafossite crystals not showing full fourfold symmetry. Although seemingly counterintuitive, our observations are consistent with the overall symmetry of the device–Fermi surface combination and fully obey the Onsager–Büttiker relations for nonlocal transport. Intense work studying the ballistic regime of electron transport in two-dimensional systems based on semiconductors and graphene had been thought to have established most of the key experimental facts of the field. In recent years, however, additional forms of ballistic transport have become accessible in the quasi–two-dimensional delafossite metals, whose Fermi wavelength is a factor of 100 shorter than those typically studied in the previous work and whose Fermi surfaces are nearly hexagonal in shape and therefore strongly faceted. This has some profound consequences for results obtained from the classic ballistic transport experiment of studying bend and Hall resistances in mesoscopic squares fabricated from delafossite single crystals. We observe pronounced anisotropies in bend resistances and even a Hall voltage that is strongly asymmetric in magnetic field. Although some of our observations are nonintuitive at first sight, we show that they can be understood within a nonlocal Landauer-Büttiker analysis tailored to the symmetries of the square/hexagonal geometries of our combined device/Fermi surface system. Signatures of nonlocal transport can be resolved for squares of linear dimension of nearly 100 µm, approximately a factor of 15 larger than the bulk mean free path of the crystal from which the device was fabricated.
Comparative Explanations of Recommendations

As recommendation is essentially a comparative (or ranking) process, a good explanation should illustrate to users why an item is believed to be better than another, i.e., comparative explanations about the recommended items. Ideally, after reading the explanations, a user should reach the same ranking of items as the system’s. Unfortunately, little research attention has yet been paid on such comparative explanations. In this work, we develop an extract-and-refine architecture to explain the relative comparisons among a set of ranked items from a recommender system. For each recommended item, we first extract one sentence from its associated reviews that best suits the desired comparison against a set of reference items. Then this extracted sentence is further articulated with respect to the target user through a generative model to better explain why the item is recommended. We design a new explanation quality metric based on BLEU to guide the end-to-end training of the extraction and refinement components, which avoids generation of generic content. Extensive offline evaluations on two large recommendation benchmark datasets and serious user studies against an array of state-of-the-art explainable recommendation algorithms demonstrate the necessity of comparative explanations and the effectiveness of our solution.
Dual Attention Network for Heart Rate and Respiratory Rate Estimation

Heart rate and respiratory rate measurement is a vital step for diagnosing many diseases. Non-contact camera based physiological measurement is more accessible and convenient in Telehealth nowadays than contact instruments such as fingertip oximeters since non-contact methods reduce risk of infection. However, remote physiological signal measurement is challenging due to environment illumination variations, head motion, facial expression, etc. It’s also desirable to have a unified network which could estimate both heart rate and respiratory rate to reduce system complexity and latency. We propose a convolutional neural network which leverages spatial attention and channel attention, which we call it dual attention network (DAN) to jointly estimate heart rate and respiratory rate with camera video as input. Extensive experiments demonstrate that our proposed system significantly improves heart rate and respiratory rate measurement accuracy.
A branching particle system as a model of semi pushed fronts

We consider a system of particles performing a one-dimensional dyadic branching Brownian motion with space-dependent branching rate, negative drift −μ and killed upon reaching 0, starting with N particles. More precisely, particles branch at rate ρ/2 in the interval [0, 1], for some ρ > 1, and at rate 1/2 in (1,+∞). The drift μ(ρ) is chosen in such a way that, heuristically, the system is critical in some sense: the number of particles stays roughly constant before it eventually dies out. This particle system can be seen as an analytically tractable model for fluctuating fronts, describing the internal mechanisms driving the invasion of a habitat by a cooperating population. Recent studies from Birzu, Hallatschek and Korolev suggest the existence of three classes of fluctuating fronts: pulled, semi pushed and pushed fronts. Here, we rigorously verify and make precise this classification and focus on the semi pushed regime. More precisely, we prove the existence of two critical values 1 < ρ1 < ρ2 such that for all ρ ∈ (ρ1, ρ2), there exists α(ρ) ∈ (1, 2) such that the rescaled number of particles in the system converges to an α-stable continuous-state branching process on the time scale Nα−1 as N goes to infinity. This complements previous results from Berestycki, Berestycki and Schweinsberg for the case ρ = 1.
Mastering Atari Games with Limited Data

Reinforcement learning has achieved great success in many applications. However, sample efficiency remains a key challenge, with prominent methods requiring millions (or even billions) of environment steps to train. Recently, there has been significant progress in sample efficient image-based RL algorithms; however, consistent human-level performance on the Atari game benchmark remains an elusive goal. We propose a sample efficient model-based visual RL algorithm built on MuZero, which we name EfficientZero. Our method achieves 194.3% mean human performance and 109.0% median performance on the Atari 100k benchmark with only two hours of real-time game experience and outperforms the state SAC in some tasks on the DMControl 100k benchmark. This is the first time an algorithm achieves super-human performance on Atari games with such little data. EfficientZero's performance is also close to DQN's performance at 200 million frames while we consume 500 times less data. EfficientZero's low sample complexity and high performance can bring RL closer to real-world applicability. We implement our algorithm in an easy-to-understand manner and it is available at https://github.com/YeWR/EfficientZero. We hope it will accelerate the research of MCTS-based RL algorithms in the wider community.
Principal Component Pursuit for Pattern Identification in Environmental Mixtures

Background: Environmental health researchers often aim to identify sources or behaviors that give rise to potentially harmful environmental exposures. Objective: We adapted principal component pursuit (PCP)—a robust and well-established technique for dimensionality reduction in computer vision and signal processing—to identify patterns in environmental mixtures. PCP decomposes the exposure mixture into a low-rank matrix containing consistent patterns of exposure across pollutants and a sparse matrix isolating unique or extreme exposure events. Methods: We adapted PCP to accommodate nonnegative data, missing data, and values below a given limit of detection (LOD). We simulated data to represent environmental mixtures of two sizes with increasing proportions <LOD and three noise structures. We applied PCP-LOD to evaluate its performance in comparison with principal component analysis (PCA). We next applied principal component pursuit with limit of detection (PCP-LOD) to an exposure mixture of 21 persistent organic pollutants (POPs) measured in 1,000 U.S. adults from the 2001–2002 National Health and Nutrition Examination Survey (NHANES). We applied singular value decomposition to the estimated low-rank matrix to characterize the patterns. Results: PCP-LOD recovered the true number of patterns through cross-validation for all simulations; based on an a priori specified criterion, PCA recovered the true number of patterns in 32% of simulations. PCP-LOD achieved lower relative predictive error than PCA for all simulated data sets with up to 50% of the data <LOD. When 75% of values were <LOD, PCP-LOD outperformed PCA only when noise was low. In the POP mixture, PCP-LOD identified a rank-three underlying structure and separated 6% of values as extreme events. One pattern represented comprehensive exposure to all POPs. The other patterns grouped chemicals based on known structure and toxicity. Discussion: PCP-LOD serves as a useful tool to express multidimensional exposures as consistent patterns that, if found to be related to adverse health, are amenable to targeted public health messaging. https://doi.org/10.1289/EHP10479
Deep Learning Transformer Architecture for Named-Entity Recognition on Low-Resourced Languages: State of the art results

This paper reports on the evaluation of Deep Learning (DL) transformer architecture models for Named-Entity Recognition (NER) on ten low-resourced South African (SA) languages. In addition, these DL transformer models were com-pared to other Neural Network and Machine Learning (ML) NER models. The findings show that transformer models substantially improve performance when applying discrete fine-tuning parameters per language. Furthermore, fine-tuned transformer models outperform other neural network and ma-chine learning models on NER with the low-resourced SA languages. For example, the transformer models obtained the highest F-scores for six of the ten SA languages and the highest average F-score surpassing the Conditional Random Fields ML model. Practical implications include developing high-performance NER capability with less effort and resource costs, potentially improving downstream NLP tasks such as Machine Translation (MT). Therefore, the application of DL trans-former architecture models for NLP NER sequence tagging tasks on low-resourced SA languages is viable. Additional re-search could evaluate the more recent transformer architecture models on other Natural Language Processing tasks and applications, such as Phrase chunking, MT, and Part-of-Speech tagging.
Measuring a Texts Fairness Dimensions Using Machine Learning Based on Social Psychological Factors

Fairness is a principal social value that can be observed in civilisations around the world. A manifestations of this is in social agreements, often described in texts, such as contracts. Yet, despite the prevalence of such, a fairness metric for texts describing a social act remains wanting. To address this, we take a step back to consider the problem based on first principals. Instead of using rules or templates, we utilise social psychology literature to determine the principal factors that humans use when making a fairness assessment. We then attempt to digitise these using word embeddings into a multi-dimensioned sentence level fairness perceptions vector to serve as an approximation for these fairness perceptions. The method leverages a pro-social bias within word embeddings, for which we obtain an F1= 81.0. A second approach, using PCA and ML based on the said fairness approximation vector produces an F1 score of 86.2. We details improvements that can be made in the methodology to incorporate the projection of sentence embedding on to a subspace representation of fairness.
Sustainable AI: Environmental Implications, Challenges and Opportunities

This paper explores the environmental impact of the super-linear growth trends for AI from a holistic perspective, spanning Data, Algorithms, and System Hardware. We characterize the carbon footprint of AI computing by examining the model development cycle across industry-scale machine learning use cases and, at the same time, considering the life cycle of system hardware. Taking a step further, we capture the operational and manufacturing carbon footprint of AI computing and present an end-to-end analysis for what and how hardware-software design and at-scale optimization can help reduce the overall carbon footprint of AI. Based on the industry experience and lessons learned, we share the key challenges and chart out important development directions across the many dimensions of AI. We hope the key messages and insights presented in this paper can inspire the community to advance the field of AI in an environmentally-responsible manner.
Learning linear non-Gaussian directed acyclic graph with diverging number of nodes

Acyclic model, often depicted as a directed acyclic graph (DAG), has been widely employed to represent directional causal relations among collected nodes. In this article, we propose an efficient method to learn linear non-Gaussian DAG in high dimensional cases, where the noises can be of any continuous non-Gaussian distribution. This is in sharp contrast to most existing DAG learning methods assuming Gaussian noise with additional variance assumptions to attain exact DAG recovery. The proposed method leverages a novel concept of topological layer to facilitate the DAG learning. Particularly, we show that the topological layers can be exactly reconstructed in a bottom-up fashion, and the parent-child relations among nodes in each layer can also be consistently established. More importantly, the proposed method does not require the faithfulness or parental faithfulness assumption which has been widely assumed in the literature of DAG learning. Its advantage is also supported by the numerical comparison against some popular competitors in various simulated examples as well as a real application on the global spread of COVID-19.
Skyformer: Remodel Self-Attention with Gaussian Kernel and Nyström Method

Transformers are expensive to train due to the quadratic time and space complexity in the self-attention mechanism. On the other hand, although kernel machines suffer from the same computation bottleneck in pairwise dot products, several approximation schemes have been successfully incorporated to considerably reduce their computational cost without sacrificing too much accuracy. In this work, we leverage the computation methods for kernel machines to alleviate the high computational cost and introduce Skyformer, which replaces the softmax structure with a Gaussian kernel to stabilize the model training and adapts the Nystr\"om method to a non-positive semidefinite matrix to accelerate the computation. We further conduct theoretical analysis by showing that the matrix approximation error of our proposed method is small in the spectral norm. Experiments on Long Range Arena benchmark show that the proposed method is sufficient in getting comparable or even better performance than the full self-attention while requiring fewer computation resources.
Hebbian learning with gradients: Hebbian convolutional neural networks with modern deep learning frameworks

Deep learning networks generally use non-biological learning methods. By contrast, networks based on more biologically plausible learning, such as Hebbian learning, show comparatively poor performance and difficulties of implementation. Here we show that Hebbian learning in hierarchical, convolutional neural networks can be implemented almost trivially with modern deep learning frameworks, by using specific losses whose gradients produce exactly the desired Hebbian updates. We provide expressions whose gradients exactly implement a plain Hebbian rule (dw ~= xy), Grossberg's instar rule (dw ~= y(x-w)), and Oja's rule (dw ~= y(x-yw)). As an application, we build Hebbian convolutional multi-layer networks for object recognition. We observe that higher layers of such networks tend to learn large, simple features (Gabor-like filters and blobs), explaining the previously reported decrease in decoding performance over successive layers. To combat this tendency, we introduce interventions (denser activations with sparse plasticity, pruning of connections between layers) which result in sparser learned features, massively increase performance, and allow information to increase over successive layers. We hypothesize that more advanced techniques (dynamic stimuli, trace learning, feedback connections, etc.), together with the massive computational boost offered by modern deep learning frameworks, could greatly improve the performance and biological relevance of multi-layer Hebbian networks.
Factorization of the Partial Covariance in Singly-Connected Path Diagrams

We extend path analysis by showing that, for a singly-connected path diagram, the partial covariance of two random variables factorizes over the nodes and edges in the path between the variables. This result allows us to determine the contribution of each node and edge to the partial covariance. It also allows us to show that Simpson's paradox cannot occur in singly-connected path diagrams.
FairBalance: Improving Machine Learning Fairness on Multiple Sensitive Attributes With Data Balancing

This paper aims to improve machine learning fairness on multiple sensitive attributes. Machine learning fairness has attracted increasing attention since machine learning software is increasingly used for high-stakes and high-risk decisions. Most existing solutions for machine learning fairness either target only one sensitive attribute (e.g. sex) at a time, or have magic parameters to tune, or have expensive computational overhead. To overcome these challenges, we propose FairBalance to balance the group distribution of training data across every sensitive attribute before training the machine learning models. Our results show that, under the assumption of unbiased ground truth labels, at low computational overhead, Fair-Balance can significantly reduce fairness metrics (AOD, EOD, and SPD) on every known sensitive attribute without much, if any damage to the prediction performance. In addition, FairBalanceClass, a variant of FairBalance, can balance the class distribution in the training data. With FairBalanceClass, predictions will no longer favor the majority class, thus achieving a higher F 1 score on the minority class. FairBalance and FairBalanceClass also outperform other state-of-the-art bias mitigation algorithms in terms of prediction performance and fairness metrics. This research will benefit society by providing a simple yet effective approach to improve fairness of machine learning software on data with multiple sensitive attributes. Our results also validate the hypothesis that, on datasets with unbiased ground truth labels, ethical biases in the learned models largely attribute to the training data having (1) difference in group size and (2) difference in class distribution within each group. To facilitate reuse, reproduction, and validation of this work, our scripts and data are available at https://anonymous.4open. science/ r/FairBalance-1E44 under an open-source Apache license (v2.0)
Quantitative insights into effects of intrapartum antibiotics and birth mode on infant gut microbiota in relation to well-being during the first year of life

ABSTRACT Birth mode and maternal intrapartum (IP) antibiotics affect infants’ gut microbiota development, but their relative contribution to absolute bacterial abundances and infant health has not been studied. We compared the effects of Cesarean section (CS) delivery and IP antibiotics on infant gut microbiota development and well-being over the first year. We focused on 92 healthy infants born between gestational weeks 37–42 vaginally without antibiotics (N = 26), with IP penicillin (N = 13) or cephalosporin (N = 7) or by CS with IP cephalosporin (N = 33) or other antibiotics (N = 13). Composition and temporal development analysis of the gut microbiota concentrated on 5 time points during the first year of life using 16S rRNA gene amplicon sequencing, integrated with qPCR to obtain absolute abundance estimates. A mediation analysis was carried out to identify taxa linked to gastrointestinal function and discomfort (crying, defecation frequency, and signs of gastrointestinal symptoms), and birth interventions. Based on absolute abundance estimates, the depletion of Bacteroides spp. was found specifically in CS birth, while decreased bifidobacteria and increased Bacilli were common in CS birth and exposure to IP antibiotics in vaginal delivery. The abundances of numerous taxa differed between the birth modes among cephalosporin-exposed infants. Penicillin had a milder impact on the infant gut microbiota than cephalosporin. CS birth and maternal IP antibiotics had both specific and overlapping effects on infants’ gut microbiota development. The resulting deviations in the gut microbiota are associated with increased defecation rate, flatulence, perceived stomach pain, and intensity of crying in infancy.