Miracle-dz/UniFAR-FTUs huggingface dataset:
Towards Label Position Bias in Graph Neural Networks

Graph Neural Networks (GNNs) have emerged as a powerful tool for semi-supervised node classification tasks. However, recent studies have revealed various biases in GNNs stemming from both node features and graph topology. In this work, we uncover a new bias - label position bias, which indicates that the node closer to the labeled nodes tends to perform better. We introduce a new metric, the Label Proximity Score, to quantify this bias, and find that it is closely related to performance disparities. To address the label position bias, we propose a novel optimization framework for learning a label position unbiased graph structure, which can be applied to existing GNNs. Extensive experiments demonstrate that our proposed method not only outperforms backbone methods but also significantly mitigates the issue of label position bias in GNNs.
IDEA: Invariant Causal Defense for Graph Adversarial Robustness

Graph neural networks (GNNs) have achieved remarkable success in various tasks, however, their vulnerability to adversarial attacks raises concerns for the real-world applications. Existing defense methods can resist some attacks, but suffer unbearable performance degradation under other unknown attacks. This is due to their reliance on either limited observed adversarial examples to optimize (adversarial training) or specific heuristics to alter graph or model structures (graph purification or robust aggregation). In this paper, we propose an Invariant causal DEfense method against adversarial Attacks (IDEA), providing a new perspective to address this issue. The method aims to learn causal features that possess strong predictability for labels and invariant predictability across attacks, to achieve graph adversarial robustness. Through modeling and analyzing the causal relationships in graph adversarial attacks, we design two invariance objectives to learn the causal features. Extensive experiments demonstrate that our IDEA significantly outperforms all the baselines under both poisoning and evasion attacks on five benchmark datasets, highlighting the strong and invariant predictability of IDEA. The implementation of IDEA is available at https://anonymous.4open. science/r/IDEA_repo-666B.
Online learning of long-range dependencies

Online learning holds the promise of enabling efficient long-term credit assignment in recurrent neural networks. However, current algorithms fall short of offline backpropagation by either not being scalable or failing to learn long-range dependencies. Here we present a high-performance online learning algorithm that merely doubles the memory and computational requirements of a single inference pass. We achieve this by leveraging independent recurrent modules in multi-layer networks, an architectural motif that has recently been shown to be particularly powerful. Experiments on synthetic memory problems and on the challenging long-range arena benchmark suite reveal that our algorithm performs competitively, establishing a new standard for what can be achieved through online learning. This ability to learn long-range dependencies offers a new perspective on learning in the brain and opens a promising avenue in neuromorphic computing.
Collective Knowledge Graph Completion with Mutual Knowledge Distillation

Knowledge graph completion (KGC), the task of predicting missing information based on the existing relational data inside a knowledge graph (KG), has drawn significant attention in recent years. However, the predictive power of KGC methods is often limited by the completeness of the existing knowledge graphs from different sources and languages. In monolingual and multilingual settings, KGs are potentially complementary to each other. In this paper, we study the problem of multi-KG completion, where we focus on maximizing the collective knowledge from different KGs to alleviate the incompleteness of individual KGs. Specifically, we propose a novel method called CKGC-CKD that uses relation-aware graph convolutional network encoder models on both individual KGs and a large fused KG in which seed alignments between KGs are regarded as edges for message propagation. An additional mutual knowledge distillation mechanism is also employed to maximize the knowledge transfer between the models of"global"fused KG and the"local"individual KGs. Experimental results on multilingual datasets have shown that our method outperforms all state-of-the-art models in the KGC task.
How to Turn Your Knowledge Graph Embeddings into Generative Models via Probabilistic Circuits

Some of the most successful knowledge graph embedding (KGE) models for link prediction -- CP, RESCAL, TuckER, ComplEx -- can be interpreted as energy-based models. Under this perspective they are not amenable for exact maximum-likelihood estimation (MLE), sampling and struggle to integrate logical constraints. This work re-interprets the score functions of these KGEs as circuits -- constrained computational graphs allowing efficient marginalisation. Then, we design two recipes to obtain efficient generative circuit models by either restricting their activations to be non-negative or squaring their outputs. Our interpretation comes with little or no loss of performance for link prediction, while the circuits framework unlocks exact learning by MLE, efficient sampling of new triples, and guarantee that logical constraints are satisfied by design. Furthermore, our models scale more gracefully than the original KGEs on graphs with millions of entities.
GFairHint: Improving Individual Fairness for Graph Neural Networks via Fairness Hint

Given the growing concerns about fairness in machine learning and the impressive performance of Graph Neural Networks (GNNs) on graph data learning, algorithmic fairness in GNNs has attracted significant attention. While many existing studies improve fairness at the group level, only a few works promote individual fairness, which renders similar outcomes for similar individuals. A desirable framework that promotes individual fairness should (1) balance between fairness and performance, (2) accommodate two commonly-used individual similarity measures (externally annotated and computed from input features), (3) generalize across various GNN models, and (4) be computationally efficient. Unfortunately, none of the prior work achieves all the desirables. In this work, we propose a novel method, GFairHint, which promotes individual fairness in GNNs and achieves all aforementioned desirables. GFairHint learns fairness representations through an auxiliary link prediction task, and then concatenates the representations with the learned node embeddings in original GNNs as a"fairness hint". Through extensive experimental investigations on five real-world graph datasets under three prevalent GNN models covering both individual similarity measures above, GFairHint achieves the best fairness results in almost all combinations of datasets with various backbone models, while generating comparable utility results, with much less computational cost compared to the previous state-of-the-art (SoTA) method.
IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languages

India has a rich linguistic landscape with languages from 4 major language families spoken by over a billion people. 22 of these languages are listed in the Constitution of India (referred to as scheduled languages) are the focus of this work. Given the linguistic diversity, high-quality and accessible Machine Translation (MT) systems are essential in a country like India. Prior to this work, there was (i) no parallel training data spanning all 22 languages, (ii) no robust benchmarks covering all these languages and containing content relevant to India, and (iii) no existing translation models which support all the 22 scheduled languages of India. In this work, we aim to address this gap by focusing on the missing pieces required for enabling wide, easy, and open access to good machine translation systems for all 22 scheduled Indian languages. We identify four key areas of improvement: curating and creating larger training datasets, creating diverse and high-quality benchmarks, training multilingual models, and releasing models with open access. Our first contribution is the release of the Bharat Parallel Corpus Collection (BPCC), the largest publicly available parallel corpora for Indic languages. BPCC contains a total of 230M bitext pairs, of which a total of 126M were newly added, including 644K manually translated sentence pairs created as part of this work. Our second contribution is the release of the first n-way parallel benchmark covering all 22 Indian languages, featuring diverse domains, Indian-origin content, and source-original test sets. Next, we present IndicTrans2, the first model to support all 22 languages, surpassing existing models on multiple existing and new benchmarks created as a part of this work. Lastly, to promote accessibility and collaboration, we release our models and associated data with permissive licenses at https://github.com/AI4Bharat/IndicTrans2.
Effective flipping, skewering and rank rigidity for cubulated groups with factor systems

Relying on work of Caprace and Sageev \cite{capracesageev:rank}, we provide an effective form of rank rigidity in the context of groups virtually acting freely cocompactly on a CAT(0) cube complex with a factor system. We accomplish this by exhibiting a special pair of hyperplanes that can be skewered uniformly quickly. Furthermore, for virtually special compact groups, we prove an effective omnibus theorem and provide a trichotomy implying a strong form of an effective Tits alternative. More generally, we provide a recipe for producing short Morse elements generating free stable subgroups in any virtually torsion-free hierarchically hyperbolic group (HHG) which recovers Mangahas' work \cite{MangahasRecipie} and provides an effective rank-rigidity dichotomy in the context of HHGs via Durham-Hagen-Sisto \cite{Durham2017-ce}. Part of our analysis involves showing that Caprace-Sageev's cubical tools of flipping and skewering can be applied to any HHG using the notion of a \emph{curtain} recently introduced by Petyt, Spriano and the author in \cite{PSZCAT}, \cite{Zalloum23_injectivity}. Indeed, our route to producing a short Morse element proceeds by exhibiting a special pair of curtains in the underlying HHG that can be skewered uniformly quickly.
Inhomogeneity-induced time-reversal symmetry breaking in cuprate twist junctions

The lowest order Josephson coupling, $J_1(\theta)\cos(\phi)$, between two d-wave superconductors with phase-difference $\phi$ across the junction vanishes when their relative orientation is rotated by $\theta=\pi/4$. However, in the presence of inhomogeneity, $J_{1}(\mathbf{r})$ is non-zero locally, with a sign that fluctuates in space. We show that such a random $J_1$ generates a global second-harmonic Josephson coupling, $J_2\cos(2\phi)$, with a sign that favors $\phi = \pm \pi/2$, i.e., spontaneous breaking of time reversal symmetry. The magnitude of $J_2$ is substantially enhanced if the spatial correlations of $J_1(\mathbf{r})$ extend over large distances, such as would be expected in the presence of large amplitude twist-angle angle disorder or significant local electronic nematicity. We argue that this effect likely accounts for the recent observations in twisted Josephson junctions between high temperature superconductors.
ASR and Emotional Speech: A Word-Level Investigation of the Mutual Impact of Speech and Emotion Recognition

In Speech Emotion Recognition (SER), textual data is often used alongside audio signals to address their inherent variability. However, the reliance on human annotated text in most research hinders the development of practical SER systems. To overcome this challenge, we investigate how Automatic Speech Recognition (ASR) performs on emotional speech by analyzing the ASR performance on emotion corpora and examining the distribution of word errors and confidence scores in ASR transcripts to gain insight into how emotion affects ASR. We utilize four ASR systems, namely Kaldi ASR, wav2vec2, Conformer, and Whisper, and three corpora: IEMOCAP, MOSI, and MELD to ensure generalizability. Additionally, we conduct text-based SER on ASR transcripts with increasing word error rates to investigate how ASR affects SER. The objective of this study is to uncover the relationship and mutual impact of ASR and SER, in order to facilitate ASR adaptation to emotional speech and the use of SER in real world.
Dynamic Context Pruning for Efficient and Interpretable Autoregressive Transformers

Autoregressive Transformers adopted in Large Language Models (LLMs) are hard to scale to long sequences. Despite several works trying to reduce their computational cost, most of LLMs still adopt attention layers between all pairs of tokens in the sequence, thus incurring a quadratic cost. In this study, we present a novel approach that dynamically prunes contextual information while preserving the model's expressiveness, resulting in reduced memory and computational requirements during inference. Our method employs a learnable mechanism that determines which uninformative tokens can be dropped from the context at any point across the generation process. By doing so, our approach not only addresses performance concerns but also enhances interpretability, providing valuable insight into the model's decision-making process. Our technique can be applied to existing pre-trained models through a straightforward fine-tuning process, and the pruning strength can be specified by a sparsity parameter. Notably, our empirical findings demonstrate that we can effectively prune up to 80\% of the context without significant performance degradation on downstream tasks, offering a valuable tool for mitigating inference costs. Our reference implementation achieves up to $2\times$ increase in inference throughput and even greater memory savings.
All Points Matter: Entropy-Regularized Distribution Alignment for Weakly-supervised 3D Segmentation

Pseudo-labels are widely employed in weakly supervised 3D segmentation tasks where only sparse ground-truth labels are available for learning. Existing methods often rely on empirical label selection strategies, such as confidence thresholding, to generate beneficial pseudo-labels for model training. This approach may, however, hinder the comprehensive exploitation of unlabeled data points. We hypothesize that this selective usage arises from the noise in pseudo-labels generated on unlabeled data. The noise in pseudo-labels may result in significant discrepancies between pseudo-labels and model predictions, thus confusing and affecting the model training greatly. To address this issue, we propose a novel learning strategy to regularize the generated pseudo-labels and effectively narrow the gaps between pseudo-labels and model predictions. More specifically, our method introduces an Entropy Regularization loss and a Distribution Alignment loss for weakly supervised learning in 3D segmentation tasks, resulting in an ERDA learning strategy. Interestingly, by using KL distance to formulate the distribution alignment loss, it reduces to a deceptively simple cross-entropy-based loss which optimizes both the pseudo-label generation network and the 3D segmentation network simultaneously. Despite the simplicity, our method promisingly improves the performance. We validate the effectiveness through extensive experiments on various baselines and large-scale datasets. Results show that ERDA effectively enables the effective usage of all unlabeled data points for learning and achieves state-of-the-art performance under different settings. Remarkably, our method can outperform fully-supervised baselines using only 1% of true annotations. Code and model will be made publicly available at https://github.com/LiyaoTang/ERDA.
Finite‐time output tracking for a class of switched nonlinear systems

This paper investigates the finite‐time output tracking for a class of switched nonlinear systems in p‐normal form. Compared with the existing results, the restrictions on power orders of the system are relaxed. Using the convex combination method and the adding a power integrator technique, a state‐dependent switching and law, and state feedback controllers of individual subsystems are constructed. It is shown that all states of the closed‐loop system are bounded, and the tracking error can converge to a small neighborhood of zero in finite time. An example is provided to show the effectiveness of the proposed method. Copyright © 2016 John Wiley & Sons, Ltd.
A Note on Efficiency Gains from Auxiliary Samples

We study the use of auxiliary samples for efficiency gains in the estimation of a finite-dimensional parameter value defined by a set of moment restrictions on a target population. The target population may be an artificial construct defined in terms of the observability of variables forming the moment vector. Multiple variables forming the moment vector may be missing jointly or individually from all but one of the primary and the auxiliary samples; hence there exist the so-called complete data cases. Efficiency gains are discussed in terms of the semiparametric efficiency bounds under the use of different auxiliary samples along with the primary sample as the observed data. The underlying population of the observed data may or may not contain the target population, and identification is obtained by a convenient version of the missing at random assumption.
Homogenization of ultrathin metallo-dielectric structures leading to transmission conditions at an equivalent interface

We present a method of homogenization of thin metallo-dielectric structures as used in the design of artificial surfaces, or metasurfaces. The approach is based on a so-called matched asymptotic expansion technique, leading to parameters being characteristic of an equivalent interface associated to jump conditions. It is applied to an array of metal strips on top of a metal-backed dielectric slab, with the strips having a small but possibly finite thickness. Solving the equivalent problem provides explicit expressions (1) of the reflection coefficient for a wave at oblique incidence and (2) of the dispersion relation of the surface waves. The results are shown to be in agreement with results coming from the transmission line theory in the limit of vanishing thickness of the metallization and for normal incidence of the wave. The influence of the finite thickness of the metallization is exemplified and validated by comparison with full wave simulations.
SIMULATIONS OF A DYNAMIC SOLAR CYCLE AND ITS EFFECTS ON THE INTERSTELLAR BOUNDARY EXPLORER RIBBON AND GLOBALLY DISTRIBUTED ENERGETIC NEUTRAL ATOM FLUX

Since 2009, observations by the Interstellar Boundary Explorer (IBEX) have vastly improved our understanding of the interaction between the solar wind (SW) and local interstellar medium through direct measurements of energetic neutral atoms (ENAs), which inform us about the heliospheric conditions that produced them. An enhanced feature of flux in the sky, the so-called IBEX ribbon, was not predicted by any global models before the first IBEX observations. A dominating theory of the origin of the ribbon, although still under debate, is a secondary charge-exchange process involving secondary ENAs originating from outside the heliopause. According to this mechanism, the evolution of the solar cycle should be visible in the ribbon flux. Therefore, in this paper we simulate a fully time-dependent ribbon flux, as well as globally distributed flux from the inner heliosheath (IHS), using time-dependent SW parameters from Sokół et al. as boundary conditions for our time-dependent heliosphere simulation. After post-processing the results to compute H ENA fluxes, our results show that the secondary ENA ribbon indeed should be time dependent, evolving with a period of approximately 11 yr, with differences depending on the energy and direction. Our results for the IHS flux show little periodic change with the 11 yr solar cycle, but rather with short-term fluctuations in the background plasma. While the secondary ENA mechanism appears to emulate several key characteristics of the observed IBEX ribbon, it appears that our simulation does not yet include all of the relevant physics that produces the observed ribbon.
Evidence for a Quasi-Biennial Variation in Eccentricity of the North Polar Vortex

Abstract The centroid of the north polar vortex has been determined from 300 mb mean-monthly maps for the 10-year period 1963–72. There is a significant correlation of 0.79 between the 12-month running average distance of the centroid from the North Pole and the 50 mb zonal wind at Balboa, Canal Zone; i.e., the eccentricity appears to be greater at the time of quasi-biennial cast wind minimum
Thickness-dependent carrier transport of PdSe2 films grown by plasma-assisted metal selenization

Atomically thin narrow-bandgap layered PdSe2 has attracted much attention due to its rich and unique electrical properties. For silicon-compatible device integration, direct wafer-scale preparation of high-quality PdSe2 thin film on a silicon substrate is highly desired. Here, we present the low-temperature synthesis of large-area polycrystalline PdSe2 films grown on SiO2/Si substrates by plasma-assisted metal selenization and investigate their charge carrier transport behaviors. Raman analysis, depth-dependent x-ray photoelectron spectroscopy, and cross-sectional transmission electron microscopy were used to reveal the selenization process. The results indicate a structural evolution from initial Pd to intermediate PdSe2–x phase and eventually to PdSe2. The field-effect transistors fabricated from these ultrathin PdSe2 films exhibit strong thickness-dependent transport behaviors. For thinner films (4.5 nm), a record high on/off ratio of 104 was obtained. While for thick ones (11 nm), the maximum hole mobility is about 0.93 cm2 V−1 S−1, which is the record high value ever reported for polycrystalline films. These findings suggest that our low-temperature-metal-selenized PdSe2 films have high quality and show great potential for applications in electrical devices.
The influence of acute dietary nitrate supplementation on skeletal muscle fatigue and recovery in older women

Older individuals fatigue more rapidly during, and recover more slowly from, dynamic exercise. Women are particularly vulnerable to these deleterious effects of aging, which increases their risk of falling. We have shown that dietary nitrate (NO3−), a source of nitric oxide (NO) via the NO3− → nitrite (NO2−) → NO pathway, enhances muscle speed and power in older individuals in the non‐fatigued state; however, it is unclear if it reduces fatigability and/or improves recoverability in this population. Using a double‐blind, placebo‐controlled, crossover design, we studied 18 older (age 70 ± 4 years) women who were administered an acute dose of beetroot juice (BRJ) containing either 15.6 ± 3.6 or <0.05 mmol of NO3−. Blood samples were drawn throughout each ~3 h visit for plasma NO3− and NO2− analysis. Peak torque was measured during, and periodically for 10 min after, 50 maximal knee extensions performed at 3.14 rad/s on an isokinetic dynamometer. Ingestion of NO3−‐containing BRJ increased plasma NO3− and NO2− concentrations by 21 ± 8 and 4 ± 4 fold, respectively. However, there were no differences in muscle fatigue or recovery. Dietary NO3− increases plasma NO3− and NO2− concentrations but does not reduce fatigability during or enhance recoverability after high intensity exercise in older women.