Miracle-dz/UniFAR-FTUs huggingface dataset:
A New cross-domain strategy based XAI models for fake news detection

In this study, we presented a four-level cross-domain strategy for fake news detection on pre-trained models. Cross-domain text classification is a task of a model adopting a target domain by using the knowledge of the source domain. Explainability is crucial in understanding the behaviour of these complex models. A fine-tune BERT model is used to. perform cross-domain classification with several experiments using datasets from different domains. Explanatory models like Anchor, ELI5, LIME and SHAP are used to design a novel explainable approach to cross-domain levels. The experimental analysis has given an ideal pair of XAI models on different levels of cross-domain.
Origin of magnetically dead layers in spinel ferrites $M\text{Fe}_2\text{O}_4$ grown on $\text{Al}_2\text{O}_3$: Effects of post-deposition annealing studied by XMCD

We study the electronic and magnetic states of as-grown and annealed $M\text{Fe}_2\text{O}_4$(111)/$\text{Al}_2\text{O}_3$(111) ($M=\text{Co, Ni}$) thin films with various thicknesses grown on Si(111) substrates with the $\gamma$-$\text{Al}_2\text{O}_3$(111) buffer layers by using x-ray absorption spectroscopy (XAS) and x-ray magnetic circular dichroism (XMCD), to investigate magnetically dead layers in these films. Although the magnetically dead layers in the as-grown samples are formed near the interface with the $\text{Al}_2\text{O}_3$ buffer layer, we reveal that ferrimagnetic order is partially recovered by post-deposition annealing at 973 K for 48 hours in air. By analyzing the line shapes of the XAS and XMCD spectra, we conclude that, in the dead layers, there are a significant number of vacancies at the $T_d$ sites of the spinel structure, which may be the microscopic origin of the degraded ferrimagnetic order in the $M\text{Fe}_2\text{O}_4$(111) thin films.
Contrastive Collaborative Filtering for Cold-Start Item Recommendation

The cold-start problem is a long-standing challenge in recommender systems. As a promising solution, content-based generative models usually project a cold-start item’s content onto a warm-start item embedding to capture collaborative signals from item content so that collaborative filtering can be applied. However, since the training of the cold-start recommendation models is conducted on warm datasets, the existent methods face the issue that the collaborative embeddings of items will be blurred, which significantly degenerates the performance of cold-start item recommendation. To address this issue, we propose a novel model called Contrastive Collaborative Filtering for Cold-start item Recommendation (CCFCRec), which capitalizes on the co-occurrence collaborative signals in warm training data to alleviate the issue of blurry collaborative embeddings for cold-start item recommendation. In particular, we devise a contrastive collaborative filtering (CF) framework, consisting of a content CF module and a co-occurrence CF module to generate the content-based collaborative embedding and the co-occurrence collaborative embedding for a training item, respectively. During the joint training of the two CF modules, we apply a contrastive learning between the two collaborative embeddings, by which the knowledge about the co-occurrence signals can be indirectly transferred to the content CF module, so that the blurry collaborative embeddings can be rectified implicitly by the memorized co-occurrence collaborative signals during the applying phase. Together with the sound theoretical analysis, the extensive experiments conducted on real datasets demonstrate the superiority of the proposed model. The codes and datasets are available on https://github.com/zzhin/CCFCRec.
Online Nonstochastic Control with Adversarial and Static Constraints

This paper studies online nonstochastic control problems with adversarial and static constraints. We propose online nonstochastic control algorithms that achieve both sublinear regret and sublinear adversarial constraint violation while keeping static constraint violation minimal against the optimal constrained linear control policy in hindsight. To establish the results, we introduce an online convex optimization with memory framework under adversarial and static constraints, which serves as a subroutine for the constrained online nonstochastic control algorithms. This subroutine also achieves the state-of-the-art regret and constraint violation bounds for constrained online convex optimization problems, which is of independent interest. Our experiments demonstrate the proposed control algorithms are adaptive to adversarial constraints and achieve smaller cumulative costs and violations. Moreover, our algorithms are less conservative and achieve significantly smaller cumulative costs than the state-of-the-art algorithm.
Computation vs. Communication Scaling for Future Transformers on Future Hardware

Scaling neural network models has delivered dramatic quality gains across ML problems. However, this scaling has increased the reliance on efficient distributed training techniques. Accordingly, as with other distributed computing scenarios, it is important to understand how will compute and communication scale relative to one another as models scale and hardware evolves? A careful study which answers this question can better guide the design of future systems which can efficiently train future large models. Accordingly, this work provides a comprehensive multi-axial (algorithmic, empirical, hardware evolution) analysis of compute vs. communication (Comp-vs.-Comm) scaling for future Transformer models on future hardware. First, our algorithmic analysis shows that compute generally enjoys an edge over communication as models scale. However, since memory capacity scales slower than compute, these trends are being stressed. Next, we quantify this edge by empirically studying how Comp-vs.-Comm scales for future models on future hardware. To avoid profiling numerous Transformer models across many setups, we extract execution regions and project costs using operator models. This allows a spectrum (hundreds) of future model/hardware scenarios to be accurately studied ($<$15% error), and reduces profiling costs by 2100$\times$. Our experiments show that communication will be a significant portion (40-75%) of runtime as models and hardware evolve. Moreover, communication which is hidden by overlapped computation in today's models often cannot be hidden in future, larger models. Overall, this work highlights the increasingly large role communication will play as models scale and discusses techniques and upcoming technologies that can help address it.
PubGraph: A Large Scale Scientific Temporal Knowledge Graph

Research publications are the primary vehicle for sharing sci-entiﬁc progress in the form of new discoveries, methods, techniques, and insights. Publications have been studied from the perspectives of both content analysis and bibliometric structure, but a barrier to more comprehensive studies of scien-tiﬁc research is a lack of publicly accessible large-scale data and resources. In this paper, we present PubGraph , a new resource for studying scientiﬁc progress that takes the form of a large-scale temporal knowledge graph (KG). It contains more than 432M nodes and 15.49B edges mapped to the popular Wikidata ontology. We extract three KGs with varying sizes from PubGraph to allow experimentation at different scales. Using these KGs, we introduce a new link prediction benchmark for transductive and inductive settings with temporally-aligned training, validation, and testing partitions. Moreover, we develop two new inductive learning methods better suited to PubGraph, operating on unseen nodes without explicit features, scaling to large KGs, and outperforming existing models. Our results demonstrate that structural features of past citations are sufﬁcient to produce high-quality predictions about new publications. We also identify new challenges for KG models, including an adversarial community-based link prediction setting, zero-shot inductive learning, and large-scale learning.
Dealing With Non-stationarity in Decentralized Cooperative Multi-Agent Deep Reinforcement Learning via Multi-Timescale Learning

Decentralized cooperative multi-agent deep reinforcement learning (MARL) can be a versatile learning framework, particularly in scenarios where centralized training is either not possible or not practical. One of the critical challenges in decentralized deep MARL is the non-stationarity of the learning environment when multiple agents are learning concurrently. A commonly used and efficient scheme for decentralized MARL is independent learning in which agents concurrently update their policies independently of each other. We first show that independent learning does not always converge, while sequential learning where agents update their policies one after another in a sequence is guaranteed to converge to an agent-by-agent optimal solution. In sequential learning, when one agent updates its policy, all other agent's policies are kept fixed, alleviating the challenge of non-stationarity due to simultaneous updates in other agents' policies. However, it can be slow because only one agent is learning at any time. Therefore it might also not always be practical. In this work, we propose a decentralized cooperative MARL algorithm based on multi-timescale learning. In multi-timescale learning, all agents learn simultaneously, but at different learning rates. In our proposed method, when one agent updates its policy, other agents are allowed to update their policies as well, but at a slower rate. This speeds up sequential learning, while also minimizing non-stationarity caused by other agents updating concurrently. Multi-timescale learning outperforms state-of-the-art decentralized learning methods on a set of challenging multi-agent cooperative tasks in the epymarl(Papoudakis et al., 2020) benchmark. This can be seen as a first step towards more general decentralized cooperative deep MARL methods based on multi-timescale learning.
The First Passage Time of a Level and the Behavior at Infinity for a Class of Processes with Independent Increments

Let $\mathfrak{S}$ be the class of homogeneous processes $\xi (t)$ with independent increments and without positive jumps, and let $\mathfrak{S}^ + $, $\mathfrak{S}^ - $ be its subclasses such that \[ \begin{gathered} {\bf P}\left\{ {\mathop {\sup }\limits_t \xi (t) = \infty } \right\} = 1,\qquad \xi (t) \in \mathfrak{S}^ + , \hfill \\ {\bf P}\left\{ {\mathop {\inf }\limits_t \xi (t) = - \infty } \right\} = 1,\qquad \xi (t) \in \mathfrak{S}^ - . \hfill \\ \end{gathered} \]It is proved that $\xi \in \mathfrak{S}^ + $ (resp. $\xi \in \mathfrak{S}^ - $) if and only if $\gamma = {\bf E}\xi (1) \geqq 0$ (resp. $\gamma \leqq 0$). The exact expression for the distribution of the first intersection moment $\tau _x $ of a level $x > 0$ by a process $\xi \in \mathfrak{S}^ + $ is found. We have also found the exact expression for the probability $P(x) = {\bf P}\{ {\sup _t \xi (t) \geqq x} \}$ as well as the implicit dependence of $Q(x) = {\bf P}\{ {\inf _t \xi (t) \leqq - x} \}$ on the distribution of $\xi (t)$. Asy.
Introspective Experience Replay: Look Back When Surprised

In reinforcement learning (RL), experience replay-based sampling techniques play a crucial role in promoting convergence by eliminating spurious correlations. However, widely used methods such as uniform experience replay (UER) and prioritized experience replay (PER) have been shown to have sub-optimal convergence and high seed sensitivity respectively. To address these issues, we propose a novel approach called IntrospectiveExperience Replay (IER) that selectively samples batches of data points prior to surprising events. Our method builds upon the theoretically sound reverse experience replay (RER) technique, which has been shown to reduce bias in the output of Q-learning-type algorithms with linear function approximation. However, this approach is not always practical or reliable when using neural function approximation. Through empirical evaluations, we demonstrate that IER with neural function approximation yields reliable and superior performance compared toUER, PER, and hindsight experience replay (HER) across most tasks.
Immersive Virtual Colonoscopy Viewer for Colorectal Diagnosis

Desktop-based virtual colonoscopy has been proven to be an asset in the identification of colon anomalies. The process is accurate, although time-consuming. The use of immersive interfaces for virtual colonoscopy is incipient and not yet understood. In this work, we present a new design exploring elements of the VR paradigm to make the immersive analysis more efficient while still effective. We also plan the conduction of experiments with experts to assess the multi-factor influences of coverage, duration, and diagnostic accuracy.
Sentiment Analysis on Movie Reviews Using Ensemble Features and Pearson Correlation Based Feature Selection

Microblogging has become the media information that is very popular among internet users. Therefore, the microblogging became a source of rich data for opinions and reviews especially on movie reviews. We proposed, sentiment analysis on movie review using ensemble features and Bag of Words and selection Features Pearson's Correlation to reduce the dimension of the feature and get the optimal feature combinations. Use the feature selection is done to improve the performance of the classification, reducing the dimension of the feature and get the optimal feature combinations. The process of classification using several models of Naïve Bayes i.e. Bernoulli Naïve Bayes for binary data, Gaussian Naïve Bayes for continuous data and Multinomial Naïve Bayes for numeric data. The results of this study indicate that by using the non-standard word on tweet evaluation results obtained accuracy 82%, precision 86%, recall 79.62% and f-measure 82.69% using Feature Selection 20%. Then after using manual standardization of word the evaluation results on the accuracy increased by 8% and then the accuracy becomes 90%, precision 92%, recall 88.46% and f-measure 90.19% using 85% feature selection. Based on these results it can be concluded that by using the standardization of word can improve the performance of classification and feature selection Pearson's provide optimal feature combinations and reducing the total number of dimensions’ feature.
Invariants for neural automata

Computational modeling of neurodynamical systems often deploys neural networks and symbolic dynamics. One particular way for combining these approaches within a framework called vector symbolic architectures leads to neural automata. Specifically, neural automata result from the assignment of symbols and symbol strings to numbers, known as Gödel encoding. Under this assignment, symbolic computation becomes represented by trajectories of state vectors in a real phase space, that allows for statistical correlation analyses with real-world measurements and experimental data. However, these assignments are usually completely arbitrary. Hence, it makes sense to address the problem which aspects of the dynamics observed under a Gödel representation is intrinsic to the dynamics and which are not. In this study, we develop a formally rigorous mathematical framework for the investigation of symmetries and invariants of neural automata under different encodings. As a central concept we define patterns of equality for such systems. We consider different macroscopic observables, such as the mean activation level of the neural network, and ask for their invariance properties. Our main result shows that only step functions that are defined over those patterns of equality are invariant under symbolic recodings, while the mean activation, e.g., is not. Our work could be of substantial importance for related regression studies of real-world measurements with neurosymbolic processors for avoiding confounding results that are dependant on a particular encoding and not intrinsic to the dynamics.
A Pre-training Framework for Knowledge Graph Completion

Knowledge graph completion (KGC) is one of the effective methods to identify new facts in knowledge graph. Except for a few methods based on graph network, most of KGC methods trend to be trained based on independent triples, while are difficult to take a full account of the information of global network connection contained in knowledge network. To address these issues, in this study, we propose a simple and effective Network-based Pre-training framework for knowledge graph completion (termed NetPeace), which takes into account the information of global network connection and local triple relationships in knowledge graph. Experiments show that in NetPeace framework, multiple KGC models yields consistent and significant improvements on benchmarks (e.g., 36.45% Hits@1 and 27.40% MRR improvements for TuckER on FB15k-237), especially dense knowledge graph. On the challenging low-resource task, NetPeace that benefits from the global features of KG achieves higher performance (104.03% MRR and 143.89% Hit@1 improvements at most) than original models.
Nonlinear elasticity complex and a finite element diagram chase

In this paper, we present a nonlinear version of the linear elasticity (Calabi, Kr\"oner, Riemannian deformation) complex which encodes isometric embedding, metric, curvature and the Bianchi identity. We reformulate the rigidity theorem and a fundamental theorem of Riemannian geometry as the exactness of this complex. Then we generalize an algebraic approach for constructing finite elements for the Bernstein-Gelfand-Gelfand (BGG) complexes. In particular, we discuss the reduction of degrees of freedom with injective connecting maps in the BGG diagrams. We derive a strain complex in two space dimensions with a diagram chase.
HAFFET: Hybrid Analytic Flux FittEr for Transients

The progenitors for many types of supernovae (SNe) are still unknown, and an approach to diagnose their physical origins is to investigate the light-curve brightness and shape of a large set of SNe. However, it is often difficult to compare and contrast the existing sample studies due to differences in their approaches and assumptions, for example, in how to eliminate host galaxy extinction, and this might lead to systematic errors when comparing the results. We therefore introduce the Hybrid Analytic Flux FittEr for Transients (HAFFET), a Python-based software package that can be applied to download photometric and spectroscopic data for transients from open online sources, derive bolometric light curves, and fit them to semianalytical models for estimation of their physical parameters. In a companion study, we have investigated a large collection of SNe Ib and Ic observed with the Zwicky Transient Facility (ZTF) with HAFFET, and here we detail the methodology and the software package to encourage more users. As large-scale surveys such as ZTF and LSST continue to discover increasing numbers of transients, tools such as HAFFET will be critical for enabling rapid comparison of models against data in statistically consistent, comparable, and reproducible ways. Additionally, HAFFET is created with a graphical user interface mode, which we hope will boost the efficiency and make the usage much easier (https://github.com/saberyoung/HAFFET).
The divergence theorem and nonlocal counterparts

We present a new proof of the classical divergence theorem in bounded domains. Our proof is based on a nonlocal analog of the divergence theorem and a rescaling argument. Main ingredients in the proof are nonlocal versions of the divergence and the normal derivative. We employ these to provide definitions of well‐known nonlocal concepts such as the fractional perimeter.
In Search of Insights, Not Magic Bullets: Towards Demystification of the Model Selection Dilemma in Heterogeneous Treatment Effect Estimation

Personalized treatment effect estimates are often of interest in high-stakes applications -- thus, before deploying a model estimating such effects in practice, one needs to be sure that the best candidate from the ever-growing machine learning toolbox for this task was chosen. Unfortunately, due to the absence of counterfactual information in practice, it is usually not possible to rely on standard validation metrics for doing so, leading to a well-known model selection dilemma in the treatment effect estimation literature. While some solutions have recently been investigated, systematic understanding of the strengths and weaknesses of different model selection criteria is still lacking. In this paper, instead of attempting to declare a global `winner', we therefore empirically investigate success- and failure modes of different selection criteria. We highlight that there is a complex interplay between selection strategies, candidate estimators and the data used for comparing them, and provide interesting insights into the relative (dis)advantages of different criteria alongside desiderata for the design of further illuminating empirical studies in this context.
Unsupervised Learning for Pilot-free Transmission in 3GPP MIMO Systems

Reference signals overhead reduction has recently evolved as an effective solution for improving the system spectral efficiency. This paper introduces a new downlink data structure that is free from demodulation reference signals (DM-RS), and hence does not require any channel estimation at the receiver. The new proposed data transmission structure involves a simple repetition step of part of the user data across the different sub-bands. Exploiting the repetition structure at the user side, it is shown that reliable recovery is possible via canonical correlation analysis. This paper also proposes two effective mechanisms for boosting the CCA performance in OFDM systems; one for repetition pattern selection and another to deal with the severe frequency selectivity issues. The proposed approach exhibits favorable complexity-performance tradeoff, rendering it appealing for practical implementation. Numerical results, using a 3GPP link-level testbench, demonstrate the superiority of the proposed approach relative to the state-of-the-art methods.
SPARLING: Learning Latent Representations with Extremely Sparse Activations

Real-world processes often contain intermediate state that can be modeled as an extremely sparse tensor. We introduce Sparling, a technique that allows you to learn models with intermediate layers that match this state from only end-to-end labeled examples (i.e., no supervision on the intermediate state). Sparling uses a new kind of informational bottleneck that enforces levels of activation sparsity unachievable using other techniques. We find that extreme sparsity is necessary to achieve good intermediate state modeling. On our synthetic DigitCircle domain as well as the LaTeX-OCR and Audio-MNIST-Sequence domains, we are able to precisely localize the intermediate states up to feature permutation with>90% accuracy, even though we only train end-to-end.