Miracle-dz/UniFAR-FTUs huggingface dataset:
XAI Establishes a Common Ground Between Machine Learning and Causality

A handful of recent works have argued on the connection between machine learning and causality. In a reverse thought process, starting from the grounding of mental models in causal models, we strengthen these initial works with results that suggest XAI essentially requiring machine learning to learn models that are causally consistent with the task at hand. By recognizing how human mental models (HMM) are naturally represented by the Pearlian Structural Causal Model (SCM), we make two key observations through the construction of an example metric space for linear SCM: ﬁrst, that the notion of a “true” data-underlying SCM is justiﬁed, and second, that an aggregation of human-derived SCM might point to said “true” SCM. Motivated by the implications of these insights, we conclude with a third observation which argues that interpretations derived from HMM must imply interpretability in the SCM framework. Following this intuition, we present an original derivation using these priorly established ﬁrst principles to reveal a human-readable interpretation scheme consistent with the given SCM, justifying the naming Structural Causal Interpretations (SCI). Going further, we analyze these SCIs and their mathematical properties theoretically and empirically. We prove that any existing graph induction method (GIM) is in fact interpretable in the SCI-sense. Our ﬁrst experiment (E1) assesses the quality of such GIM-based SCI. In (E2) we observe evidence for our conjecture on improved sample-efﬁciency for SCI-based learning. For (E3) we conduct a study ( N = 22 ) and observe superiority in human-based SCI over GIM ones, corroborating our initial hypothesis.
Residual Sparsity Connection Learning for Efficient Video Super-Resolution

Lighter and faster models are crucial for the deployment of video super-resolution (VSR) on resource-limited devices, e . g . , smartphones and wearable devices. In this paper, we develop Residual Sparsity Connection Learning (RSCL), a structured pruning scheme, to reduce the redundancy of convolution kernels and obtain a compact VSR network with a minor performance drop. However, residual blocks require the pruned ﬁlter indices of skip and residual connections to be the same, which is tricky for pruning. Thus, to mitigate the pruning restrictions of residual blocks, we design a Residual Sparsity Connection (RSC) scheme by preserving the feature channels and only operating on the important channels. Moreover, for the pixel-shufﬂe operation, we design a special pruning scheme by grouping several ﬁlters as pruning units to guarantee the accuracy of feature channel-space conversion after pruning. In addition, we introduce Temporal Finetuning (TF) to reduce the pruning error ampliﬁcation of hidden states with temporal propagation. Extensive experiments show that the proposed RSCL signiﬁcantly outperforms recent methods quantitatively and qualitatively. Codes and models will be released.
Convergence and Price of Anarchy Guarantees of the Softmax Policy Gradient in Markov Potential Games

We study the performance of policy gradient methods for the subclass of Markov games known as Markov potential games (MPGs), which extends the notion of normal-form potential games to the stateful setting and includes the important special case of the fully cooperative setting where the agents share an identical reward function. Our focus in this paper is to study the convergence of the policy gradient method for solving MPGs under softmax policy parameterization, both tabular and parameterized with general function approxima-tors such as neural networks. We first show the asymptotic convergence of this method to a Nash equilibrium of MPGs for tabular softmax policies. Second, we derive the finite-time performance of the policy gradient in two settings: 1) using the log-barrier regularization, and 2) using the natural policy gradient under the best-response dynamics (NPG-BR). Finally, extending the notion of price of anarchy (POA) and smoothness in normal-form games, we introduce the POA for MPGs and provide a POA bound for NPG-BR. To our knowledge, this is the first POA bound for solving MPGs. To support our theoretical results, we empirically compare the convergence rates and POA of exact and sample-based policy gradient variants for both tabular and neural softmax policies.
Structure and Power: an emerging landscape

In this paper, we give an overview of some recent work on applying tools from category theory in finite model theory, descriptive complexity, constraint satisfaction, and combinatorics. The motivations for this work come from Computer Science, but there may also be something of interest for model theorists and other logicians. The basic setting involves studying the category of relational structures via a resource-indexed family of adjunctions with some process category - which unfolds relational structures into tree-like forms, allowing natural resource parameters to be assigned to these unfoldings. One basic instance of this scheme allows us to recover, in a purely structural, syntax-free way: the Ehrenfeucht-Fraïssé game; the quantifier rank fragments of first-order logic; the equivalences on structures induced by (i) the quantifier rank fragments, (ii) the restriction of this fragment to the existential positive part, and (iii) the extension with counting quantifiers; and the combinatorial parameter of tree-depth (Nesetril and Ossona de Mendez). Another instance recovers the k-pebble game, the finite-variable fragments, the corresponding equivalences, and the combinatorial parameter of treewidth. Other instances cover modal, guarded and hybrid fragments, generalized quantifiers, and a wide range of combinatorial parameters. This whole scheme has been axiomatized in a very general setting, of arboreal categories and arboreal covers. Beyond this basic level, a landscape is beginning to emerge, in which structural features of the resource categories, adjunctions and comonads are reflected in degrees of logical and computational tractability of the corresponding languages. Examples include semantic characterisation and preservation theorems, and Lovász-type results on counting homomorphisms.
Safe motion planning for a mobile robot navigating in environments shared with humans

In this paper, a robot navigating an environment shared with humans is considered, and a cost function that can be exploited in RRTX, a randomized sampling-based replanning algorithm that guarantees asymptotic optimality, to allow for a safe motion is proposed. The cost function is a path length weighted by a danger index based on a prediction of human motion performed using either a linear stochastic model, assuming constant longitudinal velocity and varying lateral velocity, and a GMM/GMR-based model, computed on an experimental dataset of human trajectories. The proposed approach is validated using a dataset of human trajectories collected in a real world setting.
Epistemic Deep Learning

The belief function approach to uncertainty quantiﬁcation as proposed in the Demspter-Shafer theory of evidence is established upon the general mathematical models for set-valued observations, called random sets . Set-valued predictions are the most natural representations of uncertainty in machine learning. In this paper, we introduce a concept called epistemic deep learning based on the random-set interpretation of belief functions to model epistemic learning in deep neural networks. We propose a novel random-set convolutional neural network for classiﬁcation that produces scores for sets of classes by learning set-valued ground truth representations. We evaluate diﬀerent formulations of entropy and distance measures for belief functions as viable loss functions for these random-set networks. We also discuss methods for evaluating the quality of epistemic predictions and the performance of epistemic random-set neural networks. We demonstrate through experiments that the epistemic approach produces better performance results when compared to traditional approaches of estimating uncertainty.
A Comprehensive Survey on Deep Clustering: Taxonomy, Challenges, and Future Directions

Clustering is a fundamental machine learning task which has been widely studied in the literature. Classic clustering methods follow the assumption that data are represented as features in a vectorized form through various representation learning techniques. As the data become increasingly complicated and complex, the shallow (traditional) clustering methods can no longer handle the high-dimensional data type. With the huge success of deep learning, especially the deep unsupervised learning, many representation learning techniques with deep architectures have been proposed in the past decade. Recently, the concept of Deep Clustering, i.e., jointly optimizing the representation learning and clustering, has been proposed and hence attracted growing attention in the community. Motivated by the tremendous success of deep learning in clustering, one of the most fundamental machine learning tasks, and the large number of recent advances in this direction, in this paper we conduct a comprehensive survey on deep clustering by proposing a new taxonomy of different state-of-the-art approaches. We summarize the essential components of deep clustering and categorize existing methods by the ways they design interactions between deep representation learning and clustering. Moreover, this survey also provides the popular benchmark datasets, evaluation metrics and open-source implementations to clearly illustrate various experimental settings. Last but not least, we discuss the practical applications of deep clustering and suggest challenging topics deserving further investigations as future directions.
On the complexity of finding set repairs for data-graphs

In the deeply interconnected world we live in, pieces of information link domains all around us. As graph databases embrace effectively relationships among data and allow processing and querying these connections efficiently, they are rapidly becoming a popular platform for storage that supports a wide range of domains and applications. As in the relational case, it is expected that data preserves a set of integrity constraints that define the semantic structure of the world it represents. When a database does not satisfy its integrity constraints, a possible approach is to search for a ‘similar’ database that does satisfy the constraints, also known as a repair. In this work, we study the problem of computing subset and superset repairs for graph databases with data values using a notion of consistency based on having a set of Reg-GXPath expressions as integrity constraints. We show that for positive fragments of Reg-GXPath these problems admit a polynomial-time algorithm, while the full expressive power of the language renders them intractable.
Blind Estimation of a Doubly Selective OFDM Channel: A Deep Learning Algorithm and Theory

We provide a new generation solution to the fundamental old problem of a doubly selective fading channel estimation for orthogonal frequency division multiplexing (OFDM) systems. For systems based on OFDM, we propose a deep learning (DL)-based blind doubly selective channel estimator. This estimator does require no pilot symbols, unlike the corresponding state-of-the-art estimators, even during the estimation of a deep fading doubly selective channel. We also provide the first of its kind theory on the testing mean squared error (MSE) performance of our investigated blind OFDM channel estimator based on over-parameterized ReLU FNNs.
Beyond Words: An Experimental Study of Signaling in Crowdfunding

Increasingly, crowdfunding is transforming financing for many people worldwide. Yet we know relatively little about how, why, and when funding outcomes are impacted by signaling between funders. We conduct two studies of N=500 and N=750 participants involved in crowdfunding to investigate the effect of certain characteristics of ``crowd signals'' on the decision to fund. We find that, under a variety of conditions, contributions of heterogeneous amounts arriving at varying time intervals are significantly more likely to be selected than homogeneous contribution amounts and times. The impact of signaling is strongest among participants who are susceptible to social influence. The effect is remarkably general across different project types, fundraising goals, participant interest in the projects, and participants' altruistic attitudes. Critically, the role of crowd signals in decision-making is typically unrecognized by participants. Our results underscore the fundamental nature of social signaling in crowdfunding, informing strategies for platforms, funders, and project creators.
Local Identifiability of Deep ReLU Neural Networks: the Theory

Is a sample rich enough to determine, at least locally, the parameters of a neural network? To answer this question, we introduce a new local parameterization of a given deep ReLU neural network by fixing the values of some of its weights. This allows us to define local lifting operators whose inverses are charts of a smooth manifold of a high dimensional space. The function implemented by the deep ReLU neural network composes the local lifting with a linear operator which depends on the sample. We derive from this convenient representation a geometrical necessary and sufficient condition of local identifiability. Looking at tangent spaces, the geometrical condition provides: 1/ a sharp and testable necessary condition of identifiability and 2/ a sharp and testable sufficient condition of local identifiability. The validity of the conditions can be tested numerically using backpropagation and matrix rank computations.
Contrastive Learning as Goal-Conditioned Reinforcement Learning

In reinforcement learning (RL), it is easier to solve a task if given a good representation. While deep RL should automatically acquire such good representations, prior work often finds that learning representations in an end-to-end fashion is unstable and instead equip RL algorithms with additional representation learning parts (e.g., auxiliary losses, data augmentation). How can we design RL algorithms that directly acquire good representations? In this paper, instead of adding representation learning parts to an existing RL algorithm, we show (contrastive) representation learning methods can be cast as RL algorithms in their own right. To do this, we build upon prior work and apply contrastive representation learning to action-labeled trajectories, in such a way that the (inner product of) learned representations exactly corresponds to a goal-conditioned value function. We use this idea to reinterpret a prior RL method as performing contrastive learning, and then use the idea to propose a much simpler method that achieves similar performance. Across a range of goal-conditioned RL tasks, we demonstrate that contrastive RL methods achieve higher success rates than prior non-contrastive methods, including in the offline RL setting. We also show that contrastive RL outperforms prior methods on image-based tasks, without using data augmentation or auxiliary objectives.
Reconstruction of Coronal Magnetic Fields Using a Poloidal–Toroidal Representation

A new method for reconstruction of coronal magnetic fields as force-free fields (FFFs) is presented. Our method employs poloidal and toroidal functions to describe divergence-free magnetic fields. This magnetic field representation naturally enables us to implement the boundary conditions at the photospheric boundary, i.e., the normal magnetic field and the normal current density there, in a straightforward manner. At the upper boundary of the corona, a source surface condition can be employed, which accommodates magnetic flux imbalance at the bottom boundary. Although our iteration algorithm is inspired by extant variational methods, it is nonvariational and requires far fewer iteration steps than most others. The computational code based on our new method is tested against the analytical FFF solutions by Titov & Démoulin. It is found to excel in reproducing a tightly wound flux rope, a bald patch, and quasi-separatrix layers with a hyperbolic flux tube.
Plurality Veto: A Simple Voting Rule Achieving Optimal Metric Distortion

The metric distortion framework posits that n voters and m candidates are jointly embedded in a metric space such that voters rank candidates that are closer to them higher. A voting rule's purpose is to pick a candidate with minimum total distance to the voters, given only the rankings, but not the actual distances. As a result, in the worst case, each deterministic rule picks a candidate whose total distance is at least three times larger than that of an optimal one, i.e., has distortion at least 3. A recent breakthrough result showed that achieving this bound of 3 is possible; however, the proof is non-constructive, and the voting rule itself is a complicated exhaustive search. Our main result is an extremely simple voting rule, called Plurality Veto, which achieves the same optimal distortion of 3. Each candidate starts with a score equal to his number of first-place votes. These scores are then gradually decreased via an n-round veto process in which a candidate drops out when his score reaches zero. One after the other, voters decrement the score of their bottom choice among the standing candidates, and the last standing candidate wins. We give a one-paragraph proof that this voting rule achieves distortion 3. This rule is also immensely practical, and it only makes two queries to each voter, so it has low communication overhead. We also show that a straightforward extension can be used to give a constructive proof of the more general Ranking-Matching Lemma of Gkatzelis et al. We also generalize Plurality Veto into a class of randomized voting rules in the following way: Plurality veto is run only for k < n rounds; then, a candidate is chosen with probability proportional to his residual score. This general rule interpolates between Random Dictatorship (for k=0) and Plurality Veto (for k=n-1), and k controls the variance of the output. We show that for all k, this rule has expected distortion at most 3.
Mandheling: mixed-precision on-device DNN training with DSP offloading

This paper proposes Mandheling, the first system that enables highly resource-efficient on-device training by orchestrating mixed-precision training with on-chip Digital Signal Processor (DSP) offloading. Mandheling fully explores the advantages of DSP in integer-based numerical calculations using four novel techniques: (1) a CPU-DSP co-scheduling scheme to situationally mitigate the overhead from DSP-unfriendly operators; (2) a self-adaptive rescaling algorithm to reduce the overhead of dynamic rescaling in backward propagation; (3) a batch-splitting algorithm to improve DSP cache efficiency; (4) a DSP compute subgraph-reusing mechanism to eliminate the preparation overhead on DSP. We have fully implemented Mandheling and demonstrated its effectiveness through extensive experiments. The results show that, compared to the state-of-the-art DNN engines from TFLite and MNN, Mandheling reduces per-batch training time by 5.5X and energy consumption by 8.9X on average. In end-to-end training tasks, Mandheling reduces convergence time by up to 10.7X and energy consumption by 13.1X, with only 1.9%--2.7% accuracy loss compared to the FP32 precision setting.
Empirical likelihood estimation for ?nite populations and the e?ective usage of auxiliary informatio

SUMMARY In finite population inference problems, auxiliary population information is often available. We show in this paper that the empirical likelihood method can be naturally applied to such problems to make effective use of the auxiliary information. We prove that the resulting estimates have smaller asymptotic variances than the usual estimates which do not use auxiliary information. A Bahadur-type representation for empirical likelihood sample quantiles is given. Simulation results show that the empirical likelihood estimates perform well among a number of competitors and are model robust.
Timing and probability of arrival for sea lice dispersing between salmon farms

Sea lice are a threat to the health of both wild and farmed salmon and an economic burden for salmon farms. With a free-living larval stage, sea lice can disperse tens of kilometres in the ocean between salmon farms, leading to connected sea louse populations that are difficult to control in isolation. In this paper, we develop a simple analytical model for the dispersal of sea lice (Lepeophtheirus salmonis) between two salmon farms. From the model, we calculate the arrival time distribution of sea lice dispersing between farms, as well as the level of cross-infection of sea lice. We also use numerical flows from a hydrodynamic model, coupled with a particle tracking model, to directly calculate the arrival time of sea lice dispersing between two farms in the Broughton Archipelago, British Columbia, in order to fit our analytical model and find realistic parameter estimates. Using the parametrized analytical model, we show that there is often an intermediate interfarm spacing that maximizes the level of cross-infection between farms, and that increased temperatures will lead to increased levels of cross-infection.
Knowledge Management System with NLP-Assisted Annotations: A Brief Survey and Outlook

Knowledge management systems (KMS) are in high demand for industrial researchers, chemical or research enterprises, or evidence-based decision making. However, existing systems have limitations in categorizing and organizing paper insights or relationships. Traditional databases are usually disjoint with logging systems, which limit its utility in generating concise, collated overviews. In this work, we briefly survey existing approaches of this problem space and propose a unified framework that utilizes relational databases to log hierarchical information to facilitate the research and writing process, or generate useful knowledge from references or insights from connected concepts. Our framework of bidirectional knowledge management system (BKMS) enables novel functionalities encompassing improved hierarchical note-taking, AI-assisted brainstorming, and multi-directional relationships. Potential applications include managing inventories and changes for manufacture or research enterprises, or generating analytic reports with evidence-based decision making.
Cautious Learning of Multiattribute Preferences

This paper is dedicated to a cautious learning methodology for predicting preferences between alternatives characterized by binary attributes (for-mally, each alternative is seen as a subset of at-tributes). By “cautious”, we mean that the model learned to represent the multi-attribute preferences is general enough to be compatible with any strict weak order on the alternatives, and that we allow ourselves not to predict some preferences if the data collected are not compatible with a reliable prediction. A predicted preference will be considered reliable if all the simplest models (following Occam’s razor principle) explaining the training data agree on it. Predictions are based on an ordinal dominance relation between alternatives [Fishburn and LaValle, 1996]. The dominance relation relies on an uncertainty set encompassing the possible values of the parameters of the multi-attribute utility function. Numerical tests are provided to evaluate the richness and the reliability of the predictions made.