Miracle-dz/UniFAR-FTUs huggingface dataset:
Gaze tracking of large-billed crows (Corvus macrorhynchos) in a motion-capture system

The visually guided behaviors of corvids (Corvidae) are often examined in previous studies because they provide important clues about their perception, attention, and cognition. However, the details regarding how they orient their heads toward the visual targets (or how they use their visual fields when attending to the visual targets) remain largely unclear. This study used a newly established motion capture system to examine the visual field use of large-billed crows (Corvus macrorhynchos). Study 1 employed an established ophthalmoscopic reflex technique to identify the visual field configuration, including the binocular width and optic axes, as well as the degree of eye movement. Study 2 used the motion capture system to track the head movements of freely moving crows and examined how they oriented their reconstructed visual fields toward attention-getting objects. When visual targets were moving, the crows used their binocular visual fields, particularly around the projection of the beak-tip. When the visual targets stopped moving, crows frequently used non-binocular visual fields, particularly around the regions where their optic axes were found in Study 1 (close to their visual axes). On such occasions, the crows slightly preferred the right eye. Overall, the visual field use of crows is clearly predictable. Thus, while the untracked eye movements could introduce some level of uncertainty (typically within 15 degrees), we demonstrated the feasibility of inferring a crow’s attentional focus by 3D tracking of their heads. Our system represents a promising initial step towards establishing gaze tracking methods for studying corvid behavior and cognition.
Knowing who I am depends on who I’ve become: Linking self-concept clarity and temporal self-comparison

ABSTRACT The current research examined how temporal self-comparisons influence self-concept clarity. In Studies 1 and 2, we updated and validated self-report and indirect measures of state self-concept clarity, including specific components identified in prior research. In both studies, participants’ confidence in their self-judgments, as well as the consistency of those judgments, were associated with the self-reported state self-concept clarity. Using the updated measures, Study 3 found that self-concept clarity was highest when participants engaged in temporal comparisons that focused on positive similarities between their past and present selves and when comparisons highlighted improvement over time. These findings suggest that self-concept clarity is an emergent phenomenon that arises from comparison-based self-evaluations that fit common lay theories about how the self develops over time.
Fixational Eye Movements Enhance the Precision of Visual Information Transmitted by the Primate Retina

Fixational eye movements alter the number and timing of spikes transmitted from the retina to the brain, but whether these changes enhance or degrade the visual signal is unclear. To quantify this, we developed a Bayesian method for reconstructing natural images from the recorded spikes of hundreds of macaque retinal ganglion cells (RGCs) of the major cell types, combining a likelihood model for RGC light responses with the natural image prior implicitly embedded in an artificial neural network optimized for denoising. The method matched or surpassed the performance of previous reconstruction algorithms, and provided an interpretable framework for characterizing the retinal signal. Reconstructions were improved with artificial stimulus jitter that emulated fixational eye movements, even when the jitter trajectory was inferred from retinal spikes. Reconstructions were degraded by small artificial perturbations of spike times, revealing more precise temporal encoding than suggested by previous studies. Finally, reconstructions were substantially degraded when derived from a model that ignored cell-to-cell interactions, indicating the importance of stimulus-evoked correlations. Thus, fixational eye movements enhance the precision of the retinal representation.
Measuring and monitoring skill learning in closed-loop myoelectric hand prostheses using speed-accuracy tradeoffs

Objective Closed-loop myoelectric prostheses, which combine supplementary sensory feedback and electromyography (EMG) based control, hold the potential to narrow the divide between natural and bionic hands. The use of these devices, however, requires dedicated training. Therefore, it is crucial to develop methods that quantify how users acquire skilled control over their prostheses to effectively monitor skill progression and inform the development of interfaces that optimize this process. Approach Building on theories of skill learning in human motor control, we measured speed-accuracy tradeoff functions (SAFs) to comprehensively characterize learning-induced changes in skill – as opposed to merely tracking changes in task success across training – facilitated by a closed-loop interface that combined proportional control and EMG feedback. Sixteen able-bodied and one amputee participated in a 3-day experiment where they were instructed to perform the box-and-blocks task using a timed force-matching paradigm at four specified speeds to reach two target force levels, such that the SAF could be determined. Main results We found that the participants’ accuracy increased in a similar way across all speeds we tested. Consequently, the shape of the SAF remained similar across days, at both force levels. Further, we observed that EMG feedback enabled participants to improve their motor execution in terms of reduced trial-by-trial variability, a hallmark of skilled behavior. We then fit a power law model of the SAF, and demonstrated how the model parameters could be used to identify and monitor changes in skill. Significance We comprehensively characterized how an EMG feedback interface enabled skill acquisition, both at the level of task performance and movement execution. More generally, we believe that the proposed methods are effective for measuring and monitoring user skill progression in closed-loop prosthesis control.
A Critical Revisit of Adversarial Robustness in 3D Point Cloud Recognition with Diffusion-Driven Purification

3D point clouds serve as a crucial data representation in numerous real-world applications such as autonomous driving, robotics, and medical imaging. While the advancements in deep learning have spurred the utilization of 3D point clouds, deep models are notoriously vulnerable to adversarial attacks. Various defense solutions have been proposed to build robust models against adversarial attacks. In this work, we pinpoint a major limitation of the leading empirical defense, adversarial training, when applied to 3D point cloud models: gradient obfuscation, which significantly hampers robustness against potent attacks. To bridge the gap, we propose PointDP , a purification strategy that leverages diffusion models to defend against 3D adversarial attacks. Since PointDP does not rely on predefined adversarial examples for training, it can defend against a variety of threats. We conduct a comprehensive evaluation of PointDP across six representative 3D point cloud architectures, employing sixteen strong and adaptive attacks to manifest its foundational robustness. Our evaluation shows that PointDP achieves significantly better ( i.e., 12.6%-40.3%) adversarial robustness than state-of-the-art methods under strong attacks bounded by different ℓ p norms.
Functional Domains of Substance Use and Trauma: A Systematic Review of Neuroimaging Studies

In a framework for substance use concerning trauma, Hien and colleagues suggested three domains: reward salience, executive function, and negative emotionality. In this PRISMA-guided systematic review, we explored the neural correlates of these domains in individuals who use substances with or without trauma exposure. We included 45 studies utilizing tasks of interest in alcohol, tobacco, and cannabis use groups. Greater reward, lesser regulation of inhibitory processes, and mixed findings of negative emotionality processes in individuals who use substances versus controls were found. Specifically, greater orbitofrontal cortex, ventral tegmental area, striatum, amygdala, and hippocampal activation was found in response to reward-related tasks, and reduced activation was found in the inferior frontal gyrus and hippocampus in response to inhibition-related tasks. No studies in trauma-exposed individuals met our review criteria. Future studies examining the role of trauma-related factors are needed and should explore inhibition- and negative-emotionality domains in individuals who use substances to uncover alterations in these domains that place an individual at greater risk for developing SUD.
Network Effects in Performative Prediction Games

This paper studies the multi-agent performative prediction (Multi-PP) games over multiplex networks. We consider a distributed learning setting where agents partially cooperate on an agent network , while during learning, the data samples drawn depend on the prediction models of the agent itself and neighboring agents on a population network . The dynamics of Multi-PP games is hence affected by the interplay between both networks. This paper concentrates on this Multi-PP game with the following contributions. Firstly, we analyze sufficient conditions for the existence of the performative stable equilibrium (PSE) and Nash equilibrium (NE) of the Multi-PP games. Secondly, we analyze the changes to the equilibrium induced by perturbed data distributions, and derive the closed-form solutions where the network topologies are explicit. Our results connect the existence of PSE/NE with strengths of agents’ cooperation, and the changes of equilibrium solutions across agents with their node centrality, etc. Lastly, we show that a stochastic gradient descent (SGD) based distributed learning procedure finds the PSE under the said sufficient condition. Numerical illustrations on the network effects in Multi-PP games corroborate our findings.
Loss-Guided Diffusion Models for Plug-and-Play Controllable Generation

We consider guiding denoising diffusion models with general differentiable loss functions in a plug-and-play fashion, enabling controllable generation without additional training. This paradigm, termed Loss-Guided Diffusion (LGD), can easily be integrated into all diffusion models and leverage various efficient samplers. Despite the benefits, the resulting guidance term is, unfortunately, an intractable integral and needs to be approximated. Existing methods compute the guidance term based on a point estimate. However, we show that such approaches have significant errors over the scale of the approximations. To address this issue, we propose a Monte Carlo method that uses multiple samples from a suitable distribution to reduce bias. Our method is effective in various synthetic and real-world settings, including image super-resolution, text or label-conditional image generation, and controllable motion synthesis. Notably, we show how our method can be applied to control a pretrained motion diffusion model to follow certain paths and avoid obstacles that are proven challenging to prior methods.
Function-Space Regularization in Neural Networks: A Probabilistic Perspective

Parameter-space regularization in neural network optimization is a fundamental tool for improving generalization. However, standard parameter-space regularization methods make it challenging to encode explicit preferences about desired predictive functions into neural network training. In this work, we approach regularization in neural networks from a probabilistic perspective and show that by viewing parameter-space regularization as specifying an empirical prior distribution over the model parameters, we can derive a probabilistically well-motivated regularization technique that allows explicitly encoding information about desired predictive functions into neural network training. This method -- which we refer to as function-space empirical Bayes (FSEB) -- includes both parameter- and function-space regularization, is mathematically simple, easy to implement, and incurs only minimal computational overhead compared to standard regularization techniques. We evaluate the utility of this regularization technique empirically and demonstrate that the proposed method leads to near-perfect semantic shift detection, highly-calibrated predictive uncertainty estimates, successful task adaption from pre-trained models, and improved generalization under covariate shift.
Differential Privacy, Linguistic Fairness, and Training Data Influence: Impossibility and Possibility Theorems for Multilingual Language Models

Language models such as mBERT, XLM-R, and BLOOM aim to achieve multilingual generalization or compression to facilitate transfer to a large number of (potentially unseen) languages. However, these models should ideally also be private, linguistically fair, and transparent, by relating their predictions to training data. Can these requirements be simultaneously satisfied? We show that multilingual compression and linguistic fairness are compatible with differential privacy, but that differential privacy is at odds with training data influence sparsity, an objective for transparency. We further present a series of experiments on two common NLP tasks and evaluate multilingual compression and training data influence sparsity under different privacy guarantees, exploring these trade-offs in more detail. Our results suggest that we need to develop ways to jointly optimize for these objectives in order to find practical trade-offs.
FedHPO-Bench: A Benchmark Suite for Federated Hyperparameter Optimization

Research in the field of hyperparameter optimization (HPO) has been greatly accelerated by existing HPO benchmarks. Nonetheless, existing efforts in benchmarking all focus on HPO for traditional learning paradigms while ignoring federated learning (FL), a promising paradigm for collaboratively learning models from dispersed data. In this paper, we first identify some uniqueness of federated hyperparameter optimization (FedHPO) from various aspects, showing that existing HPO benchmarks no longer satisfy the need to study FedHPO methods. To facilitate the research of FedHPO, we propose and implement a benchmark suite F ED HPO-B ENCH that incorporates comprehensive FedHPO problems, enables flexible customization of the function evaluations, and eases continuing extensions. We conduct extensive experiments based on F ED HPO-B ENCH to provide the community with more insights into FedHPO. We open-sourced F ED HPO-B ENCH at https://github. com/alibaba/FederatedScope/tree/ master/benchmark/FedHPOBench .
Robust Explanation for Free or At the Cost of Faithfulness

Devoted to interpreting the explicit behaviors of machine learning models, explanation methods can identify implicit characteristics of models to improve trustworthiness. However, explanation methods are shown as vulnerable to adversarial perturbations, implying security concerns in high-stakes domains. In this paper, we investigate when robust explanations are necessary and what they cost. We prove that the robustness of explanations is determined by the robustness of the model to be explained. Therefore, we can have robust explanations for free for a robust model. To have robust explanations for a non-robust model, composing the original model with a kernel is proved as an effective way that returns strictly more robust explanations. Nevertheless, we argue that this also incurs a robustness-faithfulness trade-off , i.e., contrary to common expectations, an explanation method may also become less faithful when it becomes more robust. This argument holds for any model. We are the first to introduce this trade-off and theoretically prove its existence for SmoothGrad. Theoretical findings are verified by empirical evidence on six state-of-the-art explanation methods and four backbones.
The Aggregation-Heterogeneity Trade-off in Federated Learning

Conventional wisdom in machine learning holds that the more data you train your model on, the better the model can perform. Accordingly, a plethora of federated learning methods have been developed to aggregate as many local samples as possible. Contrary to this belief, this paper shows that aggregation of more data is not necessarily beneficial in the presence of heterogeneity, and reveals a fundamental trade-off between aggregation and heterogeneity in federated learning. We consider a general family of weighted M -estimators that interpolate between FedAvg and the local estimator, in which an aggregation rule is determined by the weights of local samples. We derive an upper bound for the estimation error of the weighted M -estimators, which decomposes into a bias term induced by heterogeneity and a variance term influenced by aggregation. A measure of heterogeneity, the federated smoothness β , is introduced to simplify the general result. As an important consequence, the optimal aggregation rule for each local device is to aggregate only its ⌊ K 2 β/ (2 β +1) / ( n/σ 2 ) 1 / (2 β +1) ⌋ ∨ 1 closest neighbors among the K devices, where n is the local sample size and σ 2 is the noise variance. Moreover, we show that our estimator, termed FedKNN, attains the minimax optimal rate over a certain parameter space characterized by β . This optimal procedure depends crucially on the neighboring structure among devices in terms of the proximity of local parameters. Finally, we prove that without such prior knowledge no estimator can achieve a convergence rate faster than O ( σ 2 /n ) and hence adaptation is impossible.
Towards Understanding Generalization of Graph Neural Networks

Graph neural networks (GNNs) are widely used in machine learning for graph-structured data. Even though GNNs have achieved remarkable success in real-world applications, understanding their working mechanism in theory is still on primary stage. In this paper, we move towards this goal from the perspective of generalization. Specifically, with consideration of stochastic optimization, we establish high probability bounds of generalization gap and gradients for transductive learning algorithms. After that, we provide high probability bounds of generalization gap for popular GNNs and analyze the factors affecting their generalization capability. These theoretical results reveal how the network architecture impacts the generalization gap. Experiments on benchmark datasets validate the theoretical findings. Our results provide new insights into understanding generalization of GNNs.
Understanding and Generalizing Contrastive Learning from the Inverse Optimal Transport Perspective

Previous research on contrastive learning (CL) has primarily focused on pairwise views to learn representations by attracting positive samples and repelling negative ones. In this work, we aim to understand and generalize CL from a point set matching perspective, instead of the comparison between two points. Specifically, we formulate CL as a form of inverse optimal transport (IOT), which involves a bilevel optimization procedure for learning where the outter minimization aims to learn the representations and the inner is to learn the coupling (i.e. the probability of matching matrix) between the point sets. Specifically, by adjusting the relaxation degree of constraints in the inner minimization, we obtain three contrastive losses and show that the dominant contrastive loss in literature InfoNCE falls into one of these losses. This reveals a new and more general algorithmic framework for CL. Additionally, the soft matching scheme in IOT induces a uniformity penalty to enhance representation learning which is akin to the CL’s uniformity. Results on vision benchmarks show the effectiveness of our derived loss family and the new uniformity term.
No One Idles: Efficient Heterogeneous Federated Learning with Parallel Edge and Server Computation

Federated learning suffers from a latency bottle-neck induced by network stragglers, which hampers the training efficiency significantly. In addition, due to the heterogeneous data distribution and security requirements, simple and fast averaging aggregation is not feasible anymore. Instead, complicated aggregation operations, such as knowledge distillation, are required. The time cost for complicated aggregation becomes a new bottleneck that limits the computational efficiency of FL. In this work, we claim that the root cause of training latency actually lies in the aggregation-then-broadcasting workflow of the server. By swapping the computational order of aggregation and broadcasting, we propose a novel and efficient parallel federated learning (PFL) framework that unlocks the edge nodes during global computation and the central server during local computation. This fully asynchronous and parallel pipeline enables handling complex aggregation and network stragglers, allowing flexible device participation as well as achieving scalability in computation. We theoretically prove that synchronous and asynchronous PFL can achieve a similar convergence rate as vanilla FL. Extensive experiments empirically show that our framework brings up to 5 . 56 × speedup compared with traditional FL. Code is available at: https://github.com/Hypervoyager/PFL .
Nugget: Neural Agglomerative Embeddings of Text

Embedding text sequences is a widespread requirement in modern language understanding. Existing approaches focus largely on constant-size representations. This is problematic, as the amount of information contained in text often varies with the length of the input. We propose a solution called Nugget, which encodes language into a representation based on a dynamically selected subset of input tokens. These nuggets are learned through tasks like autoencoding and machine translation, and intuitively segment language into meaningful units. We demonstrate Nugget outperforms related approaches in tasks involving semantic comparison. Finally, we illustrate these compact units allow for expanding the contextual window of a language model (LM), suggesting new future LMs that can condition on significantly larger amounts of content.
Discover-Then-Rank Unlabeled Support Vectors in the Dual Space for Multi-Class Active Learning

We propose to approach active learning (AL) from a novel perspective of discovering and then ranking potential support vectors by leveraging the key properties of the dual space of a sparse kernel max-margin predictor. We theoretically analyze the change of a hinge loss in the dual form and provide both the upper and lower bounds that are deeply connected to the key geometric properties induced by the dual space, which then help us identify various types of important data samples for AL. These bounds inform the design of a novel sampling strategy that leverages class-wise evidence as a key vehicle, formed through an affine combination of dual variables and kernel evaluation. We construct two distinct types of sampling functions, including 1) discovery, which focuses on samples with low total evidence from all classes to support exploration, and 2) ranking, which aims to further refine the decision boundary. These two functions are automatically arranged into a two-phase active sampling process to balance exploration and exploitation. Experiments on various real-world data demonstrate the state-of-the-art AL performance achieved by our model.