Miracle-dz/UniFAR-FTUs huggingface dataset:
Phase Synchrony Component Self-Organization in Brain Computer Interface

Phase synchrony information plays a crucial role in analyzing functional brain connectivity and identifying brain activities. A widely adopted feature extraction pipeline, composed of preprocessing, selection of EEG acquisition channels, and phase locking value (PLV) calculation, has achieved success in motor imagery classification (MI). However, this pipeline is manual and reliant on expert knowledge, limiting its convenience and adaptability to different application scenarios. Moreover, most studies have employed mediocre data-independent spatial filters to suppress noise, impeding the exploration of more significant phase synchronization phenomena. To address the issues, we propose the concept of phase synchrony component self-organization, which enables the adaptive learning of data-dependent spatial filters for automating both the preprocessing and channel selection procedures. Based on this concept, the first deep learning end-to-end network is developed, which directly extracts phase synchrony-based features from raw EEG signals and perform classification. The network learns optimal filters during training, which are obtained when the network achieves peak classification results. Extensive experiments have demonstrated that our network outperforms state-of-the-art methods. Remarkably, through the learned optimal filters, significant phase synchronization phenomena can be observed. Specifically, by calculating the PLV between a pair of signals extracted from each sample using two of the learned spatial filters, we have obtained an average PLV exceeding 0.87 across all tongue MI samples. This high PLV indicates a groundbreaking discovery in the synchrony pattern of tongue MI.
A Meta-Learning Perspective on Transformers for Causal Language Modeling

The Transformer architecture has become prominent in developing large causal language models. However, mechanisms to explain its capabilities are not well understood. Focused on the training process, here we establish a meta-learning view of the Transformer architecture when trained for the causal language modeling task, by explicating an inner optimization process within the Transformer. Further, within the inner optimization, we discover and theoretically analyze a special characteristic of the norms of learned token representations within Transformer-based causal language models. Our analysis is supported by experiments in various settings.
Learning A Disentangling Representation For PU Learning

In this paper, we address the problem of learning a binary (positive vs. negative) classifier given Positive and Unlabeled data commonly referred to as PU learning. Although rudimentary techniques like clustering, out-of-distribution detection, or positive density estimation can be used to solve the problem in low-dimensional settings, their efficacy progressively deteriorates with higher dimensions due to the increasing complexities in the data distribution. In this paper we propose to learn a neural network-based data representation using a loss function that can be used to project the unlabeled data into two (positive and negative) clusters that can be easily identified using simple clustering techniques, effectively emulating the phenomenon observed in low-dimensional settings. We adopt a vector quantization technique for the learned representations to amplify the separation between the learned unlabeled data clusters. We conduct experiments on simulated PU data that demonstrate the improved performance of our proposed method compared to the current state-of-the-art approaches. We also provide some theoretical justification for our two cluster-based approach and our algorithmic choices.
F\'eeton ($B-L$ Gauge Boson) Dark Matter for the 511-keV Gamma-Ray Excess and the Prediction of Low-energy Neutrino Flux

The f\'eeton is the gauge boson of the $U(1)_{B-L}$ gauge theory. If the gauge coupling constant is extremely small, it becomes a candidate for dark matter. We show that its decay to a pair of electron and positron explains the observed Galactic 511-keV gamma-ray excess in a consistent manner. This f\'eeton dark matter decays mainly into pairs of neutrino and anti-neutrino. Future low-energy experiments with improved directional capability make it possible to capture those neutrino signals. The seesaw-motivated parameter space predicts a relatively short f\'eeton lifetime comparable to the current cosmological constraint.
Visual Abductive Reasoning Meets Driving Hazard Prediction: Problem Formulation and Dataset

This paper addresses the problem of predicting hazards that drivers may encounter while driving a car. We formulate it as a task of anticipating impending accidents using a single input image captured by car dashcams. Unlike existing approaches to driving hazard prediction that rely on computational simulations or anomaly detection from videos, this study focuses on high-level inference from static images. The problem needs predicting and reasoning about future events based on uncertain observations, which falls under visual abductive reasoning. To enable research in this understudied area, a new dataset named the DHPR (Driving Hazard Prediction and Reasoning) dataset is created. The dataset consists of 15K dashcam images of street scenes, and each image is associated with a tuple containing car speed, a hypothesized hazard description, and visual entities present in the scene. These are annotated by human annotators, who identify risky scenes and provide descriptions of potential accidents that could occur a few seconds later. We present several baseline methods and evaluate their performance on our dataset, identifying remaining issues and discussing future directions. This study contributes to the field by introducing a novel problem formulation and dataset, enabling researchers to explore the potential of multi-modal AI for driving hazard prediction.
Generating scalable graph states in an atom-nanophotonic interface

Scalable graph states are essential for measurement-based quantum computation and many entanglement-assisted applications in quantum technologies. Generation of these multipartite entangled states requires a controllable and efficient quantum device with delicate design of generation protocol. Here we propose to prepare high-fidelity and scalable graph states in one and two dimensions, which can be tailored in an atom-nanophotonic cavity via state carving technique. We propose a systematic protocol to carve out unwanted state components, which facilitates scalable graph states generations via adiabatic transport of a definite number of atoms in optical tweezers. An analysis of state fidelity is also presented, and the state preparation probability can be optimized via multiqubit state carvings and sequential single-photon probes. Our results showcase the capability of an atom-nanophotonic interface for creating graph states and pave the way toward novel problem-specific applications using scalable high-dimensional graph states with stationary qubits.
Discovering interpretable physical models using symbolic regression and discrete exterior calculus

Computational modeling is a key resource to gather insight into physical systems in modern scientific research and engineering. While access to large amount of data has fueled the use of machine learning to recover physical models from experiments and increase the accuracy of physical simulations, purely data-driven models have limited generalization and interpretability. To overcome these limitations, we propose a framework that combines symbolic regression (SR) and discrete exterior calculus (DEC) for the automated discovery of physical models starting from experimental data. Since these models consist of mathematical expressions, they are interpretable and amenable to analysis, and the use of a natural, general-purpose discrete mathematical language for physics favors generalization with limited input data. Importantly, DEC provides building blocks for the discrete analog of field theories, which are beyond the state-of-the-art applications of SR to physical problems. Further, we show that DEC allows to implement a strongly-typed SR procedure that guarantees the mathematical consistency of the recovered models and reduces the search space of symbolic expressions. Finally, we prove the effectiveness of our methodology by re-discovering three models of continuum physics from synthetic experimental data: Poisson equation, the Euler’s elastica and the equations of linear elasticity. Thanks to their general-purpose nature, the methods developed in this paper may be applied to diverse contexts of physical modeling.
The Constraining Capability of BNS Dark Sirens Observed by the LIGO Gravitational Wave Detector on the Hubble Constant

The Hubble Constant observed at high redshift and low redshift are inconsistent, representing one of the urgent issues to be resolved in the field of cosmology. The discovery of gravitational waves opens a new window for addressing this problem. For instance, the GW170817 event, through the coordinated observation of electromagnetic and gravitational wave signals, allows for constraints to be imposed from a completely new perspective. However, the number of gravitational wave events where both electromagnetic and gravitational wave signals are observed simultaneously is too small, making it difficult to enhance the precision through statistical methods. In this paper, we use dark sirens as the subjects of study. Through the standard gravitational wave data simulation and the analysis process, we analyze the constraints a typical binary neutron star merger event can place on the Hubble Constant. We simulated a random event and found that it an provide an error of +0.04-0.05 for the Hubble Constant. By combining multiple events, this constraint can be improved.
Quantile-based Random Sparse Kaczmarz for Corrupted and Noisy Linear Systems

The randomzied Kaczmarz method, along with its recently developed variants, has become a popular tool for dealing with large-scale linear systems. However, these methods usually fail to converge when the linear systems are affected by heavy corruptions, which are common in many practical applications. In this study, we develop a new variant of the randomzied sparse Kaczmarz method with linear convergence guarantees, by making use of a quantile technique to detect corruptions. Moreover, we incorporate averaged block technique into the proposed method to achieve parallel computation and acceleration. Finally, the proposed algorithms are illustrated to be very efficient through extensive numerical experiments.
Using non-Markovian dynamics in effective-negative-temperature-based transient quantum Otto engines

We demonstrate that the efficiency of effective negative temperature-based quantum Otto engines, already known to outperform their traditional counterparts operating with positive-temperature thermal reservoirs, can be further improved by terminating the isochoric strokes before the working substance reaches perfect equilibrium with its environment. Our investigation encompasses both Markovian and non-Markovian dynamics during these finite-time isochoric processes while considering a weak coupling between the working substance and the reservoirs. We assess the performance of these engines as they undergo a transition from the Markovian to the non-Markovian regime using two figures of merit: maximum achievable efficiency at a certain finite time during the isochoric heating stroke, and overall performance of the engine over an extended period during the transient phase of this stroke. We show that the maximum efficiency increases with the increase of non-Markovianity. However, the overall engine performance decreases as non-Markovianity increases. Additionally, we discover the existence of effective negative temperature-based necessarily transient quantum Otto engines. These engines operate within an extended operational domain, reaching into temperature ranges where conventional effective negative temperature-based quantum Otto engines, which rely on perfect thermalization during the isochoric strokes, are unable to function. Furthermore, this extended operational domain of an effective negative temperature-based necessarily transient quantum Otto engine increases as non-Markovianity becomes more pronounced.
The Cost of Down-Scaling Language Models: Fact Recall Deteriorates before In-Context Learning

How does scaling the number of parameters in large language models (LLMs) affect their core capabilities? We study two natural scaling techniques -- weight pruning and simply training a smaller or larger model, which we refer to as dense scaling -- and their effects on two core capabilities of LLMs: (a) recalling facts presented during pre-training and (b) processing information presented in-context during inference. By curating a suite of tasks that help disentangle these two capabilities, we find a striking difference in how these two abilities evolve due to scaling. Reducing the model size by more than 30\% (via either scaling approach) significantly decreases the ability to recall facts seen in pre-training. Yet, a 60--70\% reduction largely preserves the various ways the model can process in-context information, ranging from retrieving answers from a long context to learning parameterized functions from in-context exemplars. The fact that both dense scaling and weight pruning exhibit this behavior suggests that scaling model size has an inherently disparate effect on fact recall and in-context learning.
Analyzing Key Users' behavior trends in Volunteer-Based Networks

Online social networks usage has increased significantly in the last decade and continues to grow in popularity. Multiple social platforms use volunteers as a central component. The behavior of volunteers in volunteer-based networks has been studied extensively in recent years. Here, we explore the development of volunteer-based social networks, primarily focusing on their key users' behaviors and activities. We developed two novel algorithms: the first reveals key user behavior patterns over time; the second utilizes machine learning methods to generate a forecasting model that can predict the future behavior of key users, including whether they will remain active donors or change their behavior to become mainly recipients, and vice-versa. These algorithms allowed us to analyze the factors that significantly influence behavior predictions. To evaluate our algorithms, we utilized data from over 2.4 million users on a peer-to-peer food-sharing online platform. Using our algorithm, we identified four main types of key user behavior patterns that occur over time. Moreover, we succeeded in forecasting future active donor key users and predicting the key users that would change their behavior to donors, with an accuracy of up to 89.6%. These findings provide valuable insights into the behavior of key users in volunteer-based social networks and pave the way for more effective communities-building in the future, while using the potential of machine learning for this goal.
Advancing Transformer's Capabilities in Commonsense Reasoning

Recent advances in general purpose pre-trained language models have shown great potential in commonsense reasoning. However, current works still perform poorly on standard commonsense reasoning benchmarks including the Com2Sense Dataset. We argue that this is due to a disconnect with current cutting-edge machine learning methods. In this work, we aim to bridge the gap by introducing current ML-based methods to improve general purpose pre-trained language models in the task of commonsense reasoning. Specifically, we experiment with and systematically evaluate methods including knowledge transfer, model ensemble, and introducing an additional pairwise contrastive objective. Our best model outperforms the strongest previous works by ~15\% absolute gains in Pairwise Accuracy and ~8.7\% absolute gains in Standard Accuracy.
Two-loop five-particle scattering amplitudes

I discuss the recent advances in the computation of two-loop scattering amplitudes for five-particle processes. The latter are fundamental ingredients to obtain predictions at the next-to-next-to-leading order (NNLO) in QCD for many interesting LHC processes. I discuss the state-of-the-art technology for computing scattering amplitudes analytically, and present new results relevant for the LHC phenomenology.
Optimizing Solution-Samplers for Combinatorial Problems: The Landscape of Policy-Gradient Methods

Deep Neural Networks and Reinforcement Learning methods have empirically shown great promise in tackling challenging combinatorial problems. In those methods a deep neural network is used as a solution generator which is then trained by gradient-based methods (e.g., policy gradient) to successively obtain better solution distributions. In this work we introduce a novel theoretical framework for analyzing the effectiveness of such methods. We ask whether there exist generative models that (i) are expressive enough to generate approximately optimal solutions; (ii) have a tractable, i.e, polynomial in the size of the input, number of parameters; (iii) their optimization landscape is benign in the sense that it does not contain sub-optimal stationary points. Our main contribution is a positive answer to this question. Our result holds for a broad class of combinatorial problems including Max- and Min-Cut, Max-$k$-CSP, Maximum-Weight-Bipartite-Matching, and the Traveling Salesman Problem. As a byproduct of our analysis we introduce a novel regularization process over vanilla gradient descent and provide theoretical and experimental evidence that it helps address vanishing-gradient issues and escape bad stationary points.
GReAT: A Graph Regularized Adversarial Training Method

This paper proposes a regularization method called GReAT, Graph Regularized Adversarial Training, to improve deep learning models' classification performance. Adversarial examples are a well-known challenge in machine learning, where small, purposeful perturbations to input data can mislead models. Adversarial training, a powerful and one of the most effective defense strategies, involves training models with both regular and adversarial examples. However, it often neglects the underlying structure of the data. In response, we propose GReAT, a method that leverages data graph structure to enhance model robustness. GReAT deploys the graph structure of the data into the adversarial training process, resulting in more robust models that better generalize its testing performance and defend against adversarial attacks. Through extensive evaluation on benchmark datasets, we demonstrate GReAT's effectiveness compared to state-of-the-art classification methods, highlighting its potential in improving deep learning models' classification performance.
Properly colored even cycles in edge-colored complete balanced bipartite graphs

Consider a complete balanced bipartite graph $K_{n,n}$ and let $K^c_{n,n}$ be an edge-colored version of $K_{n,n}$ that is obtained from $K_{n,n}$ by having each edge assigned a certain color. A subgraph $H$ of $K^c_{n,n}$ is called properly colored (PC) if every two adjacent edges of $H$ have distinct colors. $K_{n,n}^c$ is called properly vertex-even-pancyclic if for every vertex $u\in V(K_{n,n}^c)$ and for every even integer $k$ with $4 \leq k \leq 2n$, there exists a PC $k$-cycle containing $u$. The minimum color degree $\delta^c(K^c_{n,n})$ of $K^c_{n,n}$ is the largest integer $k$ such that for every vertex $v$, there are at least $k$ distinct colors on the edges incident to $v$. In this paper we study the existence of PC even cycles in $K_{n,n}^c$. We first show that, for every integer $t\geq 3$, every $K^c_{n,n}$ with $\delta^c(K^c_{n,n})\geq \frac{2n}{3}+t$ contains a PC 2-factor $H$ such that every cycle of $H$ has a length of at least $t$. By using the probabilistic method and absorbing technique, we use the above result to further show that, for every $\varepsilon>0$, there exists an integer $n_0(\varepsilon)$ such that every $K^c_{n,n}$ with $n\geq n_0(\varepsilon)$ is properly vertex-even-pancyclic, provided that $\delta^c(K^c_{n,n})\geq (\frac{2}{3}+\varepsilon)n$.
Learning force laws in many-body systems

Scientific laws describing natural systems may be more complex than our intuition can handle, and thus how we discover laws must change. Machine learning (ML) models can analyze large quantities of data, but their structure should match the underlying physical constraints to provide useful insight. Here we demonstrate a ML approach that incorporates such physical intuition to infer force laws in dusty plasma experiments. Trained on 3D particle trajectories, the model accounts for inherent symmetries and non-identical particles, accurately learns the effective non-reciprocal forces between particles, and extracts each particle's mass and charge. The model's accuracy (R^2>0.99) points to new physics in dusty plasma beyond the resolution of current theories and demonstrates how ML-powered approaches can guide new routes of scientific discovery in many-body systems.
FMM-Head: Enhancing Autoencoder-based ECG anomaly detection with prior knowledge

Detecting anomalies in electrocardiogram data is crucial to identifying deviations from normal heartbeat patterns and providing timely intervention to at-risk patients. Various AutoEncoder models (AE) have been proposed to tackle the anomaly detection task with ML. However, these models do not consider the specific patterns of ECG leads and are unexplainable black boxes. In contrast, we replace the decoding part of the AE with a reconstruction head (namely, FMM-Head) based on prior knowledge of the ECG shape. Our model consistently achieves higher anomaly detection capabilities than state-of-the-art models, up to 0.31 increase in area under the ROC curve (AUROC), with as little as half the original model size and explainable extracted features. The processing time of our model is four orders of magnitude lower than solving an optimization problem to obtain the same parameters, thus making it suitable for real-time ECG parameters extraction and anomaly detection.
Cross-Edge Orchestration of Serverless Functions with Probabilistic Caching

Serverless edge computing adopts an event-based paradigm that provides back-end services on an as-used basis, resulting in efficient resource utilization. To improve the end-to-end latency and revenue, service providers need to optimize the number and placement of serverless containers while considering the system cost incurred by the provisioning. The particular reason for this circumstance is that frequently creating and destroying containers not only increases the system cost but also degrades the time responsiveness due to the cold-start process. Function caching is a common approach to mitigate the coldstart issue. However, function caching requires extra hardware resources and hence incurs extra system costs. Furthermore, the dynamic and bursty nature of serverless invocations remains an under-explored area. Hence, it is vitally important for service providers to conduct a context-aware request distribution and container caching policy for serverless edge computing. In this paper, we study the request distribution and container caching problem in serverless edge computing. We prove the proposed problem is NP-hard and hence difficult to find a global optimal solution. We jointly consider the distributed and resource constrained nature of edge computing and propose an optimized request distribution algorithm that adapts to the dynamics of serverless invocations with a theoretical performance guarantee. Also, we propose a context-aware probabilistic caching policy that incorporates a number of characteristics of serverless invocations. Via simulation and implementation results, we demonstrate the superiority of the proposed algorithm by outperforming existing caching policies in terms of the overall system cost and cold-start frequency by up to 62.1% and 69.1%, respectively.