Miracle-dz/UniFAR-FTUs huggingface dataset:
The Localized Subtraction Approach For EEG and MEG Forward Modeling

In FEM-based EEG and MEG source analysis, the subtraction approach has been proposed to simulate sensor measurements generated by neural activity. While this approach possesses a rigorous foundation and produces accurate results, its major downside is that it is computationally prohibitively expensive in practical applications. To overcome this, we developed a new approach, called the localized subtraction approach. This approach is designed to preserve the mathematical foundation of the subtraction approach, while also leading to sparse right-hand sides in the FEM formulation, making it efficiently computable. We achieve this by introducing a cut-off into the subtraction, restricting its influence to the immediate neighborhood of the source. In this work, this approach will be presented, analyzed, and compared to other state-of-the-art FEM right-hand side approaches. Furthermore, we discuss how to arrive at an efficient and stable implementation. We perform validation in multi-layer sphere models where analytical solutions exist. There, we demonstrate that the localized subtraction approach is vastly more efficient than the subtraction approach. Moreover, we find that for the EEG forward problem, the localized subtraction approach is less dependent on the global structure of the FEM mesh when compared to the subtraction approach. Additionally, we show the localized subtraction approach to rival, and in many cases even surpass, the other investigated approaches in terms of accuracy. For the MEG forward problem, we show the localized subtraction approach and the subtraction approach to produce highly accurate approximations of the volume currents close to the source. The localized subtraction approach thus reduces the computational cost of the subtraction approach to an extent that makes it usable in practical applications without sacrificing rigorousness and accuracy.
MUX64, an analogue 64-to-1 multiplexer ASIC for the ATLAS high granularity timing detector

We present the design and the performance of MUX64, a 64-to-1 analogue multiplexer ASIC for the ATLAS High Granularity Timing Detector (HGTD). The MUX64 transmits one of its 64 inputs selected by six address lines for the voltages or temperatures being monitored to an lpGBT ADC channel. The prototype ASICs fabricated in TSMC 130 nm CMOS technology were prepared in wire-bonding and QFN88 packaging format. A total of 280 chips was examined for functionality and quality assurance. The accelerated aging test conducted at 85 °C shows negligible degradation over 16 days.
On the Limitations of Physics-Informed Deep Learning: Illustrations Using First-Order Hyperbolic Conservation Law-Based Traffic Flow Models

Since its introduction in 2017, physics-informed deep learning (PIDL) has garnered growing popularity in understanding the systems governed by physical laws in terms of partial differential equations (PDEs). However, empirical evidence points to the limitations of PIDL for learning certain types of PDEs. In this paper, we (a) present the challenges in training PIDL architecture, (b) contrast the performance of PIDL architecture in learning a first order scalar hyperbolic conservation law and its parabolic counterpart, (c) investigate the effect of training data sampling, which corresponds to various sensing scenarios in traffic networks, (d) comment on the implications of PIDL limitations for traffic flow estimation and prediction in practice. Case studies present the contrast in PIDL results between learning the traffic flow model (LWR PDE) and its diffusive variation. The outcome indicates that PIDL experiences significant challenges in learning the hyperbolic LWR equation due to the non-smoothness of its solution. Conversely, the architecture with parabolic PDE, augmented with the diffusion term, leads to the successful reassembly of the density data even with the shockwaves present. The paper concludes by providing a discussion on recent assessments of reasons behind the challenge PIDL encounters with hyperbolic PDEs and the corresponding mitigation strategies.
Pose-Controllable 3D Facial Animation Synthesis using Hierarchical Audio-Vertex Attention

Most of the existing audio-driven 3D facial animation methods suffered from the lack of detailed facial expression and head pose, resulting in unsatisfactory experience of human-robot interaction. In this paper, a novel pose-controllable 3D facial animation synthesis method is proposed by utilizing hierarchical audio-vertex attention. To synthesize real and detailed expression, a hierarchical decomposition strategy is proposed to encode the audio signal into both a global latent feature and a local vertex-wise control feature. Then the local and global audio features combined with vertex spatial features are used to predict the final consistent facial animation via a graph convolutional neural network by fusing the intrinsic spatial topology structure of the face model and the corresponding semantic feature of the audio. To accomplish pose-controllable animation, we introduce a novel pose attribute augmentation method by utilizing the 2D talking face technique. Experimental results indicate that the proposed method can produce more realistic facial expressions and head posture movements. Qualitative and quantitative experiments show that the proposed method achieves competitive performance against state-of-the-art methods.
Revisiting Graph Persistence for Updates and Efficiency

It is well known that ordinary persistence on graphs can be computed more efficiently than the general persistence. Recently, it has been shown that zigzag persistence on graphs also exhibits similar behavior. Motivated by these results, we revisit graph persistence and propose efficient algorithms especially for local updates on filtrations, similar to what is done in ordinary persistence for computing the vineyard. We show that, for a filtration of length $m$, (i) switches (transpositions) in ordinary graph persistence can be done in $O(\log m)$ time; (ii) zigzag persistence on graphs can be computed in $O(m\log m)$ time, which improves a recent $O(m\log^4n)$ time algorithm assuming $n$, the size of the union of all graphs in the filtration, satisfies $n\in\Omega({m^\varepsilon})$ for any fixed $0<\varepsilon<1$; (iii) open-closed, closed-open, and closed-closed bars in dimension $0$ for graph zigzag persistence can be updated in $O(\log m)$ time, whereas the open-open bars in dimension $0$ and closed-closed bars in dimension $1$ can be done in $O(\sqrt{m}\,\log m)$ time.
Conservation laws with nonlocality in density and velocity and their applicability in traffic flow modelling

In this work we present a nonlocal conservation law with a velocity depending on an integral term over a part of the space. The model class covers already existing models in literature, but it is also able to describe new dynamics mainly arising in the context of traffic flow modelling. We prove the existence and uniqueness of weak solutions of the nonlocal conservation law. Further, we provide a suitable numerical discretization and present numerical examples.
Reward Learning as Doubly Nonparametric Bandits: Optimal Design and Scaling Laws

Specifying reward functions for complex tasks like object manipulation or driving is challenging to do by hand. Reward learning seeks to address this by learning a reward model using human feedback on selected query policies. This shifts the burden of reward specification to the optimal design of the queries. We propose a theoretical framework for studying reward learning and the associated optimal experiment design problem. Our framework models rewards and policies as nonparametric functions belonging to subsets of Reproducing Kernel Hilbert Spaces (RKHSs). The learner receives (noisy) oracle access to a true reward and must output a policy that performs well under the true reward. For this setting, we first derive non-asymptotic excess risk bounds for a simple plug-in estimator based on ridge regression. We then solve the query design problem by optimizing these risk bounds with respect to the choice of query set and obtain a finite sample statistical rate, which depends primarily on the eigenvalue spectrum of a certain linear operator on the RKHSs. Despite the generality of these results, our bounds are stronger than previous bounds developed for more specialized problems. We specifically show that the well-studied problem of Gaussian process (GP) bandit optimization is a special case of our framework, and that our bounds either improve or are competitive with known regret guarantees for the Mat\'ern kernel.
On the Hardness of Robustness Transfer: A Perspective from Rademacher Complexity over Symmetric Difference Hypothesis Space

Recent studies demonstrated that the adversarially robust learning under $\ell_\infty$ attack is harder to generalize to different domains than standard domain adaptation. How to transfer robustness across different domains has been a key question in domain adaptation field. To investigate the fundamental difficulty behind adversarially robust domain adaptation (or robustness transfer), we propose to analyze a key complexity measure that controls the cross-domain generalization: the adversarial Rademacher complexity over {\em symmetric difference hypothesis space} $\mathcal{H} \Delta \mathcal{H}$. For linear models, we show that adversarial version of this complexity is always greater than the non-adversarial one, which reveals the intrinsic hardness of adversarially robust domain adaptation. We also establish upper bounds on this complexity measure. Then we extend them to the ReLU neural network class by upper bounding the adversarial Rademacher complexity in the binary classification setting. Finally, even though the robust domain adaptation is provably harder, we do find positive relation between robust learning and standard domain adaptation. We explain \emph{how adversarial training helps domain adaptation in terms of standard risk}. We believe our results initiate the study of the generalization theory of adversarially robust domain adaptation, and could shed lights on distributed adversarially robust learning from heterogeneous sources, e.g., federated learning scenario.
MUX-PLMs: Pre-training Language Models with Data Multiplexing

The widespread adoption of large language models such as ChatGPT and Bard has led to unprecedented demand for these technologies. The burgeoning cost of inference for ever-increasing model sizes coupled with hardware shortages has limited affordable access and poses a pressing need for efficiency approaches geared towards high throughput and performance. Multi-input multi-output (MIMO) algorithms such as data multiplexing, offer a promising solution with a many-fold increase in throughput by performing inference for multiple inputs at the cost of a single input. Yet these approaches are not currently performant enough to be deployed in modern systems. We change that by developing MUX-PLMs, a class of high throughput pre-trained language models (PLMs) trained with data multiplexing, that can be fine-tuned for any downstream task to yield high-throughput high-performance. Our novel multiplexing and demultiplexing modules proficiently entangle and disentangle inputs, and enable high-performance high throughput that are competitive with vanilla PLMs while achieving 2x/5x inference speedup with only a 1−4% drop on a broad suite of tasks.
3D Climate Simulations of the Archean Find That Methane has a Strong Cooling Effect at High Concentrations

Methane is thought to have been an important greenhouse gas during the Archean, although its potential warming has been found to be limited at high concentrations due to its high shortwave absorption. We use the Met Office Unified Model, a general circulation model, to further explore the climatic effect of different Archean methane concentrations. Surface warming peaks at a pressure ratio pCH4:pCO2 of approximately 0.1, reaching a maximum of up to 7 K before significant cooling above this ratio. Equator‐to‐pole temperature differences also tend to increase up to pCH4 ≤ 300 Pa, which is driven by a difference in radiative forcing at the equator and poles by methane and a reduction in the latitudinal extend of the Hadley circulation. 3D models are important to fully capture the cooling effect of methane, due to these impacts of the circulation.
A Transformer-based Deep Learning Algorithm to Auto-record Undocumented Clinical One-Lung Ventilation Events

As a team studying the predictors of complications after lung surgery, we have encountered high missingness of data on one-lung ventilation (OLV) start and end times due to high clinical workload and cognitive overload during surgery. Such missing data limit the precision and clinical applicability of our findings. We hypothesized that available intraoperative mechanical ventilation and physiological time-series data combined with other clinical events could be used to accurately predict missing start and end times of OLV. Such a predictive model can recover existing miss-documented records and relieves the documentation burden by deploying it in clinical settings. To this end, we develop a deep learning model to predict the occurrence and timing of OLV based on routinely collected intraoperative data. Our approach combines the variables' spatial and frequency domain features, using Transformer encoders to model the temporal evolution and convolutional neural network to abstract frequency-of-interest from wavelet spectrum images. The performance of the proposed method is evaluated on a benchmark dataset curated from Massachusetts General Hospital (MGH) and Brigham and Women's Hospital (BWH). Experiments show our approach outperforms baseline methods significantly and produces a satisfactory accuracy for clinical use.
Graph Laplacians on Shared Nearest Neighbor graphs and graph Laplacians on $k$-Nearest Neighbor graphs having the same limit

A Shared Nearest Neighbor (SNN) graph is a type of graph construction using shared nearest neighbor information, which is a secondary similarity measure based on the rankings induced by a primary $k$-nearest neighbor ($k$-NN) measure. SNN measures have been touted as being less prone to the curse of dimensionality than conventional distance measures, and thus methods using SNN graphs have been widely used in applications, particularly in clustering high-dimensional data sets and in finding outliers in subspaces of high dimensional data. Despite this, the theoretical study of SNN graphs and graph Laplacians remains unexplored. In this pioneering work, we make the first contribution in this direction. We show that large scale asymptotics of an SNN graph Laplacian reach a consistent continuum limit; this limit is the same as that of a $k$-NN graph Laplacian. Moreover, we show that the pointwise convergence rate of the graph Laplacian is linear with respect to $(k/n)^{1/m}$ with high probability.
Dynamic Graph Convolutional Network with Attention Fusion for Traffic Flow Prediction

Accurate and real-time traffic state prediction is of great practical importance for urban traffic control and web mapping services. With the support of massive data, deep learning methods have shown their powerful capability in capturing the complex spatialtemporal patterns of traffic networks. However, existing approaches use pre-defined graphs and a simple set of spatial-temporal components, making it difficult to model multi-scale spatial-temporal dependencies. In this paper, we propose a novel dynamic graph convolution network with attention fusion to tackle this gap. The method first enhances the interaction of temporal feature dimensions, and then it combines a dynamic graph learner with GRU to jointly model synchronous spatial-temporal correlations. We also incorporate spatial-temporal attention modules to effectively capture longrange, multifaceted domain spatial-temporal patterns. We conduct extensive experiments in four real-world traffic datasets to demonstrate that our method surpasses state-of-the-art performance compared to 18 baseline methods.
Linearization Algorithms for Fully Composite Optimization

This paper studies first-order algorithms for solving fully composite optimization problems over convex and compact sets. We leverage the structure of the objective by handling its differentiable and non-differentiable components separately, linearizing only the smooth parts. This provides us with new generalizations of the classical Frank-Wolfe method and the Conditional Gradient Sliding algorithm, that cater to a subclass of non-differentiable problems. Our algorithms rely on a stronger version of the linear minimization oracle, which can be efficiently implemented in several practical applications. We provide the basic version of our method with an affine-invariant analysis and prove global convergence rates for both convex and non-convex objectives. Furthermore, in the convex case, we propose an accelerated method with correspondingly improved complexity. Finally, we provide illustrative experiments to support our theoretical results.
Cox reduction and confidence sets of models: a theoretical elucidation

For sparse high-dimensional regression problems, Cox and Battey [1, 9] emphasised the need for confidence sets of models: an enumeration of those small sets of variables that fit the data equivalently well in a suitable statistical sense. This is to be contrasted with the single model returned by penalised regression procedures, effective for prediction but potentially misleading for subject-matter understanding. The proposed construction of such sets relied on preliminary reduction of the full set of variables, and while various possibilities could be considered for this, [9] proposed a succession of regression fits based on incomplete block designs. The purpose of the present paper is to provide insight on both aspects of that work. For an unspecified reduction strategy, we begin by characterising models that are likely to be retained in the model confidence set, emphasising geometric aspects. We then evaluate possible reduction schemes based on penalised regression or marginal screening, before theoretically elucidating the reduction of [9]. We identify features of the covariate matrix that may reduce its efficacy, and indicate improvements to the original proposal. An advantage of the approach is its ability to reveal its own stability or fragility for the data at hand.
In What Languages are Generative Language Models the Most Formal? Analyzing Formality Distribution across Languages

Multilingual generative language models (LMs) are increasingly fluent in a large variety of languages. Trained on the concatenation of corpora in multiple languages, they enable powerful transfer from high-resource languages to low-resource ones. However, it is still unknown what cultural biases are induced in the predictions of these models. In this work, we focus on one language property highly influenced by culture: formality. We analyze the formality distributions of XGLM and BLOOM's predictions, two popular generative multilingual language models, in 5 languages. We classify 1,200 generations per language as formal, informal, or incohesive and measure the impact of the prompt formality on the predictions. Overall, we observe a diversity of behaviors across the models and languages. For instance, XGLM generates informal text in Arabic and Bengali when conditioned with informal prompts, much more than BLOOM. In addition, even though both models are highly biased toward the formal style when prompted neutrally, we find that the models generate a significant amount of informal predictions even when prompted with formal text. We release with this work 6,000 annotated samples, paving the way for future work on the formality of generative multilingual LMs.
Control of multi-agent systems: Results, open problems, and applications

Abstract The purpose of this review article is to present some recent results on the modeling and control of large systems of agents. We focus on particular applications where the agents are capable of independent actions instead of simply reacting to external forces. In the literature, such agents were referred to as autonomous, intelligent, self-propelled, greedy, and others. The main applications we have in mind are social systems (as opinion dynamics), pedestrians’ movements (also called crowd dynamics), animal groups, and vehicular traffic. We note that the last three examples include physical constraints; however, the agents are able to inject energy into the system, thus preventing the typical conservation of momentum and energy. In addition, the control problems posed by such systems are new and require innovative methods. We illustrate some ideas developed recently, including the use of sparse controls, limiting the total variation of controls, and defining new control problems for measures. After reviewing various approaches, we discuss some future research directions of potential interest. The latter encompasses both new types of equations and new types of limiting procedures to connect several scales at which a system can be represented. We conclude by illustrating a recent real-life experiment using autonomous vehicles on an open highway to smooth traffic waves. This opens the door to a new era of interventions to control real-time multi-agent systems and to increase the societal impact of such interventions guided by control research.
Type-II magnetic domain contrasts caused by internal-magnetisation induced electron scattering anisotropy

Previous studies have shown that type-II magnetic domain contrasts are caused by differences in the backscattering yields of magnetic domains of opposite magnetisation. Imaging the magnetic domains when the magnetisation vectors in the opposite-magnetisation domains are perpendicular to the tilt axis of the specimen has been considered impossible, because of the lack of change in the backscattering yields between the domains. However, we found that it is possible to observe the type-II magnetic domain contrast even in cases where the backscattering coefficients of the domains are the same. The contrast arises from the electron-scattering anisotropy created because of the deflection of the electron trajectories by the Lorentz force. We verify this by distinguishing all four possible in-plane magnetisation vectors on a Fe-Si(100) surface, using an electron backscatter diffraction detector as an arrya of electron detectors. The change in contrast between the magnetic domains, with respect to the location of a virtual electron detector, can provide information on the directions of the magnetisation vectors. A method to suppress the topographic contrast superimposed on the magnetic domain contrast is also demonstrated.
Characterize the non-Gaussian diffusion property of cerebrospinal fluid using Diffusion Kurtosis Imaging and explore its diagnostic efficacy for Alzheimer's disease

Differentiating Alzheimer's disease (AD) patients from healthy controls (HCs) remains a challenge. The changes of protein level in cerebrospinal fluid (CSF) of AD patients have been reported in the literature. Macromolecules will hinder the movement of water in CSF and lead to non-Gaussian diffusion. Diffusion kurtosis imaging (DKI) is a commonly used technique for quantifying non-Gaussian diffusivity. In this study, we used DKI to evaluate the non-Gaussian diffusion of CSF in AD patients and HC. Between-group difference was explored. In addition, we have built a prediction model using cross-validation Support Vector Machines (SVM), and achieved excellent performance. The validated area under the receiver operating characteristic curve(AUC) is in the range of 0.96-1.00, and the correct prediction is in the range of 87.1% - 90.0%.