Miracle-dz/UniFAR-FTUs huggingface dataset:
Nearest Neighbor Zero-Shot Inference

Retrieval-augmented language models (LMs) use non-parametric memory to substantially outperform their non-retrieval counterparts on perplexity-based evaluations, but it is an open question whether they achieve similar gains in few- and zero-shot end-task accuracy. We extensively study one such model, the k-nearest neighbor LM (kNN-LM), showing that the gains marginally transfer. The main challenge is to achieve coverage of the verbalizer tokens that define the different end-task class labels. To address this challenge, we also introduce kNN-Prompt, a simple and effective kNN-LM with automatically expanded fuzzy verbalizers (e.g. to expand “terrible” to also include “silly” and other task-specific synonyms for sentiment classification). Across nine diverse end-tasks, using kNN-Prompt with GPT-2 large yields significant performance boosts over strong zeroshot baselines (13.4% absolute improvement over the base LM on average). We also show that other advantages of non-parametric augmentation hold for end tasks; kNN-Prompt is effective for domain adaptation with no further training, and gains increase with the size of the retrieval model.
Numerical study of floating wind turbines : hydro- and aero-mechanics

Floating wind technology has the potential to produce low-carbon electricity on a large scale: it allows the expansion of offshore wind harvesting to deep water, indicatively from 50–60 to a few hundred metres depth, where most of the worldwide technical resource is found. New design specifications are being developed for floating wind in order to meet diverse criteria such as conversion efficiency, maintainability, buoyancy stability, and structural reliability. The last is the focus of this work. The mechanics of floating wind turbines in wind and waves are investigated with an array of numerical means. They demand the simulation of multiple processes such as aerodynamics, hydrodynamics, rotor and structural dynamics; understanding their interaction is essential for engineering design, verification, and concept evaluation. The project is organised in three main parts, presented below. Aero-hydro-mechanical simulation, characterising the rigid-body motions of a floating wind turbine. An investigation of multi-physical couplings is carried out, mainly through EDF R&D’s time-domain simulator CALHYPSO. Wave forces are represented with the potential-flow panel method and the Morison equation. Aerodynamic forces are represented by a thrust model or with the blade element momentum theory. Main findings: Exposure of finite-angle coupling for semi-submersible turbines with focus on heave plate excursion; characterisation of the aerodynamic damping of pitch motion provided by an operating vertical-axis turbine. Dynamic mooring simulation, focussed on highly compliant mooring systems, where the fluid-structure interaction and mechanical inertial forces can govern line tension. EDF R&D’s general-purpose, finite-element solver Code Aster is configured for this use exploiting its nonlinear large-displacement and contact mechanics functionalities. Main findings: Demonstration of a Code Aster -based workflow for the analysis of catenary mooring systems; explanation of the dynamic mooring effects previously observed in the DeepCwind basin test campaign. Aeroelastic analysis of vertical-axis rotors, aimed at verifying novel large-scale floating wind turbine concepts in operation, when aeroelastic-rotordynamic instabilities may occur. The finite-element modal approach is used to qualify rotor vibrations and to estimate the associated damping, based on the spinning beam formulation and a linearised aerodynamic operator. Main findings: Characterisation of the vibration modes of two novel vertical-axis rotor concepts using the Campbell diagram; estimation of the related aerodynamic damping, providing information on the aeroelastic stability of these designs.
A Hybrid Neural Autoencoder for Sensory Neuroprostheses and Its Applications in Bionic Vision

Sensory neuroprostheses are emerging as a promising technology to restore lost sensory function or augment human capacities. However, sensations elicited by current devices often appear artificial and distorted. Although current models can often predict the neural or perceptual response to an electrical stimulus, an optimal stimulation strategy solves the inverse problem: what is the required stimulus to produce a desired response? Here we frame this as an end-to-end optimization problem, where a deep neural network encoder is trained to invert a known, fixed forward model that approximates the underlying biological system. As a proof of concept, we demonstrate the effectiveness of our hybrid neural autoencoder (HNA) on the use case of visual neuroprostheses. We found that HNA is able to produce high-fidelity stimuli from the MNIST and COCO datasets that outperform conventional encoding strategies and surrogate techniques across all tested conditions. Overall this is an important step towards the long-standing challenge of restoring high-quality vision to people living with incurable blindness and may prove a promising solution for a variety of neuroprosthetic technologies.
Training and Inference on Any-Order Autoregressive Models the Right Way

Conditional inference on arbitrary subsets of variables is a core problem in probabilistic inference with important applications such as masked language modeling and image inpainting. In recent years, the family of Any-Order Autoregressive Models (AO-ARMs) -- closely related to popular models such as BERT and XLNet -- has shown breakthrough performance in arbitrary conditional tasks across a sweeping range of domains. But, in spite of their success, in this paper we identify significant improvements to be made to previous formulations of AO-ARMs. First, we show that AO-ARMs suffer from redundancy in their probabilistic model, i.e., they define the same distribution in multiple different ways. We alleviate this redundancy by training on a smaller set of univariate conditionals that still maintains support for efficient arbitrary conditional inference. Second, we upweight the training loss for univariate conditionals that are evaluated more frequently during inference. Our method leads to improved performance with no compromises on tractability, giving state-of-the-art likelihoods in arbitrary conditional modeling on text (Text8), image (CIFAR10, ImageNet32), and continuous tabular data domains.
CEBaB: Estimating the Causal Effects of Real-World Concepts on NLP Model Behavior

The increasing size and complexity of modern ML systems has improved their predictive capabilities but made their behavior harder to explain. Many techniques for model explanation have been developed in response, but we lack clear criteria for assessing these techniques. In this paper, we cast model explanation as the causal inference problem of estimating causal effects of real-world concepts on the output behavior of ML models given actual input data. We introduce CEBaB, a new benchmark dataset for assessing concept-based explanation methods in Natural Language Processing (NLP). CEBaB consists of short restaurant reviews with human-generated counterfactual reviews in which an aspect (food, noise, ambiance, service) of the dining experience was modified. Original and counterfactual reviews are annotated with multiply-validated sentiment ratings at the aspect-level and review-level. The rich structure of CEBaB allows us to go beyond input features to study the effects of abstract, real-world concepts on model behavior. We use CEBaB to compare the quality of a range of concept-based explanation methods covering different assumptions and conceptions of the problem, and we seek to establish natural metrics for comparative assessments of these methods.
The Effect of Age on the Grouping of Open Clusters: II. Are There Old Binary Clusters?

In the present study, we continue testing the Primordial Group hypothesis (Casado 2022), which postulates that only sufficiently young open clusters can be binary or multiple, and old clusters are essentially single. To this end, we revisit all the remaining binary cluster candidates in the Galaxy having at least one cluster older than 100 Myr through Gaia data and careful revision of the literature. We found no convincing case for an old binary system among the 120 pairs/groups revised. Most of the pairs are optical pairs or flyby encounters. However, we found three dubious pairs that could falsify the title hypothesis upon further research. We also found two possible primordial pairs older than expected. Our results confirm that the vast majority of binary/multiple OCs in the Galaxy, if not all, are of primordial origin and are not stable for a long time. This finding is in line with similar studies of the Magellanic Clouds and theoretical N-body simulations in the Galaxy. The pairs of OCs in these groups are generally not binary systems since they are not gravitationally bound. We also point out some inconsistencies in previous works and databases, such as false open clusters and duplicities.
Spontaneous breaking of finite group symmetries at all temperatures

We study conformal field theories with finite group symmetries with spontaneous symmetry breaking (SSB) phases which persist at all temperatures. We work with two \lambda \phi^4λϕ4 theories coupled through their mass terms. The two \lambda \phi^4λϕ4 theories are chosen to preserve either the cubic symmetry group, or the tetrahedral symmetry group. A one-loop calculation in the 4-\epsilon4−ϵ expansion shows that there exist infinitely many fixed points that can host an all temperature SSB phase. An analysis of the renormalization group (RG) stability matrix of these fixed points, reveals that their spectrum contains at least three relevant operators. In other words, these fixed points are tetracritical points.
Self-Admitted Technical Debt in the Embedded Systems Industry: An Exploratory Case Study

Technical debt denotes shortcuts taken during software development, mostly for the sake of expedience. When such shortcuts are admitted explicitly by developers (e.g., writing a TODO/Fixme comment), they are termed as Self-Admitted Technical Debt or SATD. There has been a fair amount of work studying SATD management in Open Source projects, but SATD in industry is relatively unexplored. At the same time, there is no work focusing on developers’ perspectives towards SATD and its management. To address this, we conducted an exploratory case study in cooperation with an industrial partner to study how they think of SATD and how they manage it. Specifically, we collected data by identifying and characterizing SATD in different sources (issues, source code comments, and commits) and carried out a series of interviews with 12 software practitioners. The results show: 1) the core characteristics of SATD in industrial projects; 2) developers’ attitudes towards identified SATD and statistics; 3) triggers for practitioners to introduce and repay SATD; 4) relations between SATD in different sources; 5) practices used to manage SATD; 6) challenges and tooling ideas for SATD management.
Pass Evaluation in Women's Olympic Hockey

Passing during power plays in hockey is a crucial component to move one's team closer to scoring a goal. With the use of women's ice hockey event and tracking data from the elimination round games during the 2022 Winter Olympics, we evaluate passing and assess players' risk-reward behaviours in these high intensity moments. We develop a model for probabilistic passing that accounts for the order of arrival to a desired location and potential interceptions along the way. This model is based on a player-specific motion model and a puck motion model that determines how far each player can reach in the time it takes the puck to get to a target. In addition, we model the rink control for each team and the scoring probability of the offensive team. These models are then combined into novel metrics for quantifying where a pass should be made such that it would result in a high scoring opportunity or result in a high chance of maintaining possession. Finally, we use various metrics to evaluate passes made throughout the available power plays and compare them to the optimal options at that time. This can be used to identify players' risk-reward tendencies and can be used by coaches when selecting which players are best suited for a power play given the circumstances of the game.
Topological Hidden Markov Models

The hidden Markov model (HMM) is a classic modeling tool with a wide swath of applications. Its inception considered observations restricted to a finite alphabet, but it was quickly extended to multivariate continuous distributions. In this article, we further extend the HMM from mixtures of normal distributions in $d$-dimensional Euclidean space to general Gaussian measure mixtures in locally convex topological spaces. The main innovation is the use of the Onsager-Machlup functional as a proxy for the probability density function in infinite dimensional spaces. This allows for choice of a Cameron-Martin space suitable for a given application. We demonstrate the versatility of this methodology by applying it to simulated diffusion processes such as Brownian and fractional Brownian sample paths as well as the Ornstein-Uhlenbeck process. Our methodology is applied to the identification of sleep states from overnight polysomnography time series data with the aim of diagnosing Obstructive Sleep Apnea in pediatric patients. It is also applied to a series of annual cumulative snowfall curves from 1940 to 1990 in the city of Edmonton, Alberta.
Motivic spectral Mackey functors

We show that if G is a ﬁnite constant group acting on a scheme X such that | G | ∈ O × X , then the G -equivariant motivic stable homotopy category of X is equivalent to the stabilization of the category of motivic G -spaces with ﬁnite étale transfers over X at the trivial representation sphere. Along the way we obtain several results of independent interest, among them: we construct and study norms in the motivic homotopy theory of stacks, and we extend the homotopy t -structure to DM-stacks and establish some favorable properties.
Circumventing Backdoor Defenses That Are Based on Latent Separability

Recent studies revealed that deep learning is susceptible to backdoor poisoning attacks. An adversary can embed a hidden backdoor into a model to manipulate its predictions by only modifying a few training data, without controlling the training process. Currently, a tangible signature has been widely observed across a diverse set of backdoor poisoning attacks -- models trained on a poisoned dataset tend to learn separable latent representations for poison and clean samples. This latent separation is so pervasive that a family of backdoor defenses directly take it as a default assumption (dubbed latent separability assumption), based on which to identify poison samples via cluster analysis in the latent space. An intriguing question consequently follows: is the latent separation unavoidable for backdoor poisoning attacks? This question is central to understanding whether the assumption of latent separability provides a reliable foundation for defending against backdoor poisoning attacks. In this paper, we design adaptive backdoor poisoning attacks to present counter-examples against this assumption. Our methods include two key components: (1) a set of trigger-planted samples correctly labeled to their semantic classes (other than the target class) that can regularize backdoor learning; (2) asymmetric trigger planting strategies that help to boost attack success rate (ASR) as well as to diversify latent representations of poison samples. Extensive experiments on benchmark datasets verify the effectiveness of our adaptive attacks in bypassing existing latent separation based backdoor defenses. Moreover, our attacks still maintain a high attack success rate with negligible clean accuracy drop. Our studies call for defense designers to take caution when leveraging latent separation as an assumption in their defenses.
Transformer for Partial Differential Equations' Operator Learning

Data-driven learning of partial differential equations' solution operators has recently emerged as a promising paradigm for approximating the underlying solutions. The solution operators are usually parameterized by deep learning models that are built upon problem-specific inductive biases. An example is a convolutional or a graph neural network that exploits the local grid structure where functions' values are sampled. The attention mechanism, on the other hand, provides a flexible way to implicitly exploit the patterns within inputs, and furthermore, relationship between arbitrary query locations and inputs. In this work, we present an attention-based framework for data-driven operator learning, which we term Operator Transformer (OFormer). Our framework is built upon self-attention, cross-attention, and a set of point-wise multilayer perceptrons (MLPs), and thus it makes few assumptions on the sampling pattern of the input function or query locations. We show that the proposed framework is competitive on standard benchmark problems and can flexibly be adapted to randomly sampled input.
TraClets: Harnessing the power of computer vision for trajectory classification

Due to the advent of new mobile devices and tracking sensors in recent years, huge amounts of data are being produced every day. Therefore, novel methodologies need to emerge that dive through this vast sea of information and generate insights and meaningful information. To this end, researchers have developed several trajectory classification algorithms over the years that are able to annotate tracking data. Similarly, in this research, a novel methodology is presented that exploits image representations of trajectories, called TraClets, in order to classify trajectories in an intuitive humans way, through computer vision techniques. Several real-world datasets are used to evaluate the proposed approach and compare its classification performance to other state-of-the-art trajectory classification algorithms. Experimental results demonstrate that TraClets achieves a classification performance that is comparable to, or in most cases, better than the state-of-the-art, acting as a universal, high-accuracy approach for trajectory classification.
Bayesian Robust Graph Contrastive Learning

Graph Neural Networks (GNNs) have been widely used to learn node representations and with outstanding performance on various tasks such as node classification. However, noise, which inevitably exists in real-world graph data, would considerably degrade the performance of GNNs as the noise is easily propagated via the graph structure. In this work, we propose a novel and robust method, Bayesian Robust Graph Contrastive Learning (BRGCL), which trains a GNN encoder to learn robust node representations. The BRGCL encoder is a completely unsupervised encoder. Two steps are iteratively executed at each epoch of training the BRGCL encoder: (1) estimating confident nodes and computing robust cluster prototypes of node representations through a novel Bayesian nonparametric method; (2) prototypical contrastive learning between the node representations and the robust cluster prototypes. Experiments on public and large-scale benchmarks demonstrate the superior performance of BRGCL and the robustness of the learned node representations. The code of BRGCL is available at \url{https://github.com/BRGCL-code/BRGCL-code}.
On exceptional sets of radial projections

We prove two new exceptional set estimates for radial projections in the plane. If $K \subset \mathbb{R}^{2}$ is a Borel set with $\dim_{\mathrm{H}} K>1$, then $$\dim_{\mathrm{H}} \{x \in \mathbb{R}^{2} \, \setminus \, K : \dim_{\mathrm{H}} \pi_{x}(K) \leq \sigma\} \leq \max\{1 + \sigma - \dim_{\mathrm{H}} K,0\}, \qquad \sigma \in [0,1). $$ If $K \subset \mathbb{R}^{2}$ is a Borel set with $\dim_{\mathrm{H}} K \leq 1$, then $$\dim_{\mathrm{H}} \{x \in \mathbb{R}^{2} \, \setminus \, K : \dim_{\mathrm{H}} \pi_{x}(K)<\dim_{\mathrm{H}} K\} \leq 1. $$ The finite field counterparts of both results above were recently proven by Lund, Thang, and Huong Thu. Our results resolve the planar cases of conjectures of Lund-Thang-Huong Thu, and Liu.
RIGID: Robust Linear Regression with Missing Data

We present a robust framework to perform linear regression with missing entries in the features. By considering an elliptical data distribution, and speciﬁcally a multivariate normal model, we are able to conditionally formulate a distribution for the missing entries and present a robust framework, which minimizes the worst case error caused by the uncertainty about the missing data. We show that the proposed formulation, which naturally takes into account the dependency between diﬀerent variables, ultimately reduces to a convex program, for which a customized and scalable solver can be delivered. In addition to a detailed analysis to deliver such solver, we also asymptoticly analyze the behavior of the proposed framework, and present technical discussions to estimate the required input parameters. We complement our analysis with experiments performed on synthetic, semi-synthetic, and real data, and show how the proposed formulation improves the prediction accuracy and robustness, and outperforms the competing techniques. Missing data is a common problem associated with many datasets in machine learning. With the signiﬁcant increase in using robust optimization techniques to train machine learning models, this paper presents a novel robust regression framework that operates by minimizing the uncertainty associated with missing data. The proposed approach allows training models with incomplete data, while minimizing the impact of uncertainty associated with the unavailable data. The ideas developed in this paper can be generalized beyond linear models and elliptical data distributions.
A study on buoyancy-driven flows: Using particle image velocimetry for validating the Fire Dynamics Simulator

The aim of this thesis is to design and carry out bench-scale laboratory experiments specifically designed for the validation of fire models, and to use the experimental data for a validation study of the Fire Dynamics Simulator (FDS). The focus of the experiments is on one of the key components of fire models, the modeling of buoyancy-driven flows. The experimental setup is simplified by neglecting pyrolysis and combustion and its objective is to achieve high precision and reproducibility. Therefore, an electrically heated block of copper is used as a heat source and particle image velocimetry (PIV) is applied for measuring the flow velocities. Two different setups are investigated: an undisturbed open buoyant plume above the heat source, and a buoyant spill plume emerging from a compartment opening. Depending on the setup, different characteristic values of the flow are evaluated. For the undisturbed buoyant plume setup, maximum velocities in the plume, plume widths and flow integrals are determined. Furthermore, a centerline analysis is carried out in order to localize the transition from laminar to turbulent flow as a function of the Grashof number. The obtained values lie in the range 4 x 108 < Gr < 2 x 109 and therefore agree well with previous studies. For the buoyant spill plume compartment setup, position and maximum velocity of the outflow as well as volume flows in front of the opening are compared. The heat flows out of the opening cannot be measured directly and are therefore estimated based on the available data. The validation study with FDS leads to mixed results. For the open plume setup, generally a good agreement is achieved. Provided that a sufficiently fine resolution is used, the important flow characteristics are reproduced in the simulations. With the spill plume setup, significant differences between experiments and simulations are observed. In order to analyze them in detail, modifications of the experiments are required. Therefore, in conclusion of the study, areas of potential improvements for future experiments and the suitability of PIV for this kind of experiment are discussed.