Miracle-dz/UniFAR-FTUs huggingface dataset:
Evaluating LLMs for Gender Disparities in Notable Persons

This study examines the use of Large Language Models (LLMs) for retrieving factual information, addressing concerns over their propensity to produce factually incorrect"hallucinated"responses or to altogether decline to even answer prompt at all. Specifically, it investigates the presence of gender-based biases in LLMs' responses to factual inquiries. This paper takes a multi-pronged approach to evaluating GPT models by evaluating fairness across multiple dimensions of recall, hallucinations and declinations. Our findings reveal discernible gender disparities in the responses generated by GPT-3.5. While advancements in GPT-4 have led to improvements in performance, they have not fully eradicated these gender disparities, notably in instances where responses are declined. The study further explores the origins of these disparities by examining the influence of gender associations in prompts and the homogeneity in the responses.
Frequency-dependent entanglement advantage in spin-network Quantum Reservoir Computing

We study the performance of an Ising spin network for quantum reservoir computing (QRC) in linear and non-linear memory tasks. We investigate the extent to which quantumness enhances performance by monitoring the behaviour of quantum entanglement, which we quantify by means of the partial transpose of the density matrix. In the most general case where the effects of dissipation are incorporated, our results indicate that the strength of the entanglement advantage depends on the frequency of the input signal; the benefit of entanglement is greater the more rapidly fluctuating the signal, whereas a low-frequency input lends itself better to a non-entangled reservoir. This suggests that the extent to which quantumness is beneficial is dependent on the difficulty of the memory task.
CodeUltraFeedback: An LLM-as-a-Judge Dataset for Aligning Large Language Models to Coding Preferences

Evaluating the alignment of large language models (LLMs) with user-defined coding preferences is a challenging endeavour that requires assessing intricate textual LLMs' outputs. By relying on automated metrics and static analysis tools, existing benchmarks fail to assess nuances in user instructions and LLM outputs, highlighting the need for large-scale datasets and benchmarks for LLM preference alignment. In this paper, we introduce CodeUltraFeedback, a preference dataset of 10,000 complex instructions to tune and align LLMs to coding preferences through AI feedback. We generate responses to the instructions using a pool of 14 diverse LLMs, which we then annotate according to their alignment with five coding preferences using the LLM-as-a-Judge approach with GPT-3.5, producing both numerical and textual feedback. We also present CODAL-Bench, a benchmark for assessing LLM alignment with these coding preferences. Our results show that CodeLlama-7B-Instruct, aligned through reinforcement learning from AI feedback (RLAIF) with direct preference optimization (DPO) using CodeUltraFeedback's AI feedback data, outperforms 34B LLMs on CODAL-Bench, validating the utility of CodeUltraFeedback for preference tuning. Furthermore, we show our DPO-aligned CodeLlama model improves functional correctness on HumanEval+ compared to the unaligned base model. Therefore, our contributions bridge the gap in preference tuning of LLMs for code and set the stage for further advancements in model alignment and RLAIF for code intelligence. Our code and data are available at https://github.com/martin-wey/CodeUltraFeedback.
Benchmarking Distributed Coordination Systems: A Survey and Analysis

Coordination services and protocols are critical components of distributed systems and are essential for providing consistency, fault tolerance, and scalability. However, due to lack of a standard benchmarking tool for distributed coordination services, coordination service developers/researchers either use a NoSQL standard benchmark and omit evaluating consistency, distribution, and fault-tolerance; or create their own ad-hoc microbenchmarks and skip comparability with other services. In this paper, we analyze and compare known and widely used distributed coordination services, their evaluations, and the tools used to benchmark those systems. We identify important requirements of distributed coordination service benchmarking, like the metrics and parameters that need to be evaluated and their evaluation setups and tools.
Bifurcated Attention for Single-Context Large-Batch Sampling

In our study, we present bifurcated attention, a method developed for language model inference in single-context batch sampling contexts. This approach aims to reduce redundant memory IO costs, a significant factor in latency for high batch sizes and long context lengths. Bifurcated attention achieves this by dividing the attention mechanism during incremental decoding into two distinct GEMM operations, focusing on the KV cache from prefill and the decoding process. This method ensures precise computation and maintains the usual computational load (FLOPs) of standard attention mechanisms, but with reduced memory IO. Bifurcated attention is also compatible with multi-query attention mechanism known for reduced memory IO for KV cache, further enabling higher batch size and context length. The resulting efficiency leads to lower latency, improving suitability for real-time applications, e.g., enabling massively-parallel answer generation without substantially increasing latency, enhancing performance when integrated with postprocessing techniques such as reranking.
How do Machine Learning Projects use Continuous Integration Practices? An Empirical Study on GitHub Actions

Continuous Integration (CI) is a well-established practice in traditional software development, but its nuances in the domain of Machine Learning (ML) projects remain relatively unexplored. Given the distinctive nature of ML development, understanding how CI practices are adopted in this context is crucial for tailoring effective approaches. In this study, we conduct a comprehensive analysis of 185 open-source projects on GitHub (93 ML and 92 non-ML projects). Our investigation comprises both quantitative and qualitative dimensions, aiming to uncover differences in CI adoption between ML and non-ML projects. Our findings indicate that ML projects often require longer build durations, and medium-sized ML projects exhibit lower test coverage compared to non-ML projects. Moreover, small and medium-sized ML projects show a higher prevalence of increasing build duration trends compared to their non-ML counterparts. Additionally, our qualitative analysis illuminates the discussions around CI in both ML and non-ML projects, encompassing themes like CI Build Execution and Status, CI Testing, and CI Infrastructure. These insights shed light on the unique challenges faced by ML projects in adopting CI practices effectively.
Recursive Causal Discovery

Causal discovery, i.e., learning the causal graph from data, is often the first step toward the identification and estimation of causal effects, a key requirement in numerous scientific domains. Causal discovery is hampered by two main challenges: limited data results in errors in statistical testing and the computational complexity of the learning task is daunting. This paper builds upon and extends four of our prior publications (Mokhtarian et al., 2021; Akbari et al., 2021; Mokhtarian et al., 2022, 2023a). These works introduced the concept of removable variables, which are the only variables that can be removed recursively for the purpose of causal discovery. Presence and identification of removable variables allow recursive approaches for causal discovery, a promising solution that helps to address the aforementioned challenges by reducing the problem size successively. This reduction not only minimizes conditioning sets in each conditional independence (CI) test, leading to fewer errors but also significantly decreases the number of required CI tests. The worst-case performances of these methods nearly match the lower bound. In this paper, we present a unified framework for the proposed algorithms, refined with additional details and enhancements for a coherent presentation. A comprehensive literature review is also included, comparing the computational complexity of our methods with existing approaches, showcasing their state-of-the-art efficiency. Another contribution of this paper is the release of RCD, a Python package that efficiently implements these algorithms. This package is designed for practitioners and researchers interested in applying these methods in practical scenarios. The package is available at github.com/ban-epfl/rcd, with comprehensive documentation provided at rcdpackage.com.
Online and Offline Evaluation in Search Clarification

The effectiveness of clarification question models in engaging users within search systems is currently constrained, casting doubt on their overall usefulness. To improve the performance of these models, it is crucial to employ assessment approaches that encompass both real-time feedback from users (online evaluation) and the characteristics of clarification questions evaluated through human assessment (offline evaluation). However, the relationship between online and offline evaluations has been debated in information retrieval. This study aims to investigate how this discordance holds in search clarification. We use user engagement as ground truth and employ several offline labels to investigate to what extent the offline ranked lists of clarification resemble the ideal ranked lists based on online user engagement.
Existence and concentration of normalized solutions for $p$-Laplacian equations with logarithmic nonlinearity

We investigate the existence and concentration of normalized solutions for a $p$-Laplacian problem with logarithmic nonlinearity of type \[ \left\{ \begin{array}{ll} \displaystyle -\varepsilon^p\Delta_p u+V(x)|u|^{p-2}u=\lambda |u|^{p-2}u+|u|^{p-2}u\log|u|^p ~\text{in}~\mathbb R^N,\newline \displaystyle \int_{\mathbb R^N}|u|^pdx=a^p\varepsilon^N, \end{array} \right. \] where $a,\varepsilon>0$, $\lambda\in\mathbb R$ is known as the Lagrange multiplier, $\Delta_p\cdot =\text{div} (|\nabla \cdot|^{p-2}\nabla \cdot)$ denotes the usual $p$-Laplacian operator with $2\leq p<N$ and $V \in \mathcal{C}^0(\mathbb R^N)$ is the potential which satisfies some suitable assumptions. We prove that the number of positive solutions depends on the profile of $V$ and each solution concentrates around its corresponding global minimum point of $V$ in the semiclassical limit when $\varepsilon\to0^+$ using variational method. Moreover, we also get the existence of normalized solutions for some logarithmic $p$-Laplacian equations involving mass-supercritical nonlinearities.
Efficiently Computing Similarities to Private Datasets

Many methods in differentially private model training rely on computing the similarity between a query point (such as public or synthetic data) and private data. We abstract out this common subroutine and study the following fundamental algorithmic problem: Given a similarity function $f$ and a large high-dimensional private dataset $X \subset \mathbb{R}^d$, output a differentially private (DP) data structure which approximates $\sum_{x \in X} f(x,y)$ for any query $y$. We consider the cases where $f$ is a kernel function, such as $f(x,y) = e^{-\|x-y\|_2^2/\sigma^2}$ (also known as DP kernel density estimation), or a distance function such as $f(x,y) = \|x-y\|_2$, among others. Our theoretical results improve upon prior work and give better privacy-utility trade-offs as well as faster query times for a wide range of kernels and distance functions. The unifying approach behind our results is leveraging `low-dimensional structures' present in the specific functions $f$ that we study, using tools such as provable dimensionality reduction, approximation theory, and one-dimensional decomposition of the functions. Our algorithms empirically exhibit improved query times and accuracy over prior state of the art. We also present an application to DP classification. Our experiments demonstrate that the simple methodology of classifying based on average similarity is orders of magnitude faster than prior DP-SGD based approaches for comparable accuracy.
Quantum Many-Body Scars for Arbitrary Integer Spin in $2+1$D Abelian Gauge Theories

The existence of Quantum Many-Body Scars has been established across different quantum many-body systems, including spin-1/2 Quantum Link Models. We systematically identify scars for pure gauge theories with arbitrarily large integer spin $S$ in $2+1$D, concretely for Truncated Link Models, where the electric field is truncated rather than replaced by spins. Through an explicit analytic construction, we show that the presence of scars is widespread. We confirm these findings numerically for small truncated spin and $S=1$ Quantum Link Models. The proposed analytic construction establishes the presence of scars far beyond volumes and spins that can be probed with existing numerical methods.
Whittle Index Based User Association in Dense Millimeter Wave Networks

We address the problem of user association in a dense millimeter wave (mmWave) network, in which each arriving user brings a file containing a random number of packets and each time slot is divided into multiple mini-slots. This problem is an instance of the restless multi-armed bandit problem, and is provably hard to solve. Using a technique introduced by Whittle, we relax the hard per-stage constraint that each arriving user must be associated with exactly one mmWave base station (mBS) to a long-term constraint and then use the Lagrangian multiplier technique to convert the problem into an unconstrained problem. This decouples the process governing the system into separate Markov Decision Processes at different mBSs. We prove that the problem is Whittle indexable, present a scheme for computing the Whittle indices of different mBSs, and propose an association scheme under which, each arriving user is associated with the mBS with the smallest value of the Whittle index. Using extensive simulations, we show that the proposed Whittle index based scheme outperforms several user association schemes proposed in prior work in terms of various performance metrics such as average cost, delay, throughput, and Jain's fairness index.
Video Editing via Factorized Diffusion Distillation

We introduce Emu Video Edit (EVE), a model that establishes a new state-of-the art in video editing without relying on any supervised video editing data. To develop EVE we separately train an image editing adapter and a video generation adapter, and attach both to the same text-to-image model. Then, to align the adapters towards video editing we introduce a new unsupervised distillation procedure, Factorized Diffusion Distillation. This procedure distills knowledge from one or more teachers simultaneously, without any supervised data. We utilize this procedure to teach EVE to edit videos by jointly distilling knowledge to (i) precisely edit each individual frame from the image editing adapter, and (ii) ensure temporal consistency among the edited frames using the video generation adapter. Finally, to demonstrate the potential of our approach in unlocking other capabilities, we align additional combinations of adapters
On the Metallicity Gradients in the Galactic Disk using Open Clusters

We study the metallicity distribution and evolution in the Galactic disk based on the largest sample of open star clusters in the Galaxy. From the catalogue of 1879 open clusters in the range of Galactocentric distance (R_GC) from 4 to 20 kpc, we investigate the variation of metallicity in the Galactic disk as functions of R_GC, vertical distance (Z), and ages of the clusters. In the direction perpendicular to the Galactic plane, variation in metallicity is found to follow a stepped linear relation. We estimate a vertical metallicity gradient d[Fe/H]/dZ of -0.545+/-0.046 dex/kpc for |Z|<0.487 kpc, and -0.075+/-0.093 dex/kpc for 0.487<|Z|<1.8 kpc. On average, metallicity variations above and below the Galactic plane are found to change at similar rates. The change in metallicity in the radial direction is also found to follow a two-function linear relation. We obtain a radial metallicity gradient d[Fe/H]/d[R_GC] of -0.070+/-0.002 dex/kpc for 4.0<R_GC<12.8 kpc, and -0.005+/-0.018 dex/kpc for 12.8<R_GC<20.5 kpc which clearly shows a strong variation in the metallicity gradient when moving from the inner to the outer Galactic disk. Age-metallicity relation (AMR) is found to follow a steeper negative slope of -0.031+/-0.006 dex/Gyr for clusters older than 240 Myr, however, there is some hint of positive metallicity age gradient for younger clusters.
Geology and Fluid Discharge at Dorado Outcrop, a Low Temperature Ridge‐Flank Hydrothermal System

Two expeditions to Dorado Outcrop on the eastern flank of the East Pacific Rise and west of the Middle America Trench collected images, video, rocks, and sediment samples and measured temperature and fluid discharge rates to document the physical and biogeochemical characteristics of a regional, low‐temperature (~15 °C) hydrothermal system. Analysis of video and images identified lava morphologies: pillow, lobate, and sheet flows. Glasses from collected lavas were consistent with an off‐axis formation. Hydrothermal discharge generally occurs through pillow lavas but is patchy, sporadic, and sometimes ceases at particular sites of discharge. Yearlong temperature measurements at five of these discharge sites show daily ranges that oscillate with tidal frequencies by 6 °C or more. Instantaneous fluid discharge rates (0.16 to 0.19 L/s) were determined resulting in a calculated discharge of ~200 L/s when integrated over the area defined by the most robust fluid discharge. Such discharge has a power output of 10–12 MW. Hydrothermal seepage through thin sediment adjacent to the outcrop accounts for <3% of this discharge, but seepage may support an oxic sediment column. High extractable Mn concentrations and depleted δ13C in the low but variable organic solid phase suggest that hydrothermal fluids provide a source for manganese accumulation and likely enhance the oxidation of organic carbon. Comparisons of the physical and geochemical characteristics at Dorado and Baby Bare Outcrops, the latter being the only other site of ridge‐flank hydrothermal discharge that has been sampled directly, suggest commonalities and differences that have implications for future discoveries.
SkateFormer: Skeletal-Temporal Transformer for Human Action Recognition

Skeleton-based action recognition, which classifies human actions based on the coordinates of joints and their connectivity within skeleton data, is widely utilized in various scenarios. While Graph Convolutional Networks (GCNs) have been proposed for skeleton data represented as graphs, they suffer from limited receptive fields constrained by joint connectivity. To address this limitation, recent advancements have introduced transformer-based methods. However, capturing correlations between all joints in all frames requires substantial memory resources. To alleviate this, we propose a novel approach called Skeletal-Temporal Transformer (SkateFormer) that partitions joints and frames based on different types of skeletal-temporal relation (Skate-Type) and performs skeletal-temporal self-attention (Skate-MSA) within each partition. We categorize the key skeletal-temporal relations for action recognition into a total of four distinct types. These types combine (i) two skeletal relation types based on physically neighboring and distant joints, and (ii) two temporal relation types based on neighboring and distant frames. Through this partition-specific attention strategy, our SkateFormer can selectively focus on key joints and frames crucial for action recognition in an action-adaptive manner with efficient computation. Extensive experiments on various benchmark datasets validate that our SkateFormer outperforms recent state-of-the-art methods.
AI on AI: Exploring the Utility of GPT as an Expert Annotator of AI Publications

Identifying scientific publications that are within a dynamic field of research often requires costly annotation by subject-matter experts. Resources like widely-accepted classification criteria or field taxonomies are unavailable for a domain like artificial intelligence (AI), which spans emerging topics and technologies. We address these challenges by inferring a functional definition of AI research from existing expert labels, and then evaluating state-of-the-art chatbot models on the task of expert data annotation. Using the arXiv publication database as ground-truth, we experiment with prompt engineering for GPT chatbot models to identify an alternative, automated expert annotation pipeline that assigns AI labels with 94% accuracy. For comparison, we fine-tune SPECTER, a transformer language model pre-trained on scientific publications, that achieves 96% accuracy (only 2% higher than GPT) on classifying AI publications. Our results indicate that with effective prompt engineering, chatbots can be used as reliable data annotators even where subject-area expertise is required. To evaluate the utility of chatbot-annotated datasets on downstream classification tasks, we train a new classifier on GPT-labeled data and compare its performance to the arXiv-trained model. The classifier trained on GPT-labeled data outperforms the arXiv-trained model by nine percentage points, achieving 82% accuracy.
Sensitivity to sub-GeV dark matter from cosmic-ray scattering with very-high-energy gamma-ray observatories

Huge efforts have been deployed to detect dark matter (DM) in the GeV-TeV mass range involving various detection techniques, and led to strong constraints in the available parameter space. We compute here the sensitivity to sub-GeV DM that can be probed from the inevitable cosmic-ray scattering onto DM particles populating the Milky Way halo. Inelastic scattering of energetic cosmic rays off DM would produce high-energy gamma rays in the final state, providing a new avenue to probe the poorly-constrained so far sub-GeV dark matter mass range. In this work we derive sensitivity forecasts for the inelastic cosmic-ray proton - DM cross section for current and future very-high-energy gamma-ray observatories such as H.E. S.S. , LHAASO, CTA and SWGO in the 100 eV to 100 MeV mass range. These inelastic cross section constraints are converted to the elastic proton - DM cross section to highlight further complementarity with cosmological, collider and direct detection searches. The sensitivity computed at 95\% confidence level on the elastic cross section reaches $\sim$2$\times$ 10$^{-32}$ cm$^2$ for a 100 keV DM mass for H.E. S.S. -like and $\sim$7$\times$ 10$^{-34}$ cm$^2$ for a $\sim$1 keV DM mass for LHAASO. The sensitivity prospects for CTA and a strawman SWGO model reach $\sim$6$\times$ 10$^{-34}$ cm$^2$ and $\sim$4$\times$ 10$^{-35}$ cm$^2$, for DM masses of 10 keV and 1 keV, respectively. The sensitivity reach of the gamma-ray observatories considered here enables to probe an uncharted region of the DM mass - cross section parameter space.
Optimization of Vegetative Filter Strips for Mitigation of Runoff from Golf Course Turf

Joint greenhouse and field studies have been implemented to evaluate selected plants for their effectiveness in removing pesticides and nutrients from turfgrass runoff waters that enter vegetative filter strips (VFS). A greenhouse pot study determined five species (big blue stem, blue flag iris, eastern gama grass, prairie cord grass, and woolgrass) most effectively removed the six selected pesticides (2 fungicides, 2 herbicides, and 2 insecticides) from a silt loam soil.