Miracle-dz/UniFAR-FTUs huggingface dataset:
Repetition Improves Language Model Embeddings

Recent approaches to improving the extraction of text embeddings from autoregressive large language models (LLMs) have largely focused on improvements to data, backbone pretrained language models, or improving task-differentiation via instructions. In this work, we address an architectural limitation of autoregressive models: token embeddings cannot contain information from tokens that appear later in the input. To address this limitation, we propose a simple approach,"echo embeddings,"in which we repeat the input twice in context and extract embeddings from the second occurrence. We show that echo embeddings of early tokens can encode information about later tokens, allowing us to maximally leverage high-quality LLMs for embeddings. On the MTEB leaderboard, echo embeddings improve over classical embeddings by over 9% zero-shot and by around 0.7% when fine-tuned. Echo embeddings with a Mistral-7B model achieve state-of-the-art compared to prior open source models that do not leverage synthetic fine-tuning data.
A Usage-centric Take on Intent Understanding in E-Commerce

Identifying and understanding user intents is a pivotal task for E-Commerce. Despite its popularity, intent understanding has not been consistently defined or accurately benchmarked. In this paper, we focus on predicative user intents as"how a customer uses a product", and pose intent understanding as a natural language reasoning task, independent of product ontologies. We identify two weaknesses of FolkScope, the SOTA E-Commerce Intent Knowledge Graph, that limit its capacity to reason about user intents and to recommend diverse useful products. Following these observations, we introduce a Product Recovery Benchmark including a novel evaluation framework and an example dataset. We further validate the above FolkScope weaknesses on this benchmark.
Dark no more: The low luminosity stellar counterpart of a dark cloud in the Virgo cluster

We have discovered the stellar counterpart to the ALFALFA Virgo 7 cloud complex, which has been thought to be optically dark and nearly star-free since its discovery in 2007. This ~190 kpc long chain of enormous atomic gas clouds ($M_\mathrm{HI} \sim 10^9 \; \mathrm{M_\odot}$) is embedded in the hot intracluster medium of the Virgo galaxy cluster but is isolated from any galaxy. Its faint, blue stellar counterpart, BC6, was identified in a visual search of archival optical and UV imaging. Follow-up observations with the Green Bank Telescope, Hobby-Eberly Telescope, and Hubble Space Telescope demonstrate that this faint counterpart is at the same velocity as the atomic gas, is actively forming stars, and is metal-rich ($12 + \mathrm{(O/H)} = 8.58 \pm 0.25$). We estimate its stellar mass to be only $\log ( M_\ast/\mathrm{M_\odot}) \sim 4.4$, making it one of the most gas-rich stellar systems known. Aside from its extraordinary gas content, the properties of BC6 are entirely consistent with those of a recently identified class of young, low-mass, isolated, and star-forming clouds in Virgo, that appear to have formed via extreme ram pressure stripping events. We expand the existing discussion of the origin of this structure and suggest NGC 4522 as a likely candidate, however, the current evidence is not fully consistent with any of our proposed progenitor galaxies. We anticipate that other"dark"gas clouds in Virgo may have similarly faint, star-forming counterparts. We aim to identify these through the help of an ongoing citizen science search of the entire cluster.
COBIAS: Contextual Reliability in Bias Assessment

Large Language Models (LLMs) are trained on inherently biased data. Previous works on debiasing models rely on benchmark datasets to measure model performance. However, these datasets suffer from several pitfalls due to the extremely subjective understanding of bias, highlighting a critical need for contextual exploration. We propose understanding the context of user inputs with consideration of the diverse situations in which input statements are possible. This approach would allow for frameworks that foster bias awareness rather than guardrails that hurt user engagement. Our contribution is twofold: (i) we create a dataset of 2287 stereotyped statements augmented with points for adding context; (ii) we develop the Context-Oriented Bias Indicator and Assessment Score (COBIAS) to assess statements' contextual reliability in measuring bias. Our metric is a significant predictor of the contextual reliability of bias-benchmark datasets ($\chi^2=71.02, p<2.2 \cdot 10^{-16})$. COBIAS can be used to create reliable datasets, resulting in an improvement in bias mitigation works.
Understanding Oversmoothing in Diffusion-Based GNNs From the Perspective of Operator Semigroup Theory

This paper presents a novel study of the oversmoothing issue in diffusion-based Graph Neural Networks (GNNs). Diverging from extant approaches grounded in random walk analysis or particle systems, we approach this problem through operator semigroup theory. This theoretical framework allows us to rigorously prove that oversmoothing is intrinsically linked to the ergodicity of the diffusion operator. This finding further poses a general and mild ergodicity-breaking condition, encompassing the various specific solutions previously offered, thereby presenting a more universal and theoretically grounded approach to mitigating oversmoothing in diffusion-based GNNs. Additionally, we offer a probabilistic interpretation of our theory, forging a link with prior works and broadening the theoretical horizon. Our experimental results reveal that this ergodicity-breaking term effectively mitigates oversmoothing measured by Dirichlet energy, and simultaneously enhances performance in node classification tasks.
Covariance-Adaptive Least-Squares Algorithm for Stochastic Combinatorial Semi-Bandits

We address the problem of stochastic combinatorial semi-bandits, where a player can select from P subsets of a set containing d base items. Most existing algorithms (e.g. CUCB, ESCB, OLS-UCB) require prior knowledge on the reward distribution, like an upper bound on a sub-Gaussian proxy-variance, which is hard to estimate tightly. In this work, we design a variance-adaptive version of OLS-UCB, relying on an online estimation of the covariance structure. Estimating the coefficients of a covariance matrix is much more manageable in practical settings and results in improved regret upper bounds compared to proxy variance-based algorithms. When covariance coefficients are all non-negative, we show that our approach efficiently leverages the semi-bandit feedback and provably outperforms bandit feedback approaches, not only in exponential regimes where P $\gg$ d but also when P $\le$ d, which is not straightforward from most existing analyses.
The New Era of Dynamic Pricing: Synergizing Supervised Learning and Quadratic Programming

In this paper, we explore a novel combination of supervised learning and quadratic programming to refine dynamic pricing models in the car rental industry. We utilize dynamic modeling of price elasticity, informed by ordinary least squares (OLS) metrics such as p-values, homoscedasticity, error normality. These metrics, when their underlying assumptions hold, are integral in guiding a quadratic programming agent. The program is tasked with optimizing margin for a given finite set target.
Dynamics-Guided Diffusion Model for Robot Manipulator Design

We present Dynamics-Guided Diffusion Model, a data-driven framework for generating manipulator geometry designs for a given manipulation task. Instead of training different design models for each task, our approach employs a learned dynamics network shared across tasks. For a new manipulation task, we first decompose it into a collection of individual motion targets which we call target interaction profile, where each individual motion can be modeled by the shared dynamics network. The design objective constructed from the target and predicted interaction profiles provides a gradient to guide the refinement of finger geometry for the task. This refinement process is executed as a classifier-guided diffusion process, where the design objective acts as the classifier guidance. We evaluate our framework on various manipulation tasks, under the sensor-less setting using only an open-loop parallel jaw motion. Our generated designs outperform optimization-based and unguided diffusion baselines relatively by 31.5% and 45.3% on average manipulation success rate. With the ability to generate a design within 0.8 seconds, our framework could facilitate rapid design iteration and enhance the adoption of data-driven approaches for robotic mechanism design.
SoK: Analyzing Adversarial Examples: A Framework to Study Adversary Knowledge

Adversarial examples are malicious inputs to machine learning models that trigger a misclassification. This type of attack has been studied for close to a decade, and we find that there is a lack of study and formalization of adversary knowledge when mounting attacks. This has yielded a complex space of attack research with hard-to-compare threat models and attacks. We focus on the image classification domain and provide a theoretical framework to study adversary knowledge inspired by work in order theory. We present an adversarial example game, inspired by cryptographic games, to standardize attacks. We survey recent attacks in the image classification domain and classify their adversary's knowledge in our framework. From this systematization, we compile results that both confirm existing beliefs about adversary knowledge, such as the potency of information about the attacked model as well as allow us to derive new conclusions on the difficulty associated with the white-box and transferable threat models, for example, that transferable attacks might not be as difficult as previously thought.
MultiLS: A Multi-task Lexical Simplification Framework

Lexical Simplification (LS) automatically replaces difficult to read words for easier alternatives while preserving a sentence's original meaning. LS is a precursor to Text Simplification with the aim of improving text accessibility to various target demographics, including children, second language learners, individuals with reading disabilities or low literacy. Several datasets exist for LS. These LS datasets specialize on one or two sub-tasks within the LS pipeline. However, as of this moment, no single LS dataset has been developed that covers all LS sub-tasks. We present MultiLS, the first LS framework that allows for the creation of a multi-task LS dataset. We also present MultiLS-PT, the first dataset to be created using the MultiLS framework. We demonstrate the potential of MultiLS-PT by carrying out all LS sub-tasks of (1). lexical complexity prediction (LCP), (2). substitute generation, and (3). substitute ranking for Portuguese. Model performances are reported, ranging from transformer-based models to more recent large language models (LLMs).
Farsight: Fostering Responsible AI Awareness During AI Application Prototyping

Prompt-based interfaces for Large Language Models (LLMs) have made prototyping and building AI-powered applications easier than ever before. However, identifying potential harms that may arise from AI applications remains a challenge, particularly during prompt-based prototyping. To address this, we present Farsight, a novel in situ interactive tool that helps people identify potential harms from the AI applications they are prototyping. Based on a user's prompt, Farsight highlights news articles about relevant AI incidents and allows users to explore and edit LLM-generated use cases, stakeholders, and harms. We report design insights from a co-design study with 10 AI prototypers and findings from a user study with 42 AI prototypers. After using Farsight, AI prototypers in our user study are better able to independently identify potential harms associated with a prompt and find our tool more useful and usable than existing resources. Their qualitative feedback also highlights that Farsight encourages them to focus on end-users and think beyond immediate harms. We discuss these findings and reflect on their implications for designing AI prototyping experiences that meaningfully engage with AI harms. Farsight is publicly accessible at: https://PAIR-code.github. io/farsight.
New scattered subspaces in higher dimensions

Over the past few decades, there has been extensive research on scattered subspaces, partly because of their link to MRD codes. These subspaces can be characterized using linearized polynomials over finite fields. Within this context, scattered sequences extend the concept of scattered polynomials and can be viewed as geometric equivalents of exceptional MRD codes. Up to now, only scattered sequences of orders one and two have been developed. However, this paper presents an infinite series of exceptional scattered sequences of any order beyond two which correspond to scattered subspaces that cannot be obtained as direct sum of scattered subspaces in smaller dimensions. The paper also addresses equivalence concerns within this framework.
A study of the Inspiral-Merger-Ringdown Consistency Test with gravitational-wave signals from compact binaries in eccentric orbits

The Inspiral Merger Ringdown Consistency Test (IMRCT) is one among a battery of tests of general relativity (GR) employed by the LIGO-Virgo-KAGRA (LVK) collaboration. It is used to search for deviations from GR in detected gravitational waves (GWs) from compact binary coalescences (CBCs) in a model-agnostic way. The test compares source parameter estimates extracted independently from the inspiral and post-inspiral portions of the CBC signals and, therefore, crucially relies on the accurate modeling of the waveform. Current implementations of the IMRCT routinely use quasicircular waveforms, under the assumption that the residual eccentricity of the binary when the emitted GWs enter the frequency band of the LVK detector network will be negligible. In this work, we perform a detailed study to investigate the typical magnitudes of this residual eccentricity that could potentially lead to spurious violations of the IMRCT. To that end, we conduct injection campaigns for a range of eccentricities and recover with both quasicircular and eccentric waveforms. We find that an eccentric GW signal from a GW150914-like system with $e_{\text{gw}} \gtrsim$ 0.04 at an orbit averaged frequency $\langle f_{\text{ref}} \rangle=$ 25 Hz breaks the IMRCT if recovered with quasicircular waveforms at $\gtrsim 68\%$ confidence. The violation becomes more severe ($\gtrsim 90\%$ confidence) for $e_{\text{gw}} =$ 0.055 at $\langle f_{\text{ref}} \rangle=$ 25 Hz. On the other hand, when eccentric waveforms are used, the IMRCT remains intact for all eccentricities considered. We also briefly investigate the effect of the magnitude and orientation (aligned/antialigned) of the component spins of the binary on the extent of the spurious violations of the IMRCT. Our work, therefore, demonstrates the need for accurate eccentric waveform models in the context of tests of GR.
Data-Driven Ground-Fault Location Method in Distribution Power System With Distributed Generation

The recent increase in renewable energy penetration at the distribution level introduces a multi-directional power flow that outdated traditional fault location techniques. To this extent, the development of new methods is needed to ensure fast and accurate fault localization and, hence, strengthen power system reliability. This paper proposes a data-driven ground fault location method for the power distribution system. An 11-bus 20 kV power system is modeled in Matlab/Simulink to simulate ground faults. The faults are generated at different locations and under various system operational states. Time-domain faulted three-phase voltages at the system substation are then analyzed with discrete wavelet transform. Statistical quantities of the processed data are eventually used to train an Artificial Neural Network (ANN) to find a mapping between computed voltage features and faults. Specifically, three ANNs allow the prediction of faulted phase, faulted branch, and fault distance from the system substation separately. According to the results, the method shows good potential, with a total relative error of 0,4% for fault distance prediction. The method is applied to datasets with unknown system states to test robustness.
New England and northern New York forest ecosystem vulnerability assessment and synthesis: a report from the New England Climate Change Response Framework project

Forest ecosystems will face direct and indirect impacts from a changing climate over the 21st century. This assessment evaluates the vulnerability of forest ecosystems across the New England region (Connecticut, Maine, Massachusetts, New Hampshire, northern New York, Rhode Island, and Vermont) under a range of future climates. We synthesized and summarized information on the contemporary landscape, provided information on past climate trends, and described a range of projected future climates. This information was used to parameterize and run multiple vegetation impact models, which provided a range of potential vegetative responses to climate. Finally, we brought these results before a multidisciplinary panel of scientists and natural resource professionals familiar with the forests of this region to assess ecosystem vulnerability through a formal consensus-based expert elicitation process. Observed trends in climate over the historical record from 1901 through 2011 show that the mean annual temperature has increased across the region by 2.4 °F, with even greater warming during winter. Precipitation patterns also changed during this time, with a slight trend toward greater annual precipitation and a substantial increase in extreme precipitation events. Projected climate trends using downscaled global climate model data indicate a potential increase in mean annual temperature of 3 to 8 °F for the assessment area by 2100. Projections for precipitation indicate an increase in fall and winter precipitation, and spring and summer precipitation projections vary by scenario. We identified potential impacts on forests by incorporating these future climate projections into three forest impact models (DISTRIB, LINKAGES, and LANDIS PRO). Model projections suggest that many northern and boreal species, including balsam fir, red spruce, and black spruce, may fare worse under future conditions, but other species may benefit from projected changes in climate. Published literature on climate impacts related to wildfire, invasive species, and forest pests and diseases also contributed to the overall determination of climate change vulnerability. We assessed vulnerability for eight forest communities in the assessment area. The assessment was conducted through a formal elicitation process with 20 scientists and resource managers from across the area, who considered vulnerability in terms of the potential impacts and the adaptive capacity for an individual community. Montane spruce-fir, low-elevation spruce-fir, and lowland mixed conifer forests were determined to be the most vulnerable communities. Central hardwoods, transition hardwoods, and pitch pine-scrub oak forests were perceived as having lower vulnerability to projected changes in climate. These projected changes in climate and the associated impacts and vulnerabilities will have important implications for economically valuable timber species, forest-dependent animals and plants, recreation, and long-term natural resource planning.
ProTIP: Probabilistic Robustness Verification on Text-to-Image Diffusion Models against Stochastic Perturbation

Text-to-Image (T2I) Diffusion Models (DMs) have shown impressive abilities in generating high-quality images based on simple text descriptions. However, as is common with many Deep Learning (DL) models, DMs are subject to a lack of robustness. While there are attempts to evaluate the robustness of T2I DMs as a binary or worst-case problem, they cannot answer how robust in general the model is whenever an adversarial example (AE) can be found. In this study, we first introduce a probabilistic notion of T2I DMs' robustness; and then establish an efficient framework, ProTIP, to evaluate it with statistical guarantees. The main challenges stem from: i) the high computational cost of the generation process; and ii) determining if a perturbed input is an AE involves comparing two output distributions, which is fundamentally harder compared to other DL tasks like classification where an AE is identified upon misprediction of labels. To tackle the challenges, we employ sequential analysis with efficacy and futility early stopping rules in the statistical testing for identifying AEs, and adaptive concentration inequalities to dynamically determine the"just-right"number of stochastic perturbations whenever the verification target is met. Empirical experiments validate the effectiveness and efficiency of ProTIP over common T2I DMs. Finally, we demonstrate an application of ProTIP to rank commonly used defence methods.
MetaStates: An Approach for Representing Human Workers Psychophysiological States in the Industrial Metaverse

Photo-realistic avatar is a modern term referring to the digital asset that represents a human in computer graphic advance systems such as video games and simulation tools. These avatars utilize the advances in graphic technologies on both software and hardware aspects. While photorealistic avatars are increasingly used in industrial simulations, representing human factors such as human workers internal states, remains a challenge. This article addresses this issue by introducing the concept of MetaStates which are the digitization and representation of the psychophysiological states of a human worker in the digital world. The MetaStates influence the physical representation and performance of a digital human worker while performing a task. To demonstrate this concept the study presents a development of a photorealistic avatar which is integrated into a simulated environment and enhanced with a multi-level graphical representation of different psychophysiological states. This approach represents a major step forward in the use of digital humans for industrial simulations, allowing companies to better leverage the benefits of the Industrial Metaverse in their daily operations and simulations while keeping human workers at the center of the system.
Practical Software for Triangulating and Simplifying 4-Manifolds

Dimension 4 is the first dimension in which exotic smooth manifold pairs appear -- manifolds which are topologically the same but for which there is no smooth deformation of one into the other. Whilst smooth and triangulated 4-manifolds do coincide, comparatively little work has been done towards gaining an understanding of smooth 4-manifolds from the discrete and algorithmic perspective. In this paper we introduce new software tools to make this possible, including a software implementation of an algorithm which enables us to build triangulations of 4-manifolds from Kirby diagrams, as well as a new heuristic for simplifying 4-manifold triangulations. Using these tools, we present new triangulations of several bounded exotic pairs, corks and plugs (objects responsible for"exoticity"), as well as the smallest known triangulation of the fundamental K3 surface. The small size of these triangulations benefit us by revealing fine structural features in 4-manifold triangulations.