Orionnnn/hub-stats huggingface dataset:
Language Models Prefer What They Know: Relative Confidence Estimation
  via Confidence Preferences
Authors: name: Vaishnavi Shrivastava, name: Ananya Kumar, name: Percy Liang
Published: 2025-02-03 07:43:27

Language models (LMs) should provide reliable confidence estimates to help
users detect mistakes in their outputs and defer to human experts when
necessary. Asking a language model to assess its confidence ("Score your
confidence from 0-1.") is a natural way of evaluating its uncertainty. However,
models struggle to provide absolute assessments of confidence (i.e. judging
confidence in answering a question independent of other questions) and the
coarse-grained scores they produce are not useful for evaluating the
correctness of their answers. We propose relative confidence estimation, where
we match up questions against each other and ask the model to make relative
judgments of confidence ("Which question are you more confident in answering
correctly?"). Treating each question as a "player" in a series of matchups
against other questions and the model's preferences as match outcomes, we can
use rank aggregation methods like Elo rating and Bradley-Terry to translate the
model's confidence preferences into confidence scores. We evaluate relative
confidence estimation against absolute confidence estimation and
self-consistency confidence methods on five state-of-the-art LMs -- GPT-4,
GPT-4o, Gemini 1.5 Pro, Claude 3.5 Sonnet, and Llama 3.1 405B -- across 14
challenging STEM, social science, and commonsense reasoning question answering
tasks. Our results demonstrate that relative confidence estimation consistently
provides more reliable confidence scores than absolute confidence estimation,
with average gains of 3.5% in selective classification AUC over direct absolute
confidence estimation methods and 1.7% over self-consistency approaches across
all models and datasets.
AlignVLM: Bridging Vision and Language Latent Spaces for Multimodal
  Understanding
Authors: name: Ahmed Masry, name: Juan A. Rodriguez, name: Tianyu Zhang, name: Suyuchen Wang, name: Chao Wang, name: Aarash Feizi, name: Akshay Kalkunte Suresh, name: Abhay Puri, name: Xiangru Jian, name: Pierre-André Noël, name: Sathwik Tejaswi Madhusudhan, name: Marco Pedersoli, name: Bang Liu, name: Nicolas Chapados, name: Yoshua Bengio, name: Enamul Hoque, name: Christopher Pal, name: Issam H. Laradji, name: David Vazquez, name: Perouz Taslakian, name: Spandana Gella, name: Sai Rajeswar
Published: 2025-02-03 13:34:51

Aligning visual features with language embeddings is a key challenge in
vision-language models (VLMs). The performance of such models hinges on having
a good connector that maps visual features generated by a vision encoder to a
shared embedding space with the LLM while preserving semantic similarity.
Existing connectors, such as multilayer perceptrons (MLPs), often produce
out-of-distribution or noisy inputs, leading to misalignment between the
modalities. In this work, we propose a novel vision-text alignment method,
AlignVLM, that maps visual features to a weighted average of LLM text
embeddings. Our approach leverages the linguistic priors encoded by the LLM to
ensure that visual features are mapped to regions of the space that the LLM can
effectively interpret. AlignVLM is particularly effective for document
understanding tasks, where scanned document images must be accurately mapped to
their textual content. Our extensive experiments show that AlignVLM achieves
state-of-the-art performance compared to prior alignment methods. We provide
further analysis demonstrating improved vision-text feature alignment and
robustness to noise.
MM-IQ: Benchmarking Human-Like Abstraction and Reasoning in Multimodal
  Models
Authors: name: Huanqia Cai, name: Yijun Yang, name: Winston Hu
Published: 2025-02-02 07:12:03

IQ testing has served as a foundational methodology for evaluating human
cognitive capabilities, deliberately decoupling assessment from linguistic
background, language proficiency, or domain-specific knowledge to isolate core
competencies in abstraction and reasoning. Yet, artificial intelligence
research currently lacks systematic benchmarks to quantify these critical
cognitive dimensions in multimodal systems. To address this critical gap, we
propose MM-IQ, a comprehensive evaluation framework comprising 2,710
meticulously curated test items spanning 8 distinct reasoning paradigms.
  Through systematic evaluation of leading open-source and proprietary
multimodal models, our benchmark reveals striking limitations: even
state-of-the-art architectures achieve only marginally superior performance to
random chance (27.49% vs. 25% baseline accuracy). This substantial performance
chasm highlights the inadequacy of current multimodal systems in approximating
fundamental human reasoning capacities, underscoring the need for
paradigm-shifting advancements to bridge this cognitive divide.
SliderSpace: Decomposing the Visual Capabilities of Diffusion Models
Authors: name: Rohit Gandikota, name: Zongze Wu, name: Richard Zhang, name: David Bau, name: Eli Shechtman, name: Nick Kolkin
Published: 2025-02-03 18:59:55

We present SliderSpace, a framework for automatically decomposing the visual
capabilities of diffusion models into controllable and human-understandable
directions. Unlike existing control methods that require a user to specify
attributes for each edit direction individually, SliderSpace discovers multiple
interpretable and diverse directions simultaneously from a single text prompt.
Each direction is trained as a low-rank adaptor, enabling compositional control
and the discovery of surprising possibilities in the model's latent space.
Through extensive experiments on state-of-the-art diffusion models, we
demonstrate SliderSpace's effectiveness across three applications: concept
decomposition, artistic style exploration, and diversity enhancement. Our
quantitative evaluation shows that SliderSpace-discovered directions decompose
the visual structure of model's knowledge effectively, offering insights into
the latent capabilities encoded within diffusion models. User studies further
validate that our method produces more diverse and useful variations compared
to baselines. Our code, data and trained weights are available at
https://sliderspace.baulab.info
Almost Surely Safe Alignment of Large Language Models at Inference-Time
Authors: name: Xiaotong Ji, name: Shyam Sundhar Ramesh, name: Matthieu Zimmer, name: Ilija Bogunovic, name: Jun Wang, name: Haitham Bou Ammar
Published: 2025-02-03 09:59:32

Even highly capable large language models (LLMs) can produce biased or unsafe
responses, and alignment techniques, such as RLHF, aimed at mitigating this
issue, are expensive and prone to overfitting as they retrain the LLM. This
paper introduces a novel inference-time alignment approach that ensures LLMs
generate safe responses almost surely, i.e., with a probability approaching
one. We achieve this by framing the safe generation of inference-time responses
as a constrained Markov decision process within the LLM's latent space.
Crucially, we augment a safety state that tracks the evolution of safety
constraints and enables us to demonstrate formal safety guarantees upon solving
the MDP in the latent space. Building on this foundation, we propose
InferenceGuard, a practical implementation that safely aligns LLMs without
modifying the model weights. Empirically, we demonstrate InferenceGuard
effectively balances safety and task performance, outperforming existing
inference-time alignment methods in generating safe and aligned responses.
PhD Knowledge Not Required: A Reasoning Challenge for Large Language
  Models
Authors: name: Carolyn Jane Anderson, name: Joydeep Biswas, name: Aleksander Boruch-Gruszecki, name: Federico Cassano, name: Molly Q Feldman, name: Arjun Guha, name: Francesca Lucchetti, name: Zixuan Wu
Published: 2025-02-03 18:10:38

Existing benchmarks for frontier models often test specialized, ``PhD-level''
knowledge that is difficult for non-experts to grasp. In contrast, we present a
benchmark based on the NPR Sunday Puzzle Challenge that requires only general
knowledge. Our benchmark is challenging for both humans and models, however
correct solutions are easy to verify, and models' mistakes are easy to spot.
  Our work reveals capability gaps that are not evident in existing benchmarks:
OpenAI o1 significantly outperforms other reasoning models that are on par on
benchmarks that test specialized knowledge. Furthermore, our analysis of
reasoning outputs uncovers new kinds of failures. DeepSeek R1, for instance,
often concedes with ``I give up'' before providing an answer that it knows is
wrong. R1 can also be remarkably ``uncertain'' in its output and in rare cases,
it does not ``finish thinking,'' which suggests the need for an inference-time
technique to ``wrap up'' before the context window limit is reached. We also
quantify the effectiveness of reasoning longer with R1 and Gemini Thinking to
identify the point beyond which more reasoning is unlikely to improve accuracy
on our benchmark.
Current Pathology Foundation Models are unrobust to Medical Center
  Differences
Authors: name: Edwin D. de Jong, name: Eric Marcus, name: Jonas Teuwen
Published: 2025-01-29 23:38:14

Pathology Foundation Models (FMs) hold great promise for healthcare. Before
they can be used in clinical practice, it is essential to ensure they are
robust to variations between medical centers. We measure whether pathology FMs
focus on biological features like tissue and cancer type, or on the well known
confounding medical center signatures introduced by staining procedure and
other differences. We introduce the Robustness Index. This novel robustness
metric reflects to what degree biological features dominate confounding
features. Ten current publicly available pathology FMs are evaluated. We find
that all current pathology foundation models evaluated represent the medical
center to a strong degree. Significant differences in the robustness index are
observed. Only one model so far has a robustness index greater than one,
meaning biological features dominate confounding features, but only slightly. A
quantitative approach to measure the influence of medical center differences on
FM-based prediction performance is described. We analyze the impact of
unrobustness on classification performance of downstream models, and find that
cancer-type classification errors are not random, but specifically attributable
to same-center confounders: images of other classes from the same medical
center. We visualize FM embedding spaces, and find these are more strongly
organized by medical centers than by biological factors. As a consequence, the
medical center of origin is predicted more accurately than the tissue source
and cancer type. The robustness index introduced here is provided with the aim
of advancing progress towards clinical adoption of robust and reliable
pathology FMs.
DeepRAG: Thinking to Retrieval Step by Step for Large Language Models
Authors: name: Xinyan Guan, name: Jiali Zeng, name: Fandong Meng, name: Chunlei Xin, name: Yaojie Lu, name: Hongyu Lin, name: Xianpei Han, name: Le Sun, name: Jie Zhou
Published: 2025-02-03 08:22:45

Large Language Models (LLMs) have shown remarkable potential in reasoning
while they still suffer from severe factual hallucinations due to timeliness,
accuracy, and coverage of parametric knowledge. Meanwhile, integrating
reasoning with retrieval-augmented generation (RAG) remains challenging due to
ineffective task decomposition and redundant retrieval, which can introduce
noise and degrade response quality. In this paper, we propose DeepRAG, a
framework that models retrieval-augmented reasoning as a Markov Decision
Process (MDP), enabling strategic and adaptive retrieval. By iteratively
decomposing queries, DeepRAG dynamically determines whether to retrieve
external knowledge or rely on parametric reasoning at each step. Experiments
show that DeepRAG improves retrieval efficiency while improving answer accuracy
by 21.99%, demonstrating its effectiveness in optimizing retrieval-augmented
reasoning.
A Study on the Performance of U-Net Modifications in Retroperitoneal
  Tumor Segmentation
Authors: name: Moein Heidari, name: Ehsan Khodapanah Aghdam, name: Alexander Manzella, name: Daniel Hsu, name: Rebecca Scalabrino, name: Wenjin Chen, name: David J. Foran, name: Ilker Hacihaliloglu
Published: 2025-02-01 04:25:28

The retroperitoneum hosts a variety of tumors, including rare benign and
malignant types, which pose diagnostic and treatment challenges due to their
infrequency and proximity to vital structures. Estimating tumor volume is
difficult due to their irregular shapes, and manual segmentation is
time-consuming. Automatic segmentation using U-Net and its variants,
incorporating Vision Transformer (ViT) elements, has shown promising results
but struggles with high computational demands. To address this, architectures
like the Mamba State Space Model (SSM) and Extended Long-Short Term Memory
(xLSTM) offer efficient solutions by handling long-range dependencies with
lower resource consumption. This study evaluates U-Net enhancements, including
CNN, ViT, Mamba, and xLSTM, on a new in-house CT dataset and a public organ
segmentation dataset. The proposed ViLU-Net model integrates Vi-blocks for
improved segmentation. Results highlight xLSTM's efficiency in the U-Net
framework. The code is publicly accessible on GitHub.
SafeRAG: Benchmarking Security in Retrieval-Augmented Generation of
  Large Language Model
Authors: name: Xun Liang, name: Simin Niu, name: Zhiyu Li, name: Sensen Zhang, name: Hanyu Wang, name: Feiyu Xiong, name: Jason Zhaoxin Fan, name: Bo Tang, name: Shichao Song, name: Mengwei Wang, name: Jiawei Yang
Published: 2025-01-28 17:01:31

The indexing-retrieval-generation paradigm of retrieval-augmented generation
(RAG) has been highly successful in solving knowledge-intensive tasks by
integrating external knowledge into large language models (LLMs). However, the
incorporation of external and unverified knowledge increases the vulnerability
of LLMs because attackers can perform attack tasks by manipulating knowledge.
In this paper, we introduce a benchmark named SafeRAG designed to evaluate the
RAG security. First, we classify attack tasks into silver noise, inter-context
conflict, soft ad, and white Denial-of-Service. Next, we construct RAG security
evaluation dataset (i.e., SafeRAG dataset) primarily manually for each task. We
then utilize the SafeRAG dataset to simulate various attack scenarios that RAG
may encounter. Experiments conducted on 14 representative RAG components
demonstrate that RAG exhibits significant vulnerability to all attack tasks and
even the most apparent attack task can easily bypass existing retrievers,
filters, or advanced LLMs, resulting in the degradation of RAG service quality.
Code is available at: https://github.com/IAAR-Shanghai/SafeRAG.
The Differences Between Direct Alignment Algorithms are a Blur
Authors: name: Alexey Gorbatovski, name: Boris Shaposhnikov, name: Viacheslav Sinii, name: Alexey Malakhov, name: Daniil Gavrilov
Published: 2025-02-03 10:54:14

Direct Alignment Algorithms (DAAs) simplify language model alignment by
replacing reinforcement learning (RL) and reward modeling (RM) in Reinforcement
Learning from Human Feedback (RLHF) with direct policy optimization. DAAs can
be classified by their ranking losses (pairwise vs. pointwise), by the rewards
used in those losses (e.g., likelihood ratios of policy and reference policy,
or odds ratios), or by whether a Supervised Fine-Tuning (SFT) phase is required
(two-stage vs. one-stage). We first show that one-stage methods underperform
two-stage methods. To address this, we incorporate an explicit SFT phase and
introduce the beta parameter, controlling the strength of preference
optimization, into single-stage ORPO and ASFT. These modifications improve
their performance in Alpaca Eval 2 by +3.46 (ORPO) and +8.27 (ASFT),
matching two-stage methods like DPO. Further analysis reveals that the key
factor is whether the approach uses pairwise or pointwise objectives, rather
than the specific implicit reward or loss function. These results highlight the
importance of careful evaluation to avoid premature claims of performance gains
or overall superiority in alignment algorithms.
Preference Leakage: A Contamination Problem in LLM-as-a-judge
Authors: name: Dawei Li, name: Renliang Sun, name: Yue Huang, name: Ming Zhong, name: Bohan Jiang, name: Jiawei Han, name: Xiangliang Zhang, name: Wei Wang, name: Huan Liu
Published: 2025-02-03 17:13:03

Large Language Models (LLMs) as judges and LLM-based data synthesis have
emerged as two fundamental LLM-driven data annotation methods in model
development. While their combination significantly enhances the efficiency of
model training and evaluation, little attention has been given to the potential
contamination brought by this new model development paradigm. In this work, we
expose preference leakage, a contamination problem in LLM-as-a-judge caused by
the relatedness between the synthetic data generators and LLM-based evaluators.
To study this issue, we first define three common relatednesses between data
generator LLM and judge LLM: being the same model, having an inheritance
relationship, and belonging to the same model family. Through extensive
experiments, we empirically confirm the bias of judges towards their related
student models caused by preference leakage across multiple LLM baselines and
benchmarks. Further analysis suggests that preference leakage is a pervasive
issue that is harder to detect compared to previously identified biases in
LLM-as-a-judge scenarios. All of these findings imply that preference leakage
is a widespread and challenging problem in the area of LLM-as-a-judge. We
release all codes and data at:
https://github.com/David-Li0406/Preference-Leakage.
Lifelong Sequential Knowledge Editing without Model Degradation
Authors: name: Akshat Gupta, name: Phudish Prateepamornkul, name: Maochuan Lu, name: Ahmed Alaa, name: Thomas Hartvigsen, name: Gopala Anumanchipalli
Published: 2025-02-03 18:59:14

Prior work in parameter-modifying knowledge editing has shown that
large-scale sequential editing leads to significant model degradation. In this
paper, we study the reasons behind this and scale sequential knowledge editing
to 10,000 sequential edits, while maintaining the downstream performance of the
original model. We first show that locate-then-edit knowledge editing methods
lead to overfitting on the edited facts. We also show that continuous knowledge
editing using these methods leads to disproportionate growth in the norm of the
edited matrix. We then provide a crucial insight into the inner workings of
locate-then-edit methods. We show that norm-growth is a hidden trick employed
by these methods that gives larger importance to the output activations
produced from the edited layers. With this "importance hacking", the edited
layers provide a much larger contributions to the model's output. To mitigate
these issues, we present ENCORE - Early stopping and Norm-Constrained Robust
knowledge Editing. ENCORE controls for overfitting and the disproportionate
norm-growth to enable long-term sequential editing, where we are able to
perform up to 10,000 sequential edits without loss of downstream performance.
ENCORE is also 61% faster than MEMIT and 64% faster than AlphaEdit on
Llama3-8B.
FastKV: KV Cache Compression for Fast Long-Context Processing with
  Token-Selective Propagation
Authors: name: Dongwon Jo, name: Jiwon Song, name: Yulhwa Kim, name: Jae-Joon Kim
Published: 2025-02-03 05:25:09

While large language models (LLMs) excel at handling long-context sequences,
they require substantial key-value (KV) caches to store contextual information,
which can heavily burden computational efficiency and memory usage. Previous
efforts to compress these KV caches primarily focused on reducing memory
demands but were limited in enhancing latency. To address this issue, we
introduce FastKV, a KV cache compression method designed to enhance latency for
long-context sequences. To enhance processing speeds while maintaining
accuracy, FastKV adopts a novel Token-Selective Propagation (TSP) approach that
retains the full context information in the initial layers of LLMs and
selectively propagates only a portion of this information in deeper layers even
in the prefill stage. Additionally, FastKV incorporates grouped-query attention
(GQA)-aware KV cache compression to exploit the advantages of GQA in both
memory and computational efficiency. Our experimental results show that FastKV
achieves 2.00times and 1.40times improvements in time-to-first-token
(TTFT) and throughput, respectively, compared to HeadKV, the state-of-the-art
KV cache compression method. Moreover, FastKV successfully maintains accuracy
on long-context benchmarks at levels comparable to the baselines. Our code is
available at https://github.com/dongwonjo/FastKV.
OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human
  Animation Models
Authors: name: Gaojie Lin, name: Jianwen Jiang, name: Jiaqi Yang, name: Zerong Zheng, name: Chao Liang
Published: 2025-02-03 05:17:32

End-to-end human animation, such as audio-driven talking human generation,
has undergone notable advancements in the recent few years. However, existing
methods still struggle to scale up as large general video generation models,
limiting their potential in real applications. In this paper, we propose
OmniHuman, a Diffusion Transformer-based framework that scales up data by
mixing motion-related conditions into the training phase. To this end, we
introduce two training principles for these mixed conditions, along with the
corresponding model architecture and inference strategy. These designs enable
OmniHuman to fully leverage data-driven motion generation, ultimately achieving
highly realistic human video generation. More importantly, OmniHuman supports
various portrait contents (face close-up, portrait, half-body, full-body),
supports both talking and singing, handles human-object interactions and
challenging body poses, and accommodates different image styles. Compared to
existing end-to-end audio-driven methods, OmniHuman not only produces more
realistic videos, but also offers greater flexibility in inputs. It also
supports multiple driving modalities (audio-driven, video-driven and combined
driving signals). Video samples are provided on the ttfamily project page
(https://omnihuman-lab.github.io)
ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning
Authors: name: Bill Yuchen Lin, name: Ronan Le Bras, name: Kyle Richardson, name: Ashish Sabharwal, name: Radha Poovendran, name: Peter Clark, name: Yejin Choi
Published: 2025-02-03 06:44:49

We investigate the logical reasoning capabilities of large language models
(LLMs) and their scalability in complex non-monotonic reasoning. To this end,
we introduce ZebraLogic, a comprehensive evaluation framework for assessing LLM
reasoning performance on logic grid puzzles derived from constraint
satisfaction problems (CSPs). ZebraLogic enables the generation of puzzles with
controllable and quantifiable complexity, facilitating a systematic study of
the scaling limits of models such as Llama, o1 models, and DeepSeek-R1. By
encompassing a broad range of search space complexities and diverse logical
constraints, ZebraLogic provides a structured environment to evaluate reasoning
under increasing difficulty.
  Our results reveal a significant decline in accuracy as problem complexity
grows -- a phenomenon we term the curse of complexity. This limitation persists
even with larger models and increased inference-time computation, suggesting
inherent constraints in current LLM reasoning capabilities. Additionally, we
explore strategies to enhance logical reasoning, including Best-of-N sampling,
backtracking mechanisms, and self-verification prompts. Our findings offer
critical insights into the scalability of LLM reasoning, highlight fundamental
limitations, and outline potential directions for improvement.