Orionnnn/hub-stats huggingface dataset:
LServe: Efficient Long-sequence LLM Serving with Unified Sparse
  Attention
Authors: name: Shang Yang, name: Junxian Guo, name: Haotian Tang, name: Qinghao Hu, name: Guangxuan Xiao, name: Jiaming Tang, name: Yujun Lin, name: Zhijian Liu, name: Yao Lu, name: Song Han
Published: 2025-02-20 18:59:52

Large language models (LLMs) have shown remarkable potential in processing
long sequences, yet efficiently serving these long-context models remains
challenging due to the quadratic computational complexity of attention in the
prefilling stage and the large memory footprint of the KV cache in the decoding
stage. To address these issues, we introduce LServe, an efficient system that
accelerates long-sequence LLM serving via hybrid sparse attention. This method
unifies different hardware-friendly, structured sparsity patterns for both
prefilling and decoding attention into a single framework, where computations
on less important tokens are skipped block-wise. LServe demonstrates the
compatibility of static and dynamic sparsity in long-context LLM attention.
This design enables multiplicative speedups by combining these optimizations.
Specifically, we convert half of the attention heads to nearly free streaming
heads in both the prefilling and decoding stages. Additionally, we find that
only a constant number of KV pages is required to preserve long-context
capabilities, irrespective of context length. We then design a hierarchical KV
page selection policy that dynamically prunes KV pages based on query-centric
similarity. On average, LServe accelerates LLM prefilling by up to 2.9x and
decoding by 1.3-2.1x over vLLM, maintaining long-context accuracy. Code is
released at https://github.com/mit-han-lab/omniserve.
Enhancing Cognition and Explainability of Multimodal Foundation Models
  with Self-Synthesized Data
Authors: name: Yucheng Shi, name: Quanzheng Li, name: Jin Sun, name: Xiang Li, name: Ninghao Liu
Published: 2025-02-19 19:05:45

Large multimodal models (LMMs) have shown impressive capabilities in a wide
range of visual tasks. However, they often struggle with fine-grained visual
reasoning, failing to identify domain-specific objectives and provide
justifiable explanations for their predictions. To address this, we propose a
novel visual rejection sampling framework to improve the cognition and
explainability of LMMs using self-synthesized data. Specifically, visual
fine-tuning requires images, queries, and target answers. Our approach begins
by synthesizing interpretable answers that include human-verifiable visual
features. These features are based on expert-defined concepts, carefully
selected based on their alignment with the image content. After each round of
fine-tuning, we apply a reward model-free filtering mechanism to select the
highest-quality interpretable answers for the next round of tuning. This
iterative process of data synthesis and fine-tuning progressively improves the
model's ability to generate accurate and reasonable explanations. Experimental
results demonstrate the effectiveness of our method in improving both the
accuracy and explainability of specialized visual classification tasks.
NAVIG: Natural Language-guided Analysis with Vision Language Models for
  Image Geo-localization
Authors: name: Zheyuan Zhang, name: Runze Li, name: Tasnim Kabir, name: Jordan Boyd-Graber
Published: 2025-02-20 15:21:35

Image geo-localization is the task of predicting the specific location of an
image and requires complex reasoning across visual, geographical, and cultural
contexts. While prior Vision Language Models (VLMs) have the best accuracy at
this task, there is a dearth of high-quality datasets and models for analytical
reasoning. We first create NaviClues, a high-quality dataset derived from
GeoGuessr, a popular geography game, to supply examples of expert reasoning
from language. Using this dataset, we present Navig, a comprehensive image
geo-localization framework integrating global and fine-grained image
information. By reasoning with language, Navig reduces the average distance
error by 14% compared to previous state-of-the-art models while requiring fewer
than 1000 training samples. Our dataset and code are available at
https://github.com/SparrowZheyuan18/Navig/.
From RAG to Memory: Non-Parametric Continual Learning for Large Language
  Models
Authors: name: Bernal Jiménez Gutiérrez, name: Yiheng Shu, name: Weijian Qi, name: Sizhe Zhou, name: Yu Su
Published: 2025-02-20 18:26:02

Our ability to continuously acquire, organize, and leverage knowledge is a
key feature of human intelligence that AI systems must approximate to unlock
their full potential. Given the challenges in continual learning with large
language models (LLMs), retrieval-augmented generation (RAG) has become the
dominant way to introduce new information. However, its reliance on vector
retrieval hinders its ability to mimic the dynamic and interconnected nature of
human long-term memory. Recent RAG approaches augment vector embeddings with
various structures like knowledge graphs to address some of these gaps, namely
sense-making and associativity. However, their performance on more basic
factual memory tasks drops considerably below standard RAG. We address this
unintended deterioration and propose HippoRAG 2, a framework that outperforms
standard RAG comprehensively on factual, sense-making, and associative memory
tasks. HippoRAG 2 builds upon the Personalized PageRank algorithm used in
HippoRAG and enhances it with deeper passage integration and more effective
online use of an LLM. This combination pushes this RAG system closer to the
effectiveness of human long-term memory, achieving a 7% improvement in
associative memory tasks over the state-of-the-art embedding model while also
exhibiting superior factual knowledge and sense-making memory capabilities.
This work paves the way for non-parametric continual learning for LLMs. Our
code and data will be released at https://github.com/OSU-NLP-Group/HippoRAG.
CLIPPER: Compression enables long-context synthetic data generation
Authors: name: Chau Minh Pham, name: Yapei Chang, name: Mohit Iyyer
Published: 2025-02-20 18:58:03

LLM developers are increasingly reliant on synthetic data, but generating
high-quality data for complex long-context reasoning tasks remains challenging.
We introduce CLIPPER, a compression-based approach for generating synthetic
data tailored to narrative claim verification - a task that requires reasoning
over a book to verify a given claim. Instead of generating claims directly from
the raw text of the book, which results in artifact-riddled claims, CLIPPER
first compresses the book into chapter outlines and book summaries and then
uses these intermediate representations to generate complex claims and
corresponding chain-of-thoughts. Compared to naive approaches, CLIPPER produces
claims that are more valid, grounded, and complex. Using CLIPPER, we construct
a dataset of 19K synthetic book claims paired with their source texts and
chain-of-thought reasoning, and use it to fine-tune three open-weight models.
Our best model achieves breakthrough results on narrative claim verification
(from 28% to 76% accuracy on our test set) and sets a new state-of-the-art for
sub-10B models on the NoCha leaderboard. Further analysis shows that our models
generate more detailed and grounded chain-of-thought reasoning while also
improving performance on other narrative understanding tasks (e.g.,
NarrativeQA).
LLM-based User Profile Management for Recommender System
Authors: name: Seunghwan Bang, name: Hwanjun Song
Published: 2025-02-20 13:20:19

The rapid advancement of Large Language Models (LLMs) has opened new
opportunities in recommender systems by enabling zero-shot recommendation
without conventional training. Despite their potential, most existing works
rely solely on users' purchase histories, leaving significant room for
improvement by incorporating user-generated textual data, such as reviews and
product descriptions. Addressing this gap, we propose PURE, a novel LLM-based
recommendation framework that builds and maintains evolving user profiles by
systematically extracting and summarizing key information from user reviews.
PURE consists of three core components: a Review Extractor for identifying user
preferences and key product features, a Profile Updater for refining and
updating user profiles, and a Recommender for generating personalized
recommendations using the most current profile. To evaluate PURE, we introduce
a continuous sequential recommendation task that reflects real-world scenarios
by adding reviews over time and updating predictions incrementally. Our
experimental results on Amazon datasets demonstrate that PURE outperforms
existing LLM-based methods, effectively leveraging long-term user information
while managing token limitations.
How Much Knowledge Can You Pack into a LoRA Adapter without Harming LLM?
Authors: name: Sergey Pletenev, name: Maria Marina, name: Daniil Moskovskiy, name: Vasily Konovalov, name: Pavel Braslavski, name: Alexander Panchenko, name: Mikhail Salnikov
Published: 2025-02-20 12:31:03

The performance of Large Language Models (LLMs) on many tasks is greatly
limited by the knowledge learned during pre-training and stored in the model's
parameters. Low-rank adaptation (LoRA) is a popular and efficient training
technique for updating or domain-specific adaptation of LLMs. In this study, we
investigate how new facts can be incorporated into the LLM using LoRA without
compromising the previously learned knowledge. We fine-tuned
Llama-3.1-8B-instruct using LoRA with varying amounts of new knowledge. Our
experiments have shown that the best results are obtained when the training
data contains a mixture of known and new facts. However, this approach is still
potentially harmful because the model's performance on external
question-answering benchmarks declines after such fine-tuning. When the
training data is biased towards certain entities, the model tends to regress to
few overrepresented answers. In addition, we found that the model becomes more
confident and refuses to provide an answer in only few cases. These findings
highlight the potential pitfalls of LoRA-based LLM updates and underscore the
importance of training data composition and tuning parameters to balance new
knowledge integration and general model capabilities.
How Much Do LLMs Hallucinate across Languages? On Multilingual
  Estimation of LLM Hallucination in the Wild
Authors: name: Saad Obaid ul Islam, name: Anne Lauscher, name: Goran Glavaš
Published: 2025-02-18 11:32:43

In the age of misinformation, hallucination -- the tendency of Large Language
Models (LLMs) to generate non-factual or unfaithful responses -- represents the
main risk for their global utility. Despite LLMs becoming increasingly
multilingual, the vast majority of research on detecting and quantifying LLM
hallucination are (a) English-centric and (b) focus on machine translation (MT)
and summarization, tasks that are less common ``in the wild'' than open
information seeking. In contrast, we aim to quantify the extent of LLM
hallucination across languages in knowledge-intensive long-form question
answering. To this end, we train a multilingual hallucination detection model
and conduct a large-scale study across 30 languages and 6 open-source LLM
families. We start from an English hallucination detection dataset and rely on
MT to generate (noisy) training data in other languages. We also manually
annotate gold data for five high-resource languages; we then demonstrate, for
these languages, that the estimates of hallucination rates are similar between
silver (LLM-generated) and gold test sets, validating the use of silver data
for estimating hallucination rates for other languages. For the final rates
estimation, we build a knowledge-intensive QA dataset for 30 languages with
LLM-generated prompts and Wikipedia articles as references. We find that, while
LLMs generate longer responses with more hallucinated tokens for
higher-resource languages, there is no correlation between length-normalized
hallucination rates of languages and their digital representation. Further, we
find that smaller LLMs exhibit larger hallucination rates than larger models.
S^2R: Teaching LLMs to Self-verify and Self-correct via Reinforcement
  Learning
Authors: name: Ruotian Ma, name: Peisong Wang, name: Cheng Liu, name: Xingyan Liu, name: Jiaqi Chen, name: Bang Zhang, name: Xin Zhou, name: Nan Du, name: Jia Li
Published: 2025-02-18 13:40:22

Recent studies have demonstrated the effectiveness of LLM test-time scaling.
However, existing approaches to incentivize LLMs' deep thinking abilities
generally require large-scale data or significant training efforts. Meanwhile,
it remains unclear how to improve the thinking abilities of less powerful base
models. In this work, we introduce S^2R, an efficient framework that enhances
LLM reasoning by teaching models to self-verify and self-correct during
inference. Specifically, we first initialize LLMs with iterative
self-verification and self-correction behaviors through supervised fine-tuning
on carefully curated data. The self-verification and self-correction skills are
then further strengthened by both outcome-level and process-level reinforcement
learning, with minimized resource requirements, enabling the model to
adaptively refine its reasoning process during inference. Our results
demonstrate that, with only 3.1k self-verifying and self-correcting behavior
initialization samples, Qwen2.5-math-7B achieves an accuracy improvement from
51.0\% to 81.6\%, outperforming models trained on an equivalent amount of
long-CoT distilled data. Extensive experiments and analysis based on three base
models across both in-domain and out-of-domain benchmarks validate the
effectiveness of S^2R. Our code and data are available at
https://github.com/NineAbyss/S2R.
Unstructured Evidence Attribution for Long Context Query Focused
  Summarization
Authors: name: Dustin Wright, name: Zain Muhammad Mujahid, name: Lu Wang, name: Isabelle Augenstein, name: David Jurgens
Published: 2025-02-20 09:57:42

Large language models (LLMs) are capable of generating coherent summaries
from very long contexts given a user query. Extracting and properly citing
evidence spans could help improve the transparency and reliability of these
summaries. At the same time, LLMs suffer from positional biases in terms of
which information they understand and attend to, which could affect evidence
citation. Whereas previous work has focused on evidence citation with
predefined levels of granularity (e.g. sentence, paragraph, document, etc.), we
propose the task of long-context query focused summarization with unstructured
evidence citation. We show how existing systems struggle to generate and
properly cite unstructured evidence from their context, and that evidence tends
to be "lost-in-the-middle". To help mitigate this, we create the Summaries with
Unstructured Evidence Text dataset (SUnsET), a synthetic dataset generated
using a novel domain-agnostic pipeline which can be used as supervision to
adapt LLMs to this task. We demonstrate across 5 LLMs of different sizes and 4
datasets with varying document types and lengths that LLMs adapted with SUnsET
data generate more relevant and factually consistent evidence than their base
models, extract evidence from more diverse locations in their context, and can
generate more relevant and consistent summaries.
Geolocation with Real Human Gameplay Data: A Large-Scale Dataset and
  Human-Like Reasoning Framework
Authors: name: Zirui Song, name: Jingpu Yang, name: Yuan Huang, name: Jonathan Tonglet, name: Zeyu Zhang, name: Tao Cheng, name: Meng Fang, name: Iryna Gurevych, name: Xiuying Chen
Published: 2025-02-19 14:21:25

Geolocation, the task of identifying an image's location, requires complex
reasoning and is crucial for navigation, monitoring, and cultural preservation.
However, current methods often produce coarse, imprecise, and non-interpretable
localization. A major challenge lies in the quality and scale of existing
geolocation datasets. These datasets are typically small-scale and
automatically constructed, leading to noisy data and inconsistent task
difficulty, with images that either reveal answers too easily or lack
sufficient clues for reliable inference. To address these challenges, we
introduce a comprehensive geolocation framework with three key components:
GeoComp, a large-scale dataset; GeoCoT, a novel reasoning method; and GeoEval,
an evaluation metric, collectively designed to address critical challenges and
drive advancements in geolocation research. At the core of this framework is
GeoComp (Geolocation Competition Dataset), a large-scale dataset collected from
a geolocation game platform involving 740K users over two years. It comprises
25 million entries of metadata and 3 million geo-tagged locations spanning much
of the globe, with each location annotated thousands to tens of thousands of
times by human users. The dataset offers diverse difficulty levels for detailed
analysis and highlights key gaps in current models. Building on this dataset,
we propose Geographical Chain-of-Thought (GeoCoT), a novel multi-step reasoning
framework designed to enhance the reasoning capabilities of Large Vision Models
(LVMs) in geolocation tasks. GeoCoT improves performance by integrating
contextual and spatial cues through a multi-step process that mimics human
geolocation reasoning. Finally, using the GeoEval metric, we demonstrate that
GeoCoT significantly boosts geolocation accuracy by up to 25% while enhancing
interpretability.
Discovering highly efficient low-weight quantum error-correcting codes
  with reinforcement learning
Authors: name: Austin Yubo He, name: Zi-Wen Liu
Published: 2025-02-20 09:05:34

The realization of scalable fault-tolerant quantum computing is expected to
hinge on quantum error-correcting codes. In the quest for more efficient
quantum fault tolerance, a critical code parameter is the weight of
measurements that extract information about errors to enable error correction:
as higher measurement weights require higher implementation costs and introduce
more errors, it is important in code design to optimize measurement weight.
This underlies the surging interest in quantum low-density parity-check (qLDPC)
codes, the study of which has primarily focused on the asymptotic
(large-code-limit) properties. In this work, we introduce a versatile and
computationally efficient approach to stabilizer code weight reduction based on
reinforcement learning (RL), which produces new low-weight codes that
substantially outperform the state of the art in practically relevant parameter
regimes, extending significantly beyond previously accessible small distances.
For example, our approach demonstrates savings in physical qubit overhead
compared to existing results by 1 to 2 orders of magnitude for weight 6 codes
and brings the overhead into a feasible range for near-future experiments. We
also investigate the interplay between code parameters using our RL framework,
offering new insights into the potential efficiency and power of practically
viable coding strategies. Overall, our results demonstrate how RL can
effectively advance the crucial yet challenging problem of quantum code
discovery and thereby facilitate a faster path to the practical implementation
of fault-tolerant quantum technologies.
Does Time Have Its Place? Temporal Heads: Where Language Models Recall
  Time-specific Information
Authors: name: Yein Park, name: Chanwoong Yoon, name: Jungwoo Park, name: Minbyul Jeong, name: Jaewoo Kang
Published: 2025-02-20 04:52:05

While the ability of language models to elicit facts has been widely
investigated, how they handle temporally changing facts remains underexplored.
We discover Temporal Heads, specific attention heads primarily responsible for
processing temporal knowledge through circuit analysis. We confirm that these
heads are present across multiple models, though their specific locations may
vary, and their responses differ depending on the type of knowledge and its
corresponding years. Disabling these heads degrades the model's ability to
recall time-specific knowledge while maintaining its general capabilities
without compromising time-invariant and question-answering performances.
Moreover, the heads are activated not only numeric conditions ("In 2004") but
also textual aliases ("In the year ..."), indicating that they encode a
temporal dimension beyond simple numerical representation. Furthermore, we
expand the potential of our findings by demonstrating how temporal knowledge
can be edited by adjusting the values of these heads.
Dynamic Concepts Personalization from Single Videos
Authors: name: Rameen Abdal, name: Or Patashnik, name: Ivan Skorokhodov, name: Willi Menapace, name: Aliaksandr Siarohin, name: Sergey Tulyakov, name: Daniel Cohen-Or, name: Kfir Aberman
Published: 2025-02-20 18:53:39

Personalizing generative text-to-image models has seen remarkable progress,
but extending this personalization to text-to-video models presents unique
challenges. Unlike static concepts, personalizing text-to-video models has the
potential to capture dynamic concepts, i.e., entities defined not only by their
appearance but also by their motion. In this paper, we introduce
Set-and-Sequence, a novel framework for personalizing Diffusion Transformers
(DiTs)-based generative video models with dynamic concepts. Our approach
imposes a spatio-temporal weight space within an architecture that does not
explicitly separate spatial and temporal features. This is achieved in two key
stages. First, we fine-tune Low-Rank Adaptation (LoRA) layers using an
unordered set of frames from the video to learn an identity LoRA basis that
represents the appearance, free from temporal interference. In the second
stage, with the identity LoRAs frozen, we augment their coefficients with
Motion Residuals and fine-tune them on the full video sequence, capturing
motion dynamics. Our Set-and-Sequence framework results in a spatio-temporal
weight space that effectively embeds dynamic concepts into the video model's
output domain, enabling unprecedented editability and compositionality while
setting a new benchmark for personalizing dynamic concepts.
PC-Agent: A Hierarchical Multi-Agent Collaboration Framework for Complex
  Task Automation on PC
Authors: name: Haowei Liu, name: Xi Zhang, name: Haiyang Xu, name: Yuyang Wanyan, name: Junyang Wang, name: Ming Yan, name: Ji Zhang, name: Chunfeng Yuan, name: Changsheng Xu, name: Weiming Hu, name: Fei Huang
Published: 2025-02-20 05:41:55

In the field of MLLM-based GUI agents, compared to smartphones, the PC
scenario not only features a more complex interactive environment, but also
involves more intricate intra- and inter-app workflows. To address these
issues, we propose a hierarchical agent framework named PC-Agent. Specifically,
from the perception perspective, we devise an Active Perception Module (APM) to
overcome the inadequate abilities of current MLLMs in perceiving screenshot
content. From the decision-making perspective, to handle complex user
instructions and interdependent subtasks more effectively, we propose a
hierarchical multi-agent collaboration architecture that decomposes
decision-making processes into Instruction-Subtask-Action levels. Within this
architecture, three agents (i.e., Manager, Progress and Decision) are set up
for instruction decomposition, progress tracking and step-by-step
decision-making respectively. Additionally, a Reflection agent is adopted to
enable timely bottom-up error feedback and adjustment. We also introduce a new
benchmark PC-Eval with 25 real-world complex instructions. Empirical results on
PC-Eval show that our PC-Agent achieves a 32% absolute improvement of task
success rate over previous state-of-the-art methods. The code will be publicly
available.