Orionnnn/hub-stats huggingface dataset:
Analyzing LLMs' Knowledge Boundary Cognition Across Languages Through
  the Lens of Internal Representations
Authors: name: Chenghao Xiao, name: Hou Pong Chan, name: Hao Zhang, name: Mahani Aljunied, name: Lidong Bing, name: Noura Al Moubayed, name: Yu Rong
Published: 2025-04-18 17:44:12

While understanding the knowledge boundaries of LLMs is crucial to prevent
hallucination, research on knowledge boundaries of LLMs has predominantly
focused on English. In this work, we present the first study to analyze how
LLMs recognize knowledge boundaries across different languages by probing their
internal representations when processing known and unknown questions in
multiple languages. Our empirical studies reveal three key findings: 1) LLMs'
perceptions of knowledge boundaries are encoded in the middle to middle-upper
layers across different languages. 2) Language differences in knowledge
boundary perception follow a linear structure, which motivates our proposal of
a training-free alignment method that effectively transfers knowledge boundary
perception ability across languages, thereby helping reduce hallucination risk
in low-resource languages; 3) Fine-tuning on bilingual question pair
translation further enhances LLMs' recognition of knowledge boundaries across
languages. Given the absence of standard testbeds for cross-lingual knowledge
boundary analysis, we construct a multilingual evaluation suite comprising
three representative types of knowledge boundary data. Our code and datasets
are publicly available at
https://github.com/DAMO-NLP-SG/LLM-Multilingual-Knowledge-Boundaries.
Generative AI Act II: Test Time Scaling Drives Cognition Engineering
Authors: name: Shijie Xia, name: Yiwei Qin, name: Xuefeng Li, name: Yan Ma, name: Run-Ze Fan, name: Steffi Chern, name: Haoyang Zou, name: Fan Zhou, name: Xiangkun Hu, name: Jiahe Jin, name: Yanheng He, name: Yixin Ye, name: Yixiu Liu, name: Pengfei Liu
Published: 2025-04-18 17:55:58

The first generation of Large Language Models - what might be called "Act I"
of generative AI (2020-2023) - achieved remarkable success through massive
parameter and data scaling, yet exhibited fundamental limitations in knowledge
latency, shallow reasoning, and constrained cognitive processes. During this
era, prompt engineering emerged as our primary interface with AI, enabling
dialogue-level communication through natural language. We now witness the
emergence of "Act II" (2024-present), where models are transitioning from
knowledge-retrieval systems (in latent space) to thought-construction engines
through test-time scaling techniques. This new paradigm establishes a
mind-level connection with AI through language-based thoughts. In this paper,
we clarify the conceptual foundations of cognition engineering and explain why
this moment is critical for its development. We systematically break down these
advanced approaches through comprehensive tutorials and optimized
implementations, democratizing access to cognition engineering and enabling
every practitioner to participate in AI's second act. We provide a regularly
updated collection of papers on test-time scaling in the GitHub Repository:
https://github.com/GAIR-NLP/cognition-engineering
CLASH: Evaluating Language Models on Judging High-Stakes Dilemmas from
  Multiple Perspectives
Authors: name: Ayoung Lee, name: Ryan Sungmo Kwon, name: Peter Railton, name: Lu Wang
Published: 2025-04-15 02:54:16

Navigating high-stakes dilemmas involving conflicting values is challenging
even for humans, let alone for AI. Yet prior work in evaluating the reasoning
capabilities of large language models (LLMs) in such situations has been
limited to everyday scenarios. To close this gap, this work first introduces
CLASH (Character perspective-based LLM Assessments in Situations with
High-stakes), a meticulously curated dataset consisting of 345 high-impact
dilemmas along with 3,795 individual perspectives of diverse values. In
particular, we design CLASH in a way to support the study of critical aspects
of value-based decision-making processes which are missing from prior work,
including understanding decision ambivalence and psychological discomfort as
well as capturing the temporal shifts of values in characters' perspectives. By
benchmarking 10 open and closed frontier models, we uncover several key
findings. (1) Even the strongest models, such as GPT-4o and Claude-Sonnet,
achieve less than 50% accuracy in identifying situations where the decision
should be ambivalent, while they perform significantly better in clear-cut
scenarios. (2) While LLMs reasonably predict psychological discomfort as marked
by human, they inadequately comprehend perspectives involving value shifts,
indicating a need for LLMs to reason over complex values. (3) Our experiments
also reveal a significant correlation between LLMs' value preferences and their
steerability towards a given value. (4) Finally, LLMs exhibit greater
steerability when engaged in value reasoning from a third-party perspective,
compared to a first-person setup, though certain value pairs benefit uniquely
from the first-person framing.
Thought Manipulation: External Thought Can Be Efficient for Large
  Reasoning Models
Authors: name: Yule Liu, name: Jingyi Zheng, name: Zhen Sun, name: Zifan Peng, name: Wenhan Dong, name: Zeyang Sha, name: Shiwen Cui, name: Weiqiang Wang, name: Xinlei He
Published: 2025-04-18 11:07:19

Recent advancements in large reasoning models (LRMs) have demonstrated the
effectiveness of scaling test-time computation to enhance reasoning
capabilities in multiple tasks. However, LRMs typically suffer from
"overthinking" problems, where models generate significantly redundant
reasoning steps while bringing limited performance gains. Existing work relies
on fine-tuning to mitigate overthinking, which requires additional data,
unconventional training setups, risky safety misalignment, and poor
generalization.
  Through empirical analysis, we reveal an important characteristic of LRM
behaviors that placing external CoTs generated by smaller models between the
thinking token (<think> and </think>) can effectively
manipulate the model to generate fewer thoughts. Building on these insights, we
propose a simple yet efficient pipeline, ThoughtMani, to enable LRMs to bypass
unnecessary intermediate steps and reduce computational costs significantly. We
conduct extensive experiments to validate the utility and efficiency of
ThoughtMani. For instance, when applied to QwQ-32B on the LiveBench/Code
dataset, ThoughtMani keeps the original performance and reduces output token
counts by approximately 30%, with little overhead from the CoT generator.
Furthermore, we find that ThoughtMani enhances safety alignment by an average
of 10%. Since model vendors typically serve models of different sizes
simultaneously, ThoughtMani provides an effective way to construct more
efficient and accessible LRMs for real-world applications.
Tokenize Image Patches: Global Context Fusion for Effective Haze Removal
  in Large Images
Authors: name: Jiuchen Chen, name: Xinyu Yan, name: Qizhi Xu, name: Kaiqi Li
Published: 2025-04-13 15:41:25

Global contextual information and local detail features are essential for
haze removal tasks. Deep learning models perform well on small, low-resolution
images, but they encounter difficulties with large, high-resolution ones due to
GPU memory limitations. As a compromise, they often resort to image slicing or
downsampling. The former diminishes global information, while the latter
discards high-frequency details. To address these challenges, we propose
DehazeXL, a haze removal method that effectively balances global context and
local feature extraction, enabling end-to-end modeling of large images on
mainstream GPU hardware. Additionally, to evaluate the efficiency of global
context utilization in haze removal performance, we design a visual attribution
method tailored to the characteristics of haze removal tasks. Finally,
recognizing the lack of benchmark datasets for haze removal in large images, we
have developed an ultra-high-resolution haze removal dataset (8KDehaze) to
support model training and testing. It includes 10000 pairs of clear and hazy
remote sensing images, each sized at 8192 times 8192 pixels. Extensive
experiments demonstrate that DehazeXL can infer images up to 10240 times
10240 pixels with only 21 GB of memory, achieving state-of-the-art results
among all evaluated methods. The source code and experimental dataset are
available at https://github.com/CastleChen339/DehazeXL.
HiScene: Creating Hierarchical 3D Scenes with Isometric View Generation
Authors: name: Wenqi Dong, name: Bangbang Yang, name: Zesong Yang, name: Yuan Li, name: Tao Hu, name: Hujun Bao, name: Yuewen Ma, name: Zhaopeng Cui
Published: 2025-04-17 16:33:39

Scene-level 3D generation represents a critical frontier in multimedia and
computer graphics, yet existing approaches either suffer from limited object
categories or lack editing flexibility for interactive applications. In this
paper, we present HiScene, a novel hierarchical framework that bridges the gap
between 2D image generation and 3D object generation and delivers high-fidelity
scenes with compositional identities and aesthetic scene content. Our key
insight is treating scenes as hierarchical "objects" under isometric views,
where a room functions as a complex object that can be further decomposed into
manipulatable items. This hierarchical approach enables us to generate 3D
content that aligns with 2D representations while maintaining compositional
structure. To ensure completeness and spatial alignment of each decomposed
instance, we develop a video-diffusion-based amodal completion technique that
effectively handles occlusions and shadows between objects, and introduce shape
prior injection to ensure spatial coherence within the scene. Experimental
results demonstrate that our method produces more natural object arrangements
and complete object instances suitable for interactive applications, while
maintaining physical plausibility and alignment with user inputs.
It's All Connected: A Journey Through Test-Time Memorization,
  Attentional Bias, Retention, and Online Optimization
Authors: name: Ali Behrouz, name: Meisam Razaviyayn, name: Peilin Zhong, name: Vahab Mirrokni
Published: 2025-04-17 17:59:33

Designing efficient and effective architectural backbones has been in the
core of research efforts to enhance the capability of foundation models.
Inspired by the human cognitive phenomenon of attentional bias-the natural
tendency to prioritize certain events or stimuli-we reconceptualize neural
architectures, including Transformers, Titans, and modern linear recurrent
neural networks as associative memory modules that learn a mapping of keys and
values using an internal objective, referred to as attentional bias.
Surprisingly, we observed that most existing sequence models leverage either
(1) dot-product similarity, or (2) L2 regression objectives as their
attentional bias. Going beyond these objectives, we present a set of
alternative attentional bias configurations along with their effective
approximations to stabilize their training procedure. We then reinterpret
forgetting mechanisms in modern deep learning architectures as a form of
retention regularization, providing a novel set of forget gates for sequence
models. Building upon these insights, we present Miras, a general framework to
design deep learning architectures based on four choices of: (i) associative
memory architecture, (ii) attentional bias objective, (iii) retention gate, and
(iv) memory learning algorithm. We present three novel sequence models-Moneta,
Yaad, and Memora-that go beyond the power of existing linear RNNs while
maintaining a fast parallelizable training process. Our experiments show
different design choices in Miras yield models with varying strengths. For
example, certain instances of Miras achieve exceptional performance in special
tasks such as language modeling, commonsense reasoning, and recall intensive
tasks, even outperforming Transformers and other modern linear recurrent
models.
Could Thinking Multilingually Empower LLM Reasoning?
Authors: name: Changjiang Gao, name: Xu Huang, name: Wenhao Zhu, name: Shujian Huang, name: Lei Li, name: Fei Yuan
Published: 2025-04-16 07:45:10

Previous work indicates that large language models exhibit a significant
"English bias", i.e. they often perform better when tasks are presented in
English. Interestingly, we have observed that using certain other languages in
reasoning tasks can yield better performance than English. However, this
phenomenon remains under-explored. In this paper, we explore the upper bound of
harnessing multilingualism in reasoning tasks, suggesting that multilingual
reasoning promises significantly (by nearly 10 Acc@k points) and robustly
(tolerance for variations in translation quality and language choice) higher
upper bounds than English-only reasoning. Besides analyzing the reason behind
the upper bound and challenges in reaching it, we also find that common answer
selection methods cannot achieve this upper bound, due to their limitations and
biases. These insights could pave the way for future research aimed at fully
harnessing the potential of multilingual reasoning in LLMs.
Does Reinforcement Learning Really Incentivize Reasoning Capacity in
  LLMs Beyond the Base Model?
Authors: name: Yang Yue, name: Zhiqi Chen, name: Rui Lu, name: Andrew Zhao, name: Zhaokai Wang, name: Yang Yue, name: Shiji Song, name: Gao Huang
Published: 2025-04-18 17:59:56

Reinforcement Learning with Verifiable Rewards (RLVR) has recently
demonstrated notable success in enhancing the reasoning capabilities of LLMs,
particularly in mathematics and programming tasks. It is widely believed that
RLVR enables LLMs to continuously self-improve, thus acquiring novel reasoning
abilities that exceed corresponding base models' capacity. In this study,
however, we critically re-examines this assumption by measuring the
pass@k metric with large values of k to explore the reasoning
capability boundary of the models across a wide range of model families and
benchmarks. Surprisingly, the RL does not, in fact, elicit fundamentally
new reasoning patterns. While RL-trained models outperform their base models at
smaller values of k (\eg, k=1), base models can achieve a comparable or
even higher pass@k score compared to their RL counterparts at large k
values. The reasoning paths generated by RL-trained models are already included
in the base models' sampling distribution, suggesting that most reasoning
abilities manifested in RL-trained models are already obtained by base models.
Further analysis shows that RL training boosts the performance by biasing the
model's output distribution toward paths that are more likely to yield rewards,
therefore sampling correct responses more efficiently. But this also results in
a narrower reasoning capability boundary compared to base models. Similar
results are observed in visual reasoning tasks trained with RLVR. Moreover, we
find that distillation can genuinely introduce new knowledge into the model,
different from RLVR. These findings underscore a critical limitation of RLVR in
advancing LLM reasoning abilities which requires us to fundamentally rethink
the impact of RL training in reasoning LLMs and the need of a better paradigm.
Project Page: https://limit-of-RLVR.github.io
MIG: Automatic Data Selection for Instruction Tuning by Maximizing
  Information Gain in Semantic Space
Authors: name: Yicheng Chen, name: Yining Li, name: Kai Hu, name: Zerun Ma, name: Haochen Ye, name: Kai Chen
Published: 2025-04-18 17:59:46

Data quality and diversity are key to the construction of effective
instruction-tuning datasets. % With the increasing availability of open-source
instruction-tuning datasets, it is advantageous to automatically select
high-quality and diverse subsets from a vast amount of data. % Existing methods
typically prioritize instance quality and use heuristic rules to maintain
diversity. % However, this absence of a comprehensive view of the entire
collection often leads to suboptimal results. % Moreover, heuristic rules
generally focus on distance or clustering within the embedding space, which
fails to accurately capture the intent of complex instructions in the semantic
space. % To bridge this gap, we propose a unified method for quantifying the
information content of datasets. This method models the semantic space by
constructing a label graph and quantifies diversity based on the distribution
of information within the graph. % Based on such a measurement, we further
introduce an efficient sampling method that selects data samples iteratively to
Maximize the Information Gain (MIG) in semantic
space. % Experiments on various datasets and base models demonstrate that MIG
consistently outperforms state-of-the-art methods. % Notably, the model
fine-tuned with 5\% Tulu3 data sampled by MIG achieves comparable performance
to the official SFT model trained on the full dataset, with improvements of
+5.73\% on AlpacaEval and +6.89\% on Wildbench.
NodeRAG: Structuring Graph-based RAG with Heterogeneous Nodes
Authors: name: Tianyang Xu, name: Haojie Zheng, name: Chengze Li, name: Haoxiang Chen, name: Yixin Liu, name: Ruoxi Chen, name: Lichao Sun
Published: 2025-04-15 18:24:00

Retrieval-augmented generation (RAG) empowers large language models to access
external and private corpus, enabling factually consistent responses in
specific domains. By exploiting the inherent structure of the corpus,
graph-based RAG methods further enrich this process by building a knowledge
graph index and leveraging the structural nature of graphs. However, current
graph-based RAG approaches seldom prioritize the design of graph structures.
Inadequately designed graph not only impede the seamless integration of diverse
graph algorithms but also result in workflow inconsistencies and degraded
performance. To further unleash the potential of graph for RAG, we propose
NodeRAG, a graph-centric framework introducing heterogeneous graph structures
that enable the seamless and holistic integration of graph-based methodologies
into the RAG workflow. By aligning closely with the capabilities of LLMs, this
framework ensures a fully cohesive and efficient end-to-end process. Through
extensive experiments, we demonstrate that NodeRAG exhibits performance
advantages over previous methods, including GraphRAG and LightRAG, not only in
indexing time, query time, and storage efficiency but also in delivering
superior question-answering performance on multi-hop benchmarks and open-ended
head-to-head evaluations with minimal retrieval tokens. Our GitHub repository
could be seen at https://github.com/Terry-Xu-666/NodeRAG.
AerialMegaDepth: Learning Aerial-Ground Reconstruction and View
  Synthesis
Authors: name: Khiem Vuong, name: Anurag Ghosh, name: Deva Ramanan, name: Srinivasa Narasimhan, name: Shubham Tulsiani
Published: 2025-04-17 17:57:05

We explore the task of geometric reconstruction of images captured from a
mixture of ground and aerial views. Current state-of-the-art learning-based
approaches fail to handle the extreme viewpoint variation between aerial-ground
image pairs. Our hypothesis is that the lack of high-quality, co-registered
aerial-ground datasets for training is a key reason for this failure. Such data
is difficult to assemble precisely because it is difficult to reconstruct in a
scalable way. To overcome this challenge, we propose a scalable framework
combining pseudo-synthetic renderings from 3D city-wide meshes (e.g., Google
Earth) with real, ground-level crowd-sourced images (e.g., MegaDepth). The
pseudo-synthetic data simulates a wide range of aerial viewpoints, while the
real, crowd-sourced images help improve visual fidelity for ground-level images
where mesh-based renderings lack sufficient detail, effectively bridging the
domain gap between real images and pseudo-synthetic renderings. Using this
hybrid dataset, we fine-tune several state-of-the-art algorithms and achieve
significant improvements on real-world, zero-shot aerial-ground tasks. For
example, we observe that baseline DUSt3R localizes fewer than 5% of
aerial-ground pairs within 5 degrees of camera rotation error, while
fine-tuning with our data raises accuracy to nearly 56%, addressing a major
failure point in handling large viewpoint changes. Beyond camera estimation and
scene reconstruction, our dataset also improves performance on downstream tasks
like novel-view synthesis in challenging aerial-ground scenarios, demonstrating
the practical value of our approach in real-world applications.
Learning Occlusion-Robust Vision Transformers for Real-Time UAV Tracking
Authors: name: You Wu, name: Xucheng Wang, name: Xiangyang Yang, name: Mengyuan Liu, name: Dan Zeng, name: Hengzhou Ye, name: Shuiwang Li
Published: 2025-04-12 14:06:50

Single-stream architectures using Vision Transformer (ViT) backbones show
great potential for real-time UAV tracking recently. However, frequent
occlusions from obstacles like buildings and trees expose a major drawback:
these models often lack strategies to handle occlusions effectively. New
methods are needed to enhance the occlusion resilience of single-stream ViT
models in aerial tracking. In this work, we propose to learn Occlusion-Robust
Representations (ORR) based on ViTs for UAV tracking by enforcing an invariance
of the feature representation of a target with respect to random masking
operations modeled by a spatial Cox process. Hopefully, this random masking
approximately simulates target occlusions, thereby enabling us to learn ViTs
that are robust to target occlusion for UAV tracking. This framework is termed
ORTrack. Additionally, to facilitate real-time applications, we propose an
Adaptive Feature-Based Knowledge Distillation (AFKD) method to create a more
compact tracker, which adaptively mimics the behavior of the teacher model
ORTrack according to the task's difficulty. This student model, dubbed
ORTrack-D, retains much of ORTrack's performance while offering higher
efficiency. Extensive experiments on multiple benchmarks validate the
effectiveness of our method, demonstrating its state-of-the-art performance.
Codes is available at https://github.com/wuyou3474/ORTrack.
Complex-Edit: CoT-Like Instruction Generation for
  Complexity-Controllable Image Editing Benchmark
Authors: name: Siwei Yang, name: Mude Hui, name: Bingchen Zhao, name: Yuyin Zhou, name: Nataniel Ruiz, name: Cihang Xie
Published: 2025-04-17 17:51:59

We introduce Complex-Edit, a comprehensive benchmark designed to
systematically evaluate instruction-based image editing models across
instructions of varying complexity. To develop this benchmark, we harness
GPT-4o to automatically collect a diverse set of editing instructions at scale.
Our approach follows a well-structured ``Chain-of-Edit'' pipeline: we first
generate individual atomic editing tasks independently and then integrate them
to form cohesive, complex instructions. Additionally, we introduce a suite of
metrics to assess various aspects of editing performance, along with a
VLM-based auto-evaluation pipeline that supports large-scale assessments. Our
benchmark yields several notable insights: 1) Open-source models significantly
underperform relative to proprietary, closed-source models, with the
performance gap widening as instruction complexity increases; 2) Increased
instructional complexity primarily impairs the models' ability to retain key
elements from the input images and to preserve the overall aesthetic quality;
3) Decomposing a complex instruction into a sequence of atomic steps, executed
in a step-by-step manner, substantially degrades performance across multiple
metrics; 4) A straightforward Best-of-N selection strategy improves results for
both direct editing and the step-by-step sequential approach; and 5) We observe
a ``curse of synthetic data'': when synthetic data is involved in model
training, the edited images from such models tend to appear increasingly
synthetic as the complexity of the editing instructions rises -- a phenomenon
that intriguingly also manifests in the latest GPT-4o outputs.