Orionnnn/hub-stats huggingface dataset:
Diverse Controllable Diffusion Policy with Signal Temporal Logic
Authors: name: Yue Meng, name: Chuchu fan
Published: 2025-03-04 18:59:00

Generating realistic simulations is critical for autonomous system
applications such as self-driving and human-robot interactions. However,
driving simulators nowadays still have difficulty in generating controllable,
diverse, and rule-compliant behaviors for road participants: Rule-based models
cannot produce diverse behaviors and require careful tuning, whereas
learning-based methods imitate the policy from data but are not designed to
follow the rules explicitly. Besides, the real-world datasets are by nature
"single-outcome", making the learning method hard to generate diverse
behaviors. In this paper, we leverage Signal Temporal Logic (STL) and Diffusion
Models to learn controllable, diverse, and rule-aware policy. We first
calibrate the STL on the real-world data, then generate diverse synthetic data
using trajectory optimization, and finally learn the rectified diffusion policy
on the augmented dataset. We test on the NuScenes dataset and our approach can
achieve the most diverse rule-compliant trajectories compared to other
baselines, with a runtime 1/17X to the second-best approach. In the closed-loop
testing, our approach reaches the highest diversity, rule satisfaction rate,
and the least collision rate. Our method can generate varied characteristics
conditional on different STL parameters in testing. A case study on human-robot
encounter scenarios shows our approach can generate diverse and
closed-to-oracle trajectories. The annotation tool, augmented dataset, and code
are available at https://github.com/mengyuest/pSTL-diffusion-policy.
Remasking Discrete Diffusion Models with Inference-Time Scaling
Authors: name: Guanghan Wang, name: Yair Schiff, name: Subham Sekhar Sahoo, name: Volodymyr Kuleshov
Published: 2025-03-01 02:37:51

Part of the success of diffusion models stems from their ability to perform
iterative refinement, i.e., repeatedly correcting outputs during generation.
However, modern masked discrete diffusion lacks this capability: when a token
is generated, it cannot be updated again, even when it introduces an error.
Here, we address this limitation by introducing the remasking diffusion model
(ReMDM) sampler, a method that can be applied to pretrained masked diffusion
models in a principled way and that is derived from a discrete diffusion model
with a custom remasking backward process. Most interestingly, ReMDM endows
discrete diffusion with a form of inference-time compute scaling. By increasing
the number of sampling steps, ReMDM generates natural language outputs that
approach the quality of autoregressive models, whereas when the computation
budget is limited, ReMDM better maintains quality. ReMDM also improves sample
quality of masked diffusion models for discretized images, and in scientific
domains such as molecule design, ReMDM facilitates diffusion guidance and
pushes the Pareto frontier of controllability relative to classical masking and
uniform noise diffusion. We provide the code along with a blog post on the
project page: https://remdm.github.io.
Process-based Self-Rewarding Language Models
Authors: name: Shimao Zhang, name: Xiao Liu, name: Xin Zhang, name: Junxiao Liu, name: Zheheng Luo, name: Shujian Huang, name: Yeyun Gong
Published: 2025-03-05 18:58:44

Large Language Models have demonstrated outstanding performance across
various downstream tasks and have been widely applied in multiple scenarios.
Human-annotated preference data is used for training to further improve LLMs'
performance, which is constrained by the upper limit of human performance.
Therefore, Self-Rewarding method has been proposed, where LLMs generate
training data by rewarding their own outputs. However, the existing
self-rewarding paradigm is not effective in mathematical reasoning scenarios
and may even lead to a decline in performance. In this work, we propose the
Process-based Self-Rewarding pipeline for language models, which introduces
long-thought reasoning, step-wise LLM-as-a-Judge, and step-wise preference
optimization within the self-rewarding paradigm. Our new paradigm successfully
enhances the performance of LLMs on multiple mathematical reasoning benchmarks
through iterative Process-based Self-Rewarding, demonstrating the immense
potential of self-rewarding to achieve LLM reasoning that may surpass human
capabilities.
Fine-Tuning Small Language Models for Domain-Specific AI: An Edge AI
  Perspective
Authors: name: Rakshit Aralimatti, name: Syed Abdul Gaffar Shakhadri, name: Kruthika KR, name: Kartik Basavaraj Angadi
Published: 2025-03-03 04:53:55

Deploying large scale language models on edge devices faces inherent
challenges such as high computational demands, energy consumption, and
potential data privacy risks. This paper introduces the Shakti Small Language
Models (SLMs) Shakti-100M, Shakti-250M, and Shakti-500M which target these
constraints headon. By combining efficient architectures, quantization
techniques, and responsible AI principles, the Shakti series enables on-device
intelligence for smartphones, smart appliances, IoT systems, and beyond. We
provide comprehensive insights into their design philosophy, training
pipelines, and benchmark performance on both general tasks (e.g., MMLU,
Hellaswag) and specialized domains (healthcare, finance, and legal). Our
findings illustrate that compact models, when carefully engineered and
fine-tuned, can meet and often exceed expectations in real-world edge-AI
scenarios.
Mixture of Structural-and-Textual Retrieval over Text-rich Graph
  Knowledge Bases
Authors: name: Yongjia Lei, name: Haoyu Han, name: Ryan A. Rossi, name: Franck Dernoncourt, name: Nedim Lipka, name: Mahantesh M Halappanavar, name: Jiliang Tang, name: Yu Wang
Published: 2025-02-27 17:42:52

Text-rich Graph Knowledge Bases (TG-KBs) have become increasingly crucial for
answering queries by providing textual and structural knowledge. However,
current retrieval methods often retrieve these two types of knowledge in
isolation without considering their mutual reinforcement and some hybrid
methods even bypass structural retrieval entirely after neighboring
aggregation. To fill in this gap, we propose a Mixture of
Structural-and-Textual Retrieval (MoR) to retrieve these two types of knowledge
via a Planning-Reasoning-Organizing framework. In the Planning stage, MoR
generates textual planning graphs delineating the logic for answering queries.
Following planning graphs, in the Reasoning stage, MoR interweaves structural
traversal and textual matching to obtain candidates from TG-KBs. In the
Organizing stage, MoR further reranks fetched candidates based on their
structural trajectory. Extensive experiments demonstrate the superiority of MoR
in harmonizing structural and textual retrieval with insights, including uneven
retrieving performance across different query logics and the benefits of
integrating structural trajectories for candidate reranking. Our code is
available at https://github.com/Yoega/MoR.
Exploring Rewriting Approaches for Different Conversational Tasks
Authors: name: Md Mehrab Tanjim, name: Ryan A. Rossi, name: Mike Rimer, name: Xiang Chen, name: Sungchul Kim, name: Vaishnavi Muppala, name: Tong Yu, name: Zhengmian Hu, name: Ritwik Sinha, name: Wei Zhang, name: Iftikhar Ahamath Burhanuddin, name: Franck Dernoncourt
Published: 2025-02-26 06:05:29

Conversational assistants often require a question rewriting algorithm that
leverages a subset of past interactions to provide a more meaningful (accurate)
answer to the user's question or request. However, the exact rewriting approach
may often depend on the use case and application-specific tasks supported by
the conversational assistant, among other constraints. In this paper, we
systematically investigate two different approaches, denoted as rewriting and
fusion, on two fundamentally different generation tasks, including a
text-to-text generation task and a multimodal generative task that takes as
input text and generates a visualization or data table that answers the user's
question. Our results indicate that the specific rewriting or fusion approach
highly depends on the underlying use case and generative task. In particular,
we find that for a conversational question-answering assistant, the query
rewriting approach performs best, whereas for a data analysis assistant that
generates visualizations and data tables based on the user's conversation with
the assistant, the fusion approach works best. Notably, we explore two datasets
for the data analysis assistant use case, for short and long conversations, and
we find that query fusion always performs better, whereas for the
conversational text-based question-answering, the query rewrite approach
performs best.
QE4PE: Word-level Quality Estimation for Human Post-Editing
Authors: name: Gabriele Sarti, name: Vilém Zouhar, name: Grzegorz Chrupała, name: Ana Guerberof-Arenas, name: Malvina Nissim, name: Arianna Bisazza
Published: 2025-03-04 22:50:17

Word-level quality estimation (QE) detects erroneous spans in machine
translations, which can direct and facilitate human post-editing. While the
accuracy of word-level QE systems has been assessed extensively, their
usability and downstream influence on the speed, quality and editing choices of
human post-editing remain understudied. Our QE4PE study investigates the impact
of word-level QE on machine translation (MT) post-editing in a realistic
setting involving 42 professional post-editors across two translation
directions. We compare four error-span highlight modalities, including
supervised and uncertainty-based word-level QE methods, for identifying
potential errors in the outputs of a state-of-the-art neural MT model.
Post-editing effort and productivity are estimated by behavioral logs, while
quality improvements are assessed by word- and segment-level human annotation.
We find that domain, language and editors' speed are critical factors in
determining highlights' effectiveness, with modest differences between
human-made and automated QE highlights underlining a gap between accuracy and
usability in professional workflows.
Enhancing Abnormality Grounding for Vision Language Models with
  Knowledge Descriptions
Authors: name: Jun Li, name: Che Liu, name: Wenjia Bai, name: Rossella Arcucci, name: Cosmin I. Bercea, name: Julia A. Schnabel
Published: 2025-03-05 09:02:33

Visual Language Models (VLMs) have demonstrated impressive capabilities in
visual grounding tasks. However, their effectiveness in the medical domain,
particularly for abnormality detection and localization within medical images,
remains underexplored. A major challenge is the complex and abstract nature of
medical terminology, which makes it difficult to directly associate
pathological anomaly terms with their corresponding visual features. In this
work, we introduce a novel approach to enhance VLM performance in medical
abnormality detection and localization by leveraging decomposed medical
knowledge. Instead of directly prompting models to recognize specific
abnormalities, we focus on breaking down medical concepts into fundamental
attributes and common visual patterns. This strategy promotes a stronger
alignment between textual descriptions and visual features, improving both the
recognition and localization of abnormalities in medical images.We evaluate our
method on the 0.23B Florence-2 base model and demonstrate that it achieves
comparable performance in abnormality grounding to significantly larger 7B
LLaVA-based medical VLMs, despite being trained on only 1.5% of the data used
for such models. Experimental results also demonstrate the effectiveness of our
approach in both known and previously unseen abnormalities, suggesting its
strong generalization capabilities.
CrowdSelect: Synthetic Instruction Data Selection with Multi-LLM Wisdom
Authors: name: Yisen Li, name: Lingfeng Yang, name: Wenxuan Shen, name: Pan Zhou, name: Yao Wan, name: Weiwei Lin, name: Dongping Chen
Published: 2025-03-03 18:56:44

Distilling advanced Large Language Models' instruction-following capabilities
into smaller models using a selected subset has become a mainstream approach in
model training. While existing synthetic instruction data selection strategies
rely mainly on single-dimensional signals (i.e., reward scores, model
perplexity), they fail to capture the complexity of instruction-following
across diverse fields. Therefore, we investigate more diverse signals to
capture comprehensive instruction-response pair characteristics and propose
three foundational metrics that leverage Multi-LLM wisdom, informed by (1)
diverse LLM responses and (2) reward model assessment. Building upon base
metrics, we propose CrowdSelect, an integrated metric incorporating a
clustering-based approach to maintain response diversity. Our comprehensive
experiments demonstrate that our foundation metrics consistently improve
performance across 4 base models on MT-bench and Arena-Hard. CrowdSelect,
efficiently incorporating all metrics, achieves state-of-the-art performance in
both Full and LoRA fine-tuning, showing improvements of 4.81% on Arena-Hard and
11.1% on MT-bench with Llama-3.2-3b-instruct. We hope our findings will bring
valuable insights for future research in this direction. Code are available at
https://github.com/listentm/crowdselect.
Retrieval Models Aren't Tool-Savvy: Benchmarking Tool Retrieval for
  Large Language Models
Authors: name: Zhengliang Shi, name: Yuhan Wang, name: Lingyong Yan, name: Pengjie Ren, name: Shuaiqiang Wang, name: Dawei Yin, name: Zhaochun Ren
Published: 2025-03-03 17:37:16

Tool learning aims to augment large language models (LLMs) with diverse
tools, enabling them to act as agents for solving practical tasks. Due to the
limited context length of tool-using LLMs, adopting information retrieval (IR)
models to select useful tools from large toolsets is a critical initial step.
However, the performance of IR models in tool retrieval tasks remains
underexplored and unclear. Most tool-use benchmarks simplify this step by
manually pre-annotating a small set of relevant tools for each task, which is
far from the real-world scenarios. In this paper, we propose ToolRet, a
heterogeneous tool retrieval benchmark comprising 7.6k diverse retrieval tasks,
and a corpus of 43k tools, collected from existing datasets. We benchmark six
types of models on ToolRet. Surprisingly, even the models with strong
performance in conventional IR benchmarks, exhibit poor performance on ToolRet.
This low retrieval quality degrades the task pass rate of tool-use LLMs. As a
further step, we contribute a large-scale training dataset with over 200k
instances, which substantially optimizes the tool retrieval ability of IR
models.
FLAME: A Federated Learning Benchmark for Robotic Manipulation
Authors: name: Santiago Bou Betran, name: Alberta Longhini, name: Miguel Vasco, name: Yuchong Zhang, name: Danica Kragic
Published: 2025-03-03 16:49:15

Recent progress in robotic manipulation has been fueled by large-scale
datasets collected across diverse environments. Training robotic manipulation
policies on these datasets is traditionally performed in a centralized manner,
raising concerns regarding scalability, adaptability, and data privacy. While
federated learning enables decentralized, privacy-preserving training, its
application to robotic manipulation remains largely unexplored. We introduce
FLAME (Federated Learning Across Manipulation Environments), the first
benchmark designed for federated learning in robotic manipulation. FLAME
consists of: (i) a set of large-scale datasets of over 160,000 expert
demonstrations of multiple manipulation tasks, collected across a wide range of
simulated environments; (ii) a training and evaluation framework for robotic
policy learning in a federated setting. We evaluate standard federated learning
algorithms in FLAME, showing their potential for distributed policy learning
and highlighting key challenges. Our benchmark establishes a foundation for
scalable, adaptive, and privacy-aware robotic learning.
Benchmarking Large Language Models for Multi-Language Software
  Vulnerability Detection
Authors: name: Ting Zhang, name: Chengran Yang, name: Yindu Su, name: Martin Weyssow, name: Hung Nguyen, name: Tan Bui, name: Hong Jin Kang, name: Yikun Li, name: Eng Lieh Ouh, name: Lwin Khin Shar, name: David Lo
Published: 2025-03-03 11:56:00

Recent advancements in generative AI have led to the widespread adoption of
large language models (LLMs) in software engineering, addressing numerous
long-standing challenges. However, a comprehensive study examining the
capabilities of LLMs in software vulnerability detection (SVD), a crucial
aspect of software security, is currently lacking. Existing research primarily
focuses on evaluating LLMs using C/C++ datasets. It typically explores only one
or two strategies among prompt engineering, instruction tuning, and sequence
classification fine-tuning for open-source LLMs. Consequently, there is a
significant knowledge gap regarding the effectiveness of diverse LLMs in
detecting vulnerabilities across various programming languages. To address this
knowledge gap, we present a comprehensive empirical study evaluating the
performance of LLMs on the SVD task. We have compiled a comprehensive dataset
comprising 8,260 vulnerable functions in Python, 7,505 in Java, and 28,983 in
JavaScript. We assess five open-source LLMs using multiple approaches,
including prompt engineering, instruction tuning, and sequence classification
fine-tuning. These LLMs are benchmarked against five fine-tuned small language
models and two open-source static application security testing tools.
Furthermore, we explore two avenues to improve LLM performance on SVD: a) Data
perspective: Retraining models using downsampled balanced datasets. b) Model
perspective: Investigating ensemble learning methods that combine predictions
from multiple LLMs. Our comprehensive experiments demonstrate that SVD remains
a challenging task for LLMs. This study provides a thorough understanding of
the role of LLMs in SVD and offers practical insights for future advancements
in leveraging generative AI to enhance software security practices.
CognitiveDrone: A VLA Model and Evaluation Benchmark for Real-Time
  Cognitive Task Solving and Reasoning in UAVs
Authors: name: Artem Lykov, name: Valerii Serpiva, name: Muhammad Haris Khan, name: Oleg Sautenkov, name: Artyom Myshlyaev, name: Grik Tadevosyan, name: Yasheerah Yaqoot, name: Dzmitry Tsetserukou
Published: 2025-03-03 10:21:36

This paper introduces CognitiveDrone, a novel Vision-Language-Action (VLA)
model tailored for complex Unmanned Aerial Vehicles (UAVs) tasks that demand
advanced cognitive abilities. Trained on a dataset comprising over 8,000
simulated flight trajectories across three key categories-Human Recognition,
Symbol Understanding, and Reasoning-the model generates real-time 4D action
commands based on first-person visual inputs and textual instructions. To
further enhance performance in intricate scenarios, we propose
CognitiveDrone-R1, which integrates an additional Vision-Language Model (VLM)
reasoning module to simplify task directives prior to high-frequency control.
Experimental evaluations using our open-source benchmark, CognitiveDroneBench,
reveal that while a racing-oriented model (RaceVLA) achieves an overall success
rate of 31.3%, the base CognitiveDrone model reaches 59.6%, and
CognitiveDrone-R1 attains a success rate of 77.2%. These results demonstrate
improvements of up to 30% in critical cognitive tasks, underscoring the
effectiveness of incorporating advanced reasoning capabilities into UAV control
systems. Our contributions include the development of a state-of-the-art VLA
model for UAV control and the introduction of the first dedicated benchmark for
assessing cognitive tasks in drone operations. The complete repository is
available at cognitivedrone.github.io
SwiLTra-Bench: The Swiss Legal Translation Benchmark
Authors: name: Joel Niklaus, name: Jakob Merane, name: Luka Nenadic, name: Sina Ahmadi, name: Yingqiang Gao, name: Cyrill A. H. Chevalley, name: Claude Humbel, name: Christophe Gösken, name: Lorenzo Tanzi, name: Thomas Lüthi, name: Stefan Palombo, name: Spencer Poff, name: Boling Yang, name: Nan Wu, name: Matthew Guillod, name: Robin Mamié, name: Daniel Brunner, name: Julio Pereyra, name: Niko Grupen
Published: 2025-03-03 10:10:30

In Switzerland legal translation is uniquely important due to the country's
four official languages and requirements for multilingual legal documentation.
However, this process traditionally relies on professionals who must be both
legal experts and skilled translators -- creating bottlenecks and impacting
effective access to justice. To address this challenge, we introduce
SwiLTra-Bench, a comprehensive multilingual benchmark of over 180K aligned
Swiss legal translation pairs comprising laws, headnotes, and press releases
across all Swiss languages along with English, designed to evaluate LLM-based
translation systems. Our systematic evaluation reveals that frontier models
achieve superior translation performance across all document types, while
specialized translation systems excel specifically in laws but under-perform in
headnotes. Through rigorous testing and human expert validation, we demonstrate
that while fine-tuning open SLMs significantly improves their translation
quality, they still lag behind the best zero-shot prompted frontier models such
as Claude-3.5-Sonnet. Additionally, we present SwiLTra-Judge, a specialized LLM
evaluation system that aligns best with human expert assessments.
GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera
  Control
Authors: name: Xuanchi Ren, name: Tianchang Shen, name: Jiahui Huang, name: Huan Ling, name: Yifan Lu, name: Merlin Nimier-David, name: Thomas Müller, name: Alexander Keller, name: Sanja Fidler, name: Jun Gao
Published: 2025-03-05 18:59:50

We present GEN3C, a generative video model with precise Camera Control and
temporal 3D Consistency. Prior video models already generate realistic videos,
but they tend to leverage little 3D information, leading to inconsistencies,
such as objects popping in and out of existence. Camera control, if implemented
at all, is imprecise, because camera parameters are mere inputs to the neural
network which must then infer how the video depends on the camera. In contrast,
GEN3C is guided by a 3D cache: point clouds obtained by predicting the
pixel-wise depth of seed images or previously generated frames. When generating
the next frames, GEN3C is conditioned on the 2D renderings of the 3D cache with
the new camera trajectory provided by the user. Crucially, this means that
GEN3C neither has to remember what it previously generated nor does it have to
infer the image structure from the camera pose. The model, instead, can focus
all its generative power on previously unobserved regions, as well as advancing
the scene state to the next frame. Our results demonstrate more precise camera
control than prior work, as well as state-of-the-art results in sparse-view
novel view synthesis, even in challenging settings such as driving scenes and
monocular dynamic video. Results are best viewed in videos. Check out our
webpage! https://research.nvidia.com/labs/toronto-ai/GEN3C/