Orionnnn/hub-stats huggingface dataset:
Low-Precision Training of Large Language Models: Methods, Challenges,
  and Opportunities
Authors: name: Zhiwei Hao, name: Jianyuan Guo, name: Li Shen, name: Yong Luo, name: Han Hu, name: Guoxia Wang, name: Dianhai Yu, name: Yonggang Wen, name: Dacheng Tao
Published: 2025-05-02 06:33:25

Large language models (LLMs) have achieved impressive performance across
various domains. However, the substantial hardware resources required for their
training present a significant barrier to efficiency and scalability. To
mitigate this challenge, low-precision training techniques have been widely
adopted, leading to notable advancements in training efficiency. Despite these
gains, low-precision training involves several componentsx2013such
as weights, activations, and gradientsx2013each of which can be
represented in different numerical formats. The resulting diversity has created
a fragmented landscape in low-precision training research, making it difficult
for researchers to gain a unified overview of the field. This survey provides a
comprehensive review of existing low-precision training methods. To
systematically organize these approaches, we categorize them into three primary
groups based on their underlying numerical formats, which is a key factor
influencing hardware compatibility, computational efficiency, and ease of
reference for readers. The categories are: (1) fixed-point and integer-based
methods, (2) floating-point-based methods, and (3) customized format-based
methods. Additionally, we discuss quantization-aware training approaches, which
share key similarities with low-precision training during forward propagation.
Finally, we highlight several promising research directions to advance this
field. A collection of papers discussed in this survey is provided in
https://github.com/Hao840/Awesome-Low-Precision-Training.
WorldGenBench: A World-Knowledge-Integrated Benchmark for
  Reasoning-Driven Text-to-Image Generation
Authors: name: Daoan Zhang, name: Che Jiang, name: Ruoshi Xu, name: Biaoxiang Chen, name: Zijian Jin, name: Yutian Lu, name: Jianguo Zhang, name: Liang Yong, name: Jiebo Luo, name: Shengda Luo
Published: 2025-05-02 17:59:06

Recent advances in text-to-image (T2I) generation have achieved impressive
results, yet existing models still struggle with prompts that require rich
world knowledge and implicit reasoning: both of which are critical for
producing semantically accurate, coherent, and contextually appropriate images
in real-world scenarios. To address this gap, we introduce
WorldGenBench, a benchmark designed to systematically evaluate T2I
models' world knowledge grounding and implicit inferential capabilities,
covering both the humanities and nature domains. We propose the
Knowledge Checklist Score, a structured metric that measures how well
generated images satisfy key semantic expectations. Experiments across 21
state-of-the-art models reveal that while diffusion models lead among
open-source methods, proprietary auto-regressive models like GPT-4o exhibit
significantly stronger reasoning and knowledge integration. Our findings
highlight the need for deeper understanding and inference capabilities in
next-generation T2I systems. Project Page:
https://dwanzhang-ai.github.io/WorldGenBench/{https://dwanzhang-ai.github.io/WorldGenBench/}
Beyond One-Size-Fits-All: Inversion Learning for Highly Effective NLG
  Evaluation Prompts
Authors: name: Hanhua Hong, name: Chenghao Xiao, name: Yang Wang, name: Yiqi Liu, name: Wenge Rong, name: Chenghua Lin
Published: 2025-04-29 18:56:12

Evaluating natural language generation (NLG) systems is challenging due to
the diversity of valid outputs. While human evaluation is the gold standard, it
suffers from inconsistencies, lack of standardisation, and demographic biases,
limiting reproducibility. LLM-based evaluation offers a scalable alternative
but is highly sensitive to prompt design, where small variations can lead to
significant discrepancies. In this work, we propose an inversion learning
method that learns effective reverse mappings from model outputs back to their
input instructions, enabling the automatic generation of highly effective,
model-specific evaluation prompts. Our method requires only a single evaluation
sample and eliminates the need for time-consuming manual prompt engineering,
thereby improving both efficiency and robustness. Our work contributes toward a
new direction for more robust and efficient LLM-based evaluation.
PixelHacker: Image Inpainting with Structural and Semantic Consistency
Authors: name: Ziyang Xu, name: Kangsheng Duan, name: Xiaolei Shen, name: Zhifeng Ding, name: Wenyu Liu, name: Xiaohu Ruan, name: Xiaoxin Chen, name: Xinggang Wang
Published: 2025-04-29 05:28:36

Image inpainting is a fundamental research area between image editing and
image generation. Recent state-of-the-art (SOTA) methods have explored novel
attention mechanisms, lightweight architectures, and context-aware modeling,
demonstrating impressive performance. However, they often struggle with complex
structure (e.g., texture, shape, spatial relations) and semantics (e.g., color
consistency, object restoration, and logical correctness), leading to artifacts
and inappropriate generation. To address this challenge, we design a simple yet
effective inpainting paradigm called latent categories guidance, and further
propose a diffusion-based model named PixelHacker. Specifically, we first
construct a large dataset containing 14 million image-mask pairs by annotating
foreground and background (potential 116 and 21 categories, respectively).
Then, we encode potential foreground and background representations separately
through two fixed-size embeddings, and intermittently inject these features
into the denoising process via linear attention. Finally, by pre-training on
our dataset and fine-tuning on open-source benchmarks, we obtain PixelHacker.
Extensive experiments show that PixelHacker comprehensively outperforms the
SOTA on a wide range of datasets (Places2, CelebA-HQ, and FFHQ) and exhibits
remarkable consistency in both structure and semantics. Project page at
https://hustvl.github.io/PixelHacker.
TeLoGraF: Temporal Logic Planning via Graph-encoded Flow Matching
Authors: name: Yue Meng, name: Chuchu Fan
Published: 2025-05-01 14:40:07

Learning to solve complex tasks with signal temporal logic (STL)
specifications is crucial to many real-world applications. However, most
previous works only consider fixed or parametrized STL specifications due to
the lack of a diverse STL dataset and encoders to effectively extract temporal
logic information for downstream tasks. In this paper, we propose TeLoGraF,
Temporal Logic Graph-encoded Flow, which utilizes Graph Neural Networks (GNN)
encoder and flow-matching to learn solutions for general STL specifications. We
identify four commonly used STL templates and collect a total of 200K
specifications with paired demonstrations. We conduct extensive experiments in
five simulation environments ranging from simple dynamical models in the 2D
space to high-dimensional 7DoF Franka Panda robot arm and Ant quadruped
navigation. Results show that our method outperforms other baselines in the STL
satisfaction rate. Compared to classical STL planning algorithms, our approach
is 10-100X faster in inference and can work on any system dynamics. Besides, we
show our graph-encoding method's capability to solve complex STLs and
robustness to out-distribution STL specifications. Code is available at
https://github.com/mengyuest/TeLoGraF
X-Cross: Dynamic Integration of Language Models for Cross-Domain
  Sequential Recommendation
Authors: name: Guy Hadad, name: Haggai Roitman, name: Yotam Eshel, name: Bracha Shapira, name: Lior Rokach
Published: 2025-04-29 15:33:20

As new products are emerging daily, recommendation systems are required to
quickly adapt to possible new domains without needing extensive retraining.
This work presents ``X-Cross'' -- a novel cross-domain
sequential-recommendation model that recommends products in new domains by
integrating several domain-specific language models; each model is fine-tuned
with low-rank adapters (LoRA). Given a recommendation prompt, operating layer
by layer, X-Cross dynamically refines the representation of each source
language model by integrating knowledge from all other models. These refined
representations are propagated from one layer to the next, leveraging the
activations from each domain adapter to ensure domain-specific nuances are
preserved while enabling adaptability across domains. Using Amazon datasets for
sequential recommendation, X-Cross achieves performance comparable to a model
that is fine-tuned with LoRA, while using only 25% of the additional
parameters. In cross-domain tasks, such as adapting from Toys domain to Tools,
Electronics or Sports, X-Cross demonstrates robust performance, while requiring
about 50%-75% less fine-tuning data than LoRA to make fine-tuning effective.
Furthermore, X-Cross achieves significant improvement in accuracy over
alternative cross-domain baselines. Overall, X-Cross enables scalable and
adaptive cross-domain recommendations, reducing computational overhead and
providing an efficient solution for data-constrained environments.
CORG: Generating Answers from Complex, Interrelated Contexts
Authors: name: Hyunji Lee, name: Franck Dernoncourt, name: Trung Bui, name: Seunghyun Yoon
Published: 2025-04-25 02:40:48

In a real-world corpus, knowledge frequently recurs across documents but
often contains inconsistencies due to ambiguous naming, outdated information,
or errors, leading to complex interrelationships between contexts. Previous
research has shown that language models struggle with these complexities,
typically focusing on single factors in isolation. We classify these
relationships into four types: distracting, ambiguous, counterfactual, and
duplicated. Our analysis reveals that no single approach effectively addresses
all these interrelationships simultaneously. Therefore, we introduce Context
Organizer (CORG), a framework that organizes multiple contexts into
independently processed groups. This design allows the model to efficiently
find all relevant answers while ensuring disambiguation. CORG consists of three
key components: a graph constructor, a reranker, and an aggregator. Our results
demonstrate that CORG balances performance and efficiency effectively,
outperforming existing grouping methods and achieving comparable results to
more computationally intensive, single-context approaches.
Improving Editability in Image Generation with Layer-wise Memory
Authors: name: Daneul Kim, name: Jaeah Lee, name: Jaesik Park
Published: 2025-05-02 07:36:49

Most real-world image editing tasks require multiple sequential edits to
achieve desired results. Current editing approaches, primarily designed for
single-object modifications, struggle with sequential editing: especially with
maintaining previous edits along with adapting new objects naturally into the
existing content. These limitations significantly hinder complex editing
scenarios where multiple objects need to be modified while preserving their
contextual relationships. We address this fundamental challenge through two key
proposals: enabling rough mask inputs that preserve existing content while
naturally integrating new elements and supporting consistent editing across
multiple modifications. Our framework achieves this through layer-wise memory,
which stores latent representations and prompt embeddings from previous edits.
We propose Background Consistency Guidance that leverages memorized latents to
maintain scene coherence and Multi-Query Disentanglement in cross-attention
that ensures natural adaptation to existing content. To evaluate our method, we
present a new benchmark dataset incorporating semantic alignment metrics and
interactive editing scenarios. Through comprehensive experiments, we
demonstrate superior performance in iterative image editing tasks with minimal
user effort, requiring only rough masks while maintaining high-quality results
throughout multiple editing steps.
Real-World Gaps in AI Governance Research
Authors: name: Ilan Strauss, name: Isobel Moure, name: Tim O'Reilly, name: Sruly Rosenblat
Published: 2025-04-30 20:44:42

Drawing on 1,178 safety and reliability papers from 9,439 generative AI
papers (January 2020 - March 2025), we compare research outputs of leading AI
companies (Anthropic, Google DeepMind, Meta, Microsoft, and OpenAI) and AI
universities (CMU, MIT, NYU, Stanford, UC Berkeley, and University of
Washington). We find that corporate AI research increasingly concentrates on
pre-deployment areas -- model alignment and testing & evaluation -- while
attention to deployment-stage issues such as model bias has waned. Significant
research gaps exist in high-risk deployment domains, including healthcare,
finance, misinformation, persuasive and addictive features, hallucinations, and
copyright. Without improved observability into deployed AI, growing corporate
concentration could deepen knowledge deficits. We recommend expanding external
researcher access to deployment data and systematic observability of in-market
AI behaviors.
Skill Discovery for Software Scripting Automation via Offline
  Simulations with LLMs
Authors: name: Paiheng Xu, name: Gang Wu, name: Xiang Chen, name: Tong Yu, name: Chang Xiao, name: Franck Dernoncourt, name: Tianyi Zhou, name: Wei Ai, name: Viswanathan Swaminathan
Published: 2025-04-29 04:03:37

Scripting interfaces enable users to automate tasks and customize software
workflows, but creating scripts traditionally requires programming expertise
and familiarity with specific APIs, posing barriers for many users. While Large
Language Models (LLMs) can generate code from natural language queries, runtime
code generation is severely limited due to unverified code, security risks,
longer response times, and higher computational costs. To bridge the gap, we
propose an offline simulation framework to curate a software-specific skillset,
a collection of verified scripts, by exploiting LLMs and publicly available
scripting guides. Our framework comprises two components: (1) task creation,
using top-down functionality guidance and bottom-up API synergy exploration to
generate helpful tasks; and (2) skill generation with trials, refining and
validating scripts based on execution feedback. To efficiently navigate the
extensive API landscape, we introduce a Graph Neural Network (GNN)-based link
prediction model to capture API synergy, enabling the generation of skills
involving underutilized APIs and expanding the skillset's diversity.
Experiments with Adobe Illustrator demonstrate that our framework significantly
improves automation success rates, reduces response time, and saves runtime
token costs compared to traditional runtime code generation. This is the first
attempt to use software scripting interfaces as a testbed for LLM-based
systems, highlighting the advantages of leveraging execution feedback in a
controlled environment and offering valuable insights into aligning AI
capabilities with user needs in specialized software domains.
Spatial Speech Translation: Translating Across Space With Binaural
  Hearables
Authors: name: Tuochao Chen, name: Qirui Wang, name: Runlin He, name: Shyam Gollakota
Published: 2025-04-25 21:58:56

Imagine being in a crowded space where people speak a different language and
having hearables that transform the auditory space into your native language,
while preserving the spatial cues for all speakers. We introduce spatial speech
translation, a novel concept for hearables that translate speakers in the
wearer's environment, while maintaining the direction and unique voice
characteristics of each speaker in the binaural output. To achieve this, we
tackle several technical challenges spanning blind source separation,
localization, real-time expressive translation, and binaural rendering to
preserve the speaker directions in the translated audio, while achieving
real-time inference on the Apple M2 silicon. Our proof-of-concept evaluation
with a prototype binaural headset shows that, unlike existing models, which
fail in the presence of interference, we achieve a BLEU score of up to 22.01
when translating between languages, despite strong interference from other
speakers in the environment. User studies further confirm the system's
effectiveness in spatially rendering the translated speech in previously unseen
real-world reverberant environments. Taking a step back, this work marks the
first step towards integrating spatial perception into speech translation.
Self-Generated In-Context Examples Improve LLM Agents for Sequential
  Decision-Making Tasks
Authors: name: Vishnu Sarukkai, name: Zhiqiang Xie, name: Kayvon Fatahalian
Published: 2025-05-01 00:48:12

Many methods for improving Large Language Model (LLM) agents for sequential
decision-making tasks depend on task-specific knowledge engineering--such as
prompt tuning, curated in-context examples, or customized observation and
action spaces. Using these approaches, agent performance improves with the
quality or amount of knowledge engineering invested. Instead, we investigate
how LLM agents can automatically improve their performance by learning
in-context from their own successful experiences on similar tasks. Rather than
relying on task-specific knowledge engineering, we focus on constructing and
refining a database of self-generated examples. We demonstrate that even a
naive accumulation of successful trajectories across training tasks boosts test
performance on three benchmarks: ALFWorld (73% to 89%), Wordcraft (55% to 64%),
and InterCode-SQL (75% to 79%)--matching the performance the initial agent
achieves if allowed two to three attempts per task. We then introduce two
extensions: (1) database-level selection through population-based training to
identify high-performing example collections, and (2) exemplar-level selection
that retains individual trajectories based on their empirical utility as
in-context examples. These extensions further enhance performance, achieving
91% on ALFWorld--matching more complex approaches that employ task-specific
components and prompts. Our results demonstrate that automatic trajectory
database construction offers a compelling alternative to labor-intensive
knowledge engineering.
A Robust Deep Networks based Multi-Object MultiCamera Tracking System
  for City Scale Traffic
Authors: name: Muhammad Imran Zaman, name: Usama Ijaz Bajwa, name: Gulshan Saleem, name: Rana Hammad Raza
Published: 2025-05-01 14:00:25

Vision sensors are becoming more important in Intelligent Transportation
Systems (ITS) for traffic monitoring, management, and optimization as the
number of network cameras continues to rise. However, manual object tracking
and matching across multiple non-overlapping cameras pose significant
challenges in city-scale urban traffic scenarios. These challenges include
handling diverse vehicle attributes, occlusions, illumination variations,
shadows, and varying video resolutions. To address these issues, we propose an
efficient and cost-effective deep learning-based framework for Multi-Object
Multi-Camera Tracking (MO-MCT). The proposed framework utilizes Mask R-CNN for
object detection and employs Non-Maximum Suppression (NMS) to select target
objects from overlapping detections. Transfer learning is employed for
re-identification, enabling the association and generation of vehicle tracklets
across multiple cameras. Moreover, we leverage appropriate loss functions and
distance measures to handle occlusion, illumination, and shadow challenges. The
final solution identification module performs feature extraction using
ResNet-152 coupled with Deep SORT based vehicle tracking. The proposed
framework is evaluated on the 5th AI City Challenge dataset (Track 3),
comprising 46 camera feeds. Among these 46 camera streams, 40 are used for
model training and validation, while the remaining six are utilized for model
testing. The proposed framework achieves competitive performance with an IDF1
score of 0.8289, and precision and recall scores of 0.9026 and 0.8527
respectively, demonstrating its effectiveness in robust and accurate vehicle
tracking.
LLMs for Engineering: Teaching Models to Design High Powered Rockets
Authors: name: Toby Simonds
Published: 2025-04-27 23:59:39

Large Language Models (LLMs) have transformed software engineering, but their
application to physical engineering domains remains underexplored. This paper
evaluates LLMs' capabilities in high-powered rocketry design through
RocketBench, a benchmark connecting LLMs to high-fidelity rocket simulations.
We test models on two increasingly complex design tasks: target altitude
optimization and precision landing challenges. Our findings reveal that while
state-of-the-art LLMs demonstrate strong baseline engineering knowledge, they
struggle to iterate on their designs when given simulation results and
ultimately plateau below human performance levels. However, when enhanced with
reinforcement learning (RL), we show that a 7B parameter model outperforms both
SoTA foundation models and human experts. This research demonstrates that
RL-trained LLMs can serve as effective tools for complex engineering
optimization, potentially transforming engineering domains beyond software
development.
KeySync: A Robust Approach for Leakage-free Lip Synchronization in High
  Resolution
Authors: name: Antoni Bigata, name: Rodrigo Mira, name: Stella Bounareli, name: Michał Stypułkowski, name: Konstantinos Vougioukas, name: Stavros Petridis, name: Maja Pantic
Published: 2025-05-01 12:56:17

Lip synchronization, known as the task of aligning lip movements in an
existing video with new input audio, is typically framed as a simpler variant
of audio-driven facial animation. However, as well as suffering from the usual
issues in talking head generation (e.g., temporal consistency), lip
synchronization presents significant new challenges such as expression leakage
from the input video and facial occlusions, which can severely impact
real-world applications like automated dubbing, but are often neglected in
existing works. To address these shortcomings, we present KeySync, a two-stage
framework that succeeds in solving the issue of temporal consistency, while
also incorporating solutions for leakage and occlusions using a carefully
designed masking strategy. We show that KeySync achieves state-of-the-art
results in lip reconstruction and cross-synchronization, improving visual
quality and reducing expression leakage according to LipLeak, our novel leakage
metric. Furthermore, we demonstrate the effectiveness of our new masking
approach in handling occlusions and validate our architectural choices through
several ablation studies. Code and model weights can be found at
https://antonibigata.github.io/KeySync.