Orionnnn/hub-stats huggingface dataset:
Hard Negative Mining for Domain-Specific Retrieval in Enterprise Systems
Authors: name: Hansa Meghwani, name: Amit Agarwal, name: Priyaranjan Pattnayak, name: Hitesh Laxmichand Patel, name: Srikant Panda
Published: 2025-05-23 20:51:20

Enterprise search systems often struggle to retrieve accurate,
domain-specific information due to semantic mismatches and overlapping
terminologies. These issues can degrade the performance of downstream
applications such as knowledge management, customer support, and
retrieval-augmented generation agents. To address this challenge, we propose a
scalable hard-negative mining framework tailored specifically for
domain-specific enterprise data. Our approach dynamically selects semantically
challenging but contextually irrelevant documents to enhance deployed
re-ranking models.
  Our method integrates diverse embedding models, performs dimensionality
reduction, and uniquely selects hard negatives, ensuring computational
efficiency and semantic precision. Evaluation on our proprietary enterprise
corpus (cloud services domain) demonstrates substantial improvements of 15\% in
MRR@3 and 19\% in MRR@10 compared to state-of-the-art baselines and other
negative sampling techniques. Further validation on public domain-specific
datasets (FiQA, Climate Fever, TechQA) confirms our method's generalizability
and readiness for real-world applications.
Prot2Token: A Unified Framework for Protein Modeling via Next-Token
  Prediction
Authors: name: Mahdi Pourmirzaei, name: Farzaneh Esmaili, name: Salhuldin Alqarghuli, name: Mohammadreza Pourmirzaei, name: Ye Han, name: Kai Chen, name: Mohsen Rezaei, name: Duolin Wang, name: Dong Xu
Published: 2025-05-26 23:50:36

The diverse nature of protein prediction tasks has traditionally necessitated
specialized models, hindering the development of broadly applicable and
computationally efficient Protein Language Models (PLMs). In this work, we
introduce Prot2Token, a unified framework that overcomes these challenges by
converting a wide spectrum of protein-related predictions, from sequence-level
properties and residue-specific attributes to complex inter-protein
interactions, into a standardized next-token prediction format. At its core,
Prot2Token employs an autoregressive decoder, conditioned on embeddings from
pre-trained protein encoders and guided by learnable task tokens, to perform
diverse predictions. This architecture uniquely facilitates multi-task
learning, enabling a single model to master numerous tasks with improved
efficiency. We present extensive experimental validation across a variety of
benchmarks, demonstrating Prot2Tokens strong predictive power in different
types of protein-prediction tasks. Key results include significant speedups
(e.g., near 1000x over AlphaFold2 with MSA) and performance often matching or
exceeding specialized approaches. Beyond that, we introduce an auxiliary
self-supervised decoder pre-training approach to improve spatially sensitive
task performance. Prot2Token thus offers a significant step towards a
versatile, high-throughput paradigm for protein modeling, promising to
accelerate biological discovery and the development of novel therapeutics. The
code is available at https://github.com/mahdip72/prot2token .
Zero-Shot Vision Encoder Grafting via LLM Surrogates
Authors: name: Kaiyu Yue, name: Vasu Singla, name: Menglin Jia, name: John Kirchenbauer, name: Rifaa Qadri, name: Zikui Cai, name: Abhinav Bhatele, name: Furong Huang, name: Tom Goldstein
Published: 2025-05-28 17:59:59

Vision language models (VLMs) typically pair a modestly sized vision encoder
with a large language model (LLM), e.g., Llama-70B, making the decoder the
primary computational burden during training. To reduce costs, a potential
promising strategy is to first train the vision encoder using a small language
model before transferring it to the large one. We construct small "surrogate
models" that share the same embedding space and representation language as the
large target LLM by directly inheriting its shallow layers. Vision encoders
trained on the surrogate can then be directly transferred to the larger model,
a process we call zero-shot grafting -- when plugged directly into the
full-size target LLM, the grafted pair surpasses the encoder-surrogate pair
and, on some benchmarks, even performs on par with full decoder training with
the target LLM. Furthermore, our surrogate training approach reduces overall
VLM training costs by ~45% when using Llama-70B as the decoder.
Token Reduction Should Go Beyond Efficiency in Generative Models -- From
  Vision, Language to Multimodality
Authors: name: Zhenglun Kong, name: Yize Li, name: Fanhu Zeng, name: Lei Xin, name: Shvat Messica, name: Xue Lin, name: Pu Zhao, name: Manolis Kellis, name: Hao Tang, name: Marinka Zitnik
Published: 2025-05-23 11:30:30

In Transformer architectures, tokens\textemdash discrete units derived from
raw data\textemdash are formed by segmenting inputs into fixed-length chunks.
Each token is then mapped to an embedding, enabling parallel attention
computations while preserving the input's essential information. Due to the
quadratic computational complexity of transformer self-attention mechanisms,
token reduction has primarily been used as an efficiency strategy. This is
especially true in single vision and language domains, where it helps balance
computational costs, memory usage, and inference latency. Despite these
advances, this paper argues that token reduction should transcend its
traditional efficiency-oriented role in the era of large generative models.
Instead, we position it as a fundamental principle in generative modeling,
critically influencing both model architecture and broader applications.
Specifically, we contend that across vision, language, and multimodal systems,
token reduction can: (i) facilitate deeper multimodal integration and
alignment, (ii) mitigate "overthinking" and hallucinations, (iii) maintain
coherence over long inputs, and (iv) enhance training stability, etc. We
reframe token reduction as more than an efficiency measure. By doing so, we
outline promising future directions, including algorithm design, reinforcement
learning-guided token reduction, token optimization for in-context learning,
and broader ML and scientific domains. We highlight its potential to drive new
model architectures and learning strategies that improve robustness, increase
interpretability, and better align with the objectives of generative modeling.
HoPE: Hybrid of Position Embedding for Length Generalization in
  Vision-Language Models
Authors: name: Haoran Li, name: Yingjie Qin, name: Baoyuan Ou, name: Lai Xu, name: Ruiwen Xu
Published: 2025-05-26 18:37:40

Vision-Language Models (VLMs) have made significant progress in multimodal
tasks. However, their performance often deteriorates in long-context scenarios,
particularly long videos. While Rotary Position Embedding (RoPE) has been
widely adopted for length generalization in Large Language Models (LLMs),
extending vanilla RoPE to capture the intricate spatial-temporal dependencies
in videos remains an unsolved challenge. Existing methods typically allocate
different frequencies within RoPE to encode 3D positional information. However,
these allocation strategies mainly rely on heuristics, lacking in-depth
theoretical analysis. In this paper, we first study how different allocation
strategies impact the long-context capabilities of VLMs. Our analysis reveals
that current multimodal RoPEs fail to reliably capture semantic similarities
over extended contexts. To address this issue, we propose HoPE, a Hybrid of
Position Embedding designed to improve the long-context capabilities of VLMs.
HoPE introduces a hybrid frequency allocation strategy for reliable semantic
modeling over arbitrarily long context, and a dynamic temporal scaling
mechanism to facilitate robust learning and flexible inference across diverse
context lengths. Extensive experiments across four video benchmarks on long
video understanding and retrieval tasks demonstrate that HoPE consistently
outperforms existing methods, confirming its effectiveness. Code is available
at https://github.com/hrlics/HoPE.
CHIMERA: A Knowledge Base of Idea Recombination in Scientific Literature
Authors: name: Noy Sternlicht, name: Tom Hope
Published: 2025-05-27 06:36:04

A hallmark of human innovation is the process of recombination -- creating
original ideas by integrating elements of existing mechanisms and concepts. In
this work, we automatically mine the scientific literature and build CHIMERA: a
large-scale knowledge base (KB) of recombination examples. CHIMERA can be used
to empirically explore at scale how scientists recombine concepts and take
inspiration from different areas, or to train supervised machine learning
models that learn to predict new creative cross-domain directions. To build
this KB, we present a novel information extraction task of extracting
recombination from scientific paper abstracts, collect a high-quality corpus of
hundreds of manually annotated abstracts, and use it to train an LLM-based
extraction model. The model is applied to a large corpus of papers in the AI
domain, yielding a KB of over 28K recombination examples. We analyze CHIMERA to
explore the properties of recombination in different subareas of AI. Finally,
we train a scientific hypothesis generation model using the KB, which predicts
new recombination directions that real-world researchers find inspiring. Our
data and code are available at https://github.cs.huji.ac.il/tomhope-lab/CHIMERA
MangaVQA and MangaLMM: A Benchmark and Specialized Model for Multimodal
  Manga Understanding
Authors: name: Jeonghun Baek, name: Kazuki Egashira, name: Shota Onohara, name: Atsuyuki Miyai, name: Yuki Imajuku, name: Hikaru Ikuta, name: Kiyoharu Aizawa
Published: 2025-05-26 17:59:59

Manga, or Japanese comics, is a richly multimodal narrative form that blends
images and text in complex ways. Teaching large multimodal models (LMMs) to
understand such narratives at a human-like level could help manga creators
reflect on and refine their stories. To this end, we introduce two benchmarks
for multimodal manga understanding: MangaOCR, which targets in-page text
recognition, and MangaVQA, a novel benchmark designed to evaluate contextual
understanding through visual question answering. MangaVQA consists of 526
high-quality, manually constructed question-answer pairs, enabling reliable
evaluation across diverse narrative and visual scenarios. Building on these
benchmarks, we develop MangaLMM, a manga-specialized model finetuned from the
open-source LMM Qwen2.5-VL to jointly handle both tasks. Through extensive
experiments, including comparisons with proprietary models such as GPT-4o and
Gemini 2.5, we assess how well LMMs understand manga. Our benchmark and model
provide a comprehensive foundation for evaluating and advancing LMMs in the
richly narrative domain of manga.
Efficient Data Selection at Scale via Influence Distillation
Authors: name: Mahdi Nikdan, name: Vincent Cohen-Addad, name: Dan Alistarh, name: Vahab Mirrokni
Published: 2025-05-25 09:08:00

Effective data selection is critical for efficient training of modern Large
Language Models (LLMs). This paper introduces Influence Distillation, a novel,
mathematically-justified framework for data selection that employs second-order
information to optimally weight training samples. By distilling each sample's
influence on a target distribution, our method assigns model-specific weights
that are used to select training data for LLM fine-tuning, guiding it toward
strong performance on the target domain. We derive these optimal weights for
both Gradient Descent and Adam optimizers. To ensure scalability and reduce
computational cost, we propose a landmark-based approximation:
influence is precisely computed for a small subset of "landmark" samples and
then efficiently propagated to all other samples to determine their weights. We
validate Influence Distillation by applying it to instruction tuning on the
Tulu V2 dataset, targeting a range of tasks including GSM8k, SQuAD, and MMLU,
across several models from the Llama and Qwen families. Experiments show that
Influence Distillation matches or outperforms state-of-the-art performance
while achieving up to 3.5times faster selection.
Judging Quality Across Languages: A Multilingual Approach to Pretraining
  Data Filtering with Language Models
Authors: name: Mehdi Ali, name: Manuel Brack, name: Max Lübbering, name: Elias Wendt, name: Abbas Goher Khan, name: Richard Rutmann, name: Alex Jude, name: Maurice Kraus, name: Alexander Arno Weber, name: Felix Stollenwerk, name: David Kaczér, name: Florian Mai, name: Lucie Flek, name: Rafet Sifa, name: Nicolas Flores-Herr, name: Joachim Köhler, name: Patrick Schramowski, name: Michael Fromm, name: Kristian Kersting
Published: 2025-05-28 11:06:54

High-quality multilingual training data is essential for effectively
pretraining large language models (LLMs). Yet, the availability of suitable
open-source multilingual datasets remains limited. Existing state-of-the-art
datasets mostly rely on heuristic filtering methods, restricting both their
cross-lingual transferability and scalability. Here, we introduce JQL, a
systematic approach that efficiently curates diverse and high-quality
multilingual data at scale while significantly reducing computational demands.
JQL distills LLMs' annotation capabilities into lightweight annotators based on
pretrained multilingual embeddings. These models exhibit robust multilingual
and cross-lingual performance, even for languages and scripts unseen during
training. Evaluated empirically across 35 languages, the resulting annotation
pipeline substantially outperforms current heuristic filtering methods like
Fineweb2. JQL notably enhances downstream model training quality and increases
data retention rates. Our research provides practical insights and valuable
resources for multilingual data curation, raising the standards of multilingual
dataset development.
SWE-rebench: An Automated Pipeline for Task Collection and
  Decontaminated Evaluation of Software Engineering Agents
Authors: name: Ibragim Badertdinov, name: Alexander Golubev, name: Maksim Nekrashevich, name: Anton Shevtsov, name: Simon Karasik, name: Andrei Andriushchenko, name: Maria Trofimova, name: Daria Litvintseva, name: Boris Yangel
Published: 2025-05-26 18:01:00

LLM-based agents have shown promising capabilities in a growing range of
software engineering (SWE) tasks. However, advancing this field faces two
critical challenges. First, high-quality training data is scarce, especially
data that reflects real-world SWE scenarios, where agents must interact with
development environments, execute code and adapt behavior based on the outcomes
of their actions. Existing datasets are either limited to one-shot code
generation or comprise small, manually curated collections of interactive
tasks, lacking both scale and diversity. Second, the lack of fresh interactive
SWE tasks affects evaluation of rapidly improving models, as static benchmarks
quickly become outdated due to contamination issues. To address these
limitations, we introduce a novel, automated, and scalable pipeline to
continuously extract real-world interactive SWE tasks from diverse GitHub
repositories. Using this pipeline, we construct SWE-rebench, a public dataset
comprising over 21,000 interactive Python-based SWE tasks, suitable for
reinforcement learning of SWE agents at scale. Additionally, we use continuous
supply of fresh tasks collected using SWE-rebench methodology to build a
contamination-free benchmark for agentic software engineering. We compare
results of various LLMs on this benchmark to results on SWE-bench Verified and
show that performance of some language models might be inflated due to
contamination issues.
Styl3R: Instant 3D Stylized Reconstruction for Arbitrary Scenes and
  Styles
Authors: name: Peng Wang, name: Xiang Liu, name: Peidong Liu
Published: 2025-05-27 11:47:15

Stylizing 3D scenes instantly while maintaining multi-view consistency and
faithfully resembling a style image remains a significant challenge. Current
state-of-the-art 3D stylization methods typically involve computationally
intensive test-time optimization to transfer artistic features into a
pretrained 3D representation, often requiring dense posed input images. In
contrast, leveraging recent advances in feed-forward reconstruction models, we
demonstrate a novel approach to achieve direct 3D stylization in less than a
second using unposed sparse-view scene images and an arbitrary style image. To
address the inherent decoupling between reconstruction and stylization, we
introduce a branched architecture that separates structure modeling and
appearance shading, effectively preventing stylistic transfer from distorting
the underlying 3D scene structure. Furthermore, we adapt an identity loss to
facilitate pre-training our stylization model through the novel view synthesis
task. This strategy also allows our model to retain its original reconstruction
capabilities while being fine-tuned for stylization. Comprehensive evaluations,
using both in-domain and out-of-domain datasets, demonstrate that our approach
produces high-quality stylized 3D content that achieve a superior blend of
style and scene appearance, while also outperforming existing methods in terms
of multi-view consistency and efficiency.
Sherlock: Self-Correcting Reasoning in Vision-Language Models
Authors: name: Yi Ding, name: Ruqi Zhang
Published: 2025-05-28 17:58:03

Reasoning Vision-Language Models (VLMs) have shown promising performance on
complex multimodal tasks. However, they still face significant challenges: they
are highly sensitive to reasoning errors, require large volumes of annotated
data or accurate verifiers, and struggle to generalize beyond specific domains.
To address these limitations, we explore self-correction as a strategy to
enhance reasoning VLMs. We first conduct an in-depth analysis of reasoning
VLMs' self-correction abilities and identify key gaps. Based on our findings,
we introduce Sherlock, a self-correction and self-improvement training
framework. Sherlock introduces a trajectory-level self-correction objective, a
preference data construction method based on visual perturbation, and a dynamic
beta for preference tuning. Once the model acquires self-correction
capabilities using only 20k randomly sampled annotated data, it continues to
self-improve without external supervision. Built on the Llama3.2-Vision-11B
model, Sherlock achieves remarkable results across eight benchmarks, reaching
an average accuracy of 64.1 with direct generation and 65.4 after
self-correction. It outperforms LLaVA-CoT (63.2), Mulberry (63.9), and
LlamaV-o1 (63.4) while using less than 20% of the annotated data.
AITEE -- Agentic Tutor for Electrical Engineering
Authors: name: Christopher Knievel, name: Alexander Bernhardt, name: Christian Bernhardt
Published: 2025-05-27 10:07:05

Intelligent tutoring systems combined with large language models offer a
promising approach to address students' diverse needs and promote
self-efficacious learning. While large language models possess good
foundational knowledge of electrical engineering basics, they remain
insufficiently capable of addressing specific questions about electrical
circuits. In this paper, we present AITEE, an agent-based tutoring system for
electrical engineering designed to accompany students throughout their learning
process, offer individualized support, and promote self-directed learning.
AITEE supports both hand-drawn and digital circuits through an adapted circuit
reconstruction process, enabling natural interaction with students. Our novel
graph-based similarity measure identifies relevant context from lecture
materials through a retrieval augmented generation approach, while parallel
Spice simulation further enhances accuracy in applying solution methodologies.
The system implements a Socratic dialogue to foster learner autonomy through
guided questioning. Experimental evaluations demonstrate that AITEE
significantly outperforms baseline approaches in domain-specific knowledge
application, with even medium-sized LLM models showing acceptable performance.
Our results highlight the potential of agentic tutors to deliver scalable,
personalized, and effective learning environments for electrical engineering
education.
First Finish Search: Efficient Test-Time Scaling in Large Language
  Models
Authors: name: Aradhye Agarwal, name: Ayan Sengupta, name: Tanmoy Chakraborty
Published: 2025-05-23 17:57:43

Test-time scaling (TTS), which involves dynamic allocation of compute during
inference, offers a promising way to improve reasoning in large language
models. While existing TTS methods work well, they often rely on long decoding
paths or require a large number of samples to be generated, increasing the
token usage and inference latency. We observe the surprising fact that for
reasoning tasks, shorter traces are much more likely to be correct than longer
ones. Motivated by this, we introduce First Finish Search (FFS), a
training-free parallel decoding strategy that launches n independent samples
and returns as soon as any one completes. We evaluate FFS alongside simple
decoding, beam search, majority voting, and budget forcing on four reasoning
models (DeepSeek-R1, R1-Distill-Qwen-32B, QwQ-32B and Phi-4-Reasoning-Plus) and
across four datasets (AIME24, AIME25-I, AIME25-II and GPQA Diamond). With
DeepSeek-R1, FFS achieves 82.23% accuracy on the AIME datasets, a 15%
improvement over DeepSeek-R1's standalone accuracy, nearly matching OpenAI's
o4-mini performance. Our theoretical analysis explains why stopping at the
shortest trace is likely to yield a correct answer and identifies the
conditions under which early stopping may be suboptimal. The elegance and
simplicity of FFS demonstrate that straightforward TTS strategies can perform
remarkably well, revealing the untapped potential of simple approaches at
inference time.
Meta-Learning an In-Context Transformer Model of Human Higher Visual
  Cortex
Authors: name: Muquan Yu, name: Mu Nan, name: Hossein Adeli, name: Jacob S. Prince, name: John A. Pyles, name: Leila Wehbe, name: Margaret M. Henderson, name: Michael J. Tarr, name: Andrew F. Luo
Published: 2025-05-21 17:59:41

Understanding functional representations within higher visual cortex is a
fundamental question in computational neuroscience. While artificial neural
networks pretrained on large-scale datasets exhibit striking representational
alignment with human neural responses, learning image-computable models of
visual cortex relies on individual-level, large-scale fMRI datasets. The
necessity for expensive, time-intensive, and often impractical data acquisition
limits the generalizability of encoders to new subjects and stimuli. BraInCoRL
uses in-context learning to predict voxelwise neural responses from few-shot
examples without any additional finetuning for novel subjects and stimuli. We
leverage a transformer architecture that can flexibly condition on a variable
number of in-context image stimuli, learning an inductive bias over multiple
subjects. During training, we explicitly optimize the model for in-context
learning. By jointly conditioning on image features and voxel activations, our
model learns to directly generate better performing voxelwise models of higher
visual cortex. We demonstrate that BraInCoRL consistently outperforms existing
voxelwise encoder designs in a low-data regime when evaluated on entirely novel
images, while also exhibiting strong test-time scaling behavior. The model also
generalizes to an entirely new visual fMRI dataset, which uses different
subjects and fMRI data acquisition parameters. Further, BraInCoRL facilitates
better interpretability of neural signals in higher visual cortex by attending
to semantically relevant stimuli. Finally, we show that our framework enables
interpretable mappings from natural language queries to voxel selectivity.