Orionnnn/hub-stats huggingface dataset:
ResearchTown: Simulator of Human Research Community
Authors: name: Haofei Yu, name: Zhaochen Hong, name: Zirui Cheng, name: Kunlun Zhu, name: Keyang Xuan, name: Jinwei Yao, name: Tao Feng, name: Jiaxuan You
Published: 2024-12-23 18:26:53

Large Language Models (LLMs) have demonstrated remarkable potential in
scientific domains, yet a fundamental question remains unanswered: Can we
simulate human research communities with LLMs? Addressing this question can
deepen our understanding of the processes behind idea brainstorming and inspire
the automatic discovery of novel scientific insights. In this work, we propose
ResearchTown, a multi-agent framework for research community simulation. Within
this framework, the human research community is simplified and modeled as an
agent-data graph, where researchers and papers are represented as agent-type
and data-type nodes, respectively, and connected based on their collaboration
relationships. We also introduce TextGNN, a text-based inference framework that
models various research activities (e.g., paper reading, paper writing, and
review writing) as special forms of a unified message-passing process on the
agent-data graph. To evaluate the quality of the research simulation, we
present ResearchBench, a benchmark that uses a node-masking prediction task for
scalable and objective assessment based on similarity. Our experiments reveal
three key findings: (1) ResearchTown can provide a realistic simulation of
collaborative research activities, including paper writing and review writing;
(2) ResearchTown can maintain robust simulation with multiple researchers and
diverse papers; (3) ResearchTown can generate interdisciplinary research ideas
that potentially inspire novel research directions.
OpenRFT: Adapting Reasoning Foundation Model for Domain-specific Tasks
  with Reinforcement Fine-Tuning
Authors: name: Yuxiang Zhang, name: Yuqi Yang, name: Jiangming Shu, name: Yuhang Wang, name: Jinlin Xiao, name: Jitao Sang
Published: 2024-12-22 04:21:30

OpenAI's recent introduction of Reinforcement Fine-Tuning (RFT) showcases the
potential of reasoning foundation model and offers a new paradigm for
fine-tuning beyond simple pattern imitation. This technical report presents
OpenRFT, our attempt to fine-tune generalist reasoning models for
domain-specific tasks under the same settings as RFT. OpenRFT addresses two key
challenges of lacking reasoning step data and the limited quantity of training
samples, by leveraging the domain-specific samples in three ways: question
augmentation, synthesizing reasoning-process data, and few-shot ICL. The
evaluation is conducted on SciKnowEval, where OpenRFT achieves notable
performance gains with only 100 domain-specific samples for each task. More
experimental results will be updated continuously in later versions. Source
codes, datasets, and models are disclosed at:
https://github.com/ADaM-BJTU/OpenRFT
PC Agent: While You Sleep, AI Works -- A Cognitive Journey into Digital
  World
Authors: name: Yanheng He, name: Jiahe Jin, name: Shijie Xia, name: Jiadi Su, name: Runze Fan, name: Haoyang Zou, name: Xiangkun Hu, name: Pengfei Liu
Published: 2024-12-23 14:02:12

Imagine a world where AI can handle your work while you sleep - organizing
your research materials, drafting a report, or creating a presentation you need
for tomorrow. However, while current digital agents can perform simple tasks,
they are far from capable of handling the complex real-world work that humans
routinely perform. We present PC Agent, an AI system that demonstrates a
crucial step toward this vision through human cognition transfer. Our key
insight is that the path from executing simple "tasks" to handling complex
"work" lies in efficiently capturing and learning from human cognitive
processes during computer use. To validate this hypothesis, we introduce three
key innovations: (1) PC Tracker, a lightweight infrastructure that efficiently
collects high-quality human-computer interaction trajectories with complete
cognitive context; (2) a two-stage cognition completion pipeline that
transforms raw interaction data into rich cognitive trajectories by completing
action semantics and thought processes; and (3) a multi-agent system combining
a planning agent for decision-making with a grounding agent for robust visual
grounding. Our preliminary experiments in PowerPoint presentation creation
reveal that complex digital work capabilities can be achieved with a small
amount of high-quality cognitive data - PC Agent, trained on just 133 cognitive
trajectories, can handle sophisticated work scenarios involving up to 50 steps
across multiple applications. This demonstrates the data efficiency of our
approach, highlighting that the key to training capable digital agents lies in
collecting human cognitive data. By open-sourcing our complete framework,
including the data collection infrastructure and cognition completion methods,
we aim to lower the barriers for the research community to develop truly
capable digital agents.
Agent-SafetyBench: Evaluating the Safety of LLM Agents
Authors: name: Zhexin Zhang, name: Shiyao Cui, name: Yida Lu, name: Jingzhuo Zhou, name: Junxiao Yang, name: Hongning Wang, name: Minlie Huang
Published: 2024-12-19 02:35:15

As large language models (LLMs) are increasingly deployed as agents, their
integration into interactive environments and tool use introduce new safety
challenges beyond those associated with the models themselves. However, the
absence of comprehensive benchmarks for evaluating agent safety presents a
significant barrier to effective assessment and further improvement. In this
paper, we introduce Agent-SafetyBench, a comprehensive benchmark designed to
evaluate the safety of LLM agents. Agent-SafetyBench encompasses 349
interaction environments and 2,000 test cases, evaluating 8 categories of
safety risks and covering 10 common failure modes frequently encountered in
unsafe interactions. Our evaluation of 16 popular LLM agents reveals a
concerning result: none of the agents achieves a safety score above 60%. This
highlights significant safety challenges in LLM agents and underscores the
considerable need for improvement. Through quantitative analysis, we identify
critical failure modes and summarize two fundamental safety detects in current
LLM agents: lack of robustness and lack of risk awareness. Furthermore, our
findings suggest that reliance on defense prompts alone is insufficient to
address these safety issues, emphasizing the need for more advanced and robust
strategies. We release Agent-SafetyBench at
https://github.com/thu-coai/Agent-SafetyBench to facilitate further
research and innovation in agent safety evaluation and improvement.
Friends-MMC: A Dataset for Multi-modal Multi-party Conversation
  Understanding
Authors: name: Yueqian Wang, name: Xiaojun Meng, name: Yuxuan Wang, name: Jianxin Liang, name: Qun Liu, name: Dongyan Zhao
Published: 2024-12-23 05:32:48

Multi-modal multi-party conversation (MMC) is a less studied yet important
topic of research due to that it well fits real-world scenarios and thus
potentially has more widely-used applications. Compared with the traditional
multi-modal conversations, MMC requires stronger character-centered
understanding abilities as there are many interlocutors appearing in both the
visual and textual context. To facilitate the study of this problem, we present
Friends-MMC in this paper, an MMC dataset that contains 24,000+ unique
utterances paired with video context. To explore the character-centered
understanding of the dialogue, we also annotate the speaker of each utterance,
the names and bounding bboxes of faces that appear in the video. Based on this
Friends-MMC dataset, we further study two fundamental MMC tasks: conversation
speaker identification and conversation response prediction, both of which have
the multi-party nature with the video or image as visual context. For
conversation speaker identification, we demonstrate the inefficiencies of
existing methods such as pre-trained models, and propose a simple yet effective
baseline method that leverages an optimization solver to utilize the context of
two modalities to achieve better performance. For conversation response
prediction, we fine-tune generative dialogue models on Friend-MMC, and analyze
the benefits of speaker information. The code and dataset is publicly available
at https://github.com/yellow-binary-tree/Friends-MMC and thus we call for more
attention on modeling speaker information when understanding conversations.
LearnLM: Improving Gemini for Learning
Authors: name: LearnLM Team, name: Abhinit Modi, name: Aditya Srikanth Veerubhotla, name: Aliya Rysbek, name: Andrea Huber, name: Brett Wiltshire, name: Brian Veprek, name: Daniel Gillick, name: Daniel Kasenberg, name: Derek Ahmed, name: Irina Jurenka, name: James Cohan, name: Jennifer She, name: Julia Wilkowski, name: Kaiz Alarakyia, name: Kevin McKee, name: Lisa Wang, name: Markus Kunesch, name: Mike Schaekermann, name: Miruna Pîslar, name: Nikhil Joshi, name: Parsa Mahmoudieh, name: Paul Jhun, name: Sara Wiltberger, name: Shakir Mohamed, name: Shashank Agarwal, name: Shubham Milind Phal, name: Sun Jae Lee, name: Theofilos Strinopoulos, name: Wei-Jen Ko, name: Amy Wang, name: Ankit Anand, name: Avishkar Bhoopchand, name: Dan Wild, name: Divya Pandya, name: Filip Bar, name: Garth Graham, name: Holger Winnemoeller, name: Mahvish Nagda, name: Prateek Kolhar, name: Renee Schneider, name: Shaojian Zhu, name: Stephanie Chan, name: Steve Yadlowsky, name: Viknesh Sounderajah, name: Yannis Assael
Published: 2024-12-21 01:34:05

Today's generative AI systems are tuned to present information by default
rather than engage users in service of learning as a human tutor would. To
address the wide range of potential education use cases for these systems, we
reframe the challenge of injecting pedagogical behavior as one of
pedagogical instruction following, where training and evaluation
examples include system-level instructions describing the specific pedagogy
attributes present or desired in subsequent model turns. This framing avoids
committing our models to any particular definition of pedagogy, and instead
allows teachers or developers to specify desired model behavior. It also clears
a path to improving Gemini models for learning -- by enabling the addition of
our pedagogical data to post-training mixtures -- alongside their rapidly
expanding set of capabilities. Both represent important changes from our
initial tech report. We show how training with pedagogical instruction
following produces a LearnLM model (available on Google AI Studio) that is
preferred substantially by expert raters across a diverse set of learning
scenarios, with average preference strengths of 31\% over GPT-4o, 11\% over
Claude 3.5, and 13\% over the Gemini 1.5 Pro model LearnLM was based on.
DRT-o1: Optimized Deep Reasoning Translation via Long Chain-of-Thought
Authors: name: Jiaan Wang, name: Fandong Meng, name: Yunlong Liang, name: Jie Zhou
Published: 2024-12-23 11:55:33

Recently, O1-like models have emerged as representative examples,
illustrating the effectiveness of long chain-of-thought (CoT) in reasoning
tasks such as math and coding tasks. In this paper, we introduce DRT-o1, an
attempt to bring the success of long CoT to neural machine translation (MT).
Specifically, in view of the literature books that might involve similes and
metaphors, translating these texts to a target language is very difficult in
practice due to cultural differences. In such cases, literal translation often
fails to convey the intended meaning effectively. Even for professional human
translators, considerable thought must be given to preserving semantics
throughout the translation process. To simulate LLMs' long thought ability in
MT, we first mine sentences containing similes or metaphors from existing
literature books, and then develop a multi-agent framework to translate these
sentences via long thought. In the multi-agent framework, a translator is used
to iteratively translate the source sentence under the suggestions provided by
an advisor. To ensure the effectiveness of the long thoughts, an evaluator is
also employed to judge whether the translation in the current round is better
than the previous one or not. In this manner, we collect tens of thousands of
long-thought MT data, which is used to train our DRT-o1. The experimental
results on literature translation demonstrate the effectiveness of the DRT-o1.
Using Qwen2.5-7B and Qwen2.5-14B as the backbones, the improvement brought by
DRT-o1 achieves 7.33~8.26 BLEU and 1.66~3.36 CometScore. Besides, DRT-o1-7B can
outperform QwQ-32B-Preview by 7.82 BLEU and 1.46 CometScore, showing its
effectiveness. The project is available at https://github.com/krystalan/DRT-o1
Large Motion Video Autoencoding with Cross-modal Video VAE
Authors: name: Yazhou Xing, name: Yang Fei, name: Yingqing He, name: Jingye Chen, name: Jiaxin Xie, name: Xiaowei Chi, name: Qifeng Chen
Published: 2024-12-23 18:58:24

Learning a robust video Variational Autoencoder (VAE) is essential for
reducing video redundancy and facilitating efficient video generation. Directly
applying image VAEs to individual frames in isolation can result in temporal
inconsistencies and suboptimal compression rates due to a lack of temporal
compression. Existing Video VAEs have begun to address temporal compression;
however, they often suffer from inadequate reconstruction performance. In this
paper, we present a novel and powerful video autoencoder capable of
high-fidelity video encoding. First, we observe that entangling spatial and
temporal compression by merely extending the image VAE to a 3D VAE can
introduce motion blur and detail distortion artifacts. Thus, we propose
temporal-aware spatial compression to better encode and decode the spatial
information. Additionally, we integrate a lightweight motion compression model
for further temporal compression. Second, we propose to leverage the textual
information inherent in text-to-video datasets and incorporate text guidance
into our model. This significantly enhances reconstruction quality,
particularly in terms of detail preservation and temporal stability. Third, we
further improve the versatility of our model through joint training on both
images and videos, which not only enhances reconstruction quality but also
enables the model to perform both image and video autoencoding. Extensive
evaluations against strong recent baselines demonstrate the superior
performance of our method. The project website can be found
at~https://yzxing87.github.io/vae/{https://yzxing87.github.io/vae/}.
Outcome-Refining Process Supervision for Code Generation
Authors: name: Zhuohao Yu, name: Weizheng Gu, name: Yidong Wang, name: Zhengran Zeng, name: Jindong Wang, name: Wei Ye, name: Shikun Zhang
Published: 2024-12-19 17:59:42

Large Language Models have demonstrated remarkable capabilities in code
generation, yet they often struggle with complex programming tasks that require
deep algorithmic reasoning. While process supervision through learned reward
models shows promise in guiding reasoning steps, it requires expensive training
data and suffers from unreliable evaluation. We propose Outcome-Refining
Process Supervision, a novel paradigm that treats outcome refinement itself as
the process to be supervised. Our framework leverages concrete execution
signals to ground the supervision of reasoning steps, while using
tree-structured exploration to maintain multiple solution trajectories
simultaneously. Experiments demonstrate that our approach enables even smaller
models to achieve high success accuracy and performance metrics on competitive
programming tasks, creates more reliable verification than traditional reward
models without requiring training PRMs. Our approach achieves significant
improvements across 5 models and 3 datasets: an average of 26.9% increase in
correctness and 42.2% in efficiency. The results suggest that providing
structured reasoning space with concrete verification signals is crucial for
solving complex programming tasks. We open-source all our code and data at:
https://github.com/zhuohaoyu/ORPS
Deliberation in Latent Space via Differentiable Cache Augmentation
Authors: name: Luyang Liu, name: Jonas Pfeiffer, name: Jiaxing Wu, name: Jun Xie, name: Arthur Szlam
Published: 2024-12-23 18:02:25

Techniques enabling large language models (LLMs) to "think more" by
generating and attending to intermediate reasoning steps have shown promise in
solving complex problems. However, the standard approaches generate sequences
of discrete tokens immediately before responding, and so they can incur
significant latency costs and be challenging to optimize. In this work, we
demonstrate that a frozen LLM can be augmented with an offline coprocessor that
operates on the model's key-value (kv) cache. This coprocessor augments the
cache with a set of latent embeddings designed to improve the fidelity of
subsequent decoding. We train this coprocessor using the language modeling loss
from the decoder on standard pretraining data, while keeping the decoder itself
frozen. This approach enables the model to learn, in an end-to-end
differentiable fashion, how to distill additional computation into its
kv-cache. Because the decoder remains unchanged, the coprocessor can operate
offline and asynchronously, and the language model can function normally if the
coprocessor is unavailable or if a given cache is deemed not to require extra
computation. We show experimentally that when a cache is augmented, the decoder
achieves lower perplexity on numerous subsequent tokens. Furthermore, even
without any task-specific training, our experiments demonstrate that cache
augmentation consistently reduces perplexity and improves performance across a
range of reasoning-intensive tasks.
Distilled Decoding 1: One-step Sampling of Image Auto-regressive Models
  with Flow Matching
Authors: name: Enshu Liu, name: Xuefei Ning, name: Yu Wang, name: Zinan Lin
Published: 2024-12-22 20:21:54

Autoregressive (AR) models have achieved state-of-the-art performance in text
and image generation but suffer from slow generation due to the token-by-token
process. We ask an ambitious question: can a pre-trained AR model be adapted to
generate outputs in just one or two steps? If successful, this would
significantly advance the development and deployment of AR models. We notice
that existing works that try to speed up AR generation by generating multiple
tokens at once fundamentally cannot capture the output distribution due to the
conditional dependencies between tokens, limiting their effectiveness for
few-step generation. To address this, we propose Distilled Decoding (DD), which
uses flow matching to create a deterministic mapping from Gaussian distribution
to the output distribution of the pre-trained AR model. We then train a network
to distill this mapping, enabling few-step generation. DD doesn't need the
training data of the original AR model, making it more practical.We evaluate DD
on state-of-the-art image AR models and present promising results on
ImageNet-256. For VAR, which requires 10-step generation, DD enables one-step
generation (6.3times speed-up), with an acceptable increase in FID from 4.19
to 9.96. For LlamaGen, DD reduces generation from 256 steps to 1, achieving an
217.8times speed-up with a comparable FID increase from 4.11 to 11.35. In
both cases, baseline methods completely fail with FID>100. DD also excels on
text-to-image generation, reducing the generation from 256 steps to 2 for
LlamaGen with minimal FID increase from 25.70 to 28.95. As the first work to
demonstrate the possibility of one-step generation for image AR models, DD
challenges the prevailing notion that AR models are inherently slow, and opens
up new opportunities for efficient AR generation. The project website is at
https://imagination-research.github.io/distilled-decoding.
NILE: Internal Consistency Alignment in Large Language Models
Authors: name: Minda Hu, name: Qiyuan Zhang, name: Yufei Wang, name: Bowei He, name: Hongru Wang, name: Jingyan Zhou, name: Liangyou Li, name: Yasheng Wang, name: Chen Ma, name: Irwin King
Published: 2024-12-21 16:25:16

As a crucial step to enhance LLMs alignment with human intentions,
Instruction Fine-Tuning (IFT) has a high demand on dataset quality. However,
existing IFT datasets often contain knowledge that is inconsistent with LLMs'
internal knowledge learned from the pre-training phase, which can greatly
affect the efficacy of IFT. To address this issue, we introduce NILE (iNternal
consIstency aLignmEnt) framework, aimed at optimizing IFT datasets to unlock
LLMs' capability further. NILE operates by eliciting target pre-trained LLM's
internal knowledge corresponding to instruction data. The internal knowledge is
leveraged to revise the answer in IFT datasets. Additionally, we propose a
novel Internal Consistency Filtering (ICF) method to filter training samples,
ensuring its high consistency with LLM's internal knowledge. Our experiments
demonstrate that NILE-aligned IFT datasets sharply boost LLM performance across
multiple LLM ability evaluation datasets, achieving up to 66.6% gain on
Arena-Hard and 68.5% on Alpaca-Eval V2. Further analysis confirms that each
component of the NILE}framework contributes to these substantial performance
improvements, and provides compelling evidence that dataset consistency with
pre-trained internal knowledge is pivotal for maximizing LLM potential.
Diving into Self-Evolving Training for Multimodal Reasoning
Authors: name: Wei Liu, name: Junlong Li, name: Xiwen Zhang, name: Fan Zhou, name: Yu Cheng, name: Junxian He
Published: 2024-12-23 10:18:41

Reasoning ability is essential for Large Multimodal Models (LMMs). In the
absence of multimodal chain-of-thought annotated data, self-evolving training,
where the model learns from its own outputs, has emerged as an effective and
scalable approach for enhancing reasoning abilities. Despite its growing usage,
a comprehensive understanding of self-evolving training, particularly in the
context of multimodal reasoning, remains limited. In this paper, we delve into
the intricacies of self-evolving training for multimodal reasoning, pinpointing
three key factors: Training Method, Reward Model, and Prompt Variation. We
systematically examine each factor and explore how various configurations
affect the training's effectiveness. Our analysis leads to a set of best
practices for each factor, aimed at optimizing multimodal reasoning.
Furthermore, we explore the Self-Evolution Dynamics during training and the
impact of automatic balancing mechanisms in boosting performance. After all the
investigations, we present a final recipe for self-evolving training in
multimodal reasoning, encapsulating these design choices into a framework we
call MSTaR (Multimodal Self-evolving Training for Reasoning), which is
universally effective for models with different sizes on various benchmarks,
e.g., surpassing the pre-evolved model significantly on 5 multimodal reasoning
benchmarks without using additional human annotations, as demonstrated on
MiniCPM-V-2.5 (8B), Phi-3.5-Vision (4B) and InternVL2 (2B). We believe this
study fills a significant gap in the understanding of self-evolving training
for multimodal reasoning and offers a robust framework for future research. Our
policy and reward models, as well as the collected data, is released to
facilitate further investigation in multimodal reasoning.