Orionnnn/hub-stats huggingface dataset:
Mechanistic Permutability: Match Features Across Layers
Authors: name: Nikita Balagansky, name: Ian Maksimov, name: Daniil Gavrilov
Published: 2024-10-10 06:55:38

Understanding how features evolve across layers in deep neural networks is a
fundamental challenge in mechanistic interpretability, particularly due to
polysemanticity and feature superposition. While Sparse Autoencoders (SAEs)
have been used to extract interpretable features from individual layers,
aligning these features across layers has remained an open problem. In this
paper, we introduce SAE Match, a novel, data-free method for aligning SAE
features across different layers of a neural network. Our approach involves
matching features by minimizing the mean squared error between the folded
parameters of SAEs, a technique that incorporates activation thresholds into
the encoder and decoder weights to account for differences in feature scales.
Through extensive experiments on the Gemma 2 language model, we demonstrate
that our method effectively captures feature evolution across layers, improving
feature matching quality. We also show that features persist over several
layers and that our approach can approximate hidden states across layers. Our
work advances the understanding of feature dynamics in neural networks and
provides a new tool for mechanistic interpretability studies.
DA-Code: Agent Data Science Code Generation Benchmark for Large Language
  Models
Authors: name: Yiming Huang, name: Jianwen Luo, name: Yan Yu, name: Yitong Zhang, name: Fangyu Lei, name: Yifan Wei, name: Shizhu He, name: Lifu Huang, name: Xiao Liu, name: Jun Zhao, name: Kang Liu
Published: 2024-10-09 18:00:05

We introduce DA-Code, a code generation benchmark specifically designed to
assess LLMs on agent-based data science tasks. This benchmark features three
core elements: First, the tasks within DA-Code are inherently challenging,
setting them apart from traditional code generation tasks and demanding
advanced coding skills in grounding and planning. Second, examples in DA-Code
are all based on real and diverse data, covering a wide range of complex data
wrangling and analytics tasks. Third, to solve the tasks, the models must
utilize complex data science programming languages, to perform intricate data
processing and derive the answers. We set up the benchmark in a controllable
and executable environment that aligns with real-world data analysis scenarios
and is scalable. The annotators meticulously design the evaluation suite to
ensure the accuracy and robustness of the evaluation. We develop the DA-Agent
baseline. Experiments show that although the baseline performs better than
other existing frameworks, using the current best LLMs achieves only 30.5%
accuracy, leaving ample room for improvement. We release our benchmark at
https://da-code-bench.github.io.
Multi-Agent Collaborative Data Selection for Efficient LLM Pretraining
Authors: name: Tianyi Bai, name: Ling Yang, name: Zhen Hao Wong, name: Jiahui Peng, name: Xinlin Zhuang, name: Chi Zhang, name: Lijun Wu, name: Qiu Jiantao, name: Wentao Zhang, name: Binhang Yuan, name: Conghui He
Published: 2024-10-10 16:45:28

Efficient data selection is crucial to accelerate the pretraining of large
language models (LLMs). While various methods have been proposed to enhance
data efficiency, limited research has addressed the inherent conflicts between
these approaches to achieve optimal data selection for LLM pretraining. To
tackle this problem, we propose a novel multi-agent collaborative data
selection mechanism. In this framework, each data selection method serves as an
independent agent, and an agent console is designed to dynamically integrate
the information from all agents throughout the LLM training process. We conduct
extensive empirical studies to evaluate our multi-agent framework. The
experimental results demonstrate that our approach significantly improves data
efficiency, accelerates convergence in LLM training, and achieves an average
performance gain of 10.5% across multiple language model benchmarks compared to
the state-of-the-art methods.
StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via
  Inference-time Hybrid Information Structurization
Authors: name: Zhuoqun Li, name: Xuanang Chen, name: Haiyang Yu, name: Hongyu Lin, name: Yaojie Lu, name: Qiaoyu Tang, name: Fei Huang, name: Xianpei Han, name: Le Sun, name: Yongbin Li
Published: 2024-10-11 13:52:44

Retrieval-augmented generation (RAG) is a key means to effectively enhance
large language models (LLMs) in many knowledge-based tasks. However, existing
RAG methods struggle with knowledge-intensive reasoning tasks, because useful
information required to these tasks are badly scattered. This characteristic
makes it difficult for existing RAG methods to accurately identify key
information and perform global reasoning with such noisy augmentation. In this
paper, motivated by the cognitive theories that humans convert raw information
into various structured knowledge when tackling knowledge-intensive reasoning,
we proposes a new framework, StructRAG, which can identify the optimal
structure type for the task at hand, reconstruct original documents into this
structured format, and infer answers based on the resulting structure.
Extensive experiments across various knowledge-intensive tasks show that
StructRAG achieves state-of-the-art performance, particularly excelling in
challenging scenarios, demonstrating its potential as an effective solution for
enhancing LLMs in complex real-world applications.
KV Prediction for Improved Time to First Token
Authors: name: Maxwell Horton, name: Qingqing Cao, name: Chenfan Sun, name: Yanzi Jin, name: Sachin Mehta, name: Mohammad Rastegari, name: Moin Nabi
Published: 2024-10-10 21:55:11

Inference with transformer-based language models begins with a prompt
processing step. In this step, the model generates the first output token and
stores the KV cache needed for future generation steps. This prompt processing
step can be computationally expensive, taking 10s of seconds or more for
billion-parameter models on edge devices when prompt lengths or batch sizes
rise. This degrades user experience by introducing significant latency into the
model's outputs. To reduce the time spent producing the first output (known as
the ``time to first token'', or TTFT) of a pretrained model, we introduce a
novel method called KV Prediction. In our method, a small auxiliary model is
used to process the prompt and produce an approximation of the KV cache used by
a base model. This approximated KV cache is then used with the base model for
autoregressive generation without the need to query the auxiliary model again.
We demonstrate that our method produces a pareto-optimal efficiency-accuracy
trade-off when compared to baselines. On TriviaQA, we demonstrate relative
accuracy improvements in the range of 15%-50% across a range of TTFT FLOPs
budgets. We also demonstrate accuracy improvements of up to 30% on HumanEval
python code completion at fixed TTFT FLOPs budgets. Additionally, we benchmark
models on an Apple M2 Pro CPU and demonstrate that our improvement in FLOPs
translates to a TTFT speedup on hardware. We release our code at
https://github.com/apple/corenet/tree/main/projects/kv-prediction .
From Generalist to Specialist: Adapting Vision Language Models via
  Task-Specific Visual Instruction Tuning
Authors: name: Yang Bai, name: Yang Zhou, name: Jun Zhou, name: Rick Siow Mong Goh, name: Daniel Shu Wei Ting, name: Yong Liu
Published: 2024-10-09 01:24:04

Large vision language models (VLMs) combine large language models with vision
encoders, demonstrating promise across various tasks. However, they often
underperform in task-specific applications due to domain gaps between
pre-training and fine-tuning. We introduce VITask, a novel framework that
enhances task-specific adaptability of VLMs by integrating task-specific models
(TSMs). VITask employs three key strategies: exemplar prompting (EP), response
distribution alignment (RDA), and contrastive response tuning (CRT) to improve
the task-specific performance of VLMs by adjusting their response
distributions. EP allows TSM features to guide VLMs, while RDA enables VLMs to
adapt without TSMs during inference by learning from exemplar-prompted models.
CRT further optimizes the ranking of correct image-response pairs, thereby
reducing the risk of generating undesired responses. Experiments on 12 medical
diagnosis datasets across 9 imaging modalities show that VITask outperforms
both vanilla instruction-tuned VLMs and TSMs, showcasing its ability to
integrate complementary features from both models effectively. Additionally,
VITask offers practical advantages such as flexible TSM integration and
robustness to incomplete instructions, making it a versatile and efficient
solution for task-specific VLM tuning. Our code are available at
https://github.com/baiyang4/VITask.
PositionID: LLMs can Control Lengths, Copy and Paste with Explicit
  Positional Awareness
Authors: name: Zekun Wang, name: Feiyu Duan, name: Yibo Zhang, name: Wangchunshu Zhou, name: Ke Xu, name: Wenhao Huang, name: Jie Fu
Published: 2024-10-09 16:15:36

Large Language Models (LLMs) demonstrate impressive capabilities across
various domains, including role-playing, creative writing, mathematical
reasoning, and coding. Despite these advancements, LLMs still encounter
challenges with length control, frequently failing to adhere to specific length
constraints due to their token-level operations and insufficient training on
data with strict length limitations. We identify this issue as stemming from a
lack of positional awareness and propose novel approaches--PositionID Prompting
and PositionID Fine-Tuning--to address it. These methods enhance the model's
ability to continuously monitor and manage text length during generation.
Additionally, we introduce PositionID CP Prompting to enable LLMs to perform
copy and paste operations accurately. Furthermore, we develop two benchmarks
for evaluating length control and copy-paste abilities. Our experiments
demonstrate that our methods significantly improve the model's adherence to
length constraints and copy-paste accuracy without compromising response
quality.
Meissonic: Revitalizing Masked Generative Transformers for Efficient
  High-Resolution Text-to-Image Synthesis
Authors: name: Jinbin Bai, name: Tian Ye, name: Wei Chow, name: Enxin Song, name: Qing-Guo Chen, name: Xiangtai Li, name: Zhen Dong, name: Lei Zhu, name: Shuicheng Yan
Published: 2024-10-10 17:59:17

Diffusion models, such as Stable Diffusion, have made significant strides in
visual generation, yet their paradigm remains fundamentally different from
autoregressive language models, complicating the development of unified
language-vision models. Recent efforts like LlamaGen have attempted
autoregressive image generation using discrete VQVAE tokens, but the large
number of tokens involved renders this approach inefficient and slow. In this
work, we present Meissonic, which elevates non-autoregressive masked image
modeling (MIM) text-to-image to a level comparable with state-of-the-art
diffusion models like SDXL. By incorporating a comprehensive suite of
architectural innovations, advanced positional encoding strategies, and
optimized sampling conditions, Meissonic substantially improves MIM's
performance and efficiency. Additionally, we leverage high-quality training
data, integrate micro-conditions informed by human preference scores, and
employ feature compression layers to further enhance image fidelity and
resolution. Our model not only matches but often exceeds the performance of
existing models like SDXL in generating high-quality, high-resolution images.
Extensive experiments validate Meissonic's capabilities, demonstrating its
potential as a new standard in text-to-image synthesis. We release a model
checkpoint capable of producing 1024 times 1024 resolution images.
Baichuan-Omni Technical Report
Authors: name: Yadong Li, name: Haoze Sun, name: Mingan Lin, name: Tianpeng Li, name: Guosheng Dong, name: Tao Zhang, name: Bowen Ding, name: Wei Song, name: Zhenglin Cheng, name: Yuqi Huo, name: Song Chen, name: Xu Li, name: Da Pan, name: Shusen Zhang, name: Xin Wu, name: Zheng Liang, name: Jun Liu, name: Tao Zhang, name: Keer Lu, name: Yaqi Zhao, name: Yanjun Shen, name: Fan Yang, name: Kaicheng Yu, name: Tao Lin, name: Jianhua Xu, name: Zenan Zhou, name: Weipeng Chen
Published: 2024-10-11 06:44:31

The salient multimodal capabilities and interactive experience of GPT-4o
highlight its critical role in practical applications, yet it lacks a
high-performing open-source counterpart. In this paper, we introduce
Baichuan-Omni, the first open-source 7B Multimodal Large Language Model (MLLM)
adept at concurrently processing and analyzing modalities of image, video,
audio, and text, while delivering an advanced multimodal interactive experience
and strong performance. We propose an effective multimodal training schema
starting with 7B model and proceeding through two stages of multimodal
alignment and multitask fine-tuning across audio, image, video, and text modal.
This approach equips the language model with the ability to handle visual and
audio data effectively. Demonstrating strong performance across various
omni-modal and multimodal benchmarks, we aim for this contribution to serve as
a competitive baseline for the open-source community in advancing multimodal
understanding and real-time interaction.
Semantic Score Distillation Sampling for Compositional Text-to-3D
  Generation
Authors: name: Ling Yang, name: Zixiang Zhang, name: Junlin Han, name: Bohan Zeng, name: Runjia Li, name: Philip Torr, name: Wentao Zhang
Published: 2024-10-11 17:26:00

Generating high-quality 3D assets from textual descriptions remains a pivotal
challenge in computer graphics and vision research. Due to the scarcity of 3D
data, state-of-the-art approaches utilize pre-trained 2D diffusion priors,
optimized through Score Distillation Sampling (SDS). Despite progress, crafting
complex 3D scenes featuring multiple objects or intricate interactions is still
difficult. To tackle this, recent methods have incorporated box or layout
guidance. However, these layout-guided compositional methods often struggle to
provide fine-grained control, as they are generally coarse and lack
expressiveness. To overcome these challenges, we introduce a novel SDS
approach, Semantic Score Distillation Sampling (SemanticSDS), designed to
effectively improve the expressiveness and accuracy of compositional text-to-3D
generation. Our approach integrates new semantic embeddings that maintain
consistency across different rendering views and clearly differentiate between
various objects and parts. These embeddings are transformed into a semantic
map, which directs a region-specific SDS process, enabling precise optimization
and compositional generation. By leveraging explicit semantic guidance, our
method unlocks the compositional capabilities of existing pre-trained diffusion
models, thereby achieving superior quality in 3D content generation,
particularly for complex objects and scenes. Experimental results demonstrate
that our SemanticSDS framework is highly effective for generating
state-of-the-art complex 3D content. Code:
https://github.com/YangLing0818/SemanticSDS-3D
SuperCorrect: Supervising and Correcting Language Models with
  Error-Driven Insights
Authors: name: Ling Yang, name: Zhaochen Yu, name: Tianjun Zhang, name: Minkai Xu, name: Joseph E. Gonzalez, name: Bin Cui, name: Shuicheng Yan
Published: 2024-10-11 17:25:52

Large language models (LLMs) like GPT-4, PaLM, and LLaMA have shown
significant improvements in various reasoning tasks. However, smaller models
such as Llama-3-8B and DeepSeekMath-Base still struggle with complex
mathematical reasoning because they fail to effectively identify and correct
reasoning errors. Recent reflection-based methods aim to address these issues
by enabling self-reflection and self-correction, but they still face challenges
in independently detecting errors in their reasoning steps. To overcome these
limitations, we propose SuperCorrect, a novel two-stage framework that uses a
large teacher model to supervise and correct both the reasoning and reflection
processes of a smaller student model. In the first stage, we extract
hierarchical high-level and detailed thought templates from the teacher model
to guide the student model in eliciting more fine-grained reasoning thoughts.
In the second stage, we introduce cross-model collaborative direct preference
optimization (DPO) to enhance the self-correction abilities of the student
model by following the teacher's correction traces during training. This
cross-model DPO approach teaches the student model to effectively locate and
resolve erroneous thoughts with error-driven insights from the teacher model,
breaking the bottleneck of its thoughts and acquiring new skills and knowledge
to tackle challenging problems. Extensive experiments consistently demonstrate
our superiority over previous methods. Notably, our SuperCorrect-7B model
significantly surpasses powerful DeepSeekMath-7B by 7.8%/5.3% and
Qwen2.5-Math-7B by 15.1%/6.3% on MATH/GSM8K benchmarks, achieving new SOTA
performance among all 7B models. Code:
https://github.com/YangLing0818/SuperCorrect-llm
EvolveDirector: Approaching Advanced Text-to-Image Generation with Large
  Vision-Language Models
Authors: name: Rui Zhao, name: Hangjie Yuan, name: Yujie Wei, name: Shiwei Zhang, name: Yuchao Gu, name: Lingmin Ran, name: Xiang Wang, name: Zhangjie Wu, name: Junhao Zhang, name: Yingya Zhang, name: Mike Zheng Shou
Published: 2024-10-09 17:52:28

Recent advancements in generation models have showcased remarkable
capabilities in generating fantastic content. However, most of them are trained
on proprietary high-quality data, and some models withhold their parameters and
only provide accessible application programming interfaces (APIs), limiting
their benefits for downstream tasks. To explore the feasibility of training a
text-to-image generation model comparable to advanced models using publicly
available resources, we introduce EvolveDirector. This framework interacts with
advanced models through their public APIs to obtain text-image data pairs to
train a base model. Our experiments with extensive data indicate that the model
trained on generated data of the advanced model can approximate its generation
capability. However, it requires large-scale samples of 10 million or more.
This incurs significant expenses in time, computational resources, and
especially the costs associated with calling fee-based APIs. To address this
problem, we leverage pre-trained large vision-language models (VLMs) to guide
the evolution of the base model. VLM continuously evaluates the base model
during training and dynamically updates and refines the training dataset by the
discrimination, expansion, deletion, and mutation operations. Experimental
results show that this paradigm significantly reduces the required data volume.
Furthermore, when approaching multiple advanced models, EvolveDirector can
select the best samples generated by them to learn powerful and balanced
abilities. The final trained model Edgen is demonstrated to outperform these
advanced models. The code and model weights are available at
https://github.com/showlab/EvolveDirector.
Accelerated Preference Optimization for Large Language Model Alignment
Authors: name: Jiafan He, name: Huizhuo Yuan, name: Quanquan Gu
Published: 2024-10-08 18:51:01

Reinforcement Learning from Human Feedback (RLHF) has emerged as a pivotal
tool for aligning large language models (LLMs) with human preferences. Direct
Preference Optimization (DPO), one of the most popular approaches, formulates
RLHF as a policy optimization problem without explicitly estimating the reward
function. It overcomes the stability and efficiency issues of two-step
approaches, which typically involve first estimating the reward function and
then optimizing the policy via proximal policy optimization (PPO). Since RLHF
is essentially an optimization problem, and it is well-known that momentum
techniques can accelerate optimization both theoretically and empirically, a
natural question arises: Can RLHF be accelerated by momentum? This paper
answers this question in the affirmative. In detail, we first show that the
iterative preference optimization method can be viewed as a proximal point
method. Based on this observation, we propose a general Accelerated Preference
Optimization (APO) framework, which unifies many existing preference
optimization algorithms and employs Nesterov's momentum technique to speed up
the alignment of LLMs. Theoretically, we demonstrate that APO can achieve a
faster convergence rate than the standard iterative preference optimization
methods, including DPO and Self-Play Preference Optimization (SPPO).
Empirically, we show the superiority of APO over DPO, iterative DPO, and other
strong baselines for RLHF on the AlpacaEval 2.0 benchmark.
Data Advisor: Dynamic Data Curation for Safety Alignment of Large
  Language Models
Authors: name: Fei Wang, name: Ninareh Mehrabi, name: Palash Goyal, name: Rahul Gupta, name: Kai-Wei Chang, name: Aram Galstyan
Published: 2024-10-07 17:59:58

Data is a crucial element in large language model (LLM) alignment. Recent
studies have explored using LLMs for efficient data collection. However,
LLM-generated data often suffers from quality issues, with underrepresented or
absent aspects and low-quality datapoints. To address these problems, we
propose Data Advisor, an enhanced LLM-based method for generating data that
takes into account the characteristics of the desired dataset. Starting from a
set of pre-defined principles in hand, Data Advisor monitors the status of the
generated data, identifies weaknesses in the current dataset, and advises the
next iteration of data generation accordingly. Data Advisor can be easily
integrated into existing data generation methods to enhance data quality and
coverage. Experiments on safety alignment of three representative LLMs (i.e.,
Mistral, Llama2, and Falcon) demonstrate the effectiveness of Data Advisor in
enhancing model safety against various fine-grained safety issues without
sacrificing model utility.
WALL-E: World Alignment by Rule Learning Improves World Model-based LLM
  Agents
Authors: name: Siyu Zhou, name: Tianyi Zhou, name: Yijun Yang, name: Guodong Long, name: Deheng Ye, name: Jing Jiang, name: Chengqi Zhang
Published: 2024-10-09 23:37:36

Can large language models (LLMs) directly serve as powerful world models for
model-based agents? While the gaps between the prior knowledge of LLMs and the
specified environment's dynamics do exist, our study reveals that the gaps can
be bridged by aligning an LLM with its deployed environment and such "world
alignment" can be efficiently achieved by rule learning on LLMs. Given the rich
prior knowledge of LLMs, only a few additional rules suffice to align LLM
predictions with the specified environment dynamics. To this end, we propose a
neurosymbolic approach to learn these rules gradient-free through LLMs, by
inducing, updating, and pruning rules based on comparisons of agent-explored
trajectories and world model predictions. The resulting world model is composed
of the LLM and the learned rules. Our embodied LLM agent "WALL-E" is built upon
model-predictive control (MPC). By optimizing look-ahead actions based on the
precise world model, MPC significantly improves exploration and learning
efficiency. Compared to existing LLM agents, WALL-E's reasoning only requires a
few principal rules rather than verbose buffered trajectories being included in
the LLM input. On open-world challenges in Minecraft and ALFWorld, WALL-E
achieves higher success rates than existing methods, with lower costs on
replanning time and the number of tokens used for reasoning. In Minecraft,
WALL-E exceeds baselines by 15-30% in success rate while costing 8-20 fewer
replanning rounds and only 60-80% of tokens. In ALFWorld, its success rate
surges to a new record high of 95% only after 6 iterations.