Orionnnn/hub-stats huggingface dataset:
SemiEvol: Semi-supervised Fine-tuning for LLM Adaptation
Authors: name: Junyu Luo, name: Xiao Luo, name: Xiusi Chen, name: Zhiping Xiao, name: Wei Ju, name: Ming Zhang
Published: 2024-10-17 16:59:46

Supervised fine-tuning (SFT) is crucial in adapting large language models
(LLMs) to a specific domain or task. However, only a limited amount of labeled
data is available in practical applications, which poses a severe challenge for
SFT in yielding satisfactory results. Therefore, a data-efficient framework
that can fully exploit labeled and unlabeled data for LLM fine-tuning is highly
anticipated. Towards this end, we introduce a semi-supervised fine-tuning
framework named SemiEvol for LLM adaptation from a propagate-and-select manner.
For knowledge propagation, SemiEvol adopts a bi-level approach, propagating
knowledge from labeled data to unlabeled data through both in-weight and
in-context methods. For knowledge selection, SemiEvol incorporates a
collaborative learning mechanism, selecting higher-quality pseudo-response
samples. We conducted experiments using GPT-4o-mini and Llama-3.1 on seven
general or domain-specific datasets, demonstrating significant improvements in
model performance on target data. Furthermore, we compared SemiEvol with SFT
and self-evolution methods, highlighting its practicality in hybrid data
scenarios.
CBT-Bench: Evaluating Large Language Models on Assisting Cognitive
  Behavior Therapy
Authors: name: Mian Zhang, name: Xianjun Yang, name: Xinlu Zhang, name: Travis Labrum, name: Jamie C. Chiu, name: Shaun M. Eack, name: Fei Fang, name: William Yang Wang, name: Zhiyu Zoey Chen
Published: 2024-10-17 04:52:57

There is a significant gap between patient needs and available mental health
support today. In this paper, we aim to thoroughly examine the potential of
using Large Language Models (LLMs) to assist professional psychotherapy. To
this end, we propose a new benchmark, CBT-BENCH, for the systematic evaluation
of cognitive behavioral therapy (CBT) assistance. We include three levels of
tasks in CBT-BENCH: I: Basic CBT knowledge acquisition, with the task of
multiple-choice questions; II: Cognitive model understanding, with the tasks of
cognitive distortion classification, primary core belief classification, and
fine-grained core belief classification; III: Therapeutic response generation,
with the task of generating responses to patient speech in CBT therapy
sessions. These tasks encompass key aspects of CBT that could potentially be
enhanced through AI assistance, while also outlining a hierarchy of capability
requirements, ranging from basic knowledge recitation to engaging in real
therapeutic conversations. We evaluated representative LLMs on our benchmark.
Experimental results indicate that while LLMs perform well in reciting CBT
knowledge, they fall short in complex real-world scenarios requiring deep
analysis of patients' cognitive structures and generating effective responses,
suggesting potential future work.
SHAKTI: A 2.5 Billion Parameter Small Language Model Optimized for Edge
  AI and Low-Resource Environments
Authors: name: Syed Abdul Gaffar Shakhadri, name: Kruthika KR, name: Rakshit Aralimatti
Published: 2024-10-15 06:59:44

We introduce Shakti, a 2.5 billion parameter language model specifically
optimized for resource-constrained environments such as edge devices, including
smartphones, wearables, and IoT systems. Shakti combines high-performance NLP
with optimized efficiency and precision, making it ideal for real-time AI
applications where computational resources and memory are limited. With support
for vernacular languages and domain-specific tasks, Shakti excels in industries
such as healthcare, finance, and customer service. Benchmark evaluations
demonstrate that Shakti performs competitively against larger models while
maintaining low latency and on-device efficiency, positioning it as a leading
solution for edge AI.
FiTv2: Scalable and Improved Flexible Vision Transformer for Diffusion
  Model
Authors: name: ZiDong Wang, name: Zeyu Lu, name: Di Huang, name: Cai Zhou, name: Wanli Ouyang, name: and Lei Bai
Published: 2024-10-17 15:51:49

Nature is infinitely resolution-free. In the context of this
reality, existing diffusion models, such as Diffusion Transformers, often face
challenges when processing image resolutions outside of their trained domain.
To address this limitation, we conceptualize images as sequences of tokens with
dynamic sizes, rather than traditional methods that perceive images as
fixed-resolution grids. This perspective enables a flexible training strategy
that seamlessly accommodates various aspect ratios during both training and
inference, thus promoting resolution generalization and eliminating biases
introduced by image cropping. On this basis, we present the Flexible
Vision Transformer (FiT), a transformer architecture specifically designed for
generating images with unrestricted resolutions and aspect ratios. We
further upgrade the FiT to FiTv2 with several innovative designs, includingthe
Query-Key vector normalization, the AdaLN-LoRA module, a rectified flow
scheduler, and a Logit-Normal sampler. Enhanced by a meticulously adjusted
network structure, FiTv2 exhibits 2times convergence speed of FiT. When
incorporating advanced training-free extrapolation techniques, FiTv2
demonstrates remarkable adaptability in both resolution extrapolation and
diverse resolution generation. Additionally, our exploration of the scalability
of the FiTv2 model reveals that larger models exhibit better computational
efficiency. Furthermore, we introduce an efficient post-training strategy to
adapt a pre-trained model for the high-resolution generation. Comprehensive
experiments demonstrate the exceptional performance of FiTv2 across a broad
range of resolutions. We have released all the codes and models at
https://github.com/whlzy/FiT to promote the exploration of diffusion
transformer models for arbitrary-resolution image generation.
Mini-Omni2: Towards Open-source GPT-4o with Vision, Speech and Duplex
  Capabilities
Authors: name: Zhifei Xie, name: Changqiao Wu
Published: 2024-10-15 02:10:45

GPT-4o, an all-encompassing model, represents a milestone in the development
of large multi-modal language models. It can understand visual, auditory, and
textual modalities, directly output audio, and support flexible duplex
interaction. Models from the open-source community often achieve some
functionalities of GPT-4o, such as visual understanding and voice chat.
Nevertheless, training a unified model that incorporates all modalities is
challenging due to the complexities of multi-modal data, intricate model
architectures, and training processes. In this paper, we introduce Mini-Omni2,
a visual-audio assistant capable of providing real-time, end-to-end voice
responses to visoin and audio queries. By integrating pretrained visual and
auditory encoders, Mini-Omni2 maintains performance in individual modalities.
We propose a three-stage training process to align modalities, allowing the
language model to handle multi-modal inputs and outputs after training on a
limited dataset. For interaction, we introduce a command-based interruption
mechanism, enabling more flexible interaction with users. To the best of our
knowledge, Mini-Omni2 is one of the closest reproductions of GPT-4o, which have
similar form of functionality, and we hope it can offer valuable insights for
subsequent research.
HART: Efficient Visual Generation with Hybrid Autoregressive Transformer
Authors: name: Haotian Tang, name: Yecheng Wu, name: Shang Yang, name: Enze Xie, name: Junsong Chen, name: Junyu Chen, name: Zhuoyang Zhang, name: Han Cai, name: Yao Lu, name: Song Han
Published: 2024-10-14 17:59:42

We introduce Hybrid Autoregressive Transformer (HART), an autoregressive (AR)
visual generation model capable of directly generating 1024x1024 images,
rivaling diffusion models in image generation quality. Existing AR models face
limitations due to the poor image reconstruction quality of their discrete
tokenizers and the prohibitive training costs associated with generating 1024px
images. To address these challenges, we present the hybrid tokenizer, which
decomposes the continuous latents from the autoencoder into two components:
discrete tokens representing the big picture and continuous tokens representing
the residual components that cannot be represented by the discrete tokens. The
discrete component is modeled by a scalable-resolution discrete AR model, while
the continuous component is learned with a lightweight residual diffusion
module with only 37M parameters. Compared with the discrete-only VAR tokenizer,
our hybrid approach improves reconstruction FID from 2.11 to 0.30 on MJHQ-30K,
leading to a 31% generation FID improvement from 7.85 to 5.38. HART also
outperforms state-of-the-art diffusion models in both FID and CLIP score, with
4.5-7.7x higher throughput and 6.9-13.4x lower MACs. Our code is open sourced
at https://github.com/mit-han-lab/hart.
BiGR: Harnessing Binary Latent Codes for Image Generation and Improved
  Visual Representation Capabilities
Authors: name: Shaozhe Hao, name: Xuantong Liu, name: Xianbiao Qi, name: Shihao Zhao, name: Bojia Zi, name: Rong Xiao, name: Kai Han, name: Kwan-Yee K. Wong
Published: 2024-10-18 17:59:04

We introduce BiGR, a novel conditional image generation model using compact
binary latent codes for generative training, focusing on enhancing both
generation and representation capabilities. BiGR is the first conditional
generative model that unifies generation and discrimination within the same
framework. BiGR features a binary tokenizer, a masked modeling mechanism, and a
binary transcoder for binary code prediction. Additionally, we introduce a
novel entropy-ordered sampling method to enable efficient image generation.
Extensive experiments validate BiGR's superior performance in generation
quality, as measured by FID-50k, and representation capabilities, as evidenced
by linear-probe accuracy. Moreover, BiGR showcases zero-shot generalization
across various vision tasks, enabling applications such as image inpainting,
outpainting, editing, interpolation, and enrichment, without the need for
structural modifications. Our findings suggest that BiGR unifies generative and
discriminative tasks effectively, paving the way for further advancements in
the field.
Montessori-Instruct: Generate Influential Training Data Tailored for
  Student Learning
Authors: name: Xiaochuan Li, name: Zichun Yu, name: Chenyan Xiong
Published: 2024-10-18 06:50:15

Synthetic data has been widely used to train large language models, but their
generative nature inevitably introduces noisy, non-informative, and misleading
learning signals. In this paper, we propose Montessori-Instruct, a novel data
synthesis framework that tailors the data synthesis ability of the teacher
language model toward the student language model's learning process.
Specifically, we utilize local data influence of synthetic training data points
on students to characterize students' learning preferences. Then, we train the
teacher model with Direct Preference Optimization (DPO) to generate synthetic
data tailored toward student learning preferences. Experiments with
Llama3-8B-Instruct (teacher) and Llama3-8B (student) on Alpaca Eval and
MT-Bench demonstrate that Montessori-Instruct significantly outperforms
standard synthesis methods by 18.35\% and 46.24\% relatively. Our method also
beats data synthesized by a stronger teacher model, GPT-4o. Further analysis
confirms the benefits of teacher's learning to generate more influential
training data in the student's improved learning, the advantages of local data
influence in accurately measuring student preferences, and the robustness of
Montessori-Instruct across different student models. Our code and data are
open-sourced at https://github.com/cxcscmu/Montessori-Instruct.
Teaching Models to Balance Resisting and Accepting Persuasion
Authors: name: Elias Stengel-Eskin, name: Peter Hase, name: Mohit Bansal
Published: 2024-10-18 16:49:36

Large language models (LLMs) are susceptible to persuasion, which can pose
risks when models are faced with an adversarial interlocutor. We take a first
step towards defending models against persuasion while also arguing that
defense against adversarial (i.e. negative) persuasion is only half of the
equation: models should also be able to accept beneficial (i.e. positive)
persuasion to improve their answers. We show that optimizing models for only
one side results in poor performance on the other. In order to balance positive
and negative persuasion, we introduce Persuasion-Balanced Training (or PBT),
which leverages multi-agent recursive dialogue trees to create data and trains
models via preference optimization to accept persuasion when appropriate. PBT
consistently improves resistance to misinformation and resilience to being
challenged while also resulting in the best overall performance on holistic
data containing both positive and negative persuasion. Crucially, we show that
PBT models are better teammates in multi-agent debates. We find that without
PBT, pairs of stronger and weaker models have unstable performance, with the
order in which the models present their answers determining whether the team
obtains the stronger or weaker model's performance. PBT leads to better and
more stable results and less order dependence, with the stronger model
consistently pulling the weaker one up.
Looking Inward: Language Models Can Learn About Themselves by
  Introspection
Authors: name: Felix J Binder, name: James Chua, name: Tomek Korbak, name: Henry Sleight, name: John Hughes, name: Robert Long, name: Ethan Perez, name: Miles Turpin, name: Owain Evans
Published: 2024-10-17 17:24:10

Humans acquire knowledge by observing the external world, but also by
introspection. Introspection gives a person privileged access to their current
state of mind (e.g., thoughts and feelings) that is not accessible to external
observers. Can LLMs introspect? We define introspection as acquiring knowledge
that is not contained in or derived from training data but instead originates
from internal states. Such a capability could enhance model interpretability.
Instead of painstakingly analyzing a model's internal workings, we could simply
ask the model about its beliefs, world models, and goals. More speculatively,
an introspective model might self-report on whether it possesses certain
internal states such as subjective feelings or desires and this could inform us
about the moral status of these states. Such self-reports would not be entirely
dictated by the model's training data.
  We study introspection by finetuning LLMs to predict properties of their own
behavior in hypothetical scenarios. For example, "Given the input P, would your
output favor the short- or long-term option?" If a model M1 can introspect, it
should outperform a different model M2 in predicting M1's behavior even if M2
is trained on M1's ground-truth behavior. The idea is that M1 has privileged
access to its own behavioral tendencies, and this enables it to predict itself
better than M2 (even if M2 is generally stronger).
  In experiments with GPT-4, GPT-4o, and Llama-3 models (each finetuned to
predict itself), we find that the model M1 outperforms M2 in predicting itself,
providing evidence for introspection. Notably, M1 continues to predict its
behavior accurately even after we intentionally modify its ground-truth
behavior. However, while we successfully elicit introspection on simple tasks,
we are unsuccessful on more complex tasks or those requiring
out-of-distribution generalization.
How Do Training Methods Influence the Utilization of Vision Models?
Authors: name: Paul Gavrikov, name: Shashank Agnihotri, name: Margret Keuper, name: Janis Keuper
Published: 2024-10-18 13:54:46

Not all learnable parameters (e.g., weights) contribute equally to a neural
network's decision function. In fact, entire layers' parameters can sometimes
be reset to random values with little to no impact on the model's decisions. We
revisit earlier studies that examined how architecture and task complexity
influence this phenomenon and ask: is this phenomenon also affected by how we
train the model? We conducted experimental evaluations on a diverse set of
ImageNet-1k classification models to explore this, keeping the architecture and
training data constant but varying the training pipeline. Our findings reveal
that the training method strongly influences which layers become critical to
the decision function for a given task. For example, improved training regimes
and self-supervised training increase the importance of early layers while
significantly under-utilizing deeper layers. In contrast, methods such as
adversarial training display an opposite trend. Our preliminary results extend
previous findings, offering a more nuanced understanding of the inner mechanics
of neural networks.
  Code: https://github.com/paulgavrikov/layer_criticality
NaturalBench: Evaluating Vision-Language Models on Natural Adversarial
  Samples
Authors: name: Baiqi Li, name: Zhiqiu Lin, name: Wenxuan Peng, name: Jean de Dieu Nyandwi, name: Daniel Jiang, name: Zixian Ma, name: Simran Khanuja, name: Ranjay Krishna, name: Graham Neubig, name: Deva Ramanan
Published: 2024-10-18 17:58:21

Vision-language models (VLMs) have made significant progress in recent
visual-question-answering (VQA) benchmarks that evaluate complex
visio-linguistic reasoning. However, are these models truly effective? In this
work, we show that VLMs still struggle with natural images and questions that
humans can easily answer, which we term natural adversarial samples. We also
find it surprisingly easy to generate these VQA samples from natural image-text
corpora using off-the-shelf models like CLIP and ChatGPT. We propose a
semi-automated approach to collect a new benchmark, NaturalBench, for reliably
evaluating VLMs with 10,000 human-verified VQA samples. Crucially, we adopt a
vision-centric design by pairing each question with two images that
yield different answers, preventing blind solutions from answering without
using the images. This makes NaturalBench more challenging than previous
benchmarks that can be solved with commonsense priors. We evaluate 53
state-of-the-art VLMs on NaturalBench, showing that models like
LLaVA-OneVision, Cambrian-1, Llama3.2-Vision, Molmo, Qwen2-VL, and even GPT-4o
lag 50%-70% behind human performance (over 90%). We analyze why NaturalBench is
hard from two angles: (1) Compositionality: Solving NaturalBench requires
diverse visio-linguistic skills, including understanding attribute bindings,
object relationships, and advanced reasoning like logic and counting. To this
end, unlike prior work that uses a single tag per sample, we tag each
NaturalBench sample with 1 to 8 skill tags for fine-grained evaluation. (2)
Biases: NaturalBench exposes severe biases in VLMs, as models often choose the
same answer regardless of the image. Lastly, we apply our benchmark curation
method to diverse data sources, including long captions (over 100 words) and
non-English languages like Chinese and Hindi, highlighting its potential for
dynamic evaluations of VLMs.
DAWN: Dynamic Frame Avatar with Non-autoregressive Diffusion Framework
  for Talking Head Video Generation
Authors: name: Hanbo Cheng, name: Limin Lin, name: Chenyu Liu, name: Pengcheng Xia, name: Pengfei Hu, name: Jiefeng Ma, name: Jun Du, name: Jia Pan
Published: 2024-10-17 16:32:36

Talking head generation intends to produce vivid and realistic talking head
videos from a single portrait and speech audio clip. Although significant
progress has been made in diffusion-based talking head generation, almost all
methods rely on autoregressive strategies, which suffer from limited context
utilization beyond the current generation step, error accumulation, and slower
generation speed. To address these challenges, we present DAWN (Dynamic frame
Avatar With Non-autoregressive diffusion), a framework that enables all-at-once
generation of dynamic-length video sequences. Specifically, it consists of two
main components: (1) audio-driven holistic facial dynamics generation in the
latent motion space, and (2) audio-driven head pose and blink generation.
Extensive experiments demonstrate that our method generates authentic and vivid
videos with precise lip motions, and natural pose/blink movements.
Additionally, with a high generation speed, DAWN possesses strong extrapolation
capabilities, ensuring the stable production of high-quality long videos. These
results highlight the considerable promise and potential impact of DAWN in the
field of talking head video generation. Furthermore, we hope that DAWN sparks
further exploration of non-autoregressive approaches in diffusion models. Our
code will be publicly at https://github.com/Hanbo-Cheng/DAWN-pytorch.
A Common Pitfall of Margin-based Language Model Alignment: Gradient
  Entanglement
Authors: name: Hui Yuan, name: Yifan Zeng, name: Yue Wu, name: Huazheng Wang, name: Mengdi Wang, name: Liu Leqi
Published: 2024-10-17 17:52:01

Reinforcement Learning from Human Feedback (RLHF) has become the predominant
approach for language model (LM) alignment. At its core, RLHF uses a
margin-based loss for preference optimization, specifying ideal LM behavior
only by the difference between preferred and dispreferred responses. In this
paper, we identify a common pitfall of margin-based methods -- the
under-specification of ideal LM behavior on preferred and dispreferred
responses individually, which leads to two unintended consequences as the
margin increases: (1) The probability of dispreferred (e.g., unsafe) responses
may increase, resulting in potential safety alignment failures. (2) The
probability of preferred responses may decrease, even when those responses are
ideal. We demystify the reasons behind these problematic behaviors:
margin-based losses couple the change in the preferred probability to the
gradient of the dispreferred one, and vice versa, often preventing the
preferred probability from increasing while the dispreferred one decreases, and
thus causing a synchronized increase or decrease in both probabilities. We term
this effect, inherent in margin-based objectives, gradient entanglement.
Formally, we derive conditions for general margin-based alignment objectives
under which gradient entanglement becomes concerning: the inner product of the
gradients of preferred and dispreferred log-probabilities is large relative to
the individual gradient norms. We theoretically investigate why such inner
products can be large when aligning language models and empirically validate
our findings. Empirical implications of our framework extend to explaining
important differences in the training dynamics of various preference
optimization algorithms, and suggesting potential algorithm designs to mitigate
the under-specification issue of margin-based methods and thereby improving
language model alignment.