Orionnnn/hub-stats huggingface dataset:
Jailbreaking as a Reward Misspecification Problem
Authors: name: Zhihui Xie, name: Jiahui Gao, name: Lei Li, name: Zhenguo Li, name: Qi Liu, name: Lingpeng Kong
Published: 2024-06-20 15:12:27

The widespread adoption of large language models (LLMs) has raised concerns
about their safety and reliability, particularly regarding their vulnerability
to adversarial attacks. In this paper, we propose a novel perspective that
attributes this vulnerability to reward misspecification during the alignment
process. We introduce a metric ReGap to quantify the extent of reward
misspecification and demonstrate its effectiveness and robustness in detecting
harmful backdoor prompts. Building upon these insights, we present ReMiss, a
system for automated red teaming that generates adversarial prompts against
various target aligned LLMs. ReMiss achieves state-of-the-art attack success
rates on the AdvBench benchmark while preserving the human readability of the
generated prompts. Detailed analysis highlights the unique advantages brought
by the proposed reward misspecification objective compared to previous methods.
ToVo: Toxicity Taxonomy via Voting
Authors: name: Tinh Son Luong, name: Thanh-Thien Le, name: Thang Viet Doan, name: Linh Ngo Van, name: Thien Huu Nguyen, name: Diep Thi-Ngoc Nguyen
Published: 2024-06-21 02:35:30

Existing toxic detection models face significant limitations, such as lack of
transparency, customization, and reproducibility. These challenges stem from
the closed-source nature of their training data and the paucity of explanations
for their evaluation mechanism. To address these issues, we propose a dataset
creation mechanism that integrates voting and chain-of-thought processes,
producing a high-quality open-source dataset for toxic content detection. Our
methodology ensures diverse classification metrics for each sample and includes
both classification scores and explanatory reasoning for the classifications.
  We utilize the dataset created through our proposed mechanism to train our
model, which is then compared against existing widely-used detectors. Our
approach not only enhances transparency and customizability but also
facilitates better fine-tuning for specific use cases. This work contributes a
robust framework for developing toxic content detection models, emphasizing
openness and adaptability, thus paving the way for more effective and
user-specific content moderation solutions.
How Well Do LLMs Represent Values Across Cultures? Empirical Analysis of
  LLM Responses Based on Hofstede Cultural Dimensions
Authors: name: Julia Kharchenko, name: Tanya Roosta, name: Aman Chadha, name: Chirag Shah
Published: 2024-06-21 00:58:01

Large Language Models (LLMs) attempt to imitate human behavior by responding
to humans in a way that pleases them, including by adhering to their values.
However, humans come from diverse cultures with different values. It is
critical to understand whether LLMs showcase different values to the user based
on the stereotypical values of a user's known country. We prompt different LLMs
with a series of advice requests based on 5 Hofstede Cultural Dimensions -- a
quantifiable way of representing the values of a country. Throughout each
prompt, we incorporate personas representing 36 different countries and,
separately, languages predominantly tied to each country to analyze the
consistency in the LLMs' cultural understanding. Through our analysis of the
responses, we found that LLMs can differentiate between one side of a value and
another, as well as understand that countries have differing values, but will
not always uphold the values when giving advice, and fail to understand the
need to answer differently based on different cultural values. Rooted in these
findings, we present recommendations for training value-aligned and culturally
sensitive LLMs. More importantly, the methodology and the framework developed
here can help further understand and mitigate culture and language alignment
issues with LLMs.
Sampling 3D Gaussian Scenes in Seconds with Latent Diffusion Models
Authors: name: Paul Henderson, name: Melonie de Almeida, name: Daniela Ivanova, name: Titas Anciukevičius
Published: 2024-06-18 23:14:29

We present a latent diffusion model over 3D scenes, that can be trained using
only 2D image data. To achieve this, we first design an autoencoder that maps
multi-view images to 3D Gaussian splats, and simultaneously builds a compressed
latent representation of these splats. Then, we train a multi-view diffusion
model over the latent space to learn an efficient generative model. This
pipeline does not require object masks nor depths, and is suitable for complex
scenes with arbitrary camera positions. We conduct careful experiments on two
large-scale datasets of complex real-world scenes -- MVImgNet and
RealEstate10K. We show that our approach enables generating 3D scenes in as
little as 0.2 seconds, either from scratch, from a single input view, or from
sparse input views. It produces diverse and high-quality results while running
an order of magnitude faster than non-latent diffusion models and earlier
NeRF-based generative models
A Systematic Survey of Text Summarization: From Statistical Methods to
  Large Language Models
Authors: name: Haopeng Zhang, name: Philip S. Yu, name: Jiawei Zhang
Published: 2024-06-17 07:52:32

Text summarization research has undergone several significant transformations
with the advent of deep neural networks, pre-trained language models (PLMs),
and recent large language models (LLMs). This survey thus provides a
comprehensive review of the research progress and evolution in text
summarization through the lens of these paradigm shifts. It is organized into
two main parts: (1) a detailed overview of datasets, evaluation metrics, and
summarization methods before the LLM era, encompassing traditional statistical
methods, deep learning approaches, and PLM fine-tuning techniques, and (2) the
first detailed examination of recent advancements in benchmarking, modeling,
and evaluating summarization in the LLM era. By synthesizing existing
literature and presenting a cohesive overview, this survey also discusses
research trends, open challenges, and proposes promising research directions in
summarization, aiming to guide researchers through the evolving landscape of
summarization research.
StableSemantics: A Synthetic Language-Vision Dataset of Semantic
  Representations in Naturalistic Images
Authors: name: Rushikesh Zawar, name: Shaurya Dewan, name: Andrew F. Luo, name: Margaret M. Henderson, name: Michael J. Tarr, name: Leila Wehbe
Published: 2024-06-19 17:59:40

Understanding the semantics of visual scenes is a fundamental challenge in
Computer Vision. A key aspect of this challenge is that objects sharing similar
semantic meanings or functions can exhibit striking visual differences, making
accurate identification and categorization difficult. Recent advancements in
text-to-image frameworks have led to models that implicitly capture natural
scene statistics. These frameworks account for the visual variability of
objects, as well as complex object co-occurrences and sources of noise such as
diverse lighting conditions. By leveraging large-scale datasets and
cross-attention conditioning, these models generate detailed and contextually
rich scene representations. This capability opens new avenues for improving
object recognition and scene understanding in varied and challenging
environments. Our work presents StableSemantics, a dataset comprising 224
thousand human-curated prompts, processed natural language captions, over 2
million synthetic images, and 10 million attention maps corresponding to
individual noun chunks. We explicitly leverage human-generated prompts that
correspond to visually interesting stable diffusion generations, provide 10
generations per phrase, and extract cross-attention maps for each image. We
explore the semantic distribution of generated images, examine the distribution
of objects within images, and benchmark captioning and open vocabulary
segmentation methods on our data. To the best of our knowledge, we are the
first to release a diffusion dataset with semantic attributions. We expect our
proposed dataset to catalyze advances in visual semantic understanding and
provide a foundation for developing more sophisticated and effective visual
models. Website: https://stablesemantics.github.io/StableSemantics
Model Merging and Safety Alignment: One Bad Model Spoils the Bunch
Authors: name: Hasan Abed Al Kader Hammoud, name: Umberto Michieli, name: Fabio Pizzati, name: Philip Torr, name: Adel Bibi, name: Bernard Ghanem, name: Mete Ozay
Published: 2024-06-20 17:59:58

Merging Large Language Models (LLMs) is a cost-effective technique for
combining multiple expert LLMs into a single versatile model, retaining the
expertise of the original ones. However, current approaches often overlook the
importance of safety alignment during merging, leading to highly misaligned
models. This work investigates the effects of model merging on alignment. We
evaluate several popular model merging techniques, demonstrating that existing
methods do not only transfer domain expertise but also propagate misalignment.
We propose a simple two-step approach to address this problem: (i) generating
synthetic safety and domain-specific data, and (ii) incorporating these
generated data into the optimization process of existing data-aware model
merging techniques. This allows us to treat alignment as a skill that can be
maximized in the resulting merged LLM. Our experiments illustrate the
effectiveness of integrating alignment-related data during merging, resulting
in models that excel in both domain expertise and alignment.
From Insights to Actions: The Impact of Interpretability and Analysis
  Research on NLP
Authors: name: Marius Mosbach, name: Vagrant Gautam, name: Tomás Vergara-Browne, name: Dietrich Klakow, name: Mor Geva
Published: 2024-06-18 13:45:07

Interpretability and analysis (IA) research is a growing subfield within NLP
with the goal of developing a deeper understanding of the behavior or inner
workings of NLP systems and methods. Despite growing interest in the subfield,
a commonly voiced criticism is that it lacks actionable insights and therefore
has little impact on NLP. In this paper, we seek to quantify the impact of IA
research on the broader field of NLP. We approach this with a mixed-methods
analysis of: (1) a citation graph of 185K+ papers built from all papers
published at ACL and EMNLP conferences from 2018 to 2023, and (2) a survey of
138 members of the NLP community. Our quantitative results show that IA work is
well-cited outside of IA, and central in the NLP citation graph. Through
qualitative analysis of survey responses and manual annotation of 556 papers,
we find that NLP researchers build on findings from IA work and perceive it is
important for progress in NLP, multiple subfields, and rely on its findings and
terminology for their own work. Many novel methods are proposed based on IA
findings and highly influenced by them, but highly influential non-IA work
cites IA findings without being driven by them. We end by summarizing what is
missing in IA work today and provide a call to action, to pave the way for a
more impactful future of IA research.
Model Internals-based Answer Attribution for Trustworthy
  Retrieval-Augmented Generation
Authors: name: Jirui Qi, name: Gabriele Sarti, name: Raquel Fernández, name: Arianna Bisazza
Published: 2024-06-19 16:10:26

Ensuring the verifiability of model answers is a fundamental challenge for
retrieval-augmented generation (RAG) in the question answering (QA) domain.
Recently, self-citation prompting was proposed to make large language models
(LLMs) generate citations to supporting documents along with their answers.
However, self-citing LLMs often struggle to match the required format, refer to
non-existent sources, and fail to faithfully reflect LLMs' context usage
throughout the generation. In this work, we present MIRAGE --Model
Internals-based RAG Explanations -- a plug-and-play approach using model
internals for faithful answer attribution in RAG applications. MIRAGE detects
context-sensitive answer tokens and pairs them with retrieved documents
contributing to their prediction via saliency methods. We evaluate our proposed
approach on a multilingual extractive QA dataset, finding high agreement with
human answer attribution. On open-ended QA, MIRAGE achieves citation quality
and efficiency comparable to self-citation while also allowing for a
finer-grained control of attribution parameters. Our qualitative evaluation
highlights the faithfulness of MIRAGE's attributions and underscores the
promising application of model internals for RAG answer attribution.
The Devil is in the Details: StyleFeatureEditor for Detail-Rich StyleGAN
  Inversion and High Quality Image Editing
Authors: name: Denis Bobkov, name: Vadim Titov, name: Aibek Alanov, name: Dmitry Vetrov
Published: 2024-06-15 11:28:32

The task of manipulating real image attributes through StyleGAN inversion has
been extensively researched. This process involves searching latent variables
from a well-trained StyleGAN generator that can synthesize a real image,
modifying these latent variables, and then synthesizing an image with the
desired edits. A balance must be struck between the quality of the
reconstruction and the ability to edit. Earlier studies utilized the
low-dimensional W-space for latent search, which facilitated effective editing
but struggled with reconstructing intricate details. More recent research has
turned to the high-dimensional feature space F, which successfully inverses the
input image but loses much of the detail during editing. In this paper, we
introduce StyleFeatureEditor -- a novel method that enables editing in both
w-latents and F-latents. This technique not only allows for the reconstruction
of finer image details but also ensures their preservation during editing. We
also present a new training pipeline specifically designed to train our model
to accurately edit F-latents. Our method is compared with state-of-the-art
encoding approaches, demonstrating that our model excels in terms of
reconstruction quality and is capable of editing even challenging out-of-domain
examples. Code is available at
https://github.com/AIRI-Institute/StyleFeatureEditor.
Improving Visual Commonsense in Language Models via Multiple Image
  Generation
Authors: name: Guy Yariv, name: Idan Schwartz, name: Yossi Adi, name: Sagie Benaim
Published: 2024-06-19 15:17:10

Commonsense reasoning is fundamentally based on multimodal knowledge.
However, existing large language models (LLMs) are primarily trained using
textual data only, limiting their ability to incorporate essential visual
information. In contrast, Visual Language Models, which excel at
visually-oriented tasks, often fail at non-visual tasks such as basic
commonsense reasoning. This divergence highlights a critical challenge - the
integration of robust visual understanding with foundational text-based
language reasoning. To this end, we introduce a method aimed at enhancing LLMs'
visual commonsense. Specifically, our method generates multiple images based on
the input text prompt and integrates these into the model's decision-making
process by mixing their prediction probabilities. To facilitate multimodal
grounded language modeling, we employ a late-fusion layer that combines the
projected visual features with the output of a pre-trained LLM conditioned on
text only. This late-fusion layer enables predictions based on comprehensive
image-text knowledge as well as text only when this is required. We evaluate
our approach using several visual commonsense reasoning tasks together with
traditional NLP tasks, including common sense reasoning and reading
comprehension. Our experimental results demonstrate significant superiority
over existing baselines. When applied to recent state-of-the-art LLMs (e.g.,
Llama3), we observe improvements not only in visual common sense but also in
traditional NLP benchmarks. Code and models are available under
https://github.com/guyyariv/vLMIG.
Self-play with Execution Feedback: Improving Instruction-following
  Capabilities of Large Language Models
Authors: name: Guanting Dong, name: Keming Lu, name: Chengpeng Li, name: Tingyu Xia, name: Bowen Yu, name: Chang Zhou, name: Jingren Zhou
Published: 2024-06-19 13:29:53

One core capability of large language models (LLMs) is to follow natural
language instructions. However, the issue of automatically constructing
high-quality training data to enhance the complex instruction-following
abilities of LLMs without manual annotation remains unresolved. In this paper,
we introduce AutoIF, the first scalable and reliable method for automatically
generating instruction-following training data. AutoIF transforms the
validation of instruction-following data quality into code verification,
requiring LLMs to generate instructions, the corresponding code to check the
correctness of the instruction responses, and unit test samples to verify the
code's correctness. Then, execution feedback-based rejection sampling can
generate data for Supervised Fine-Tuning (SFT) and Reinforcement Learning from
Human Feedback (RLHF) training. AutoIF achieves significant improvements across
three training algorithms, SFT, Offline DPO, and Online DPO, when applied to
the top open-source LLMs, Qwen2 and LLaMA3, in self-alignment and
strong-to-weak distillation settings. Our code is publicly available at
https://github.com/QwenLM/AutoIF.
LiveMind: Low-latency Large Language Models with Simultaneous Inference
Authors: name: Chuangtao Chen, name: Grace Li Zhang, name: Xunzhao Yin, name: Cheng Zhuo, name: Ulf Schlichtmann, name: Bing Li
Published: 2024-06-20 13:52:30

In this paper, we introduce a novel low-latency inference framework for large
language models (LLMs) inference which enables LLMs to perform inferences with
incomplete prompts. By reallocating computational processes to prompt input
phase, we achieve a substantial reduction in latency, thereby significantly
enhancing the interactive experience for users of LLMs. The framework adeptly
manages the visibility of the streaming prompt to the model, allowing it to
infer from incomplete prompts or await additional prompts. Compared with
traditional inference methods that utilize complete prompts, our approach
demonstrates an average reduction of 59% in response latency on the MMLU-Pro
dataset, while maintaining comparable accuracy. Additionally, our framework
facilitates collaborative inference and output across different models. By
employing an LLM for inference and a small language model (SLM) for output, we
achieve an average 68% reduction in response latency, alongside a 5.5%
improvement in accuracy on the MMLU-Pro dataset compared with the SLM baseline.
For long prompts exceeding 20 sentences, the response latency can be reduced by
up to 93%.
Invertible Consistency Distillation for Text-Guided Image Editing in
  Around 7 Steps
Authors: name: Nikita Starodubcev, name: Mikhail Khoroshikh, name: Artem Babenko, name: Dmitry Baranchuk
Published: 2024-06-20 17:49:11

Diffusion distillation represents a highly promising direction for achieving
faithful text-to-image generation in a few sampling steps. However, despite
recent successes, existing distilled models still do not provide the full
spectrum of diffusion abilities, such as real image inversion, which enables
many precise image manipulation methods. This work aims to enrich distilled
text-to-image diffusion models with the ability to effectively encode real
images into their latent space. To this end, we introduce invertible
Consistency Distillation (iCD), a generalized consistency distillation
framework that facilitates both high-quality image synthesis and accurate image
encoding in only 3-4 inference steps. Though the inversion problem for
text-to-image diffusion models gets exacerbated by high classifier-free
guidance scales, we notice that dynamic guidance significantly reduces
reconstruction errors without noticeable degradation in generation performance.
As a result, we demonstrate that iCD equipped with dynamic guidance may serve
as a highly effective tool for zero-shot text-guided image editing, competing
with more expensive state-of-the-art alternatives.
HARE: HumAn pRiors, a key to small language model Efficiency
Authors: name: Lingyun Zhang, name: Bin jin, name: Gaojian Ge, name: Lunhui Liu, name: Xuewen Shen, name: Mingyong Wu, name: Houqian Zhang, name: Yongneng Jiang, name: Shiqi Chen, name: Shi Pu
Published: 2024-06-17 10:56:03

Human priors play a crucial role in efficiently utilizing data in deep
learning. However, with the development of large language models (LLMs), there
is an increasing emphasis on scaling both model size and data volume, which
often diminishes the importance of human priors in data construction.
Influenced by these trends, existing Small Language Models (SLMs) mainly rely
on web-scraped large-scale training data, neglecting the proper incorporation
of human priors. This oversight limits the training efficiency of language
models in resource-constrained settings. In this paper, we propose a principle
to leverage human priors for data construction. This principle emphasizes
achieving high-performance SLMs by training on a concise dataset that
accommodates both semantic diversity and data quality consistency, while
avoiding benchmark data leakage. Following this principle, we train an SLM
named HARE-1.1B. Extensive experiments on large-scale benchmark datasets
demonstrate that HARE-1.1B performs favorably against state-of-the-art SLMs,
validating the effectiveness of the proposed principle. Additionally, this
provides new insights into efficient language model training in
resource-constrained environments from the view of human priors.
nabla^2DFT: A Universal Quantum Chemistry Dataset of Drug-Like
  Molecules and a Benchmark for Neural Network Potentials
Authors: name: Kuzma Khrabrov, name: Anton Ber, name: Artem Tsypin, name: Konstantin Ushenin, name: Egor Rumiantsev, name: Alexander Telepov, name: Dmitry Protasov, name: Ilya Shenbin, name: Anton Alekseev, name: Mikhail Shirokikh, name: Sergey Nikolenko, name: Elena Tutubalina, name: Artur Kadurin
Published: 2024-06-20 14:14:59

Methods of computational quantum chemistry provide accurate approximations of
molecular properties crucial for computer-aided drug discovery and other areas
of chemical science. However, high computational complexity limits the
scalability of their applications. Neural network potentials (NNPs) are a
promising alternative to quantum chemistry methods, but they require large and
diverse datasets for training. This work presents a new dataset and benchmark
called nabla^2DFT that is based on the nablaDFT. It contains twice as much
molecular structures, three times more conformations, new data types and tasks,
and state-of-the-art models. The dataset includes energies, forces, 17
molecular properties, Hamiltonian and overlap matrices, and a wavefunction
object. All calculations were performed at the DFT level
(omegaB97X-D/def2-SVP) for each conformation. Moreover, nabla^2DFT is the
first dataset that contains relaxation trajectories for a substantial number of
drug-like molecules. We also introduce a novel benchmark for evaluating NNPs in
molecular property prediction, Hamiltonian prediction, and conformational
optimization tasks. Finally, we propose an extendable framework for training
NNPs and implement 10 models within it.