Orionnnn/hub-stats huggingface dataset:
ProgressGym: Alignment with a Millennium of Moral Progress
Authors: name: Tianyi Qiu, name: Yang Zhang, name: Xuchuan Huang, name: Jasmine Xinze Li, name: Jiaming Ji, name: Yaodong Yang
Published: 2024-06-28 17:55:24

Frontier AI systems, including large language models (LLMs), hold increasing
influence over the epistemology of human users. Such influence can reinforce
prevailing societal values, potentially contributing to the lock-in of
misguided moral beliefs and, consequently, the perpetuation of problematic
moral practices on a broad scale. We introduce progress alignment as a
technical solution to mitigate this imminent risk. Progress alignment
algorithms learn to emulate the mechanics of human moral progress, thereby
addressing the susceptibility of existing alignment methods to contemporary
moral blindspots. To empower research in progress alignment, we introduce
ProgressGym, an experimental framework allowing the learning of moral progress
mechanics from history, in order to facilitate future progress in real-world
moral decisions. Leveraging 9 centuries of historical text and 18 historical
LLMs, ProgressGym enables codification of real-world progress alignment
challenges into concrete benchmarks. Specifically, we introduce three core
challenges: tracking evolving values (PG-Follow), preemptively anticipating
moral progress (PG-Predict), and regulating the feedback loop between human and
AI value shifts (PG-Coevolve). Alignment methods without a temporal dimension
are inapplicable to these tasks. In response, we present lifelong and
extrapolative algorithms as baseline methods of progress alignment, and build
an open leaderboard soliciting novel algorithms and challenges. The framework
and the leaderboard are available at
https://github.com/PKU-Alignment/ProgressGym and
https://huggingface.co/spaces/PKU-Alignment/ProgressGym-LeaderBoard
respectively.
RealTalk: Real-time and Realistic Audio-driven Face Generation with 3D
  Facial Prior-guided Identity Alignment Network
Authors: name: Xiaozhong Ji, name: Chuming Lin, name: Zhonggan Ding, name: Ying Tai, name: Jian Yang, name: Junwei Zhu, name: Xiaobin Hu, name: Jiangning Zhang, name: Donghao Luo, name: Chengjie Wang
Published: 2024-06-26 12:09:59

Person-generic audio-driven face generation is a challenging task in computer
vision. Previous methods have achieved remarkable progress in audio-visual
synchronization, but there is still a significant gap between current results
and practical applications. The challenges are two-fold: 1) Preserving unique
individual traits for achieving high-precision lip synchronization. 2)
Generating high-quality facial renderings in real-time performance. In this
paper, we propose a novel generalized audio-driven framework RealTalk, which
consists of an audio-to-expression transformer and a high-fidelity
expression-to-face renderer. In the first component, we consider both identity
and intra-personal variation features related to speaking lip movements. By
incorporating cross-modal attention on the enriched facial priors, we can
effectively align lip movements with audio, thus attaining greater precision in
expression prediction. In the second component, we design a lightweight facial
identity alignment (FIA) module which includes a lip-shape control structure
and a face texture reference structure. This novel design allows us to generate
fine details in real-time, without depending on sophisticated and inefficient
feature alignment modules. Our experimental results, both quantitative and
qualitative, on public datasets demonstrate the clear advantages of our method
in terms of lip-speech synchronization and generation quality. Furthermore, our
method is efficient and requires fewer computational resources, making it
well-suited to meet the needs of practical applications.
Token Erasure as a Footprint of Implicit Vocabulary Items in LLMs
Authors: name: Sheridan Feucht, name: David Atkinson, name: Byron Wallace, name: David Bau
Published: 2024-06-28 17:54:47

LLMs process text as sequences of tokens that roughly correspond to words,
where less common words are represented by multiple tokens. However, individual
tokens are often semantically unrelated to the meanings of the words/concepts
they comprise. For example, Llama-2-7b's tokenizer splits the word
"northeastern" into the tokens ['_n', 'ort', 'he', 'astern'], none of which
correspond to semantically meaningful units like "north" or "east." Similarly,
the overall meanings of named entities like "Neil Young" and multi-word
expressions like "break a leg" cannot be directly inferred from their
constituent tokens. Mechanistically, how do LLMs convert such arbitrary groups
of tokens into useful higher-level representations? In this work, we find that
last token representations of named entities and multi-token words exhibit a
pronounced "erasure" effect, where information about previous and current
tokens is rapidly forgotten in early layers. Using this observation, we propose
a method to "read out" the implicit vocabulary of an autoregressive LLM by
examining differences in token representations across layers, and present
results of this method for Llama-2-7b and Llama-3-8B. To our knowledge, this is
the first attempt to probe the implicit vocabulary of an LLM.
Is It Really Long Context if All You Need Is Retrieval? Towards
  Genuinely Difficult Long Context NLP
Authors: name: Omer Goldman, name: Alon Jacovi, name: Aviv Slobodkin, name: Aviya Maimon, name: Ido Dagan, name: Reut Tsarfaty
Published: 2024-06-29 11:09:47

Improvements in language models' capabilities have pushed their applications
towards longer contexts, making long-context evaluation and development an
active research area. However, many disparate use-cases are grouped together
under the umbrella term of "long-context", defined simply by the total length
of the model's input, including - for example - Needle-in-a-Haystack tasks,
book summarization, and information aggregation. Given their varied difficulty,
in this position paper we argue that conflating different tasks by their
context length is unproductive. As a community, we require a more precise
vocabulary to understand what makes long-context tasks similar or different. We
propose to unpack the taxonomy of long-context based on the properties that
make them more difficult with longer contexts. We propose two orthogonal axes
of difficulty: (I) Diffusion: How hard is it to find the necessary information
in the context? (II) Scope: How much necessary information is there to find? We
survey the literature on long-context, provide justification for this taxonomy
as an informative descriptor, and situate the literature with respect to it. We
conclude that the most difficult and interesting settings, whose necessary
information is very long and highly diffused within the input, is severely
under-explored. By using a descriptive vocabulary and discussing the relevant
properties of difficulty in long-context, we can implement more informed
research in this area. We call for a careful design of tasks and benchmarks
with distinctly long context, taking into account the characteristics that make
it qualitatively different from shorter context.
T-MAC: CPU Renaissance via Table Lookup for Low-Bit LLM Deployment on
  Edge
Authors: name: Jianyu Wei, name: Shijie Cao, name: Ting Cao, name: Lingxiao Ma, name: Lei Wang, name: Yanyong Zhang, name: Mao Yang
Published: 2024-06-25 08:38:38

The deployment of Large Language Models (LLMs) on edge devices is
increasingly important to enhance on-device intelligence. Weight quantization
is crucial for reducing the memory footprint of LLMs on devices. However,
low-bit LLMs necessitate mixed precision matrix multiplication (mpGEMM) of low
precision weights and high precision activations during inference. Existing
systems, lacking native support for mpGEMM, resort to dequantize weights for
high precision computation. Such an indirect way can lead to a significant
inference overhead.
  In this paper, we introduce T-MAC, an innovative lookup table(LUT)-based
method designed for efficient low-bit LLM (i.e., weight-quantized LLM)
inference on CPUs. T-MAC directly supports mpGEMM without dequantization, while
simultaneously eliminating multiplications and reducing additions required.
Specifically, T-MAC transforms the traditional data-type-centric multiplication
to bit-wise table lookup, and enables a unified and scalable mpGEMM solution.
  Our LUT-based kernels scale linearly to the weight bit-width. Evaluated on
low-bit Llama and BitNet models, T-MAC demonstrates up to 4x increase in
throughput and 70% reduction in energy consumption compared to llama.cpp. For
BitNet-b1.58-3B, T-MAC delivers a token generation throughput of 30 tokens/s
with a single core and 71 tokens/s with eight cores on M2-Ultra, and 11
tokens/s on lower-end devices like Raspberry Pi 5, which significantly exceeds
the adult average reading speed. T-MAC with LUT-based computing paradigm, paves
the way for the practical deployment of low-bit LLMs on resource-constrained
edge devices without compromising computational efficiency. The system is
open-sourced at https://github.com/microsoft/T-MAC.
ROS-LLM: A ROS framework for embodied AI with task feedback and
  structured reasoning
Authors: name: Christopher E. Mower, name: Yuhui Wan, name: Hongzhan Yu, name: Antoine Grosnit, name: Jonas Gonzalez-Billandon, name: Matthieu Zimmer, name: Jinlong Wang, name: Xinyu Zhang, name: Yao Zhao, name: Anbang Zhai, name: Puze Liu, name: Davide Tateo, name: Cesar Cadena, name: Marco Hutter, name: Jan Peters, name: Guangjian Tian, name: Yuzheng Zhuang, name: Kun Shao, name: Xingyue Quan, name: Jianye Hao, name: Jun Wang, name: Haitham Bou-Ammar
Published: 2024-06-28 08:28:38

We present a framework for intuitive robot programming by non-experts,
leveraging natural language prompts and contextual information from the Robot
Operating System (ROS). Our system integrates large language models (LLMs),
enabling non-experts to articulate task requirements to the system through a
chat interface. Key features of the framework include: integration of ROS with
an AI agent connected to a plethora of open-source and commercial LLMs,
automatic extraction of a behavior from the LLM output and execution of ROS
actions/services, support for three behavior modes (sequence, behavior tree,
state machine), imitation learning for adding new robot actions to the library
of possible actions, and LLM reflection via human and environment feedback.
Extensive experiments validate the framework, showcasing robustness,
scalability, and versatility in diverse scenarios, including long-horizon
tasks, tabletop rearrangements, and remote supervisory control. To facilitate
the adoption of our framework and support the reproduction of our results, we
have made our code open-source. You can access it at:
https://github.com/huawei-noah/HEBO/tree/master/ROSLLM.
Accurate Prediction of Ligand-Protein Interaction Affinities with
  Fine-Tuned Small Language Models
Authors: name: Ben Fauber
Published: 2024-06-27 13:04:58

We describe the accurate prediction of ligand-protein interaction (LPI)
affinities, also known as drug-target interactions (DTI), with instruction
fine-tuned pretrained generative small language models (SLMs). We achieved
accurate predictions for a range of affinity values associated with
ligand-protein interactions on out-of-sample data in a zero-shot setting. Only
the SMILES string of the ligand and the amino acid sequence of the protein were
used as the model inputs. Our results demonstrate a clear improvement over
machine learning (ML) and free-energy perturbation (FEP+) based methods in
accurately predicting a range of ligand-protein interaction affinities, which
can be leveraged to further accelerate drug discovery campaigns against
challenging therapeutic targets.
Auto Cherry-Picker: Learning from High-quality Generative Data Driven by
  Language
Authors: name: Yicheng Chen, name: Xiangtai Li, name: Yining Li, name: Yanhong Zeng, name: Jianzong Wu, name: Xiangyu Zhao, name: Kai Chen
Published: 2024-06-28 17:53:18

Diffusion-based models have shown great potential in generating high-quality
images with various layouts, which can benefit downstream perception tasks.
However, a fully automatic layout generation driven only by language and a
suitable metric for measuring multiple generated instances has not been well
explored. In this work, we present Auto Cherry-Picker (ACP), a novel framework
that generates high-quality multi-modal training examples to augment perception
and multi-modal training. Starting with a simple list of natural language
concepts, we prompt large language models (LLMs) to generate a detailed
description and design reasonable layouts. Next, we use an off-the-shelf
text-to-image model to generate multiple images. Then, the generated data are
refined using a comprehensively designed metric to ensure quality. In
particular, we present a new metric, Composite Layout and Image Score (CLIS),
to evaluate the generated images fairly. Our synthetic high-quality examples
boost performance in various scenarios by customizing the initial concept list,
especially in addressing challenges associated with long-tailed distribution
and imbalanced datasets. Experiment results on downstream tasks demonstrate
that Auto Cherry-Picker can significantly improve the performance of existing
models. In addition, we have thoroughly investigated the correlation between
CLIS and performance gains in downstream tasks, and we find that a better CLIS
score results in better performance. This finding shows the potential for
evaluation metrics as the role for various visual perception and MLLM tasks.
Code will be available.
Show Less, Instruct More: Enriching Prompts with Definitions and
  Guidelines for Zero-Shot NER
Authors: name: Andrew Zamai, name: Andrea Zugarini, name: Leonardo Rigutini, name: Marco Ernandes, name: Marco Maggini
Published: 2024-07-01 13:25:33

Recently, several specialized instruction-tuned Large Language Models (LLMs)
for Named Entity Recognition (NER) have emerged. Compared to traditional NER
approaches, these models have strong generalization capabilities. Existing LLMs
mainly focus on zero-shot NER in out-of-domain distributions, being fine-tuned
on an extensive number of entity classes that often highly or completely
overlap with test sets. In this work instead, we propose SLIMER, an approach
designed to tackle never-seen-before named entity tags by instructing the model
on fewer examples, and by leveraging a prompt enriched with definition and
guidelines. Experiments demonstrate that definition and guidelines yield better
performance, faster and more robust learning, particularly when labelling
unseen Named Entities. Furthermore, SLIMER performs comparably to
state-of-the-art approaches in out-of-domain zero-shot NER, while being trained
on a reduced tag set.
InstantStyle-Plus: Style Transfer with Content-Preserving in
  Text-to-Image Generation
Authors: name: Haofan Wang, name: Peng Xing, name: Renyuan Huang, name: Hao Ai, name: Qixun Wang, name: Xu Bai
Published: 2024-06-30 18:05:33

Style transfer is an inventive process designed to create an image that
maintains the essence of the original while embracing the visual style of
another. Although diffusion models have demonstrated impressive generative
power in personalized subject-driven or style-driven applications, existing
state-of-the-art methods still encounter difficulties in achieving a seamless
balance between content preservation and style enhancement. For example,
amplifying the style's influence can often undermine the structural integrity
of the content. To address these challenges, we deconstruct the style transfer
task into three core elements: 1) Style, focusing on the image's aesthetic
characteristics; 2) Spatial Structure, concerning the geometric arrangement and
composition of visual elements; and 3) Semantic Content, which captures the
conceptual meaning of the image. Guided by these principles, we introduce
InstantStyle-Plus, an approach that prioritizes the integrity of the original
content while seamlessly integrating the target style. Specifically, our method
accomplishes style injection through an efficient, lightweight process,
utilizing the cutting-edge InstantStyle framework. To reinforce the content
preservation, we initiate the process with an inverted content latent noise and
a versatile plug-and-play tile ControlNet for preserving the original image's
intrinsic layout. We also incorporate a global semantic adapter to enhance the
semantic content's fidelity. To safeguard against the dilution of style
information, a style extractor is employed as discriminator for providing
supplementary style guidance. Codes will be available at
https://github.com/instantX-research/InstantStyle-Plus.
OmniJARVIS: Unified Vision-Language-Action Tokenization Enables
  Open-World Instruction Following Agents
Authors: name: Zihao Wang, name: Shaofei Cai, name: Zhancun Mu, name: Haowei Lin, name: Ceyao Zhang, name: Xuejie Liu, name: Qing Li, name: Anji Liu, name: Xiaojian Ma, name: Yitao Liang
Published: 2024-06-27 13:46:11

We present OmniJARVIS, a novel Vision-Language-Action (VLA) model for
open-world instruction-following agents in open-world Minecraft. Compared to
prior works that either emit textual goals to separate controllers or produce
the control command directly, OmniJARVIS seeks a different path to ensure both
strong reasoning and efficient decision-making capabilities via unified
tokenization of multimodal interaction data. First, we introduce a
self-supervised approach to learn a behavior encoder that produces discretized
tokens for behavior trajectories tau = {o_0, a_0, dots} and an
imitation learning (IL) policy decoder conditioned on these tokens. These
additional behavior tokens will be augmented to the vocabulary of pretrained
Multimodal Language Models (MLMs). With this encoder, we then pack long-term
multimodal interactions involving task instructions, memories, thoughts,
observations, textual responses, behavior trajectories, etc. into unified token
sequences and model them with autoregressive transformers. Thanks to the
semantically meaningful behavior tokens, the resulting VLA model, OmniJARVIS,
can reason (by producing chain-of-thoughts), plan, answer questions, and act
(by producing behavior tokens for the IL policy decoder). OmniJARVIS
demonstrates excellent performances on a comprehensive collection of atomic,
programmatic, and open-ended tasks in open-world Minecraft. Our analysis
further unveils the crucial design principles in interaction data formation,
unified tokenization, and its scaling potentials.
Chain-of-Knowledge: Integrating Knowledge Reasoning into Large Language
  Models by Learning from Knowledge Graphs
Authors: name: Yifei Zhang, name: Xintao Wang, name: Jiaqing Liang, name: Sirui Xia, name: Lida Chen, name: Yanghua Xiao
Published: 2024-06-30 10:49:32

Large Language Models (LLMs) have exhibited impressive proficiency in various
natural language processing (NLP) tasks, which involve increasingly complex
reasoning. Knowledge reasoning, a primary type of reasoning, aims at deriving
new knowledge from existing one.While it has been widely studied in the context
of knowledge graphs (KGs), knowledge reasoning in LLMs remains underexplored.
In this paper, we introduce Chain-of-Knowledge, a comprehensive framework for
knowledge reasoning, including methodologies for both dataset construction and
model learning. For dataset construction, we create KnowReason via rule mining
on KGs. For model learning, we observe rule overfitting induced by naive
training. Hence, we enhance CoK with a trial-and-error mechanism that simulates
the human process of internal knowledge exploration. We conduct extensive
experiments with KnowReason. Our results show the effectiveness of CoK in
refining LLMs in not only knowledge reasoning, but also general reasoning
benchmarkms.
Wavelets Are All You Need for Autoregressive Image Generation
Authors: name: Wael Mattar, name: Idan Levy, name: Nir Sharon, name: Shai Dekel
Published: 2024-06-28 15:32:59

In this paper, we take a new approach to autoregressive image generation that
is based on two main ingredients. The first is wavelet image coding, which
allows to tokenize the visual details of an image from coarse to fine details
by ordering the information starting with the most significant bits of the most
significant wavelet coefficients. The second is a variant of a language
transformer whose architecture is re-designed and optimized for token sequences
in this 'wavelet language'. The transformer learns the significant statistical
correlations within a token sequence, which are the manifestations of
well-known correlations between the wavelet subbands at various resolutions. We
show experimental results with conditioning on the generation process.
UnUnlearning: Unlearning is not sufficient for content regulation in
  advanced generative AI
Authors: name: Ilia Shumailov, name: Jamie Hayes, name: Eleni Triantafillou, name: Guillermo Ortiz-Jimenez, name: Nicolas Papernot, name: Matthew Jagielski, name: Itay Yona, name: Heidi Howard, name: Eugene Bagdasaryan
Published: 2024-06-27 10:24:35

Exact unlearning was first introduced as a privacy mechanism that allowed a
user to retract their data from machine learning models on request. Shortly
after, inexact schemes were proposed to mitigate the impractical costs
associated with exact unlearning. More recently unlearning is often discussed
as an approach for removal of impermissible knowledge i.e. knowledge that the
model should not possess such as unlicensed copyrighted, inaccurate, or
malicious information. The promise is that if the model does not have a certain
malicious capability, then it cannot be used for the associated malicious
purpose. In this paper we revisit the paradigm in which unlearning is used for
in Large Language Models (LLMs) and highlight an underlying inconsistency
arising from in-context learning. Unlearning can be an effective control
mechanism for the training phase, yet it does not prevent the model from
performing an impermissible act during inference. We introduce a concept of
ununlearning, where unlearned knowledge gets reintroduced in-context,
effectively rendering the model capable of behaving as if it knows the
forgotten knowledge. As a result, we argue that content filtering for
impermissible knowledge will be required and even exact unlearning schemes are
not enough for effective content regulation. We discuss feasibility of
ununlearning for modern LLMs and examine broader implications.
DogeRM: Equipping Reward Models with Domain Knowledge through Model
  Merging
Authors: name: Tzu-Han Lin, name: Chen-An Li, name: Hung-yi Lee, name: Yun-Nung Chen
Published: 2024-07-01 17:01:54

Reinforcement learning from human feedback (RLHF) is a popular strategy for
aligning large language models (LLMs) with desired behaviors. Reward modeling
is a crucial step in RLHF. However, collecting paired preference data for
training reward models is often costly and time-consuming, especially for
domain-specific preferences requiring expert annotation. To address this
challenge, we propose the Domain knowledge merged
Reward Model (DogeRM), a novel framework that integrates
domain-specific knowledge into a general reward model by model merging. The
experiments demonstrate that DogeRM enhances performance across different
benchmarks and provide a detailed analysis showcasing the effects of model
merging, showing the great potential of facilitating model alignment.