Orionnnn/hub-stats huggingface dataset:
SocialGPT: Prompting LLMs for Social Relation Reasoning via Greedy
  Segment Optimization
Authors: name: Wanhua Li, name: Zibin Meng, name: Jiawei Zhou, name: Donglai Wei, name: Chuang Gan, name: Hanspeter Pfister
Published: 2024-10-28 18:10:26

Social relation reasoning aims to identify relation categories such as
friends, spouses, and colleagues from images. While current methods adopt the
paradigm of training a dedicated network end-to-end using labeled image data,
they are limited in terms of generalizability and interpretability. To address
these issues, we first present a simple yet well-crafted framework named
{\name}, which combines the perception capability of Vision Foundation Models
(VFMs) and the reasoning capability of Large Language Models (LLMs) within a
modular framework, providing a strong baseline for social relation recognition.
Specifically, we instruct VFMs to translate image content into a textual social
story, and then utilize LLMs for text-based reasoning. {\name} introduces
systematic design principles to adapt VFMs and LLMs separately and bridge their
gaps. Without additional model training, it achieves competitive zero-shot
results on two databases while offering interpretable answers, as LLMs can
generate language-based explanations for the decisions. The manual prompt
design process for LLMs at the reasoning phase is tedious and an automated
prompt optimization method is desired. As we essentially convert a visual
classification task into a generative task of LLMs, automatic prompt
optimization encounters a unique long prompt optimization issue. To address
this issue, we further propose the Greedy Segment Prompt Optimization (GSPO),
which performs a greedy search by utilizing gradient information at the segment
level. Experimental results show that GSPO significantly improves performance,
and our method also generalizes to different image styles. The code is
available at https://github.com/Mengzibin/SocialGPT.
Robots Pre-train Robots: Manipulation-Centric Robotic Representation
  from Large-Scale Robot Dataset
Authors: name: Guangqi Jiang, name: Yifei Sun, name: Tao Huang, name: Huanyu Li, name: Yongyuan Liang, name: Huazhe Xu
Published: 2024-10-29 17:58:13

The pre-training of visual representations has enhanced the efficiency of
robot learning. Due to the lack of large-scale in-domain robotic datasets,
prior works utilize in-the-wild human videos to pre-train robotic visual
representation. Despite their promising results, representations from human
videos are inevitably subject to distribution shifts and lack the dynamics
information crucial for task completion. We first evaluate various pre-trained
representations in terms of their correlation to the downstream robotic
manipulation tasks (i.e., manipulation centricity). Interestingly, we find that
the "manipulation centricity" is a strong indicator of success rates when
applied to downstream tasks. Drawing from these findings, we propose
Manipulation Centric Representation (MCR), a foundation representation learning
framework capturing both visual features and the dynamics information such as
actions and proprioceptions of manipulation tasks to improve manipulation
centricity. Specifically, we pre-train a visual encoder on the DROID robotic
dataset and leverage motion-relevant data such as robot proprioceptive states
and actions. We introduce a novel contrastive loss that aligns visual
observations with the robot's proprioceptive state-action dynamics, combined
with a behavior cloning (BC)-like actor loss to predict actions during
pre-training, along with a time contrastive loss. Empirical results across 4
simulation domains with 20 tasks verify that MCR outperforms the strongest
baseline method by 14.8%. Moreover, MCR boosts the performance of
data-efficient learning with a UR5e arm on 3 real-world tasks by 76.9%. Project
website: https://robots-pretrain-robots.github.io/.
Flow-DPO: Improving LLM Mathematical Reasoning through Online
  Multi-Agent Learning
Authors: name: Yihe Deng, name: Paul Mineiro
Published: 2024-10-29 17:50:31

Mathematical reasoning is a crucial capability for Large Language Models
(LLMs), yet generating detailed and accurate reasoning traces remains a
significant challenge. This paper introduces a novel approach to produce
high-quality reasoning traces for LLM fine-tuning using online learning
Flows. Our method employs an incremental output production Flow, where
component LLMs collaboratively construct solutions through iterative
communication. We train the Flow using online Direct Preference Optimization
(DPO) learning with rollouts, generating DPO pairs for each training example
and updating models in real-time. We directly compare the quality of reasoning
traces generated by our method with those produced through direct model
inference, demonstrating the effectiveness of our approach in improving LLM
performance in mathematical reasoning tasks.
ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM
  Inference
Authors: name: Hanshi Sun, name: Li-Wen Chang, name: Wenlei Bao, name: Size Zheng, name: Ningxin Zheng, name: Xin Liu, name: Harry Dong, name: Yuejie Chi, name: Beidi Chen
Published: 2024-10-28 19:08:12

With the widespread deployment of long-context large language models (LLMs),
there has been a growing demand for efficient support of high-throughput
inference. However, as the key-value (KV) cache expands with the sequence
length, the increasing memory footprint and the need to access it for each
token generation both result in low throughput when serving long-context LLMs.
While various dynamic sparse attention methods have been proposed to speed up
inference while maintaining generation quality, they either fail to
sufficiently reduce GPU memory consumption or introduce significant decoding
latency by offloading the KV cache to the CPU. We present ShadowKV, a
high-throughput long-context LLM inference system that stores the low-rank key
cache and offloads the value cache to reduce the memory footprint for larger
batch sizes and longer sequences. To minimize decoding latency, ShadowKV
employs an accurate KV selection strategy that reconstructs minimal sparse KV
pairs on-the-fly. By evaluating ShadowKV on a broad range of benchmarks,
including RULER, LongBench, and Needle In A Haystack, and models like
Llama-3.1-8B, Llama-3-8B-1M, GLM-4-9B-1M, Yi-9B-200K, Phi-3-Mini-128K, and
Qwen2-7B-128K, we demonstrate that it can support up to 6times larger batch
sizes and boost throughput by up to 3.04times on an A100 GPU without
sacrificing accuracy, even surpassing the performance achievable with infinite
batch size under the assumption of infinite GPU memory. The code is available
at https://github.com/bytedance/ShadowKV.
Precise and Dexterous Robotic Manipulation via Human-in-the-Loop
  Reinforcement Learning
Authors: name: Jianlan Luo, name: Charles Xu, name: Jeffrey Wu, name: Sergey Levine
Published: 2024-10-29 08:12:20

Reinforcement learning (RL) holds great promise for enabling autonomous
acquisition of complex robotic manipulation skills, but realizing this
potential in real-world settings has been challenging. We present a
human-in-the-loop vision-based RL system that demonstrates impressive
performance on a diverse set of dexterous manipulation tasks, including dynamic
manipulation, precision assembly, and dual-arm coordination. Our approach
integrates demonstrations and human corrections, efficient RL algorithms, and
other system-level design choices to learn policies that achieve near-perfect
success rates and fast cycle times within just 1 to 2.5 hours of training. We
show that our method significantly outperforms imitation learning baselines and
prior RL approaches, with an average 2x improvement in success rate and 1.8x
faster execution. Through extensive experiments and analysis, we provide
insights into the effectiveness of our approach, demonstrating how it learns
robust, adaptive policies for both reactive and predictive control strategies.
Our results suggest that RL can indeed learn a wide range of complex
vision-based manipulation policies directly in the real world within practical
training times. We hope this work will inspire a new generation of learned
robotic manipulation techniques, benefiting both industrial applications and
research advancements. Videos and code are available at our project website
https://hil-serl.github.io/.
EoRA: Training-free Compensation for Compressed LLM with Eigenspace
  Low-Rank Approximation
Authors: name: Shih-Yang Liu, name: Huck Yang, name: Chein-Yi Wang, name: Nai Chit Fung, name: Hongxu Yin, name: Charbel Sakr, name: Saurav Muralidharan, name: Kwang-Ting Cheng, name: Jan Kautz, name: Yu-Chiang Frank Wang, name: Pavlo Molchanov, name: Min-Hung Chen
Published: 2024-10-28 17:59:03

In this work, we re-formulate the model compression problem into the
customized compensation problem: Given a compressed model, we aim to introduce
residual low-rank paths to compensate for compression errors under customized
requirements from users (e.g., tasks, compression ratios), resulting in greater
flexibility in adjusting overall capacity without being constrained by specific
compression formats. However, naively applying SVD to derive residual paths
causes suboptimal utilization of the low-rank representation capacity. Instead,
we propose Training-free Eigenspace Low-Rank Approximation (EoRA), a method
that directly minimizes compression-induced errors without requiring
gradient-based training, achieving fast optimization in minutes using a small
amount of calibration data. EoRA projects compression errors into the
eigenspace of input activations, leveraging eigenvalues to effectively
prioritize the reconstruction of high-importance error components. Moreover,
EoRA can be seamlessly integrated with fine-tuning and quantization to further
improve effectiveness and efficiency. EoRA consistently outperforms previous
methods in compensating errors for compressed LLaMA2/3 models on various tasks,
such as language generation, commonsense reasoning, and math reasoning tasks
(e.g., 31.31%/12.88% and 9.69% improvements on ARC-Easy/ARC-Challenge and
MathQA when compensating LLaMA3-8B that is quantized to 4-bit and pruned to 2:4
sparsity). EoRA offers a scalable, training-free solution to compensate for
compression errors, making it a powerful tool to deploy LLMs in various
capacity and efficiency requirements.
Bi-Level Motion Imitation for Humanoid Robots
Authors: name: Wenshuai Zhao, name: Yi Zhao, name: Joni Pajarinen, name: Michael Muehlebach
Published: 2024-10-02 19:21:15

Imitation learning from human motion capture (MoCap) data provides a
promising way to train humanoid robots. However, due to differences in
morphology, such as varying degrees of joint freedom and force limits, exact
replication of human behaviors may not be feasible for humanoid robots.
Consequently, incorporating physically infeasible MoCap data in training
datasets can adversely affect the performance of the robot policy. To address
this issue, we propose a bi-level optimization-based imitation learning
framework that alternates between optimizing both the robot policy and the
target MoCap data. Specifically, we first develop a generative latent dynamics
model using a novel self-consistent auto-encoder, which learns sparse and
structured motion representations while capturing desired motion patterns in
the dataset. The dynamics model is then utilized to generate reference motions
while the latent representation regularizes the bi-level motion imitation
process. Simulations conducted with a realistic model of a humanoid robot
demonstrate that our method enhances the robot policy by modifying reference
motions to be physically consistent.
VideoWebArena: Evaluating Long Context Multimodal Agents with Video
  Understanding Web Tasks
Authors: name: Lawrence Jang, name: Yinheng Li, name: Charles Ding, name: Justin Lin, name: Paul Pu Liang, name: Dan Zhao, name: Rogerio Bonatti, name: Kazuhito Koishida
Published: 2024-10-24 19:03:01

Videos are often used to learn or extract the necessary information to
complete tasks in ways different than what text and static imagery alone can
provide. However, many existing agent benchmarks neglect long-context video
understanding, instead focusing on text or static image inputs. To bridge this
gap, we introduce VideoWebArena (VideoWA), a benchmark for evaluating the
capabilities of long-context multimodal agents for video understanding. VideoWA
consists of 2,021 web agent tasks based on manually crafted video tutorials,
which total almost four hours of content. For our benchmark, we define a
taxonomy of long-context video-based agent tasks with two main areas of focus:
skill retention and factual retention. While skill retention tasks evaluate
whether an agent can use a given human demonstration to complete a task
efficiently, the factual retention task evaluates whether an agent can retrieve
instruction-relevant information from a video to complete a task. We find that
the best model achieves 13.3% success on factual retention tasks and 45.8% on
factual retention QA pairs, far below human performance at 73.9% and 79.3%,
respectively. On skill retention tasks, long-context models perform worse with
tutorials than without, exhibiting a 5% performance decrease in WebArena tasks
and a 10.3% decrease in VisualWebArena tasks. Our work highlights the need to
improve the agentic abilities of long-context multimodal models and provides a
testbed for future development with long-context video agents.
Relaxed Recursive Transformers: Effective Parameter Sharing with
  Layer-wise LoRA
Authors: name: Sangmin Bae, name: Adam Fisch, name: Hrayr Harutyunyan, name: Ziwei Ji, name: Seungyeon Kim, name: Tal Schuster
Published: 2024-10-28 02:15:45

Large language models (LLMs) are expensive to deploy. Parameter sharing
offers a possible path towards reducing their size and cost, but its
effectiveness in modern LLMs remains fairly limited. In this work, we revisit
"layer tying" as form of parameter sharing in Transformers, and introduce novel
methods for converting existing LLMs into smaller "Recursive Transformers" that
share parameters across layers, with minimal loss of performance. Here, our
Recursive Transformers are efficiently initialized from standard pretrained
Transformers, but only use a single block of unique layers that is then
repeated multiple times in a loop. We further improve performance by
introducing Relaxed Recursive Transformers that add flexibility to the layer
tying constraint via depth-wise low-rank adaptation (LoRA) modules, yet still
preserve the compactness of the overall model. We show that our recursive
models (e.g., recursive Gemma 1B) outperform both similar-sized vanilla
pretrained models (such as TinyLlama 1.1B and Pythia 1B) and knowledge
distillation baselines -- and can even recover most of the performance of the
original "full-size" model (e.g., Gemma 2B with no shared parameters). Finally,
we propose Continuous Depth-wise Batching, a promising new inference paradigm
enabled by the Recursive Transformer when paired with early exiting. In a
theoretical analysis, we show that this has the potential to lead to
significant (2-3x) gains in inference throughput.
Dialog2Flow: Pre-training Soft-Contrastive Action-Driven Sentence
  Embeddings for Automatic Dialog Flow Extraction
Authors: name: Sergio Burdisso, name: Srikanth Madikeri, name: Petr Motlicek
Published: 2024-10-24 07:10:18

Efficiently deriving structured workflows from unannotated dialogs remains an
underexplored and formidable challenge in computational linguistics. Automating
this process could significantly accelerate the manual design of workflows in
new domains and enable the grounding of large language models in
domain-specific flowcharts, enhancing transparency and controllability. In this
paper, we introduce Dialog2Flow (D2F) embeddings, which differ from
conventional sentence embeddings by mapping utterances to a latent space where
they are grouped according to their communicative and informative functions
(i.e., the actions they represent). D2F allows for modeling dialogs as
continuous trajectories in a latent space with distinct action-related regions.
By clustering D2F embeddings, the latent space is quantized, and dialogs can be
converted into sequences of region/action IDs, facilitating the extraction of
the underlying workflow. To pre-train D2F, we build a comprehensive dataset by
unifying twenty task-oriented dialog datasets with normalized per-turn action
annotations. We also introduce a novel soft contrastive loss that leverages the
semantic information of these actions to guide the representation learning
process, showing superior performance compared to standard supervised
contrastive loss. Evaluation against various sentence embeddings, including
dialog-specific ones, demonstrates that D2F yields superior qualitative and
quantitative results across diverse domains.
Leveraging Locality to Boost Sample Efficiency in Robotic Manipulation
Authors: name: Tong Zhang, name: Yingdong Hu, name: Jiacheng You, name: Yang Gao
Published: 2024-06-15 12:27:35

Given the high cost of collecting robotic data in the real world, sample
efficiency is a consistently compelling pursuit in robotics. In this paper, we
introduce SGRv2, an imitation learning framework that enhances sample
efficiency through improved visual and action representations. Central to the
design of SGRv2 is the incorporation of a critical inductive bias-action
locality, which posits that robot's actions are predominantly influenced by the
target object and its interactions with the local environment. Extensive
experiments in both simulated and real-world settings demonstrate that action
locality is essential for boosting sample efficiency. SGRv2 excels in RLBench
tasks with keyframe control using merely 5 demonstrations and surpasses the RVT
baseline in 23 of 26 tasks. Furthermore, when evaluated on ManiSkill2 and
MimicGen using dense control, SGRv2's success rate is 2.54 times that of SGR.
In real-world environments, with only eight demonstrations, SGRv2 can perform a
variety of tasks at a markedly higher success rate compared to baseline models.
Project website: http://sgrv2-robot.github.io
Document Parsing Unveiled: Techniques, Challenges, and Prospects for
  Structured Information Extraction
Authors: name: Qintong Zhang, name: Victor Shea-Jay Huang, name: Bin Wang, name: Junyuan Zhang, name: Zhengren Wang, name: Hao Liang, name: Shawn Wang, name: Matthieu Lin, name: Wentao Zhang, name: Conghui He
Published: 2024-10-28 16:11:35

Document parsing is essential for converting unstructured and semi-structured
documents-such as contracts, academic papers, and invoices-into structured,
machine-readable data. Document parsing extract reliable structured data from
unstructured inputs, providing huge convenience for numerous applications.
Especially with recent achievements in Large Language Models, document parsing
plays an indispensable role in both knowledge base construction and training
data generation. This survey presents a comprehensive review of the current
state of document parsing, covering key methodologies, from modular pipeline
systems to end-to-end models driven by large vision-language models. Core
components such as layout detection, content extraction (including text,
tables, and mathematical expressions), and multi-modal data integration are
examined in detail. Additionally, this paper discusses the challenges faced by
modular document parsing systems and vision-language models in handling complex
layouts, integrating multiple modules, and recognizing high-density text. It
emphasizes the importance of developing larger and more diverse datasets and
outlines future research directions.
Bielik 7B v0.1: A Polish Language Model -- Development, Insights, and
  Evaluation
Authors: name: Krzysztof Ociepa, name: Łukasz Flis, name: Krzysztof Wróbel, name: Adrian Gwoździej, name: Remigiusz Kinas
Published: 2024-10-24 09:16:09

We introduce Bielik 7B v0.1, a 7-billion-parameter generative text model for
Polish language processing. Trained on curated Polish corpora, this model
addresses key challenges in language model development through innovative
techniques. These include Weighted Instruction Cross-Entropy Loss, which
balances the learning of different instruction types, and Adaptive Learning
Rate, which dynamically adjusts the learning rate based on training progress.
To evaluate performance, we created the Open PL LLM Leaderboard and Polish
MT-Bench, novel frameworks assessing various NLP tasks and conversational
abilities. Bielik 7B v0.1 demonstrates significant improvements, achieving a 9
percentage point increase in average score compared to Mistral-7B-v0.1 on the
RAG Reader task. It also excels in the Polish MT-Bench, particularly in
Reasoning (6.15/10) and Role-playing (7.83/10) categories. This model
represents a substantial advancement in Polish language AI, offering a powerful
tool for diverse linguistic applications and setting new benchmarks in the
field.
AgentStore: Scalable Integration of Heterogeneous Agents As Specialized
  Generalist Computer Assistant
Authors: name: Chengyou Jia, name: Minnan Luo, name: Zhuohang Dang, name: Qiushi Sun, name: Fangzhi Xu, name: Junlin Hu, name: Tianbao Xie, name: Zhiyong Wu
Published: 2024-10-24 09:58:40

Digital agents capable of automating complex computer tasks have attracted
considerable attention due to their immense potential to enhance human-computer
interaction. However, existing agent methods exhibit deficiencies in their
generalization and specialization capabilities, especially in handling
open-ended computer tasks in real-world environments. Inspired by the rich
functionality of the App store, we present AgentStore, a scalable platform
designed to dynamically integrate heterogeneous agents for automating computer
tasks. AgentStore empowers users to integrate third-party agents, allowing the
system to continuously enrich its capabilities and adapt to rapidly evolving
operating systems. Additionally, we propose a novel core MetaAgent
with the AgentToken strategy to efficiently manage diverse agents and
utilize their specialized and generalist abilities for both domain-specific and
system-wide tasks. Extensive experiments on three challenging benchmarks
demonstrate that AgentStore surpasses the limitations of previous systems with
narrow capabilities, particularly achieving a significant improvement from
11.21\% to 23.85\% on the OSWorld benchmark, more than doubling the previous
results. Comprehensive quantitative and qualitative results further demonstrate
AgentStore's ability to enhance agent systems in both generalization and
specialization, underscoring its potential for developing the specialized
generalist computer assistant. All our codes will be made publicly available in
https://chengyou-jia.github.io/AgentStore-Home.
Fast Best-of-N Decoding via Speculative Rejection
Authors: name: Hanshi Sun, name: Momin Haider, name: Ruiqi Zhang, name: Huitao Yang, name: Jiahao Qiu, name: Ming Yin, name: Mengdi Wang, name: Peter Bartlett, name: Andrea Zanette
Published: 2024-10-26 23:20:48

The safe and effective deployment of Large Language Models (LLMs) involves a
critical step called alignment, which ensures that the model's responses are in
accordance with human preferences. Prevalent alignment techniques, such as DPO,
PPO and their variants, align LLMs by changing the pre-trained model weights
during a phase called post-training. While predominant, these post-training
methods add substantial complexity before LLMs can be deployed. Inference-time
alignment methods avoid the complex post-training step and instead bias the
generation towards responses that are aligned with human preferences. The
best-known inference-time alignment method, called Best-of-N, is as effective
as the state-of-the-art post-training procedures. Unfortunately, Best-of-N
requires vastly more resources at inference time than standard decoding
strategies, which makes it computationally not viable. In this work, we
introduce Speculative Rejection, a computationally-viable inference-time
alignment algorithm. It generates high-scoring responses according to a given
reward model, like Best-of-N does, while being between 16 to 32 times more
computationally efficient.