LLaVA-Interactive: An All-in-One Demo for Image Chat, Segmentation,
  Generation and Editing
Authors: name: Wei-Ge Chen; name: Irina Spiridonova; name: Jianwei Yang; name: Jianfeng Gao; name: Chunyuan Li
Published: 2023-11-01 15:13:43

Abstract: LLaVA-Interactive is a research prototype for multimodal human-AI
interaction. The system can have multi-turn dialogues with human users by
taking multimodal user inputs and generating multimodal responses. Importantly,
LLaVA-Interactive goes beyond language prompt, where visual prompt is enabled
to align human intents in the interaction. The development of LLaVA-Interactive
is extremely cost-efficient as the system combines three multimodal skills of
pre-built AI models without additional model training: visual chat of LLaVA,
image segmentation from SEEM, as well as image generation and editing from
GLIGEN. A diverse set of application scenarios is presented to demonstrate the
promises of LLaVA-Interactive and to inspire future research in multimodal
interactive systems.