File size: 2,093 Bytes
958ba9c 9d87f8d 958ba9c 65bc71a 958ba9c 65bc71a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 |
---
inference: false
pipeline_tag: image-text-to-text
license: apache-2.0
datasets:
- VIMA/VIMA-Data
tags:
- llara
- llava
- robotics
- vlm
---
<br>
<be>
# LLaRA Model Card
This model is released with paper **[LLaRA: Supercharging Robot Learning Data for Vision-Language Policy](https://arxiv.org/abs/2406.20095)**
[Xiang Li](https://xxli.me)<sup>1</sup>, [Cristina Mata](https://openreview.net/profile?id=~Cristina_Mata1)<sup>1</sup>, [Jongwoo Park](https://github.com/jongwoopark7978)<sup>1</sup>, [Kumara Kahatapitiya](https://www3.cs.stonybrook.edu/~kkahatapitiy)<sup>1</sup>, [Yoo Sung Jang](https://yjang43.github.io/)<sup>1</sup>, [Jinghuan Shang](https://elicassion.github.io/)<sup>1</sup>, [Kanchana Ranasinghe](https://kahnchana.github.io/)<sup>1</sup>, [Ryan Burgert](https://ryanndagreat.github.io/)<sup>1</sup>, [Mu Cai](https://pages.cs.wisc.edu/~mucai/)<sup>2</sup>, [Yong Jae Lee](https://pages.cs.wisc.edu/~yongjaelee/)<sup>2</sup>, and [Michael S. Ryoo](http://michaelryoo.com/)<sup>1</sup>
<sup>1</sup>Stony Brook University <sup>2</sup>University of Wisconsin-Madison
## Model details
**Model type:**
LLaRA is an open-source visuomotor policy trained by fine-tuning [LLaVA-7b-v1.5](https://huggingface.co/liuhaotian/llava-v1.5-7b) on instruction-following data `D-inBC`, converted from [VIMA-Data](https://huggingface.co/datasets/VIMA/VIMA-Data).
For the conversion code, please refer to [convert_vima.ipynb](https://github.com/LostXine/LLaRA/blob/main/datasets/convert_vima.ipynb)
**Model date:**
llava-1.5-7b-llara-D-inBC-VIMA-80k was trained in June 2024.
**Paper or resources for more information:**
https://github.com/LostXine/LLaRA
**Where to send questions or comments about the model:**
https://github.com/LostXine/LLaRA/issues
## Intended use
**Primary intended uses:**
The primary use of LLaRA is research on large multimodal models for robotics.
**Primary intended users:**
The primary intended users of the model are researchers and hobbyists in robotics, computer vision, natural language processing, machine learning, and artificial intelligence. |