File size: 2,136 Bytes
64b04d1 cc33b4e 64b04d1 cc33b4e 64b04d1 72be1d8 5332b77 3ed8821 9991254 64b04d1 cc33b4e 64b04d1 cc33b4e 64b04d1 cc33b4e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 |
---
license: apache-2.0
---
# Model Card for SpaceLLaVA
**SpaceLLaVA** uses LoRA to fine-tune [LLaVA](https://github.com/haotian-liu/LLaVA/tree/main) on a dataset designed with [VQASynth](https://github.com/remyxai/VQASynth/tree/main) to enhance spatial reasoning as in [SpatialVLM](https://spatial-vlm.github.io/)
## Model Details
### Model Description
This model uses data synthesis techniques and publically available models to reproduce the work described in SpatialVLM to enhance the spatial reasoning of multimodal models.
With a pipeline of expert models, we can infer spatial relationships between objects in a scene to create VQA dataset for spatial reasoning.
- **Developed by:** remyx.ai
- **Model type:** MultiModal Model, Vision Language Model, LLaVA
- **License:** Apache-2.0
- **Finetuned from model:** LLaVA
### Model Sources
- **Repository:** [VQASynth](https://github.com/remyxai/VQASynth/tree/main)
- **Paper:** [SpatialVLM](https://arxiv.org/abs/2401.12168)
## Uses
Use this model to query spatial relationships between objects in a scene.
[](https://colab.research.google.com/drive/1WPE7Br5A5ERSij8BL1M22EoEMLVkD8EP?usp=sharing)
Try it on Discord: http://discord.gg/b2yGuCNpuC
## Citation
```
@article{chen2024spatialvlm,
title = {SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities},
author = {Chen, Boyuan and Xu, Zhuo and Kirmani, Sean and Ichter, Brian and Driess, Danny and Florence, Pete and Sadigh, Dorsa and Guibas, Leonidas and Xia, Fei},
journal = {arXiv preprint arXiv:2401.12168},
year = {2024},
url = {https://arxiv.org/abs/2401.12168},
}
@misc{liu2023llava,
title={Visual Instruction Tuning},
author={Liu, Haotian and Li, Chunyuan and Wu, Qingyang and Lee, Yong Jae},
publisher={NeurIPS},
year={2023},
}
``` |