README.md

---
license: cc-by-sa-4.0
language:
- en
tags:
- Multimodal
- StableLM
datasets:
- LDJnr/Capybara
- LDJnr/LessWrong-Amplify-Instruct
- LDJnr/Pure-Dove
- LDJnr/Verified-Camel
---

# Obsidian: Worlds smallest multi-modal LLM. First multi-modal model in size 3B

## Model Name: Obsidian-3B-V0.5

Obsidian is a brand new series of Multimodal Language Models. This first project is led by Quan N. and Luigi D.(LDJ).

Obsidian-3B-V0.5 is a multi-modal AI model that has vision! it's smarts are built on [Capybara-3B-V1.9](https://huggingface.co/NousResearch/Capybara-3B-V1.9) based on [StableLM-3B-4e1t](https://huggingface.co/stabilityai/stablelm-3b-4e1t). Capybara-3B-V1.9 achieves state-of-the-art performance when compared to model with similar size, even beats some 7B models.

Current finetuning and inference code is available on our GitHub repo: [Here](https://github.com/NousResearch/Obsidian)

## Acknowledgement

Obsidian-3B-V0.5 was developed and finetuned by [Nous Research](https://huggingface.co/NousResearch), in collaboration with [Virtual Interactive](https://huggingface.co/vilm).
Special thank you to **LDJ** for the wonderful Capybara dataset, and **qnguyen3** for the model training procedure.
## Model Training

Obsidian-3B-V0.5 followed the same training procedure as LLaVA 1.5

## Prompt Format

The model followed ChatML format. However, with `###` as the seperator

```
<|im_start|>user
What is this sign about?\n<image>
###
<|im_start|>assistant
The sign is about bullying, and it is placed on a black background with a red background.
###
```

## Benchmarks

Coming Soon!


Citation:

```
@article{nguyen2023Obsidian-3B,
  title={Obsidian-3B: First Multi-modal below 7B Parameters.},
  author={Nguyen, Quan and Daniele},
  journal={HuggingFace:https://huggingface.co/NousResearch/Obsidian-3B-V0.5},
  year={2023}
}
```

Acknowledgements:

```
@article{daniele2023amplify-instruct,
  title={Amplify-Instruct: Synthetically Generated Diverse Multi-turn Conversations for Effecient LLM Training.},
  author={Daniele, Luigi and Suphavadeeprasit},
  journal={arXiv preprint arXiv:(comming soon)},
  year={2023}
}
```