---
license: apache-2.0
datasets:
- HuggingFaceFW/fineweb
- PleIAs/YouTube-Commons
- allenai/WildChat-1M
language:
- de
- en
- ja
- fr
library_name: mlx
tags:
- moe
- multimodal
- j.o.s.i.e.
---

# This will be the repo for J.O.S.I.E.v4o

Like **OpenAIs GPT-4o**, it's natively Multimodal, based on the **NExT-GPT** combined with **ROPE**, **RMS Normalisation**, and **MoE**, parred with the **GPT-4o Tokenizer** from OpenAI.
This is a *future project* and will take it's time.

Further more, I will probably make a **UI application** with that model too.

Further updates comming soon!!!


First architecture Overview:

First Beta will utilize the already pretrained ImageBind Model. The linear input Projection is because the outputs of the ImageBind model are not in the correct dimensions.
Later on the input and output projections will be removed.

<img src="Architecture_overview_beta3.png" width="100%" height="auto"/>


Source code and more info will be available on my <a href="https://github.com/Goekdeniz-Guelmez/J.O.S.I.E.v4-o.git">GitHub Repo</a>