Instructions to use Unchanted/omi-models with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Unchanted/omi-models with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible embeddings server: llama serve -hf Unchanted/omi-models:Q8_0 --embeddings
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible embeddings server: llama serve -hf Unchanted/omi-models:Q8_0 --embeddings
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible embeddings server: ./build/bin/llama serve -hf Unchanted/omi-models:Q8_0 --embeddings
- Llama new
- LM Studio
- Jan
- Ollama
How to use Unchanted/omi-models with Ollama:
ollama run hf.co/Unchanted/omi-models:Q8_0
- Unsloth Desktop
- Docker Model Runner
How to use Unchanted/omi-models with Docker Model Runner:
docker model run hf.co/Unchanted/omi-models:Q8_0
- Lemonade
How to use Unchanted/omi-models with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Unchanted/omi-models:Q8_0
Run and chat with the model
lemonade run user.omi-models-Q8_0
List all available models
lemonade list
- Atomic Chat
Omi models
Exact model files the Omi Android app downloads. The app checks each file's size and sha256 before it uses it.
Span reader
span-ft-tiny-q8_0.gguf with span_heads.json and span_heads.bin.
What it does: it reads up to 5 short message rows that are already on screen and marks the words in one row that answer the question. When no row holds the answer, it marks nothing. It never writes words of its own.
Base model: deepset/tinyroberta-squad2 (82M, CC BY 4.0).
Tuning: one pass over 18,000 rows of open chat and question data:
- SQuAD 2.0 (CC BY-SA 4.0);
- Molweni (Apache 2.0);
- QAConv (CC0).
No private or personal data was used.
Files:
- the encoder exported to GGUF at q8_0 and run with llama.cpp (
--embeddings --pooling none); - the start and end head (
span_heads.jsondescribes it,span_heads.binholds its weights).
- the encoder exported to GGUF at q8_0 and run with llama.cpp (
Status: a first version, used in shadow while it is tested.
| File | sha256 |
|---|---|
| span-ft-tiny-q8_0.gguf | c11f51e44af49e2e3f48260370c4b32c615099583b37e63e33a80027750ed905 |
| span_heads.json | 75dba1f656cb74bd72df91be77ca4a0d255e6960eeb520b0bd8abaabf32e45c1 |
| span_heads.bin | 4bc194c0d95b6a70cf3de3171d735e061f1c0120dbaeb86bade2a5bffadbe10d |
Licence and credit
The weights are derived from tinyroberta-squad2 (deepset, CC BY 4.0) and tuned partly on SQuAD 2.0 (Rajpurkar et al., CC BY-SA 4.0). They are shared under CC BY-SA 4.0.
- Downloads last month
- -
8-bit
Model tree for Unchanted/omi-models
Base model
deepset/tinyroberta-squad2