Instructions to use vinyteles/mobilerag-plus-assets with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use vinyteles/mobilerag-plus-assets with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible embeddings server: llama serve -hf vinyteles/mobilerag-plus-assets:Q8_0 --embeddings
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible embeddings server: llama serve -hf vinyteles/mobilerag-plus-assets:Q8_0 --embeddings
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible embeddings server: ./build/bin/llama serve -hf vinyteles/mobilerag-plus-assets:Q8_0 --embeddings
- Llama new
- LM Studio
- Jan
- Ollama
How to use vinyteles/mobilerag-plus-assets with Ollama:
ollama run hf.co/vinyteles/mobilerag-plus-assets:Q8_0
- Unsloth Desktop
- Docker Model Runner
How to use vinyteles/mobilerag-plus-assets with Docker Model Runner:
docker model run hf.co/vinyteles/mobilerag-plus-assets:Q8_0
- Lemonade
How to use vinyteles/mobilerag-plus-assets with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull vinyteles/mobilerag-plus-assets:Q8_0
Run and chat with the model
lemonade run user.mobilerag-plus-assets-Q8_0
List all available models
lemonade list
- Atomic Chat
MobileRAG+ assets for the CEMIG mobile assistant
Files the CEMIG mobile app downloads to run MobileRAG+ on the phone over the benchmark_mobile
corpus: retrieval with an EcoVector index, file-neighbor expansion, and Selective Content
Reduction (SCR) with precomputed window vectors. The answer model (Energy-Mobile, LiteRT) is
downloaded separately from Fazzioni/Energy-Mobile.
mobilerag-plus.json lists every file with its size and SHA-256, and the parameters the app
checks before using them. The app pins one commit of this repository.
Files
| Path | Size | What |
|---|---|---|
embedders/qwen3-embedding-0.6b-v2-q8_0.gguf |
639 MB | Question embedder for retrieval (last-token pooling, 1,024 dims) |
embedders/granite-embedding-107m-multilingual-q8_0.gguf |
121 MB | Question embedder for SCR (CLS pooling, 384 dims) |
benchmark_mobile/corpus.sqlite |
31 MB | 3,764 chunks with their Qwen vectors (sqlite-vec vec0 table) |
benchmark_mobile/eco-vector.sqlite |
17 MB | EcoVector index: centroid graph + 19 cluster HNSW graphs, one row each |
benchmark_mobile/windows-int8.sqlite |
34 MB | 72,061 SCR window vectors (3-sentence windows, step 1), int8 |
mobilerag-plus.json |
2 KB | Manifest |
How they were built
- Embedders: converted with
convert_hf_to_gguf.pyof llama.cpp b10256 (the version llama.rn 0.12.9 bundles) to Q8_0. On 500 benchmark_mobile questions, MobileRAG+ with these query vectors keeps hit, evidence recall, nDCG@5 and MRR@5 within 0.4 points of the float32 sentence-transformers models.- Qwen: prepend the query instruction stored in the manifest, with no separator; the model
appends
<|endoftext|>and pools the last token. - granite: no instruction.
- Qwen: prepend the query instruction stored in the manifest, with no separator; the model
appends
- Corpus: chunks of the benchmark_mobile source documents, embedded with
UFGCEMIGONA/Qwen3-Embedding-0.6B-v2. - EcoVector index: k-means partitions (about 200 vectors each), an HNSW graph over the
centroids and one per cluster; searched with
n_probe6 andef_search32. Built and exported bymobile_rag(build-index,export-mobile-index). - SCR windows: every 3-sentence window of every chunk embedded with granite-107m (float32)
and quantized to int8 with one scale per window (
build-scr-windows).
Sources and licenses
| Source | License |
|---|---|
UFGCEMIGONA/Qwen3-Embedding-0.6B-v2, fine-tuned from Qwen/Qwen3-Embedding-0.6B |
Base model Apache 2.0; see the fine-tune's card |
ibm-granite/granite-embedding-107m-multilingual |
Apache 2.0 |
UFGCEMIGONA/benchmark_mobile source documents (corpus text) |
other |
- Downloads last month
- 71
Hardware compatibility
Log In to add your hardware
8-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐ Ask for provider support