Instructions to use elley-ai/elley-1b-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use elley-ai/elley-1b-v1 with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf elley-ai/elley-1b-v1:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf elley-ai/elley-1b-v1:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf elley-ai/elley-1b-v1:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf elley-ai/elley-1b-v1:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf elley-ai/elley-1b-v1:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf elley-ai/elley-1b-v1:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf elley-ai/elley-1b-v1:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf elley-ai/elley-1b-v1:Q4_K_M

Use Docker

docker model run hf.co/elley-ai/elley-1b-v1:Q4_K_M

LM Studio
Jan

vLLM

How to use elley-ai/elley-1b-v1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "elley-ai/elley-1b-v1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "elley-ai/elley-1b-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/elley-ai/elley-1b-v1:Q4_K_M

Ollama
How to use elley-ai/elley-1b-v1 with Ollama:
```
ollama run hf.co/elley-ai/elley-1b-v1:Q4_K_M
```
Unsloth Desktop

How to use elley-ai/elley-1b-v1 with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf elley-ai/elley-1b-v1:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "elley-ai/elley-1b-v1:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use elley-ai/elley-1b-v1 with Docker Model Runner:
```
docker model run hf.co/elley-ai/elley-1b-v1:Q4_K_M
```

Lemonade

How to use elley-ai/elley-1b-v1 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull elley-ai/elley-1b-v1:Q4_K_M

Run and chat with the model

lemonade run user.elley-1b-v1-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use elley-ai/elley-1b-v1 with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf elley-ai/elley-1b-v1:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default elley-ai/elley-1b-v1:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use elley-ai/elley-1b-v1 with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf elley-ai/elley-1b-v1:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "elley-ai/elley-1b-v1:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Elley Model v1 (1B Instruct — Distilled Health Companion)

Release: v1.0.0
Base Architecture: meta-llama/Llama-3.2-1B-Instruct (1.25B Parameters)
Distillation Teachers: meta-llama/Llama-3.3-70B-Instruct & deepseek-ai/DeepSeek-V3
Physical Edge Target: Android Devices (<= 4 GB RAM, ARM64 Cortex-A76/A55)
Hugging Face Repository: elley-ai/elley-1b-v1

1. Executive Summary

Elley Model v1 is a domain-adapted conversational model optimized for mobile health accompaniment, symptom navigation, lifestyle memory, and patient safety triage.

Distilled from an audited 1,906-scenario multi-source corpus (1,000 open instruction/chat pairs, 736 clinical supervision teacher pairs, 170 pilot seed pairs), Elley Model v1 resolves generic assistant boilerplate into a calm, empathetic, and safety-gated edge companion.

2. Model Artifacts & Physical Benchmarks on Motorola Moto G35 5G

All quantized GGUF variants were quantized directly from master FP16 weights with zero requantization using llama-quantize (build 10549). Physical benchmarks were executed directly on the Motorola Moto G35 5G (ZA223CJ5CX, Unisoc T760 5G, 3.46 GB RAM, Android 15) using a continuous sustained decode loop ($\ge 90$ seconds wall-clock):

Artifact File	Format	Size	Physical Decode (tok/s)*	TTFT (Mean)	Sustained Stability**	Host Quality (Phase H)	Safety Benchmark (Phase H)	Recommended Role
`elley-1b-v1-fp16.gguf`	`F16`	2,364.7 MB	N/A (Master Base)	N/A	N/A	75.1%	80.0%	Master Unquantized Base Reference
`elley-1b-v1-q3_k_m.gguf`	`Q3_K_M`	658.8 MB	10.85 tok/s	574 ms	0.946	77.0%	90.0%	Primary Mobile Deployment Candidate
`elley-1b-v1-q4_k_m.gguf`	`Q4_K_M`	770.3 MB	9.68 tok/s	528 ms	0.694	70.7%	70.0%	Historical Reference / Standard
`elley-1b-v1-q5_k_m.gguf`	`Q5_K_M`	869.3 MB	6.67 tok/s	1,266 ms	0.693	77.6%	80.0%	High Quality Retention Candidate

\nIn addition to the standalone GGUF binaries, this repository houses the full LoRA adapter fine-tuning weights (training/lora_adapter_full/), the complete 1,906-scenario unified training/validation corpus (datasets/), and the 60-scenario held-out clinical test suite (datasets/held_out_test_suite.jsonl) for 100% reproducibility and offline redundancy.

*Sustained decode throughput measured across repeated generation runs satisfying an actual >= 90-second monotonic wall-clock workload.
**Sustained Stability: Ratio of final checkpoint throughput to initial checkpoint throughput (1.0 = zero thermal degradation).

3. Training & Provenance Summary

Training Stack: Unsloth / PyTorch 2.5.1 on NVIDIA RTX 3060 Laptop GPU (6,144 MB VRAM).
LoRA Target Matrices: All 7 projection layers (q, k, v, o, gate, up, down), $r=16, lpha=32$.
Trainable Parameters: 11,272,192 (0.90% of model).
Schedule & Convergence: 2 epochs, 382 optimizer steps, cosine decay ($1.5 imes 10^-4 ightarrow 0$), train loss 2.2676 $ ightarrow$ 0.3117.
Zero Contamination Assertion: Cryptographically verified 0.0% overlap with the 60-scenario held-out evaluation test suite.

4. Prompt Template & System Persona

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are Elley, a calm, empathetic, and knowledgeable AI health companion. Your communication is warm, supportive, and natural. You speak directly to the user without robotic boilerplate or repetitive disclaimers. You prioritize patient safety, recognize medical emergencies with urgency, adhere strictly to user constraints and formatting instructions, and handle memory seamlessly.
[KNOWN USER MEMORY: <Optional Context>]<|eot_id|><|start_header_id|>user<|end_header_id|>

<User Prompt><|eot_id|><|start_header_id|>assistant<|end_header_id|>

5. Important Clinical Disclaimer & Intended Use

Not a Standalone Medical Diagnostic Device:
Elley Model v1 is designed for conversational wellness accompaniment, lifestyle tracking, and supportive health navigation. It is NOT a clinically certified or regulatory-approved diagnostic system. It does not replace professional clinical evaluation, diagnosis, or prescription from licensed healthcare providers. The model incorporates strict emergency escalation protocols for acute presentations (such as acute chest pain, anaphylaxis, overdose, and psychiatric emergencies).

6. Cryptographic Integrity (SHA-256)

fd32a4e0c08e42750d1673e1f5e182037b8fee332852a71fc0aeeb20b819c133  models/elley-1b-v1-q3_k_m.gguf
81ed143d312a0b6b77ba80baaff6b77ac629ecd2ebef9c8be1ee026804b90cae  models/elley-1b-v1-q4_k_m.gguf
58ed113002c1632476c0e31ca6be6d6d32f3729ff472fe8d543ae9de5aa7d3ff  models/elley-1b-v1-q5_k_m.gguf
c68e438091f98465229cd0df4e78943612206914e661c06ec7812c0c11a4247d  provenance/quantization_manifest.json
9023776692562f6e27c8b443a5cc14d1093a6c8d3603b590118c919ed069d6a8  provenance/TEACHER_MODEL_MANIFEST.md
3216d131d02222664917cf5062e6bf954076dfbfca0a9d512efbcfe38e0defe8  provenance/unified_dataset_manifest.json
4447ff2991a1617d2627bb5f63c1111630389cfcbc15d597bac34b0c5f943234  evaluations/PHASE_H_HOST_QUANTIZATION_REPORT.md
792e63de59bf7455163b888ec26c339e675ad4bf4d12e9c383d153eda7a7105d  evaluations/phase_i_device_snapshot.json
2ebb6d80bd489481affbe273d58d0c3c4c01957880932590e1db3cad0681e0f1  evaluations/PHASE_I_MOBILE_BENCHMARK_REPORT.md
7d864cf8cddcda6d08a9a42b60dfe41c3a02e3636847580464f8727a1defcada  evaluations/phase_i_results.json

Downloads last month: 78

GGUF

Model size

1B params

Architecture

llama

Hardware compatibility

3-bit

4-bit

5-bit

View +1 variant

Model tree for elley-ai/elley-1b-v1

Base model

meta-llama/Llama-3.2-1B-Instruct

Quantized

(424)

this model