Instructions to use pinkelephantlimited/phone-llm with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use pinkelephantlimited/phone-llm with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="pinkelephantlimited/phone-llm")

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("pinkelephantlimited/phone-llm", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use pinkelephantlimited/phone-llm with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "pinkelephantlimited/phone-llm"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "pinkelephantlimited/phone-llm",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/pinkelephantlimited/phone-llm

SGLang

How to use pinkelephantlimited/phone-llm with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "pinkelephantlimited/phone-llm" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "pinkelephantlimited/phone-llm",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "pinkelephantlimited/phone-llm" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "pinkelephantlimited/phone-llm",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use pinkelephantlimited/phone-llm with Docker Model Runner:
```
docker model run hf.co/pinkelephantlimited/phone-llm
```

PhoneLLM — Sovereign On-Device LLM for Android

Own 1.2B model from scratch • DeepSeek-like chat • Local inference + Web Search • HF as storage

48B was too large for a phone — so we distilled our own. Pink Elephant 48B-S (47.7B MoE) is the teacher; PhoneLLM 1.2B is the student trained from scratch with own tokenizer (32k BPE) and own architecture (24L, 2048H, GQA, SwiGLU). No Qwen, no copy — sovereign weights, MIT-licensed. Runs fully offline on your iQOO Neo5 SE via llama.cpp, with optional web fetch for freshness.

Why Own Model

We learned from the whole HF universe (not just pinkelephantlimited):

PocketPal AI / Maid / MLC Chat (2026) — GGUF Q4_K_M is the community standard; base APK ~20MB, models downloadable, Vulkan/CPU universal. We adopt llama.cpp via NDK + GGUF (already proven in PinkElephantAgent) for max portability, not vendor-locked LiteRT/ML Kit.
HF RAG Spaces — best pattern is local-first, not local-only: sequential local RAG + web search, concatenated with section headers ([Local], [Web Search Results]) so LLM knows source. Dark mode retrieval discussions confirm structure-aware chunking + 384d embedders beat naive splitting.
PromptQuorum / ItsFOSS 2026 tests — Snapdragon 8 Elite ~40 tok/s (NPU), Snapdragon 870 (iQOO Neo5 SE) ~12-18 tok/s CPU/Vulkan on Phi-3-mini/Gemma2B class. Our 1.2B Q4_K_M targets ~15 tok/s on your device — DeepSeek-like streaming, no cloud.

We do not bundle Qwen. Own tokenizer + own 1.2B config (phone_llm_train/model_config.json) ensures brand sovereignty.

Architecture: Local + Web, DeepSeek UI

User → WebSearchService (Jina Reader + DDG lite, 8s timeout, capped 1500 chars)
     └→ local prompt (PhoneLLM GGUF via NativeEngine.nativeLlamaChat, 2048 ctx, 4 threads)
        → DeepSeek-like RecyclerView (streaming bubble, markdown, history, web toggle)

Base APK ~20MB — models not bundled (learned from Pocket LLM v1.4: 200MB base → downloadable). APK lives in apk/ on HF for storage.
Storage: app-private getExternalFilesDir()/models — correct scoped storage (PocketPal pattern).
Training vehicle: molab RTX PRO 6000 102GB (sb-bf9f066c447abaab), BF16, PagedAdamW8bit, gradient checkpointing — same QLoRA lineage as 48B but from scratch.

The Phone Model at a Glance

Spec	Value
Params	1.2B dense, 24L, 16 heads, 4 KV, hidden 2048, intermediate 5504, vocab 32000, RoPE 10k, 4096 ctx
Tokenizer	Own BPE 32k (`[PAD][BOS][EOS][WEB][THINK]`), trained on FineWeb-Edu + OpenHermes + StarCoder (HF datasets)
Quant	Q4_K_M ~0.8GB (phone), Q4_0 ~0.6GB (low-end), F16 ~2.4GB
Teacher	Pink Elephant 48B-S (47.7B) — distillation, not copy
Dataset	HF permissive mix 60% web / 30% chat / 10% code — curation is the moat
Hardware	molab 102GB → iQOO Neo5 SE 8GB offline
Inference	llama.cpp JNI (`peagent.cpp`), Vulkan available, 4 threads

Repo Layout (HF as storage)

pinkelephantlimited/phone-llm/
  README.md
  config.json                 # own 1.2B config (sovereign)
  tokenizer.json              # own BPE (after train)
  train_tokenizer.py          # from-scratch BPE
  train_pretrain.py           # 1.2B pretrain on molab
  OWN_MODEL_PLAN.md
  phone-llm-Q4_K_M.gguf       # quantized for APK (after training)
  phone-llm-f16.gguf
  training/phone-llm/         # logs, loss curves
  apk/phone-llm-v1.0.apk      # base APK

Quick Start (on-device)

Install APK from apk/phone-llm-v1.0.apk via adb install
First launch → taps Download model (HF) → fetches phone-llm-Q4_K_M.gguf to models/ with progress
Toggle Web Search for hybrid mode

adb install apk/phone-llm-v1.0.apk
# or via HF
huggingface-cli download pinkelephantlimited/phone-llm phone-llm-Q4_K_M.gguf --local-dir ./models

Build From Source

cd app/PhoneLLM
./gradlew assembleDebug  # ~20MB base APK
adb install app/build/outputs/apk/debug/app-debug.apk

Training (molab GPU)

python phone_llm_train/train_tokenizer.py
python phone_llm_train/train_pretrain.py  # 102GB GPU, 5000 steps v0.1
python -m llama_cpp.convert_hf_to_gguf --outfile phone-llm-Q4_K_M.gguf --outtype q4_k_m /home/marimo/phone-llm-out/final

APK

See app/PhoneLLM — DeepSeek-like chat, Hybrid RAG, HF storage. Test on iQOO Neo5 SE via adb/scrcpy.

Pink Elephant Limited — sovereign on-device intelligence, 2026

Downloads last month: 356

Model tree for pinkelephantlimited/phone-llm

Unable to build the model tree, the base model loops to the model itself. Learn more.