Instructions to use Mpairwe49/retinalai-narrator-135m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Mpairwe49/retinalai-narrator-135m with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Mpairwe49/retinalai-narrator-135m")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Mpairwe49/retinalai-narrator-135m")
model = AutoModelForCausalLM.from_pretrained("Mpairwe49/retinalai-narrator-135m", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Mpairwe49/retinalai-narrator-135m with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Mpairwe49/retinalai-narrator-135m"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Mpairwe49/retinalai-narrator-135m",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Mpairwe49/retinalai-narrator-135m

SGLang

How to use Mpairwe49/retinalai-narrator-135m with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Mpairwe49/retinalai-narrator-135m" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Mpairwe49/retinalai-narrator-135m",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Mpairwe49/retinalai-narrator-135m" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Mpairwe49/retinalai-narrator-135m",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Mpairwe49/retinalai-narrator-135m with Docker Model Runner:
```
docker model run hf.co/Mpairwe49/retinalai-narrator-135m
```

RetinalAI compact narrator (135M, vocabulary-pruned)

Writes the free-text clinical narrative for a retinal screening result. It is a research artifact, not a medical device — see Limitations.

Distilled (sequence-level KD) from a self-hosted Qwen/Qwen3-8B-AWQ teacher onto SmolLM2-135M-Instruct, then vocabulary-pruned for footprint.

What is different about it

Prose only. It emits the narrative as plain text. The structured triage decision is served separately by a 3 KB linear head, so this model has no JSON contract to break. An earlier JSON-emitting version of the same base model failed to parse on 100% of cases once 4-bit quantized; prose-only generation is 1.000 at every precision.
Pruned vocabulary, 49,152 -> 920 tokens. Embedding rows are the original tokenizer ids listed in keep_ids.json, in order. The tokenizer is unmodified; callers map ids old->new before the model and new->old before decoding. Tokens outside the kept set are re-encoded byte-wise rather than dropped. The vocabulary is seeded from all 45 disease names the upstream classifier can emit, and all 45 are verified to survive the round trip.

Use src/narrator/compact.py from the OptiscanAI repository — the id remap is required, so AutoModelForCausalLM alone will produce garbage.

Evaluation

24 held-out cases, scored against the teacher's own narrative for the same case:

precision	size	generation rate	omission	misquoted probabilities	acuity divergence
bf16	213.5 MB	1.000	0.000	0.000	0.250
int8	107.3 MB	—	—	—	— (37.5 s/case, impractical)
nf4	54.2 MB	1.000	0.417	0.167	0.583

bf16 is the only precision recommended. 4-bit reaches 54 MB but drops findings the teacher reported in 42% of cases while remaining fluent, which makes the omissions hard to notice.

"Acuity divergence" counts narratives using urgency language the teacher did not use for that same case. It is a broad screening signal, not a hallucination count.

Limitations

Not a medical device. Not for clinical use. No regulatory clearance, no clinical validation, no prospective study.
Evaluated on 24 held-out cases from a single RFMiD-derived sample. That is enough to distinguish "works" from "broken" and little else.
No clinician review of generated text has been performed.
Trained on 56 examples. It reproduces the teacher's register on cases that look like its training distribution and has not been probed outside it.
No EMERGENCY case appeared anywhere in the source data, so emergency phrasing is untested. Escalation is handled deterministically upstream, not here.
Callers must append their own AI-disclosure statement; the teacher traces did not contain one, so the model does not produce it.

Downloads last month: 87

Safetensors

Model size

0.1B params

Tensor type

F16

Model tree for Mpairwe49/retinalai-narrator-135m

Base model

HuggingFaceTB/SmolLM2-135M

Quantized

HuggingFaceTB/SmolLM2-135M-Instruct

Finetuned

(363)

this model