Instructions to use NoemaAI-labs/Noema-1.5-2B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use NoemaAI-labs/Noema-1.5-2B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="NoemaAI-labs/Noema-1.5-2B") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("NoemaAI-labs/Noema-1.5-2B") model = AutoModelForCausalLM.from_pretrained("NoemaAI-labs/Noema-1.5-2B", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use NoemaAI-labs/Noema-1.5-2B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "NoemaAI-labs/Noema-1.5-2B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "NoemaAI-labs/Noema-1.5-2B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/NoemaAI-labs/Noema-1.5-2B
- SGLang
How to use NoemaAI-labs/Noema-1.5-2B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "NoemaAI-labs/Noema-1.5-2B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "NoemaAI-labs/Noema-1.5-2B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "NoemaAI-labs/Noema-1.5-2B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "NoemaAI-labs/Noema-1.5-2B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use NoemaAI-labs/Noema-1.5-2B with Docker Model Runner:
docker model run hf.co/NoemaAI-labs/Noema-1.5-2B
Noema 1.5 2B
Noema 1.5 2B is an open-weight, local-first language model optimized for broad knowledge retention, precise instruction following, multi-turn constraint persistence, short-form code generation, and efficient reasoning.
It is the most balanced release in the Noema 2B lineage.
Compared with the previous Noema release, Noema 1.5 improves pooled knowledge, MMLU-Pro development, HumanEval+, strict IFEval, verified mathematics, and novel constraint following. Against stock Qwen3.5-2B in final evaluation, it improves strict IFEval by 7.02 points, complete three-turn Multi-IF success by 3.11 points, HumanEval+ by 3.05 points, and reasoning efficiency by 13.5%, while passing a preregistered full MMLU-Pro knowledge-retention gate.
No retrieval, external tools, test-time answer repair, or external model calls were used in the reported benchmark results.
Highlights
- Knowledge recovery: +1.42 points on the 1,900-item pooled knowledge development composite over Noema v2.
- MMLU-Pro recovery: +2.86 points over v2 on the matched development panel.
- Stronger code: HumanEval+ increased from 50.00% to 53.66% over v2.
- Better instructions: strict IFEval increased from 70.24% to 73.01% over v2.
- Better multi-turn behavior: complete three-turn Multi-IF success was 21.1% higher than stock Qwen3.5-2B.
- Efficient reasoning: 13.5% fewer completion tokens than stock Qwen3.5-2B on the final thinking panel.
- Broad knowledge retained: passed the preregistered MMLU-Pro non-inferiority gate across 11,332 untouched questions.
- Open-weight deployment: downloadable weights built for private and resource-conscious inference.
Release-over-release results
The table below compares Noema 1.5 with the exact previous Noema checkpoint from which training began.
| Benchmark | Noema v2 | Noema 1.5 2B | Delta |
|---|---|---|---|
| Pooled knowledge composite | 61.84 | 63.26 | +1.42 pp |
| MMLU-Pro development | 50.14 | 53.00 | +2.86 pp |
| HumanEval+ pass@1 | 50.00 | 53.66 | +3.66 pp |
| IFEval prompt strict | 70.24 | 73.01 | +2.77 pp |
| Verified math development | 78.00 | 92.00 | +14.00 pp |
| Novel constraints prompt strict | 45.83 | 51.67 | +5.83 pp |
These are paired development and selection results. They demonstrate release-over-release progress but are not untouched final-lockbox estimates.
Final benchmark results
The final evaluation compared the frozen Noema 1.5 candidate with the exact stock Qwen3.5-2B foundation using identical prompts, generation settings, and graders.
| Benchmark | Samples | Stock Qwen3.5-2B | Noema 1.5 2B | Delta |
|---|---|---|---|---|
| MMLU-Pro | 11,332 | 53.88 | 53.17 | -0.71 pp |
| GPQA-Diamond | 198 | 45.45 | 40.40 | -5.05 pp |
| HumanEval+ pass@1 | 164 | 50.61 | 53.66 | +3.05 pp |
| IFEval prompt strict | 541 | 65.06 | 72.09 | +7.02 pp |
| IFBench prompt loose | 300 | 31.00 | 27.33 | -3.67 pp |
| Multi-IF mean per turn | 4,501 conversations | 33.49 | 35.84 | +2.36 pp |
| Multi-IF all three turns | 4,501 conversations | 14.77 | 17.88 | +3.11 pp |
| Thinking native accuracy | 300 | 67.33 | 68.67 | +1.33 pp |
| Thinking mean tokens | 300 | 7,968.95 | 6,894.59 | -13.48% |
Statistical interpretation
- MMLU-Pro: passed the preregistered non-inferiority test. The one-sided 95% lower bound was -1.438 points against a -1.5-point margin.
- IFEval: +7.02 points, statistically significant (
p = 0.000475). - Multi-IF: +3.11 points on complete three-turn success, statistically significant (
p = 4.44e-8). - HumanEval+ and thinking: favorable point estimates that were not statistically resolved.
- GPQA-Diamond and IFBench: negative point estimates and active areas for future research.
MMLU-Pro was the strict confirmatory endpoint. The final evaluation report is available in the Noema training repository.
Model details
| Property | Value |
|---|---|
| Public name | Noema 1.5 2B |
| Internal research candidate | noema-v3-lite-l4-step-0150-bf16 |
| Architecture | Qwen3.5 hybrid Gated DeltaNet and gated attention |
| Parameters | Approximately 2B |
| Layers | 24 |
| Hidden dimension | 2,048 |
| Vocabulary | 248,320 tokens |
| Training method | Restoration-aware on-policy distillation |
| Weight update | Rank-128 all-projection LoRA, merged for release |
| Native backbone context | Up to 262,144 tokens |
| Independently validated context | Up to 24,576 tokens |
| Primary evaluated language | English |
| Release status | Open-weight; not open source |
| MTP | Disabled |
Noema 1.5 is the text-generation trunk. Multimodal capabilities were not trained or evaluated in this release.
Training
Noema 1.5 began from the exact previous Noema 2B release and used a restoration-aware on-policy distillation program:
- 150 optimizer steps;
- 19,200 on-policy trajectories;
- 7,982,979 valid completion tokens;
- 32 unique prompts per step;
- four student rollouts per prompt;
- rank-128 LoRA with alpha 256;
- Gated DeltaNet, ordinary-attention, and MLP projections targeted;
- tied embedding and output matrix kept frozen.
Seventy percent of assignments used Qwen3.5-9B as the capability scorer. Thirty percent used untouched Qwen3.5-2B as a restoration scorer to protect general behavior.
The training mixture contained:
- 52.5% broad knowledge;
- 17.5% mathematics and code;
- 22.5% general-chat restoration;
- 7.5% novel composed constraints.
An experimental secondary constraint-RL stage was tested, failed its first mandatory gate, and was rejected. It did not modify the released checkpoint.
Intended uses
Noema 1.5 is well suited to:
- local and privacy-sensitive assistants;
- structured and format-constrained generation;
- multi-turn conversations;
- short Python program synthesis;
- educational and lightweight reasoning tasks;
- offline applications;
- memory- and compute-constrained deployments.
For knowledge-intensive applications, use retrieval or authoritative tools and verify important factual claims.
Usage
Noema 1.5 requires a transformers version with qwen3_5_text support. The release was built and validated with Transformers 5.12.1.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "NoemaAI-labs/Noema-1.5-2B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
dtype=torch.bfloat16,
device_map="auto",
)
messages = [
{
"role": "user",
"content": "Write a Python function that merges two sorted lists.",
}
]
inputs = tokenizer.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
enable_thinking=False,
return_dict=True,
return_tensors="pt",
).to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=1024,
do_sample=False,
)
response = tokenizer.decode(
outputs[0][inputs["input_ids"].shape[-1]:],
skip_special_tokens=True,
)
print(response)
Non-thinking mode is recommended for concise instruction following, structured generation, and code. The packaged defaults are greedy (do_sample=false) and stop at <|im_end|>.
For harder reasoning tasks, use enable_thinking=True, do_sample=True, temperature=1.0, top_p=0.95, and top_k=20. Deployments should still enforce output limits and repeated-span detection.
Evaluation methodology
The final evaluation used:
- frozen model and tokenizer revisions;
- identical prompts and settings across both arms;
- BF16 inference on NVIDIA L40S hardware;
- vLLM 0.19.1;
- pinned official graders;
- paired bootstrap intervals and McNemar tests;
- deterministic keys and exact-count verification;
- no score inspection before both arms were complete.
Limitations
- The release-over-release v2 comparison uses matched development suites, not a third untouched final arm.
- Broad MMLU-Pro knowledge was statistically retained against stock but not improved in the final point estimate.
- GPQA-Diamond and IFBench produced lower final point estimates.
- The release is based on one primary training seed and does not establish that on-policy distillation beats matched supervised training.
- Evaluation was primarily English and text-only.
- Native 262K context support comes from the Qwen3.5 architecture; Noema independently evaluated contexts only up to 24,576 tokens.
- Thinking mode can still produce long or repetitive responses.
- The model can hallucinate and should not be treated as the sole authority for high-stakes decisions.
Release status
Noema 1.5 2B is an open-weight release, not an open-source release. The weights are publicly downloadable, but no open-source license is granted with this repository at this time.
Citation
@misc{noema15_2b_2026,
title = {Noema 1.5 2B: Restoration-Aware On-Policy Distillation for Local AI},
author = {NoemaAI Labs},
year = {2026},
howpublished = {\url{https://huggingface.co/NoemaAI-labs/Noema-1.5-2B}}
}
- Downloads last month
- 329