Numo-1-E2B
Numo-1 reads a paper's title and abstract and answers a defined question about it, such as what kind of study is this?, with a probability for every possible answer. Each decision takes one model pass, about 124 ms on a consumer RTX 3060, and nothing leaves your machine.
- Accurate on its tasks. It comes within 1–2 points of the best hosted decision models, which are much larger.
- Probabilities you can trust. It has the lowest calibration error of every model we tested. You can therefore let it decide the papers it is sure about and send the rest to a person.
- Open and local. Apache-2.0 weights and code. It runs offline after the download.
It is a specialist, not a general assistant. On tasks it was not trained for, it is weak (see limitations).
Research release. Probability scores are estimates. Keep a person in the loop for decisions that matter.
What it decides
Table as text
| Task ID | Question | Answers |
|---|---|---|
study_approach |
What kind of study is this paper? | Six study types, such as trial, observational, review or protocol |
research_area |
What is its main field? | Medicine · Biology · Physics · Computer science · Other |
biomedical_relevance |
Is a paper relevant to a seed paper? | Irrelevant · Partially · Relevant |
scientific_field |
Which broad arXiv field? | 2–8 of eight fields |
support_boolean |
Does the evidence fully support a hypothesis? | No · Yes |
support_grade3 |
How much does it support it? | None · Partial · Full |
consensus4, consensus5 |
How strongly did annotators agree it is supported? | Four or five ordered bands |
The exact wording of every question and answer is in task-contracts.json.
Who it is for
- Systematic-review and screening tools. Flag trials, observational studies, reviews and protocols in thousands of abstracts, and send only the uncertain ones to a reviewer.
- Research libraries and reference managers. Tag study type and field automatically, offline.
- Groups with confidential or unpublished work. The paper text never leaves your machine.
Results
Every model answered the same inputs. Hosted models were queried in October 2026 and were never used to train Numo.
Study design on PubMed
There are 600 recent PubMed papers, 150 for each design. The correct answers are the publication types assigned by the indexers at the US National Library of Medicine (NLM), not by us. Numo-1 never saw these papers during training.
With 95% intervals:
Table as text
| Model | Accuracy | Calibration error | Numo-1 minus model (95% interval) |
|---|---|---|---|
| Jev 1.13 (TypeSafe) | 96.2% | 1.8% | −1.8 (−3.2 to −0.5) |
| Perplexity Decider v1.2 | 96.0% | 2.1% | −1.7 (−3.0 to −0.5) |
| Jev 1.14 (TypeSafe) | 95.8% | 1.8% | −1.5 (−2.8 to −0.2) |
| Perplexity Decider Flash v1.2 | 95.2% | 2.6% | −0.8 (−2.7 to +1.0) |
| GPT-6 Luna Decisions | 95.0% | 2.4% | −0.7 (−2.3 to +1.0) |
| Clef Flash (Cloudflare) | 94.8% | 6.6% | −0.5 (−2.0 to +1.0) |
| Numo-1-E2B (local) | 94.3% | 1.6% | — |
| Microsoft Decision-1 | 93.5% | 4.2% | +0.8 (−1.0 to +2.7) |
| Gemma 4 E2B, untrained (local) | 92.7% | 7.2% | +1.7 (−0.3 to +3.5) |
| VegaML 4B (local) | 61.5% | 24.3% | +32.8 (+28.8 to +36.8) |
| Laya (local) | 51.3% | 19.1% | +43.0 (+39.0 to +47.0) |
Numo-1's accuracy by design: trials 88.0%, observational 96.0%, reviews 95.3%, protocols 98.0%.
Study approach and research area across four fields
What are Tests C and D? They are two test sets we built to measure Numo-1 on papers it has never seen.
- The papers. Each set has 1,600 papers. There are 400 medical papers from Europe PMC, and 400 each from biology, physics and computer science, drawn at random from arXiv by primary category. No paper appears in the training data or in the other set.
- The answers. Two AI reviewers labelled every paper independently, using the same written guidelines. A third AI reviewer settled the papers where they disagreed, about 1 in 15.
- Locked before scoring. The answers were fixed before any model was scored, and were read once, after Numo-1 was final.
- One caveat. Part of Numo's training data was labelled in the same way, which could slightly favour Numo. The PubMed benchmark above uses independent NLM labels and does not have this issue.
- Not in this release. The test sets are not included in this repository (see why).
Table as text
| Model | Study approach, Test C | Study approach, Test D | Research area, Test C | Research area, Test D |
|---|---|---|---|---|
| Jev 1.14 | 91.8% | 92.7% | 90.1% | 89.6% |
| Perplexity Decider v1.2 | 92.0% | 92.7% | 90.6% | 90.1% |
| Jev 1.13 | 91.2% | 92.0% | 89.1% | 89.8% |
| Numo-1-E2B (local) | 91.3% | 90.8% | 88.9% | 87.5% |
| GPT-6 Luna Decisions | 90.6% | 91.6% | 87.8% | 87.4% |
| Perplexity Decider Flash v1.2 | 90.5% | 91.7% | 87.1% | 86.4% |
| Microsoft Decision-1 | 91.0% | 91.4% | 82.6% | 82.7% |
| Clef Flash | 89.9% | 90.9% | 84.6% | 86.8% |
| Gemma 4 E2B, untrained (local) | 84.8% | 84.6% | 71.8% | 73.6% |
- On Test C, Numo-1 is statistically level with Jev 1.14 on study approach.
- On Test D, the best hosted models lead by about 2 points.
- Numo-1 is the best-calibrated model on both tasks and both tests. Its calibration error on Test D is 2.3% for study approach and 1.2% for research area. Jev 1.14 scores 2.5% and 2.2%.
Let it decide when it is sure
Table as text
| Papers decided automatically | Study approach | Research area |
|---|---|---|
| All | 91.0% | 88.2% |
| Most confident 90% | 95.3% | 92.3% |
| Most confident 80% | 98.0% | 95.7% |
| Most confident 70% | 99.2% | 97.8% |
| Most confident 50% | 99.9% | 99.4% |
These figures cover Tests C and D together, 3,200 papers per task. Check any threshold on a sample of your own papers before relying on it.
Speed and footprint
Table as text
| Numo-1-E2B | |
|---|---|
| Time per decision | 124 ms median, 175 ms p95 (RTX 3060 12 GB, one at a time) |
| GPU memory | 4.7 GB peak; the large embedding table stays in CPU memory |
| Network | Not needed after download |
| For comparison | Hosted Jev 1.13: 240 ms median, including network |
More results are in EVALUATION.md, including biomedical relevance, two further test sets (Tests A and B), a check against each paper's source category and the original evidence tasks. Machine-readable values are in evaluation-summary.json.
Limitations
- A specialist, not a general model. On six general science benchmarks from the public Decision Index, Numo-1 averages 32.8. That is below its untrained Gemma base (48.0) and far below Jev 1.13 (77.9). Use it for the tasks above only.
- Input limits. Each decision is limited to 1,024 tokens, including the question and the answers, and to 2–8 answers. Longer inputs are refused, never cut short. English only.
- How the references were made. The references for Tests A–D and most of the training labels come from AI reviewers, not human experts. The PubMed benchmark uses NLM indexer labels.
- Small trade-offs on older tasks. The final training stage cost a little accuracy on SciFact claim direction (86.0%, down from 87.7%) and on RELISH relevance (48.6%, down from 52.4%).
- Not a clinical or safety system. It says what kind of study a paper is, not whether the study is good. Do not use it as the only basis for including a paper in a systematic review or for clinical decisions.
Quickstart
You need Linux, an NVIDIA GPU and Python 3.12 with the versions in requirements.txt. You also need a C compiler and the Python development headers, because PyTorch compiles a small helper on first use (see setup).
import torch
from transformers import AutoModel, AutoTokenizer
path = "./Numo-1-E2B" # your local model folder
tokenizer = AutoTokenizer.from_pretrained(path, trust_remote_code=True, local_files_only=True)
model = AutoModel.from_pretrained(
path, trust_remote_code=True, local_files_only=True, dtype=torch.bfloat16
).prepare_for_inference("cuda")
result = model.predict(
tokenizer,
task="study_approach",
text="Title: Vitamin D and fracture risk in older adults\n"
"Abstract: We randomly assigned 2,000 participants to vitamin D or placebo ...",
)
print(result["answer"], result["probabilities_by_id"])
trust_remote_code=True runs the loading code that ships in this folder.
After the download, everything works offline. The usage guide
covers every task, the input formats and custom questions.
How it was built
Numo-1-E2B starts from Google's Gemma 4 E2B. We added a small decision head and trained it, together with low-rank (LoRA) adapters on the backbone, in several supervised stages. The training data were public scientific datasets and AI-reviewed papers. The adapters are merged into the released weights. No hosted comparison model was ever used as a teacher or as a source of labels. See training and the model reference.
From Laya to Numo
Laya is an open decision engine built on ModernBERT. It gives the model explicit answer options and makes typed decisions directly. Numo adapts that design to a Gemma backbone and to scientific tasks. It contains no Laya weights. See why Numo and credits.
Release facts
Links: google/gemma-4-E2B-it · LICENSE · NOTICE
Table as text
| Model ID | mycel-ai/Numo-1-E2B |
| Version | numo-1-e2b-v1 |
| Base model | google/gemma-4-E2B-it, revision 3e22461f |
| Parameters | 4.63 B stored, of which 2.35 B are an embedding table kept on CPU |
| Weights | 9.27 GB in four safetensors files |
| Tested on | RTX 3060 12 GB with 64 GB system RAM |
| Licence | Apache-2.0 for weights and code (LICENSE, NOTICE) |
People and credit
Links: dev@mycel-ai.de · mycel-ai.de
Table as text
| Author | Alphin Tom |
| Rights holder | Mycel UG (haftungsbeschränkt), Germany |
| Contact | dev@mycel-ai.de · mycel-ai.de |
Numo-1-E2B builds on the work of others, who keep their own rights:
- Google DeepMind, for Gemma 4 E2B;
- Nandha Kishor M, Convai Innovations and the Laya contributors, for the decision design;
- the authors of the datasets listed in CREDITS.md.
Naming them does not imply that they endorse Numo.
Citation
If you use Numo-1-E2B in your work, please cite:
@misc{tom2026numo1e2b,
title = {Numo-1-E2B: Calibrated Scientific-Paper Decisions on Local Hardware},
author = {Tom, Alphin},
year = {2026},
publisher = {Mycel UG (haftungsbeschr{\"a}nkt)},
doi = {10.57967/hf/11188},
howpublished = {\url{https://huggingface.co/mycel-ai/Numo-1-E2B}},
note = {Model weights and code, version numo-1-e2b-v1, Apache-2.0}
}
Or in text:
Tom, A. (2026). Numo-1-E2B: Calibrated scientific-paper decisions on local hardware (Version numo-1-e2b-v1) [Computer software]. Mycel UG (haftungsbeschränkt). https://doi.org/10.57967/hf/11188
When you describe how Numo works, please also credit the Gemma 4 base model and the Laya decision design.
- Downloads last month
- -