Gargi-Base

The pretrained base model behind Gargi: a 384M-parameter English model trained from scratch by UdgamLabs on about 17 billion tokens, with a 32K-token context window (reliable recall up to ~16K). It is not chat-tuned; use Gargi for chat and step-by-step maths.

On the same evaluation harness, Gargi-Base comes within 2 points of Qwen2.5-0.5B on ARC-Easy, ARC-Challenge and WinoGrande, although Qwen2.5-0.5B was trained on ~18 trillion tokens: roughly 1,000× more data.

Data efficiency: base models

Model	Params	Training tokens*	MMLU	ARC-E	ARC-C	HellaSwag	WinoGrande	PIQA
Gargi-Base	384M	~0.017T	25.5	62.8	32.3	41.5	54.6	66.8
Qwen2.5-0.5B	494M	~18T	47.3	64.7	32.4	52.1	56.1	70.0
SmolLM2-360M	360M	~4T	25.6	70.2	38.3	56.3	59.0	71.5
Granite-4.0-350M (base)	352M	–	32.0	65.4	33.7	52.0	56.0	70.7
Gemma-3-270M (pt)	268M	~6T	26.1	58.8	28.2	41.3	53.4	68.1

*Training-token counts for other models are as published by their authors.

Gargi trades breadth for efficiency: on reasoning-style multiple choice (ARC, WinoGrande) it is close to models trained on hundreds to thousands of times more data, while knowledge-heavy and everyday-commonsense benchmarks (MMLU, HellaSwag, PIQA) still scale with data, and there it is clearly behind.

Long-context recall (needle in a haystack)

A 5-digit code is hidden early in a long document of real text and asked for at the end; 30 trials per length. Each model gets its own start token; lengths beyond a model's declared context are marked n/a.

Model	Params	8K	16K	20K	24K	32K
Gargi	384M	100	93	90	13	0
Qwen3-0.6B	600M	100	100	100	100	100
Hunyuan-0.5B-Instruct	500M	100	100	100	100	100
Falcon-H1-Tiny-R-0.6B	600M	100	97	100	100	97
Granite-4.0-H-350M	350M	100	100	100	100	100
ERNIE-4.5-0.3B	300M	100	100	100	100	100
Gemma-3-270M-it	270M	100	100	100	100	63
LFM2.5-350M	350M	90	0	0	0	0
SmolLM2-360M-Instruct	360M	60	n/a	n/a	n/a	n/a

Use Gargi with inputs up to about 16K tokens.

Use

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("udgamlabs/Gargi-Base")
model = AutoModelForCausalLM.from_pretrained("udgamlabs/Gargi-Base", dtype="auto")
ids = tok("The three states of matter are", return_tensors="pt")   # the tokenizer adds <bos>, which the model expects
print(tok.decode(model.generate(**ids, max_new_tokens=40, do_sample=False)[0]))

Limitations

A base model continues text; it does not follow instructions. Limited world knowledge (MMLU near chance), often wrong on facts and arithmetic, English only, not safety-tuned. Reliable recall only up to ~16K tokens.

Data acknowledgements

Gargi was trained from scratch by UdgamLabs on openly licensed and public-domain text, plus synthetic text written by openly licensed models. We gratefully acknowledge the following sources, used under their licences:

• Web, books, papers and public records: FineWeb / Ultra-FineWeb (ODC-By 1.0 / Apache-2.0); Wikipedia and Wikisource (CC-BY-SA); the Common Pile collections (public-domain books from Project Gutenberg, the Library of Congress and pre-1929 publications; openly licensed arXiv papers, DOAB books and PubMed Central articles under CC-BY / CC-BY-SA / CC0; US Government Publishing Office records; permissively licensed source code).
• UK Parliament: Contains Parliamentary information licensed under the Open Parliament Licence v3.0.
• Synthetic and instruction data: PleIAs SYNTH (CC-BY-4.0), FinePhrase (ODC-By), NVIDIA Nemotron datasets (CC-BY-4.0 and the NVIDIA Open Data License), Salesforce xLAM function-calling (CC-BY-4.0), Table-GPT (MIT), and step-by-step solutions and other examples written by OpenAI's gpt-oss models (Apache-2.0).
• Maths problems: GSM8K (MIT) and MATH (MIT) training splits. Their test splits were used only for evaluation.

Licences of individual documents vary within collections; we kept only documents whose licence allows this use.

Licence

Apache-2.0.