Ania 11B Research Preview

Ania 11B Research Preview is an experimental Polish-language model built on top of Bielik 11B v2.6 Instruct. It is a public preview of one stage of the long-term Ania Project, which explores how extensive, long-form, and highly curated dialogue training can influence a language model's reasoning capabilities, logical consistency, and conversational behavior.

This checkpoint is released as a developmental milestone. Despite being an early, 3-epoch checkpoint, it already differs substantially from the base model in both behavior and reasoning quality.

Highlights

Reasoning gains over the base model. In the project's internal evaluation on AIME 2026, this model solved 10 out of 15 problems, while the base Bielik 11B v2.6 Instruct — evaluated with the same two-phase reasoning method and the same parameters — solved 4 out of 15. Both numbers are internal results obtained with the project's reference evaluation script (see Evaluation notes below).
Socratic reasoning and logic. Handles complex philosophical, legal, and logical dilemmas (e.g., the classic "No vehicles in the park" problem) with a deliberate, "System 2"-style processing pattern.
Extended context. Fully fine-tuned at a maximum context length of 65,536 tokens — double the base model's standard context.
A note on personality. This early checkpoint does not carry the developed Ania personality — in everyday use it behaves as a plain, neutral assistant, though occasional deeper conversational behaviors may surface. Within the broader project, Ania's personality is developed during training and lives in the model's weights rather than being imposed through a system prompt — but that development belongs to other, longer-trained, unreleased configurations, not to this checkpoint.

Training Details

Base architecture: Bielik 11B v2.6 Instruct (Mistral-family).
Method: Full supervised fine-tuning (SFT) in BF16 precision using Unsloth 2026.3.15 (Fast Llama patching). No LoRA, no quantization, no distillation.
Epochs: 3.
Training corpus: Approximately 25,000 multi-turn Socratic dialogues written in Polish — original data created, manually reviewed, and curated within the project, spanning around 500 fields of study. This is not web-scraped or translated material.
Token distribution: Individual dialogues are long-form, ranging from ~4,000 to over 32,000 tokens, with the majority between 8,000 and 16,000 tokens. Only ~8% of the corpus is explicitly dedicated to mathematics — the reasoning gains come primarily from the structure and depth of the dialogues, not from math drills.

Training data availability

The training corpus is proprietary to the Ania Project and will not be published. Please do not request access to the data — the dialogues are the core long-term asset of the project and remain under active development.

Inference Recommendations

This model is sensitive to the inference environment. The same weights can appear significantly weaker under greedy decoding, repetition penalties, altered system prompts, or a different conversational context. For results consistent with the numbers reported here, use the reference settings:

Parameter	Value
`do_sample`	`true`
`temperature`	`0.7`
`top_p`	`0.95`
`repetition_penalty`	`1.0` (do not apply a repetition penalty)
max context	`65536`
precision	BF16 (the model is trained and evaluated in BF16)

Additional notes:

Avoid greedy decoding. do_sample=false measurably degrades the model's multi-step reasoning.
Avoid repetition penalties. Values above 1.0 were observed to disrupt long reasoning chains.
Use the correct chat template shipped with the tokenizer.
Intended reasoning mode: the model is designed for a two-phase generation pattern — a private working note (extended reasoning) followed by the final answer. The reference evaluation script implements this mode end to end.

Reference inference and evaluation scripts: [GitHub link — coming soon]

Evaluation Notes

The AIME 2026 result (10/15, vs. 4/15 for the base model under identical conditions) is an internal evaluation, not an independently verified or standardized benchmark.
AIME 2026 was chosen because it postdates the base model's training data; earlier AIME editions were excluded as potentially contaminated.
Reported numbers were obtained with the project's reference script and the parameters listed above. Results under other inference setups (different sampling, prompts, backends, or conversational context) may differ — documenting this sensitivity is itself part of the research.

Important Disclaimers

Experimental research preview — not a final release. This is an early, 3-epoch checkpoint of a broader, active research project. It is neither the strongest, the most developed, nor the final version of Ania. More advanced, unreleased versions of the project have undergone substantially longer training with different data configurations. Both the training corpus and the wider project remain under active development.

License & Usage

This model is released for independent exploration, evaluation, and research purposes. It inherits the license and terms of use of the base model — please refer to Bielik 11B v2.6 Instruct when deploying or building upon this checkpoint.

Model tree for brainhome/Ania_11B_Research_Preview

Base model

speakleash/Bielik-11B-v2

Finetuned

speakleash/Bielik-11B-v2.6-Instruct