DesaiJayM/Bharat-24B-Math-Code

Bharat 24B by Leo Labs

Strong math. Practical Python code.

Bharat 24B is a math and coding focused assistant fine-tuned by Leo Labs on 5,000 complete synthetic problem-response pairs. It builds on the 24-billion-parameter Mistral Small 3.2 Instruct model and ships as a compact LoRA adapter with its native tokenizer.

Math and code scorecard

Test	Correct	Score
GSM8K — numeric final-answer match, fixed 128-case subset	123 / 128	96.09%
HumanEval — original Python tests, fixed 64-case subset, greedy pass@1	51 / 64	79.69%

These math/code results cover 192 cases from a completed 222-case evaluation run. Each code problem received one generation attempt. Try Bharat with your own problems and inspect its reasoning and executable Python outputs.

Test method and exact scope

Evaluation used the saved step 625 adapter with an NF4 backbone, original native prompt IDs and greedy decoding. Math used published GSM8K numeric answers and MathVerify 0.9.0; code used the original HumanEval tests in an isolated process. The exact normalized training-question overlap check found zero matches for these frozen evaluation prompts.

The math number above is a format-tolerant secondary analysis. Its checker was chosen after an earlier answer-format artifact and declared before the remaining 100 responses were generated. The registered #### answer-format metric is 111/128 (86.72%), retained in evaluation-results.json. Six math responses reached the generation-token limit; none of these math/code cases was cut by the evaluation wall-time limit. Results are from fixed subsets, not full benchmark or leaderboard evaluations.

Training

Setting	Value
Complete problem-response pairs	5,000
Math / code balance	2,500 / 2,500
Assistant target tokens	17,502,357
Total native training tokens	18,171,052
Epochs / optimizer updates	1 / 625
QLoRA rank / alpha	16 / 32
Peak learning rate	0.00001
Examples per optimizer update	8
Target modules	Text attention and MLP

Training used assistant-only labels, complete response reasoning/content, and the original native fine-tuning format. All 5,000 examples were consumed exactly once. Vision parameters were frozen. The adapter and native tokenizer are byte-for-byte preserved from the evaluated checkpoint.

Run Bharat

This repository supplies the adapter and native tokenizer. Load it with the original base weights on a CUDA GPU. Install requirements.txt and run the included helper:

python -m pip install -r requirements.txt
python bharat_inference.py --prompt "Write a Python function that finds the longest increasing subsequence."

Use --revision RELEASE_COMMIT_SHA to pin a public release commit. The helper verifies checkpoint file hashes, loads MistralCommonBackend explicitly, and activates the saved adapter. --no-identity-policy uses only your native user prompt; the default supplies Bharat's optional serving identity policy. --raw-output also displays the full generated response.

Base and data attribution

Base model: Mistral Small 3.2 24B Instruct, by Mistral AI, revision 95a6d26c4bfb886c58daf9d3f7332c857cb27b43. Released under Apache 2.0; see LICENSE and NOTICE.

Question sources: NuminaMath-TIR, by Project Numina (Apache 2.0), and OpenCodeInstruct, by NVIDIA (CC BY 4.0). Only the selected questions were used; the original dataset answers were replaced with newly generated synthetic response targets. See DATA_ATTRIBUTION.md for source credits and license links.

Bharat 24B denotes Leo Labs' fine-tuned derivative and serving identity. The optional identity system policy supplies that name during ordinary serving. This release provides a model artifact and inference code; it does not send user prompts to a hosted inference API.