Kaveri H-GRPO 0.5B INT8

Kaveri H-GRPO 0.5B INT8 is an INT8-quantized release of dharun2049/kaveri-hgrpo-0.5b.

The original Kaveri H-GRPO model was developed as an experimental compact reasoning and coding model using Hypergraph Group Relative Policy Optimization (H-GRPO).

This release uses bitsandbytes LLM.int8() quantization to reduce the memory and storage requirements of the model while retaining higher-precision computation for numerically sensitive components.

Model Details

Property	Value
Model	Kaveri H-GRPO 0.5B INT8
Parent model	dharun2049/kaveri-hgrpo-0.5b
Base architecture	Qwen2 0.5B
Approximate parameters	0.5B
Quantization	INT8
Quantization backend	bitsandbytes
Quantization method	LLM.int8()
Framework	Hugging Face Transformers
Primary use	Reasoning, coding and experimentation
Saved checkpoint size	~613 MB

What is H-GRPO?

H-GRPO stands for Hypergraph Group Relative Policy Optimization.

The method extends group-relative reinforcement learning by representing relationships between generated solutions as a hypergraph.

Instead of treating every candidate response only as an independent sample, H-GRPO can model relationships between multiple candidate solutions using features such as:

• strategy similarity
• code similarity
• test-case behavior
• structural relationships between solutions
• group-relative reward signals

The objective is to encourage useful reasoning structures and solution diversity while maintaining the relative-reward advantages of GRPO.

Kaveri H-GRPO was trained as an experimental investigation into whether these structured relationships can improve reasoning capability in very small language models.

INT8 Quantization

This checkpoint was produced using Hugging Face Transformers with bitsandbytes:

from transformers import AutoModelForCausalLM, BitsAndBytesConfig

quant_config = BitsAndBytesConfig(
    load_in_8bit=True,
    llm_int8_threshold=6.0,
    llm_int8_skip_modules=["lm_head"],
)

model = AutoModelForCausalLM.from_pretrained(
    "dharun2049/kaveri-hgrpo-0.5b",
    quantization_config=quant_config,
    device_map="auto",
)

The majority of supported linear layers are executed using the bitsandbytes Linear8bitLt implementation.

LLM.int8() is not equivalent to naively casting every model tensor to torch.int8. Numerically sensitive operations and outlier features may continue to use higher-precision computation.

Size

The serialized INT8 checkpoint is approximately:

613 MB

The measured in-memory model footprint during the quantization run was approximately:

601 MB

This makes the INT8 release considerably smaller than the original higher-precision checkpoint.

Usage

Install the required packages:

pip install -U transformers accelerate bitsandbytes

Load the model:

import torch

from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_ID = "dharun2049/kaveri-hgrpo-0.5b-int8"

tokenizer = AutoTokenizer.from_pretrained(
    MODEL_ID,
    trust_remote_code=True,
)

model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    device_map="auto",
    dtype=torch.float16,
    trust_remote_code=True,
)

model.eval()

Generate a response:

messages = [
    {
        "role": "user",
        "content": "Write a Python function to solve the Two Sum problem."
    }
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
)

device = next(model.parameters()).device

inputs = {
    key: value.to(device)
    for key, value in inputs.items()
}

with torch.inference_mode():

    output = model.generate(
        **inputs,
        max_new_tokens=256,
        do_sample=False,
        pad_token_id=tokenizer.eos_token_id,
    )

generated = output[
    0,
    inputs["input_ids"].shape[1]:
]

response = tokenizer.decode(
    generated,
    skip_special_tokens=True,
)

print(response)

Intended Uses

Kaveri H-GRPO 0.5B INT8 is primarily intended for:

• lightweight local inference
• coding experiments
• algorithmic problem solving
• reasoning research
• reinforcement-learning research
• quantization experiments
• agent and tool-use research
• educational experimentation
• deployment on systems where memory is constrained

The model is particularly useful for studying how much reasoning capability can be retained in sub-billion-parameter models.

Benchmarks

The parent Kaveri H-GRPO 0.5B checkpoint has been evaluated separately.

The INT8 checkpoint has not yet been fully re-evaluated across the complete benchmark suite.

Because quantization can slightly alter model outputs, benchmark numbers from the original model should not automatically be treated as results for this INT8 checkpoint.

Recommended post-quantization evaluations include:

Benchmark	Purpose
MMLU	General knowledge and reasoning
HumanEval	Code generation
MBPP / MBPP+	Python programming
LiveCodeBench	Competitive coding
GSM8K	Mathematical reasoning
ARC	Scientific and logical reasoning

Post-quantization results will be added once evaluation is completed.

Quantization Validation

During conversion, the model successfully loaded using INT8 quantization and produced a serialized checkpoint.

The resulting checkpoint size was approximately 613.16 MB, with a measured memory footprint of approximately 601.04 MB.

bitsandbytes may display messages similar to:

MatMul8bitLt: inputs will be cast from torch.bfloat16 to float16 during quantization

These are warnings associated with the numerical precision used by MatMul8bitLt and do not by themselves indicate that INT8 quantization failed.

Limitations

Kaveri H-GRPO is a small experimental language model.

Users should expect limitations including:

• hallucinated information
• incorrect reasoning
• incorrect or uncompilable code
• limited long-context reasoning
• weaker factual knowledge than much larger models
• sensitivity to prompt formatting
• potential degradation from quantization
• inconsistent performance on difficult competitive-programming problems

Generated code should be reviewed and tested before use.

The model should not be relied upon as an authoritative source for medical, legal, financial, safety-critical or other high-stakes decisions.

Research Status

This model is an experimental research release.

Kaveri is part of an effort to investigate how reinforcement-learning methods, structured reward relationships, distillation, inference optimization and quantization can improve the capability-to-size ratio of compact language models.

Parent Model

Full-precision / higher-precision Kaveri H-GRPO:

dharun2049/kaveri-hgrpo-0.5b

Author

Dharun Narayan V J

Hugging Face: dharun2049

Citation

If you use Kaveri H-GRPO in research, experiments or derivative work, please reference the model repository.

@misc{kaveri_hgrpo_2026,
  author = {Dharun Narayan V J},
  title = {Kaveri H-GRPO 0.5B},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/dharun2049/kaveri-hgrpo-0.5b}}
}

Disclaimer

This checkpoint is provided for research, experimentation and development.

The INT8 version is a quantized derivative of Kaveri H-GRPO 0.5B. Performance characteristics may differ from the parent checkpoint, and users should independently evaluate the model for their intended use case.