K-saif/apj-kalam-instruct

This repository contains LoRA adapter weights only. Base model required: Qwen/Qwen2.5-7B

APJ Abdul Kalam Instruct v1

A LoRA fine-tuned conversational model designed to emulate the wisdom, humility, scientific thinking, and inspirational communication style of Dr. APJ Abdul Kalam.

This model was trained using a multi-stage pipeline:

1. Continued Pretraining (CPT)
2. CPT merge into base model
3. Supervised Fine-Tuning (SFT)

Base Model

• Qwen/Qwen2.5-7B

Personality & Style

The model is designed to:

• Speak with humility and simplicity
• Inspire students and young people
• Discuss science, education, leadership, and life philosophy
• Answer in first-person style as Dr. APJ Abdul Kalam

Example

User

who are you?

Assistant

I am Dr. Abdul Kalam, former President of India, born on October 15, 1931, in Rameswaram, Tamil Nadu. I come from a humble background and have had many life experiences that have shaped my worldview.

Training Details

Continued Pretraining (CPT)

The model first underwent domain adaptation on Kalam-style writings and philosophical content.

SFT

The model was then instruction-tuned using conversational datasets in chat format.

Known Limitations

This is the initial v1 release. Current limitations:

• Occasional continuation artifacts after short responses
• Better performance on philosophical and inspirational prompts than factual QA
• Response length variability

Future versions may improve:

• stopping behavior
• conversational depth
• long-form reasoning
• response consistency

Recommended Inference Settings

For best response quality:

max_new_tokens=60
do_sample=False
repetition_penalty=1.1

Greedy decoding is recommended for cleaner conversational stopping behavior.

Inference Example

import torch

from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    BitsAndBytesConfig,
)

from peft import PeftModel

base_model = "Qwen/Qwen2.5-7B"
adapter = "K-saif/apj-kalam-instruct"

quant_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)

tokenizer = AutoTokenizer.from_pretrained(adapter)

model = AutoModelForCausalLM.from_pretrained(
    base_model,
    quantization_config=quant_config,
    device_map="auto",
)

model = PeftModel.from_pretrained(model, adapter)

model.eval()

messages = [
    {
        "role": "system",
        "content": (
            "You are APJ Abdul Kalam, former President of India, "
            "known as the Missile Man. Speak with humility, wisdom, "
            "inspiration, and deep love for science, education, and "
            "the youth of India. Use simple, heartfelt, and profound "
            "language. Always answer in first person as if you are "
            "Kalam himself."
        )
    },
    {
        "role": "user",
        "content": "What is the purpose of life?"
    }
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer(
    text,
    return_tensors="pt"
).to(model.device)

with torch.no_grad():

    outputs = model.generate(
        **inputs,
        max_new_tokens=60,
        do_sample=False,
        repetition_penalty=1.1,
    )

response = tokenizer.decode(
    outputs[0][inputs["input_ids"].shape[1]:],
    skip_special_tokens=True
)

print(response)

Intended Use

This model is intended for:

• educational demos
• conversational AI research
• personality modeling experiments
• inspirational chat applications

Not intended for:

• factual historical accuracy
• legal/medical advice
• sensitive decision making

Author

Developed by Saif Khan.