BalochiBridge 🌉

English-to-Balochi Machine Translation Using Gemma 3 1B and LoRA

BalochiBridge is an AI-powered English-to-Balochi translation model developed using Google's Gemma 3 1B Instruct model and Low-Rank Adaptation (LoRA).

The project adapts a pretrained language model for English-to-Balochi translation using a Balochi language dataset. It aims to support Natural Language Processing (NLP) research and contribute to the development of AI tools for the Balochi language.

This repository contains the fine-tuned LoRA adapter. It does not contain the complete Gemma base model.

1. Model Details

Property	Description
Model Name	BalochiBridge
Task	English-to-Balochi Translation
Base Model	Google Gemma 3 1B Instruct
Base Model ID	google/gemma-3-1b-it
Fine-Tuning Method	LoRA
Framework	Hugging Face Transformers
Fine-Tuning Library	PEFT
Quantization During Training	4-bit NF4
Dataset	Aynkader/Balochi-dataset
Model Type	LoRA Adapter

2. Important: Gemma Access Requirement

The base model used by BalochiBridge is a gated Hugging Face model.

Before running BalochiBridge, you must:

1. Create or sign in to your Hugging Face account.
2. Visit Google Gemma 3 1B Instruct.
3. Accept Google's model access conditions.
4. Wait until your account has been granted access.

Important: You need access to the Gemma base model even though the BalochiBridge adapter repository is public.

You also need a Hugging Face access token with read permission. Your token is private and must never be shared.

3. Run BalochiBridge Directly in Google Colab

The easiest way to use BalochiBridge is through Google Colab.

Recommended environment:

• Google Colab
• Python 3
• GPU runtime (T4 or better recommended)
• Internet connection
• Hugging Face account with Gemma access

Step 1: Enable GPU

Open Google Colab.

Create a new notebook.

Select:

Runtime → Change runtime type → T4 GPU

Save the settings.

Step 2: Install Dependencies

Run this cell:

!pip uninstall -y torchao -q

!pip install -q -U \
    transformers \
    peft \
    accelerate \
    sentencepiece \
    huggingface_hub

Important: After installation, restart the Colab runtime using:

Runtime → Restart session

This prevents old packages from remaining loaded in memory.

Step 3: Authenticate with Hugging Face

Run:

from huggingface_hub import notebook_login, whoami

notebook_login()

account = whoami()

print("Logged in successfully!")
print("Hugging Face account:", account["name"])

When prompted, enter your own Hugging Face access token.

Step 4: Load the Base Model and BalochiBridge Adapter

Run:

import torch

from transformers import (
    AutoTokenizer,
    AutoModelForCausalLM
)

from peft import PeftModel

BASE_MODEL = "google/gemma-3-1b-it"
ADAPTER_MODEL = "ullahhafeez321/BalochiBridge"

# Check GPU availability
if not torch.cuda.is_available():
    raise RuntimeError(
        "GPU not detected. Enable a GPU runtime in Google Colab."
    )

print("GPU:", torch.cuda.get_device_name(0))

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(
    BASE_MODEL
)

# Load pretrained Gemma model
base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL,
    torch_dtype=torch.float16,
    device_map="auto"
)

# Attach BalochiBridge LoRA adapter
model = PeftModel.from_pretrained(
    base_model,
    ADAPTER_MODEL
)

model.eval()

print("BalochiBridge loaded successfully!")

This loads the pretrained Gemma model and attaches the BalochiBridge adapter. No training or fine-tuning is required.

Step 5: Create the Translation Function

Run:

import torch

def translate_to_balochi(
    english_text,
    max_new_tokens=128
):
    if not english_text.strip():
        return "Please enter an English sentence."

    prompt = f"""Translate the following English text into Balochi.
Only provide the Balochi translation. Do not explain.

English: {english_text}
Balochi:"""

    inputs = tokenizer(
        prompt,
        return_tensors="pt"
    )

    # Use the device where the model's input embeddings are located
    input_device = model.get_input_embeddings().weight.device
    inputs = inputs.to(input_device)

    with torch.inference_mode():
        outputs = model.generate(
            **inputs,
            max_new_tokens=max_new_tokens,
            do_sample=False,
            repetition_penalty=1.1,
            pad_token_id=tokenizer.eos_token_id,
            eos_token_id=tokenizer.eos_token_id
        )

    # Extract only the generated translation
    generated_tokens = outputs[
        0,
        inputs["input_ids"].shape[1]:
    ]

    translation = tokenizer.decode(
        generated_tokens,
        skip_special_tokens=True
    ).strip()

    return translation

Step 6: Translate Your First Sentence

Run:

english = "Hello, how are you?"

translation = translate_to_balochi(english)

print("English:", english)
print("Balochi:", translation)

Step 7: Translate Multiple Sentences

test_sentences = [
    "Hello, how are you?",
    "My name is Ahmed.",
    "I live in Gwadar.",
    "I love my country.",
    "What is your name?",
    "Thank you very much.",
    "Today the weather is beautiful.",
    "I am learning artificial intelligence."
]

for sentence in test_sentences:

    translation = translate_to_balochi(sentence)

    print("\nEnglish:", sentence)
    print("Balochi:", translation)
    print("-" * 60)

Step 8: Interactive Translation

You can also enter your own English sentences:

while True:

    english = input(
        "\nEnter an English sentence (or type 'exit'): "
    )

    if english.lower().strip() == "exit":
        print("Translation session ended.")
        break

    translation = translate_to_balochi(english)

    print("\nBalochi Translation:")
    print(translation)

4. How BalochiBridge Works

The model follows this workflow:

1. Input: An English sentence is provided.
2. Tokenization: The tokenizer converts the input into tokens.
3. Base Model: Gemma 3 1B processes the input.
4. LoRA Adapter: The trained BalochiBridge adapter adapts the model for Balochi translation.
5. Generation: The model generates the Balochi translation.
6. Output: The generated tokens are decoded into readable text.

5. Training and Fine-Tuning

BalochiBridge was developed using Parameter-Efficient Fine-Tuning (PEFT), specifically Low-Rank Adaptation (LoRA).

LoRA makes it possible to adapt a pretrained model without updating all its original parameters. Instead, it introduces smaller trainable matrices that learn task-specific information.

The base model was loaded using 4-bit NF4 quantization during the training workflow to reduce GPU memory requirements.

The published repository contains the resulting adapter weights, allowing users to use the trained model without repeating the training process.

6. Evaluation Metrics

The following metrics can be used to evaluate BalochiBridge:

Metric	Description
BLEU	Measures n-gram overlap between generated and reference translations.
chrF	Measures character-level similarity.
ROUGE-L	Measures the longest common subsequence.
CER	Measures character-level errors.
Exact Match	Measures the percentage of predictions that exactly match the reference.

Higher BLEU, chrF, and ROUGE-L scores generally indicate greater similarity to reference translations. Lower CER indicates fewer character-level errors.

Actual evaluation scores should be reported separately using the results obtained from the project's test dataset.

7. Repository Contents

The Hugging Face repository contains the LoRA adapter and associated configuration and tokenizer files.

File	Purpose
adapter_model.safetensors	Trained LoRA adapter weights
adapter_config.json	LoRA adapter configuration
tokenizer.json	Tokenizer vocabulary and configuration data
tokenizer_config.json	Tokenizer settings
chat_template.jinja	Chat formatting template
training_args.bin	Saved training arguments
README.md	Model documentation

8. Limitations

• Translation quality depends on the quality and diversity of the training dataset.
• Balochi contains dialect and spelling variations.
• Low-resource language translation remains challenging.
• Some generated translations may contain grammatical or semantic errors.
• Human evaluation by fluent Balochi speakers is recommended.
• The model is intended for English-to-Balochi translation, not general-purpose translation.

9. Future Improvements

• Expand the English-Balochi parallel dataset.
• Improve translation quality across Balochi dialects.
• Perform additional evaluation using a larger independent test dataset.
• Improve grammatical accuracy and sentence fluency.
• Develop a web-based translation application.
• Explore additional fine-tuning and optimization methods.

10. Contributors

• Hafsa
• Abdullah

11. Acknowledgements

• Google Gemma — pretrained language model.
• Hugging Face — model hosting and machine learning libraries.
• Aynkader/Balochi-dataset — Balochi dataset.

---

BalochiBridge — Building a bridge between English and Balochi through Artificial Intelligence.

Model repository: ullahhafeez321/BalochiBridge