Instructions to use MKSHRESTHA/Gemma-3-270m_Newari_autocomplete with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use MKSHRESTHA/Gemma-3-270m_Newari_autocomplete with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("google/gemma-3-270m")
model = PeftModel.from_pretrained(base_model, "MKSHRESTHA/Gemma-3-270m_Newari_autocomplete")

Transformers

How to use MKSHRESTHA/Gemma-3-270m_Newari_autocomplete with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="MKSHRESTHA/Gemma-3-270m_Newari_autocomplete")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("MKSHRESTHA/Gemma-3-270m_Newari_autocomplete", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use MKSHRESTHA/Gemma-3-270m_Newari_autocomplete with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "MKSHRESTHA/Gemma-3-270m_Newari_autocomplete"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MKSHRESTHA/Gemma-3-270m_Newari_autocomplete",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/MKSHRESTHA/Gemma-3-270m_Newari_autocomplete

SGLang

How to use MKSHRESTHA/Gemma-3-270m_Newari_autocomplete with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "MKSHRESTHA/Gemma-3-270m_Newari_autocomplete" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MKSHRESTHA/Gemma-3-270m_Newari_autocomplete",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "MKSHRESTHA/Gemma-3-270m_Newari_autocomplete" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MKSHRESTHA/Gemma-3-270m_Newari_autocomplete",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use MKSHRESTHA/Gemma-3-270m_Newari_autocomplete with Docker Model Runner:
```
docker model run hf.co/MKSHRESTHA/Gemma-3-270m_Newari_autocomplete
```

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Gemma-3-270M-Newari-Autocomplete

Model Description

This is a fine-tuned version of Google's Gemma-3-270M base model, adapted for Newari (Nepal Bhasa) text autocomplete using Parameter-Efficient Fine-Tuning (PEFT) with LoRA. The model is specifically trained to continue and complete Newari language text sequences, making it suitable for applications requiring Newari language generation, autocomplete, and text completion tasks.

This project was partly inspired by the work of Himalaya AI Labs (publishing as HimalayaGPT on Hugging Face), a Nepal-based research lab building open-source Nepali foundation models. Their custom Devanagari/Nepali tokenizer (from himalaya-ai/himalayagpt-0.5b-it) was explored during early testing of this project, though it is not used in the final released model.

Developed by: Pragesh Shrestha Model type: Causal Language Model with LoRA adapters Language(s): Newari (Nepal Bhasa), English License: Gemma Terms of Use Finetuned from model: google/gemma-3-270m Base model: google/gemma-3-270m

Uses

Direct Use

This model is intended for Newari language text autocomplete and completion tasks. It can be used directly for:

Newari language sentence completion
Text generation in Newari
Autocomplete for Newari writing tools
Research on low-resource language modeling

Out-of-Scope Use

The model is not intended for production deployment without proper evaluation
Not suitable for ASR tasks (this is a text-only language model)
May not generalize well to domains significantly different from the training data
Should not be used for automated decision-making without human oversight

Bias, Risks, and Limitations

Language Limitation: This model is specifically tuned for Newari and may not perform well for other languages
Domain Limitation: Performance is limited to the training data distribution (ASR transcripts and book text)
Resource Constraints: As a 270M parameter model, it has inherent limitations compared to larger language models
Training Data Quality: The training data quality and representativeness reflect the source datasets
Cultural Sensitivity: Newari is a low-resource language with cultural significance; outputs should be reviewed for cultural appropriateness

Recommendations

Users should validate model outputs, especially for production applications. This model is most suitable for research and development rather than production deployment without thorough testing in the target use case context.

How to Get Started with the Model

Loading the LoRA Adapter

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel, PeftConfig
import torch

# Load the base model with the adapter
model_name = "google/gemma-3-270m"
adapter_path = "MKSHRESTHA/gemma3-270m-Newari-autocomplete"

# Load the tokenizer from base model (it's the same as base Gemma tokenizer)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Load the base model
base_model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Load the LoRA adapter
model = PeftModel.from_pretrained(base_model, adapter_path)

# Generate text
prompt = "म्हसीका"  # Example Newari text start
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
    **inputs,
    max_new_tokens=50,
    temperature=0.7,
    do_sample=True,
    pad_token_id=tokenizer.eos_token_id
)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Merged Model (Optional)

If you prefer a merged version:

# Merge adapter into base model
merged_model = model.merge_and_unload()

# Save the complete model
merged_model.save_pretrained("gemma3-270m-newari-merged")
tokenizer.save_pretrained("gemma3-270m-newari-merged")

Training Details

Training Data

Source: Newari text corpus compiled from two sources:

ASR transcripts extracted from ILPRL's Nwacha Muna Newari ASR Dataset (text column only)
Digitized text from an older Newari-language book

Dataset Format: JSONL with a "text" field containing Newari sentences Preprocessing: The dataset was formatted as causal LM training examples with Newari text sequences

Training Procedure

Framework: TRL (Transformer Reinforcement Learning) SFTTrainer Methodology: Parameter-Efficient Fine-Tuning (PEFT) using LoRA (Low-Rank Adaptation)

Preprocessing

Text tokenized using the base Gemma tokenizer
Maximum sequence length: 512 tokens
No special preprocessing beyond standard causal LM formatting
Note: A custom Nepali/Devanagari tokenizer from Himalaya AI Labs was tested during early experimentation but was not used in the final training run; the base Gemma tokenizer was used instead.

Training Hyperparameters

Base Model: google/gemma-3-270m
Training Method: LoRA Fine-tuning
LoRA Rank (r): 16
LoRA Alpha: 32
Target Modules: all-linear
LoRA Dropout: 0.05
Batch Size (per device): 4
Gradient Accumulation Steps: 4
Effective Batch Size: 16
Learning Rate: 2e-4
Optimizer: AdamW torch
Scheduler: Linear
Number of Epochs: 5
Precision: bfloat16 (bf16)
Max Sequence Length: 512
Dataset Format: JSONL with 'text' field
Framework Versions: PEFT 0.19.1

Speeds, Sizes, Times

Training Regime: bf16 mixed precision
Output: LoRA adapter weights only (not merged with base model)
Total Parameters Trained: ~21M (LoRA adapter parameters only)
Checkpoint Size: Adapter-only saves (significantly smaller than full model)

Technical Specifications

Model Architecture and Objective

Architecture: Gemma-3-270M (transformer-based decoder-only)
Objective: Causal Language Modeling (next-token prediction)
Fine-tuning Method: LoRA adapters with rank 16
Parameters (Base): 270M
Parameters (Trainable): ~21M (LoRA adapters)

Compute Infrastructure

Hardware: NVIDIA RTX 5060 Ti, 32GB RAM

Software:

Python 3.12
Transformers library
PEFT 0.19.1
TRL (SFTTrainer)
PyTorch
bfloat16 support

Citation

If you use this model, please cite both the original Gemma model and this fine-tuned version:

Gemma Model:

@misc{gemma2024,
  title={Gemma: Open Models Based on Gemini Research and Technology},
  author={Google},
  year={2024},
  url={https://ai.google.dev/gemma}
}

Base Dataset:

ILPRL, Kathmandu University - Nwacha Muna Newari ASR Dataset
https://huggingface.co/datasets/ilprl-docse/Nwacha_Muna_A_Newari_ASR_Dataset

This Model:

@misc{shrestha2025gemma3newari,
  title={Gemma-3-270M-Newari-Autocomplete: LoRA Fine-tuned Newari Language Model},
  author={Pragesh Shrestha},
  year={2025},
  url={https://huggingface.co/MKSHRESTHA/Gemma-3-270m_Newari_autocomplete}
}

Model Card Authors

Pragesh Shrestha

Framework Versions

PEFT 0.19.1
Transformers: [Your version]
TRL: [Your version]
PyTorch: [Your version]

Model Card Contact

For questions, issues, or collaboration inquiries regarding this model, please reach out via:

Hugging Face: MKSHRESTHA
GitHub: prageshshrestha
Kaggle: prageshshrestha

More Information

Training Configuration File (For Reference)

If you want to reproduce this fine-tuning, here is the key LoRA configuration used:

from peft import LoraConfig

lora_config = LoraConfig(
    r=16,
    lora_alpha=32,
    target_modules="all-linear",
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM"
)

Recommended Generation Parameters

For best results with Newari text generation:

generation_config = {
    "max_new_tokens": 128,
    "temperature": 0.7,
    "top_p": 0.9,
    "do_sample": True,
    "pad_token_id": tokenizer.eos_token_id,
    "repetition_penalty": 1.1
}

Known Issues and Limitations

Tokenization: The base Gemma tokenizer may not optimally handle all Newari characters and compounds, which could affect generation quality
Dialect Variation: Newari has multiple dialects; this model may perform better on some than others
Code-switching: The model may sometimes produce mixed Newari-English output
Length: The 512 token limit may truncate longer texts

Glossary

Newari (Nepal Bhasa): A Sino-Tibetan language spoken primarily in the Kathmandu Valley of Nepal
LoRA (Low-Rank Adaptation): Parameter-efficient fine-tuning method that trains low-rank matrices to adapt pretrained models
PEFT (Parameter-Efficient Fine-Tuning): Approach that fine-tunes a small subset of parameters while keeping the base model frozen
Causal Language Model: Model trained to predict the next token given previous tokens
ASR (Automatic Speech Recognition): Technology that converts spoken language to text

Acknowledgements

Google for releasing the Gemma-3-270M base model
ILPRL, Kathmandu University for making the Nwacha Muna Newari ASR Dataset available
The Hugging Face and PEFT teams for providing the fine-tuning infrastructure
Himalaya AI Labs (HimalayaGPT) — a Nepal-based research lab building open-source Nepali foundation models, whose work and custom Nepali/Devanagari tokenizer (himalaya-ai/himalayagpt-0.5b-it) served as inspiration and an early testing reference for this project

Disclaimer

This model is provided "as is" for research and educational purposes. While efforts have been made to ensure quality, no guarantees are made regarding accuracy, bias, or suitability for any specific use case. Users are responsible for evaluating the model's outputs and ensuring compliance with applicable laws and regulations, particularly regarding the use of low-resource languages and cultural content.

Version: 1.0 Last Updated: 2026-06-26

Downloads last month: -

Model tree for MKSHRESTHA/Gemma-3-270m_Newari_autocomplete

Base model

google/gemma-3-270m

Adapter

(58)

this model