arunmcops/Param-1-7B-GGUF

Param-1 7B MoE - GGUF

General Purpose - Multilingual MoE  |  BharatGenAI  |  Hindi + English

Param-1-7B is BharatGenAI's Mixture-of-Experts multilingual model with 7 billion total parameters and 64 specialized experts (top-8 routing, ~1.04B active per token). Pretrained on 4 trillion tokens covering English, Hindi, and 14 additional Indian languages.

---

Available Quantizations

Quantization	File	Size	Description
F16	Param-1-7B-F16.gguf	~12.4 GB	Full float 16, near-lossless, GPU recommended
Q2_K	Param-1-7B-Q2_K.gguf	~2.4 GB	2-bit, smallest, significant quality loss
Q3_K_M	Param-1-7B-Q3_K_M.gguf	~3.0 GB	3-bit medium, moderate quality
Q4_K_M	Param-1-7B-Q4_K_M.gguf	~3.8 GB	4-bit medium - recommended for most users
Q5_K_M	Param-1-7B-Q5_K_M.gguf	~4.4 GB	5-bit medium, great quality
Q6_K	Param-1-7B-Q6_K.gguf	~5.1 GB	6-bit, excellent quality
Q8_0	Param-1-7B-Q8_0.gguf	~6.6 GB	8-bit, best quality/size balance

Not sure which to pick?

• Q4_K_M for everyday use on CPU/GPU
• Q8_0 for best quality on GPU with >= 6 GB VRAM
• F16/BF16 for full precision research use

---

Use Cases

• Multilingual text generation across 16 Indian languages
• Hindi and English conversational AI
• Cross-lingual knowledge transfer
• Region-specific NLP applications
• Foundation for multilingual fine-tuning

---

Model Details

Property	Value
Architecture	Mixture of Experts (Mixtral-style, Decoder-only Transformer)
Total Parameters	~7B
Active Parameters	~1.04B per token
Hidden Size	1536
Expert FFN Size	512 per expert
Layers	40
Attention Heads	16 (4 KV heads, GQA)
Head Dim	96
Experts	64 total, 8 active per token (top-8 softmax routing)
Context Length	4096 tokens
Activation	SiLU
Positional Encoding	RoPE (theta=1000)
Precision	BF16 (original)
Vocab Size	128,000
License	BharatGen Non-Commercial

Supported Languages

English, Hindi + 14 Indian languages: Assamese, Bengali, Gujarati, Kannada, Maithili, Malayalam, Marathi, Nepali, Oriya, Punjabi, Sanskrit, Sindhi, Tamil, Telugu.

Training

Pretrained from scratch on 4 trillion tokens with two phases:

Phase 1 (2T tokens):

Data Category	Tokens
English + Hindi	~500B
Code + Math	~500B
14 Indian Languages	~1.0T

Phase 2 (2T tokens):

Data Category	Tokens
General Knowledge	~470B
Technical (Research, STEM, India-focused)	~270B
Education, Exams, Math, Code	~330B
Indic Multilingual Expansion (16 langs)	~920B
Alignment & Stabilization	~50B

Data mixing curated with CLIMB (Clustering-based Iterative Data Mixture Bootstrapping) from NVIDIA.

• Framework: NVIDIA NeMo
• Hardware: NVIDIA H100 GPUs (Yotta Shakti Cloud)

Architecture Notes

This model uses sparse Mixture of Experts with token-level routing. Each input token is dynamically routed to 8 of 64 specialized expert sub-networks. This gives the model 7B total knowledge capacity while only activating ~1.04B parameters per token, enabling efficient inference on consumer hardware.

Note: This is a pretrained base model. For best results in chat or instruction-following, fine-tune on your target task.

Original Model

Source: bharatgenai/Param-1-7B

---

Quick Start

llama.cpp

# Windows
winget install llama.cpp

# Run directly from Hugging Face
llama-cli -hf arunmcops/Param-1-7B-GGUF:Param-1-7B-Q4_K_M.gguf

# OpenAI-compatible server
llama-server -hf arunmcops/Param-1-7B-GGUF:Param-1-7B-Q4_K_M.gguf --port 8080

Ollama

ollama run hf.co/arunmcops/Param-1-7B-GGUF:Param-1-7B-Q4_K_M.gguf

LM Studio / Jan

Search arunmcops/Param-1-7B-GGUF in the model browser - GGUF files appear automatically.

Python (llama-cpp-python)

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="arunmcops/Param-1-7B-GGUF",
    filename="Param-1-7B-Q4_K_M.gguf",
)

response = llm.create_chat_completion(
    messages=[{"role": "user", "content": "Namaste! Aap kaun hain?"}]
)
print(response["choices"][0]["message"]["content"])

---

About BharatGenAI

BharatGenAI builds open-source language models for India - bilingual (Hindi + English) and domain-specialized for agriculture, healthcare, legal, and general-purpose use.

• GitHub: BharatGenAI
• HuggingFace: arunmcops

---

License

Apache 2.0 - free for personal, research, and commercial use.