🚀 NexAI-v1 (7B Instruct - Q4_K_M GGUF)

Official repository of NexAI-v1, an enterprise-grade instruction-following model fine-tuned using QLoRA (SFT) on Alibaba's robust Qwen2.5-7B-Instruct architecture, and quantized into high-performance Q4_K_M GGUF format.

NexAI-v1 is engineered specifically for fast, localized CPU inference, retrieval-augmented generation (RAG) tasks, and secure, privacy-preserving local assistant applications.

---

📊 Model Summary

Attribute	Details
Developed by	Anoop Singh
Model Type	Causal Language Model (Transformer-based)
Base Architecture	Qwen2.5 (Dense)
Parameter Count	7.61 Billion
Quantization Format	GGUF (Q4_K_M - 4.91 Bits Per Weight)
File Size	4.46 GB
Supported Languages	English, Hindi (Hinglish supported natively)
Context Length	32,768 Tokens
License	Apache 2.0

---

🌟 Key Capabilities & Features

• 🧠 Instruction Following: Fine-tuned to respond precisely to system prompts, system instructions, and multi-turn chat templates.
• 🔍 RAG Optimized: Enhanced comprehension for feeding external context documents (PDFs, text, web scrapes) without losing tracking accuracy.
• 🤖 Stealth & Safety: Integrated system prompts designed to prevent bot detection and bypass standard security filter constraints.
• ⚡ Ultra-low Resource Execution: Fits comfortably within 8GB System RAM, executing at high token-generation speeds on consumer-grade laptops.

---

🛠️ Installation & Quickstart

1. Python Integration (via llama-cpp-python)

For programmatic use cases, install the Python binding:

pip install llama-cpp-python

Initialize and query the model directly from Hugging Face:

from llama_cpp import Llama

# Download and load the model locally
llm = Llama.from_pretrained(
    repo_id="Anoopsingh53/nexai-v1",
    filename="nexai-v1-Q4_K_M.gguf",
    n_ctx=2048,
    n_threads=4
)

# ChatML Format Prompt
prompt = '<|im_start|>system\n' \
         'You are NexAI, a helpful, intelligent assistant.\n' \
         '<|im_end|>\n' \
         '<|im_start|>user\n' \
         'Write a python function to check if a number is prime.\n' \
         '<|im_end|>\n' \
         '<|im_start|>assistant\n'

response = llm(
    prompt,
    max_tokens=256,
    stop=["<|im_end|>"],
    echo=False
)

print(response['choices'][0]['text'])

2. Desktop deployment (via llama.cpp CLI)

Download the .gguf file manually and run:

./llama-cli \
  -m nexai-v1-Q4_K_M.gguf \
  -p "<|im_start|>system\nYou are NexAI, a helpful assistant.<|im_end|>\n<|im_start|>user\nHello!\n<|im_end|><|im_start|>assistant\n" \
  -n 128 \
  -c 2048

---

⚙️ Training Details (QLoRA)

NexAI-v1 was trained using parameter-efficient fine-tuning (PEFT) on Google Colab hardware.

• Optimizer: AdamW (8-bit)
• Learning Rate: 2e-4
• Precision: mixed 16-bit / 4-bit SFT
• LoRA Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• LoRA Rank (R): 64 | LoRA Alpha: 16
• Training Epochs: 1 Epoch over specialized multi-turn data

---

⚠️ Intended Use & Limitations

• Intended Use: Personal assistance, local coding assistant, offline document RAG, and educational demonstrations.
• Limitations: Like all language models, NexAI-v1 may occasionally hallucinate or generate inaccurate facts. It is not intended for mission-critical medical, financial, or legal advice without human-in-the-loop validation.
• Biases: The model's outputs are heavily influenced by the pre-training data and instructions. Ensure proper filtering if deploying in customer-facing public products.

---

📄 License & Terms

This model is released under the Apache 2.0 License. Qwen2.5 base weights are governed by Alibaba's original terms of use.