Instructions to use sangtran12/eduflowai-gemma4b-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use sangtran12/eduflowai-gemma4b-gguf with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf sangtran12/eduflowai-gemma4b-gguf:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf sangtran12/eduflowai-gemma4b-gguf:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf sangtran12/eduflowai-gemma4b-gguf:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf sangtran12/eduflowai-gemma4b-gguf:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf sangtran12/eduflowai-gemma4b-gguf:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf sangtran12/eduflowai-gemma4b-gguf:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf sangtran12/eduflowai-gemma4b-gguf:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf sangtran12/eduflowai-gemma4b-gguf:Q4_K_M

Use Docker

docker model run hf.co/sangtran12/eduflowai-gemma4b-gguf:Q4_K_M

LM Studio
Jan

vLLM

How to use sangtran12/eduflowai-gemma4b-gguf with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "sangtran12/eduflowai-gemma4b-gguf"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sangtran12/eduflowai-gemma4b-gguf",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/sangtran12/eduflowai-gemma4b-gguf:Q4_K_M

Ollama
How to use sangtran12/eduflowai-gemma4b-gguf with Ollama:
```
ollama run hf.co/sangtran12/eduflowai-gemma4b-gguf:Q4_K_M
```
Unsloth Desktop
Docker Model Runner
How to use sangtran12/eduflowai-gemma4b-gguf with Docker Model Runner:
```
docker model run hf.co/sangtran12/eduflowai-gemma4b-gguf:Q4_K_M
```

Lemonade

How to use sangtran12/eduflowai-gemma4b-gguf with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull sangtran12/eduflowai-gemma4b-gguf:Q4_K_M

Run and chat with the model

lemonade run user.eduflowai-gemma4b-gguf-Q4_K_M

List all available models

lemonade list

Atomic Chat

🎓 EduFlowAI Gemma 3 4B GGUF (Vietnamese Exam & Quiz Generator)

Made with Unsloth

EduFlowAI Gemma 3 4B GGUF is a specialized instruction-tuned language model designed for Vietnamese educational assessment, automatic question extraction, and quiz generation for the EduFlow Multi-Tenant SaaS LMS.

It is quantized and packaged in GGUF format for direct, high-performance offline inference via Ollama, LM Studio, and llama.cpp on Apple Silicon (M1/M2/M3/M4), Windows, and Linux.

⚡ Quickstart with Ollama (Single Command)

You can run this model locally with zero setup using Ollama:

# Run optimal 4-bit quantized version (~2.4 GB, ~25-35 tokens/sec on Apple Silicon)
ollama run hf.co/sangtran12/eduflowai-gemma4b-gguf:Q4_K_M

# Or run high-precision 8-bit quantized version (~3.9 GB)
ollama run hf.co/sangtran12/eduflowai-gemma4b-gguf:Q8_0

📦 Available GGUF Quantizations

File	Size	RAM Required	Best Use Case
`gemma-3-4b-it.Q4_K_M.gguf`	2.4 GB	$\ge$ 4 GB	Recommended: Best balance of speed, low memory, and near-lossless JSON syntax validity.
`gemma-3-4b-it.Q8_0.gguf`	3.9 GB	$\ge$ 6 GB	High-precision evaluation, benchmark replication, academic verification.
`gemma-3-4b-it.BF16-mmproj.gguf`	812 MB	-	Multimodal vision-language projection asset from base Gemma 3.

🎯 Key Capabilities

VNHSGE Exam Parsing: Parses past exam papers (Đề thi Tốt nghiệp THPT Quốc gia) across STEM (Toán, Lý, Hóa, Sinh) and Social Sciences (Sử, Địa, GDCD, Tiếng Anh).
Strict Schema Adherence: Outputs strict JSON arrays directly consumable by EduFlow SaaS without manual post-processing:
- type: mcq (multiple choice), true_false, fill_blank, short_answer.
- content: Extracted question body.
- options: 4 standardized choices (A, B, C, D).
- correct_index: 0-indexed correct answer (0: A, 1: B, 2: C, 3: D).
- difficulty: easy, medium, hard.
- points: Fractional scoring on Vietnamese 10-point scale (e.g. 0.25).
LaTeX Formula Preservation: Mathematical and chemical notation are automatically formatted in KaTeX/LaTeX (e.g. $Al^{3+}$ , $H_2SO_4$ , $\int_0^1 f(x)dx$ ).

💻 Sample Usage & Prompting

Prompt:

<start_of_turn>user
Bạn là chuyên gia trích xuất đề thi THPT Quốc gia. Hãy chuyển đổi văn bản đề thi sau thành mảng JSON theo chuẩn EduFlow:

Câu 1: Kim loại nào sau đây có nhiệt độ nóng chảy cao nhất?
A. Na.    B. K.    C. Cu.    D. W.
Câu 2: Dung dịch nào sau đây làm quỳ tím chuyển sang màu đỏ?
A. HCl.   B. NaOH. C. NaCl.  D. Ba(OH)2.
<end_of_turn>
<start_of_turn>model

Response:

[
  {
    "type": "mcq",
    "content": "Câu 1: Kim loại nào sau đây có nhiệt độ nóng chảy cao nhất?",
    "options": ["Na.", "K.", "Cu.", "W."],
    "correct_index": 3,
    "difficulty": "medium",
    "points": 0.25,
    "subject": "chemistry"
  },
  {
    "type": "mcq",
    "content": "Câu 2: Dung dịch nào sau đây làm quỳ tím chuyển sang màu đỏ?",
    "options": ["HCl.", "NaOH.", "NaCl.", "Ba(OH)2."],
    "correct_index": 0,
    "difficulty": "medium",
    "points": 0.25,
    "subject": "chemistry"
  }
]

🏛️ Research Context (Master's Thesis)

Author: Trần Quang Sang (Student ID: 24MSE23205)
Program: Master of Software Engineering (MSE), FPT University
Supervisor: Dr. Nguyễn Xuân Sâm
Topic: Investigating trade-offs between Parameter-Efficient Fine-Tuning (QLoRA vs LoRA) and inference validity for Vietnamese Educational Systems.

Downloads last month: 155

GGUF

Model size

4B params

Architecture

gemma3

Hardware compatibility

4-bit

8-bit

Model tree for sangtran12/eduflowai-gemma4b-gguf

Base model

google/gemma-3-4b-pt

Finetuned

google/gemma-3-4b-it

Quantized

(487)

this model