Instructions to use sarathrkrishna/python_coding_assistant with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use sarathrkrishna/python_coding_assistant with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf sarathrkrishna/python_coding_assistant:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf sarathrkrishna/python_coding_assistant:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf sarathrkrishna/python_coding_assistant:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf sarathrkrishna/python_coding_assistant:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf sarathrkrishna/python_coding_assistant:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf sarathrkrishna/python_coding_assistant:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf sarathrkrishna/python_coding_assistant:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf sarathrkrishna/python_coding_assistant:Q4_K_M

Use Docker

docker model run hf.co/sarathrkrishna/python_coding_assistant:Q4_K_M

LM Studio
Jan
Ollama
How to use sarathrkrishna/python_coding_assistant with Ollama:
```
ollama run hf.co/sarathrkrishna/python_coding_assistant:Q4_K_M
```
Unsloth Desktop

How to use sarathrkrishna/python_coding_assistant with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf sarathrkrishna/python_coding_assistant:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "sarathrkrishna/python_coding_assistant:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use sarathrkrishna/python_coding_assistant with Docker Model Runner:
```
docker model run hf.co/sarathrkrishna/python_coding_assistant:Q4_K_M
```

Lemonade

How to use sarathrkrishna/python_coding_assistant with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull sarathrkrishna/python_coding_assistant:Q4_K_M

Run and chat with the model

lemonade run user.python_coding_assistant-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use sarathrkrishna/python_coding_assistant with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf sarathrkrishna/python_coding_assistant:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default sarathrkrishna/python_coding_assistant:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use sarathrkrishna/python_coding_assistant with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf sarathrkrishna/python_coding_assistant:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "sarathrkrishna/python_coding_assistant:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Model Specifications

Architecture

Property	Value
Base Model	IBM Granite 4.1
Architecture Type	Decoder-Only Transformer
Transformer Layers	40
Hidden Size	4096
Attention Heads	32
KV Heads (GQA)	8
Head Dimension	128
Intermediate Size	12800
Vocabulary Size	100,352
Context Length	131,072 Tokens
Activation Function	SiLU
RoPE Theta	10,000,000

Fine-Tuning Statistics

Metric	Value
Fine-Tuning Method	LoRA
Trainable Parameters	98,959,360
Total Parameters	4,494,921,728
Trainable Percentage	2.20%
Base Parameters Frozen	97.80%
Training Framework	Unsloth
Optimizer	AdamW 8-bit

Quantization Details

Property	Value
Output Format	GGUF
Quantization Method	Q4_K_M
Quantization Type	K-Quant Medium
Deployment Size	~5 GB
Runtime Engine	Ollama / llama.cpp

Memory Analysis

KV Cache Formula

KV Cache Per Token:

KV Cache = 2 × Layers × KV Heads × Head Dimension × 2 Bytes

Calculation:

2 × 40 × 8 × 128 × 2

= 163,840 Bytes

≈ 160 KB per Token

Estimated Runtime Memory Usage

Context Length	KV Cache	Total Runtime Memory
4K Tokens	~655 MB	~6.2 GB
8K Tokens	~1.31 GB	~7.0 GB
16K Tokens	~2.62 GB	~8–9 GB
32K Tokens	~5.24 GB	~11 GB

Hardware Requirements

Training Environment

Component	Value
GPU	NVIDIA T4
VRAM	16 GB
Quantization	4-bit NF4
Fine-Tuning Method	LoRA

Inference Environment

Component	Value
GPU	RTX 2080
VRAM	8 GB
System RAM	32 GB
Recommended Context	8192 Tokens
Quantization	Q4_K_M

Deployment Artifacts

Artifact	Purpose
granite_python_lora.zip	LoRA Adapter Backup
adapter_model.safetensors	Fine-Tuned Weights
granite-4.1-8b.Q4_K_M.gguf	Deployable Model
Modelfile	Ollama Configuration
granite-python	Ollama Model Name

Project Workflow

Dataset → LoRA Fine-Tuning → Adapter Export → Model Merge → GGUF Conversion → Q4_K_M Quantization → Ollama Deployment → VS Code Integration → Custom Python Code Generation AI Model

Downloads last month: 41

GGUF

Model size

9B params

Architecture

granite

Hardware compatibility

4-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support