Text Generation
Transformers
GGUF
Russian
English
lora
qwen2.5
fine-tuned
function-calling
tool-use
system-administration
ollama
Instructions to use ckacor/my-lora-model with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ckacor/my-lora-model with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="ckacor/my-lora-model")# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("ckacor/my-lora-model", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use ckacor/my-lora-model with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "ckacor/my-lora-model" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ckacor/my-lora-model", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/ckacor/my-lora-model
- SGLang
How to use ckacor/my-lora-model with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "ckacor/my-lora-model" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ckacor/my-lora-model", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "ckacor/my-lora-model" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ckacor/my-lora-model", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use ckacor/my-lora-model with Docker Model Runner:
docker model run hf.co/ckacor/my-lora-model
🛠️ my-lora-model (Qwen2.5-1.5B Sysadmin Tool-Use)
A lightweight LoRA fine-tune of Qwen2.5-1.5B-Instruct, specialized for fast function calling and tool use in system administration tasks. Designed for local execution on low-resource hardware, CPUs, or edge devices.
💡 What it does
The model processes natural language requests in Russian and English and outputs structured tool execution parameters.
Supported Tools out-of-the-box:
get_gpu_status— Retrieve GPU temperature, utilization, and memory usage.execute_terminal_command— Run an arbitrary Bash command.read_file— Read the contents of a specified file path.
Example Prompts handled:
- 🇬🇧 "What's the GPU temperature?" ➔ Calls
get_gpu_status - 🇬🇧 "Uptime?" ➔ Calls
execute_terminal_commandwithuptime - 🇷🇺 "Покажи конфиг config.json" ➔ Calls
read_filewith pathconfig.json
🚀 How to use
Via Ollama (GGUF version)
- Download the
.gguffile from the repository. - Create a
Modelfile:FROM ./my-lora-model-q8_0.gguf SYSTEM "You are a helpful system administration assistant capable of using system tools."