Instructions to use SandLogicTechnologies/Qwen3.8-27B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use SandLogicTechnologies/Qwen3.8-27B-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf SandLogicTechnologies/Qwen3.8-27B-GGUF:IQ3_M
# Run inference directly in the terminal:
llama cli -hf SandLogicTechnologies/Qwen3.8-27B-GGUF:IQ3_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf SandLogicTechnologies/Qwen3.8-27B-GGUF:IQ3_M
# Run inference directly in the terminal:
llama cli -hf SandLogicTechnologies/Qwen3.8-27B-GGUF:IQ3_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf SandLogicTechnologies/Qwen3.8-27B-GGUF:IQ3_M
# Run inference directly in the terminal:
./llama-cli -hf SandLogicTechnologies/Qwen3.8-27B-GGUF:IQ3_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf SandLogicTechnologies/Qwen3.8-27B-GGUF:IQ3_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf SandLogicTechnologies/Qwen3.8-27B-GGUF:IQ3_M

Use Docker

docker model run hf.co/SandLogicTechnologies/Qwen3.8-27B-GGUF:IQ3_M

LM Studio
Jan

vLLM

How to use SandLogicTechnologies/Qwen3.8-27B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "SandLogicTechnologies/Qwen3.8-27B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SandLogicTechnologies/Qwen3.8-27B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/SandLogicTechnologies/Qwen3.8-27B-GGUF:IQ3_M

Ollama
How to use SandLogicTechnologies/Qwen3.8-27B-GGUF with Ollama:
```
ollama run hf.co/SandLogicTechnologies/Qwen3.8-27B-GGUF:IQ3_M
```
Unsloth Desktop

How to use SandLogicTechnologies/Qwen3.8-27B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf SandLogicTechnologies/Qwen3.8-27B-GGUF:IQ3_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "SandLogicTechnologies/Qwen3.8-27B-GGUF:IQ3_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use SandLogicTechnologies/Qwen3.8-27B-GGUF with Docker Model Runner:
```
docker model run hf.co/SandLogicTechnologies/Qwen3.8-27B-GGUF:IQ3_M
```

Lemonade

How to use SandLogicTechnologies/Qwen3.8-27B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull SandLogicTechnologies/Qwen3.8-27B-GGUF:IQ3_M

Run and chat with the model

lemonade run user.Qwen3.8-27B-GGUF-IQ3_M

List all available models

lemonade list

Hermes Agent

How to use SandLogicTechnologies/Qwen3.8-27B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf SandLogicTechnologies/Qwen3.8-27B-GGUF:IQ3_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default SandLogicTechnologies/Qwen3.8-27B-GGUF:IQ3_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use SandLogicTechnologies/Qwen3.8-27B-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf SandLogicTechnologies/Qwen3.8-27B-GGUF:IQ3_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "SandLogicTechnologies/Qwen3.8-27B-GGUF:IQ3_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Qwen3.8-27B

Qwen3.8-27B is a native multimodal dense model from the Qwen family, designed for advanced reasoning, coding, professional workflows, research, and long-horizon agentic tasks. The model combines a 27B-parameter language model with a vision encoder, enabling it to process both textual and visual information.

The model supports native image and video understanding alongside text generation, with flexible thinking controls for adjusting reasoning behavior. It is designed for complex multi-step tasks, software engineering, agentic workflows, professional work, research, and visual understanding while remaining suitable for local deployment relative to larger Qwen models.

Qwen3.8-27B provides a 262K-token native context window and can be extended to up to 1M tokens using YaRN. Its combination of multimodal understanding, long-context processing, reasoning, and agent execution makes it suitable for demanding workloads requiring sustained context and multi-step task completion.

Model Overview

Model Name: Qwen3.8-27B
Architecture: Causal Language Model with Vision Encoder
Parameter Count: 27B parameters
Context Window: 262K tokens
Modalities: Text, Image, Video
Primary Languages: Multilingual
Developer: Qwen / Alibaba Cloud
License: Apache 2.0

Quantization Details

This repository provides various GGUF quantized versions of the Qwen3.8-27B model, optimized for efficient local inference using llama.cpp. Below are the details of the available I-Matrix (IQ) formats.

Quantization Formats (I-Quants)

IQ3_M

Size reduction of approx 76.64% (11.89 GB) compared to 16-bit (50.90 GB)
IQ3_M is an aggressive low-bit quantization format intended for deployments where reducing the memory footprint is a major priority.
It compresses the model into a compact 3-bit representation while retaining important model information through importance-aware quantization.
This format is suitable for local inference on systems with substantially lower memory capacity than required by the original model.
The reduced precision can have a greater effect on complex reasoning, long-context processing, multimodal understanding, and agentic workloads compared with higher-precision variants.

IQ4_XS

Size reduction of approx 71.79% (14.36 GB) compared to 16-bit (50.90 GB)
IQ4_XS provides a balanced 4-bit quantization option designed to reduce memory consumption while maintaining a relatively high level of model fidelity.
Its importance-aware representation makes it suitable for general local inference workloads involving reasoning, coding, conversational generation, and multimodal tasks.
The format provides a practical deployment compromise for users who need substantially lower memory requirements without moving to the more aggressive 3-bit representation.
Additional reconstruction complexity may affect prompt-processing performance depending on the inference backend and hardware configuration.

IQ4_NL

Size reduction of approx 70.49% (15.02 GB) compared to 16-bit (50.90 GB)
IQ4_NL is a 4-bit non-linear quantization format designed to provide higher-fidelity weight representation than more aggressive low-bit formats.
Its non-linear mapping helps represent the original weight distribution more effectively, making it appropriate for quality-sensitive reasoning, coding, multimodal understanding, and structured generation workloads.
This format provides a strong balance between the original model's capabilities and the reduced memory requirements of GGUF deployment.
The larger model footprint and additional dequantization complexity may require more memory and compute resources than the smaller IQ3_M and IQ4_XS variants.

Training Overview

Pretraining

The Qwen3.8-27B model is developed as a large-scale multimodal model with training across language and visual information, enabling unified processing of text and visual inputs.

Training focuses on:

Large-scale language modeling
Multimodal representation learning
Vision-language understanding
Image and video comprehension
Long-context learning
General reasoning and knowledge acquisition

Alignment and Optimization

Post-training is used to improve the model's performance on complex real-world tasks and interactive use cases:

Instruction following and conversational behavior
Reasoning and flexible thinking control
Agentic task execution
Coding and software-engineering workflows
Professional and research-oriented tasks
Multimodal grounding and visual understanding
Improved long-context task completion

Core Capabilities

Multimodal understanding Processes text, images, and videos to generate context-aware responses and perform visual reasoning.
Advanced reasoning Supports multi-step reasoning across mathematical, analytical, research, and general problem-solving tasks.
Agentic task execution Designed for long-horizon workflows involving planning, environment feedback, tool usage, and multi-step task completion.
Coding and software engineering Supports code generation, debugging, software-engineering workflows, and complex development tasks.
Long-context processing Supports a native 262K-token context window and can be extended toward 1M tokens using YaRN.
Flexible thinking control Supports configurable reasoning behavior, allowing thinking to be enabled or disabled and reasoning depth to be adjusted.
Professional and research workflows Designed for demanding knowledge-intensive tasks spanning research, analysis, document-oriented work, and professional applications.
Multilingual support Supports multilingual interaction and mixed-language inputs across a broad range of languages.

Example Usage

llama.cpp

./llama-cli \
  -m SandLogicTechnologies/Qwen3.8-27B_IQ4_NL.gguf \
  -p "Explain the architectural differences between linear attention and full attention."

Recommended Use Cases

Advanced reasoning and analytical AI systems
Agentic AI and long-horizon task execution
Software engineering and coding assistants
Multimodal question answering
Image and video understanding
Research and professional knowledge workflows
Long-context document analysis
Multimodal document understanding
Intelligent automation systems
Enterprise AI assistants
Local deployment of large multimodal language models
Applications requiring sustained context and complex multi-step reasoning

Acknowledgments

These quantized models are based on the original work by the Qwen / Alibaba Cloud development team.

Special thanks to:

The Qwen team for developing and releasing the Qwen3.8-27B model.
Georgi Gerganov and the llama.cpp open-source community for enabling efficient quantization and inference via the GGUF format.

Contact

For any inquiries or support, please contact us at support@sandlogic.com or visit our Website.

Downloads last month: 662

GGUF

Model size

27B params

Architecture

qwen35

Hardware compatibility

3-bit

4-bit

Model tree for SandLogicTechnologies/Qwen3.8-27B-GGUF

Base model

Qwen/Qwen3.8-27B

Quantized

(1040)

this model