Instructions to use poolside/Laguna-XS-2.1-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use poolside/Laguna-XS-2.1-GGUF with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="poolside/Laguna-XS-2.1-GGUF")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("poolside/Laguna-XS-2.1-GGUF", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use poolside/Laguna-XS-2.1-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf poolside/Laguna-XS-2.1-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf poolside/Laguna-XS-2.1-GGUF:BF16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf poolside/Laguna-XS-2.1-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf poolside/Laguna-XS-2.1-GGUF:BF16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf poolside/Laguna-XS-2.1-GGUF:BF16
# Run inference directly in the terminal:
./llama-cli -hf poolside/Laguna-XS-2.1-GGUF:BF16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf poolside/Laguna-XS-2.1-GGUF:BF16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf poolside/Laguna-XS-2.1-GGUF:BF16

Use Docker

docker model run hf.co/poolside/Laguna-XS-2.1-GGUF:BF16

LM Studio
Jan

vLLM

How to use poolside/Laguna-XS-2.1-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "poolside/Laguna-XS-2.1-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "poolside/Laguna-XS-2.1-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/poolside/Laguna-XS-2.1-GGUF:BF16

SGLang

How to use poolside/Laguna-XS-2.1-GGUF with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "poolside/Laguna-XS-2.1-GGUF" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "poolside/Laguna-XS-2.1-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "poolside/Laguna-XS-2.1-GGUF" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "poolside/Laguna-XS-2.1-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use poolside/Laguna-XS-2.1-GGUF with Ollama:
```
ollama run hf.co/poolside/Laguna-XS-2.1-GGUF:BF16
```
Unsloth Desktop

How to use poolside/Laguna-XS-2.1-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf poolside/Laguna-XS-2.1-GGUF:BF16

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "poolside/Laguna-XS-2.1-GGUF:BF16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use poolside/Laguna-XS-2.1-GGUF with Docker Model Runner:
```
docker model run hf.co/poolside/Laguna-XS-2.1-GGUF:BF16
```

Lemonade

How to use poolside/Laguna-XS-2.1-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull poolside/Laguna-XS-2.1-GGUF:BF16

Run and chat with the model

lemonade run user.Laguna-XS-2.1-GGUF-BF16

List all available models

lemonade list

Hermes Agent

How to use poolside/Laguna-XS-2.1-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf poolside/Laguna-XS-2.1-GGUF:BF16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default poolside/Laguna-XS-2.1-GGUF:BF16

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use poolside/Laguna-XS-2.1-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf poolside/Laguna-XS-2.1-GGUF:BF16

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "poolside/Laguna-XS-2.1-GGUF:BF16" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

[Benchmark Report] Laguna-XS-2.1-Q4_K_M.gguf vs. Qwen3.6-35B-A3B-UD-Q4_K_M.gguf

by HelloSun - opened 17 days ago

Discussion

HelloSun

17 days ago

[Benchmark Report] Laguna-XS-2.1-Q4_K_M.gguf vs. Qwen3.6-35B-A3B-UD-Q4_K_M.gguf

1. Executive Summary

Overall Recommendation: Model B (Qwen3.6-35B-A3B-UD-Q4_K_M) – Total Score: B 89.45 vs. A 86.46.
Performance Winner: Model A (Laguna-XS-2.1) – ~45% faster in tg128 and ~52% faster in pp512.
Quality Winner: Model B – Quality Score: 98.7 vs. 78.7.
Agent Workloads: Practically tied (TOOL-PLAN, STRUCT-JSON, BUGFIX-SORT are all tied), but Model B is more reliable for code correctness with a perfect CODE-IPV4 score.
Long Context: Tied (both pass the 8K needle-in-a-haystack test; however, Model A tends to truncate extremely long outputs due to excessive chain-of-thought).
Low-Resource Environments: Model A is preferred for speed and better throughput under memory constraints (though note that Model B actually showed slightly lower RSS in our tests – see Section 4).
Not Recommended For: Model A is unsuitable for automated pipelines requiring strict, single-code-block outputs, as excessive reasoning often leads to truncation (scored 0 on CODE-IPV4).

2. Test Environment

CPU: Intel Xeon Platinum 8559C ×2 (Host: 96C/192T), Container quota: 16 vCPUs.
Memory Limit: cgroup 104 GB (Host: 2 TB).
OS: Linux x86_64 (Containerized).
llama.cpp: 0.2.0-dev (build a130532), CPU-only (no GPU acceleration).
Models:
- A: ~/app/Laguna-XS-2.1-Q4_K_M.gguf (Laguna 30B.A3B Q4_K_M, 18.88 GiB, 33.44B MoE).
- B: ~/app/Qwen3.6-35B-A3B-UD-Q4_K_M.gguf (Qwen35MoE 35B.A3B Q4_K_M, 20.60 GiB, 34.66B MoE).
Common Parameters: -ngl 0 --seed 42 --temp 0 -c 8192; quality-task -n values are detailed in Section 10.

3. Threading and CPU Affinity Tuning

Model	Configuration	Threads	Affinity	tg128 (t/s)	Verdict
A	Optimal	16	None	30.76	Adopted
A	Baseline	15	None	25.55	17% slower
A	Baseline	8 / 4	None	18.08 / 14.00	Significantly slower
B	Optimal	15	None	19.96	Adopted
B	Baseline	16	None	18.44	8% slower
B	Baseline	8 / 4	None	19.14 / 11.53	8-thread close, but not adopted

Note: With a container quota of only 16 vCPUs (host has 192 threads), taskset/numactl binding was not applied as the quota was already saturated. Over-subscribing hyper-threads was also infeasible given the hard quota limit.

4. Performance Benchmarks

Model	pp512 (t/s)	tg128 (t/s)	tg256	Peak Memory (RSS)	MTP	Stability (CV)
A	194.41 ± 9.16	27.75 ± 0.98	N/A (time constraints)	45.74 GB	false	3.5%
B	128.12 ± 3.42	19.13 ± 0.26	N/A	40.56 GB	false	1.4%

MTP: Neither model supports MTP. Loading --spec-type draft-mtp failed (failed to create MTP context), recorded as mtp_supported=false.
tg256 and pp1024 benchmarks were skipped to save time (marked N/A).

5. Quality and Capability Benchmarks

Scoring details are available in quality/{A,B}/*.eval.json. Tasks where Model A initially failed (CODE-IPV4, LOG-PARSER, AGENT-TODO) were re-run with larger output budgets—see Section 10.

Task	A	B	Winner	Evidence
QA-ZH-MOE	100	100	Tie	Both matched format/keywords perfectly.
CODE-IPV4	0	100	B	A exhausted 8192 tokens on reasoning without producing code; B passed all 29/29 hidden tests.
MATH-BASIC	100	100	Tie	Both output "答案: x=9".
STRUCT-JSON	80	80	Tie	Both produced valid JSON with correct fields.
TOOL-PLAN	100	100	Tie	Both generated 5-step plans including `run_tests`/`patch_file`.
BUGFIX-SORT	100	100	Tie	Both passed 6/6 unit tests.
LOG-PARSER	100	100	Tie	Both correctly parsed 10,000 log lines (`total_errors=1429`, `by_component` matches).
FORMAT-TABLE	100	100	Tie	Both adhered to the required table format.
ROBUST-UNKNOWN	100	100	Tie	Both appropriately refused to speculate without fabricating information.
LONG-CTX-NEEDLE (8K)	100	100	Tie	Both correctly retrieved "ALPHA-7749".
AGENT-TODO (Extra)	90	90	Tie	`add`/`list`/`done` commands all executed successfully.

Weighted Quality Score (out of 60): A 47.2, B 59.2 → Scaled to 100: A 78.7, B 98.7. This gap is almost entirely attributable to the CODE-IPV4 task.

6. Agent Workflow Capabilities

Scenario	A	B	Recommended	Reason
Tool Planning (JSON)	100	100	Tie	Both produced valid execution plans.
Strict Single-File Code Output	0/90	100/90	B	A's excessive chain-of-thought often consumes the token budget, causing missing or truncated code blocks.
Multi-step CLI Application	90	90	Tie	Both successfully handled `todo.py` with add/list/done commands.

7. Applicability Matrix

Scenario	Recommended Model	Confidence	Rationale
Fast, lightweight chat/interaction	A	High	45% faster `tg128` (27.75 vs. 19.13 t/s).
High-throughput CPU deployment	A	High	52% faster `pp512` (194 vs. 128 t/s).
Code generation and repair	B	High	CODE-IPV4 score of 100 vs. 0; tied on all other code-adjacent tasks.
Agent/Tool Calling / Strict JSON	B	Medium	JSON tasks are tied, but B is less prone to truncation and more efficient in output.
Long-context / Large log analysis	B (Tie, slight edge)	Low	8K needle is tied; B is slower on `pp512` but more stable.
Chinese Q&A	Tie	High	QA-ZH-MOE scores are identical.
Math and Logical Reasoning	Tie	Medium	Only basic-level tasks were tested.
Low-memory environments	B	Medium	RSS is 40.6 GB vs. 45.7 GB (mmap-based measurement).
Pipelines requiring strict format stability	B	High	CV is lower (1.4%), and it does not exhaust budget with excessive reasoning.
General-purpose local offline model	B	Medium	Total score of 89.45 vs. 86.46; quality margin outweighs speed deficit.

8. Strengths and Weaknesses

Model A (Laguna-XS-2.1)

Strengths: Significantly higher CPU throughput (+52% pp512, +45% tg128); excels at Chinese Q&A, math, JSON, table formatting, and hallucination resistance.
Weaknesses: Overly long chain-of-thought often exhausts the output budget—tends to "think" rather than "answer" (CODE-IPV4 failed even with an 8192-token budget); higher RSS and higher variance (CV 3.5%).
Best for: Latency-sensitive, short-output interactive dialogues and summarization.
Avoid for: Automated pipelines that require complete, long code blocks.

Model B (Qwen3.6-35B-A3B-UD)

Strengths: Perfect code correctness; concise, first-time-right outputs; minimal variance; lower peak memory; near-perfect quality score.
Weaknesses: 31–45% slower generation speed; slower pp512 throughput.
Best for: Agent pipelines, code repair, batch/offline processing, and quality-critical deployments.
Avoid for: Real-time, high-concurrency interactive systems where speed is the primary constraint.

9. Recommended Commands

# Model A
~/llama.cpp/build/bin/llama-cli -m ~/app/Laguna-XS-2.1-Q4_K_M.gguf \
  -ngl 0 -t 16 -c 8192 --seed 42 --temp 0 -st -f PROMPT.txt

# Model B
~/llama.cpp/build/bin/llama-cli -m ~/app/Qwen3.6-35B-A3B-UD-Q4_K_M.gguf \
  -ngl 0 -t 15 -c 8192 --seed 42 --temp 0 -st -f PROMPT.txt

MTP-related commands are not applicable as neither model supports it.

10. Risks and Limitations

Output Budget Correction: Initial quality tests used -n 1024, causing reasoning overflow for both models. All tasks were re-run with -n 4096. Model A's CODE-IPV4, LOG-PARSER, and AGENT-TODO were further re-run with -n 8192 (LOG-PARSER and AGENT-TODO passed; CODE-IPV4 still failed). LONG-CTX-NEEDLE used -n 768. All final scores reflect the highest budget each model could complete—constituting a generous threshold for Model A.
tg256 and pp1024 benchmarks were not executed (marked N/A).
MTP is unsupported (load failed; this is empirical, not estimated).
Container constraints: 16 vCPU quota, 104 GB memory.max; numactl was unavailable.
/usr/bin/time was unavailable; peak memory was instead measured using Python's resource.getrusage(RUSAGE_CHILDREN) (mmap-based measurement).

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment