Instructions to use poolside/Laguna-XS-2.1-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use poolside/Laguna-XS-2.1-GGUF with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="poolside/Laguna-XS-2.1-GGUF")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("poolside/Laguna-XS-2.1-GGUF", device_map="auto")

llama-cpp-python

How to use poolside/Laguna-XS-2.1-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="poolside/Laguna-XS-2.1-GGUF",
	filename="Laguna-XS-2.1-BF16.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use poolside/Laguna-XS-2.1-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf poolside/Laguna-XS-2.1-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf poolside/Laguna-XS-2.1-GGUF:BF16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf poolside/Laguna-XS-2.1-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf poolside/Laguna-XS-2.1-GGUF:BF16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf poolside/Laguna-XS-2.1-GGUF:BF16
# Run inference directly in the terminal:
./llama-cli -hf poolside/Laguna-XS-2.1-GGUF:BF16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf poolside/Laguna-XS-2.1-GGUF:BF16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf poolside/Laguna-XS-2.1-GGUF:BF16

Use Docker

docker model run hf.co/poolside/Laguna-XS-2.1-GGUF:BF16

LM Studio
Jan

vLLM

How to use poolside/Laguna-XS-2.1-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "poolside/Laguna-XS-2.1-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "poolside/Laguna-XS-2.1-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/poolside/Laguna-XS-2.1-GGUF:BF16

SGLang

How to use poolside/Laguna-XS-2.1-GGUF with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "poolside/Laguna-XS-2.1-GGUF" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "poolside/Laguna-XS-2.1-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "poolside/Laguna-XS-2.1-GGUF" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "poolside/Laguna-XS-2.1-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use poolside/Laguna-XS-2.1-GGUF with Ollama:
```
ollama run hf.co/poolside/Laguna-XS-2.1-GGUF:BF16
```

Unsloth Studio

How to use poolside/Laguna-XS-2.1-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for poolside/Laguna-XS-2.1-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for poolside/Laguna-XS-2.1-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for poolside/Laguna-XS-2.1-GGUF to start chatting

How to use poolside/Laguna-XS-2.1-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf poolside/Laguna-XS-2.1-GGUF:BF16

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "poolside/Laguna-XS-2.1-GGUF:BF16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use poolside/Laguna-XS-2.1-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf poolside/Laguna-XS-2.1-GGUF:BF16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default poolside/Laguna-XS-2.1-GGUF:BF16

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use poolside/Laguna-XS-2.1-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf poolside/Laguna-XS-2.1-GGUF:BF16

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "poolside/Laguna-XS-2.1-GGUF:BF16" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use poolside/Laguna-XS-2.1-GGUF with Docker Model Runner:
```
docker model run hf.co/poolside/Laguna-XS-2.1-GGUF:BF16
```

Lemonade

How to use poolside/Laguna-XS-2.1-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull poolside/Laguna-XS-2.1-GGUF:BF16

Run and chat with the model

lemonade run user.Laguna-XS-2.1-GGUF-BF16

List all available models

lemonade list

Pretty good

by flyingweasel - opened 6 days ago

Discussion

flyingweasel

6 days ago

Been using it with oh-my-pi and llama.cpp (pull/25165/head:laguna) on a V100 32GB for development of personal rust, python, c++ and typescript projects since a couple days.
With Q4 KV-Quant it uses ~24GB, with full precision ~30GB of VRAM.

llama.cpp/build/bin/llama-server \
    -hf poolside/Laguna-XS-2.1-GGUF:Q4_K_M \
    --host 0.0.0.0 \
    --parallel 1 \
    --batch-size 1024 \
    --ubatch-size 512 \
    --temp 0.6 \
    --top-p 0.95 \
    --top-k 20 \
    --min-p 0.05 \
    --metrics \
    -lv 4 \
    --flash-attn on \
    --jinja \
    --cont-batching

15.48.449.247 I slot   operator(): id  0 | task 29489 | cached n_tokens = 102829, memory_seq_rm [102829, end)
15.50.701.264 I slot print_timing: id  0 | task 29489 | prompt processing, n_tokens =  22528, progress = 0.98, t =  47.90 s / 470.32 tokens per second
15.50.701.267 I slot   operator(): id  0 | task 29489 | cached n_tokens = 103853, memory_seq_rm [103853, end)
15.52.965.633 I slot print_timing: id  0 | task 29489 | prompt processing, n_tokens =  23552, progress = 0.99, t =  50.16 s / 469.50 tokens per second
15.52.965.637 I slot   operator(): id  0 | task 29489 | cached n_tokens = 104877, memory_seq_rm [104877, end)
15.53.052.741 I slot print_timing: id  0 | task 29489 | prompt processing, n_tokens =  23601, progress = 1.00, t =  50.25 s / 469.66 tokens per second
15.53.052.746 I slot   operator(): id  0 | task 29489 | cached n_tokens = 104926, memory_seq_rm [104926, end)
15.53.052.904 I slot create_check: id  0 | task 29489 | erasing context checkpoint too close to an earlier one (pos_min = 80011, pos_max = 81034, n_tokens = 81035, size = 60.007 MiB)
15.53.053.366 I slot create_check: id  0 | task 29489 | erasing context checkpoint too close to an earlier one (pos_min = 80302, pos_max = 81325, n_tokens = 81326, size = 60.007 MiB)
15.53.143.004 I slot create_check: id  0 | task 29489 | created context checkpoint 3 of 32 (pos_min = 103902, pos_max = 104925, n_tokens = 104926, size = 60.007 MiB)
15.54.253.398 I slot print_timing: id  0 | task 29489 | prompt processing, n_tokens =  24099, progress = 1.00, t =  51.45 s / 468.38 tokens per second
15.54.253.402 I slot   operator(): id  0 | task 29489 | cached n_tokens = 105424, memory_seq_rm [105424, end)
15.54.285.387 I slot create_check: id  0 | task 29489 | created context checkpoint 4 of 32 (pos_min = 104400, pos_max = 105423, n_tokens = 105424, size = 60.007 MiB)
15.54.318.583 I slot print_timing: id  0 | task 29489 | prompt processing, n_tokens =  24113, progress = 1.00, t =  51.52 s / 468.06 tokens per second
15.54.318.586 I slot   operator(): id  0 | task 29489 | cached n_tokens = 105438, memory_seq_rm [105438, end)
15.54.330.225 I slot init_sampler: id  0 | task 29489 | init sampler, took 11.50 ms, tokens: text = 105442, total = 105442
15.54.368.023 I slot create_check: id  0 | task 29489 | created context checkpoint 5 of 32 (pos_min = 104414, pos_max = 105437, n_tokens = 105438, size = 60.007 MiB)
15.57.407.944 I slot print_timing: id  0 | task 29489 | n_decoded =    141, tg =  46.84 t/s, tg_3s =  46.84 t/s
16.00.415.901 I slot print_timing: id  0 | task 29489 | n_decoded =    281, tg =  46.69 t/s, tg_3s =  46.54 t/s
16.03.430.496 I slot print_timing: id  0 | task 29489 | n_decoded =    421, tg =  46.61 t/s, tg_3s =  46.44 t/s
16.06.445.043 I slot print_timing: id  0 | task 29489 | n_decoded =    561, tg =  46.57 t/s, tg_3s =  46.44 t/s
16.09.459.962 I slot print_timing: id  0 | task 29489 | n_decoded =    701, tg =  46.54 t/s, tg_3s =  46.44 t/s
16.09.761.423 I cmn  common_reaso: deactivated (natural end)

Speed is ~30-80 inference token per second, 400-700 prompt processing token per second.

Works pretty well and seems to be on par with Qwen3.6 27B MTP (was using that before).
It has issues with looping sometimes, but I've encountered this with all MoE LLM's (and even some dense ones). And that can usually be fixed with a "/compact" as well as "take a different approach" (or similar).
Also it seems to sometimes "keep" context (references information from a previous session without reading that file or recalling memories, but that seems to be in line with linuxid10t's comment (https://github.com/ggml-org/llama.cpp/pull/25165#issuecomment-4985453742)?).

It seems to be more useful than any of the Gemma 4 (26B A4B / 31B) or Ornith 35B for my work.
It also has less issues with tool calling (e.g. OMP's hashline editing, glab, git and other CLI tool usage than Qwen 3.6 27B. Gemma 4 Models were pretty much unusable for OMP and Ornith stopped properly working after a ~60k context.
It also seems like it does not dilly-dally around like Qwen3.6 does (I've found Qwen tends to over-think or think for way too long).

Of course it's not almighty, so nudging it sometimes is necessary.
Also I've noticed it does not like to fill big gaps like Qwen does, so I feel like I have to use other wording or be more specific for certain tasks.

Anyways, these are my two cents, thanks for the model.

flyingweasel

6 days ago

Just noticed that I have been using the wrong / non-recommended parameters.
With

llama.cpp/build/bin/llama-server \
    -hf poolside/Laguna-XS-2.1-GGUF:Q4_K_M \
    --host 0.0.0.0 \
    --parallel 1 \
    --batch-size 1024 \
    --ubatch-size 512 \
    --temp 1.0 \
    --top-p 1.0 \
    --top-k 20 \
    --metrics \
    -lv 4 \
    --flash-attn on \
    --jinja \
    --cont-batching

it works really good, have not seen looping again (yet?).

gammaray-co

4 days ago

I'm getting unknown architecture still in the upstream llama.cpp b10091. But i thought laguna support was merged as of b10087.

0.00.465.633 E llama_model_load: error loading model: unknown model architecture: 'laguna'
0.00.465.636 E llama_model_load_from_file_impl: failed to load model
$ llama-server --version
version: 10091 (b4d6c7d)
built with GNU 15.2.0 for Linux x86_64

What am i doing wrong?

flyingweasel

4 days ago

•

edited 4 days ago

I'm getting unknown architecture still in the upstream llama.cpp b10091. But i thought laguna support was merged as of b10087.
0.00.465.633 E llama_model_load: error loading model: unknown model architecture: 'laguna'
0.00.465.636 E llama_model_load_from_file_impl: failed to load model
$ llama-server --version
version: 10091 (b4d6c7d)
built with GNU 15.2.0 for Linux x86_64
What am i doing wrong?

Just updated my llama-cpp to the latest state, but I can not reproduce your issue.

$ ./host_laguna_xs_2_1.sh 
0.00.341.680 I cmn  common_param: common_params_print_info: build 10103 (c588c4f47) with GNU 14.2.0 for Linux x86_64
...
0.00.465.229 I srv    load_model: loading model 'poolside/Laguna-XS-2.1-GGUF:Q4_K_M'
0.00.465.231 I srv    load_model: local path '/home/flyingweasel/.cache/huggingface/hub/models--poolside--Laguna-XS-2.1-GGUF/snapshots/1a37c0a5fb8c7a18e6106decb6be6327d1b63fa6/Laguna-XS-2.1-Q4_K_M.gguf'
0.00.465.240 I cmn  common_init_: fitting params to device memory ...
0.00.465.241 I cmn  common_init_: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
0.00.465.244 I common_params_fit_impl: getting device memory data for initial parameters:
0.00.665.225 I common_memory_breakdown_print: | memory breakdown [MiB]     | total    free     self   model   context   compute    unaccounted |
0.00.665.230 I common_memory_breakdown_print: |   - CUDA0 (V100-PCIE-32GB) | 32494 = 32184 + (29903 = 19221 +   10360 +     322) +      -29593 |
0.00.665.231 I common_memory_breakdown_print: |   - Host                   |                    375 =   110 +       0 +     265                |
0.00.680.866 I common_params_fit_impl: projected to use 29903 MiB of device memory vs. 32184 MiB of free device memory
0.00.680.872 I common_params_fit_impl: will leave 2280 >= 1024 MiB of free device memory, no changes needed
0.00.680.872 I common_fit_params: successfully fit params to free device memory
0.00.680.876 I common_fit_params: fitting params to free memory took 0.22 seconds
0.00.702.810 I llama_model_loader: loaded meta data with 56 key-value pairs and 678 tensors from /home/flyingweasel/.cache/huggingface/hub/models--poolside--Laguna-XS-2.1-GGUF/snapshots/1a37c0a5fb8c7a18e6106decb6be6327d1b63fa6/Laguna-XS-2.1-Q4_K_M.gguf (version GGUF V3 (latest))
...
0.22.959.117 I sched_reserve: reserve took 192.40 ms, sched copies = 1
0.22.959.159 I cmn  common_init_: warming up the model with an empty run - please wait ... (--no-warmup to disable)
0.23.063.623 I srv    load_model: initializing, n_slots = 1, n_ctx_slot = 262144, kv_unified = 'false'
0.23.063.639 I spec common_specu: no implementations specified for speculative decoding
0.23.063.640 I slot   load_model: id  0 | task -1 | new slot, n_ctx = 262144
0.23.063.705 I srv    load_model: prompt cache is enabled, size limit: 8192 MiB
0.23.063.706 I srv    load_model: use `--cache-ram 0` to disable the prompt cache
0.23.063.706 I srv    load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
0.23.063.706 I srv    load_model: context checkpoints enabled, max = 32, min spacing = 8192
0.23.063.733 I srv          init: idle slots will be saved to prompt cache upon starting a new task
0.23.075.813 I srv          init: init: chat template, example_format: '<system>You are a helpful assistant</system>
<user>Hello</user>
<assistant><think></think>Hi there</assistant>
<user>How are you?</user>
<assistant><think>'
0.23.085.450 I srv          init: init: chat template, thinking = 1
0.23.085.484 I srv  llama_server: model loaded
0.23.085.487 I srv  llama_server: listening on http://0.0.0.0:8080
0.23.085.492 I srv  update_slots: all slots are idle

gammaray-co

3 days ago

found it. some stale linked libraries that didn't get removed by my pipeline cleanup because they were renamed in the repo. all good now.

Damn this thing is fast. haven't tested the accuracy. but seeing about 33t/s token generation on my little 780M vs about 18t/s with Ornith.

Will do some more benchmarks tonight.

gammaray-co

3 days ago

Looks like tg falls off pretty badly as the context increases. Ornith at Q8 is slower, but a lot more consistent.

$ llama-bench -hf poolside/Laguna-XS-2.1-GGUF:Q4_K_M -ngl 999 -t 16 -mmp 0 -fa 1 -ctk q8_0 -ctv q8_0 -p 0 -n 128 -d 0,16384,65536
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon 780M Graphics (RADV PHOENIX) (radv) | uma: 1 | fp16: dot2 | bf16: 0 | fp4: 0 | warp size: 64 | shared memory: 65536 | int dot: 1 | matrix cores: KHR_coopmat
| model                          |       size |     params | backend    | ngl | threads | type_k | type_v |  fa | mmap |            test |                  t/s |
| ------------------------------ | ---------: | ---------: | ---------- | --: | ------: | -----: | -----: | --: | ---: | --------------: | -------------------: |
| laguna 30B.A3B Q4_K - Medium   |  18.88 GiB |    33.44 B | Vulkan     | 999 |      16 |   q8_0 |   q8_0 |   1 |    0 |           tg128 |         36.16 ± 0.03 |
| laguna 30B.A3B Q4_K - Medium   |  18.88 GiB |    33.44 B | Vulkan     | 999 |      16 |   q8_0 |   q8_0 |   1 |    0 |  tg128 @ d16384 |         28.18 ± 0.08 |
| laguna 30B.A3B Q4_K - Medium   |  18.88 GiB |    33.44 B | Vulkan     | 999 |      16 |   q8_0 |   q8_0 |   1 |    0 |  tg128 @ d65536 |         17.11 ± 0.01 |

build: 0278d83 (10098)

alexzkhr

1 day ago

•

edited 1 day ago

I asked Laguna 2.1 XS model to solve simple logical task - 4 travelers cross the bridge with 1 light source (a must), different walk times. Question is: what is minimal time?
Catch is capacity of the bridge is not set, so valid answer is 4 walk at once with slowest walker is the answer.
Also I used my custom prompt which switches Gemma 4 to explore all possibilities and than Gemma finds "4 at once".
Well, this model was thinking for long (in output chat, not in tag thinking), than lost between 2 answers, decided to check internet, found the typical standard question (without catch) with different timings, and used that timings to compute answer for me.
So what it did - it invented new task and solved it instead mine.
So I'm not really sure what for this model could be used...to follow super detailed instructions?

akierum

1 day ago

Anyone thinking it's anything like qwen 3.6 27B ask to replicate any tokenchaser prompt he used for qwen 3.6 27B. See for yourself that is no where close.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment