# 03 — Serve and thinking

Primary: Hub model card · models page · blog.

## OpenAI-compatible API

```python
from openai import OpenAI
import os

client = OpenAI(
    api_key=os.getenv("POOLSIDE_API_KEY"),
    base_url="https://inference.poolside.ai/v1",
)
client.chat.completions.create(
    model="poolside/laguna-s-2.1",
    messages=[{"role": "user", "content": "…"}],
    stream=True,
)
```

## vLLM (Hub)

```
vllm serve \
  --model poolside/Laguna-S-2.1 \
  --tensor-parallel-size 4 \
  --tool-call-parser poolside_v1 \
  --reasoning-parser poolside_v1 \
  --enable-auto-tool-choice \
  --served-model-name laguna \
  --default-chat-template-kwargs '{"enable_thinking": true}'
```

Optional DFlash speculative decoding (`generation_config` uses **15** speculative tokens; card examples sometimes show 7 — prefer config):

```
--speculative-config '{"model":"poolside/Laguna-S-2.1-DFlash","num_speculative_tokens":15,"method":"dflash"}'
```

## SGLang

```
python -m sglang.launch_server \
  --model-path poolside/Laguna-S-2.1 \
  --tp-size 4 \
  --reasoning-parser poolside_v1 \
  --tool-call-parser poolside_v1 \
  --trust-remote-code
```

## TRT-LLM

```
trtllm-serve poolside/Laguna-S-2.1 --trust-remote-code \
  --tool_parser poolside_v1 --reasoning_parser laguna
```

Note: flag names differ from vLLM (`--tool_parser`; reasoning parser is `laguna`).

## llama.cpp / GGUF

- Weights: `poolside/Laguna-S-2.1-GGUF`  
- Fork: `poolsideai/llama.cpp` branch `laguna` (DFlash support)  
- Upstream PR also in flight for base Laguna  

## Thinking contract

| Item | Detail |
|---|---|
| Modes | **off** / **max** (default max) |
| Per request | `extra_body={"chat_template_kwargs": {"enable_thinking": false}}` |
| Server default | `--default-chat-template-kwargs '{"enable_thinking": true}'` |
| Preserved thinking | Keep prior assistant `reasoning_content` in history |
| pool toggle | `/thought-level` |
| Tool format | XML-like `<tool_call>…</tool_call>` with `arg_key` / `arg_value` |

Card: model works best with preserved thinking; may stop reasoning in follow-ups if prior thinking blocks are dropped.

## Footguns

| Issue | Mitigation |
|---|---|
| Harness schema drift | Reject + retry; prefer pool / ACP |
| Nested JSON tool args | Schema validation; simplify tools |
| Overthinking | `enable_thinking=false` when latency matters |
| BF16 VRAM | TP≥4-class; use FP8/INT4/GGUF for smaller nodes |
| Parser mismatch | Always set tool + reasoning parsers |

## Memory (order of magnitude)

BF16 weights ≈ **236 GB**. SWA (window 512) caps local KV growth; global layers still scale with sequence. See [memory_notes](../derived/memory_notes.md).