poolside

/

Laguna-S-2.1-GGUF

like 111

Follow

Poolside 960

Model info: Q4_K_M
Settings: llama-server -m laguna-s-2.1-Q4_K_M.gguf --jinja --chat-template-file chat_template.jinja --port 8000 -fa on

One-line summary
The model invoked edit without the required newText argument and mixed tool-markup residue into the payload, so validation failed and no code change was applied.

Problem
• Coding-agent evaluation via local llama-server (OpenAI-compatible API)
• Agent attempted a file edit via the edit tool, but tool arguments failed schema validation
• No source change was applied (empty diff)
• Not an eval-server / network / tool-runtime outage — model tool-call generation quality issue

───

Observed outputs

1. Invalid tool call from the model
   edits[0] contained only oldText; newText was missing. Corrupted markup leaked into the argument string:

path: /testbed/django/db/migrations/serializer.py
edits: [
{
"oldText": "class EnumSerializer(BaseSerializer): def serialize(self): ..."
// newText missing
// pollution at end of oldText:
// ...importsnewText ]...
}
]

2. Tool runtime response: Validation failed for tool "edit":
   • edits.0.newText: must have required properties newText

3. Expected schema (as produced on successful PASS runs)

{
"path": "...",
"edits": [
{
"oldText": "...",
"newText": "..."
}
]
}

4. Additional observations
   • Same environment: other tasks produced valid edit(oldText, newText) and applied successfully (PASS)
   • Around the failing turn: stopReason: length observed → possible truncation mid tool-call generation
   • On some tasks, almost no edit tool calls; long reasoning only, then timeout

───

Suspected root cause (model-side)

1. Broken tool-call argument generation
   • Missing required field newText
   • Tool/XML leftovers (, , etc.) leaked into arguments
2. Unstable structured output
   • Multi-field tool args not completed as valid schema-compliant JSON
3. Possible max-token / length stop
   • Generation cut off during tool call → missing fields + corrupted payload

Ruled out: eval server failure, tool executor bug, API endpoint outage
(Valid tool calls succeed in the same session with the same tool definitions.)