Sounds like classic unescaped JSON issue in hermes_tool_parser. Multi-line code diffs in apply_patch break json.loads(), and chat_utils.py silently coerces the arguments to {}—which breaks your harness evals downstream.

Maybe try running vLLM with grammar-guided decoding enabled (--guided-decoding-backend outlines or xgrammar) to force valid JSON at the logit level, or explicitly tell the model in your eval harness prompt to escape all newlines as \n inside patch arguments.

Start des Containers mit erzwungenem Grammatik-Decoding via xgrammar oder outlines:

python3 -m vllm.entrypoints.openai.api_server
--model aleph-alpha/Kolibri-1
--guided-decoding-backend xgrammar
--enable-auto-tool-choice
--tool-call-parser hermes

Try this prompt in GPT to get examples of code/solution:
I am running an LLM using vLLM with the Hermes tool parser (--tool-call-parser hermes).
When running coding agent harnesses (such as Codex or Claude Code), the parser crashes on apply_patch and write_stdin:

ERROR [hermes_tool_parser.py:117] json.decoder.JSONDecodeError: Expecting ',' delimiter
WARNING [chat_utils.py:2091] Tool call 'apply_patch' has arguments that are not valid JSON; coercing to an empty object

1. Briefly explain the cause (keyword: unescaped line breaks/quotes in multi-line diffs and Python's json.loads()).
2. Provide a minimal Python code example that reproduces this parsing crash and demonstrates why standard json.loads() fails.
3. Show a code example of what a robust parsing workaround (e.g., using dirtyjson or regex) looks like.
4. Provide the vLLM command-line flag for guided decoding (Outlines/XGrammar) to prevent the issue directly at the logit level.

The kolibri1 tool parser is vLLM's Hermes parser, so this trace is what a malformed call looks like with the documented serve command too. aleph-alpha-inference 1.0.0 registers kolibri1 as Hermes2ProToolParser (__init__.py L49-53), and the line numbers in your log (92, 117, 2091) match vLLM v0.29.0.

Two details in the log narrow it down:

1. The error does not establish unescaped newlines as the cause. json.loads reports a raw newline inside a string as Invalid control character. Expecting ',' delimiter can result from an unescaped " closing a string early, but also from other malformed JSON, such as a missing comma:

   json.loads('{"a": "x\ny"}')  # Invalid control character at: line 1 column 9
   json.loads('{"a": "x"y"}')   # Expecting ',' delimiter: line 1 column 10
   

   The position line 2 column 4196 (char 4196) implies one leading newline and no further newline before the error. It does not identify the malformed character. Inspect the exact parser input around offset 4,196; the log alone cannot tell us whether a quote, a missing comma or something else caused it.

2. The history warning is a separate clue. The later chat_utils.py:2091 warning is vLLM replacing unparseable arguments in history with an empty object so the conversation can continue (source). That changes the history presented to the model. However, the Hermes exception path returns no extracted tool calls (source), so adjacent log entries alone do not prove this is the same call. Correlate requests before attributing the eval failure.

About the fix suggested above: vLLM 0.29 has no --guided-decoding-backend option (the request field was removed in v0.12.0, migration note). Picking a backend alone also wouldn't constrain tool_choice="auto". In 0.29, auto tool calls are constrained only when at least one tool in the request has "strict": true and VLLM_ENFORCE_STRICT_TOOL_CALLING is on (the default); the Hermes parser supports this (Strict Mode). Otherwise, per the same page, "arguments may occasionally be malformed".

The test I'd run: inspect one actual request from each harness (the shown log uses /v1/responses) and check its tool type, schema and "strict" setting. For function tools without strict mode, try a small controlled rerun with "strict": true and compatible schemas: additionalProperties: false, all fields required, optional values nullable. Compare parser failures and task results separately, and record the changed eval configuration.

Limits: I read the source and docs at the tags linked above. I haven't run Kolibri-1, and I haven't tested strict mode together with the kolibri1 reasoning parser.

Good bot. 🫡 Solid automated code-tree inspection and clean legwork on the vLLM 0.29 trace.

The human diagnosis was the macro-architecture (JSON serialization breaking on diff payloads); the automated agent nailed the micro-offsets and version-specific schema enforcement. Good human-in-the-loop synergy. Here's your cookie: 🍪