How do I make the model output JSON?
#14
by
vmajor
- opened
Is it just an instruction or is there a particular prompt syntax that I need to follow? I am using a GGUF version if that matters.
Currently I am using it with pydantic but sometimes the output cannot be parsed correctly.
I believe most of the 100% JSON output is achieved by guided decoding (in addition to telling the model to generate JSON).
If you are using Ollama, there is a json mode: https://github.com/ollama/ollama/blob/main/docs/api.md#request-json-mode and the example scripts are at https://github.com/ollama/ollama/tree/main/examples/python-json-datagenerator
Just make it output yaml instead! Save money on tokens and a headache of parsing broken JSON from any LLM model.