How to use LLM for efficient text outputs longer than 4k tokens?
If you ask an LLM to improve your text, it will reply with the whole text. This is time consuming and expensive. In this article we discuss how to use patching prompts and diffs to modify texts efficiently.
Tags: llm, streaming, ai, patch