*   [](/v0.2/)
*   [How-to guides](/v0.2/docs/how_to/)
*   How to stream chat model responses

On this page

How to stream chat model responses
==================================

All [chat models](https://api.python.langchain.com/en/latest/language_models/langchain_core.language_models.chat_models.BaseChatModel.html) implement the [Runnable interface](https://api.python.langchain.com/en/latest/runnables/langchain_core.runnables.base.Runnable.html#langchain_core.runnables.base.Runnable), which comes with a **default** implementations of standard runnable methods (i.e. `ainvoke`, `batch`, `abatch`, `stream`, `astream`, `astream_events`).

The **default** streaming implementation provides an`Iterator` (or `AsyncIterator` for asynchronous streaming) that yields a single value: the final output from the underlying chat model provider.

tip

The **default** implementation does **not** provide support for token-by-token streaming, but it ensures that the the model can be swapped in for any other model as it supports the same standard interface.

The ability to stream the output token-by-token depends on whether the provider has implemented proper streaming support.

See which [integrations support token-by-token streaming here](/v0.2/docs/integrations/chat/).

Sync streaming[​](#sync-streaming "Direct link to Sync streaming")
------------------------------------------------------------------

Below we use a `|` to help visualize the delimiter between tokens.

    from langchain_anthropic.chat_models import ChatAnthropicchat = ChatAnthropic(model="claude-3-haiku-20240307")for chunk in chat.stream("Write me a 1 verse song about goldfish on the moon"):    print(chunk.content, end="|", flush=True)

**API Reference:**[ChatAnthropic](https://api.python.langchain.com/en/latest/chat_models/langchain_anthropic.chat_models.ChatAnthropic.html)

    Here| is| a| |1| |verse| song| about| gol|dfish| on| the| moon|:|Floating| up| in| the| star|ry| night|,|Fins| a|-|gl|im|mer| in| the| pale| moon|light|.|Gol|dfish| swimming|,| peaceful| an|d free|,|Se|ren|ely| |drif|ting| across| the| lunar| sea|.|

Async Streaming[​](#async-streaming "Direct link to Async Streaming")
---------------------------------------------------------------------

    from langchain_anthropic.chat_models import ChatAnthropicchat = ChatAnthropic(model="claude-3-haiku-20240307")async for chunk in chat.astream("Write me a 1 verse song about goldfish on the moon"):    print(chunk.content, end="|", flush=True)

**API Reference:**[ChatAnthropic](https://api.python.langchain.com/en/latest/chat_models/langchain_anthropic.chat_models.ChatAnthropic.html)

    Here| is| a| |1| |verse| song| about| gol|dfish| on| the| moon|:|Floating| up| above| the| Earth|,|Gol|dfish| swim| in| alien| m|irth|.|In| their| bowl| of| lunar| dust|,|Gl|it|tering| scales| reflect| the| trust|Of| swimming| free| in| this| new| worl|d,|Where| their| aqu|atic| dream|'s| unf|ur|le|d.|

Astream events[​](#astream-events "Direct link to Astream events")
------------------------------------------------------------------

Chat models also support the standard [astream events](https://api.python.langchain.com/en/latest/runnables/langchain_core.runnables.base.Runnable.html#langchain_core.runnables.base.Runnable.astream_events) method.

This method is useful if you're streaming output from a larger LLM application that contains multiple steps (e.g., an LLM chain composed of a prompt, llm and parser).

    from langchain_anthropic.chat_models import ChatAnthropicchat = ChatAnthropic(model="claude-3-haiku-20240307")idx = 0async for event in chat.astream_events(    "Write me a 1 verse song about goldfish on the moon", version="v1"):    idx += 1    if idx >= 5:  # Truncate the output        print("...Truncated")        break    print(event)

**API Reference:**[ChatAnthropic](https://api.python.langchain.com/en/latest/chat_models/langchain_anthropic.chat_models.ChatAnthropic.html)

    {'event': 'on_chat_model_start', 'run_id': '08da631a-12a0-4f07-baee-fc9a175ad4ba', 'name': 'ChatAnthropic', 'tags': [], 'metadata': {}, 'data': {'input': 'Write me a 1 verse song about goldfish on the moon'}}{'event': 'on_chat_model_stream', 'run_id': '08da631a-12a0-4f07-baee-fc9a175ad4ba', 'tags': [], 'metadata': {}, 'name': 'ChatAnthropic', 'data': {'chunk': AIMessageChunk(content='Here', id='run-08da631a-12a0-4f07-baee-fc9a175ad4ba')}}{'event': 'on_chat_model_stream', 'run_id': '08da631a-12a0-4f07-baee-fc9a175ad4ba', 'tags': [], 'metadata': {}, 'name': 'ChatAnthropic', 'data': {'chunk': AIMessageChunk(content="'s", id='run-08da631a-12a0-4f07-baee-fc9a175ad4ba')}}{'event': 'on_chat_model_stream', 'run_id': '08da631a-12a0-4f07-baee-fc9a175ad4ba', 'tags': [], 'metadata': {}, 'name': 'ChatAnthropic', 'data': {'chunk': AIMessageChunk(content=' a', id='run-08da631a-12a0-4f07-baee-fc9a175ad4ba')}}...Truncated

[Edit this page](https://github.com/langchain-ai/langchain/edit/master/docs/docs/how_to/chat_streaming.ipynb)

* * *

#### Was this page helpful?

  

#### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchain/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E).

[

Previous

How to invoke runnables in parallel

](/v0.2/docs/how_to/parallel/)[

Next

How to add default invocation args to a Runnable

](/v0.2/docs/how_to/binding/)

*   [Sync streaming](#sync-streaming)
*   [Async Streaming](#async-streaming)
*   [Astream events](#astream-events)