## Chat models

## Features (natively supported)[​](#features-natively-supported "Direct link to Features (natively supported)")

All ChatModels implement the Runnable interface, which comes with default implementations of all methods, ie. `ainvoke`, `batch`, `abatch`, `stream`, `astream`. This gives all ChatModels basic support for async, streaming and batch, which by default is implemented as below:

*   _Async_ support defaults to calling the respective sync method in asyncio's default thread pool executor. This lets other async functions in your application make progress while the ChatModel is being executed, by moving this call to a background thread.
*   _Streaming_ support defaults to returning an `Iterator` (or `AsyncIterator` in the case of async streaming) of a single value, the final result returned by the underlying ChatModel provider. This obviously doesn't give you token-by-token streaming, which requires native support from the ChatModel provider, but ensures your code that expects an iterator of tokens can work for any of our ChatModel integrations.
*   _Batch_ support defaults to calling the underlying ChatModel in parallel for each input by making use of a thread pool executor (in the sync batch case) or `asyncio.gather` (in the async batch case). The concurrency can be controlled with the `max_concurrency` key in `RunnableConfig`.

Each ChatModel integration can optionally provide native implementations to truly enable async or streaming. The table shows, for each integration, which features have been implemented with native support. Yellow circles (🟡) indicates partial support - for example, if the model supports tool calling but not tool messages for agents.

| Model | Invoke | Async invoke | Stream | Async stream | [Tool calling](https://python.langchain.com/docs/modules/model_io/chat/function_calling/) | [Structured output](https://python.langchain.com/docs/modules/model_io/chat/structured_output/) | Python Package |
| --- | --- | --- | --- | --- | --- | --- | --- |
| AzureChatOpenAI | ✅   | ❌   | ❌   | ❌   | ✅   | ✅   | langchain-openai |
| BedrockChat | ✅   | ❌   | ✅   | ❌   | ❌   | ❌   | langchain-community |
| ChatAnthropic | ✅   | ❌   | ❌   | ❌   | ✅   | ✅   | langchain-anthropic |
| ChatAnyscale | ✅   | ✅   | ✅   | ✅   | ❌   | ❌   | langchain-community |
| ChatBaichuan | ✅   | ❌   | ✅   | ❌   | ❌   | ❌   | langchain-community |
| ChatCohere | ✅   | ❌   | ❌   | ❌   | 🟡  | ✅   | langchain-cohere |
| ChatDatabricks | ✅   | ❌   | ✅   | ❌   | ❌   | ❌   | langchain-community |
| ChatDeepInfra | ✅   | ✅   | ✅   | ✅   | ❌   | ❌   | langchain-community |
| ChatEverlyAI | ✅   | ✅   | ✅   | ✅   | ❌   | ❌   | langchain-community |
| ChatFireworks | ✅   | ❌   | ❌   | ❌   | ✅   | ✅   | langchain-fireworks |
| ChatFriendli | ✅   | ✅   | ✅   | ✅   | ❌   | ❌   | langchain-community |
| ChatGooglePalm | ✅   | ✅   | ❌   | ❌   | ❌   | ❌   | langchain-community |
| ChatGroq | ✅   | ❌   | ❌   | ❌   | 🟡  | ✅   | langchain-groq |
| ChatHuggingFace | ✅   | ✅   | ✅   | ✅   | ❌   | ❌   | langchain-community |
| ChatHunyuan | ✅   | ❌   | ✅   | ❌   | ❌   | ❌   | langchain-community |
| ChatJavelinAIGateway | ✅   | ✅   | ❌   | ❌   | ❌   | ❌   | langchain-community |
| ChatKinetica | ✅   | ❌   | ❌   | ❌   | ❌   | ❌   | langchain-community |
| ChatKonko | ✅   | ❌   | ❌   | ❌   | ❌   | ❌   | langchain-community |
| ChatLiteLLM | ✅   | ✅   | ✅   | ✅   | ❌   | ❌   | langchain-community |
| ChatLiteLLMRouter | ✅   | ✅   | ✅   | ✅   | ❌   | ❌   | langchain-community |
| ChatMLX | ✅   | ✅   | ✅   | ❌   | ❌   | ❌   | langchain-community |
| ChatMLflowAIGateway | ✅   | ❌   | ❌   | ❌   | ❌   | ❌   | langchain-community |
| ChatMaritalk | ✅   | ❌   | ❌   | ❌   | ❌   | ❌   | langchain-community |
| ChatMistralAI | ✅   | ❌   | ❌   | ❌   | ✅   | ✅   | langchain-mistralai |
| ChatMlflow | ✅   | ❌   | ✅   | ❌   | ❌   | ❌   | langchain-community |
| ChatOctoAI | ✅   | ✅   | ✅   | ✅   | ❌   | ❌   | langchain-community |
| ChatOllama | ✅   | ✅   | ✅   | ✅   | ❌   | ❌   | langchain-community |
| ChatOpenAI | ✅   | ❌   | ❌   | ❌   | ✅   | ✅   | langchain-openai |
| ChatPerplexity | ✅   | ❌   | ✅   | ❌   | ❌   | ❌   | langchain-community |
| ChatPremAI | ✅   | ❌   | ✅   | ❌   | ❌   | ❌   | langchain-community |
| ChatSparkLLM | ✅   | ❌   | ✅   | ❌   | ❌   | ❌   | langchain-community |
| ChatTongyi | ✅   | ✅   | ✅   | ✅   | ❌   | ❌   | langchain-community |
| ChatVertexAI | ✅   | ❌   | ❌   | ❌   | ✅   | ✅   | langchain-google-vertexai |
| ChatYandexGPT | ✅   | ✅   | ❌   | ❌   | ❌   | ❌   | langchain-community |
| ChatYuan2 | ✅   | ✅   | ✅   | ✅   | ❌   | ❌   | langchain-community |
| ChatZhipuAI | ✅   | ✅   | ✅   | ✅   | ❌   | ❌   | langchain-community |
| ErnieBotChat | ✅   | ❌   | ❌   | ❌   | ❌   | ❌   | langchain-community |
| GPTRouter | ✅   | ✅   | ✅   | ✅   | ❌   | ❌   | langchain-community |
| GigaChat | ✅   | ✅   | ✅   | ✅   | ❌   | ❌   | langchain-community |
| JinaChat | ✅   | ✅   | ✅   | ✅   | ❌   | ❌   | langchain-community |
| LlamaEdgeChatService | ✅   | ❌   | ✅   | ❌   | ❌   | ❌   | langchain-community |
| MiniMaxChat | ✅   | ✅   | ❌   | ❌   | ❌   | ❌   | langchain-community |
| PaiEasChatEndpoint | ✅   | ❌   | ❌   | ✅   | ❌   | ❌   | langchain-community |
| PromptLayerChatOpenAI | ✅   | ❌   | ❌   | ❌   | ❌   | ❌   | langchain-community |
| QianfanChatEndpoint | ✅   | ✅   | ✅   | ✅   | ❌   | ❌   | langchain-community |
| SolarChat | ✅   | ✅   | ✅   | ✅   | ❌   | ❌   | langchain-community |
| VolcEngineMaasChat | ✅   | ❌   | ✅   | ❌   | ❌   | ❌   | langchain-community |