le server 
  * ` /stream_log/ ` endpoint for streaming all (or some) intermediate steps from your chain/agent 
  * **new** as of 0.0.40, supports ` astream_events ` to make it easier to stream without needing to parse the output of ` stream_log ` . 
  * Playground page at ` /playground/ ` with streaming output and intermediate steps 
  * Built-in (optional) tracing to [ LangSmith ](https://www.langchain.com/langsmith) , just add your API key (see [ Instructions ](https://docs.smith.langchain.com/) ) 
  * All built with battle-tested open-source Python libraries like FastAPI, Pydantic, uvloop and asyncio. 
  * Use the client SDK to call a LangServe server as if it was a Runnable running locally (or call the HTTP API directly) 
  * [ LangServe Hub ](https://github.com/langchain-ai/langchain/blob/master/templates/README.md)

##  Limitations  ​

  * Client callbacks are not yet supported for events that originate on the server 
  * OpenAPI docs will not be generated when using Pydantic V2. Fast API does not support [ mixing pydantic v1 and v2 namespaces ](https://github.com/tiangolo/fastapi/issues/10360) . See section below for more details. 

##  Hosted LangServe  ​

We will be releasing a hosted version of LangServe for one-click deployments
of LangChain applications. [ Sign up here
](https://airtable.com/apppQ9p5XuujRl3wJ/shrABpHWdxry8Bacm) to get on the
waitlist.

##  Security  ​

  * Vulnerability in Versions 0.0.13 - 0.0.15 -- playground endpoint allows accessing arbitrary files on server. [ Resolved in 0.0.16 ](https://github.com/langchain-ai/langserve/pull/98) . 

##  Installation  ​

For both client and server:

    
    
    pip install "langserve[all]"  
    

or ` pip install "langserve[client]" ` for client code, and ` pip install
"langserve[server]" ` for server code.

##  LangChain CLI 🛠️  ​

Use the ` LangChain ` CLI to bootstrap a ` LangServe ` project quickly.

To use the langchain CLI make sure that you have a recent version of `
langchain-cli ` installed. You can install it with ` pip install -U langchain-
cli ` .

##  Setup  ​

**Note** : We use ` poetry ` for dependency management. Please follow poetry [
doc ](https://python-poetry.org/docs/) to learn more about it.

###  1\. Create new app using langchain cli command  ​

    
    
    langchain app new my-app  
    

###  2\. Define the runnable in add_routes. Go to server.py and edit  ​

    
    
    add_routes(app. NotImplemented)  
    

###  3\. Use ` poetry ` to add 3rd party packages (e.g., langchain-openai,
langchain-anthropic, langchain-mistral etc).  ​

    
    
    poetry add [package-name] // e.g `poetry add langchain-openai`  
    

###  4\. Set up relevant env variables. For example,  ​

    
    
    export OPENAI_API_KEY="sk-..."  
    

###  5\. Serve your app  ​

    
    
    poetry run langchain serve --port=8100  
    

##  Examples  ​

Get your LangServe instance started quickly with [ LangChain Templates
](https://github.com/langchain-ai/langchain/blob/master/templates/README.md) .

For more examples, see the templates [ index ](https://github.com/langchain-
ai/langchain/blob/master/templates/docs/INDEX.md) or the [ examples
](https://github.com/langchain-ai/langserve/tree/main/examples) directory.

Description  |  Links   
---|---  
**LLMs** Minimal example that reserves OpenAI and Anthropic chat models. Uses async, supports batching and streaming.  |  [ server ](https://github.com/langchain-ai/langserve/tree/main/examples/llm/server.py) , [ client ](https://github.com/langchain-ai/langserve/blob/main/examples/llm/client.ipynb)  
**Retriever** Simple server that exposes a retriever as a runnable.  |  [ server ](https://github.com/langchain-ai/langserve/tree/main/examples/retrieval/server.py) , [ client ](https://github.com/langchain-ai/langserve/tree/main/examples/retrieval/client.ipynb)  
**Conversational Retriever** A [ Conversational Retriever ](https://python.langchain.com/docs/expression_language/cookbook/retrieval#conversationa