[ ](https://pytorch.org/)

  * [ Get Started ](https://pytorch.org/get-started)
  * [ Ecosystem ](https://pytorch.org/ecosystem)
  * Edge 

[ About PyTorch Edge  ](https://pytorch.org/edge) [ ExecuTorch
](https://pytorch.org/executorch)

  * [ Blog ](https://pytorch.org/blog/)
  * [ Tutorials ](https://pytorch.org/tutorials)
  * Docs 

[ PyTorch  ](https://pytorch.org/docs/stable/index.html) [ torchaudio
](https://pytorch.org/audio/stable/index.html) [ torchtext
](https://pytorch.org/text/stable/index.html) [ torchvision
](https://pytorch.org/vision/stable/index.html) [ torcharrow
](https://pytorch.org/torcharrow) [ TorchData  ](https://pytorch.org/data) [
TorchRec  ](https://pytorch.org/torchrec) [ TorchServe
](https://pytorch.org/serve/) [ TorchX  ](https://pytorch.org/torchx/) [
PyTorch on XLA Devices  ](https://pytorch.org/xla)

  * Resources 

[ About  Learn about PyTorch’s features and capabilities
](https://pytorch.org/features) [ PyTorch Foundation  Learn about the PyTorch
foundation  ](https://pytorch.org/foundation) [ Community  Join the PyTorch
developer community to contribute, learn, and get your questions answered.
](https://pytorch.org/#community-module) [ Community Stories  Learn how our
community solves real, everyday machine learning problems with PyTorch.
](https://pytorch.org/community-stories) [ Developer Resources  Find resources
and get questions answered  ](https://pytorch.org/resources) [ Events  Find
events, webinars, and podcasts  ](https://pytorch.org/events) [ Forums  A
place to discuss PyTorch code, issues, install, research
](https://discuss.pytorch.org/) [ Models (Beta)  Discover, publish, and reuse
pre-trained models  ](https://pytorch.org/hub)

  * [ GitHub ](https://github.com/pytorch/pytorch)

Table of Contents

master

Contents:

  * TorchServe 
  * [ Troubleshooting Guide ](Troubleshooting.html)
  * [ Performance Guide ](performance_guide.html)
  * [ Batch Inference with TorchServe ](batch_inference_with_ts.html)
  * [ Code Coverage ](code_coverage.html)
  * [ Advanced configuration ](configuration.html)
  * [ Custom Service ](custom_service.html)
  * [ TorchServe default inference handlers ](default_handlers.html)
  * [ Logging in Torchserve ](logging.html)
  * [ TorchServe Metrics ](metrics.html)
  * [ Model Zoo ](model_zoo.html)
  * [ Request Envelopes ](request_envelopes.html)
  * [ Running TorchServe ](server.html)
  * [ Running TorchServe with NVIDIA MPS ](nvidia_mps.html)
  * [ TorchServe model snapshot ](snapshot.html)
  * [ TorchServe on Windows ](torchserve_on_win_native.html)
  * [ TorchServe on Windows Subsystem for Linux (WSL) ](torchserve_on_wsl.html)
  * [ Torchserve Use Cases ](use_cases.html)
  * [ TorchServe Workflows ](workflows.html)
  * [ Serving large models with Torchserve ](large_model_inference.html)
  * [ Security Policy ](Security.html)
  * [ FAQ’S ](FAQs.html)

Service APIs:

  * [ TorchServe gRPC API ](grpc_api.html)
  * [ Inference API ](inference_api.html)
  * [ Management API ](management_api.html)
  * [ Metrics API ](metrics_api.html)
  * [ TorchServe REST API ](rest_api.html)
  * [ Workflow Inference API ](workflow_inference_api.html)
  * [ Management API ](workflow_management_api.html)

Developer APIs:

  * [ ts package ](api/ts.html)
  * [ ts.torch_handler.request_envelope package ](api/ts.torch_handler.request_envelope.html)
  * [ ts.torch_handler package ](api/ts.torch_handler.html)
  * [ ts.metrics package ](api/ts.metrics.html)
  * [ ts.model_service package ](api/ts.model_service.html)
  * [ ts.protocol package ](api/ts.protocol.html)
  * [ serve ](api/modules.html)

  * [ Docs ](contents.html) >
  * TorchServe 
  * [ ![](_static/images/view-page-source-icon.svg) ](_sources/index.rst.txt)

Shortcuts

#  TorchServe  ¶

TorchServe is a performant, flexible and easy to use tool for serving PyTorch
models in production.

What’s going on in TorchServe?

  * [ High performance Llama 2 deployments with AWS Inferentia2 using TorchServe ](https://pytorch.org