for PyTorch (Part 2).  Model-Optimization,Production
![](_static/img/thumbnails/cropped/generic-pytorch-logo.png)
](intermediate/torchserve_with_ipex_2)

#### [ Multi-Objective Neural Architecture Search with Ax  Learn how to use Ax
to search over architectures find optimal tradeoffs between accuracy and
latency.  Model-Optimization,Best-Practice,Ax,TorchX
![](_static/img/ax_logo.png)
](intermediate/ax_multiobjective_nas_tutorial.html)

#### [ torch.compile Tutorial  Speed up your models with minimal code changes
using torch.compile, the latest PyTorch compiler solution.  Model-Optimization
![](_static/img/thumbnails/cropped/generic-pytorch-logo.png)
](intermediate/torch_compile_tutorial.html)

#### [ Inductor CPU Backend Debugging and Profiling  Learn the usage,
debugging and performance profiling for ``torch.compile`` with Inductor CPU
backend.  Model-Optimization  ![](_static/img/thumbnails/cropped/generic-
pytorch-logo.png) ](intermediate/inductor_debug_cpu.html)

#### [ (beta) Implementing High-Performance Transformers with SCALED DOT
PRODUCT ATTENTION  This tutorial explores the new
torch.nn.functional.scaled_dot_product_attention and how it can be used to
construct Transformer components.  Model-Optimization,Attention,Transformer
![](_static/img/thumbnails/cropped/pytorch-logo.png)
](intermediate/scaled_dot_product_attention_tutorial.html)

#### [ Knowledge Distillation in Convolutional Neural Networks  Learn how to
improve the accuracy of lightweight models using more powerful models as
teachers.  Model-Optimization,Image/Video
![](_static/img/thumbnails/cropped/knowledge_distillation_pytorch_logo.png)
](beginner/knowledge_distillation_tutorial.html)

#### [ PyTorch Distributed Overview  Briefly go over all concepts and features
in the distributed package. Use this document to find the distributed training
technology that can best serve your application.  Parallel-and-Distributed-
Training  ![](_static/img/thumbnails/cropped/PyTorch-Distributed-Overview.png)
](beginner/dist_overview.html)

#### [ Distributed Data Parallel in PyTorch - Video Tutorials  This series of
video tutorials walks you through distributed training in PyTorch via DDP.
Parallel-and-Distributed-Training  ![](_static/img/thumbnails/cropped/PyTorch-
Distributed-Overview.png) ](beginner/ddp_series_intro.html)

#### [ Single-Machine Model Parallel Best Practices  Learn how to implement
model parallel, a distributed training technique which splits a single model
onto different GPUs, rather than replicating the entire model on each GPU
Parallel-and-Distributed-Training  ![](_static/img/thumbnails/cropped/Model-
Parallel-Best-Practices.png) ](intermediate/model_parallel_tutorial.html)

#### [ Getting Started with Distributed Data Parallel  Learn the basics of
when to use distributed data paralle versus data parallel and work through an
example to set it up.  Parallel-and-Distributed-Training
![](_static/img/thumbnails/cropped/Getting-Started-with-Distributed-Data-
Parallel.png) ](intermediate/ddp_tutorial.html)

#### [ Writing Distributed Applications with PyTorch  Set up the distributed
package of PyTorch, use the different communication strategies, and go over
some the internals of the package.  Parallel-and-Distributed-Training
![](_static/img/thumbnails/cropped/Writing-Distributed-Applications-with-
PyTorch.png) ](intermediate/dist_tuto.html)

#### [ Large Scale Transformer model training with Tensor Parallel  Learn how
to train large models with Tensor Parallel package.  Parallel-and-Distributed-
Training  ![](_static/img/thumbnails/cropped/Large-Scale-Transformer-model-
training-with-Tensor-Parallel.png) ](intermediate/TP_tutorial.html)

#### [ Customize Process Group Backends Using Cpp Extensions  Extend
ProcessGroup with custom collective communication implementations.  Parallel-
and-Distributed-Training  ![](_static/img/thumbnails/cropped/Customize-
Process-Group-Backends-Using-Cpp-Extensions.png)
](intermediate/process_group_cpp_extension_tutori