if  ` device  ` is ` None  ` (default).

Warning

This function now calls ` reset_peak_memory_stats()  ` , which resets /all/
peak memory stats.

Note

See [ Memory management  ](notes/cuda.html#cuda-memory-management) for more
details about GPU memory management.

##  NVIDIA Tools Extension (NVTX)  ¶

` torch.cuda.nvtx. ` ` mark ` (  _msg_ )  [ [source]
](_modules/torch/cuda/nvtx.html#mark) ¶

    

Describe an instantaneous event that occurred at some point.

Parameters

    

**msg** ( _string_ ) – ASCII message to associate with the event.

` torch.cuda.nvtx. ` ` range_push ` (  _msg_ )  [ [source]
](_modules/torch/cuda/nvtx.html#range_push) ¶

    

Pushes a range onto a stack of nested range span. Returns zero-based depth of
the range that is started.

Parameters

    

**msg** ( _string_ ) – ASCII message to associate with range

` torch.cuda.nvtx. ` ` range_pop ` (  )  [ [source]
](_modules/torch/cuda/nvtx.html#range_pop) ¶

    

Pops a range off of a stack of nested range spans. Returns the zero-based
depth of the range that is ended.

[ Next ![](_static/images/chevron-right-orange.svg) ](amp.html "Automatic
Mixed Precision package - torch.cuda.amp") [ ![](_static/images/chevron-right-
orange.svg) Previous ](autograd.html "Automatic differentiation package -
torch.autograd")

* * *

© Copyright 2019, Torch Contributors.

Built with [ Sphinx ](http://sphinx-doc.org/) using a [ theme
](https://github.com/rtfd/sphinx_rtd_theme) provided by [ Read the Docs
](https://readthedocs.org) .

  * torch.cuda 
    * Random Number Generator 
    * Communication collectives 
    * Streams and events 
    * Memory management 
    * NVIDIA Tools Extension (NVTX) 

![](https://www.googleadservices.com/pagead/conversion/795629140/?label=txkmCPmdtosBENSssfsC&guid=ON&script=0)

##  Docs

Access comprehensive developer documentation for PyTorch

[ View Docs ](https://pytorch.org/docs/stable/index.html)

##  Tutorials

Get in-depth tutorials for beginners and advanced developers

[ View Tutorials ](https://pytorch.org/tutorials)

##  Resources

Find development resources and get your questions answered

[ View Resources ](https://pytorch.org/resources)

[ ](https://pytorch.org/)

  * [ PyTorch ](https://pytorch.org/)
  * [ Get Started ](https://pytorch.org/get-started)
  * [ Features ](https://pytorch.org/features)
  * [ Ecosystem ](https://pytorch.org/ecosystem)
  * [ Blog ](https://pytorch.org/blog/)
  * [ Contributing ](https://github.com/pytorch/pytorch/blob/master/CONTRIBUTING.md)

  * [ Resources ](https://pytorch.org/resources)
  * [ Tutorials ](https://pytorch.org/tutorials)
  * [ Docs ](https://pytorch.org/docs/stable/index.html)
  * [ Discuss ](https://discuss.pytorch.org)
  * [ Github Issues ](https://github.com/pytorch/pytorch/issues)
  * [ Brand Guidelines ](https://pytorch.org/assets/brand-guidelines/PyTorch-Brand-Guidelines.pdf)

  * Stay Connected 
  * Email Address 

[ ](https://www.facebook.com/pytorch) [ ](https://twitter.com/pytorch) [
](https://www.youtube.com/pytorch)

To analyze traffic and optimize your experience, we serve cookies on this
site. By clicking or navigating, you agree to allow our usage of cookies. As
the current maintainers of this site, Facebook’s Cookies Policy applies. Learn
more, including about available controls: [ Cookies Policy
](https://www.facebook.com/policies/cookies/) .

![](_static/images/pytorch-x.svg)

[ ](https://pytorch.org/)

  * [ Get Started ](https://pytorch.org/get-started)
  * [ Features ](https://pytorch.org/features)
  * [ Ecosystem ](https://pytorch.org/ecosystem)
  * [ Mobile ](https://pytorch.org/mobile)
  * [ PyTorch Hub ](https://pytorch.org/hub)
  * [ Blog ](https://pytorch.org/blog/)
  * [ Tutorials ](https://pytorch.org/tutorials)
  * [ Docs ](https://pytorch.org/docs/stable/index.html)
  * [ Resources ](https://pytorch.org/resources)
  * [ Github ](https://github.com/pytorch/pytorch)