shrinath-suresh/blogs-docs-splitted huggingface dataset:
subclass of infra.Diagnostic, and adds additional methods to add
   more information to the diagnostic.
record_cpp_call_stack(frames_to_skip)
  Records the current C++ call stack in the diagnostic.

record_python_call_stack(frames_to_skip)
  Records the current Python call stack in the diagnostic.

class torch.onnx._internal.diagnostics.infra.DiagnosticEngine
A generic diagnostic engine based on SARIF.
This class is the main interface for diagnostics. It manages the
   creation of diagnostic contexts. A DiagnosticContext provides the
   entry point for recording Diagnostics. See infra.DiagnosticContext
   for more details.
-[ Examples ]-
Step 1: Create a set of rules. >>> rules =
   infra.RuleCollection.custom_collection_from_list( ...
   "CustomRuleCollection", ...     [ ...         infra.Rule( ...
   id="r1", ...             name="rule-1", ...
   message_default_template="Mising xxx", ...         ), ...     ],
   ... )
... )
Step 2: Create a diagnostic engine. >>> engine = DiagnosticEngine()
Step 3: Start a new diagnostic context. >>> with
   engine.create_diagnostic_context("torch.onnx.export",
   version="1.0") as context: ...     ...
Step 4: Add diagnostics in your code. ...
   context.diagnose(rules.rule1, infra.Level.ERROR)
Step 5: Afterwards, get the SARIF log. >>> sarif_log =
   engine.sarif_log()
clear()
  Clears all diagnostic contexts.

create_diagnostic_context(name, version, options=None, diagnostic_type=)
  Creates a new diagnostic context.

  Parameters:
     * **name** (*str*) -- The subject name for the diagnostic
       context.

     * **version** (*str*) -- The subject version for the
       diagnostic context.

     * **options** (*Optional**[**DiagnosticOptions**]*) -- The
       options for the diagnostic context.

  Returns:
     A new diagnostic context.
Returns:
         A new diagnostic context.
  Return type:
     *DiagnosticContext*

pretty_print(verbose=False, level=Level.ERROR)
  Pretty prints all diagnostics in the diagnostic contexts.

  Parameters:
     * **verbose** (*bool*) -- Whether to print the diagnostics in
       verbose mode. See Diagnostic.pretty_print.

     * **level** (*Level*) -- The minimum level of diagnostics to
       print.
Benchmark Utils - torch.utils.benchmark
class torch.utils.benchmark.Timer(stmt='pass', setup='pass', global_setup='', timer=, globals=None, label=None, sub_label=None, description=None, env=None, num_threads=1, language=Language.PYTHON)
Helper class for measuring execution time of PyTorch statements.
For a full tutorial on how to use this class, see:
   https://pytorch.org/tutorials/recipes/recipes/benchmark.html
The PyTorch Timer is based on timeit.Timer (and in fact uses
   timeit.Timer internally), but with several key differences:


Runtime aware:
         Timer will perform warmups (important as some elements of
         PyTorch are lazily initialized), set threadpool size so that
         comparisons are apples-to-apples, and synchronize
         asynchronous CUDA functions when necessary.


Focus on replicates:
         When measuring code, and particularly complex kernels /
models, run-to-run variation is a significant confounding
         factor. It is expected that all measurements should include
         replicates to quantify noise and allow median computation,
         which is more robust than mean. To that effect, this class
         deviates from the timeit API by conceptually merging
         timeit.Timer.repeat and timeit.Timer.autorange. (Exact
         algorithms are discussed in method docstrings.) The timeit
         method is replicated for cases where an adaptive strategy is
         not desired.


Optional metadata:
         When defining a Timer, one can optionally specify label,
         sub_label, description, and env. (Defined later) These
         fields are included in the representation of result object
         and by the Compare class to group and display results for
         comparison.


Instruction counts
         In addition to wall times, Timer can run a statement under
Callgrind and report instructions executed.
Directly analogous to timeit.Timer constructor arguments:
  *stmt*, *setup*, *timer*, *globals*

PyTorch Timer specific constructor arguments:
  *label*, *sub_label*, *description*, *env*, *num_threads*

Parameters:
      * stmt (str) -- Code snippet to be run in a loop and
        timed.
  * **setup** (*str*) -- Optional setup code. Used to define
    variables used in *stmt*

  * **global_setup** (*str*) -- (C++ only) Code which is placed at
    the top level of the file for things like *#include*
    statements.

  * **timer** (*Callable**[**[**]**, **float**]*) -- Callable
    which returns the current time. If PyTorch was built without
    CUDA or there is no GPU present, this defaults to
    *timeit.default_timer*; otherwise it will synchronize CUDA
    before measuring the time.

  * **globals** (*Optional**[**Dict**[**str**, **Any**]**]*) -- A
dict which defines the global variables when stmt is being
        executed. This is the other method for providing variables
        which stmt needs.
  * **label** (*Optional**[**str**]*) -- String which summarizes
    *stmt*. For instance, if *stmt* is
    "torch.nn.functional.relu(torch.add(x, 1, out=out))" one might
    set label to "ReLU(x + 1)" to improve readability.

  * **sub_label** (*Optional**[**str**]*) --

    Provide supplemental information to disambiguate measurements
    with identical stmt or label. For instance, in our example
    above sub_label might be "float" or "int", so that it is easy
    to differentiate: "ReLU(x + 1): (float)"

    "ReLU(x + 1): (int)" when printing Measurements or summarizing
    using *Compare*.

  * **description** (*Optional**[**str**]*) --

    String to distinguish measurements with identical label and
    sub_label. The principal use of *description* is to signal to
Compare the columns of data. For instance one might set it
        based on the input size  to create a table of the form:
                               | n=1 | n=4 | ...
                               ------------- ...
       ReLU(x + 1): (float)    | ... | ... | ...
       ReLU(x + 1): (int)      | ... | ... | ...

    using *Compare*. It is also included when printing a
    Measurement.

  * **env** (*Optional**[**str**]*) -- This tag indicates that
    otherwise identical tasks were run in different environments,
    and are therefore not equivalent, for instance when A/B
    testing a change to a kernel. *Compare* will treat
    Measurements with different *env* specification as distinct
    when merging replicate runs.

  * **num_threads** (*int*) -- The size of the PyTorch threadpool
    when executing *stmt*. Single threaded performance is
    important as both a key inference workload and a good
indicator of intrinsic algorithmic efficiency, so the default
        is set to one. This is in contrast to the default PyTorch
        threadpool size which tries to utilize all cores.
blocked_autorange(callback=None, min_run_time=0.2)
  Measure many replicates while keeping timer overhead to a
  minimum.

  At a high level, blocked_autorange executes the following
  pseudo-code:

     `setup`

     total_time = 0
     while total_time < min_run_time
         start = timer()
         for _ in range(block_size):
             `stmt`
         total_time += (timer() - start)

  Note the variable *block_size* in the inner loop. The choice of
  block size is important to measurement quality, and must balance
  two competing objectives:

     1. A small block size results in more replicates and
        generally better statistics.

     2. A large block size better amortizes the cost of *timer*
invocation, and results in a less biased measurement. This
            is important because CUDA synchronization time is non-
            trivial (order single to low double digit microseconds)
            and would otherwise bias the measurement.
  blocked_autorange sets block_size by running a warmup period,
  increasing block size until timer overhead is less than 0.1% of
  the overall computation. This value is then used for the main
  measurement loop.

  Returns:
     A *Measurement* object that contains measured runtimes and
     repetition counts, and can be used to compute statistics.
     (mean, median, etc.)

  Return type:
     *Measurement*

collect_callgrind(number: int, , repeats: None, collect_baseline: bool, retain_out_file: bool) -> CallgrindStats
   collect_callgrind(number: int, , repeats: int, collect_baseline: bool, retain_out_file: bool) -> Tuple[CallgrindStats, ...]
  Collect instruction counts using Callgrind.
Collect instruction counts using Callgrind.
  Unlike wall times, instruction counts are deterministic (modulo
  non-determinism in the program itself and small amounts of
  jitter from the Python interpreter.) This makes them ideal for
  detailed performance analysis. This method runs *stmt* in a
  separate process so that Valgrind can instrument the program.
  Performance is severely degraded due to the instrumentation,
  however this is ameliorated by the fact that a small number of
  iterations is generally sufficient to obtain good measurements.

  In order to to use this method *valgrind*, *callgrind_control*,
  and *callgrind_annotate* must be installed.

  Because there is a process boundary between the caller (this
  process) and the *stmt* execution, *globals* cannot contain
  arbitrary in-memory data structures. (Unlike timing methods)
  Instead, globals are restricted to builtins, *nn.Modules*'s, and
TorchScripted functions/modules to reduce the surprise factor
      from serialization and subsequent deserialization. The
      GlobalsBridge class provides more detail on this subject. Take
      particular care with nn.Modules: they rely on pickle and you may
      need to add an import to setup for them to transfer properly.
  By default, a profile for an empty statement will be collected
  and cached to indicate how many instructions are from the Python
  loop which drives *stmt*.

  Returns:
     A *CallgrindStats* object which provides instruction counts
     and some basic facilities for analyzing and manipulating
     results.

timeit(number=1000000)
  Mirrors the semantics of timeit.Timer.timeit().

  Execute the main statement (*stmt*) *number* times. https://doc
  s.python.org/3/library/timeit.html#timeit.Timer.timeit

  Return type:
     *Measurement*
Return type:
         Measurement
class torch.utils.benchmark.Measurement(number_per_run, raw_times, task_spec, metadata=None)
The result of a Timer measurement.
This class stores one or more measurements of a given statement. It
   is serializable and provides several convenience methods (including
   a detailed repr) for downstream consumers.
static merge(measurements)
  Convenience method for merging replicates.

  Merge will extrapolate times to *number_per_run=1* and will not
  transfer any metadata. (Since it might differ between
  replicates)

  Return type:
     *List*[*Measurement*]

property significant_figures: int
  Approximate significant figure estimate.

  This property is intended to give a convenient way to estimate
  the precision of a measurement. It only uses the interquartile
  region to estimate statistics to try to mitigate skew from the
  tails, and uses a static z value of 1.645 since it is not
expected to be used for small values of n, so z can
      approximate t.
  The significant figure estimation used in conjunction with the
  *trim_sigfig* method to provide a more human interpretable data
  summary. __repr__ does not use this method; it simply displays
  raw values. Significant figure estimation is intended for
  *Compare*.

class torch.utils.benchmark.CallgrindStats(task_spec, number_per_run, built_with_debug_symbols, baseline_inclusive_stats, baseline_exclusive_stats, stmt_inclusive_stats, stmt_exclusive_stats, stmt_callgrind_out)
Top level container for Callgrind results collected by Timer.
Manipulation is generally done using the FunctionCounts class,
   which is obtained by calling CallgrindStats.stats(...). Several
   convenience methods are provided as well; the most significant is
   CallgrindStats.as_standardized().
as_standardized()
  Strip library names and some prefixes from function strings.
When comparing two different sets of instruction counts, on
      stumbling block can be path prefixes. Callgrind includes the
      full filepath when reporting a function (as it should). However,
      this can cause issues when diffing profiles. If a key component
      such as Python or PyTorch was built in separate locations in the
      two profiles, which can result in something resembling:
     23234231 /tmp/first_build_dir/thing.c:foo(...)
      9823794 /tmp/first_build_dir/thing.c:bar(...)
       ...
        53453 .../aten/src/Aten/...:function_that_actually_changed(...)
       ...
      -9823794 /tmp/second_build_dir/thing.c:bar(...)
     -23234231 /tmp/second_build_dir/thing.c:foo(...)

  Stripping prefixes can ameliorate this issue by regularizing the
  strings and causing better cancellation of equivalent call sites
  when diffing.

  Return type:
     *CallgrindStats*

counts(*, denoise=False)
counts(*, denoise=False)
  Returns the total number of instructions executed.

  See *FunctionCounts.denoise()* for an explanation of the
  *denoise* arg.

  Return type:
     int

delta(other, inclusive=False)
  Diff two sets of counts.

  One common reason to collect instruction counts is to determine
  the the effect that a particular change will have on the number
  of instructions needed to perform some unit of work. If a change
  increases that number, the next logical question is "why". This
  generally involves looking at what part if the code increased in
  instruction count. This function automates that process so that
  one can easily diff counts on both an inclusive and exclusive
  basis.

  Return type:
     *FunctionCounts*

stats(inclusive=False)
  Returns detailed function counts.

  Conceptually, the FunctionCounts returned can be thought of as a
  tuple of (count, path_and_function_name) tuples.
inclusive matches the semantics of callgrind. If True, the
      counts include instructions executed by children.
      inclusive=True is useful for identifying hot spots in code;
      inclusive=False is useful for reducing noise when diffing
      counts from two different runs. (See CallgrindStats.delta(...)
      for more details)
  Return type:
     *FunctionCounts*

class torch.utils.benchmark.FunctionCounts(_data, inclusive, truncate_rows=True, _linewidth=None)
Container for manipulating Callgrind results.
It supports:
      1. Addition and subtraction to combine or diff results.
  2. Tuple-like indexing.

  3. A *denoise* function which strips CPython calls which are
     known to be non-deterministic and quite noisy.

  4. Two higher order methods (*filter* and *transform*) for
     custom manipulation.

denoise()
  Remove known noisy instructions.

  Several instructions in the CPython interpreter are rather
noisy. These instructions involve unicode to dictionary lookups
      which Python uses to map variable names. FunctionCounts is
      generally a content agnostic container, however this is
      sufficiently important for obtaining reliable results to warrant
      an exception.
  Return type:
     *FunctionCounts*

filter(filter_fn)
  Keep only the elements where *filter_fn* applied to function
  name returns True.

  Return type:
     *FunctionCounts*

transform(map_fn)
  Apply *map_fn* to all of the function names.

  This can be used to regularize function names (e.g. stripping
  irrelevant parts of the file path), coalesce entries by mapping
  multiple functions to the same name (in which case the counts
  are added together), etc.

  Return type:
     *FunctionCounts*
CUDA Stream Sanitizer
Note:
This is a prototype feature, which means it is at an early stage for
  feedback and testing, and its components are subject to change.
Overview
This module introduces CUDA Sanitizer, a tool for detecting
synchronization errors between kernels ran on different streams. It
stores information on accesses to tensors to determine if they are
synchronized or not. When enabled in a python program and a possible
data race is detected, a detailed warning will be printed and the
program will exit.
It can be enabled either by importing this module and calling
"enable_cuda_sanitizer()" or by exporting the "TORCH_CUDA_SANITIZER"
environment variable.
Usage
Here is an example of a simple synchronization error in PyTorch:
import torch
a = torch.rand(4, 2, device="cuda")
with torch.cuda.stream(torch.cuda.Stream()):
       torch.mul(a, 5, out=a)
The "a" tensor is initialized on the default stream and, without any
synchronization methods, modified on a new stream. The two kernels
will run concurrently on the same tensor, which might cause the second
kernel to read uninitialized data before the first one was able to
write it, or the first kernel might overwrite part of the result of
the second. When this script is run on the commandline with:
TORCH_CUDA_SANITIZER=1 python example_error.py
the following output is printed by CSAN:
============================
   CSAN detected a possible data race on tensor with data pointer 139719969079296
   Access by stream 94646435460352 during kernel:
   aten::mul.out(Tensor self, Tensor other, *, Tensor(a!) out) -> Tensor(a!)
   writing to argument(s) self, out, and to the output
   With stack trace:
     File "example_error.py", line 6, in 
       torch.mul(a, 5, out=a)
     ...
     File "pytorch/torch/cuda/_sanitizer.py", line 364, in _handle_kernel_launch
       stack_trace = traceback.StackSummary.extract(
Previous access by stream 0 during kernel:
Previous access by stream 0 during kernel:
   aten::rand(int[] size, *, int? dtype=None, Device? device=None) -> Tensor
   writing to the output
   With stack trace:
     File "example_error.py", line 3, in 
       a = torch.rand(10000, device="cuda")
     ...
     File "pytorch/torch/cuda/_sanitizer.py", line 364, in _handle_kernel_launch
       stack_trace = traceback.StackSummary.extract(
Tensor was allocated with stack trace:
     File "example_error.py", line 3, in 
       a = torch.rand(10000, device="cuda")
     ...
     File "pytorch/torch/cuda/_sanitizer.py", line 420, in _handle_memory_allocation
       traceback.StackSummary.extract(
This gives extensive insight into the origin of the error:


A tensor was incorrectly accessed from streams with ids: 0 (default
  stream) and 94646435460352 (new stream)


The tensor was allocated by invoking "a = torch.rand(10000,
  device="cuda")"


The faulty accesses were caused by operators
The faulty accesses were caused by operators


"a = torch.rand(10000, device="cuda")" on stream 0


"torch.mul(a, 5, out=a)" on stream 94646435460352




The error message also displays the schemas of the invoked
  operators, along with a note showing which arguments of the
  operators correspond to the affected tensor.


In the example, it can be seen that tensor "a" corresponds to
    arguments "self", "out" and the "output" value of the invoked
    operator "torch.mul".


See also:
The list of supported torch operators and their schemas can be
  viewed here.
The bug can be fixed by forcing the new stream to wait for the default
stream:
with torch.cuda.stream(torch.cuda.Stream()):
       torch.cuda.current_stream().wait_stream(torch.cuda.default_stream())
       torch.mul(a, 5, out=a)
When the script is run again, there are no errors reported.
API Reference
torch.cuda._sanitizer.enable_cuda_sanitizer()
Enables CUDA Sanitizer.
Complex Numbers
Note:
When using complex numbers, use Pytorch with CUDA 11.6 downloaded
  via pip wheel as described in Get Started and select the CUDA 11.6
  pip package.
Complex numbers are numbers that can be expressed in the form a + bj,
where a and b are real numbers, and j is called the imaginary unit,
which satisfies the equation j^2 = -1. Complex numbers frequently
occur in mathematics and engineering, especially in topics like signal
processing. Traditionally many users and libraries (e.g., TorchAudio)
have handled complex numbers by representing the data in float tensors
with shape (..., 2) where the last dimension contains the real and
imaginary values.
Tensors of complex dtypes provide a more natural user experience while
working with complex numbers. Operations on complex tensors (e.g.,
"torch.mv()", "torch.matmul()") are likely to be faster and more
memory efficient than operations on float tensors mimicking them.
Operations involving complex numbers in PyTorch are optimized to use
vectorized assembly instructions and specialized kernels (e.g. LAPACK,
cuBlas).
Note:
Spectral operations in the torch.fft module support native complex
  tensors.
Warning:
Complex tensors is a beta feature and subject to change.
Creating Complex Tensors
We support two complex dtypes: torch.cfloat and torch.cdouble



x = torch.randn(2,2, dtype=torch.cfloat)
x
   tensor([[-0.4621-0.0303j, -0.2438-0.5874j],
        [ 0.7706+0.1421j,  1.2110+0.1918j]])



Note:
The default dtype for complex tensors is determined by the default
  floating point dtype. If the default floating point dtype is
  torch.float64 then complex numbers are inferred to have a dtype of
  torch.complex128, otherwise they are assumed to have a dtype of
  torch.complex64.
All factory functions apart from "torch.linspace()",
"torch.logspace()", and "torch.arange()" are supported for complex
tensors.
tensors.
Transition from the old representation
Users who currently worked around the lack of complex tensors with
real tensors of shape (..., 2) can easily to switch using the complex
tensors in their code using "torch.view_as_complex()" and
"torch.view_as_real()". Note that these functions donât perform any
copy and return a view of the input tensor.



x = torch.randn(3, 2)
x
   tensor([[ 0.6125, -0.1681],
        [-0.3773,  1.3487],
        [-0.0861, -0.7981]])
y = torch.view_as_complex(x)
y
   tensor([ 0.6125-0.1681j, -0.3773+1.3487j, -0.0861-0.7981j])
torch.view_as_real(y)
   tensor([[ 0.6125, -0.1681],
        [-0.3773,  1.3487],
        [-0.0861, -0.7981]])



Accessing real and imag
The real and imaginary values of a complex tensor can be accessed
using the "real" and "imag".
Note:
Accessing real and imag attributes doesn't allocate any memory,
and in-place updates on the real and imag tensors will update
  the original complex tensor. Also, the returned real and imag
  tensors are not contiguous.



y.real
   tensor([ 0.6125, -0.3773, -0.0861])
y.imag
   tensor([-0.1681,  1.3487, -0.7981])
y.real.mul_(2)
   tensor([ 1.2250, -0.7546, -0.1722])
y
   tensor([ 1.2250-0.1681j, -0.7546+1.3487j, -0.1722-0.7981j])
y.real.stride()
   (2,)



Angle and abs
The angle and absolute values of a complex tensor can be computed
using "torch.angle()" and "torch.abs()".



x1=torch.tensor([3j, 4+4j])
x1.abs()
   tensor([3.0000, 5.6569])
x1.angle()
   tensor([1.5708, 0.7854])



Linear Algebra
Many linear algebra operations, like "torch.matmul()", "torch.svd()",
"torch.solve()" etc., support complex numbers. If you'd like to
request an operation we don't currently support, please search if an
issue has already been filed and if not, file one.
Serialization
Serialization
Complex tensors can be serialized, allowing data to be saved as
complex values.



torch.save(y, 'complex_tensor.pt')
torch.load('complex_tensor.pt')
   tensor([ 0.6125-0.1681j, -0.3773+1.3487j, -0.0861-0.7981j])



Autograd
PyTorch supports autograd for complex tensors. The gradient computed
is the Conjugate Wirtinger derivative, the negative of which is
precisely the direction of steepest descent used in Gradient Descent
algorithm. Thus, all the existing optimizers work out of the box with
complex parameters. For more details, check out the note Autograd for
Complex Numbers.
We do not fully support the following subsystems:


Quantization


JIT


Sparse Tensors


Distributed


If any of these would help your use case, please search if an issue
has already been filed and if not, file one.