shrinath-suresh/blogs-docs-splitted huggingface dataset:
torch.jit.load
torch.jit.load(f, map_location=None, _extra_files=None)
Load a "ScriptModule" or "ScriptFunction" previously saved with
   "torch.jit.save"
All previously saved modules, no matter their device, are first
   loaded onto CPU, and then are moved to the devices they were saved
   from. If this fails (e.g. because the run time system doesn't have
   certain devices), an exception is raised.
Parameters:
      * f -- a file-like object (has to implement read, readline,
        tell, and seek), or a string containing a file name
  * **map_location** (*string** or **torch.device*) -- A
    simplified version of "map_location" in *torch.jit.save* used
    to dynamically remap storages to an alternative set of
    devices.

  * **_extra_files** (*dictionary of filename to content*) -- The
    extra filenames given in the map would be loaded and their
    content would be stored in the provided map.

Returns:
      A "ScriptModule" object.
Returns:
      A "ScriptModule" object.
Example:
  import torch
  import io

  torch.jit.load('scriptmodule.pt')

  # Load ScriptModule from io.BytesIO object
  with open('scriptmodule.pt', 'rb') as f:
      buffer = io.BytesIO(f.read())

  # Load all tensors to the original device
  torch.jit.load(buffer)

  # Load all tensors onto CPU, using a device
  buffer.seek(0)
  torch.jit.load(buffer, map_location=torch.device('cpu'))

  # Load all tensors onto CPU, using a string
  buffer.seek(0)
  torch.jit.load(buffer, map_location='cpu')

  # Load with extra files.
  extra_files = {'foo.txt': ''}  # values will be replaced with data
  torch.jit.load('scriptmodule.pt', _extra_files=extra_files)
  print(extra_files['foo.txt'])
torch.complex
torch.complex(real, imag, *, out=None) -> Tensor
Constructs a complex tensor with its real part equal to "real" and
   its imaginary part equal to "imag".
Parameters:
      * real (Tensor) -- The real part of the complex tensor.
        Must be float or double.
  * **imag** (*Tensor*) -- The imaginary part of the complex
    tensor. Must be same dtype as "real".

Keyword Arguments:
      out (Tensor) -- If the inputs are "torch.float32", must be
      "torch.complex64". If the inputs are "torch.float64", must be
      "torch.complex128".
Example:
  >>> real = torch.tensor([1, 2], dtype=torch.float32)
  >>> imag = torch.tensor([3, 4], dtype=torch.float32)
  >>> z = torch.complex(real, imag)
  >>> z
  tensor([(1.+3.j), (2.+4.j)])
  >>> z.dtype
  torch.complex64
torch.lcm
torch.lcm(input, other, *, out=None) -> Tensor
Computes the element-wise least common multiple (LCM) of "input"
   and "other".
Both "input" and "other" must have integer types.
Note:
 This defines lcm(0, 0) = 0 and lcm(0, a) = 0.

Parameters:
      * input (Tensor) -- the input tensor.
  * **other** (*Tensor*) -- the second input tensor

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.tensor([5, 10, 15])
  >>> b = torch.tensor([3, 4, 5])
  >>> torch.lcm(a, b)
  tensor([15, 20, 15])
  >>> c = torch.tensor([3])
  >>> torch.lcm(a, c)
  tensor([15, 30, 15])
torch.isposinf
torch.isposinf(input, *, out=None) -> Tensor
Tests if each element of "input" is positive infinity or not.
Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.tensor([-float('inf'), float('inf'), 1.2])
  >>> torch.isposinf(a)
  tensor([False,  True, False])
ConvBn1d
class torch.ao.nn.intrinsic.qat.ConvBn1d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=None, padding_mode='zeros', eps=1e-05, momentum=0.1, freeze_bn=False, qconfig=None)
A ConvBn1d module is a module fused from Conv1d and BatchNorm1d,
   attached with FakeQuantize modules for weight, used in quantization
   aware training.
We combined the interface of "torch.nn.Conv1d" and
   "torch.nn.BatchNorm1d".
Similar to "torch.nn.Conv1d", with FakeQuantize modules initialized
   to default.
Variables:
      * freeze_bn --
  * **weight_fake_quant** -- fake quant module for weight
RNN
class torch.nn.RNN(args, *kwargs)
Applies a multi-layer Elman RNN with \tanh or \text{ReLU} non-
   linearity to an input sequence.
For each element in the input sequence, each layer computes the
   following function:
  h_t = \tanh(x_t W_{ih}^T + b_{ih} + h_{t-1}W_{hh}^T + b_{hh})

where h_t is the hidden state at time t, x_t is the input at time
   t, and h_{(t-1)} is the hidden state of the previous layer at
   time t-1 or the initial hidden state at time 0. If
   "nonlinearity" is "'relu'", then \text{ReLU} is used instead of
   \tanh.
Parameters:
      * input_size -- The number of expected features in the input
        x
  * **hidden_size** -- The number of features in the hidden state
    *h*

  * **num_layers** -- Number of recurrent layers. E.g., setting
    "num_layers=2" would mean stacking two RNNs together to form a
    *stacked RNN*, with the second RNN taking in outputs of the
first RNN and computing the final results. Default: 1
  * **nonlinearity** -- The non-linearity to use. Can be either
    "'tanh'" or "'relu'". Default: "'tanh'"

  * **bias** -- If "False", then the layer does not use bias
    weights *b_ih* and *b_hh*. Default: "True"

  * **batch_first** -- If "True", then the input and output
    tensors are provided as *(batch, seq, feature)* instead of
    *(seq, batch, feature)*. Note that this does not apply to
    hidden or cell states. See the Inputs/Outputs sections below
    for details.  Default: "False"

  * **dropout** -- If non-zero, introduces a *Dropout* layer on
    the outputs of each RNN layer except the last layer, with
    dropout probability equal to "dropout". Default: 0

  * **bidirectional** -- If "True", becomes a bidirectional RNN.
    Default: "False"

Inputs: input, h_0
      * input: tensor of shape (L, H_{in}) for unbatched input,
(L, N, H_{in}) when "batch_first=False" or (N, L, H_{in}) when
        "batch_first=True" containing the features of the input
        sequence.  The input can also be a packed variable length
        sequence. See "torch.nn.utils.rnn.pack_padded_sequence()" or
        "torch.nn.utils.rnn.pack_sequence()" for details.
  * **h_0**: tensor of shape (D * \text{num\_layers}, H_{out}) for
    unbatched input or (D * \text{num\_layers}, N, H_{out})
    containing the initial hidden state for the input sequence
    batch. Defaults to zeros if not provided.

  where:

     \begin{aligned}     N ={} & \text{batch size} \\     L ={} &
     \text{sequence length} \\     D ={} & 2 \text{ if
     bidirectional=True otherwise } 1 \\     H_{in} ={} &
     \text{input\_size} \\     H_{out} ={} & \text{hidden\_size}
     \end{aligned}

Outputs: output, h_n
      * output: tensor of shape (L, D * H_{out}) for unbatched
input, (L, N, D * H_{out}) when "batch_first=False" or (N, L,
        D * H_{out}) when "batch_first=True" containing the output
        features (h_t) from the last layer of the RNN, for each t.
        If a "torch.nn.utils.rnn.PackedSequence" has been given as the
        input, the output will also be a packed sequence.
  * **h_n**: tensor of shape (D * \text{num\_layers}, H_{out}) for
    unbatched input or (D * \text{num\_layers}, N, H_{out})
    containing the final hidden state for each element in the
    batch.

Variables:
      * weight_ih_l[k] -- the learnable input-hidden weights of
        the k-th layer, of shape (hidden_size, input_size) for k =
        0. Otherwise, the shape is (hidden_size, num_directions *
        hidden_size)
  * **weight_hh_l[k]** -- the learnable hidden-hidden weights of
    the k-th layer, of shape *(hidden_size, hidden_size)*

  * **bias_ih_l[k]** -- the learnable input-hidden bias of the
k-th layer, of shape (hidden_size)
  * **bias_hh_l[k]** -- the learnable hidden-hidden bias of the
    k-th layer, of shape *(hidden_size)*

Note:
 All the weights and biases are initialized from
 \mathcal{U}(-\sqrt{k}, \sqrt{k}) where k =
 \frac{1}{\text{hidden\_size}}

Note:
 For bidirectional RNNs, forward and backward are directions 0 and
 1 respectively. Example of splitting the output layers when
 "batch_first=False": "output.view(seq_len, batch, num_directions,
 hidden_size)".

Note:
 "batch_first" argument is ignored for unbatched inputs.

Warning:
 There are known non-determinism issues for RNN functions on some
 versions of cuDNN and CUDA. You can enforce deterministic
 behavior by setting the following environment variables:On CUDA
 10.1, set environment variable "CUDA_LAUNCH_BLOCKING=1". This may
 affect performance.On CUDA 10.2 or later, set environment
 variable (note the leading colon symbol)
variable (note the leading colon symbol)
     "CUBLAS_WORKSPACE_CONFIG=:16:8" or
     "CUBLAS_WORKSPACE_CONFIG=:4096:2".See the cuDNN 8 Release Notes
     for more information.
Note:
 If the following conditions are satisfied: 1) cudnn is enabled,
 2) input data is on the GPU 3) input data has dtype
 "torch.float16" 4) V100 GPU is used, 5) input data is not in
 "PackedSequence" format persistent algorithm can be selected to
 improve performance.

Examples:
  >>> rnn = nn.RNN(10, 20, 2)
  >>> input = torch.randn(5, 3, 10)
  >>> h0 = torch.randn(2, 3, 20)
  >>> output, hn = rnn(input, h0)
torch.deg2rad
torch.deg2rad(input, *, out=None) -> Tensor
Returns a new tensor with each of the elements of "input" converted
   from angles in degrees to radians.
Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.tensor([[180.0, -180.0], [360.0, -360.0], [90.0, -90.0]])
  >>> torch.deg2rad(a)
  tensor([[ 3.1416, -3.1416],
          [ 6.2832, -6.2832],
          [ 1.5708, -1.5708]])
torch.rand
torch.rand(size, , generator=None, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False, pin_memory=False) -> Tensor
Returns a tensor filled with random numbers from a uniform
   distribution on the interval [0, 1)
The shape of the tensor is defined by the variable argument "size".
Parameters:
      size (int...) -- a sequence of integers defining the
      shape of the output tensor. Can be a variable number of
      arguments or a collection like a list or tuple.
Keyword Arguments:
      * generator ("torch.Generator", optional) -- a pseudorandom
        number generator for sampling
  * **out** (*Tensor**, **optional*) -- the output tensor.

  * **dtype** ("torch.dtype", optional) -- the desired data type
    of returned tensor. Default: if "None", uses a global default
    (see "torch.set_default_tensor_type()").

  * **layout** ("torch.layout", optional) -- the desired layout of
returned Tensor. Default: "torch.strided".
  * **device** ("torch.device", optional) -- the desired device of
    returned tensor. Default: if "None", uses the current device
    for the default tensor type (see
    "torch.set_default_tensor_type()"). "device" will be the CPU
    for CPU tensor types and the current CUDA device for CUDA
    tensor types.

  * **requires_grad** (*bool**, **optional*) -- If autograd should
    record operations on the returned tensor. Default: "False".

  * **pin_memory** (*bool**, **optional*) -- If set, returned
    tensor would be allocated in the pinned memory. Works only for
    CPU tensors. Default: "False".

Example:
  >>> torch.rand(4)
  tensor([ 0.5204,  0.2503,  0.3525,  0.5673])
  >>> torch.rand(2, 3)
  tensor([[ 0.8237,  0.5781,  0.6879],
          [ 0.3816,  0.7249,  0.0998]])
check_sparse_tensor_invariants
class torch.sparse.check_sparse_tensor_invariants(enable=True)
A tool to control checking sparse tensor invariants.
The following options exists to manage sparsr tensor invariants
   checking in sparse tensor construction:


Using a context manager:
 with torch.sparse.check_sparse_tensor_invariants():
     run_my_model()



Using a procedural approach:
 prev_checks_enabled = torch.sparse.check_sparse_tensor_invariants.is_enabled()
 torch.sparse.check_sparse_tensor_invariants.enable()

 run_my_model()

 if not prev_checks_enabled:
     torch.sparse.check_sparse_tensor_invariants.disable()



Using function decoration:
 @torch.sparse.check_sparse_tensor_invariants()
 def run_my_model():
     ...

 run_my_model()



Using "check_invariants" keyword argument in sparse tensor
      constructor call. For example:
constructor call. For example:
  >>> torch.sparse_csr_tensor([0, 1, 3], [0, 1], [1, 2], check_invariants=True)
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
  RuntimeError: `crow_indices[..., -1] == nnz` is not satisfied.

static disable()
  Disable sparse tensor invariants checking in sparse tensor
  constructors.

  See "torch.sparse.check_sparse_tensor_invariants.enable()" for
  more information.

static enable()
  Enable sparse tensor invariants checking in sparse tensor
  constructors.

  Note:

    By default, the sparse tensor invariants checks are disabled.
    Use "torch.sparse.check_sparse_tensor_invariants.is_enabled()"
    to retrieve the current state of sparse tensor invariants
    checking.

  Note:

    The sparse tensor invariants check flag is effective to all
    sparse tensor constructors, both in Python and ATen.The flag
can be locally overridden by the "check_invariants" optional
        argument of the sparse tensor constructor functions.
static is_enabled()
  Returns True if the sparse tensor invariants checking is
  enabled.

  Note:

    Use "torch.sparse.check_sparse_tensor_invariants.enable()" or
    "torch.sparse.check_sparse_tensor_invariants.disable()" to
    manage the state of the sparse tensor invariants checks.
torch.sin
torch.sin(input, *, out=None) -> Tensor
Returns a new tensor with the sine of the elements of "input".
  \text{out}_{i} = \sin(\text{input}_{i})

Parameters:
      input (Tensor) -- the input tensor.
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.randn(4)
  >>> a
  tensor([-0.5461,  0.1347, -2.7266, -0.2746])
  >>> torch.sin(a)
  tensor([-0.5194,  0.1343, -0.4032, -0.2711])
torch.autograd.graph.Node.register_prehook
abstract Node.register_prehook(fn)
Registers a backward pre-hook.
The hook will be called every time a gradient with respect to the
   Node is computed. The hook should have the following signature:
  hook(grad_outputs: Tuple[Tensor]) -> Tuple[Tensor] or None

The hook should not modify its argument, but it can optionally
   return a new gradient which will be used in place of
   "grad_outputs".
This function returns a handle with a method "handle.remove()" that
   removes the hook from the module.
Note:
 See Backward Hooks execution for more information on how when
 this hook is executed, and how its execution is ordered relative
 to other hooks.

Example:
  >>> a = torch.tensor([0., 0., 0.], requires_grad=True)
  >>> b = a.clone()
  >>> assert isinstance(b.grad_fn, torch.autograd.graph.Node)
  >>> handle = b.grad_fn.register_prehook(lambda gI: (gI[0] * 2,))
torch.nn.utils.rnn.pack_padded_sequence
torch.nn.utils.rnn.pack_padded_sequence(input, lengths, batch_first=False, enforce_sorted=True)
Packs a Tensor containing padded sequences of variable length.
"input" can be of size "T x B x " where T is the length of the
   longest sequence (equal to "lengths[0]"), "B" is the batch size,
   and "" is any number of dimensions (including 0). If "batch_first"
   is "True", "B x T x *" "input" is expected.
For unsorted sequences, use enforce_sorted = False. If
   "enforce_sorted" is "True", the sequences should be sorted by
   length in a decreasing order, i.e. "input[:,0]" should be the
   longest sequence, and "input[:,B-1]" the shortest one.
   enforce_sorted = True is only necessary for ONNX export.
Note:
 This function accepts any input that has at least two dimensions.
 You can apply it to pack the labels, and use the output of the
RNN with them to compute the loss directly. A Tensor can be
     retrieved from a "PackedSequence" object by accessing its ".data"
     attribute.
Parameters:
      * input (Tensor) -- padded batch of variable length
        sequences.
  * **lengths** (*Tensor** or **list**(**int**)*) -- list of
    sequence lengths of each batch element (must be on the CPU if
    provided as a tensor).

  * **batch_first** (*bool**, **optional*) -- if "True", the input
    is expected in "B x T x *" format.

  * **enforce_sorted** (*bool**, **optional*) -- if "True", the
    input is expected to contain sequences sorted by length in a
    decreasing order. If "False", the input will get sorted
    unconditionally. Default: "True".

Returns:
      a "PackedSequence" object
Return type:
      PackedSequence
torch.nn.utils.clip_grad_value_
torch.nn.utils.clip_grad_value_(parameters, clip_value, foreach=None)
Clips gradient of an iterable of parameters at specified value.
Gradients are modified in-place.
Parameters:
      * parameters (Iterable[Tensor] or Tensor) -- an
        iterable of Tensors or a single Tensor that will have
        gradients normalized
  * **clip_value** (*float*) -- maximum allowed value of the
    gradients. The gradients are clipped in the range
    \left[\text{-clip\_value}, \text{clip\_value}\right]

  * **foreach** (*bool*) -- use the faster foreach-based
    implementation If "None", use the foreach implementation for
    CUDA and CPU tensors and silently fall back to the slow
    implementation for other device types. Default: "None"
torch.autograd.functional.hessian
torch.autograd.functional.hessian(func, inputs, create_graph=False, strict=False, vectorize=False, outer_jacobian_strategy='reverse-mode')
Function that computes the Hessian of a given scalar function.
Parameters:
      * func (function) -- a Python function that takes Tensor
        inputs and returns a Tensor with a single element.
  * **inputs** (*tuple of Tensors** or **Tensor*) -- inputs to the
    function "func".

  * **create_graph** (*bool**, **optional*) -- If "True", the
    Hessian will be computed in a differentiable manner. Note that
    when "strict" is "False", the result can not require gradients
    or be disconnected from the inputs. Defaults to "False".

  * **strict** (*bool**, **optional*) -- If "True", an error will
    be raised when we detect that there exists an input such that
    all the outputs are independent of it. If "False", we return a
Tensor of zeros as the hessian for said inputs, which is the
        expected mathematical value. Defaults to "False".
  * **vectorize** (*bool**, **optional*) -- This feature is
    experimental. Please consider using "torch.func.hessian()"
    instead if you are looking for something less experimental and
    more performant. When computing the hessian, usually we invoke
    "autograd.grad" once per row of the hessian. If this flag is
    "True", we use the vmap prototype feature as the backend to
    vectorize calls to "autograd.grad" so we only invoke it once
    instead of once per row. This should lead to performance
    improvements in many use cases, however, due to this feature
    being incomplete, there may be performance cliffs. Please use
    *torch._C._debug_only_display_vmap_fallback_warnings(True)* to
    show any performance warnings and file us issues if warnings
    exist for your use case. Defaults to "False".
outer_jacobian_strategy (str, optional) -- The
        Hessian is computed by computing the Jacobian of a Jacobian.
        The inner Jacobian is always computed in reverse-mode AD.
        Setting strategy to ""forward-mode"" or ""reverse-mode""
        determines whether the outer Jacobian will be computed with
        forward or reverse mode AD. Currently, computing the outer
        Jacobian in ""forward-mode"" requires "vectorized=True".
        Defaults to ""reverse-mode"".

Returns:
      if there is a single input, this will be a single Tensor
      containing the Hessian for the input. If it is a tuple, then the
      Hessian will be a tuple of tuples where "Hessian[i][j]" will
      contain the Hessian of the "i"th input and "j"th input with size
      the sum of the size of the "i"th input plus the size of the
      "j"th input. "Hessian[i][j]" will have the same dtype and device
      as the corresponding "i"th input.
Return type:
Return type:
      Hessian (Tensor or a tuple of tuple of Tensors)
-[ Example ]-



def pow_reducer(x):
   ...     return x.pow(3).sum()
inputs = torch.rand(2, 2)
hessian(pow_reducer, inputs)
   tensor([[[[5.2265, 0.0000],
             [0.0000, 0.0000]],
            [[0.0000, 4.8221],
             [0.0000, 0.0000]]],
           [[[0.0000, 0.0000],
             [1.9456, 0.0000]],
            [[0.0000, 0.0000],
             [0.0000, 3.2550]]]])
hessian(pow_reducer, inputs, create_graph=True)
   tensor([[[[5.2265, 0.0000],
             [0.0000, 0.0000]],
            [[0.0000, 4.8221],
             [0.0000, 0.0000]]],
           [[[0.0000, 0.0000],
             [1.9456, 0.0000]],
            [[0.0000, 0.0000],
             [0.0000, 3.2550]]]], grad_fn=)
def pow_adder_reducer(x, y):
   ...     return (2 * x.pow(2) + 3 * y.pow(2)).sum()
inputs = (torch.rand(2), torch.rand(2))
hessian(pow_adder_reducer, inputs)
   ((tensor([[4., 0.],
torch.hstack
torch.hstack(tensors, *, out=None) -> Tensor
Stack tensors in sequence horizontally (column wise).
This is equivalent to concatenation along the first axis for 1-D
   tensors, and along the second axis for all other tensors.
Parameters:
      tensors (sequence of Tensors) -- sequence of tensors to
      concatenate
Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.tensor([1, 2, 3])
  >>> b = torch.tensor([4, 5, 6])
  >>> torch.hstack((a,b))
  tensor([1, 2, 3, 4, 5, 6])
  >>> a = torch.tensor([[1],[2],[3]])
  >>> b = torch.tensor([[4],[5],[6]])
  >>> torch.hstack((a,b))
  tensor([[1, 4],
          [2, 5],
          [3, 6]])
torch.vmap
torch.vmap(func, in_dims=0, out_dims=0, randomness='error', *, chunk_size=None)
vmap is the vectorizing map; "vmap(func)" returns a new function
   that maps "func" over some dimension of the inputs. Semantically,
   vmap pushes the map into PyTorch operations called by "func",
   effectively vectorizing those operations.
vmap is useful for handling batch dimensions: one can write a
   function "func" that runs on examples and then lift it to a
   function that can take batches of examples with "vmap(func)". vmap
   can also be used to compute batched gradients when composed with
   autograd.
Note:
 "torch.vmap()" is aliased to "torch.func.vmap()" for convenience.
 Use whichever one you'd like.

Parameters:
      * func (function) -- A Python function that takes one or
        more arguments. Must return one or more Tensors.
  * **in_dims** (*int** or **nested structure*) -- Specifies which
torch.Tensor.numpy
Tensor.numpy(*, force=False) -> numpy.ndarray
Returns the tensor as a NumPy "ndarray".
If "force" is "False" (the default), the conversion is performed
   only if the tensor is on the CPU, does not require grad, does not
   have its conjugate bit set, and is a dtype and layout that NumPy
   supports. The returned ndarray and the tensor will share their
   storage, so changes to the tensor will be reflected in the ndarray
   and vice versa.
If "force" is "True" this is equivalent to calling
   "t.detach().cpu().resolve_conj().resolve_neg().numpy()". If the
   tensor isn't on the CPU or the conjugate or negative bit is set,
   the tensor won't share its storage with the returned ndarray.
   Setting "force" to "True" can be a useful shorthand.
Parameters:
      force (bool) -- if "True", the ndarray may be a copy of
      the tensor instead of always sharing memory, defaults to
      "False".