shrinath-suresh/blogs-docs-splitted huggingface dataset:
new_a = {'foo', torch.zeros(()), 'foo_tied': torch.zeros(())}
        >>> functional_call(mod, new_a, torch.zeros()) # tensor(0.)



Parameters:
      * module (torch.nn.Module) -- the module to call
  * **parameters_and_buffers** (*dict of str and Tensor*) -- the
    parameters that will be used in the module call.

  * **args** (*Any** or **tuple*) -- arguments to be passed to the
    module call. If not a tuple, considered a single argument.

  * **kwargs** (*dict*) -- keyword arguments to be passed to the
    module call

  * **tie_weights** (*bool**, **optional*) -- If True, then
    parameters and buffers tied in the original model will be
    treated as tied in the reparamaterized version. Therefore, if
    True and different values are passed for the tied paramaters
    and buffers, it will error. If False, it will not respect the
    originally tied parameters and buffers unless the values
torch.nn.utils.prune.custom_from_mask
torch.nn.utils.prune.custom_from_mask(module, name, mask)
Prunes tensor corresponding to parameter called "name" in "module"
   by applying the pre-computed mask in "mask". Modifies module in
   place (and also return the modified module) by:


adding a named buffer called "name+'_mask'" corresponding to the
      binary mask applied to the parameter "name" by the pruning
      method.


replacing the parameter "name" by its pruned version, while the
      original (unpruned) parameter is stored in a new parameter named
      "name+'_orig'".


Parameters:
      * module (nn.Module) -- module containing the tensor to
        prune
  * **name** (*str*) -- parameter name within "module" on which
    pruning will act.

  * **mask** (*Tensor*) -- binary mask to be applied to the
    parameter.

Returns:
      modified (i.e. pruned) version of the input module
Return type:
torch.autograd.functional.jvp
torch.autograd.functional.jvp(func, inputs, v=None, create_graph=False, strict=False)
Function that computes the dot product between  the Jacobian of the
   given function at the point given by the inputs and a vector "v".
Parameters:
      * func (function) -- a Python function that takes Tensor
        inputs and returns a tuple of Tensors or a Tensor.
  * **inputs** (*tuple of Tensors** or **Tensor*) -- inputs to the
    function "func".

  * **v** (*tuple of Tensors** or **Tensor*) -- The vector for
    which the Jacobian vector product is computed. Must be the
    same size as the input of "func". This argument is optional
    when the input to "func" contains a single element and (if it
    is not provided) will be set as a Tensor containing a single
    "1".

  * **create_graph** (*bool**, **optional*) -- If "True", both the
    output and result will be computed in a differentiable way.
Note that when "strict" is "False", the result can not require
        gradients or be disconnected from the inputs.  Defaults to
        "False".
  * **strict** (*bool**, **optional*) -- If "True", an error will
    be raised when we detect that there exists an input such that
    all the outputs are independent of it. If "False", we return a
    Tensor of zeros as the jvp for said inputs, which is the
    expected mathematical value. Defaults to "False".

Returns:
      tuple with:
         func_output (tuple of Tensors or Tensor): output of
         "func(inputs)"
     jvp (tuple of Tensors or Tensor): result of the dot product
     with the same shape as the output.

Return type:
      output (tuple)
Note:
 "autograd.functional.jvp" computes the jvp by using the backward
 of the backward (sometimes called the double backwards trick).
 This is not the most performant way of computing the jvp. Please
consider using "torch.func.jvp()" or the low-level forward-mode
     AD API instead.
-[ Example ]-



def exp_reducer(x):
   ...     return x.exp().sum(dim=1)
inputs = torch.rand(4, 4)
v = torch.ones(4, 4)
jvp(exp_reducer, inputs, v)
   (tensor([6.3090, 4.6742, 7.9114, 8.2106]),
    tensor([6.3090, 4.6742, 7.9114, 8.2106]))
jvp(exp_reducer, inputs, v, create_graph=True)
   (tensor([6.3090, 4.6742, 7.9114, 8.2106], grad_fn=),
    tensor([6.3090, 4.6742, 7.9114, 8.2106], grad_fn=))
def adder(x, y):
   ...     return 2 * x + 3 * y
inputs = (torch.rand(2), torch.rand(2))
v = (torch.ones(2), torch.ones(2))
jvp(adder, inputs, v)
   (tensor([2.2399, 2.5005]),
    tensor([5., 5.]))
torch.func.grad_and_value
torch.func.grad_and_value(func, argnums=0, has_aux=False)
Returns a function to compute a tuple of the gradient and primal,
   or forward, computation.
Parameters:
      * func (Callable) -- A Python function that takes one or
        more arguments. Must return a single-element Tensor. If
        specified "has_aux" equals "True", function can return a tuple
        of single-element Tensor and other auxiliary objects:
        "(output, aux)".
  * **argnums** (*int** or **Tuple**[**int**]*) -- Specifies
    arguments to compute gradients with respect to. "argnums" can
    be single integer or tuple of integers. Default: 0.

  * **has_aux** (*bool*) -- Flag indicating that "func" returns a
    tensor and other auxiliary objects: "(output, aux)". Default:
    False.

Returns:
      Function to compute a tuple of gradients with respect to its
      inputs and the forward computation. By default, the output of
the function is a tuple of the gradient tensor(s) with respect
      to the first argument and the primal computation. If specified
      "has_aux" equals "True", tuple of gradients and tuple of the
      forward computation with output auxiliary objects is returned.
      If "argnums" is a tuple of integers, a tuple of a tuple of the
      output gradients with respect to each "argnums" value and the
      forward computation is returned.
Return type:
      Callable
See "grad()" for examples
torch.Tensor.size
Tensor.size(dim=None) -> torch.Size or int
Returns the size of the "self" tensor. If "dim" is not specified,
   the returned value is a "torch.Size", a subclass of "tuple". If
   "dim" is specified, returns an int holding the size of that
   dimension.
Parameters:
      dim (int, optional) -- The dimension for which to
      retrieve the size.
Example:
  >>> t = torch.empty(3, 4, 5)
  >>> t.size()
  torch.Size([3, 4, 5])
  >>> t.size(dim=1)
  4
torch.func.hessian
torch.func.hessian(func, argnums=0)
Computes the Hessian of "func" with respect to the arg(s) at index
   "argnum" via a forward-over-reverse strategy.
The forward-over-reverse strategy (composing
   "jacfwd(jacrev(func))") is a good default for good performance. It
   is possible to compute Hessians through other compositions of
   "jacfwd()" and "jacrev()" like "jacfwd(jacfwd(func))" or
   "jacrev(jacrev(func))".
Parameters:
      * func (function) -- A Python function that takes one or
        more arguments, one of which must be a Tensor, and returns one
        or more Tensors
  * **argnums** (*int** or **Tuple**[**int**]*) -- Optional,
    integer or tuple of integers, saying which arguments to get
    the Hessian with respect to. Default: 0.

Returns:
      Returns a function that takes in the same inputs as "func" and
      returns the Hessian of "func" with respect to the arg(s) at
      "argnums".
Note:
"argnums".
Note:
 You may see this API error out with "forward-mode AD not
 implemented for operator X". If so, please file a bug report and
 we will prioritize it. An alternative is to use
 "jacrev(jacrev(func))", which has better operator coverage.

A basic usage with a R^N -> R^1 function gives a N x N Hessian:



from torch.func import hessian
def f(x):
  return x.sin().sum()
x = torch.randn(5)
hess = hessian(f)(x)  # equivalent to jacfwd(jacrev(f))(x)
assert torch.allclose(hess, torch.diag(-x.sin()))
torch.broadcast_tensors
torch.broadcast_tensors(*tensors) -> List of Tensors
Broadcasts the given tensors according to Broadcasting semantics.
Parameters:
      *tensors -- any number of tensors of the same type
Warning:
 More than one element of a broadcasted tensor may refer to a
 single memory location. As a result, in-place operations
 (especially ones that are vectorized) may result in incorrect
 behavior. If you need to write to the tensors, please clone them
 first.

Example:
  >>> x = torch.arange(3).view(1, 3)
  >>> y = torch.arange(2).view(2, 1)
  >>> a, b = torch.broadcast_tensors(x, y)
  >>> a.size()
  torch.Size([2, 3])
  >>> a
  tensor([[0, 1, 2],
          [0, 1, 2]])
ObservationType
class torch.ao.quantization.backend_config.ObservationType(value)
An enum that represents different ways of how an operator/operator
   pattern should be observed
OUTPUT_SHARE_OBSERVER_WITH_INPUT = 1
  this means the output will use the same observer instance as
  input, based on qconfig.activation example: torch.cat, maxpool

OUTPUT_USE_DIFFERENT_OBSERVER_AS_INPUT = 0
  this means input and output are observed with different
  observers, based on qconfig.activation example: conv, linear,
  softmax
torch.jit.script
torch.jit.script(obj, optimize=None, _frames_up=0, _rcb=None, example_inputs=None)
Scripting a function or "nn.Module" will inspect the source code,
   compile it as TorchScript code using the TorchScript compiler, and
   return a "ScriptModule" or "ScriptFunction". TorchScript itself is
   a subset of the Python language, so not all features in Python
   work, but we provide enough functionality to compute on tensors and
   do control-dependent operations. For a complete guide, see the
   TorchScript Language Reference.
Scripting a dictionary or list copies the data inside it into a
   TorchScript instance than can be subsequently passed by reference
   between Python and TorchScript with zero copy overhead.
"torch.jit.script" can be used as a function for modules,
   functions, dictionaries and lists
      and as a decorator "@torch.jit.script" for TorchScript Classes
      and functions.
Parameters:
and functions.
Parameters:
      * obj (Callable, class, or nn.Module) -- The
        "nn.Module", function, class type, dictionary, or list to
        compile.
  * **example_inputs** (*Union**[**List**[**Tuple**]**,
    **Dict**[**Callable**, **List**[**Tuple**]**]**, **None**]*)
    -- Provide example inputs to annotate the arguments for a
    function or "nn.Module".

Returns:
      If "obj" is "nn.Module", "script" returns a "ScriptModule"
      object. The returned "ScriptModule" will have the same set of
      sub-modules and parameters as the original "nn.Module". If "obj"
      is a standalone function, a "ScriptFunction" will be returned.
      If "obj" is a "dict", then "script" returns an instance of
      torch._C.ScriptDict. If "obj" is a "list", then "script"
      returns an instance of torch._C.ScriptList.
Scripting a function
      The "@torch.jit.script" decorator will construct a
"ScriptFunction" by compiling the body of the function.
  Example (scripting a function):

     import torch

     @torch.jit.script
     def foo(x, y):
         if x.max() > y.max():
             r = x
         else:
             r = y
         return r

     print(type(foo))  # torch.jit.ScriptFunction

     # See the compiled graph as Python code
     print(foo.code)

     # Call the function using the TorchScript interpreter
     foo(torch.ones(2, 2), torch.ones(2, 2))

**Scripting a function using example_inputs
      Example inputs can be used to annotate a function arguments.
  Example (annotating a function before scripting):

     import torch

     def test_sum(a, b):
         return a + b

     # Annotate the arguments to be int
     scripted_fn = torch.jit.script(test_sum, example_inputs=[(3, 4)])

     print(type(scripted_fn))  # torch.jit.ScriptFunction
See the compiled graph as Python code
     print(scripted_fn.code)

     # Call the function using the TorchScript interpreter
     scripted_fn(20, 100)

Scripting an nn.Module
      Scripting an "nn.Module" by default will compile the "forward"
      method and recursively compile any methods, submodules, and
      functions called by "forward". If a "nn.Module" only uses
      features supported in TorchScript, no changes to the original
      module code should be necessary. "script" will construct
      "ScriptModule" that has copies of the attributes, parameters,
      and methods of the original module.
  Example (scripting a simple module with a Parameter):

     import torch

     class MyModule(torch.nn.Module):
         def __init__(self, N, M):
             super(MyModule, self).__init__()
             # This parameter will be copied to the new ScriptModule
             self.weight = torch.nn.Parameter(torch.rand(N, M))
When this submodule is used, it will be compiled
             self.linear = torch.nn.Linear(N, M)

         def forward(self, input):
             output = self.weight.mv(input)

             # This calls the `forward` method of the `nn.Linear` module, which will
             # cause the `self.linear` submodule to be compiled to a `ScriptModule` here
             output = self.linear(output)
             return output

     scripted_module = torch.jit.script(MyModule(2, 3))

  Example (scripting a module with traced submodules):

     import torch
     import torch.nn as nn
     import torch.nn.functional as F

     class MyModule(nn.Module):
         def __init__(self):
             super(MyModule, self).__init__()
             # torch.jit.trace produces a ScriptModule's conv1 and conv2
             self.conv1 = torch.jit.trace(nn.Conv2d(1, 20, 5), torch.rand(1, 1, 16, 16))
self.conv2 = torch.jit.trace(nn.Conv2d(20, 20, 5), torch.rand(1, 20, 16, 16))
         def forward(self, input):
             input = F.relu(self.conv1(input))
             input = F.relu(self.conv2(input))
             return input

     scripted_module = torch.jit.script(MyModule())

  To compile a method other than "forward" (and recursively
  compile anything it calls), add the "@torch.jit.export"
  decorator to the method. To opt out of compilation use
  "@torch.jit.ignore" or "@torch.jit.unused".

  Example (an exported and ignored method in a module):

     import torch
     import torch.nn as nn

     class MyModule(nn.Module):
         def __init__(self):
             super(MyModule, self).__init__()

         @torch.jit.export
         def some_entry_point(self, input):
             return input + 10

         @torch.jit.ignore
         def python_only_fn(self, input):
def python_only_fn(self, input):
                 # This function won't be compiled, so any
                 # Python APIs can be used
                 import pdb
                 pdb.set_trace()
         def forward(self, input):
             if self.training:
                 self.python_only_fn(input)
             return input * 99

     scripted_module = torch.jit.script(MyModule())
     print(scripted_module.some_entry_point(torch.randn(2, 2)))
     print(scripted_module(torch.randn(2, 2)))

  Example ( Annotating forward of nn.Module using example_inputs):

     import torch
     import torch.nn as nn
     from typing import NamedTuple

     class MyModule(NamedTuple):
     result: List[int]

     class TestNNModule(torch.nn.Module):
         def forward(self, a) -> MyModule:
             result = MyModule(result=a)
             return result

     pdt_model = TestNNModule()
ConvReLU3d
class torch.ao.nn.intrinsic.qat.ConvReLU3d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True, padding_mode='zeros', qconfig=None)
A ConvReLU3d module is a fused module of Conv3d and ReLU, attached
   with FakeQuantize modules for weight for quantization aware
   training.
We combined the interface of "Conv3d" and "BatchNorm3d".
Variables:
      weight_fake_quant -- fake quant module for weight
torch.nn.functional.softmin
torch.nn.functional.softmin(input, dim=None, _stacklevel=3, dtype=None)
Applies a softmin function.
Note that \text{Softmin}(x) = \text{Softmax}(-x). See softmax
   definition for mathematical formula.
See "Softmin" for more details.
Parameters:
      * input (Tensor) -- input
  * **dim** (*int*) -- A dimension along which softmin will be
    computed (so every slice along dim will sum to 1).

  * **dtype** ("torch.dtype", optional) -- the desired data type
    of returned tensor. If specified, the input tensor is casted
    to "dtype" before the operation is performed. This is useful
    for preventing data type overflows. Default: None.

Return type:
      Tensor
prepare
class torch.quantization.prepare(model, inplace=False, allow_list=None, observer_non_leaf_module_list=None, prepare_custom_config_dict=None)
Prepares a copy of the model for quantization calibration or
   quantization-aware training.
Quantization configuration should be assigned preemptively to
   individual submodules in .qconfig attribute.
The model will be attached with observer or fake quant modules, and
   qconfig will be propagated.
Parameters:
      * model -- input model to be modified in-place
  * **inplace** -- carry out model transformations in-place, the
    original module is mutated

  * **allow_list** -- list of quantizable modules

  * **observer_non_leaf_module_list** -- list of non-leaf modules
    we want to add observer

  * **prepare_custom_config_dict** -- customization configuration
    dictionary for prepare function

  # Example of prepare_custom_config_dict:
  prepare_custom_config_dict = {
GaussianNLLLoss
class torch.nn.GaussianNLLLoss(*, full=False, eps=1e-06, reduction='mean')
Gaussian negative log likelihood loss.
The targets are treated as samples from Gaussian distributions with
   expectations and variances predicted by the neural network. For a
   "target" tensor modelled as having Gaussian distribution with a
   tensor of expectations "input" and a tensor of positive variances
   "var" the loss is:
  \text{loss} =
  \frac{1}{2}\left(\log\left(\text{max}\left(\text{var}, \
  \text{eps}\right)\right) + \frac{\left(\text{input} -
  \text{target}\right)^2} {\text{max}\left(\text{var}, \
  \text{eps}\right)}\right) + \text{const.}

where "eps" is used for stability. By default, the constant term of
   the loss function is omitted unless "full" is "True". If "var" is
   not the same size as "input" (due to a homoscedastic assumption),
   it must either have a final dimension of 1 or have one fewer
dimension (with all other sizes being the same) for correct
   broadcasting.
Parameters:
      * full (bool, optional) -- include the constant term
        in the loss calculation. Default: "False".
  * **eps** (*float**, **optional*) -- value used to clamp "var"
    (see note below), for stability. Default: 1e-6.

  * **reduction** (*str**, **optional*) -- specifies the reduction
    to apply to the output:"'none'" | "'mean'" | "'sum'".
    "'none'": no reduction will be applied, "'mean'": the output
    is the average of all batch member losses, "'sum'": the output
    is the sum of all batch member losses. Default: "'mean'".

Shape:
      * Input: (N, ) or () where * means any number of additional
        dimensions
  * Target: (N, *) or (*), same shape as the input, or same shape
    as the input but with one dimension equal to 1 (to allow for
    broadcasting)

  * Var: (N, *) or (*), same shape as the input, or same shape as
the input but with one dimension equal to 1, or same shape as
        the input but with one fewer dimension (to allow for
        broadcasting)
  * Output: scalar if "reduction" is "'mean'" (default) or
    "'sum'". If "reduction" is "'none'", then (N, *), same shape
    as the input

Examples::
      >>> loss = nn.GaussianNLLLoss()
      >>> input = torch.randn(5, 2, requires_grad=True)
      >>> target = torch.randn(5, 2)
      >>> var = torch.ones(5, 2, requires_grad=True)  # heteroscedastic
      >>> output = loss(input, target, var)
      >>> output.backward()
  >>> loss = nn.GaussianNLLLoss()
  >>> input = torch.randn(5, 2, requires_grad=True)
  >>> target = torch.randn(5, 2)
  >>> var = torch.ones(5, 1, requires_grad=True)  # homoscedastic
  >>> output = loss(input, target, var)
  >>> output.backward()

Note:
 The clamping of "var" is ignored with respect to autograd, and so
 the gradients are unaffected by it.

Reference:
Reference:
      Nix, D. A. and Weigend, A. S., "Estimating the mean and variance
      of the target probability distribution", Proceedings of 1994
      IEEE International Conference on Neural Networks (ICNN'94),
      Orlando, FL, USA, 1994, pp. 55-60 vol.1, doi:
      10.1109/ICNN.1994.374138.
convert_fx
class torch.quantization.quantize_fx.convert_fx(graph_module, convert_custom_config=None, _remove_qconfig=True, qconfig_mapping=None, backend_config=None)
Convert a calibrated or trained model to a quantized model
Parameters:
      * graph_module (***) -- A prepared and calibrated/trained
        model (GraphModule)
  * **convert_custom_config** (***) -- custom configurations for
    convert function. See "ConvertCustomConfig" for more details

  * **_remove_qconfig** (***) -- Option to remove the qconfig
    attributes in the model after convert.

  * **qconfig_mapping** (***) --

    config for specifying how to convert a model for quantization.

          The keys must include the ones in the qconfig_mapping
          passed to *prepare_fx* or *prepare_qat_fx*, with the
          same values or *None*. Additional keys can be specified
          with values set to *None*.
with values set to None.
       For each entry whose value is set to None, we skip
       quantizing that entry in the model:

          qconfig_mapping = QConfigMapping
              .set_global(qconfig_from_prepare)
              .set_object_type(torch.nn.functional.add, None)  # skip quantizing torch.nn.functional.add
              .set_object_type(torch.nn.functional.linear, qconfig_from_prepare)
              .set_module_name("foo.bar", None)  # skip quantizing module "foo.bar"

    * *backend_config* (BackendConfig): A configuration for the
      backend which describes how
         operators should be quantized in the backend, this
         includes quantization mode support
         (static/dynamic/weight_only), dtype support (quint8/qint8
         etc.), observer placement for each operators and fused
         operators. See "BackendConfig" for more details

Returns:
      A quantized model (torch.nn.Module)
Return type:
Return type:
      Module
Example:
  # prepared_model: the model after prepare_fx/prepare_qat_fx and calibration/training
  # convert_fx converts a calibrated/trained model to a quantized model for the
  # target hardware, this includes converting the model first to a reference
  # quantized model, and then lower the reference quantized model to a backend
  # Currently, the supported backends are fbgemm (onednn), qnnpack (xnnpack) and
  # they share the same set of quantized operators, so we are using the same
  # lowering procedure
  #
  # backend_config defines the corresponding reference quantized module for
  # the weighted modules in the model, e.g. nn.Linear
  # TODO: add backend_config after we split the backend_config for fbgemm and qnnpack
  # e.g. backend_config = get_default_backend_config("fbgemm")
  quantized_model = convert_fx(prepared_model)
torch.gcd
torch.gcd(input, other, *, out=None) -> Tensor
Computes the element-wise greatest common divisor (GCD) of "input"
   and "other".
Both "input" and "other" must have integer types.
Note:
 This defines gcd(0, 0) = 0.

Parameters:
      * input (Tensor) -- the input tensor.
  * **other** (*Tensor*) -- the second input tensor

Keyword Arguments:
      out (Tensor, optional) -- the output tensor.
Example:
  >>> a = torch.tensor([5, 10, 15])
  >>> b = torch.tensor([3, 4, 5])
  >>> torch.gcd(a, b)
  tensor([1, 2, 5])
  >>> c = torch.tensor([3])
  >>> torch.gcd(a, c)
  tensor([1, 1, 3])
torch.Tensor.log_normal_
Tensor.log_normal_(mean=1, std=2, *, generator=None)
Fills "self" tensor with numbers samples from the log-normal
   distribution parameterized by the given mean \mu and standard
   deviation \sigma. Note that "mean" and "std" are the mean and
   standard deviation of the underlying normal distribution, and not
   of the returned distribution:
  f(x) = \dfrac{1}{x \sigma \sqrt{2\pi}}\ e^{-\frac{(\ln x -
  \mu)^2}{2\sigma^2}}