shrinath-suresh/blogs-docs-splitted huggingface dataset:
from torch.func import jacfwd, vmap
x = torch.randn(64, 5)
jacobian = vmap(jacfwd(torch.sin))(x)
assert jacobian.shape == (64, 5, 5)



If you would like to compute the output of the function as well as
   the jacobian of the function, use the "has_aux" flag to return the
   output as an auxiliary object:



from torch.func import jacfwd
x = torch.randn(5)
def f(x):
  return x.sin()
def g(x):
  result = f(x)
  return result, result
jacobian_f, f_x = jacfwd(g, has_aux=True)(x)
assert torch.allclose(f_x, f(x))



Additionally, "jacrev()" can be composed with itself or "jacrev()"
   to produce Hessians



from torch.func import jacfwd, jacrev
def f(x):
  return x.sin().sum()
x = torch.randn(5)
hessian = jacfwd(jacrev(f))(x)
assert torch.allclose(hessian, torch.diag(-x.sin()))



By default, "jacfwd()" computes the Jacobian with respect to the
first input. However, it can compute the Jacboian with respect to a
   different argument by using "argnums":



from torch.func import jacfwd
def f(x, y):
  return x + y ** 2
x, y = torch.randn(5), torch.randn(5)
jacobian = jacfwd(f, argnums=1)(x, y)
expected = torch.diag(2 * y)
assert torch.allclose(jacobian, expected)



Additionally, passing a tuple to "argnums" will compute the
   Jacobian with respect to multiple arguments



from torch.func import jacfwd
def f(x, y):
  return x + y ** 2
x, y = torch.randn(5), torch.randn(5)
jacobian = jacfwd(f, argnums=(0, 1))(x, y)
expectedX = torch.diag(torch.ones_like(x))
expectedY = torch.diag(2 * y)
assert torch.allclose(jacobian[0], expectedX)
assert torch.allclose(jacobian[1], expectedY)
torch.linalg.solve_ex
torch.linalg.solve_ex(A, B, *, left=True, check_errors=False, out=None)
A version of "solve()" that does not perform error checks unless
   "check_errors"= True. It also returns the "info" tensor returned
   by LAPACK's getrf.
Note:
 When the inputs are on a CUDA device, this function synchronizes
 only when "check_errors"*= True*.

Warning:
 This function is "experimental" and it may change in a future
 PyTorch release.

Parameters:
      A (Tensor) -- tensor of shape (, n, n)* where *** is
      zero or more batch dimensions.
Keyword Arguments:
      * left (bool, optional) -- whether to solve the system
        AX=B or XA = B. Default: True.
  * **check_errors** (*bool**, **optional*) -- controls whether to
    check the content of "infos" and raise an error if it is non-
    zero. Default: *False*.

  * **out** (*tuple**, **optional*) -- tuple of two tensors to
write the output to. Ignored if None. Default: None.
Returns:
      A named tuple (result, info).
Examples:
  >>> A = torch.randn(3, 3)
  >>> Ainv, info = torch.linalg.solve_ex(A)
  >>> torch.dist(torch.linalg.inv(A), Ainv)
  tensor(0.)
  >>> info
  tensor(0, dtype=torch.int32)
MaxPool2d
class torch.nn.MaxPool2d(kernel_size, stride=None, padding=0, dilation=1, return_indices=False, ceil_mode=False)
Applies a 2D max pooling over an input signal composed of several
   input planes.
In the simplest case, the output value of the layer with input size
   (N, C, H, W), output (N, C, H_{out}, W_{out}) and "kernel_size"
   (kH, kW) can be precisely described as:
  \begin{aligned}     out(N_i, C_j, h, w) ={} & \max_{m=0, \ldots,
  kH-1} \max_{n=0, \ldots, kW-1} \\                             &
  \text{input}(N_i, C_j, \text{stride[0]} \times h + m,
  \text{stride[1]} \times w + n) \end{aligned}

If "padding" is non-zero, then the input is implicitly padded with
   negative infinity on both sides for "padding" number of points.
   "dilation" controls the spacing between the kernel points. It is
   harder to describe, but this link has a nice visualization of what
   "dilation" does.
Note:
"dilation" does.
Note:
 When ceil_mode=True, sliding windows are allowed to go off-bounds
 if they start within the left padding or the input. Sliding
 windows that would start in the right padded region are ignored.

The parameters "kernel_size", "stride", "padding", "dilation" can
   either be:
  * a single "int" -- in which case the same value is used for the
    height and width dimension

  * a "tuple" of two ints -- in which case, the first *int* is
    used for the height dimension, and the second *int* for the
    width dimension

Parameters:
      * kernel_size (Union[int, Tuple[int,
        int]]) -- the size of the window to take a max over
  * **stride** (*Union**[**int**, **Tuple**[**int**, **int**]**]*)
    -- the stride of the window. Default value is "kernel_size"

  * **padding** (*Union**[**int**, **Tuple**[**int**,
    **int**]**]*) -- Implicit negative infinity padding to be
added on both sides
  * **dilation** (*Union**[**int**, **Tuple**[**int**,
    **int**]**]*) -- a parameter that controls the stride of
    elements in the window

  * **return_indices** (*bool*) -- if "True", will return the max
    indices along with the outputs. Useful for
    "torch.nn.MaxUnpool2d" later

  * **ceil_mode** (*bool*) -- when True, will use *ceil* instead
    of *floor* to compute the output shape

Shape:
      * Input: (N, C, H_{in}, W_{in}) or (C, H_{in}, W_{in})
  * Output: (N, C, H_{out}, W_{out}) or (C, H_{out}, W_{out}),
    where

       H_{out} = \left\lfloor\frac{H_{in} + 2 * \text{padding[0]}
       - \text{dilation[0]}       \times (\text{kernel\_size[0]} -
       1) - 1}{\text{stride[0]}} + 1\right\rfloor

       W_{out} = \left\lfloor\frac{W_{in} + 2 * \text{padding[1]}
       - \text{dilation[1]}       \times (\text{kernel\_size[1]} -
       1) - 1}{\text{stride[1]}} + 1\right\rfloor

Examples:
torch.jit.fork
torch.jit.fork(func, args, *kwargs)
Creates an asynchronous task executing func and a reference to
   the value of the result of this execution. fork will return
   immediately, so the return value of func may not have been
   computed yet. To force completion of the task and access the return
   value invoke torch.jit.wait on the Future. fork invoked with a
   func which returns T is typed as torch.jit.Future[T]. fork
   calls can be arbitrarily nested, and may be invoked with positional
   and keyword arguments. Asynchronous execution will only occur when
   run in TorchScript. If run in pure python, fork will not execute
   in parallel. fork will also not execute in parallel when invoked
   while tracing, however the fork and wait calls will be captured
   in the exported IR Graph.
Warning:
 *fork* tasks will execute non-deterministically. We recommend
 only spawning parallel fork tasks for pure functions that do not
modify their inputs, module attributes, or global state.
Parameters:
      * func (callable or torch.nn.Module) -- A Python
        function or torch.nn.Module that will be invoked. If
        executed in TorchScript, it will execute asynchronously,
        otherwise it will not. Traced invocations of fork will be
        captured in the IR.
  * ***args** -- arguments to invoke *func* with.

  * ****kwargs** -- arguments to invoke *func* with.

Returns:
      a reference to the execution of func. The value T can only
      be accessed by forcing completion of func through
      torch.jit.wait.
Return type:
      torch.jit.Future[T]
Example (fork a free function):
  import torch
  from torch import Tensor
  def foo(a : Tensor, b : int) -> Tensor:
      return a + b
  def bar(a):
      fut : torch.jit.Future[Tensor] = torch.jit.fork(foo, a, b=2)
      return torch.jit.wait(fut)
  script_bar = torch.jit.script(bar)
script_bar = torch.jit.script(bar)
      input = torch.tensor(2)
      # only the scripted version executes asynchronously
      assert script_bar(input) == bar(input)
      # trace is not run asynchronously, but fork is captured in IR
      graph = torch.jit.trace(bar, (input,)).graph
      assert "fork" in str(graph)
Example (fork a module method):
  import torch
  from torch import Tensor
  class AddMod(torch.nn.Module):
      def forward(self, a: Tensor, b : int):
          return a + b
  class Mod(torch.nn.Module):
      def __init__(self):
          super(self).__init__()
          self.mod = AddMod()
      def forward(self, input):
          fut = torch.jit.fork(self.mod, a, b=2)
          return torch.jit.wait(fut)
  input = torch.tensor(2)
  mod = Mod()
  assert mod(input) == torch.jit.script(mod).forward(input)
Parameter
class torch.nn.parameter.Parameter(data=None, requires_grad=True)
A kind of Tensor that is to be considered a module parameter.
Parameters are "Tensor" subclasses, that have a very special
   property when used with "Module" s - when they're assigned as
   Module attributes they are automatically added to the list of its
   parameters, and will appear e.g. in "parameters()" iterator.
   Assigning a Tensor doesn't have such effect. This is because one
   might want to cache some temporary state, like last hidden state of
   the RNN, in the model. If there was no such class as "Parameter",
   these temporaries would get registered too.
Parameters:
      * data (Tensor) -- parameter tensor.
  * **requires_grad** (*bool**, **optional*) -- if the parameter
    requires gradient. See Locally disabling gradient computation
    for more details. Default: *True*
PlaceholderObserver
class torch.quantization.observer.PlaceholderObserver(dtype=torch.float32, custom_op_name='', compute_dtype=None, quant_min=None, quant_max=None, is_dynamic=False)
Observer that doesn't do anything and just passes its configuration
   to the quantized module's ".from_float()".
Can be used for quantization to float16 which doesn't require
   determining ranges.
Parameters:
      * dtype -- dtype argument to the quantize node needed to
        implement the reference model spec.
  * **quant_min** -- minimum value in quantized domain (TODO:
    align behavior with other observers)

  * **quant_min** -- maximum value in quantized domain

  * **custom_op_name** -- (temporary) specify this observer for an
    operator that doesn't require any observation (Can be used in
    Graph Mode Passes for special case ops).

  * **compute_dtype** (*deprecated*) -- if set, marks the future
torch.nn.utils.parametrize.is_parametrized
torch.nn.utils.parametrize.is_parametrized(module, tensor_name=None)
Returns "True" if module has an active parametrization.
If the argument "tensor_name" is specified, returns "True" if
   "module[tensor_name]" is parametrized.
Parameters:
      * module (nn.Module) -- module to query
  * **tensor_name** (*str**, **optional*) -- attribute in the
    module to query Default: "None"

Return type:
      bool
torch.Tensor.scatter_reduce_
Tensor.scatter_reduce_(dim, index, src, reduce, *, include_self=True) -> Tensor
Reduces all values from the "src" tensor to the indices specified
   in the "index" tensor in the "self" tensor using the applied
   reduction defined via the "reduce" argument (""sum"", ""prod"",
   ""mean"", ""amax"", ""amin""). For each value in "src", it is
   reduced to an index in "self" which is specified by its index in
   "src" for "dimension != dim" and by the corresponding value in
   "index" for "dimension = dim". If "include_self="True"", the values
   in the "self" tensor are included in the reduction.
"self", "index" and "src" should all have the same number of
   dimensions. It is also required that "index.size(d) <= src.size(d)"
   for all dimensions "d", and that "index.size(d) <= self.size(d)"
   for all dimensions "d != dim". Note that "index" and "src" do not
   broadcast.
For a 3-D tensor with "reduce="sum"" and "include_self=True" the
output is given as:
  self[index[i][j][k]][j][k] += src[i][j][k]  # if dim == 0
  self[i][index[i][j][k]][k] += src[i][j][k]  # if dim == 1
  self[i][j][index[i][j][k]] += src[i][j][k]  # if dim == 2

Note:
 This operation may behave nondeterministically when given tensors
 on a CUDA device. See Reproducibility for more information.

Note:
 The backward pass is implemented only for "src.shape ==
 index.shape".

Warning:
 This function is in beta and may change in the near future.

Parameters:
      * dim (int) -- the axis along which to index
  * **index** (*LongTensor*) -- the indices of elements to scatter
    and reduce.

  * **src** (*Tensor*) -- the source elements to scatter and
    reduce

  * **reduce** (*str*) -- the reduction operation to apply for
    non-unique indices (""sum"", ""prod"", ""mean"", ""amax"",
    ""amin"")

  * **include_self** (*bool*) -- whether elements from the "self"
tensor are included in the reduction
Example:
  >>> src = torch.tensor([1., 2., 3., 4., 5., 6.])
  >>> index = torch.tensor([0, 1, 0, 1, 2, 1])
  >>> input = torch.tensor([1., 2., 3., 4.])
  >>> input.scatter_reduce(0, index, src, reduce="sum")
  tensor([5., 14., 8., 4.])
  >>> input.scatter_reduce(0, index, src, reduce="sum", include_self=False)
  tensor([4., 12., 5., 4.])
  >>> input2 = torch.tensor([5., 4., 3., 2.])
  >>> input2.scatter_reduce(0, index, src, reduce="amax")
  tensor([5., 6., 5., 2.])
  >>> input2.scatter_reduce(0, index, src, reduce="amax", include_self=False)
  tensor([3., 6., 5., 2.])
ReflectionPad3d
class torch.nn.ReflectionPad3d(padding)
Pads the input tensor using the reflection of the input boundary.
For N-dimensional padding, use "torch.nn.functional.pad()".
Parameters:
      padding (int, tuple) -- the size of the padding. If is
      int, uses the same padding in all boundaries. If a 6-tuple,
      uses (\text{padding_left}, \text{padding_right},
      \text{padding_top}, \text{padding_bottom},
      \text{padding_front}, \text{padding_back})
Shape:
      * Input: (N, C, D_{in}, H_{in}, W_{in}) or (C, D_{in}, H_{in},
        W_{in}).
  * Output: (N, C, D_{out}, H_{out}, W_{out}) or (C, D_{out},
    H_{out}, W_{out}), where

    D_{out} = D_{in} + \text{padding\_front} +
    \text{padding\_back}

    H_{out} = H_{in} + \text{padding\_top} +
    \text{padding\_bottom}

    W_{out} = W_{in} + \text{padding\_left} +
    \text{padding\_right}

Examples:
  >>> m = nn.ReflectionPad3d(1)
Examples:
  >>> m = nn.ReflectionPad3d(1)
  >>> input = torch.arange(8, dtype=torch.float).reshape(1, 1, 2, 2, 2)
  >>> m(input)
  tensor([[[[[7., 6., 7., 6.],
             [5., 4., 5., 4.],
             [7., 6., 7., 6.],
             [5., 4., 5., 4.]],
            [[3., 2., 3., 2.],
             [1., 0., 1., 0.],
             [3., 2., 3., 2.],
             [1., 0., 1., 0.]],
            [[7., 6., 7., 6.],
             [5., 4., 5., 4.],
             [7., 6., 7., 6.],
             [5., 4., 5., 4.]],
            [[3., 2., 3., 2.],
             [1., 0., 1., 0.],
             [3., 2., 3., 2.],
             [1., 0., 1., 0.]]]]])
torch.nn.functional.grid_sample
torch.nn.functional.grid_sample(input, grid, mode='bilinear', padding_mode='zeros', align_corners=None)
Given an "input" and a flow-field "grid", computes the "output"
   using "input" values and pixel locations from "grid".
Currently, only spatial (4-D) and volumetric (5-D) "input" are
   supported.
In the spatial (4-D) case, for "input" with shape (N, C,
   H_\text{in}, W_\text{in}) and "grid" with shape (N, H_\text{out},
   W_\text{out}, 2), the output will have shape (N, C, H_\text{out},
   W_\text{out}).
For each output location "output[n, :, h, w]", the size-2 vector
   "grid[n, h, w]" specifies "input" pixel locations "x" and "y",
   which are used to interpolate the output value "output[n, :, h,
   w]". In the case of 5D inputs, "grid[n, d, h, w]" specifies the
   "x", "y", "z" pixel locations for interpolating "output[n, :, d, h,
   w]". "mode" argument specifies "nearest" or "bilinear"
interpolation method to sample the input pixels.
"grid" specifies the sampling pixel locations normalized by the
   "input" spatial dimensions. Therefore, it should have most values
   in the range of "[-1, 1]". For example, values "x = -1, y = -1" is
   the left-top pixel of "input", and values  "x = 1, y = 1" is the
   right-bottom pixel of "input".
If "grid" has values outside the range of "[-1, 1]", the
   corresponding outputs are handled as defined by "padding_mode".
   Options are
  * "padding_mode="zeros"": use "0" for out-of-bound grid
    locations,

  * "padding_mode="border"": use border values for out-of-bound
    grid locations,

  * "padding_mode="reflection"": use values at locations reflected
    by the border for out-of-bound grid locations. For location
    far away from the border, it will keep being reflected until
    becoming in bound, e.g., (normalized) pixel location "x =
    -3.5" reflects by border "-1" and becomes "x' = 1.5", then
reflects by border "1" and becomes "x'' = -0.5".
Note:
 This function is often used in conjunction with "affine_grid()"
 to build Spatial Transformer Networks .

Note:
 When using the CUDA backend, this operation may induce
 nondeterministic behaviour in its backward pass that is not
 easily switched off. Please see the notes on Reproducibility for
 background.

Note:
 NaN values in "grid" would be interpreted as "-1".

Parameters:
      * input (Tensor) -- input of shape (N, C, H_\text{in},
        W_\text{in}) (4-D case) or (N, C, D_\text{in}, H_\text{in},
        W_\text{in}) (5-D case)
  * **grid** (*Tensor*) -- flow-field of shape (N, H_\text{out},
    W_\text{out}, 2) (4-D case) or (N, D_\text{out}, H_\text{out},
    W_\text{out}, 3) (5-D case)

  * **mode** (*str*) -- interpolation mode to calculate output
    values "'bilinear'" | "'nearest'" | "'bicubic'". Default:
"'bilinear'" Note: "mode='bicubic'" supports only 4-D input.
        When "mode='bilinear'" and the input is 5-D, the interpolation
        mode used internally will actually be trilinear. However, when
        the input is 4-D, the interpolation mode will legitimately be
        bilinear.
  * **padding_mode** (*str*) -- padding mode for outside grid
    values "'zeros'" | "'border'" | "'reflection'". Default:
    "'zeros'"

  * **align_corners** (*bool**, **optional*) -- Geometrically, we
    consider the pixels of the input  as squares rather than
    points. If set to "True", the extrema ("-1" and "1") are
    considered as referring to the center points of the input's
    corner pixels. If set to "False", they are instead considered
    as referring to the corner points of the input's corner
    pixels, making the sampling more resolution agnostic. This
    option parallels the "align_corners" option in
"interpolate()", and so whichever option is used here should
        also be used there to resize the input image before grid
        sampling. Default: "False"
Returns:
      output Tensor
Return type:
      output (Tensor)
Warning:
 When "align_corners = True", the grid positions depend on the
 pixel size relative to the input image size, and so the locations
 sampled by "grid_sample()" will differ for the same input given
 at different resolutions (that is, after being upsampled or
 downsampled). The default behavior up to version 1.2.0 was
 "align_corners = True". Since then, the default behavior has been
 changed to "align_corners = False", in order to bring it in line
 with the default for "interpolate()".

Note:
 "mode='bicubic'" is implemented using the cubic convolution
 algorithm with \alpha=-0.75. The constant \alpha might be
 different from packages to packages. For example, PIL and OpenCV
torch.isin
torch.isin(elements, test_elements, *, assume_unique=False, invert=False) -> Tensor
Tests if each element of "elements" is in "test_elements". Returns
   a boolean tensor of the same shape as "elements" that is True for
   elements in "test_elements" and False otherwise.
Note:
 One of "elements" or "test_elements" can be a scalar, but not
 both.

Parameters:
      * elements (Tensor or Scalar) -- Input elements
  * **test_elements** (*Tensor** or **Scalar*) -- Values against
    which to test for each input element

  * **assume_unique** (*bool**, **optional*) -- If True, assumes
    both "elements" and "test_elements" contain unique elements,
    which can speed up the calculation. Default: False

  * **invert** (*bool**, **optional*) -- If True, inverts the
    boolean return tensor, resulting in True values for elements
    *not* in "test_elements". Default: False

Returns:
BackendPatternConfig
class torch.ao.quantization.backend_config.BackendPatternConfig(pattern=None)
Config object that specifies quantization behavior for a given
   operator pattern. For a detailed example usage, see
   "BackendConfig".
add_dtype_config(dtype_config)
  Add a set of supported data types passed as arguments to
  quantize ops in the reference model spec.

  Return type:
     *BackendPatternConfig*

classmethod from_dict(backend_pattern_config_dict)
  Create a "BackendPatternConfig" from a dictionary with the
  following items:

     "pattern": the pattern being configured "observation_type":
     the "ObservationType" that specifies how observers should be
     inserted for this pattern "dtype_configs": a list of
     dictionaries that represents "DTypeConfig" s "root_module": a
     "torch.nn.Module" that represents the root for this pattern
     "qat_module": a "torch.nn.Module" that represents the QAT
implementation for this pattern "reference_quantized_module":
         a "torch.nn.Module" that represents the reference quantized
         implementation for this pattern's root module.
         "fused_module": a "torch.nn.Module" that represents the fused
         implementation for this pattern "fuser_method": a function
         that specifies how to fuse the pattern for this pattern
         "pattern_complex_format": the pattern specified in the
         reversed nested tuple format (deprecated)
  Return type:
     *BackendPatternConfig*

set_dtype_configs(dtype_configs)
  Set the supported data types passed as arguments to quantize ops
  in the reference model spec, overriding all previously
  registered data types.

  Return type:
     *BackendPatternConfig*

set_fused_module(fused_module)
  Set the module that represents the fused implementation for this
  pattern.

  Return type:
     *BackendPatternConfig*
BackendPatternConfig
set_fuser_method(fuser_method)
  Set the function that specifies how to fuse this
  BackendPatternConfig's pattern.

  The first argument of this function should be *is_qat*, and the
  rest of the arguments should be the items in the tuple pattern.
  The return value of this function should be the resulting fused
  module.

  For example, the fuser method for the pattern *(torch.nn.Linear,
  torch.nn.ReLU)* can be:

     def fuse_linear_relu(is_qat, linear, relu):
        return torch.ao.nn.intrinsic.LinearReLU(linear, relu)

  For a more complicated example, see https://gist.github.com/jer
  ryzh168/8bea7180a8ba3c279f2c9b050f2a69a6.

  Return type:
     *BackendPatternConfig*

set_observation_type(observation_type)
  Set how observers should be inserted in the graph for this
  pattern.

  Observation type here refers to how observers (or quant-dequant
ops) will be placed in the graph. This is used to produce the
      desired reference patterns understood by the backend. Weighted
      ops such as linear and conv require different observers (or
      quantization parameters passed to quantize ops in the reference
      model) for the input and the output.
  There are two observation types:

     *OUTPUT_USE_DIFFERENT_OBSERVER_AS_INPUT* (default): the
     output observer instance will be different from the input.
     This is the most common observation type.

     *OUTPUT_SHARE_OBSERVER_WITH_INPUT*: the output observer
     instance will be the same as the input. This is useful for
     operators like *cat*.

  Note: This will be renamed in the near future, since we will
  soon insert QuantDeQuantStubs with observers (and fake
  quantizes) attached instead of observers themselves.

  Return type:
     *BackendPatternConfig*

set_pattern(pattern)
  Set the pattern to configure.