shrinath-suresh/blogs-docs-splitted huggingface dataset:
'output_dtype': DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=0, quant_max_upper_bound=255, scale_min_lower_bound=None, scale_max_upper_bound=None, scale_exact_match=0.00390625, zero_point_exact_match=0),
       },
     ],
     'observation_type': ObservationType.OUTPUT_USE_DIFFERENT_OBSERVER_AS_INPUT,
   },
   {
     'pattern': ,
     'dtype_configs': [
       {
         'input_dtype': DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=None, quant_max_upper_bound=None, scale_min_lower_bound=None, scale_max_upper_bound=None, scale_exact_match=None, zero_point_exact_match=None),
         'output_dtype': DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=None, quant_max_upper_bound=None, scale_min_lower_bound=None, scale_max_upper_bound=None, scale_exact_match=None, zero_point_exact_match=None),
       },
     ],
     'observation_type': ObservationType.OUTPUT_SHARE_OBSERVER_WITH_INPUT,
   },
   {
},
   {
     'pattern': squeeze,
     'dtype_configs': [
       {
         'input_dtype': DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=None, quant_max_upper_bound=None, scale_min_lower_bound=None, scale_max_upper_bound=None, scale_exact_match=None, zero_point_exact_match=None),
         'output_dtype': DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=None, quant_max_upper_bound=None, scale_min_lower_bound=None, scale_max_upper_bound=None, scale_exact_match=None, zero_point_exact_match=None),
       },
     ],
     'observation_type': ObservationType.OUTPUT_SHARE_OBSERVER_WITH_INPUT,
   },
   {
     'pattern': squeeze_,
     'dtype_configs': [
       {
         'input_dtype': DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=None, quant_max_upper_bound=None, scale_min_lower_bound=None, scale_max_upper_bound=None, scale_exact_match=None, zero_point_exact_match=None),
},
   {
     'pattern': ,
     'dtype_configs': [
       {
         'input_dtype': DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=0, quant_max_upper_bound=255, scale_min_lower_bound=None, scale_max_upper_bound=None, scale_exact_match=0.0078125, zero_point_exact_match=128),
         'output_dtype': DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=0, quant_max_upper_bound=255, scale_min_lower_bound=None, scale_max_upper_bound=None, scale_exact_match=0.0078125, zero_point_exact_match=128),
       },
     ],
     'observation_type': ObservationType.OUTPUT_USE_DIFFERENT_OBSERVER_AS_INPUT,
   },
   {
     'pattern': ,
     'dtype_configs': [
       {
         'input_dtype': DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=0, quant_max_upper_bound=255, scale_min_lower_bound=None, scale_max_upper_bound=None, scale_exact_match=0.0078125, zero_point_exact_match=128),
'output_dtype': DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=0, quant_max_upper_bound=255, scale_min_lower_bound=None, scale_max_upper_bound=None, scale_exact_match=0.0078125, zero_point_exact_match=128),
       },
     ],
     'observation_type': ObservationType.OUTPUT_USE_DIFFERENT_OBSERVER_AS_INPUT,
   },
   {
     'pattern': tanh,
     'dtype_configs': [
       {
         'input_dtype': DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=0, quant_max_upper_bound=255, scale_min_lower_bound=None, scale_max_upper_bound=None, scale_exact_match=0.0078125, zero_point_exact_match=128),
         'output_dtype': DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=0, quant_max_upper_bound=255, scale_min_lower_bound=None, scale_max_upper_bound=None, scale_exact_match=0.0078125, zero_point_exact_match=128),
       },
     ],
     'observation_type': ObservationType.OUTPUT_USE_DIFFERENT_OBSERVER_AS_INPUT,
   },
   {
     'pattern': tanh_,
     'dtype_configs': [
'pattern': tanh_,
     'dtype_configs': [
       {
         'input_dtype': DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=0, quant_max_upper_bound=255, scale_min_lower_bound=None, scale_max_upper_bound=None, scale_exact_match=0.0078125, zero_point_exact_match=128),
         'output_dtype': DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=0, quant_max_upper_bound=255, scale_min_lower_bound=None, scale_max_upper_bound=None, scale_exact_match=0.0078125, zero_point_exact_match=128),
       },
     ],
     'observation_type': ObservationType.OUTPUT_USE_DIFFERENT_OBSERVER_AS_INPUT,
   },
   {
     'pattern': ,
     'dtype_configs': [
       {
         'input_dtype': DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=None, quant_max_upper_bound=None, scale_min_lower_bound=None, scale_max_upper_bound=None, scale_exact_match=None, zero_point_exact_match=None),
'output_dtype': DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=None, quant_max_upper_bound=None, scale_min_lower_bound=None, scale_max_upper_bound=None, scale_exact_match=None, zero_point_exact_match=None),
       },
     ],
     'observation_type': ObservationType.OUTPUT_SHARE_OBSERVER_WITH_INPUT,
   },
   {
     'pattern': transpose,
     'dtype_configs': [
       {
         'input_dtype': DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=None, quant_max_upper_bound=None, scale_min_lower_bound=None, scale_max_upper_bound=None, scale_exact_match=None, zero_point_exact_match=None),
         'output_dtype': DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=None, quant_max_upper_bound=None, scale_min_lower_bound=None, scale_max_upper_bound=None, scale_exact_match=None, zero_point_exact_match=None),
       },
     ],
     'observation_type': ObservationType.OUTPUT_SHARE_OBSERVER_WITH_INPUT,
   },
   {
},
   {
     'pattern': ,
     'dtype_configs': [
       {
         'input_dtype': DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=None, quant_max_upper_bound=None, scale_min_lower_bound=None, scale_max_upper_bound=None, scale_exact_match=None, zero_point_exact_match=None),
         'output_dtype': DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=None, quant_max_upper_bound=None, scale_min_lower_bound=None, scale_max_upper_bound=None, scale_exact_match=None, zero_point_exact_match=None),
       },
     ],
     'observation_type': ObservationType.OUTPUT_SHARE_OBSERVER_WITH_INPUT,
   },
   {
     'pattern': unsqueeze,
     'dtype_configs': [
       {
         'input_dtype': DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=None, quant_max_upper_bound=None, scale_min_lower_bound=None, scale_max_upper_bound=None, scale_exact_match=None, zero_point_exact_match=None),
'output_dtype': DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=None, quant_max_upper_bound=None, scale_min_lower_bound=None, scale_max_upper_bound=None, scale_exact_match=None, zero_point_exact_match=None),
       },
     ],
     'observation_type': ObservationType.OUTPUT_SHARE_OBSERVER_WITH_INPUT,
   },
   {
     'pattern': unsqueeze_,
     'dtype_configs': [
       {
         'input_dtype': DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=None, quant_max_upper_bound=None, scale_min_lower_bound=None, scale_max_upper_bound=None, scale_exact_match=None, zero_point_exact_match=None),
         'output_dtype': DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=None, quant_max_upper_bound=None, scale_min_lower_bound=None, scale_max_upper_bound=None, scale_exact_match=None, zero_point_exact_match=None),
       },
     ],
     'observation_type': ObservationType.OUTPUT_SHARE_OBSERVER_WITH_INPUT,
   },
   {
     'pattern': view,
     'dtype_configs': [
       {
'dtype_configs': [
       {
         'input_dtype': DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=None, quant_max_upper_bound=None, scale_min_lower_bound=None, scale_max_upper_bound=None, scale_exact_match=None, zero_point_exact_match=None),
         'output_dtype': DTypeWithConstraints(dtype=torch.quint8, quant_min_lower_bound=None, quant_max_upper_bound=None, scale_min_lower_bound=None, scale_max_upper_bound=None, scale_exact_match=None, zero_point_exact_match=None),
       },
     ],
     'observation_type': ObservationType.OUTPUT_SHARE_OBSERVER_WITH_INPUT,
   }
torch.utils.checkpoint
Note:
Checkpointing is implemented by rerunning a forward-pass segment for
  each checkpointed segment during backward.  This can cause
  persistent states like the RNG state to be advanced than they would
  without checkpointing.  By default, checkpointing includes logic to
  juggle the RNG state such that checkpointed passes making use of RNG
  (through dropout for example) have deterministic output as compared
  to non-checkpointed passes.  The logic to stash and restore RNG
  states can incur a moderate performance hit depending on the runtime
  of checkpointed operations.  If deterministic output compared to
  non-checkpointed passes is not required, supply
  "preserve_rng_state=False" to "checkpoint" or
  "checkpoint_sequential" to omit stashing and restoring the RNG state
  during each checkpoint.The stashing logic saves and restores the RNG
  state for the current device and the device of all cuda Tensor
arguments to the "run_fn". However, the logic has no way to
  anticipate if the user will move Tensors to a new device within the
  "run_fn" itself.  Therefore, if you move Tensors to a new device
  ("new" meaning not belonging to the set of [current device + devices
  of Tensor arguments]) within "run_fn", deterministic output compared
  to non-checkpointed passes is never guaranteed.
torch.utils.checkpoint.checkpoint(function, args, use_reentrant=True, *kwargs)
Checkpoint a model or part of the model
Checkpointing works by trading compute for memory. Rather than
   storing all intermediate activations of the entire computation
   graph for computing backward, the checkpointed part does not
   save intermediate activations, and instead recomputes them in
   backward pass. It can be applied on any part of a model.
Specifically, in the forward pass, "function" will run in
   "torch.no_grad()" manner, i.e., not storing the intermediate
activations. Instead, the forward pass saves the inputs tuple and
   the "function" parameter. In the backwards pass, the saved inputs
   and "function" is retrieved, and the forward pass is computed on
   "function" again, now tracking the intermediate activations, and
   then the gradients are calculated using these activation values.
The output of "function" can contain non-Tensor values and gradient
   recording is only performed for the Tensor values. Note that if the
   output consists of nested structures (ex: custom objects, lists,
   dicts etc.) consisting of Tensors, these Tensors nested in custom
   structures will not be considered as part of autograd.
Warning:
 If "function" invocation during backward does anything different
 than the one during forward, e.g., due to some global variable,
 the checkpointed version won't be equivalent, and unfortunately
 it can't be detected.

Warning:
 If "use_reentrant=True" is specified, then if the checkpointed
segment contains tensors detached from the computational graph by
     detach() or torch.no_grad(), the backward pass will raise an
     error. This is because checkpoint makes all the outputs require
     gradients which causes issues when a tensor is defined to have no
     gradient in the model. To circumvent this, detach the tensors
     outside of the checkpoint function. Note that the checkpointed
     segment can contain tensors detached from the computational graph
     if "use_reentrant=False" is specified.
Warning:
 If "use_reentrant=True" is specified, at least one of the inputs
 needs to have "requires_grad=True" if grads are needed for model
 inputs, otherwise the checkpointed part of the model won't have
 gradients. At least one of the outputs needs to have
 "requires_grad=True" as well. Note that this does not apply if
 "use_reentrant=False" is specified.

Warning:
 If "use_reentrant=True" is specified, checkpointing currently
only supports "torch.autograd.backward()" and only if its
     inputs argument is not passed. "torch.autograd.grad()" is not
     supported. If "use_reentrant=False" is specified, checkpointing
     will work with "torch.autograd.grad()".
Parameters:
      * function -- describes what to run in the forward pass of
        the model or part of the model. It should also know how to
        handle the inputs passed as the tuple. For example, in LSTM,
        if user passes "(activation, hidden)", "function" should
        correctly use the first input as "activation" and the second
        input as "hidden"
  * **preserve_rng_state** (*bool**, **optional*) -- Omit stashing
    and restoring the RNG state during each checkpoint. Default:
    "True"

  * **use_reentrant** (*bool**, **optional*) -- Use checkpointing
    implementation that requires re-entrant autograd. If
    "use_reentrant=False" is specified, "checkpoint" will use an
implementation that does not require re-entrant autograd. This
        allows "checkpoint" to support additional functionality, such
        as working as expected with "torch.autograd.grad" and support
        for keyword arguments input into the checkpointed function.
        Note that future versions of PyTorch will default to
        "use_reentrant=False". Default: "True"
  * **args** -- tuple containing inputs to the "function"

Returns:
      Output of running "function" on "*args"
torch.utils.checkpoint.checkpoint_sequential(functions, segments, input, use_reentrant=True, **kwargs)
A helper function for checkpointing sequential models.
Sequential models execute a list of modules/functions in order
   (sequentially). Therefore, we can divide such a model in various
   segments and checkpoint each segment. All segments except the last
   will run in "torch.no_grad()" manner, i.e., not storing the
   intermediate activations. The inputs of each checkpointed segment
will be saved for re-running the segment in the backward pass.
See "checkpoint()" on how checkpointing works.
Warning:
 Checkpointing currently only supports "torch.autograd.backward()"
 and only if its *inputs* argument is not passed.
 "torch.autograd.grad()" is not supported.

Parameters:
      * functions -- A "torch.nn.Sequential" or the list of
        modules or functions (comprising the model) to run
        sequentially.
  * **segments** -- Number of chunks to create in the model

  * **input** -- A Tensor that is input to "functions"

  * **preserve_rng_state** (*bool**, **optional*) -- Omit stashing
    and restoring the RNG state during each checkpoint. Default:
    "True"

  * **use_reentrant** (*bool**, **optional*) -- Use checkpointing
    implementation that requires re-entrant autograd. If
    "use_reentrant=False" is specified, "checkpoint" will use an
    implementation that does not require re-entrant autograd. This
torch.func
torch.func, previously known as "functorch", is JAX-like composable
function transforms for PyTorch.
Note:
This library is currently in beta. What this means is that the
  features generally work (unless otherwise documented) and we (the
  PyTorch team) are committed to bringing this library forward.
  However, the APIs may change under user feedback and we don't have
  full coverage over PyTorch operations.If you have suggestions on the
  API or use-cases you'd like to be covered, please open an GitHub
  issue or reach out. We'd love to hear about how you're using the
  library.
What are composable function transforms?


A "function transform" is a higher-order function that accepts a
  numerical function and returns a new function that computes a
  different quantity.


"torch.func" has auto-differentiation transforms ("grad(f)" returns
  a function that computes the gradient of "f"), a
a function that computes the gradient of "f"), a
  vectorization/batching transform ("vmap(f)" returns a function that
  computes "f" over batches of inputs), and others.

These function transforms can compose with each other arbitrarily.
  For example, composing "vmap(grad(f))" computes a quantity called
  per-sample-gradients that stock PyTorch cannot efficiently compute
  today.

Why composable function transforms?
There are a number of use cases that are tricky to do in PyTorch
today:


computing per-sample-gradients (or other per-sample quantities)


running ensembles of models on a single machine


efficiently batching together tasks in the inner-loop of MAML


efficiently computing Jacobians and Hessians


efficiently computing batched Jacobians and Hessians


Composing "vmap()", "grad()", and "vjp()" transforms allows us to
express the above without designing a separate subsystem for each.
This idea of composable function transforms comes from the JAX
framework.
Read More


torch.func Whirlwind Tour


What is torch.func?


Why composable function transforms?


What are the transforms?


torch.func API Reference


Function Transforms


Utilities for working with torch.nn.Modules


UX Limitations


General limitations


torch.autograd APIs


vmap limitations


Randomness


Migrating from functorch to torch.func


function transforms


NN module utilities


functorch.compile
torch.ao.ns._numeric_suite
Warning:
This module is an early prototype and is subject to change.
torch.ao.ns._numeric_suite.compare_weights(float_dict, quantized_dict)
Compare the weights of the float module with its corresponding
   quantized module. Return a dict with key corresponding to module
   names and each entry being a dictionary with two keys 'float' and
   'quantized', containing the float and quantized weights. This dict
   can be used to compare and compute the quantization error of the
   weights of float and quantized models.
Example usage:
  wt_compare_dict = compare_weights(
      float_model.state_dict(), qmodel.state_dict())
  for key in wt_compare_dict:
      print(
          key,
          compute_error(
              wt_compare_dict[key]['float'],
              wt_compare_dict[key]['quantized'].dequantize()
          )
      )

Parameters:
)
          )
Parameters:
      * float_dict (Dict[str, Any]) -- state dict of
        the float model
  * **quantized_dict** (*Dict**[**str**, **Any**]*) -- state dict
    of the quantized model

Returns:
      dict with key corresponding to module names and each entry being
      a dictionary with two keys 'float' and 'quantized', containing
      the float and quantized weights
Return type:
      weight_dict
torch.ao.ns._numeric_suite.get_logger_dict(mod, prefix='')
Traverse the modules and save all logger stats into target dict.
   This is mainly used for quantization accuracy debug.
Type of loggers supported:
      ShadowLogger: used to log the outputs of the quantized module
      and its matching float shadow module, OutputLogger: used to log
      the outputs of the modules
Parameters:
      * mod (Module) -- module we want to save all logger stats
  * **prefix** (*str*) -- prefix for the current module

Returns:
Returns:
      the dictionary used to save all logger stats
Return type:
      target_dict
class torch.ao.ns._numeric_suite.Logger
Base class for stats logging
forward(x)
class torch.ao.ns._numeric_suite.ShadowLogger
Class used in Shadow module to record the outputs of the original
   and shadow modules.
forward(x, y)
class torch.ao.ns._numeric_suite.OutputLogger
Class used to log the outputs of the module
forward(x)
class torch.ao.ns._numeric_suite.Shadow(q_module, float_module, logger_cls)
Shadow module attaches the float module to its matching quantized
   module as the shadow. Then it uses Logger module to process the
   outputs of both modules.
Parameters:
      * q_module -- module quantized from float_module that we
        want to shadow
  * **float_module** -- float module used to shadow q_module

  * **logger_cls** -- type of logger used to process the outputs
    of q_module and float_module. ShadowLogger or custom loggers
can be used.
forward(*x)
  Return type:
     *Tensor*

add(x, y)
  Return type:
     *Tensor*

add_scalar(x, y)
  Return type:
     *Tensor*

mul(x, y)
  Return type:
     *Tensor*

mul_scalar(x, y)
  Return type:
     *Tensor*

cat(x, dim=0)
  Return type:
     *Tensor*

add_relu(x, y)
  Return type:
     *Tensor*

torch.ao.ns._numeric_suite.prepare_model_with_stubs(float_module, q_module, module_swap_list, logger_cls)
Prepare the model by attaching the float module to its matching
   quantized module as the shadow if the float module type is in
   module_swap_list.
Example usage:
  prepare_model_with_stubs(float_model, q_model, module_swap_list, Logger)
  q_model(data)
  ob_dict = get_logger_dict(q_model)

Parameters:
      * float_module (Module) -- float module used to generate
        the q_module
  * **q_module** (*Module*) -- module quantized from float_module
module_swap_list (Set[type]) -- list of float
        module types to attach the shadow

logger_cls (Callable) -- type of logger to be used in
    shadow module to process the outputs of quantized module and
    its float shadow module



torch.ao.ns._numeric_suite.compare_model_stub(float_model, q_model, module_swap_list, *data, logger_cls=)
Compare quantized module in a model with its floating point
   counterpart, feeding both of them the same input. Return a dict
   with key corresponding to module names and each entry being a
   dictionary with two keys 'float' and 'quantized', containing the
   output tensors of quantized and its matching float shadow module.
   This dict can be used to compare and compute the module level
   quantization error.
This function first call prepare_model_with_stubs() to swap the
   quantized module that we want to compare with the Shadow module,
which takes quantized module, corresponding float module and logger
   as input, and creates a forward path inside to make the float
   module to shadow quantized module sharing the same input. The
   logger can be customizable, default logger is ShadowLogger and it
   will save the outputs of the quantized module and float module that
   can be used to compute the module level quantization error.
Example usage:
  module_swap_list = [torchvision.models.quantization.resnet.QuantizableBasicBlock]
  ob_dict = compare_model_stub(float_model,qmodel,module_swap_list, data)
  for key in ob_dict:
      print(key, compute_error(ob_dict[key]['float'], ob_dict[key]['quantized'].dequantize()))

Parameters:
      * float_model (Module) -- float model used to generate the
        q_model
  * **q_model** (*Module*) -- model quantized from float_model

  * **module_swap_list** (*Set**[**type**]*) -- list of float
    module types at which shadow modules will be attached.
data -- input data used to run the prepared q_model

logger_cls -- type of logger to be used in shadow module
    to process the outputs of quantized module and its float
    shadow module



Return type:
      Dict[str, Dict]
torch.ao.ns._numeric_suite.get_matching_activations(float_module, q_module)
Find the matching activation between float and quantized modules.
Parameters:
      * float_module (Module) -- float module used to generate
        the q_module
  * **q_module** (*Module*) -- module quantized from float_module

Returns:
      dict with key corresponding to quantized module names and each
      entry being a dictionary with two keys 'float' and 'quantized',
      containing the matching float and quantized activations
Return type:
      act_dict
torch.ao.ns._numeric_suite.prepare_model_outputs(float_module, q_module, logger_cls=, allow_list=None)