neulab/docprompting-conala huggingface dataset:
class torch.autograd.function._ContextMethodMixin [source]
 
 
mark_dirty(*args) [source]
 
Marks given tensors as modified in an in-place operation. This should be called at most once, only from inside the forward() method, and all arguments should be inputs. Every tensor that’s been modified in-place in a call to forward() should be given to this function, to ensure correctness of our checks. It doesn’t matter whether the function is called before or after modification. 
  
mark_non_differentiable(*args) [source]
 
Marks outputs as non-differentiable. This should be called at most once, only from inside the forward() method, and all arguments should be outputs. This will mark outputs as not requiring gradients, increasing the efficiency of backward computation. You still need to accept a gradient for each output in backward(), but it’s always going to be a zero tensor with the same shape as the shape of a corresponding output. This is used e.g. for indices returned from a max Function. 
  
save_for_backward(*tensors) [source]
 
Saves given tensors for a future call to backward(). This should be called at most once, and only from inside the forward() method. Later, saved tensors can be accessed through the saved_tensors attribute. Before returning them to the user, a check is made to ensure they weren’t used in any in-place operation that modified their content. Arguments can also be None. 
  
set_materialize_grads(value) [source]
 
Sets whether to materialize output grad tensors. Default is true. This should be called only from inside the forward() method If true, undefined output grad tensors will be expanded to tensors full of zeros prior to calling the backward() method.
mark_dirty(*args) [source]
 
Marks given tensors as modified in an in-place operation. This should be called at most once, only from inside the forward() method, and all arguments should be inputs. Every tensor that’s been modified in-place in a call to forward() should be given to this function, to ensure correctness of our checks. It doesn’t matter whether the function is called before or after modification.
mark_non_differentiable(*args) [source]
 
Marks outputs as non-differentiable. This should be called at most once, only from inside the forward() method, and all arguments should be outputs. This will mark outputs as not requiring gradients, increasing the efficiency of backward computation. You still need to accept a gradient for each output in backward(), but it’s always going to be a zero tensor with the same shape as the shape of a corresponding output. This is used e.g. for indices returned from a max Function.
save_for_backward(*tensors) [source]
 
Saves given tensors for a future call to backward(). This should be called at most once, and only from inside the forward() method. Later, saved tensors can be accessed through the saved_tensors attribute. Before returning them to the user, a check is made to ensure they weren’t used in any in-place operation that modified their content. Arguments can also be None.
set_materialize_grads(value) [source]
 
Sets whether to materialize output grad tensors. Default is true. This should be called only from inside the forward() method If true, undefined output grad tensors will be expanded to tensors full of zeros prior to calling the backward() method.
torch.autograd.functional.hessian(func, inputs, create_graph=False, strict=False, vectorize=False) [source]
 
Function that computes the Hessian of a given scalar function.  Parameters 
 
func (function) – a Python function that takes Tensor inputs and returns a Tensor with a single element. 
inputs (tuple of Tensors or Tensor) – inputs to the function func. 
create_graph (bool, optional) – If True, the Hessian will be computed in a differentiable manner. Note that when strict is False, the result can not require gradients or be disconnected from the inputs. Defaults to False. 
strict (bool, optional) – If True, an error will be raised when we detect that there exists an input such that all the outputs are independent of it. If False, we return a Tensor of zeros as the hessian for said inputs, which is the expected mathematical value. Defaults to False. 
vectorize (bool, optional) – This feature is experimental, please use at your own risk. When computing the hessian, usually we invoke autograd.grad once per row of the hessian. If this flag is True, we use the vmap prototype feature as the backend to vectorize calls to autograd.grad so we only invoke it once instead of once per row. This should lead to performance improvements in many use cases, however, due to this feature being incomplete, there may be performance cliffs. Please use torch._C._debug_only_display_vmap_fallback_warnings(True) to show any performance warnings and file us issues if warnings exist for your use case. Defaults to False.   Returns 
if there is a single input, this will be a single Tensor containing the Hessian for the input. If it is a tuple, then the Hessian will be a tuple of tuples where Hessian[i][j] will contain the Hessian of the ith input and jth input with size the sum of the size of the ith input plus the size of the jth input. Hessian[i][j] will have the same dtype and device as the corresponding ith input.  Return type 
Hessian (Tensor or a tuple of tuple of Tensors)   Example >>> def pow_reducer(x):
...   return x.pow(3).sum()
>>> inputs = torch.rand(2, 2)
>>> hessian(pow_reducer, inputs)
tensor([[[[5.2265, 0.0000],
          [0.0000, 0.0000]],
         [[0.0000, 4.8221],
          [0.0000, 0.0000]]],
        [[[0.0000, 0.0000],
          [1.9456, 0.0000]],
         [[0.0000, 0.0000],
          [0.0000, 3.2550]]]])
 >>> hessian(pow_reducer, inputs, create_graph=True)
tensor([[[[5.2265, 0.0000],
          [0.0000, 0.0000]],
         [[0.0000, 4.8221],
          [0.0000, 0.0000]]],
        [[[0.0000, 0.0000],
          [1.9456, 0.0000]],
         [[0.0000, 0.0000],
          [0.0000, 3.2550]]]], grad_fn=<ViewBackward>)
 >>> def pow_adder_reducer(x, y):
...   return (2 * x.pow(2) + 3 * y.pow(2)).sum()
>>> inputs = (torch.rand(2), torch.rand(2))
>>> hessian(pow_adder_reducer, inputs)
((tensor([[4., 0.],
          [0., 4.]]),
  tensor([[0., 0.],
          [0., 0.]])),
 (tensor([[0., 0.],
          [0., 0.]]),
  tensor([[6., 0.],
          [0., 6.]])))
torch.autograd.functional.hvp(func, inputs, v=None, create_graph=False, strict=False) [source]
 
Function that computes the dot product between the Hessian of a given scalar function and a vector v at the point given by the inputs.  Parameters 
 
func (function) – a Python function that takes Tensor inputs and returns a Tensor with a single element. 
inputs (tuple of Tensors or Tensor) – inputs to the function func. 
v (tuple of Tensors or Tensor) – The vector for which the Hessian vector product is computed. Must be the same size as the input of func. This argument is optional when func’s input contains a single element and (if it is not provided) will be set as a Tensor containing a single 1. 
create_graph (bool, optional) – If True, both the output and result will be computed in a differentiable way. Note that when strict is False, the result can not require gradients or be disconnected from the inputs. Defaults to False. 
strict (bool, optional) – If True, an error will be raised when we detect that there exists an input such that all the outputs are independent of it. If False, we return a Tensor of zeros as the hvp for said inputs, which is the expected mathematical value. Defaults to False.   Returns 
 tuple with:

func_output (tuple of Tensors or Tensor): output of func(inputs) hvp (tuple of Tensors or Tensor): result of the dot product with the same shape as the inputs.    Return type 
output (tuple)   Example >>> def pow_reducer(x):
...   return x.pow(3).sum()
>>> inputs = torch.rand(2, 2)
>>> v = torch.ones(2, 2)
>>> hvp(pow_reducer, inputs, v)
(tensor(0.1448),
 tensor([[2.0239, 1.6456],
         [2.4988, 1.4310]]))
 >>> hvp(pow_reducer, inputs, v, create_graph=True)
(tensor(0.1448, grad_fn=<SumBackward0>),
 tensor([[2.0239, 1.6456],
         [2.4988, 1.4310]], grad_fn=<MulBackward0>))
 >>> def pow_adder_reducer(x, y):
...   return (2 * x.pow(2) + 3 * y.pow(2)).sum()
>>> inputs = (torch.rand(2), torch.rand(2))
>>> v = (torch.zeros(2), torch.ones(2))
>>> hvp(pow_adder_reducer, inputs, v)
(tensor(2.3030),
 (tensor([0., 0.]),
  tensor([6., 6.])))
  Note This function is significantly slower than vhp due to backward mode AD constraints. If your functions is twice continuously differentiable, then hvp = vhp.t(). So if you know that your function satisfies this condition, you should use vhp instead that is much faster with the current implementation.
torch.autograd.functional.jacobian(func, inputs, create_graph=False, strict=False, vectorize=False) [source]
 
Function that computes the Jacobian of a given function.  Parameters 
 
func (function) – a Python function that takes Tensor inputs and returns a tuple of Tensors or a Tensor. 
inputs (tuple of Tensors or Tensor) – inputs to the function func. 
create_graph (bool, optional) – If True, the Jacobian will be computed in a differentiable manner. Note that when strict is False, the result can not require gradients or be disconnected from the inputs. Defaults to False. 
strict (bool, optional) – If True, an error will be raised when we detect that there exists an input such that all the outputs are independent of it. If False, we return a Tensor of zeros as the jacobian for said inputs, which is the expected mathematical value. Defaults to False. 
vectorize (bool, optional) – This feature is experimental, please use at your own risk. When computing the jacobian, usually we invoke autograd.grad once per row of the jacobian. If this flag is True, we use the vmap prototype feature as the backend to vectorize calls to autograd.grad so we only invoke it once instead of once per row. This should lead to performance improvements in many use cases, however, due to this feature being incomplete, there may be performance cliffs. Please use torch._C._debug_only_display_vmap_fallback_warnings(True) to show any performance warnings and file us issues if warnings exist for your use case. Defaults to False.   Returns 
if there is a single input and output, this will be a single Tensor containing the Jacobian for the linearized inputs and output. If one of the two is a tuple, then the Jacobian will be a tuple of Tensors. If both of them are tuples, then the Jacobian will be a tuple of tuple of Tensors where Jacobian[i][j] will contain the Jacobian of the ith output and jth input and will have as size the concatenation of the sizes of the corresponding output and the corresponding input and will have same dtype and device as the corresponding input.  Return type 
Jacobian (Tensor or nested tuple of Tensors)   Example >>> def exp_reducer(x):
...   return x.exp().sum(dim=1)
>>> inputs = torch.rand(2, 2)
>>> jacobian(exp_reducer, inputs)
tensor([[[1.4917, 2.4352],
         [0.0000, 0.0000]],
        [[0.0000, 0.0000],
         [2.4369, 2.3799]]])
 >>> jacobian(exp_reducer, inputs, create_graph=True)
tensor([[[1.4917, 2.4352],
         [0.0000, 0.0000]],
        [[0.0000, 0.0000],
         [2.4369, 2.3799]]], grad_fn=<ViewBackward>)
 >>> def exp_adder(x, y):
...   return 2 * x.exp() + 3 * y
>>> inputs = (torch.rand(2), torch.rand(2))
>>> jacobian(exp_adder, inputs)
(tensor([[2.8052, 0.0000],
        [0.0000, 3.3963]]),
 tensor([[3., 0.],
         [0., 3.]]))
torch.autograd.functional.jvp(func, inputs, v=None, create_graph=False, strict=False) [source]
 
Function that computes the dot product between the Jacobian of the given function at the point given by the inputs and a vector v.  Parameters 
 
func (function) – a Python function that takes Tensor inputs and returns a tuple of Tensors or a Tensor. 
inputs (tuple of Tensors or Tensor) – inputs to the function func. 
v (tuple of Tensors or Tensor) – The vector for which the Jacobian vector product is computed. Must be the same size as the input of func. This argument is optional when the input to func contains a single element and (if it is not provided) will be set as a Tensor containing a single 1. 
create_graph (bool, optional) – If True, both the output and result will be computed in a differentiable way. Note that when strict is False, the result can not require gradients or be disconnected from the inputs. Defaults to False. 
strict (bool, optional) – If True, an error will be raised when we detect that there exists an input such that all the outputs are independent of it. If False, we return a Tensor of zeros as the jvp for said inputs, which is the expected mathematical value. Defaults to False.   Returns 
 tuple with:

func_output (tuple of Tensors or Tensor): output of func(inputs) jvp (tuple of Tensors or Tensor): result of the dot product with the same shape as the output.    Return type 
output (tuple)   Example >>> def exp_reducer(x):
...   return x.exp().sum(dim=1)
>>> inputs = torch.rand(4, 4)
>>> v = torch.ones(4, 4)
>>> jvp(exp_reducer, inputs, v)
(tensor([6.3090, 4.6742, 7.9114, 8.2106]),
 tensor([6.3090, 4.6742, 7.9114, 8.2106]))
 >>> jvp(exp_reducer, inputs, v, create_graph=True)
(tensor([6.3090, 4.6742, 7.9114, 8.2106], grad_fn=<SumBackward1>),
 tensor([6.3090, 4.6742, 7.9114, 8.2106], grad_fn=<SqueezeBackward1>))
 >>> def adder(x, y):
...   return 2 * x + 3 * y
>>> inputs = (torch.rand(2), torch.rand(2))
>>> v = (torch.ones(2), torch.ones(2))
>>> jvp(adder, inputs, v)
(tensor([2.2399, 2.5005]),
 tensor([5., 5.]))
  Note The jvp is currently computed by using the backward of the backward (sometimes called the double backwards trick) as we don’t have support for forward mode AD in PyTorch at the moment.
torch.autograd.functional.vhp(func, inputs, v=None, create_graph=False, strict=False) [source]
 
Function that computes the dot product between a vector v and the Hessian of a given scalar function at the point given by the inputs.  Parameters 
 
func (function) – a Python function that takes Tensor inputs and returns a Tensor with a single element. 
inputs (tuple of Tensors or Tensor) – inputs to the function func. 
v (tuple of Tensors or Tensor) – The vector for which the vector Hessian product is computed. Must be the same size as the input of func. This argument is optional when func’s input contains a single element and (if it is not provided) will be set as a Tensor containing a single 1. 
create_graph (bool, optional) – If True, both the output and result will be computed in a differentiable way. Note that when strict is False, the result can not require gradients or be disconnected from the inputs. Defaults to False. 
strict (bool, optional) – If True, an error will be raised when we detect that there exists an input such that all the outputs are independent of it. If False, we return a Tensor of zeros as the vhp for said inputs, which is the expected mathematical value. Defaults to False.   Returns 
 tuple with:

func_output (tuple of Tensors or Tensor): output of func(inputs) vhp (tuple of Tensors or Tensor): result of the dot product with the same shape as the inputs.    Return type 
output (tuple)   Example >>> def pow_reducer(x):
...   return x.pow(3).sum()
>>> inputs = torch.rand(2, 2)
>>> v = torch.ones(2, 2)
>>> vhp(pow_reducer, inputs, v)
(tensor(0.5591),
 tensor([[1.0689, 1.2431],
         [3.0989, 4.4456]]))
>>> vhp(pow_reducer, inputs, v, create_graph=True)
(tensor(0.5591, grad_fn=<SumBackward0>),
 tensor([[1.0689, 1.2431],
         [3.0989, 4.4456]], grad_fn=<MulBackward0>))
>>> def pow_adder_reducer(x, y):
...   return (2 * x.pow(2) + 3 * y.pow(2)).sum()
>>> inputs = (torch.rand(2), torch.rand(2))
>>> v = (torch.zeros(2), torch.ones(2))
>>> vhp(pow_adder_reducer, inputs, v)
(tensor(4.8053),
 (tensor([0., 0.]),
  tensor([6., 6.])))
torch.autograd.functional.vjp(func, inputs, v=None, create_graph=False, strict=False) [source]
 
Function that computes the dot product between a vector v and the Jacobian of the given function at the point given by the inputs.  Parameters 
 
func (function) – a Python function that takes Tensor inputs and returns a tuple of Tensors or a Tensor. 
inputs (tuple of Tensors or Tensor) – inputs to the function func. 
v (tuple of Tensors or Tensor) – The vector for which the vector Jacobian product is computed. Must be the same size as the output of func. This argument is optional when the output of func contains a single element and (if it is not provided) will be set as a Tensor containing a single 1. 
create_graph (bool, optional) – If True, both the output and result will be computed in a differentiable way. Note that when strict is False, the result can not require gradients or be disconnected from the inputs. Defaults to False. 
strict (bool, optional) – If True, an error will be raised when we detect that there exists an input such that all the outputs are independent of it. If False, we return a Tensor of zeros as the vjp for said inputs, which is the expected mathematical value. Defaults to False.   Returns 
 tuple with:

func_output (tuple of Tensors or Tensor): output of func(inputs) vjp (tuple of Tensors or Tensor): result of the dot product with the same shape as the inputs.    Return type 
output (tuple)   Example >>> def exp_reducer(x):
...   return x.exp().sum(dim=1)
>>> inputs = torch.rand(4, 4)
>>> v = torch.ones(4)
>>> vjp(exp_reducer, inputs, v)
(tensor([5.7817, 7.2458, 5.7830, 6.7782]),
 tensor([[1.4458, 1.3962, 1.3042, 1.6354],
        [2.1288, 1.0652, 1.5483, 2.5035],
        [2.2046, 1.1292, 1.1432, 1.3059],
        [1.3225, 1.6652, 1.7753, 2.0152]]))
 >>> vjp(exp_reducer, inputs, v, create_graph=True)
(tensor([5.7817, 7.2458, 5.7830, 6.7782], grad_fn=<SumBackward1>),
 tensor([[1.4458, 1.3962, 1.3042, 1.6354],
        [2.1288, 1.0652, 1.5483, 2.5035],
        [2.2046, 1.1292, 1.1432, 1.3059],
        [1.3225, 1.6652, 1.7753, 2.0152]], grad_fn=<MulBackward0>))
 >>> def adder(x, y):
...   return 2 * x + 3 * y
>>> inputs = (torch.rand(2), torch.rand(2))
>>> v = torch.ones(2)
>>> vjp(adder, inputs, v)
(tensor([2.4225, 2.3340]),
 (tensor([2., 2.]), tensor([3., 3.])))
torch.autograd.grad(outputs, inputs, grad_outputs=None, retain_graph=None, create_graph=False, only_inputs=True, allow_unused=False) [source]
 
Computes and returns the sum of gradients of outputs w.r.t. the inputs. grad_outputs should be a sequence of length matching output containing the “vector” in Jacobian-vector product, usually the pre-computed gradients w.r.t. each of the outputs. If an output doesn’t require_grad, then the gradient can be None). If only_inputs is True, the function will only return a list of gradients w.r.t the specified inputs. If it’s False, then gradient w.r.t. all remaining leaves will still be computed, and will be accumulated into their .grad attribute.  Note If you run any forward ops, create grad_outputs, and/or call grad in a user-specified CUDA stream context, see Stream semantics of backward passes.   Parameters 
 
outputs (sequence of Tensor) – outputs of the differentiated function. 
inputs (sequence of Tensor) – Inputs w.r.t. which the gradient will be returned (and not accumulated into .grad). 
grad_outputs (sequence of Tensor) – The “vector” in the Jacobian-vector product. Usually gradients w.r.t. each output. None values can be specified for scalar Tensors or ones that don’t require grad. If a None value would be acceptable for all grad_tensors, then this argument is optional. Default: None. 
retain_graph (bool, optional) – If False, the graph used to compute the grad will be freed. Note that in nearly all cases setting this option to True is not needed and often can be worked around in a much more efficient way. Defaults to the value of create_graph. 
create_graph (bool, optional) – If True, graph of the derivative will be constructed, allowing to compute higher order derivative products. Default: False. 
allow_unused (bool, optional) – If False, specifying inputs that were not used when computing outputs (and therefore their grad is always zero) is an error. Defaults to False.
torch.autograd.gradcheck(func, inputs, eps=1e-06, atol=1e-05, rtol=0.001, raise_exception=True, check_sparse_nnz=False, nondet_tol=0.0, check_undefined_grad=True, check_grad_dtypes=False, check_batched_grad=False) [source]
 
Check gradients computed via small finite differences against analytical gradients w.r.t. tensors in inputs that are of floating point or complex type and with requires_grad=True. The check between numerical and analytical gradients uses allclose(). For complex functions, no notion of Jacobian exists. Gradcheck verifies if the numerical and analytical values of Wirtinger and Conjugate Wirtinger derivative are consistent. The gradient computation is done under the assumption that the overall function has a real valued output. For functions with complex output, gradcheck compares the numerical and analytical gradients for two values of grad_output: 1 and 1j. For more details, check out Autograd for Complex Numbers.  Note The default values are designed for input of double precision. This check will likely fail if input is of less precision, e.g., FloatTensor.   Warning If any checked tensor in input has overlapping memory, i.e., different indices pointing to the same memory address (e.g., from torch.expand()), this check will likely fail because the numerical gradients computed by point perturbation at such indices will change values at all other indices that share the same memory address.   Parameters 
 
func (function) – a Python function that takes Tensor inputs and returns a Tensor or a tuple of Tensors 
inputs (tuple of Tensor or Tensor) – inputs to the function 
eps (float, optional) – perturbation for finite differences 
atol (float, optional) – absolute tolerance 
rtol (float, optional) – relative tolerance 
raise_exception (bool, optional) – indicating whether to raise an exception if the check fails. The exception gives more information about the exact nature of the failure. This is helpful when debugging gradchecks. 
check_sparse_nnz (bool, optional) – if True, gradcheck allows for SparseTensor input, and for any SparseTensor at input, gradcheck will perform check at nnz positions only. 
nondet_tol (float, optional) – tolerance for non-determinism. When running identical inputs through the differentiation, the results must either match exactly (default, 0.0) or be within this tolerance. 
check_undefined_grad (bool, optional) – if True, check if undefined output grads are supported and treated as zeros, for Tensor outputs. 
check_batched_grad (bool, optional) – if True, check if we can compute batched gradients using prototype vmap support. Defaults to False.   Returns 
True if all differences satisfy allclose condition