neulab/docprompting-conala huggingface dataset:
classmethod apply(module, name, amount) [source]
 
Adds the forward pre-hook that enables pruning on the fly and the reparametrization of a tensor in terms of the original tensor and the pruning mask.  Parameters 
 
module (nn.Module) – module containing the tensor to prune 
name (str) – parameter name within module on which pruning will act. 
amount (int or float) – quantity of parameters to prune. If float, should be between 0.0 and 1.0 and represent the fraction of parameters to prune. If int, it represents the absolute number of parameters to prune.
torch.nn.utils.prune.random_structured(module, name, amount, dim) [source]
 
Prunes tensor corresponding to parameter called name in module by removing the specified amount of (currently unpruned) channels along the specified dim selected at random. Modifies module in place (and also return the modified module) by: 1) adding a named buffer called name+'_mask' corresponding to the binary mask applied to the parameter name by the pruning method. 2) replacing the parameter name by its pruned version, while the original (unpruned) parameter is stored in a new parameter named name+'_orig'.  Parameters 
 
module (nn.Module) – module containing the tensor to prune 
name (str) – parameter name within module on which pruning will act. 
amount (int or float) – quantity of parameters to prune. If float, should be between 0.0 and 1.0 and represent the fraction of parameters to prune. If int, it represents the absolute number of parameters to prune. 
dim (int) – index of the dim along which we define channels to prune.   Returns 
modified (i.e. pruned) version of the input module  Return type 
module (nn.Module)   Examples >>> m = prune.random_structured(
        nn.Linear(5, 3), 'weight', amount=3, dim=1
    )
>>> columns_pruned = int(sum(torch.sum(m.weight, dim=0) == 0))
>>> print(columns_pruned)
3
torch.nn.utils.prune.random_unstructured(module, name, amount) [source]
 
Prunes tensor corresponding to parameter called name in module by removing the specified amount of (currently unpruned) units selected at random. Modifies module in place (and also return the modified module) by: 1) adding a named buffer called name+'_mask' corresponding to the binary mask applied to the parameter name by the pruning method. 2) replacing the parameter name by its pruned version, while the original (unpruned) parameter is stored in a new parameter named name+'_orig'.  Parameters 
 
module (nn.Module) – module containing the tensor to prune 
name (str) – parameter name within module on which pruning will act. 
amount (int or float) – quantity of parameters to prune. If float, should be between 0.0 and 1.0 and represent the fraction of parameters to prune. If int, it represents the absolute number of parameters to prune.   Returns 
modified (i.e. pruned) version of the input module  Return type 
module (nn.Module)   Examples >>> m = prune.random_unstructured(nn.Linear(2, 3), 'weight', amount=1)
>>> torch.sum(m.weight_mask == 0)
tensor(1)
torch.nn.utils.prune.remove(module, name) [source]
 
Removes the pruning reparameterization from a module and the pruning method from the forward hook. The pruned parameter named name remains permanently pruned, and the parameter named name+'_orig' is removed from the parameter list. Similarly, the buffer named name+'_mask' is removed from the buffers.  Note Pruning itself is NOT undone or reversed!   Parameters 
 
module (nn.Module) – module containing the tensor to prune 
name (str) – parameter name within module on which pruning will act.    Examples >>> m = random_unstructured(nn.Linear(5, 7), name='weight', amount=0.2)
>>> m = remove(m, name='weight')
torch.nn.utils.remove_spectral_norm(module, name='weight') [source]
 
Removes the spectral normalization reparameterization from a module.  Parameters 
 
module (Module) – containing module 
name (str, optional) – name of weight parameter    Example >>> m = spectral_norm(nn.Linear(40, 10))
>>> remove_spectral_norm(m)
torch.nn.utils.remove_weight_norm(module, name='weight') [source]
 
Removes the weight normalization reparameterization from a module.  Parameters 
 
module (Module) – containing module 
name (str, optional) – name of weight parameter    Example >>> m = weight_norm(nn.Linear(20, 40))
>>> remove_weight_norm(m)
class torch.nn.utils.rnn.PackedSequence [source]
 
Holds the data and list of batch_sizes of a packed sequence. All RNN modules accept packed sequences as inputs.  Note Instances of this class should never be created manually. They are meant to be instantiated by functions like pack_padded_sequence(). Batch sizes represent the number elements at each sequence step in the batch, not the varying sequence lengths passed to pack_padded_sequence(). For instance, given data abc and x the PackedSequence would contain data axbc with batch_sizes=[2,1,1].   Variables 
 
~PackedSequence.data (Tensor) – Tensor containing packed sequence 
~PackedSequence.batch_sizes (Tensor) – Tensor of integers holding information about the batch size at each sequence step 
~PackedSequence.sorted_indices (Tensor, optional) – Tensor of integers holding how this PackedSequence is constructed from sequences. 
~PackedSequence.unsorted_indices (Tensor, optional) – Tensor of integers holding how this to recover the original sequences with correct order.     Note data can be on arbitrary device and of arbitrary dtype. sorted_indices and unsorted_indices must be torch.int64 tensors on the same device as data. However, batch_sizes should always be a CPU torch.int64 tensor. This invariant is maintained throughout PackedSequence class, and all functions that construct a :class:PackedSequence in PyTorch (i.e., they only pass in tensors conforming to this constraint).   
property batch_sizes  
Alias for field number 1 
  
count()  
Return number of occurrences of value. 
  
property data  
Alias for field number 0 
  
index()  
Return first index of value. Raises ValueError if the value is not present. 
  
property is_cuda  
Returns true if self.data stored on a gpu 
  
is_pinned() [source]
 
Returns true if self.data stored on in pinned memory 
  
property sorted_indices  
Alias for field number 2 
  
to(*args, **kwargs) [source]
 
Performs dtype and/or device conversion on self.data. It has similar signature as torch.Tensor.to(), except optional arguments like non_blocking and copy should be passed as kwargs, not args, or they will not apply to the index tensors.  Note If the self.data Tensor already has the correct torch.dtype and torch.device, then self is returned. Otherwise, returns a copy with the desired configuration.  
  
property unsorted_indices  
Alias for field number 3
is_pinned() [source]
 
Returns true if self.data stored on in pinned memory
to(*args, **kwargs) [source]
 
Performs dtype and/or device conversion on self.data. It has similar signature as torch.Tensor.to(), except optional arguments like non_blocking and copy should be passed as kwargs, not args, or they will not apply to the index tensors.  Note If the self.data Tensor already has the correct torch.dtype and torch.device, then self is returned. Otherwise, returns a copy with the desired configuration.
torch.nn.utils.rnn.pack_padded_sequence(input, lengths, batch_first=False, enforce_sorted=True) [source]
 
Packs a Tensor containing padded sequences of variable length. input can be of size T x B x * where T is the length of the longest sequence (equal to lengths[0]), B is the batch size, and * is any number of dimensions (including 0). If batch_first is True, B x T x * input is expected. For unsorted sequences, use enforce_sorted = False. If enforce_sorted is True, the sequences should be sorted by length in a decreasing order, i.e. input[:,0] should be the longest sequence, and input[:,B-1] the shortest one. enforce_sorted = True is only necessary for ONNX export.  Note This function accepts any input that has at least two dimensions. You can apply it to pack the labels, and use the output of the RNN with them to compute the loss directly. A Tensor can be retrieved from a PackedSequence object by accessing its .data attribute.   Parameters 
 
input (Tensor) – padded batch of variable length sequences. 
lengths (Tensor or list(int)) – list of sequence lengths of each batch element (must be on the CPU if provided as a tensor). 
batch_first (bool, optional) – if True, the input is expected in B x T x * format. 
enforce_sorted (bool, optional) – if True, the input is expected to contain sequences sorted by length in a decreasing order. If False, the input will get sorted unconditionally. Default: True.   Returns 
a PackedSequence object
torch.nn.utils.rnn.pack_sequence(sequences, enforce_sorted=True) [source]
 
Packs a list of variable length Tensors sequences should be a list of Tensors of size L x *, where L is the length of a sequence and * is any number of trailing dimensions, including zero. For unsorted sequences, use enforce_sorted = False. If enforce_sorted is True, the sequences should be sorted in the order of decreasing length. enforce_sorted = True is only necessary for ONNX export. Example >>> from torch.nn.utils.rnn import pack_sequence
>>> a = torch.tensor([1,2,3])
>>> b = torch.tensor([4,5])
>>> c = torch.tensor([6])
>>> pack_sequence([a, b, c])
PackedSequence(data=tensor([ 1,  4,  6,  2,  5,  3]), batch_sizes=tensor([ 3,  2,  1]))
  Parameters 
 
sequences (list[Tensor]) – A list of sequences of decreasing length. 
enforce_sorted (bool, optional) – if True, checks that the input contains sequences sorted by length in a decreasing order. If False, this condition is not checked. Default: True.   Returns 
a PackedSequence object
torch.nn.utils.rnn.pad_packed_sequence(sequence, batch_first=False, padding_value=0.0, total_length=None) [source]
 
Pads a packed batch of variable length sequences. It is an inverse operation to pack_padded_sequence(). The returned Tensor’s data will be of size T x B x *, where T is the length of the longest sequence and B is the batch size. If batch_first is True, the data will be transposed into B x T x * format. Example >>> from torch.nn.utils.rnn import pack_padded_sequence, pad_packed_sequence
>>> seq = torch.tensor([[1,2,0], [3,0,0], [4,5,6]])
>>> lens = [2, 1, 3]
>>> packed = pack_padded_sequence(seq, lens, batch_first=True, enforce_sorted=False)
>>> packed
PackedSequence(data=tensor([4, 1, 3, 5, 2, 6]), batch_sizes=tensor([3, 2, 1]),
               sorted_indices=tensor([2, 0, 1]), unsorted_indices=tensor([1, 2, 0]))
>>> seq_unpacked, lens_unpacked = pad_packed_sequence(packed, batch_first=True)
>>> seq_unpacked
tensor([[1, 2, 0],
        [3, 0, 0],
        [4, 5, 6]])
>>> lens_unpacked
tensor([2, 1, 3])
  Note total_length is useful to implement the pack sequence -> recurrent network -> unpack sequence pattern in a Module wrapped in DataParallel. See this FAQ section for details.   Parameters 
 
sequence (PackedSequence) – batch to pad 
batch_first (bool, optional) – if True, the output will be in B x T x * format. 
padding_value (float, optional) – values for padded elements. 
total_length (int, optional) – if not None, the output will be padded to have length total_length. This method will throw ValueError if total_length is less than the max sequence length in sequence.   Returns 
Tuple of Tensor containing the padded sequence, and a Tensor containing the list of lengths of each sequence in the batch. Batch elements will be re-ordered as they were ordered originally when the batch was passed to pack_padded_sequence or pack_sequence.
torch.nn.utils.rnn.pad_sequence(sequences, batch_first=False, padding_value=0.0) [source]
 
Pad a list of variable length Tensors with padding_value pad_sequence stacks a list of Tensors along a new dimension, and pads them to equal length. For example, if the input is list of sequences with size L x * and if batch_first is False, and T x B x * otherwise. B is batch size. It is equal to the number of elements in sequences. T is length of the longest sequence. L is length of the sequence. * is any number of trailing dimensions, including none. Example >>> from torch.nn.utils.rnn import pad_sequence
>>> a = torch.ones(25, 300)
>>> b = torch.ones(22, 300)
>>> c = torch.ones(15, 300)
>>> pad_sequence([a, b, c]).size()
torch.Size([25, 3, 300])
  Note This function returns a Tensor of size T x B x * or B x T x * where T is the length of the longest sequence. This function assumes trailing dimensions and type of all the Tensors in sequences are same.   Parameters 
 
sequences (list[Tensor]) – list of variable length sequences. 
batch_first (bool, optional) – output will be in B x T x * if True, or in T x B x * otherwise 
padding_value (float, optional) – value for padded elements. Default: 0.   Returns 
Tensor of size T x B x * if batch_first is False. Tensor of size B x T x * otherwise
torch.nn.utils.spectral_norm(module, name='weight', n_power_iterations=1, eps=1e-12, dim=None) [source]
 
Applies spectral normalization to a parameter in the given module.  WSN=Wσ(W),σ(W)=max⁡h:h≠0∥Wh∥2∥h∥2\mathbf{W}_{SN} = \dfrac{\mathbf{W}}{\sigma(\mathbf{W})}, \sigma(\mathbf{W}) = \max_{\mathbf{h}: \mathbf{h} \ne 0} \dfrac{\|\mathbf{W} \mathbf{h}\|_2}{\|\mathbf{h}\|_2}  
Spectral normalization stabilizes the training of discriminators (critics) in Generative Adversarial Networks (GANs) by rescaling the weight tensor with spectral norm σ\sigma  of the weight matrix calculated using power iteration method. If the dimension of the weight tensor is greater than 2, it is reshaped to 2D in power iteration method to get spectral norm. This is implemented via a hook that calculates spectral norm and rescales weight before every forward() call. See Spectral Normalization for Generative Adversarial Networks .  Parameters 
 
module (nn.Module) – containing module 
name (str, optional) – name of weight parameter 
n_power_iterations (int, optional) – number of power iterations to calculate spectral norm 
eps (float, optional) – epsilon for numerical stability in calculating norms 
dim (int, optional) – dimension corresponding to number of outputs, the default is 0, except for modules that are instances of ConvTranspose{1,2,3}d, when it is 1
   Returns 
The original module with the spectral norm hook   Example: >>> m = spectral_norm(nn.Linear(20, 40))
>>> m
Linear(in_features=20, out_features=40, bias=True)
>>> m.weight_u.size()
torch.Size([40])
torch.nn.utils.vector_to_parameters(vec, parameters) [source]
 
Convert one vector to the parameters  Parameters 
 
vec (Tensor) – a single vector represents the parameters of a model. 
parameters (Iterable[Tensor]) – an iterator of Tensors that are the parameters of a model.
torch.nn.utils.weight_norm(module, name='weight', dim=0) [source]
 
Applies weight normalization to a parameter in the given module.  w=gv∥v∥\mathbf{w} = g \dfrac{\mathbf{v}}{\|\mathbf{v}\|}  
Weight normalization is a reparameterization that decouples the magnitude of a weight tensor from its direction. This replaces the parameter specified by name (e.g. 'weight') with two parameters: one specifying the magnitude (e.g. 'weight_g') and one specifying the direction (e.g. 'weight_v'). Weight normalization is implemented via a hook that recomputes the weight tensor from the magnitude and direction before every forward() call. By default, with dim=0, the norm is computed independently per output channel/plane. To compute a norm over the entire weight tensor, use dim=None. See https://arxiv.org/abs/1602.07868  Parameters 
 
module (Module) – containing module 
name (str, optional) – name of weight parameter 
dim (int, optional) – dimension over which to compute the norm   Returns 
The original module with the weight norm hook   Example: >>> m = weight_norm(nn.Linear(20, 40), name='weight')
>>> m
Linear(in_features=20, out_features=40, bias=True)
>>> m.weight_g.size()
torch.Size([40, 1])
>>> m.weight_v.size()
torch.Size([40, 20])
class torch.nn.ZeroPad2d(padding) [source]
 
Pads the input tensor boundaries with zero. For N-dimensional padding, use torch.nn.functional.pad().  Parameters 
padding (int, tuple) – the size of the padding. If is int, uses the same padding in all boundaries. If a 4-tuple, uses (padding_left\text{padding\_left} , padding_right\text{padding\_right} , padding_top\text{padding\_top} , padding_bottom\text{padding\_bottom} )    Shape:

 Input: (N,C,Hin,Win)(N, C, H_{in}, W_{in}) 
 
Output: (N,C,Hout,Wout)(N, C, H_{out}, W_{out})  where Hout=Hin+padding_top+padding_bottomH_{out} = H_{in} + \text{padding\_top} + \text{padding\_bottom}  Wout=Win+padding_left+padding_rightW_{out} = W_{in} + \text{padding\_left} + \text{padding\_right}      Examples: >>> m = nn.ZeroPad2d(2)
>>> input = torch.randn(1, 1, 3, 3)
>>> input
tensor([[[[-0.1678, -0.4418,  1.9466],
          [ 0.9604, -0.4219, -0.5241],
          [-0.9162, -0.5436, -0.6446]]]])
>>> m(input)
tensor([[[[ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000],
          [ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000],
          [ 0.0000,  0.0000, -0.1678, -0.4418,  1.9466,  0.0000,  0.0000],
          [ 0.0000,  0.0000,  0.9604, -0.4219, -0.5241,  0.0000,  0.0000],
          [ 0.0000,  0.0000, -0.9162, -0.5436, -0.6446,  0.0000,  0.0000],
          [ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000],
          [ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000]]]])
>>> # using different paddings for different sides
>>> m = nn.ZeroPad2d((1, 1, 2, 0))
>>> m(input)
tensor([[[[ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000],
          [ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000],
          [ 0.0000, -0.1678, -0.4418,  1.9466,  0.0000],
          [ 0.0000,  0.9604, -0.4219, -0.5241,  0.0000],
          [ 0.0000, -0.9162, -0.5436, -0.6446,  0.0000]]]])
torch.nonzero(input, *, out=None, as_tuple=False) → LongTensor or tuple of LongTensors  
 Note torch.nonzero(..., as_tuple=False) (default) returns a 2-D tensor where each row is the index for a nonzero value. torch.nonzero(..., as_tuple=True) returns a tuple of 1-D index tensors, allowing for advanced indexing, so x[x.nonzero(as_tuple=True)] gives all nonzero values of tensor x. Of the returned tuple, each index tensor contains nonzero indices for a certain dimension. See below for more details on the two behaviors. When input is on CUDA, torch.nonzero() causes host-device synchronization.  When as_tuple is ``False`` (default): Returns a tensor containing the indices of all non-zero elements of input. Each row in the result contains the indices of a non-zero element in input. The result is sorted lexicographically, with the last index changing the fastest (C-style). If input has nn  dimensions, then the resulting indices tensor out is of size (z×n)(z \times n) , where zz  is the total number of non-zero elements in the input tensor. When as_tuple is ``True``: Returns a tuple of 1-D tensors, one for each dimension in input, each containing the indices (in that dimension) of all non-zero elements of input . If input has nn  dimensions, then the resulting tuple contains nn  tensors of size zz , where zz  is the total number of non-zero elements in the input tensor. As a special case, when input has zero dimensions and a nonzero scalar value, it is treated as a one-dimensional tensor with one element.  Parameters 
input (Tensor) – the input tensor.  Keyword Arguments 
out (LongTensor, optional) – the output tensor containing indices  Returns 
If as_tuple is False, the output tensor containing indices. If as_tuple is True, one 1-D tensor for each dimension, containing the indices of each nonzero element along that dimension.  Return type 
LongTensor or tuple of LongTensor   Example: >>> torch.nonzero(torch.tensor([1, 1, 1, 0, 1]))
tensor([[ 0],
        [ 1],
        [ 2],
        [ 4]])
>>> torch.nonzero(torch.tensor([[0.6, 0.0, 0.0, 0.0],
...                             [0.0, 0.4, 0.0, 0.0],
...                             [0.0, 0.0, 1.2, 0.0],
...                             [0.0, 0.0, 0.0,-0.4]]))
tensor([[ 0,  0],
        [ 1,  1],
        [ 2,  2],
        [ 3,  3]])
>>> torch.nonzero(torch.tensor([1, 1, 1, 0, 1]), as_tuple=True)
(tensor([0, 1, 2, 4]),)
>>> torch.nonzero(torch.tensor([[0.6, 0.0, 0.0, 0.0],
...                             [0.0, 0.4, 0.0, 0.0],
...                             [0.0, 0.0, 1.2, 0.0],
...                             [0.0, 0.0, 0.0,-0.4]]), as_tuple=True)
(tensor([0, 1, 2, 3]), tensor([0, 1, 2, 3]))
>>> torch.nonzero(torch.tensor(5), as_tuple=True)
(tensor([0]),)
torch.norm(input, p='fro', dim=None, keepdim=False, out=None, dtype=None) [source]
 
Returns the matrix norm or vector norm of a given tensor.  Warning torch.norm is deprecated and may be removed in a future PyTorch release. Use torch.linalg.norm() instead, but note that torch.linalg.norm() has a different signature and slightly different behavior that is more consistent with NumPy’s numpy.linalg.norm.   Parameters 
 
input (Tensor) – The input tensor. Its data type must be either a floating point or complex type. For complex inputs, the norm is calculated using the absolute value of each element. If the input is complex and neither dtype nor out is specified, the result’s data type will be the corresponding floating point type (e.g. float if input is complexfloat). 
p (int, float, inf, -inf, 'fro', 'nuc', optional) – 
the order of norm. Default: 'fro' The following norms can be calculated:   
ord matrix norm vector norm   
’fro’ Frobenius norm –  
‘nuc’ nuclear norm –  
Number – sum(abs(x)**ord)**(1./ord)   The vector norm can be calculated across any number of dimensions. The corresponding dimensions of input are flattened into one dimension, and the norm is calculated on the flattened dimension. Frobenius norm produces the same result as p=2 in all cases except when dim is a list of three or more dims, in which case Frobenius norm throws an error. Nuclear norm can only be calculated across exactly two dimensions.  
dim (int, tuple of python:ints, list of python:ints, optional) – Specifies which dimension or dimensions of input to calculate the norm across. If dim is None, the norm will be calculated across all dimensions of input. If the norm type indicated by p does not support the specified number of dimensions, an error will occur. 
keepdim (bool, optional) – whether the output tensors have dim retained or not. Ignored if dim = None and out = None. Default: False
 
out (Tensor, optional) – the output tensor. Ignored if dim = None and out = None. 
dtype (torch.dtype, optional) – the desired data type of returned tensor. If specified, the input tensor is casted to :attr:’dtype’ while performing the operation. Default: None.     Note Even though p='fro' supports any number of dimensions, the true mathematical definition of Frobenius norm only applies to tensors with exactly two dimensions. torch.linalg.norm() with ord='fro' aligns with the mathematical definition, since it can only be applied across exactly two dimensions.  Example: >>> import torch
>>> a = torch.arange(9, dtype= torch.float) - 4
>>> b = a.reshape((3, 3))
>>> torch.norm(a)
tensor(7.7460)
>>> torch.norm(b)
tensor(7.7460)
>>> torch.norm(a, float('inf'))
tensor(4.)
>>> torch.norm(b, float('inf'))
tensor(4.)
>>> c = torch.tensor([[ 1, 2, 3],[-1, 1, 4]] , dtype= torch.float)
>>> torch.norm(c, dim=0)
tensor([1.4142, 2.2361, 5.0000])
>>> torch.norm(c, dim=1)
tensor([3.7417, 4.2426])
>>> torch.norm(c, p=1, dim=1)
tensor([6., 6.])
>>> d = torch.arange(8, dtype= torch.float).reshape(2,2,2)
>>> torch.norm(d, dim=(1,2))
tensor([ 3.7417, 11.2250])
>>> torch.norm(d[0, :, :]), torch.norm(d[1, :, :])
(tensor(3.7417), tensor(11.2250))