neulab/docprompting-conala huggingface dataset:
torch.cuda.memory_stats(device=None) [source]
 
Returns a dictionary of CUDA memory allocator statistics for a given device. The return value of this function is a dictionary of statistics, each of which is a non-negative integer. Core statistics:  
"allocated.{all,large_pool,small_pool}.{current,peak,allocated,freed}": number of allocation requests received by the memory allocator. 
"allocated_bytes.{all,large_pool,small_pool}.{current,peak,allocated,freed}": amount of allocated memory. 
"segment.{all,large_pool,small_pool}.{current,peak,allocated,freed}": number of reserved segments from cudaMalloc(). 
"reserved_bytes.{all,large_pool,small_pool}.{current,peak,allocated,freed}": amount of reserved memory. 
"active.{all,large_pool,small_pool}.{current,peak,allocated,freed}": number of active memory blocks. 
"active_bytes.{all,large_pool,small_pool}.{current,peak,allocated,freed}": amount of active memory. 
"inactive_split.{all,large_pool,small_pool}.{current,peak,allocated,freed}": number of inactive, non-releasable memory blocks. 
"inactive_split_bytes.{all,large_pool,small_pool}.{current,peak,allocated,freed}": amount of inactive, non-releasable memory.  For these core statistics, values are broken down as follows. Pool type:  
all: combined statistics across all memory pools. 
large_pool: statistics for the large allocation pool (as of October 2019, for size >= 1MB allocations). 
small_pool: statistics for the small allocation pool (as of October 2019, for size < 1MB allocations).  Metric type:  
current: current value of this metric. 
peak: maximum value of this metric. 
allocated: historical total increase in this metric. 
freed: historical total decrease in this metric.  In addition to the core statistics, we also provide some simple event counters:  
"num_alloc_retries": number of failed cudaMalloc calls that result in a cache flush and retry. 
"num_ooms": number of out-of-memory errors thrown.   Parameters 
device (torch.device or int, optional) – selected device. Returns statistics for the current device, given by current_device(), if device is None (default).    Note See Memory management for more details about GPU memory management.
torch.cuda.memory_summary(device=None, abbreviated=False) [source]
 
Returns a human-readable printout of the current memory allocator statistics for a given device. This can be useful to display periodically during training, or when handling out-of-memory exceptions.  Parameters 
 
device (torch.device or int, optional) – selected device. Returns printout for the current device, given by current_device(), if device is None (default). 
abbreviated (bool, optional) – whether to return an abbreviated summary (default: False).     Note See Memory management for more details about GPU memory management.
torch.cuda.nvtx.mark(msg) [source]
 
Describe an instantaneous event that occurred at some point.  Parameters 
msg (string) – ASCII message to associate with the event.
torch.cuda.nvtx.range_pop() [source]
 
Pops a range off of a stack of nested range spans. Returns the zero-based depth of the range that is ended.
torch.cuda.nvtx.range_push(msg) [source]
 
Pushes a range onto a stack of nested range span. Returns zero-based depth of the range that is started.  Parameters 
msg (string) – ASCII message to associate with range
torch.cuda.reset_max_memory_allocated(device=None) [source]
 
Resets the starting point in tracking maximum GPU memory occupied by tensors for a given device. See max_memory_allocated() for details.  Parameters 
device (torch.device or int, optional) – selected device. Returns statistic for the current device, given by current_device(), if device is None (default).    Warning This function now calls reset_peak_memory_stats(), which resets /all/ peak memory stats.   Note See Memory management for more details about GPU memory management.
torch.cuda.reset_max_memory_cached(device=None) [source]
 
Resets the starting point in tracking maximum GPU memory managed by the caching allocator for a given device. See max_memory_cached() for details.  Parameters 
device (torch.device or int, optional) – selected device. Returns statistic for the current device, given by current_device(), if device is None (default).    Warning This function now calls reset_peak_memory_stats(), which resets /all/ peak memory stats.   Note See Memory management for more details about GPU memory management.
torch.cuda.seed() [source]
 
Sets the seed for generating random numbers to a random number for the current GPU. It’s safe to call this function if CUDA is not available; in that case, it is silently ignored.  Warning If you are working with a multi-GPU model, this function will only initialize the seed on one GPU. To initialize all GPUs, use seed_all().
torch.cuda.seed_all() [source]
 
Sets the seed for generating random numbers to a random number on all GPUs. It’s safe to call this function if CUDA is not available; in that case, it is silently ignored.
torch.cuda.set_device(device) [source]
 
Sets the current device. Usage of this function is discouraged in favor of device. In most cases it’s better to use CUDA_VISIBLE_DEVICES environmental variable.  Parameters 
device (torch.device or int) – selected device. This function is a no-op if this argument is negative.
torch.cuda.set_per_process_memory_fraction(fraction, device=None) [source]
 
Set memory fraction for a process. The fraction is used to limit an caching allocator to allocated memory on a CUDA device. The allowed value equals the total visible memory multiplied fraction. If trying to allocate more than the allowed value in a process, will raise an out of memory error in allocator.  Parameters 
 
fraction (float) – Range: 0~1. Allowed memory equals total_memory * fraction. 
device (torch.device or int, optional) – selected device. If it is None the default CUDA device is used.     Note In general, the total available free memory is less than the total capacity.
torch.cuda.set_rng_state(new_state, device='cuda') [source]
 
Sets the random number generator state of the specified GPU.  Parameters 
 
new_state (torch.ByteTensor) – The desired state 
device (torch.device or int, optional) – The device to set the RNG state. Default: 'cuda' (i.e., torch.device('cuda'), the current CUDA device).
torch.cuda.set_rng_state_all(new_states) [source]
 
Sets the random number generator state of all devices.  Parameters 
new_states (Iterable of torch.ByteTensor) – The desired state for each device
class torch.cuda.Stream [source]
 
Wrapper around a CUDA stream. A CUDA stream is a linear sequence of execution that belongs to a specific device, independent from other streams. See CUDA semantics for details.  Parameters 
 
device (torch.device or int, optional) – a device on which to allocate the stream. If device is None (default) or a negative integer, this will use the current device. 
priority (int, optional) – priority of the stream. Can be either -1 (high priority) or 0 (low priority). By default, streams have priority 0.     Note Although CUDA versions >= 11 support more than two levels of priorities, in PyTorch, we only support two levels of priorities.   
query() [source]
 
Checks if all the work submitted has been completed.  Returns 
A boolean indicating if all kernels in this stream are completed.   
  
record_event(event=None) [source]
 
Records an event.  Parameters 
event (Event, optional) – event to record. If not given, a new one will be allocated.  Returns 
Recorded event.   
  
synchronize() [source]
 
Wait for all the kernels in this stream to complete.  Note This is a wrapper around cudaStreamSynchronize(): see CUDA Stream documentation for more info.  
  
wait_event(event) [source]
 
Makes all future work submitted to the stream wait for an event.  Parameters 
event (Event) – an event to wait for.    Note This is a wrapper around cudaStreamWaitEvent(): see CUDA Stream documentation for more info. This function returns without waiting for event: only future operations are affected.  
  
wait_stream(stream) [source]
 
Synchronizes with another stream. All future work submitted to this stream will wait until all kernels submitted to a given stream at the time of call complete.  Parameters 
stream (Stream) – a stream to synchronize.    Note This function returns without waiting for currently enqueued kernels in stream: only future operations are affected.
torch.cuda.stream(stream) [source]
 
Context-manager that selects a given stream. All CUDA kernels queued within its context will be enqueued on a selected stream.  Parameters 
stream (Stream) – selected stream. This manager is a no-op if it’s None.    Note Streams are per-device. If the selected stream is not on the current device, this function will also change the current device to match the stream.
query() [source]
 
Checks if all the work submitted has been completed.  Returns 
A boolean indicating if all kernels in this stream are completed.
record_event(event=None) [source]
 
Records an event.  Parameters 
event (Event, optional) – event to record. If not given, a new one will be allocated.  Returns 
Recorded event.
synchronize() [source]
 
Wait for all the kernels in this stream to complete.  Note This is a wrapper around cudaStreamSynchronize(): see CUDA Stream documentation for more info.
wait_event(event) [source]
 
Makes all future work submitted to the stream wait for an event.  Parameters 
event (Event) – an event to wait for.    Note This is a wrapper around cudaStreamWaitEvent(): see CUDA Stream documentation for more info. This function returns without waiting for event: only future operations are affected.
wait_stream(stream) [source]
 
Synchronizes with another stream. All future work submitted to this stream will wait until all kernels submitted to a given stream at the time of call complete.  Parameters 
stream (Stream) – a stream to synchronize.    Note This function returns without waiting for currently enqueued kernels in stream: only future operations are affected.
torch.cuda.synchronize(device=None) [source]
 
Waits for all kernels in all streams on a CUDA device to complete.  Parameters 
device (torch.device or int, optional) – device for which to synchronize. It uses the current device, given by current_device(), if device is None (default).
torch.cummax(input, dim, *, out=None) -> (Tensor, LongTensor)  
Returns a namedtuple (values, indices) where values is the cumulative maximum of elements of input in the dimension dim. And indices is the index location of each maximum value found in the dimension dim.  yi=max(x1,x2,x3,…,xi)y_i = max(x_1, x_2, x_3, \dots, x_i)  
 Parameters 
 
input (Tensor) – the input tensor. 
dim (int) – the dimension to do the operation over   Keyword Arguments 
out (tuple, optional) – the result tuple of two output tensors (values, indices)   Example: >>> a = torch.randn(10)
>>> a
tensor([-0.3449, -1.5447,  0.0685, -1.5104, -1.1706,  0.2259,  1.4696, -1.3284,
     1.9946, -0.8209])
>>> torch.cummax(a, dim=0)
torch.return_types.cummax(
    values=tensor([-0.3449, -0.3449,  0.0685,  0.0685,  0.0685,  0.2259,  1.4696,  1.4696,
     1.9946,  1.9946]),
    indices=tensor([0, 0, 2, 2, 2, 5, 6, 6, 8, 8]))
torch.cummin(input, dim, *, out=None) -> (Tensor, LongTensor)  
Returns a namedtuple (values, indices) where values is the cumulative minimum of elements of input in the dimension dim. And indices is the index location of each maximum value found in the dimension dim.  yi=min(x1,x2,x3,…,xi)y_i = min(x_1, x_2, x_3, \dots, x_i)  
 Parameters 
 
input (Tensor) – the input tensor. 
dim (int) – the dimension to do the operation over   Keyword Arguments 
out (tuple, optional) – the result tuple of two output tensors (values, indices)   Example: >>> a = torch.randn(10)
>>> a
tensor([-0.2284, -0.6628,  0.0975,  0.2680, -1.3298, -0.4220, -0.3885,  1.1762,
     0.9165,  1.6684])
>>> torch.cummin(a, dim=0)
torch.return_types.cummin(
    values=tensor([-0.2284, -0.6628, -0.6628, -0.6628, -1.3298, -1.3298, -1.3298, -1.3298,
    -1.3298, -1.3298]),
    indices=tensor([0, 1, 1, 1, 4, 4, 4, 4, 4, 4]))
torch.cumprod(input, dim, *, dtype=None, out=None) → Tensor  
Returns the cumulative product of elements of input in the dimension dim. For example, if input is a vector of size N, the result will also be a vector of size N, with elements.  yi=x1×x2×x3×⋯×xiy_i = x_1 \times x_2\times x_3\times \dots \times x_i  
 Parameters 
 
input (Tensor) – the input tensor. 
dim (int) – the dimension to do the operation over   Keyword Arguments 
 
dtype (torch.dtype, optional) – the desired data type of returned tensor. If specified, the input tensor is casted to dtype before the operation is performed. This is useful for preventing data type overflows. Default: None. 
out (Tensor, optional) – the output tensor.    Example: >>> a = torch.randn(10)
>>> a
tensor([ 0.6001,  0.2069, -0.1919,  0.9792,  0.6727,  1.0062,  0.4126,
        -0.2129, -0.4206,  0.1968])
>>> torch.cumprod(a, dim=0)
tensor([ 0.6001,  0.1241, -0.0238, -0.0233, -0.0157, -0.0158, -0.0065,
         0.0014, -0.0006, -0.0001])

>>> a[5] = 0.0
>>> torch.cumprod(a, dim=0)
tensor([ 0.6001,  0.1241, -0.0238, -0.0233, -0.0157, -0.0000, -0.0000,
         0.0000, -0.0000, -0.0000])
torch.cumsum(input, dim, *, dtype=None, out=None) → Tensor  
Returns the cumulative sum of elements of input in the dimension dim. For example, if input is a vector of size N, the result will also be a vector of size N, with elements.  yi=x1+x2+x3+⋯+xiy_i = x_1 + x_2 + x_3 + \dots + x_i  
 Parameters 
 
input (Tensor) – the input tensor. 
dim (int) – the dimension to do the operation over   Keyword Arguments 
 
dtype (torch.dtype, optional) – the desired data type of returned tensor. If specified, the input tensor is casted to dtype before the operation is performed. This is useful for preventing data type overflows. Default: None. 
out (Tensor, optional) – the output tensor.    Example: >>> a = torch.randn(10)
>>> a
tensor([-0.8286, -0.4890,  0.5155,  0.8443,  0.1865, -0.1752, -2.0595,
         0.1850, -1.1571, -0.4243])
>>> torch.cumsum(a, dim=0)
tensor([-0.8286, -1.3175, -0.8020,  0.0423,  0.2289,  0.0537, -2.0058,
        -1.8209, -2.9780, -3.4022])
torch.deg2rad(input, *, out=None) → Tensor  
Returns a new tensor with each of the elements of input converted from angles in degrees to radians.  Parameters 
input (Tensor) – the input tensor.  Keyword Arguments 
out (Tensor, optional) – the output tensor.   Example: >>> a = torch.tensor([[180.0, -180.0], [360.0, -360.0], [90.0, -90.0]])
>>> torch.deg2rad(a)
tensor([[ 3.1416, -3.1416],
        [ 6.2832, -6.2832],
        [ 1.5708, -1.5708]])
torch.dequantize(tensor) → Tensor  
Returns an fp32 Tensor by dequantizing a quantized Tensor  Parameters 
tensor (Tensor) – A quantized Tensor    
torch.dequantize(tensors) → sequence of Tensors 
 Given a list of quantized Tensors, dequantize them and return a list of fp32 Tensors  Parameters 
tensors (sequence of Tensors) – A list of quantized Tensors
torch.det(input) → Tensor  
Calculates determinant of a square matrix or batches of square matrices.  Note torch.det() is deprecated. Please use torch.linalg.det() instead.   Note Backward through detdet  internally uses SVD results when input is not invertible. In this case, double backward through detdet  will be unstable when input doesn’t have distinct singular values. See torch.svd~torch.svd  for details.   Parameters 
input (Tensor) – the input tensor of size (*, n, n) where * is zero or more batch dimensions.   Example: >>> A = torch.randn(3, 3)
>>> torch.det(A)
tensor(3.7641)

>>> A = torch.randn(3, 2, 2)
>>> A
tensor([[[ 0.9254, -0.6213],
         [-0.5787,  1.6843]],

        [[ 0.3242, -0.9665],
         [ 0.4539, -0.0887]],

        [[ 1.1336, -0.4025],
         [-0.7089,  0.9032]]])
>>> A.det()
tensor([1.1990, 0.4099, 0.7386])
torch.diag(input, diagonal=0, *, out=None) → Tensor  
 If input is a vector (1-D tensor), then returns a 2-D square tensor with the elements of input as the diagonal. If input is a matrix (2-D tensor), then returns a 1-D tensor with the diagonal elements of input.  The argument diagonal controls which diagonal to consider:  If diagonal = 0, it is the main diagonal. If diagonal > 0, it is above the main diagonal. If diagonal < 0, it is below the main diagonal.   Parameters 
 
input (Tensor) – the input tensor. 
diagonal (int, optional) – the diagonal to consider   Keyword Arguments 
out (Tensor, optional) – the output tensor.    See also torch.diagonal() always returns the diagonal of its input. torch.diagflat() always constructs a tensor with diagonal elements specified by the input.  Examples: Get the square matrix where the input vector is the diagonal: >>> a = torch.randn(3)
>>> a
tensor([ 0.5950,-0.0872, 2.3298])
>>> torch.diag(a)
tensor([[ 0.5950, 0.0000, 0.0000],
        [ 0.0000,-0.0872, 0.0000],
        [ 0.0000, 0.0000, 2.3298]])
>>> torch.diag(a, 1)
tensor([[ 0.0000, 0.5950, 0.0000, 0.0000],
        [ 0.0000, 0.0000,-0.0872, 0.0000],
        [ 0.0000, 0.0000, 0.0000, 2.3298],
        [ 0.0000, 0.0000, 0.0000, 0.0000]])
 Get the k-th diagonal of a given matrix: >>> a = torch.randn(3, 3)
>>> a
tensor([[-0.4264, 0.0255,-0.1064],
        [ 0.8795,-0.2429, 0.1374],
        [ 0.1029,-0.6482,-1.6300]])
>>> torch.diag(a, 0)
tensor([-0.4264,-0.2429,-1.6300])
>>> torch.diag(a, 1)
tensor([ 0.0255, 0.1374])
torch.diagflat(input, offset=0) → Tensor  
 If input is a vector (1-D tensor), then returns a 2-D square tensor with the elements of input as the diagonal. If input is a tensor with more than one dimension, then returns a 2-D tensor with diagonal elements equal to a flattened input.  The argument offset controls which diagonal to consider:  If offset = 0, it is the main diagonal. If offset > 0, it is above the main diagonal. If offset < 0, it is below the main diagonal.   Parameters 
 
input (Tensor) – the input tensor. 
offset (int, optional) – the diagonal to consider. Default: 0 (main diagonal).    Examples: >>> a = torch.randn(3)
>>> a
tensor([-0.2956, -0.9068,  0.1695])
>>> torch.diagflat(a)
tensor([[-0.2956,  0.0000,  0.0000],
        [ 0.0000, -0.9068,  0.0000],
        [ 0.0000,  0.0000,  0.1695]])
>>> torch.diagflat(a, 1)
tensor([[ 0.0000, -0.2956,  0.0000,  0.0000],
        [ 0.0000,  0.0000, -0.9068,  0.0000],
        [ 0.0000,  0.0000,  0.0000,  0.1695],
        [ 0.0000,  0.0000,  0.0000,  0.0000]])

>>> a = torch.randn(2, 2)
>>> a
tensor([[ 0.2094, -0.3018],
        [-0.1516,  1.9342]])
>>> torch.diagflat(a)
tensor([[ 0.2094,  0.0000,  0.0000,  0.0000],
        [ 0.0000, -0.3018,  0.0000,  0.0000],
        [ 0.0000,  0.0000, -0.1516,  0.0000],
        [ 0.0000,  0.0000,  0.0000,  1.9342]])
torch.diagonal(input, offset=0, dim1=0, dim2=1) → Tensor  
Returns a partial view of input with the its diagonal elements with respect to dim1 and dim2 appended as a dimension at the end of the shape. The argument offset controls which diagonal to consider:  If offset = 0, it is the main diagonal. If offset > 0, it is above the main diagonal. If offset < 0, it is below the main diagonal.  Applying torch.diag_embed() to the output of this function with the same arguments yields a diagonal matrix with the diagonal entries of the input. However, torch.diag_embed() has different default dimensions, so those need to be explicitly specified.  Parameters 
 
input (Tensor) – the input tensor. Must be at least 2-dimensional. 
offset (int, optional) – which diagonal to consider. Default: 0 (main diagonal). 
dim1 (int, optional) – first dimension with respect to which to take diagonal. Default: 0. 
dim2 (int, optional) – second dimension with respect to which to take diagonal. Default: 1.     Note To take a batch diagonal, pass in dim1=-2, dim2=-1.  Examples: >>> a = torch.randn(3, 3)
>>> a
tensor([[-1.0854,  1.1431, -0.1752],
        [ 0.8536, -0.0905,  0.0360],
        [ 0.6927, -0.3735, -0.4945]])


>>> torch.diagonal(a, 0)
tensor([-1.0854, -0.0905, -0.4945])


>>> torch.diagonal(a, 1)
tensor([ 1.1431,  0.0360])


>>> x = torch.randn(2, 5, 4, 2)
>>> torch.diagonal(x, offset=-1, dim1=1, dim2=2)
tensor([[[-1.2631,  0.3755, -1.5977, -1.8172],
         [-1.1065,  1.0401, -0.2235, -0.7938]],

        [[-1.7325, -0.3081,  0.6166,  0.2335],
         [ 1.0500,  0.7336, -0.3836, -1.1015]]])
torch.diag_embed(input, offset=0, dim1=-2, dim2=-1) → Tensor  
Creates a tensor whose diagonals of certain 2D planes (specified by dim1 and dim2) are filled by input. To facilitate creating batched diagonal matrices, the 2D planes formed by the last two dimensions of the returned tensor are chosen by default. The argument offset controls which diagonal to consider:  If offset = 0, it is the main diagonal. If offset > 0, it is above the main diagonal. If offset < 0, it is below the main diagonal.  The size of the new matrix will be calculated to make the specified diagonal of the size of the last input dimension. Note that for offset other than 00 , the order of dim1 and dim2 matters. Exchanging them is equivalent to changing the sign of offset. Applying torch.diagonal() to the output of this function with the same arguments yields a matrix identical to input. However, torch.diagonal() has different default dimensions, so those need to be explicitly specified.  Parameters 
 
input (Tensor) – the input tensor. Must be at least 1-dimensional. 
offset (int, optional) – which diagonal to consider. Default: 0 (main diagonal). 
dim1 (int, optional) – first dimension with respect to which to take diagonal. Default: -2. 
dim2 (int, optional) – second dimension with respect to which to take diagonal. Default: -1.    Example: >>> a = torch.randn(2, 3)
>>> torch.diag_embed(a)
tensor([[[ 1.5410,  0.0000,  0.0000],
         [ 0.0000, -0.2934,  0.0000],
         [ 0.0000,  0.0000, -2.1788]],

        [[ 0.5684,  0.0000,  0.0000],
         [ 0.0000, -1.0845,  0.0000],
         [ 0.0000,  0.0000, -1.3986]]])

>>> torch.diag_embed(a, offset=1, dim1=0, dim2=2)
tensor([[[ 0.0000,  1.5410,  0.0000,  0.0000],
         [ 0.0000,  0.5684,  0.0000,  0.0000]],

        [[ 0.0000,  0.0000, -0.2934,  0.0000],
         [ 0.0000,  0.0000, -1.0845,  0.0000]],

        [[ 0.0000,  0.0000,  0.0000, -2.1788],
         [ 0.0000,  0.0000,  0.0000, -1.3986]],

        [[ 0.0000,  0.0000,  0.0000,  0.0000],
         [ 0.0000,  0.0000,  0.0000,  0.0000]]])
torch.diff(input, n=1, dim=-1, prepend=None, append=None) → Tensor  
Computes the n-th forward difference along the given dimension. The first-order differences are given by out[i] = input[i + 1] - input[i]. Higher-order differences are calculated by using torch.diff() recursively.  Note Only n = 1 is currently supported   Parameters 
 
input (Tensor) – the tensor to compute the differences on 
n (int, optional) – the number of times to recursively compute the difference 
dim (int, optional) – the dimension to compute the difference along. Default is the last dimension. 
append (prepend,) – values to prepend or append to input along dim before computing the difference. Their dimensions must be equivalent to that of input, and their shapes must match input’s shape except on dim.   Keyword Arguments 
out (Tensor, optional) – the output tensor.   Example: >>> a = torch.tensor([1, 3, 2])
>>> torch.diff(a)
tensor([ 2, -1])
>>> b = torch.tensor([4, 5])
>>> torch.diff(a, append=b)
tensor([ 2, -1,  2,  1])
>>> c = torch.tensor([[1, 2, 3], [3, 4, 5]])
>>> torch.diff(c, dim=0)
tensor([[2, 2, 2]])
>>> torch.diff(c, dim=1)
tensor([[1, 1],
        [1, 1]])
torch.digamma(input, *, out=None) → Tensor  
Computes the logarithmic derivative of the gamma function on input.  ψ(x)=ddxln⁡(Γ(x))=Γ′(x)Γ(x)\psi(x) = \frac{d}{dx} \ln\left(\Gamma\left(x\right)\right) = \frac{\Gamma'(x)}{\Gamma(x)}  
 Parameters 
input (Tensor) – the tensor to compute the digamma function on  Keyword Arguments 
out (Tensor, optional) – the output tensor.    Note This function is similar to SciPy’s scipy.special.digamma.   Note From PyTorch 1.8 onwards, the digamma function returns -Inf for 0. Previously it returned NaN for 0.  Example: >>> a = torch.tensor([1, 0.5])
>>> torch.digamma(a)
tensor([-0.5772, -1.9635])