neulab/docprompting-conala huggingface dataset:
torch.cross(input, other, dim=None, *, out=None) → Tensor  
Returns the cross product of vectors in dimension dim of input and other. input and other must have the same size, and the size of their dim dimension should be 3. If dim is not given, it defaults to the first dimension found with the size 3. Note that this might be unexpected.  Parameters 
 
input (Tensor) – the input tensor. 
other (Tensor) – the second input tensor 
dim (int, optional) – the dimension to take the cross-product in.   Keyword Arguments 
out (Tensor, optional) – the output tensor.   Example: >>> a = torch.randn(4, 3)
>>> a
tensor([[-0.3956,  1.1455,  1.6895],
        [-0.5849,  1.3672,  0.3599],
        [-1.1626,  0.7180, -0.0521],
        [-0.1339,  0.9902, -2.0225]])
>>> b = torch.randn(4, 3)
>>> b
tensor([[-0.0257, -1.4725, -1.2251],
        [-1.1479, -0.7005, -1.9757],
        [-1.3904,  0.3726, -1.1836],
        [-0.9688, -0.7153,  0.2159]])
>>> torch.cross(a, b, dim=1)
tensor([[ 1.0844, -0.5281,  0.6120],
        [-2.4490, -1.5687,  1.9792],
        [-0.8304, -1.3037,  0.5650],
        [-1.2329,  1.9883,  1.0551]])
>>> torch.cross(a, b)
tensor([[ 1.0844, -0.5281,  0.6120],
        [-2.4490, -1.5687,  1.9792],
        [-0.8304, -1.3037,  0.5650],
        [-1.2329,  1.9883,  1.0551]])
torch.cuda.amp.custom_bwd(bwd) [source]
 
Helper decorator for backward methods of custom autograd functions (subclasses of torch.autograd.Function). Ensures that backward executes with the same autocast state as forward. See the example page for more detail.
torch.cuda.amp.custom_fwd(fwd=None, **kwargs) [source]
 
Helper decorator for forward methods of custom autograd functions (subclasses of torch.autograd.Function). See the example page for more detail.  Parameters 
cast_inputs (torch.dtype or None, optional, default=None) – If not None, when forward runs in an autocast-enabled region, casts incoming floating-point CUDA Tensors to the target dtype (non-floating-point Tensors are not affected), then executes forward with autocast disabled. If None, forward’s internal ops execute with the current autocast state.    Note If the decorated forward is called outside an autocast-enabled region, custom_fwd is a no-op and cast_inputs has no effect.
get_backoff_factor() [source]
 
Returns a Python float containing the scale backoff factor.
get_growth_factor() [source]
 
Returns a Python float containing the scale growth factor.
get_growth_interval() [source]
 
Returns a Python int containing the growth interval.
get_scale() [source]
 
Returns a Python float containing the current scale, or 1.0 if scaling is disabled.  Warning get_scale() incurs a CPU-GPU sync.
load_state_dict(state_dict) [source]
 
Loads the scaler state. If this instance is disabled, load_state_dict() is a no-op.  Parameters 
state_dict (dict) – scaler state. Should be an object returned from a call to state_dict().
scale(outputs) [source]
 
Multiplies (‘scales’) a tensor or list of tensors by the scale factor. Returns scaled outputs. If this instance of GradScaler is not enabled, outputs are returned unmodified.  Parameters 
outputs (Tensor or iterable of Tensors) – Outputs to scale.
set_backoff_factor(new_factor) [source]
 
 Parameters 
new_scale (float) – Value to use as the new scale backoff factor.
set_growth_factor(new_factor) [source]
 
 Parameters 
new_scale (float) – Value to use as the new scale growth factor.
set_growth_interval(new_interval) [source]
 
 Parameters 
new_interval (int) – Value to use as the new growth interval.
state_dict() [source]
 
Returns the state of the scaler as a dict. It contains five entries:  
"scale" - a Python float containing the current scale 
"growth_factor" - a Python float containing the current growth factor 
"backoff_factor" - a Python float containing the current backoff factor 
"growth_interval" - a Python int containing the current growth interval 
"_growth_tracker" - a Python int containing the number of recent consecutive unskipped steps.  If this instance is not enabled, returns an empty dict.  Note If you wish to checkpoint the scaler’s state after a particular iteration, state_dict() should be called after update().
step(optimizer, *args, **kwargs) [source]
 
step() carries out the following two operations:  Internally invokes unscale_(optimizer) (unless unscale_() was explicitly called for optimizer earlier in the iteration). As part of the unscale_(), gradients are checked for infs/NaNs. If no inf/NaN gradients are found, invokes optimizer.step() using the unscaled gradients. Otherwise, optimizer.step() is skipped to avoid corrupting the params.  *args and **kwargs are forwarded to optimizer.step(). Returns the return value of optimizer.step(*args, **kwargs).  Parameters 
 
optimizer (torch.optim.Optimizer) – Optimizer that applies the gradients. 
args – Any arguments. 
kwargs – Any keyword arguments.     Warning Closure use is not currently supported.
unscale_(optimizer) [source]
 
Divides (“unscales”) the optimizer’s gradient tensors by the scale factor. unscale_() is optional, serving cases where you need to modify or inspect gradients between the backward pass(es) and step(). If unscale_() is not called explicitly, gradients will be unscaled automatically during step(). Simple example, using unscale_() to enable clipping of unscaled gradients: ...
scaler.scale(loss).backward()
scaler.unscale_(optimizer)
torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm)
scaler.step(optimizer)
scaler.update()
  Parameters 
optimizer (torch.optim.Optimizer) – Optimizer that owns the gradients to be unscaled.    Note unscale_() does not incur a CPU-GPU sync.   Warning unscale_() should only be called once per optimizer per step() call, and only after all gradients for that optimizer’s assigned parameters have been accumulated. Calling unscale_() twice for a given optimizer between each step() triggers a RuntimeError.   Warning unscale_() may unscale sparse gradients out of place, replacing the .grad attribute.
update(new_scale=None) [source]
 
Updates the scale factor. If any optimizer steps were skipped the scale is multiplied by backoff_factor to reduce it. If growth_interval unskipped iterations occurred consecutively, the scale is multiplied by growth_factor to increase it. Passing new_scale sets the scale directly.  Parameters 
new_scale (float or torch.cuda.FloatTensor, optional, default=None) – New scale factor.    Warning update() should only be called at the end of the iteration, after scaler.step(optimizer) has been invoked for all optimizers used this iteration.
torch.cuda.can_device_access_peer(device, peer_device) [source]
 
Checks if peer access between two devices is possible.
torch.cuda.comm.broadcast(tensor, devices=None, *, out=None) [source]
 
Broadcasts a tensor to specified GPU devices.  Parameters 
 
tensor (Tensor) – tensor to broadcast. Can be on CPU or GPU. 
devices (Iterable[torch.device, str or int], optional) – an iterable of GPU devices, among which to broadcast. 
out (Sequence[Tensor], optional, keyword-only) – the GPU tensors to store output results.     Note Exactly one of devices and out must be specified.   Returns 
 
 
If devices is specified, 

a tuple containing copies of tensor, placed on devices.    
 
If out is specified, 

a tuple containing out tensors, each containing a copy of tensor.
torch.cuda.comm.broadcast_coalesced(tensors, devices, buffer_size=10485760) [source]
 
Broadcasts a sequence tensors to the specified GPUs. Small tensors are first coalesced into a buffer to reduce the number of synchronizations.  Parameters 
 
tensors (sequence) – tensors to broadcast. Must be on the same device, either CPU or GPU. 
devices (Iterable[torch.device, str or int]) – an iterable of GPU devices, among which to broadcast. 
buffer_size (int) – maximum size of the buffer used for coalescing   Returns 
A tuple containing copies of tensor, placed on devices.
torch.cuda.comm.gather(tensors, dim=0, destination=None, *, out=None) [source]
 
Gathers tensors from multiple GPU devices.  Parameters 
 
tensors (Iterable[Tensor]) – an iterable of tensors to gather. Tensor sizes in all dimensions other than dim have to match. 
dim (int, optional) – a dimension along which the tensors will be concatenated. Default: 0. 
destination (torch.device, str, or int, optional) – the output device. Can be CPU or CUDA. Default: the current CUDA device. 
out (Tensor, optional, keyword-only) – the tensor to store gather result. Its sizes must match those of tensors, except for dim, where the size must equal sum(tensor.size(dim) for tensor in tensors). Can be on CPU or CUDA.     Note destination must not be specified when out is specified.   Returns 
 
 
If destination is specified, 

a tensor located on destination device, that is a result of concatenating tensors along dim.    
 
If out is specified, 

the out tensor, now containing results of concatenating tensors along dim.
torch.cuda.comm.reduce_add(inputs, destination=None) [source]
 
Sums tensors from multiple GPUs. All inputs should have matching shapes, dtype, and layout. The output tensor will be of the same shape, dtype, and layout.  Parameters 
 
inputs (Iterable[Tensor]) – an iterable of tensors to add. 
destination (int, optional) – a device on which the output will be placed (default: current device).   Returns 
A tensor containing an elementwise sum of all inputs, placed on the destination device.
torch.cuda.comm.scatter(tensor, devices=None, chunk_sizes=None, dim=0, streams=None, *, out=None) [source]
 
Scatters tensor across multiple GPUs.  Parameters 
 
tensor (Tensor) – tensor to scatter. Can be on CPU or GPU. 
devices (Iterable[torch.device, str or int], optional) – an iterable of GPU devices, among which to scatter. 
chunk_sizes (Iterable[int], optional) – sizes of chunks to be placed on each device. It should match devices in length and sums to tensor.size(dim). If not specified, tensor will be divided into equal chunks. 
dim (int, optional) – A dimension along which to chunk tensor. Default: 0. 
streams (Iterable[Stream], optional) – an iterable of Streams, among which to execute the scatter. If not specified, the default stream will be utilized. 
out (Sequence[Tensor], optional, keyword-only) – the GPU tensors to store output results. Sizes of these tensors must match that of tensor, except for dim, where the total size must sum to tensor.size(dim).     Note Exactly one of devices and out must be specified. When out is specified, chunk_sizes must not be specified and will be inferred from sizes of out.   Returns 
 
 
If devices is specified, 

a tuple containing chunks of tensor, placed on devices.    
 
If out is specified, 

a tuple containing out tensors, each containing a chunk of tensor.
torch.cuda.current_blas_handle() [source]
 
Returns cublasHandle_t pointer to current cuBLAS handle
torch.cuda.current_device() [source]
 
Returns the index of a currently selected device.
torch.cuda.current_stream(device=None) [source]
 
Returns the currently selected Stream for a given device.  Parameters 
device (torch.device or int, optional) – selected device. Returns the currently selected Stream for the current device, given by current_device(), if device is None (default).
torch.cuda.default_stream(device=None) [source]
 
Returns the default Stream for a given device.  Parameters 
device (torch.device or int, optional) – selected device. Returns the default Stream for the current device, given by current_device(), if device is None (default).
class torch.cuda.device(device) [source]
 
Context-manager that changes the selected device.  Parameters 
device (torch.device or int) – device index to select. It’s a no-op if this argument is a negative integer or None.
torch.cuda.device_count() [source]
 
Returns the number of GPUs available.
class torch.cuda.device_of(obj) [source]
 
Context-manager that changes the current device to that of given object. You can use both tensors and storages as arguments. If a given object is not allocated on a GPU, this is a no-op.  Parameters 
obj (Tensor or Storage) – object allocated on the selected device.
torch.cuda.empty_cache() [source]
 
Releases all unoccupied cached memory currently held by the caching allocator so that those can be used in other GPU application and visible in nvidia-smi.  Note empty_cache() doesn’t increase the amount of GPU memory available for PyTorch. However, it may help reduce fragmentation of GPU memory in certain cases. See Memory management for more details about GPU memory management.
class torch.cuda.Event [source]
 
Wrapper around a CUDA event. CUDA events are synchronization markers that can be used to monitor the device’s progress, to accurately measure timing, and to synchronize CUDA streams. The underlying CUDA events are lazily initialized when the event is first recorded or exported to another process. After creation, only streams on the same device may record the event. However, streams on any device can wait on the event.  Parameters 
 
enable_timing (bool, optional) – indicates if the event should measure time (default: False) 
blocking (bool, optional) – if True, wait() will be blocking (default: False) 
interprocess (bool) – if True, the event can be shared between processes (default: False)     
elapsed_time(end_event) [source]
 
Returns the time elapsed in milliseconds after the event was recorded and before the end_event was recorded. 
  
classmethod from_ipc_handle(device, handle) [source]
 
Reconstruct an event from an IPC handle on the given device. 
  
ipc_handle() [source]
 
Returns an IPC handle of this event. If not recorded yet, the event will use the current device. 
  
query() [source]
 
Checks if all work currently captured by event has completed.  Returns 
A boolean indicating if all work currently captured by event has completed.   
  
record(stream=None) [source]
 
Records the event in a given stream. Uses torch.cuda.current_stream() if no stream is specified. The stream’s device must match the event’s device. 
  
synchronize() [source]
 
Waits for the event to complete. Waits until the completion of all work currently captured in this event. This prevents the CPU thread from proceeding until the event completes.  Note This is a wrapper around cudaEventSynchronize(): see CUDA Event documentation for more info.  
  
wait(stream=None) [source]
 
Makes all future work submitted to the given stream wait for this event. Use torch.cuda.current_stream() if no stream is specified.
elapsed_time(end_event) [source]
 
Returns the time elapsed in milliseconds after the event was recorded and before the end_event was recorded.
classmethod from_ipc_handle(device, handle) [source]
 
Reconstruct an event from an IPC handle on the given device.
ipc_handle() [source]
 
Returns an IPC handle of this event. If not recorded yet, the event will use the current device.
query() [source]
 
Checks if all work currently captured by event has completed.  Returns 
A boolean indicating if all work currently captured by event has completed.
record(stream=None) [source]
 
Records the event in a given stream. Uses torch.cuda.current_stream() if no stream is specified. The stream’s device must match the event’s device.
synchronize() [source]
 
Waits for the event to complete. Waits until the completion of all work currently captured in this event. This prevents the CPU thread from proceeding until the event completes.  Note This is a wrapper around cudaEventSynchronize(): see CUDA Event documentation for more info.
wait(stream=None) [source]
 
Makes all future work submitted to the given stream wait for this event. Use torch.cuda.current_stream() if no stream is specified.
torch.cuda.get_arch_list() [source]
 
Returns list CUDA architectures this library was compiled for.
torch.cuda.get_device_capability(device=None) [source]
 
Gets the cuda capability of a device.  Parameters 
device (torch.device or int, optional) – device for which to return the device capability. This function is a no-op if this argument is a negative integer. It uses the current device, given by current_device(), if device is None (default).  Returns 
the major and minor cuda capability of the device  Return type 
tuple(int, int)
torch.cuda.get_device_name(device=None) [source]
 
Gets the name of a device.  Parameters 
device (torch.device or int, optional) – device for which to return the name. This function is a no-op if this argument is a negative integer. It uses the current device, given by current_device(), if device is None (default).  Returns 
the name of the device  Return type 
str
torch.cuda.get_device_properties(device) [source]
 
Gets the properties of a device.  Parameters 
device (torch.device or int or str) – device for which to return the properties of the device.  Returns 
the properties of the device  Return type 
_CudaDeviceProperties
torch.cuda.get_gencode_flags() [source]
 
Returns NVCC gencode flags this library were compiled with.
torch.cuda.get_rng_state(device='cuda') [source]
 
Returns the random number generator state of the specified GPU as a ByteTensor.  Parameters 
device (torch.device or int, optional) – The device to return the RNG state of. Default: 'cuda' (i.e., torch.device('cuda'), the current CUDA device).    Warning This function eagerly initializes CUDA.
torch.cuda.get_rng_state_all() [source]
 
Returns a list of ByteTensor representing the random number states of all devices.
torch.cuda.init() [source]
 
Initialize PyTorch’s CUDA state. You may need to call this explicitly if you are interacting with PyTorch via its C API, as Python bindings for CUDA functionality will not be available until this initialization takes place. Ordinary users should not need this, as all of PyTorch’s CUDA methods automatically initialize CUDA state on-demand. Does nothing if the CUDA state is already initialized.
torch.cuda.initial_seed() [source]
 
Returns the current random seed of the current GPU.  Warning This function eagerly initializes CUDA.
torch.cuda.ipc_collect() [source]
 
Force collects GPU memory after it has been released by CUDA IPC.  Note Checks if any sent CUDA tensors could be cleaned from the memory. Force closes shared memory file used for reference counting if there is no active counters. Useful when the producer process stopped actively sending tensors and want to release unused memory.
torch.cuda.is_available() [source]
 
Returns a bool indicating if CUDA is currently available.
torch.cuda.is_initialized() [source]
 
Returns whether PyTorch’s CUDA state has been initialized.
torch.cuda.list_gpu_processes(device=None) [source]
 
Returns a human-readable printout of the running processes and their GPU memory use for a given device. This can be useful to display periodically during training, or when handling out-of-memory exceptions.  Parameters 
device (torch.device or int, optional) – selected device. Returns printout for the current device, given by current_device(), if device is None (default).
torch.cuda.manual_seed(seed) [source]
 
Sets the seed for generating random numbers for the current GPU. It’s safe to call this function if CUDA is not available; in that case, it is silently ignored.  Parameters 
seed (int) – The desired seed.    Warning If you are working with a multi-GPU model, this function is insufficient to get determinism. To seed all GPUs, use manual_seed_all().
torch.cuda.manual_seed_all(seed) [source]
 
Sets the seed for generating random numbers on all GPUs. It’s safe to call this function if CUDA is not available; in that case, it is silently ignored.  Parameters 
seed (int) – The desired seed.
torch.cuda.max_memory_allocated(device=None) [source]
 
Returns the maximum GPU memory occupied by tensors in bytes for a given device. By default, this returns the peak allocated memory since the beginning of this program. reset_peak_stats() can be used to reset the starting point in tracking this metric. For example, these two functions can measure the peak allocated memory usage of each iteration in a training loop.  Parameters 
device (torch.device or int, optional) – selected device. Returns statistic for the current device, given by current_device(), if device is None (default).    Note See Memory management for more details about GPU memory management.
torch.cuda.max_memory_cached(device=None) [source]
 
Deprecated; see max_memory_reserved().
torch.cuda.max_memory_reserved(device=None) [source]
 
Returns the maximum GPU memory managed by the caching allocator in bytes for a given device. By default, this returns the peak cached memory since the beginning of this program. reset_peak_stats() can be used to reset the starting point in tracking this metric. For example, these two functions can measure the peak cached memory amount of each iteration in a training loop.  Parameters 
device (torch.device or int, optional) – selected device. Returns statistic for the current device, given by current_device(), if device is None (default).    Note See Memory management for more details about GPU memory management.
torch.cuda.memory_allocated(device=None) [source]
 
Returns the current GPU memory occupied by tensors in bytes for a given device.  Parameters 
device (torch.device or int, optional) – selected device. Returns statistic for the current device, given by current_device(), if device is None (default).    Note This is likely less than the amount shown in nvidia-smi since some unused memory can be held by the caching allocator and some context needs to be created on GPU. See Memory management for more details about GPU memory management.
torch.cuda.memory_cached(device=None) [source]
 
Deprecated; see memory_reserved().
torch.cuda.memory_reserved(device=None) [source]
 
Returns the current GPU memory managed by the caching allocator in bytes for a given device.  Parameters 
device (torch.device or int, optional) – selected device. Returns statistic for the current device, given by current_device(), if device is None (default).    Note See Memory management for more details about GPU memory management.
torch.cuda.memory_snapshot() [source]
 
Returns a snapshot of the CUDA memory allocator state across all devices. Interpreting the output of this function requires familiarity with the memory allocator internals.  Note See Memory management for more details about GPU memory management.