neulab/docprompting-conala huggingface dataset:
tf.nn.safe_embedding_lookup_sparse     View source on GitHub    Lookup embedding results, accounting for invalid IDs and empty features. 
tf.nn.safe_embedding_lookup_sparse(
    embedding_weights, sparse_ids, sparse_weights=None, combiner='mean',
    default_id=None, max_norm=None, name=None
)
 The partitioned embedding in embedding_weights must all be the same shape except for the first dimension. The first dimension is allowed to vary as the vocabulary size is not necessarily a multiple of num of shards. Invalid IDs (< 0) are pruned from input IDs and weights, as well as any IDs with non-positive weight. For an entry with no features, the embedding vector for default_id is returned, or the 0-vector if default_id is not supplied. The ids and weights may be multi-dimensional. Embeddings are always aggregated along the last dimension. If len(embedding_weights) > 1, each element id of ids is partitioned between the elements of embedding_weights according to the "div" partition strategy, which means we assign ids to partitions in a contiguous manner. For instance, 13 ids are split across 5 partitions as: [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9, 10], [11, 12]]. If the id space does not evenly divide the number of partitions, each of the first (max_id + 1) % len(embedding_weights) partitions will be assigned one more id.
 


 Args
  embedding_weights   A single tensor representing the complete embedding tensor, or a list of tensors all of same shape except for the first dimension, representing sharded embedding tensors following "div" partition strategy.  
  sparse_ids   SparseTensor of shape [d_0, d_1, ..., d_n] containing the ids. d_0 is typically batch size.  
  sparse_weights   SparseTensor of same shape as sparse_ids, containing float weights corresponding to sparse_ids, or None if all weights are be assumed to be 1.0.  
  combiner   A string specifying how to combine embedding results for each entry. Currently "mean", "sqrtn" and "sum" are supported, with "mean" the default.  
  default_id   The id to use for an entry with no features. Defaults to 0-vector.  
  max_norm   If not None, all embeddings are l2-normalized to max_norm before combining.  
  name   A name for this operation (optional).   
 


 Returns   A dense tensor representing the combined embeddings for the sparse ids. For each row in the dense tensor represented by sparse_ids, the op looks up the embeddings for all ids in that row, multiplies them by the corresponding weight, and combines these embeddings as specified. In other words, if shape(combined embedding_weights) = [p0, p1, ..., pm] and shape(sparse_ids) = shape(sparse_weights) = [d0, d1, ..., dn] then shape(output) = [d0, d1, ... dn-1, p1, ..., pm]. For instance, if params is a 10x20 matrix, and sp_ids / sp_weights are [0, 0]: id 1, weight 2.0
[0, 1]: id 3, weight 0.5
[1, 0]: id -1, weight 1.0
[2, 3]: id 1, weight 3.0
 default_id is 0. with combiner="mean", then the output will be a 3x20 matrix where output[0, :] = (params[1, :] * 2.0 + params[3, :] * 0.5) / (2.0 + 0.5)
output[1, :] = (params[0, :] * 1.0) / 1.0
output[2, :] = (params[1, :] * 3.0) / 3.0

 

 


 Raises
  ValueError   if embedding_weights is empty.
tf.nn.sampled_softmax_loss     View source on GitHub    Computes and returns the sampled softmax training loss. 
tf.nn.sampled_softmax_loss(
    weights, biases, labels, inputs, num_sampled, num_classes, num_true=1,
    sampled_values=None, remove_accidental_hits=True, seed=None,
    name='sampled_softmax_loss'
)
 This is a faster way to train a softmax classifier over a huge number of classes. This operation is for training only. It is generally an underestimate of the full softmax loss. A common use case is to use this method for training, and calculate the full sigmoid loss for evaluation or inference as in the following example: if mode == "train":
  loss = tf.nn.sampled_softmax_loss(
      weights=weights,
      biases=biases,
      labels=labels,
      inputs=inputs,
      ...)
elif mode == "eval":
  logits = tf.matmul(inputs, tf.transpose(weights))
  logits = tf.nn.bias_add(logits, biases)
  labels_one_hot = tf.one_hot(labels, n_classes)
  loss = tf.nn.softmax_cross_entropy_with_logits(
      labels=labels_one_hot,
      logits=logits)
 See our Candidate Sampling Algorithms Reference Also see Section 3 of Jean et al., 2014 (pdf) for the math. 
Note: when doing embedding lookup on weights and bias, "div" partition strategy will be used. Support for other partition strategy will be added later.

 


 Args
  weights   A Tensor of shape [num_classes, dim], or a list of Tensor objects whose concatenation along dimension 0 has shape [num_classes, dim]. The (possibly-sharded) class embeddings.  
  biases   A Tensor of shape [num_classes]. The class biases.  
  labels   A Tensor of type int64 and shape [batch_size, num_true]. The target classes. Note that this format differs from the labels argument of nn.softmax_cross_entropy_with_logits.  
  inputs   A Tensor of shape [batch_size, dim]. The forward activations of the input network.  
  num_sampled   An int. The number of classes to randomly sample per batch.  
  num_classes   An int. The number of possible classes.  
  num_true   An int. The number of target classes per training example.  
  sampled_values   a tuple of (sampled_candidates, true_expected_count, sampled_expected_count) returned by a *_candidate_sampler function. (if None, we default to log_uniform_candidate_sampler)  
  remove_accidental_hits   A bool. whether to remove "accidental hits" where a sampled class equals one of the target classes. Default is True.  
  seed   random seed for candidate sampling. Default to None, which doesn't set the op-level random seed for candidate sampling.  
  name   A name for the operation (optional).   
 


 Returns   A batch_size 1-D tensor of per-example sampled softmax losses.
tf.nn.scale_regularization_loss     View source on GitHub    Scales the sum of the given regularization losses by number of replicas.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.nn.scale_regularization_loss  
tf.nn.scale_regularization_loss(
    regularization_loss
)
 Usage with distribution strategy and custom training loop: with strategy.scope():
  def compute_loss(self, label, predictions):
    per_example_loss = tf.keras.losses.sparse_categorical_crossentropy(
        labels, predictions)

    # Compute loss that is scaled by sample_weight and by global batch size.
    loss = tf.nn.compute_average_loss(
        per_example_loss,
        sample_weight=sample_weight,
        global_batch_size=GLOBAL_BATCH_SIZE)

    # Add scaled regularization losses.
    loss += tf.nn.scale_regularization_loss(tf.nn.l2_loss(weights))
    return loss

 


 Args
  regularization_loss   Regularization loss.   
 


 Returns   Scalar loss value.
tf.nn.selu Computes scaled exponential linear: scale * alpha * (exp(features) - 1)  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.nn.selu  
tf.nn.selu(
    features, name=None
)
 if < 0, scale * features otherwise. To be used together with initializer = tf.variance_scaling_initializer(factor=1.0, mode='FAN_IN'). For correct dropout, use tf.contrib.nn.alpha_dropout. See Self-Normalizing Neural Networks
 


 Args
  features   A Tensor. Must be one of the following types: half, bfloat16, float32, float64.  
  name   A name for the operation (optional).   
 


 Returns   A Tensor. Has the same type as features.
tf.nn.separable_conv2d     View source on GitHub    2-D convolution with separable filters. 
tf.nn.separable_conv2d(
    input, depthwise_filter, pointwise_filter, strides, padding, data_format=None,
    dilations=None, name=None
)
 Performs a depthwise convolution that acts separately on channels followed by a pointwise convolution that mixes channels. Note that this is separability between dimensions [1, 2] and 3, not spatial separability between dimensions 1 and 2. In detail, with the default NHWC format, output[b, i, j, k] = sum_{di, dj, q, r}
    input[b, strides[1] * i + di, strides[2] * j + dj, q] *
    depthwise_filter[di, dj, q, r] *
    pointwise_filter[0, 0, q * channel_multiplier + r, k]
 strides controls the strides for the depthwise convolution only, since the pointwise convolution has implicit strides of [1, 1, 1, 1]. Must have strides[0] = strides[3] = 1. For the most common case of the same horizontal and vertical strides, strides = [1, stride, stride, 1]. If any value in rate is greater than 1, we perform atrous depthwise convolution, in which case all values in the strides tensor must be equal to 1.
 


 Args
  input   4-D Tensor with shape according to data_format.  
  depthwise_filter   4-D Tensor with shape [filter_height, filter_width, in_channels, channel_multiplier]. Contains in_channels convolutional filters of depth 1.  
  pointwise_filter   4-D Tensor with shape [1, 1, channel_multiplier * in_channels, out_channels]. Pointwise filter to mix channels after depthwise_filter has convolved spatially.  
  strides   1-D of size 4. The strides for the depthwise convolution for each dimension of input.  
  padding   Controls how to pad the image before applying the depthwise convolution. Can be the string "SAME" or "VALID" indicating the type of padding algorithm to use, or a Python list indicating the explicit paddings at the start and end of each dimension. When explicit padding is used and data_format is "NHWC", this should be in the form [[0, 0], [pad_top, pad_bottom], [pad_left, pad_right], [0, 0]]. When explicit padding used and data_format is "NCHW", this should be in the form [[0, 0], [0, 0], [pad_top, pad_bottom], [pad_left, pad_right]].  
  data_format   The data format for input. Either "NHWC" (default) or "NCHW".  
  dilations   1-D of size 2. The dilation rate in which we sample input values across the height and width dimensions in atrous convolution. If it is greater than 1, then all values of strides must be 1.  
  name   A name for this operation (optional).   
 


 Returns   A 4-D Tensor with shape according to 'data_format'. For example, with data_format="NHWC", shape is [batch, out_height, out_width, out_channels].
tf.nn.sigmoid_cross_entropy_with_logits     View source on GitHub    Computes sigmoid cross entropy given logits. 
tf.nn.sigmoid_cross_entropy_with_logits(
    labels=None, logits=None, name=None
)
 Measures the probability error in discrete classification tasks in which each class is independent and not mutually exclusive. For instance, one could perform multilabel classification where a picture can contain both an elephant and a dog at the same time. For brevity, let x = logits, z = labels. The logistic loss is   z * -log(sigmoid(x)) + (1 - z) * -log(1 - sigmoid(x))
= z * -log(1 / (1 + exp(-x))) + (1 - z) * -log(exp(-x) / (1 + exp(-x)))
= z * log(1 + exp(-x)) + (1 - z) * (-log(exp(-x)) + log(1 + exp(-x)))
= z * log(1 + exp(-x)) + (1 - z) * (x + log(1 + exp(-x))
= (1 - z) * x + log(1 + exp(-x))
= x - x * z + log(1 + exp(-x))
 For x < 0, to avoid overflow in exp(-x), we reformulate the above   x - x * z + log(1 + exp(-x))
= log(exp(x)) - x * z + log(1 + exp(-x))
= - x * z + log(1 + exp(x))
 Hence, to ensure stability and avoid overflow, the implementation uses this equivalent formulation max(x, 0) - x * z + log(1 + exp(-abs(x)))
 logits and labels must have the same type and shape.
 


 Args
  labels   A Tensor of the same type and shape as logits.  
  logits   A Tensor of type float32 or float64.  
  name   A name for the operation (optional).   
 


 Returns   A Tensor of the same shape as logits with the componentwise logistic losses.  

 


 Raises
  ValueError   If logits and labels do not have the same shape.
tf.nn.silu Computes the SiLU or Swish activation function: x * sigmoid(x).  View aliases  Main aliases 
tf.nn.swish Compat aliases for migration See Migration guide for more details. tf.compat.v1.nn.silu, tf.compat.v1.nn.swish  
tf.nn.silu(
    features
)
 The SiLU activation function was introduced in "Gaussian Error Linear Units (GELUs)" Hendrycks et al. 2016 and "Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning" Elfwing et al. 2017 and was independently discovered (and called swish) in "Searching for Activation Functions" Ramachandran et al. 2017
 


 Args
  features   A Tensor representing preactivation values.  
  name   A name for the operation (optional).   
 


 Returns   The activation value.
tf.nn.softmax     View source on GitHub    Computes softmax activations.  View aliases  Main aliases 
tf.math.softmax  
tf.nn.softmax(
    logits, axis=None, name=None
)
 This function performs the equivalent of softmax = tf.exp(logits) / tf.reduce_sum(tf.exp(logits), axis)

 


 Args
  logits   A non-empty Tensor. Must be one of the following types: half, float32, float64.  
  axis   The dimension softmax would be performed on. The default is -1 which indicates the last dimension.  
  name   A name for the operation (optional).   
 


 Returns   A Tensor. Has the same type and shape as logits.  

 


 Raises
  InvalidArgumentError   if logits is empty or axis is beyond the last dimension of logits.
tf.nn.softmax_cross_entropy_with_logits     View source on GitHub    Computes softmax cross entropy between logits and labels. 
tf.nn.softmax_cross_entropy_with_logits(
    labels, logits, axis=-1, name=None
)
 Measures the probability error in discrete classification tasks in which the classes are mutually exclusive (each entry is in exactly one class). For example, each CIFAR-10 image is labeled with one and only one label: an image can be a dog or a truck, but not both. 
Note: While the classes are mutually exclusive, their probabilities need not be. All that is required is that each row of labels is a valid probability distribution. If they are not, the computation of the gradient will be incorrect.
 If using exclusive labels (wherein one and only one class is true at a time), see sparse_softmax_cross_entropy_with_logits. Usage: 
logits = [[4.0, 2.0, 1.0], [0.0, 5.0, 1.0]]
labels = [[1.0, 0.0, 0.0], [0.0, 0.8, 0.2]]
tf.nn.softmax_cross_entropy_with_logits(labels=labels, logits=logits)
<tf.Tensor: shape=(2,), dtype=float32,
numpy=array([0.16984604, 0.82474494], dtype=float32)>
 Warning: This op expects unscaled logits, since it performs a softmax on logits internally for efficiency. Do not call this op with the output of softmax, as it will produce incorrect results. A common use case is to have logits and labels of shape [batch_size, num_classes], but higher dimensions are supported, with the axis argument specifying the class dimension. logits and labels must have the same dtype (either float16, float32, or float64). Backpropagation will happen into both logits and labels. To disallow backpropagation into labels, pass label tensors through tf.stop_gradient before feeding it to this function. Note that to avoid confusion, it is required to pass only named arguments to this function.
 


 Args
  labels   Each vector along the class dimension should hold a valid probability distribution e.g. for the case in which labels are of shape [batch_size, num_classes], each row of labels[i] must be a valid probability distribution.  
  logits   Per-label activations, typically a linear output. These activation energies are interpreted as unnormalized log probabilities.  
  axis   The class dimension. Defaulted to -1 which is the last dimension.  
  name   A name for the operation (optional).   
 


 Returns   A Tensor that contains the softmax cross entropy loss. Its type is the same as logits and its shape is the same as labels except that it does not have the last dimension of labels.
tf.nn.softsign Computes softsign: features / (abs(features) + 1).  View aliases  Main aliases 
tf.math.softsign Compat aliases for migration See Migration guide for more details. tf.compat.v1.math.softsign, tf.compat.v1.nn.softsign  
tf.nn.softsign(
    features, name=None
)

 


 Args
  features   A Tensor. Must be one of the following types: half, bfloat16, float32, float64.  
  name   A name for the operation (optional).   
 


 Returns   A Tensor. Has the same type as features.
tf.nn.space_to_depth     View source on GitHub    SpaceToDepth for tensors of type T. 
tf.nn.space_to_depth(
    input, block_size, data_format='NHWC', name=None
)
 Rearranges blocks of spatial data, into depth. More specifically, this op outputs a copy of the input tensor where values from the height and width dimensions are moved to the depth dimension. The attr block_size indicates the input block size.  Non-overlapping blocks of size block_size x block size are rearranged into depth at each location. The depth of the output tensor is block_size * block_size * input_depth. The Y, X coordinates within each block of the input become the high order component of the output channel index. The input tensor's height and width must be divisible by block_size.  The data_format attr specifies the layout of the input and output tensors with the following options: "NHWC": [ batch, height, width, channels ] "NCHW": [ batch, channels, height, width ] "NCHW_VECT_C": qint8 [ batch, channels / 4, height, width, 4 ] It is useful to consider the operation as transforming a 6-D Tensor. e.g. for data_format = NHWC, Each element in the input tensor can be specified via 6 coordinates, ordered by decreasing memory layout significance as: n,oY,bY,oX,bX,iC (where n=batch index, oX, oY means X or Y coordinates within the output image, bX, bY means coordinates within the input block, iC means input channels). The output would be a transpose to the following layout: n,oY,oX,bY,bX,iC This operation is useful for resizing the activations between convolutions (but keeping all data), e.g. instead of pooling. It is also useful for training purely convolutional models. For example, given an input of shape [1, 2, 2, 1], data_format = "NHWC" and block_size = 2: x = [[[[1], [2]],
      [[3], [4]]]]
 This operation will output a tensor of shape [1, 1, 1, 4]: [[[[1, 2, 3, 4]]]]
 Here, the input has a batch of 1 and each batch element has shape [2, 2, 1], the corresponding output will have a single element (i.e. width and height are both 1) and will have a depth of 4 channels (1 * block_size * block_size). The output element shape is [1, 1, 4]. For an input tensor with larger depth, here of shape [1, 2, 2, 3], e.g. x = [[[[1, 2, 3], [4, 5, 6]],
      [[7, 8, 9], [10, 11, 12]]]]
 This operation, for block_size of 2, will return the following tensor of shape [1, 1, 1, 12] [[[[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]]]]
 Similarly, for the following input of shape [1 4 4 1], and a block size of 2: x = [[[[1],   [2],  [5],  [6]],
      [[3],   [4],  [7],  [8]],
      [[9],  [10], [13],  [14]],
      [[11], [12], [15],  [16]]]]
 the operator will return the following tensor of shape [1 2 2 4]: x = [[[[1, 2, 3, 4],
       [5, 6, 7, 8]],
      [[9, 10, 11, 12],
       [13, 14, 15, 16]]]]

 


 Args
  input   A Tensor.  
  block_size   An int that is >= 2. The size of the spatial block.  
  data_format   An optional string from: "NHWC", "NCHW", "NCHW_VECT_C". Defaults to "NHWC".  
  name   A name for the operation (optional).   
 


 Returns   A Tensor. Has the same type as input.
tf.nn.sparse_softmax_cross_entropy_with_logits     View source on GitHub    Computes sparse softmax cross entropy between logits and labels. 
tf.nn.sparse_softmax_cross_entropy_with_logits(
    labels, logits, name=None
)
 Measures the probability error in discrete classification tasks in which the classes are mutually exclusive (each entry is in exactly one class). For example, each CIFAR-10 image is labeled with one and only one label: an image can be a dog or a truck, but not both. 
Note: For this operation, the probability of a given label is considered exclusive. That is, soft classes are not allowed, and the labels vector must provide a single specific index for the true class for each row of logits (each minibatch entry). For soft softmax classification with a probability distribution for each entry, see softmax_cross_entropy_with_logits_v2.

Warning: This op expects unscaled logits, since it performs a softmax on logits internally for efficiency. Do not call this op with the output of softmax, as it will produce incorrect results. A common use case is to have logits of shape [batch_size, num_classes] and have labels of shape [batch_size], but higher dimensions are supported, in which case the dim-th dimension is assumed to be of size num_classes. logits must have the dtype of float16, float32, or float64, and labels must have the dtype of int32 or int64. Note that to avoid confusion, it is required to pass only named arguments to this function.
 


 Args
  labels   Tensor of shape [d_0, d_1, ..., d_{r-1}] (where r is rank of labels and result) and dtype int32 or int64. Each entry in labels must be an index in [0, num_classes). Other values will raise an exception when this op is run on CPU, and return NaN for corresponding loss and gradient rows on GPU.  
  logits   Unscaled log probabilities of shape [d_0, d_1, ..., d_{r-1}, num_classes] and dtype float16, float32, or float64.  
  name   A name for the operation (optional).   
 


 Returns   A Tensor of the same shape as labels and of the same type as logits with the softmax cross entropy loss.  

 


 Raises
  ValueError   If logits are scalars (need to have rank >= 1) or if the rank of the labels is not equal to the rank of the logits minus one.
tf.nn.sufficient_statistics     View source on GitHub    Calculate the sufficient statistics for the mean and variance of x. 
tf.nn.sufficient_statistics(
    x, axes, shift=None, keepdims=False, name=None
)
 These sufficient statistics are computed using the one pass algorithm on an input that's optionally shifted. See: https://en.wikipedia.org/wiki/Algorithms_for_calculating_variance#Computing_shifted_data
 


 Args
  x   A Tensor.  
  axes   Array of ints. Axes along which to compute mean and variance.  
  shift   A Tensor containing the value by which to shift the data for numerical stability, or None if no shift is to be performed. A shift close to the true mean provides the most numerically stable results.  
  keepdims   produce statistics with the same dimensionality as the input.  
  name   Name used to scope the operations that compute the sufficient stats.   
 


 Returns   Four Tensor objects of the same type as x:  the count (number of elements to average over). the (possibly shifted) sum of the elements in the array. the (possibly shifted) sum of squares of the elements in the array. the shift by which the mean must be corrected or None if shift is None.