neulab/docprompting-conala huggingface dataset:
tf.compat.v1.sparse_segment_sqrt_n Computes the sum along sparse segments of a tensor divided by the sqrt(N).  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.sparse.segment_sqrt_n  
tf.compat.v1.sparse_segment_sqrt_n(
    data, indices, segment_ids, name=None, num_segments=None
)
 N is the size of the segment being reduced.
 


 Args
  data   A Tensor with data that will be assembled in the output.  
  indices   A 1-D Tensor with indices into data. Has same rank as segment_ids.  
  segment_ids   A 1-D Tensor with indices into the output Tensor. Values should be sorted and can be repeated.  
  name   A name for the operation (optional).  
  num_segments   An optional int32 scalar. Indicates the size of the output Tensor.   
 


 Returns   A tensor of the shape as data, except for dimension 0 which has size k, the number of segments specified via num_segments or inferred for the last element in segments_ids.
tf.compat.v1.sparse_segment_sum Computes the sum along sparse segments of a tensor.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.sparse.segment_sum  
tf.compat.v1.sparse_segment_sum(
    data, indices, segment_ids, name=None, num_segments=None
)
 Read the section on segmentation for an explanation of segments. Like tf.math.segment_sum, but segment_ids can have rank less than data's first dimension, selecting a subset of dimension 0, specified by indices. segment_ids is allowed to have missing ids, in which case the output will be zeros at those indices. In those cases num_segments is used to determine the size of the output. For example: c = tf.constant([[1,2,3,4], [-1,-2,-3,-4], [5,6,7,8]])

# Select two rows, one segment.
tf.sparse.segment_sum(c, tf.constant([0, 1]), tf.constant([0, 0]))
# => [[0 0 0 0]]

# Select two rows, two segment.
tf.sparse.segment_sum(c, tf.constant([0, 1]), tf.constant([0, 1]))
# => [[ 1  2  3  4]
#     [-1 -2 -3 -4]]

# With missing segment ids.
tf.sparse.segment_sum(c, tf.constant([0, 1]), tf.constant([0, 2]),
                      num_segments=4)
# => [[ 1  2  3  4]
#     [ 0  0  0  0]
#     [-1 -2 -3 -4]
#     [ 0  0  0  0]]

# Select all rows, two segments.
tf.sparse.segment_sum(c, tf.constant([0, 1, 2]), tf.constant([0, 0, 1]))
# => [[0 0 0 0]
#     [5 6 7 8]]

# Which is equivalent to:
tf.math.segment_sum(c, tf.constant([0, 0, 1]))

 


 Args
  data   A Tensor with data that will be assembled in the output.  
  indices   A 1-D Tensor with indices into data. Has same rank as segment_ids.  
  segment_ids   A 1-D Tensor with indices into the output Tensor. Values should be sorted and can be repeated.  
  name   A name for the operation (optional).  
  num_segments   An optional int32 scalar. Indicates the size of the output Tensor.   
 


 Returns   A tensor of the shape as data, except for dimension 0 which has size k, the number of segments specified via num_segments or inferred for the last element in segments_ids.
tf.compat.v1.sparse_split Split a SparseTensor into num_split tensors along axis. (deprecated arguments)  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.sparse.split  
tf.compat.v1.sparse_split(
    keyword_required=KeywordRequired(), sp_input=None, num_split=None, axis=None,
    name=None, split_dim=None
)
 Warning: SOME ARGUMENTS ARE DEPRECATED: (split_dim). They will be removed in a future version. Instructions for updating: split_dim is deprecated, use axis instead If the sp_input.dense_shape[axis] is not an integer multiple of num_split each slice starting from 0:shape[axis] % num_split gets extra one dimension. For example, if axis = 1 and num_split = 2 and the input is: input_tensor = shape = [2, 7]
[    a   d e  ]
[b c          ]
 Graphically the output tensors are: output_tensor[0] =
[    a   ]
[b c     ]

output_tensor[1] =
[ d e  ]
[      ]

 


 Args
  keyword_required   Python 2 standin for * (temporary for argument reorder)  
  sp_input   The SparseTensor to split.  
  num_split   A Python integer. The number of ways to split.  
  axis   A 0-D int32 Tensor. The dimension along which to split. Must be in range [-rank, rank), where rank is the number of dimensions in the input SparseTensor.  
  name   A name for the operation (optional).  
  split_dim   Deprecated old name for axis.   
 


 Returns   num_split SparseTensor objects resulting from splitting value.  

 


 Raises
  TypeError   If sp_input is not a SparseTensor.  
  ValueError   If the deprecated split_dim and axis are both non None.
tf.compat.v1.sparse_to_dense Converts a sparse representation into a dense tensor. (deprecated) 
tf.compat.v1.sparse_to_dense(
    sparse_indices, output_shape, sparse_values, default_value=0,
    validate_indices=True, name=None
)
 Warning: THIS FUNCTION IS DEPRECATED. It will be removed in a future version. Instructions for updating: Create a tf.sparse.SparseTensor and use tf.sparse.to_dense instead. Builds an array dense with shape output_shape such that # If sparse_indices is scalar
dense[i] = (i == sparse_indices ? sparse_values : default_value)

# If sparse_indices is a vector, then for each i
dense[sparse_indices[i]] = sparse_values[i]

# If sparse_indices is an n by d matrix, then for each i in [0, n)
dense[sparse_indices[i][0], ..., sparse_indices[i][d-1]] = sparse_values[i]
 All other values in dense are set to default_value. If sparse_values is a scalar, all sparse indices are set to this single value. Indices should be sorted in lexicographic order, and indices must not contain any repeats. If validate_indices is True, these properties are checked during execution.
 


 Args
  sparse_indices   A 0-D, 1-D, or 2-D Tensor of type int32 or int64. sparse_indices[i] contains the complete index where sparse_values[i] will be placed.  
  output_shape   A 1-D Tensor of the same type as sparse_indices. Shape of the dense output tensor.  
  sparse_values   A 0-D or 1-D Tensor. Values corresponding to each row of sparse_indices, or a scalar value to be used for all sparse indices.  
  default_value   A 0-D Tensor of the same type as sparse_values. Value to set for indices not specified in sparse_indices. Defaults to zero.  
  validate_indices   A boolean value. If True, indices are checked to make sure they are sorted in lexicographic order and that there are no repeats.  
  name   A name for the operation (optional).   
 


 Returns   Dense Tensor of shape output_shape. Has the same type as sparse_values.
Module: tf.compat.v1.spectral Public API for tf.spectral namespace. Functions dct(...): Computes the 1D [Discrete Cosine Transform (DCT)][dct] of input. fft(...): Fast Fourier transform. fft2d(...): 2D fast Fourier transform. fft3d(...): 3D fast Fourier transform. idct(...): Computes the 1D [Inverse Discrete Cosine Transform (DCT)][idct] of input. ifft(...): Inverse fast Fourier transform. ifft2d(...): Inverse 2D fast Fourier transform. ifft3d(...): Inverse 3D fast Fourier transform. irfft(...): Inverse real-valued fast Fourier transform. irfft2d(...): Inverse 2D real-valued fast Fourier transform. irfft3d(...): Inverse 3D real-valued fast Fourier transform. rfft(...): Real-valued fast Fourier transform. rfft2d(...): 2D real-valued fast Fourier transform. rfft3d(...): 3D real-valued fast Fourier transform.
tf.compat.v1.squeeze Removes dimensions of size 1 from the shape of a tensor. (deprecated arguments) 
tf.compat.v1.squeeze(
    input, axis=None, name=None, squeeze_dims=None
)
 Warning: SOME ARGUMENTS ARE DEPRECATED: (squeeze_dims). They will be removed in a future version. Instructions for updating: Use the axis argument instead Given a tensor input, this operation returns a tensor of the same type with all dimensions of size 1 removed. If you don't want to remove all size 1 dimensions, you can remove specific size 1 dimensions by specifying axis. For example: 
# 't' is a tensor of shape [1, 2, 1, 3, 1, 1]
t = tf.ones([1, 2, 1, 3, 1, 1])
print(tf.shape(tf.squeeze(t)).numpy())
[2 3]
 Or, to remove specific size 1 dimensions: 
# 't' is a tensor of shape [1, 2, 1, 3, 1, 1]
t = tf.ones([1, 2, 1, 3, 1, 1])
print(tf.shape(tf.squeeze(t, [2, 4])).numpy())
[1 2 3 1]
 
Note: if input is a tf.RaggedTensor, then this operation takes O(N) time, where N is the number of elements in the squeezed dimensions.

 


 Args
  input   A Tensor. The input to squeeze.  
  axis   An optional list of ints. Defaults to []. If specified, only squeezes the dimensions listed. The dimension index starts at 0. It is an error to squeeze a dimension that is not 1. Must be in the range [-rank(input), rank(input)). Must be specified if input is a RaggedTensor.  
  name   A name for the operation (optional).  
  squeeze_dims   Deprecated keyword argument that is now axis.   
 


 Returns   A Tensor. Has the same type as input. Contains the same data as input, but has one or more dimensions of size 1 removed.  

 


 Raises
  ValueError   When both squeeze_dims and axis are specified.
Module: tf.compat.v1.strings Operations for working with string Tensors. Functions as_string(...): Converts each entry in the given tensor to strings. bytes_split(...): Split string elements of input into bytes. format(...): Formats a string template using a list of tensors. join(...): Perform element-wise concatenation of a list of string tensors. length(...): Computes the length of each string given in the input tensor. lower(...): Converts all uppercase characters into their respective lowercase replacements. ngrams(...): Create a tensor of n-grams based on data. reduce_join(...): Joins all strings into a single string, or joins along an axis. regex_full_match(...): Check if the input matches the regex pattern. regex_replace(...): Replace elements of input matching regex pattern with rewrite. split(...): Split elements of input based on sep. strip(...): Strip leading and trailing whitespaces from the Tensor. substr(...): Return substrings from Tensor of strings. to_hash_bucket(...): Converts each string in the input Tensor to its hash mod by a number of buckets. to_hash_bucket_fast(...): Converts each string in the input Tensor to its hash mod by a number of buckets. to_hash_bucket_strong(...): Converts each string in the input Tensor to its hash mod by a number of buckets. to_number(...): Converts each string in the input Tensor to the specified numeric type. unicode_decode(...): Decodes each string in input into a sequence of Unicode code points. unicode_decode_with_offsets(...): Decodes each string into a sequence of code points with start offsets. unicode_encode(...): Encodes each sequence of Unicode code points in input into a string. unicode_script(...): Determine the script codes of a given tensor of Unicode integer code points. unicode_split(...): Splits each string in input into a sequence of Unicode code points. unicode_split_with_offsets(...): Splits each string into a sequence of code points with start offsets. unicode_transcode(...): Transcode the input text from a source encoding to a destination encoding. unsorted_segment_join(...): Joins the elements of inputs based on segment_ids. upper(...): Converts all lowercase characters into their respective uppercase replacements.
tf.compat.v1.strings.length Computes the length of each string given in the input tensor. 
tf.compat.v1.strings.length(
    input, name=None, unit='BYTE'
)
 
strings = tf.constant(['Hello','TensorFlow', '🙂'])
tf.strings.length(strings).numpy() # default counts bytes
array([ 5, 10, 4], dtype=int32)
tf.strings.length(strings, unit="UTF8_CHAR").numpy()
array([ 5, 10, 1], dtype=int32)

 


 Args
  input   A Tensor of type string. The strings for which to compute the length for each element.  
  name   A name for the operation (optional).  
  unit   An optional string from: "BYTE", "UTF8_CHAR". Defaults to "BYTE". The unit that is counted to compute string length. One of: "BYTE" (for the number of bytes in each string) or "UTF8_CHAR" (for the number of UTF-8 encoded Unicode code points in each string). Results are undefined if unit=UTF8_CHAR and the input strings do not contain structurally valid UTF-8.   
 


 Returns   A Tensor of type int32, containing the length of the input string in the same element of the input tensor.
tf.compat.v1.strings.split Split elements of input based on sep. 
tf.compat.v1.strings.split(
    input=None, sep=None, maxsplit=-1, result_type='SparseTensor',
    source=None, name=None
)
 Let N be the size of input (typically N will be the batch size). Split each element of input based on sep and return a SparseTensor or RaggedTensor containing the split tokens. Empty tokens are ignored. Examples: 
print(tf.compat.v1.strings.split(['hello world', 'a b c']))
SparseTensor(indices=tf.Tensor( [[0 0] [0 1] [1 0] [1 1] [1 2]], ...),
             values=tf.Tensor([b'hello' b'world' b'a' b'b' b'c'], ...),
             dense_shape=tf.Tensor([2 3], shape=(2,), dtype=int64))
 
print(tf.compat.v1.strings.split(['hello world', 'a b c'],
    result_type="RaggedTensor"))
<tf.RaggedTensor [[b'hello', b'world'], [b'a', b'b', b'c']]>
 If sep is given, consecutive delimiters are not grouped together and are deemed to delimit empty strings. For example, input of "1<>2<><>3" and sep of "<>" returns ["1", "2", "", "3"]. If sep is None or an empty string, consecutive whitespace are regarded as a single separator, and the result will contain no empty strings at the start or end if the string has leading or trailing whitespace. Note that the above mentioned behavior matches python's str.split.
 


 Args
  input   A string Tensor of rank N, the strings to split. If rank(input) is not known statically, then it is assumed to be 1.  
  sep   0-D string Tensor, the delimiter character.  
  maxsplit   An int. If maxsplit > 0, limit of the split of the result.  
  result_type   The tensor type for the result: one of "RaggedTensor" or "SparseTensor".  
  source   alias for "input" argument.  
  name   A name for the operation (optional).   
 


 Raises
  ValueError   If sep is not a string.   
 


 Returns   A SparseTensor or RaggedTensor of rank N+1, the strings split according to the delimiter.
tf.compat.v1.strings.substr Return substrings from Tensor of strings. 
tf.compat.v1.strings.substr(
    input, pos, len, name=None, unit='BYTE'
)
 For each string in the input Tensor, creates a substring starting at index pos with a total length of len. If len defines a substring that would extend beyond the length of the input string, or if len is negative, then as many characters as possible are used. A negative pos indicates distance within the string backwards from the end. If pos specifies an index which is out of range for any of the input strings, then an InvalidArgumentError is thrown. pos and len must have the same shape, otherwise a ValueError is thrown on Op creation. 
Note: Substr supports broadcasting up to two dimensions. More about broadcasting here
 Examples Using scalar pos and len: input = [b'Hello', b'World']
position = 1
length = 3

output = [b'ell', b'orl']
 Using pos and len with same shape as input: input = [[b'ten', b'eleven', b'twelve'],
         [b'thirteen', b'fourteen', b'fifteen'],
         [b'sixteen', b'seventeen', b'eighteen']]
position = [[1, 2, 3],
            [1, 2, 3],
            [1, 2, 3]]
length =   [[2, 3, 4],
            [4, 3, 2],
            [5, 5, 5]]

output = [[b'en', b'eve', b'lve'],
          [b'hirt', b'urt', b'te'],
          [b'ixtee', b'vente', b'hteen']]
 Broadcasting pos and len onto input: input = [[b'ten', b'eleven', b'twelve'],
         [b'thirteen', b'fourteen', b'fifteen'],
         [b'sixteen', b'seventeen', b'eighteen'],
         [b'nineteen', b'twenty', b'twentyone']]
position = [1, 2, 3]
length =   [1, 2, 3]

output = [[b'e', b'ev', b'lve'],
          [b'h', b'ur', b'tee'],
          [b'i', b've', b'hte'],
          [b'i', b'en', b'nty']]
 Broadcasting input onto pos and len: input = b'thirteen'
position = [1, 5, 7]
length =   [3, 2, 1]

output = [b'hir', b'ee', b'n']

 


 Raises    
ValueError: If the first argument cannot be converted to a Tensor of dtype string. 
InvalidArgumentError: If indices are out of range. 
ValueError: If pos and len are not the same shape. 

 

 


 Args
  input   A Tensor of type string. Tensor of strings  
  pos   A Tensor. Must be one of the following types: int32, int64. Scalar defining the position of first character in each substring  
  len   A Tensor. Must have the same type as pos. Scalar defining the number of characters to include in each substring  
  unit   An optional string from: "BYTE", "UTF8_CHAR". Defaults to "BYTE". The unit that is used to create the substring. One of: "BYTE" (for defining position and length by bytes) or "UTF8_CHAR" (for the UTF-8 encoded Unicode code points). The default is "BYTE". Results are undefined if unit=UTF8_CHAR and the input strings do not contain structurally valid UTF-8.  
  name   A name for the operation (optional).   
 


 Returns   A Tensor of type string.
tf.compat.v1.string_split Split elements of source based on delimiter. (deprecated arguments) 
tf.compat.v1.string_split(
    source, sep=None, skip_empty=True, delimiter=None,
    result_type='SparseTensor', name=None
)
 Warning: SOME ARGUMENTS ARE DEPRECATED: (delimiter). They will be removed in a future version. Instructions for updating: delimiter is deprecated, please use sep instead. Let N be the size of source (typically N will be the batch size). Split each element of source based on delimiter and return a SparseTensor or RaggedTensor containing the split tokens. Empty tokens are ignored. If sep is an empty string, each element of the source is split into individual strings, each containing one byte. (This includes splitting multibyte sequences of UTF-8.) If delimiter contains multiple bytes, it is treated as a set of delimiters with each considered a potential split point. Examples: 
print(tf.compat.v1.string_split(['hello world', 'a b c']))
SparseTensor(indices=tf.Tensor( [[0 0] [0 1] [1 0] [1 1] [1 2]], ...),
             values=tf.Tensor([b'hello' b'world' b'a' b'b' b'c'], ...),
             dense_shape=tf.Tensor([2 3], shape=(2,), dtype=int64))
 
print(tf.compat.v1.string_split(['hello world', 'a b c'],
    result_type="RaggedTensor"))
<tf.RaggedTensor [[b'hello', b'world'], [b'a', b'b', b'c']]>

 


 Args
  source   1-D string Tensor, the strings to split.  
  sep   0-D string Tensor, the delimiter character, the string should be length 0 or 1. Default is ' '.  
  skip_empty   A bool. If True, skip the empty strings from the result.  
  delimiter   deprecated alias for sep.  
  result_type   The tensor type for the result: one of "RaggedTensor" or "SparseTensor".  
  name   A name for the operation (optional).   
 


 Raises
  ValueError   If delimiter is not a string.   
 


 Returns   A SparseTensor or RaggedTensor of rank 2, the strings split according to the delimiter. The first column of the indices corresponds to the row in source and the second column corresponds to the index of the split component in this row.
tf.compat.v1.string_to_hash_bucket Converts each string in the input Tensor to its hash mod by a number of buckets.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.strings.to_hash_bucket  
tf.compat.v1.string_to_hash_bucket(
    string_tensor=None, num_buckets=None, name=None, input=None
)
 The hash function is deterministic on the content of the string within the process. Note that the hash function may change from time to time. This functionality will be deprecated and it's recommended to use tf.string_to_hash_bucket_fast() or tf.string_to_hash_bucket_strong().
 


 Args
  string_tensor   A Tensor of type string.  
  num_buckets   An int that is >= 1. The number of buckets.  
  name   A name for the operation (optional).   
 


 Returns   A Tensor of type int64.
tf.compat.v1.string_to_number Converts each string in the input Tensor to the specified numeric type.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.strings.to_number  
tf.compat.v1.string_to_number(
    string_tensor=None, out_type=tf.dtypes.float32, name=None, input=None
)
 (Note that int32 overflow results in an error while float overflow results in a rounded value.) Example: 
strings = ["5.0", "3.0", "7.0"]
tf.strings.to_number(strings)
<tf.Tensor: shape=(3,), dtype=float32, numpy=array([5., 3., 7.], dtype=float32)>

 


 Args
  string_tensor   A Tensor of type string.  
  out_type   An optional tf.DType from: tf.float32, tf.float64, tf.int32, tf.int64. Defaults to tf.float32. The numeric type to interpret each string in string_tensor as.  
  name   A name for the operation (optional).   
 


 Returns   A Tensor of type out_type.
tf.compat.v1.substr Return substrings from Tensor of strings. 
tf.compat.v1.substr(
    input, pos, len, name=None, unit='BYTE'
)
 For each string in the input Tensor, creates a substring starting at index pos with a total length of len. If len defines a substring that would extend beyond the length of the input string, or if len is negative, then as many characters as possible are used. A negative pos indicates distance within the string backwards from the end. If pos specifies an index which is out of range for any of the input strings, then an InvalidArgumentError is thrown. pos and len must have the same shape, otherwise a ValueError is thrown on Op creation. 
Note: Substr supports broadcasting up to two dimensions. More about broadcasting here
 Examples Using scalar pos and len: input = [b'Hello', b'World']
position = 1
length = 3

output = [b'ell', b'orl']
 Using pos and len with same shape as input: input = [[b'ten', b'eleven', b'twelve'],
         [b'thirteen', b'fourteen', b'fifteen'],
         [b'sixteen', b'seventeen', b'eighteen']]
position = [[1, 2, 3],
            [1, 2, 3],
            [1, 2, 3]]
length =   [[2, 3, 4],
            [4, 3, 2],
            [5, 5, 5]]

output = [[b'en', b'eve', b'lve'],
          [b'hirt', b'urt', b'te'],
          [b'ixtee', b'vente', b'hteen']]
 Broadcasting pos and len onto input: input = [[b'ten', b'eleven', b'twelve'],
         [b'thirteen', b'fourteen', b'fifteen'],
         [b'sixteen', b'seventeen', b'eighteen'],
         [b'nineteen', b'twenty', b'twentyone']]
position = [1, 2, 3]
length =   [1, 2, 3]

output = [[b'e', b'ev', b'lve'],
          [b'h', b'ur', b'tee'],
          [b'i', b've', b'hte'],
          [b'i', b'en', b'nty']]
 Broadcasting input onto pos and len: input = b'thirteen'
position = [1, 5, 7]
length =   [3, 2, 1]

output = [b'hir', b'ee', b'n']

 


 Raises    
ValueError: If the first argument cannot be converted to a Tensor of dtype string. 
InvalidArgumentError: If indices are out of range. 
ValueError: If pos and len are not the same shape. 

 

 


 Args
  input   A Tensor of type string. Tensor of strings  
  pos   A Tensor. Must be one of the following types: int32, int64. Scalar defining the position of first character in each substring  
  len   A Tensor. Must have the same type as pos. Scalar defining the number of characters to include in each substring  
  unit   An optional string from: "BYTE", "UTF8_CHAR". Defaults to "BYTE". The unit that is used to create the substring. One of: "BYTE" (for defining position and length by bytes) or "UTF8_CHAR" (for the UTF-8 encoded Unicode code points). The default is "BYTE". Results are undefined if unit=UTF8_CHAR and the input strings do not contain structurally valid UTF-8.  
  name   A name for the operation (optional).   
 


 Returns   A Tensor of type string.