neulab/docprompting-conala huggingface dataset:
tf.saved_model.experimental.VariablePolicy Enum defining options for variable handling when saving.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.saved_model.experimental.VariablePolicy  NONE No policy applied: Distributed variables are saved as one variable, with no device attached. SAVE_VARIABLE_DEVICES When saving variables, also save their device assignment. This is useful if one wants to hardcode devices in saved models, but it also makes them non-portable if soft device placement is disabled (more details in tf.config.set_soft_device_placement). This is currently not fully supported by saved_model.load, and is mainly intended to be used when one will be reading the saved model at a lower API level. In the example below, the graph saved by the call to saved_model.save will have the variable devices correctly specified: exported = tf.train.Checkpoint()
with tf.device('/GPU:0'):
  exported.x_gpu = tf.Variable(1.0)
with tf.device('/CPU:0'):
  exported.x_cpu = tf.Variable(1.0)
tf.saved_model.save(exported, export_dir,
    options = tf.saved_model.SaveOptions(
        experimental_variable_policy=
          tf.saved_model.experimental.VariablePolicy.SAVE_VARIABLE_DEVICES))
 Distributed variables are still saved as one variable under this policy. EXPAND_DISTRIBUTED_VARIABLES Distributed variables will be saved with information about their components, allowing for their restoration on load. Also, the saved graph will contain references to those variables. This is useful when one wants to use the model for training in environments where the original distribution strategy is not available.
 


 Class Variables
  EXPAND_DISTRIBUTED_VARIABLES   tf.saved_model.experimental.VariablePolicy  
  NONE   tf.saved_model.experimental.VariablePolicy  
  SAVE_VARIABLE_DEVICES   tf.saved_model.experimental.VariablePolicy
tf.saved_model.LoadOptions Options for loading a SavedModel. 
tf.saved_model.LoadOptions(
    experimental_io_device=None
)
 This function may be used in the options argument in functions that load a SavedModel (tf.saved_model.load, tf.keras.models.load_model).
 


 Args
  experimental_io_device   string. Applies in a distributed setting. Tensorflow device to use to access the filesystem. If None (default) then for each variable the filesystem is accessed from the CPU:0 device of the host where that variable is assigned. If specified, the filesystem is instead accessed from that device for all variables. This is for example useful if you want to load from a local directory, such as "/tmp" when running in a distributed setting. In that case pass a device for the host where the "/tmp" directory is accessible.   
 


 Class Variables
  experimental_io_device
tf.saved_model.SaveOptions Options for saving to SavedModel.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.saved_model.SaveOptions  
tf.saved_model.SaveOptions(
    namespace_whitelist=None, save_debug_info=False, function_aliases=None,
    experimental_io_device=None, experimental_variable_policy=None
)
 This function may be used in the options argument in functions that save a SavedModel (tf.saved_model.save, tf.keras.models.save_model).
 


 Args
  namespace_whitelist   List of strings containing op namespaces to whitelist when saving a model. Saving an object that uses namespaced ops must explicitly add all namespaces to the whitelist. The namespaced ops must be registered into the framework when loading the SavedModel.  
  save_debug_info   Boolean indicating whether debug information is saved. If True, then a debug/saved_model_debug_info.pb file will be written with the contents of a GraphDebugInfo binary protocol buffer containing stack trace information for all ops and functions that are saved.  
  function_aliases   Python dict. Mapping from string to object returned by @tf.function. A single tf.function can generate many ConcreteFunctions. If a downstream tool wants to refer to all concrete functions generated by a single tf.function you can use the function_aliases argument to store a map from the alias name to all concrete function names. E.g. class MyModel:
@tf.function
def func():
...

@tf.function
def serve():
...
func()

model = MyModel()
signatures = {
'serving_default': model.serve.get_concrete_function(),
}
options = tf.saved_model.SaveOptions(function_aliases={
'my_func': func,
})
tf.saved_model.save(model, export_dir, signatures, options)

 
  experimental_io_device   string. Applies in a distributed setting. Tensorflow device to use to access the filesystem. If None (default) then for each variable the filesystem is accessed from the CPU:0 device of the host where that variable is assigned. If specified, the filesystem is instead accessed from that device for all variables. This is for example useful if you want to save to a local directory, such as "/tmp" when running in a distributed setting. In that case pass a device for the host where the "/tmp" directory is accessible. 
 
  experimental_variable_policy   The policy to apply to variables when saving. This is either a saved_model.experimental.VariablePolicy enum instance or one of its value strings (case is not important). See that enum documentation for details. A value of None corresponds to the default policy.   
 


 Class Variables
  experimental_io_device  
 
  experimental_variable_policy  
 
  function_aliases  
 
  namespace_whitelist  
 
  save_debug_info
tf.searchsorted     View source on GitHub    Searches input tensor for values on the innermost dimension.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.searchsorted  
tf.searchsorted(
    sorted_sequence, values, side='left', out_type=tf.dtypes.int32,
    name=None
)
 A 2-D example: sorted_sequence = [[0, 3, 9, 9, 10],
                   [1, 2, 3, 4, 5]]
values = [[2, 4, 9],
          [0, 2, 6]]

result = searchsorted(sorted_sequence, values, side="left")

result == [[1, 2, 2],
           [0, 1, 5]]

result = searchsorted(sorted_sequence, values, side="right")

result == [[1, 2, 4],
           [0, 2, 5]]

 


 Args
  sorted_sequence   N-D Tensor containing a sorted sequence.  
  values   N-D Tensor containing the search values.  
  side   'left' or 'right'; 'left' corresponds to lower_bound and 'right' to upper_bound.  
  out_type   The output type (int32 or int64). Default is tf.int32.  
  name   Optional name for the operation.   
 


 Returns   An N-D Tensor the size of values containing the result of applying either lower_bound or upper_bound (depending on side) to each value. The result is not a global index to the entire Tensor, but the index in the last dimension.  

 


 Raises
  ValueError   If the last dimension of sorted_sequence >= 2^31-1 elements. If the total size of values exceeds 2^31 - 1 elements. If the first N-1 dimensions of the two tensors don't match.
tf.sequence_mask     View source on GitHub    Returns a mask tensor representing the first N positions of each cell.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.sequence_mask  
tf.sequence_mask(
    lengths, maxlen=None, dtype=tf.dtypes.bool, name=None
)
 If lengths has shape [d_1, d_2, ..., d_n] the resulting tensor mask has dtype dtype and shape [d_1, d_2, ..., d_n, maxlen], with mask[i_1, i_2, ..., i_n, j] = (j < lengths[i_1, i_2, ..., i_n])
 Examples: tf.sequence_mask([1, 3, 2], 5)  # [[True, False, False, False, False],
                                #  [True, True, True, False, False],
                                #  [True, True, False, False, False]]

tf.sequence_mask([[1, 3],[2,0]])  # [[[True, False, False],
                                  #   [True, True, True]],
                                  #  [[True, True, False],
                                  #   [False, False, False]]]

 


 Args
  lengths   integer tensor, all its values <= maxlen.  
  maxlen   scalar integer tensor, size of last dimension of returned tensor. Default is the maximum value in lengths.  
  dtype   output type of the resulting tensor.  
  name   name of the op.   
 


 Returns   A mask tensor of shape lengths.shape + (maxlen,), cast to specified dtype.  

 


 Raises
  ValueError   if maxlen is not a scalar.
Module: tf.sets Tensorflow set operations. Functions difference(...): Compute set difference of elements in last dimension of a and b. intersection(...): Compute set intersection of elements in last dimension of a and b. size(...): Compute number of unique elements along last dimension of a. union(...): Compute set union of elements in last dimension of a and b.
tf.sets.difference     View source on GitHub    Compute set difference of elements in last dimension of a and b.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.sets.difference, tf.compat.v1.sets.set_difference  
tf.sets.difference(
    a, b, aminusb=True, validate_indices=True
)
 All but the last dimension of a and b must match. Example: import tensorflow as tf
import collections

# Represent the following array of sets as a sparse tensor:
# a = np.array([[{1, 2}, {3}], [{4}, {5, 6}]])
a = collections.OrderedDict([
    ((0, 0, 0), 1),
    ((0, 0, 1), 2),
    ((0, 1, 0), 3),
    ((1, 0, 0), 4),
    ((1, 1, 0), 5),
    ((1, 1, 1), 6),
])
a = tf.sparse.SparseTensor(list(a.keys()), list(a.values()),
                           dense_shape=[2, 2, 2])

# np.array([[{1, 3}, {2}], [{4, 5}, {5, 6, 7, 8}]])
b = collections.OrderedDict([
    ((0, 0, 0), 1),
    ((0, 0, 1), 3),
    ((0, 1, 0), 2),
    ((1, 0, 0), 4),
    ((1, 0, 1), 5),
    ((1, 1, 0), 5),
    ((1, 1, 1), 6),
    ((1, 1, 2), 7),
    ((1, 1, 3), 8),
])
b = tf.sparse.SparseTensor(list(b.keys()), list(b.values()),
                           dense_shape=[2, 2, 4])

# `set_difference` is applied to each aligned pair of sets.
tf.sets.difference(a, b)

# The result will be equivalent to either of:
#
# np.array([[{2}, {3}], [{}, {}]])
#
# collections.OrderedDict([
#     ((0, 0, 0), 2),
#     ((0, 1, 0), 3),
# ])

 


 Args
  a   Tensor or SparseTensor of the same type as b. If sparse, indices must be sorted in row-major order.  
  b   Tensor or SparseTensor of the same type as a. If sparse, indices must be sorted in row-major order.  
  aminusb   Whether to subtract b from a, vs vice versa.  
  validate_indices   Whether to validate the order and range of sparse indices in a and b.   
 


 Returns   A SparseTensor whose shape is the same rank as a and b, and all but the last dimension the same. Elements along the last dimension contain the differences.  

 


 Raises
  TypeError   If inputs are invalid types, or if a and b have different types.  
  ValueError   If a is sparse and b is dense.  
  errors_impl.InvalidArgumentError   If the shapes of a and b do not match in any dimension other than the last dimension.
tf.sets.intersection     View source on GitHub    Compute set intersection of elements in last dimension of a and b.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.sets.intersection, tf.compat.v1.sets.set_intersection  
tf.sets.intersection(
    a, b, validate_indices=True
)
 All but the last dimension of a and b must match. Example: import tensorflow as tf
import collections

# Represent the following array of sets as a sparse tensor:
# a = np.array([[{1, 2}, {3}], [{4}, {5, 6}]])
a = collections.OrderedDict([
    ((0, 0, 0), 1),
    ((0, 0, 1), 2),
    ((0, 1, 0), 3),
    ((1, 0, 0), 4),
    ((1, 1, 0), 5),
    ((1, 1, 1), 6),
])
a = tf.sparse.SparseTensor(list(a.keys()), list(a.values()),
                           dense_shape=[2,2,2])

# b = np.array([[{1}, {}], [{4}, {5, 6, 7, 8}]])
b = collections.OrderedDict([
    ((0, 0, 0), 1),
    ((1, 0, 0), 4),
    ((1, 1, 0), 5),
    ((1, 1, 1), 6),
    ((1, 1, 2), 7),
    ((1, 1, 3), 8),
])
b = tf.sparse.SparseTensor(list(b.keys()), list(b.values()),
                           dense_shape=[2, 2, 4])

# `tf.sets.intersection` is applied to each aligned pair of sets.
tf.sets.intersection(a, b)

# The result will be equivalent to either of:
#
# np.array([[{1}, {}], [{4}, {5, 6}]])
#
# collections.OrderedDict([
#     ((0, 0, 0), 1),
#     ((1, 0, 0), 4),
#     ((1, 1, 0), 5),
#     ((1, 1, 1), 6),
# ])

 


 Args
  a   Tensor or SparseTensor of the same type as b. If sparse, indices must be sorted in row-major order.  
  b   Tensor or SparseTensor of the same type as a. If sparse, indices must be sorted in row-major order.  
  validate_indices   Whether to validate the order and range of sparse indices in a and b.   
 


 Returns   A SparseTensor whose shape is the same rank as a and b, and all but the last dimension the same. Elements along the last dimension contain the intersections.
tf.sets.size     View source on GitHub    Compute number of unique elements along last dimension of a.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.sets.set_size, tf.compat.v1.sets.size  
tf.sets.size(
    a, validate_indices=True
)

 


 Args
  a   SparseTensor, with indices sorted in row-major order.  
  validate_indices   Whether to validate the order and range of sparse indices in a.   
 


 Returns   int32 Tensor of set sizes. For a ranked n, this is a Tensor with rank n-1, and the same 1st n-1 dimensions as a. Each value is the number of unique elements in the corresponding [0...n-1] dimension of a.  

 


 Raises
  TypeError   If a is an invalid types.
tf.sets.union     View source on GitHub    Compute set union of elements in last dimension of a and b.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.sets.set_union, tf.compat.v1.sets.union  
tf.sets.union(
    a, b, validate_indices=True
)
 All but the last dimension of a and b must match. Example: import tensorflow as tf
import collections

# [[{1, 2}, {3}], [{4}, {5, 6}]]
a = collections.OrderedDict([
    ((0, 0, 0), 1),
    ((0, 0, 1), 2),
    ((0, 1, 0), 3),
    ((1, 0, 0), 4),
    ((1, 1, 0), 5),
    ((1, 1, 1), 6),
])
a = tf.sparse.SparseTensor(list(a.keys()), list(a.values()),
                           dense_shape=[2, 2, 2])

# [[{1, 3}, {2}], [{4, 5}, {5, 6, 7, 8}]]
b = collections.OrderedDict([
    ((0, 0, 0), 1),
    ((0, 0, 1), 3),
    ((0, 1, 0), 2),
    ((1, 0, 0), 4),
    ((1, 0, 1), 5),
    ((1, 1, 0), 5),
    ((1, 1, 1), 6),
    ((1, 1, 2), 7),
    ((1, 1, 3), 8),
])
b = tf.sparse.SparseTensor(list(b.keys()), list(b.values()),
                           dense_shape=[2, 2, 4])

# `set_union` is applied to each aligned pair of sets.
tf.sets.union(a, b)

# The result will be a equivalent to either of:
#
# np.array([[{1, 2, 3}, {2, 3}], [{4, 5}, {5, 6, 7, 8}]])
#
# collections.OrderedDict([
#     ((0, 0, 0), 1),
#     ((0, 0, 1), 2),
#     ((0, 0, 2), 3),
#     ((0, 1, 0), 2),
#     ((0, 1, 1), 3),
#     ((1, 0, 0), 4),
#     ((1, 0, 1), 5),
#     ((1, 1, 0), 5),
#     ((1, 1, 1), 6),
#     ((1, 1, 2), 7),
#     ((1, 1, 3), 8),
# ])

 


 Args
  a   Tensor or SparseTensor of the same type as b. If sparse, indices must be sorted in row-major order.  
  b   Tensor or SparseTensor of the same type as a. If sparse, indices must be sorted in row-major order.  
  validate_indices   Whether to validate the order and range of sparse indices in a and b.   
 


 Returns   A SparseTensor whose shape is the same rank as a and b, and all but the last dimension the same. Elements along the last dimension contain the unions.
tf.shape     View source on GitHub    Returns a tensor containing the shape of the input tensor. 
tf.shape(
    input, out_type=tf.dtypes.int32, name=None
)
 See also tf.size, tf.rank. tf.shape returns a 1-D integer tensor representing the shape of input. For a scalar input, the tensor returned has a shape of (0,) and its value is the empty vector (i.e. []). For example: 
tf.shape(1.)
<tf.Tensor: shape=(0,), dtype=int32, numpy=array([], dtype=int32)>
 
t = tf.constant([[[1, 1, 1], [2, 2, 2]], [[3, 3, 3], [4, 4, 4]]])
tf.shape(t)
<tf.Tensor: shape=(3,), dtype=int32, numpy=array([2, 2, 3], dtype=int32)>
 
Note: When using symbolic tensors, such as when using the Keras API, tf.shape() will return the shape of the symbolic tensor.
 
a = tf.keras.layers.Input((None, 10))
tf.shape(a)
<... shape=(3,) dtype=int32...>
 In these cases, using tf.Tensor.shape will return more informative results. 
a.shape
TensorShape([None, None, 10])
 (The first None represents the as yet unknown batch size.) tf.shape and Tensor.shape should be identical in eager mode. Within tf.function or within a compat.v1 context, not all dimensions may be known until execution time. Hence when defining custom layers and models for graph mode, prefer the dynamic tf.shape(x) over the static x.shape.
 


 Args
  input   A Tensor or SparseTensor.  
  out_type   (Optional) The specified output type of the operation (int32 or int64). Defaults to tf.int32.  
  name   A name for the operation (optional).   
 


 Returns   A Tensor of type out_type.
tf.shape_n     View source on GitHub    Returns shape of tensors.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.shape_n  
tf.shape_n(
    input, out_type=tf.dtypes.int32, name=None
)

 


 Args
  input   A list of at least 1 Tensor object with the same type.  
  out_type   The specified output type of the operation (int32 or int64). Defaults to tf.int32(optional).  
  name   A name for the operation (optional).   
 


 Returns   A list with the same length as input of Tensor objects with type out_type.
Module: tf.signal Signal processing operations. See the tf.signal guide. Functions dct(...): Computes the 1D [Discrete Cosine Transform (DCT)][dct] of input. fft(...): Fast Fourier transform. fft2d(...): 2D fast Fourier transform. fft3d(...): 3D fast Fourier transform. fftshift(...): Shift the zero-frequency component to the center of the spectrum. frame(...): Expands signal's axis dimension into frames of frame_length. hamming_window(...): Generate a Hamming window. hann_window(...): Generate a Hann window. idct(...): Computes the 1D [Inverse Discrete Cosine Transform (DCT)][idct] of input. ifft(...): Inverse fast Fourier transform. ifft2d(...): Inverse 2D fast Fourier transform. ifft3d(...): Inverse 3D fast Fourier transform. ifftshift(...): The inverse of fftshift. inverse_mdct(...): Computes the inverse modified DCT of mdcts. inverse_stft(...): Computes the inverse Short-time Fourier Transform of stfts. inverse_stft_window_fn(...): Generates a window function that can be used in inverse_stft. irfft(...): Inverse real-valued fast Fourier transform. irfft2d(...): Inverse 2D real-valued fast Fourier transform. irfft3d(...): Inverse 3D real-valued fast Fourier transform. kaiser_bessel_derived_window(...): Generate a [Kaiser Bessel derived window][kbd]. kaiser_window(...): Generate a [Kaiser window][kaiser]. linear_to_mel_weight_matrix(...): Returns a matrix to warp linear scale spectrograms to the mel scale. mdct(...): Computes the [Modified Discrete Cosine Transform][mdct] of signals. mfccs_from_log_mel_spectrograms(...): Computes MFCCs of log_mel_spectrograms. overlap_and_add(...): Reconstructs a signal from a framed representation. rfft(...): Real-valued fast Fourier transform. rfft2d(...): 2D real-valued fast Fourier transform. rfft3d(...): 3D real-valued fast Fourier transform. stft(...): Computes the Short-time Fourier Transform of signals. vorbis_window(...): Generate a [Vorbis power complementary window][vorbis].
tf.signal.dct     View source on GitHub    Computes the 1D Discrete Cosine Transform (DCT) of input.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.signal.dct, tf.compat.v1.spectral.dct  
tf.signal.dct(
    input, type=2, n=None, axis=-1, norm=None, name=None
)
 Types I, II, III and IV are supported. Type I is implemented using a length 2N padded tf.signal.rfft. Type II is implemented using a length 2N padded tf.signal.rfft, as described here: Type 2 DCT using 2N FFT padded (Makhoul). Type III is a fairly straightforward inverse of Type II (i.e. using a length 2N padded tf.signal.irfft). Type IV is calculated through 2N length DCT2 of padded signal and picking the odd indices.
 


 Args
  input   A [..., samples] float32/float64 Tensor containing the signals to take the DCT of.  
  type   The DCT type to perform. Must be 1, 2, 3 or 4.  
  n   The length of the transform. If length is less than sequence length, only the first n elements of the sequence are considered for the DCT. If n is greater than the sequence length, zeros are padded and then the DCT is computed as usual.  
  axis   For future expansion. The axis to compute the DCT along. Must be -1.  
  norm   The normalization to apply. None for no normalization or 'ortho' for orthonormal normalization.  
  name   An optional name for the operation.   
 


 Returns   A [..., samples] float32/float64 Tensor containing the DCT of input.  

 


 Raises
  ValueError   If type is not 1, 2, 3 or 4, axis is not -1, n is not None or greater than 0, or norm is not None or 'ortho'.  
  ValueError   If type is 1 and norm is ortho.    Scipy Compatibility Equivalent to scipy.fftpack.dct for Type-I, Type-II, Type-III and Type-IV DCT.
tf.signal.fft Fast Fourier transform.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.fft, tf.compat.v1.signal.fft, tf.compat.v1.spectral.fft  
tf.signal.fft(
    input, name=None
)
 Computes the 1-dimensional discrete Fourier transform over the inner-most dimension of input.
 


 Args
  input   A Tensor. Must be one of the following types: complex64, complex128. A complex tensor.  
  name   A name for the operation (optional).   
 


 Returns   A Tensor. Has the same type as input.
tf.signal.fft2d 2D fast Fourier transform.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.fft2d, tf.compat.v1.signal.fft2d, tf.compat.v1.spectral.fft2d  
tf.signal.fft2d(
    input, name=None
)
 Computes the 2-dimensional discrete Fourier transform over the inner-most 2 dimensions of input.
 


 Args
  input   A Tensor. Must be one of the following types: complex64, complex128. A complex tensor.  
  name   A name for the operation (optional).   
 


 Returns   A Tensor. Has the same type as input.
tf.signal.fft3d 3D fast Fourier transform.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.fft3d, tf.compat.v1.signal.fft3d, tf.compat.v1.spectral.fft3d  
tf.signal.fft3d(
    input, name=None
)
 Computes the 3-dimensional discrete Fourier transform over the inner-most 3 dimensions of input.
 


 Args
  input   A Tensor. Must be one of the following types: complex64, complex128. A complex tensor.  
  name   A name for the operation (optional).   
 


 Returns   A Tensor. Has the same type as input.
tf.signal.fftshift     View source on GitHub    Shift the zero-frequency component to the center of the spectrum.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.signal.fftshift  
tf.signal.fftshift(
    x, axes=None, name=None
)
 This function swaps half-spaces for all axes listed (defaults to all). Note that y[0] is the Nyquist component only if len(x) is even. For example: x = tf.signal.fftshift([ 0.,  1.,  2.,  3.,  4., -5., -4., -3., -2., -1.])
x.numpy() # array([-5., -4., -3., -2., -1.,  0.,  1.,  2.,  3.,  4.])

 


 Args
  x   Tensor, input tensor.  
  axes   int or shape tuple, optional Axes over which to shift. Default is None, which shifts all axes.  
  name   An optional name for the operation.   
 


 Returns   A Tensor, The shifted tensor.  
 Numpy Compatibility Equivalent to numpy.fft.fftshift. https://docs.scipy.org/doc/numpy/reference/generated/numpy.fft.fftshift.html