neulab/docprompting-conala huggingface dataset:
Module: tf.distribute.experimental.partitioners Public API for tf.distribute.experimental.partitioners namespace. Classes class FixedShardsPartitioner: Partitioner that allocates a fixed number of shards. class MaxSizePartitioner: Partitioner that keeps shards below max_shard_bytes. class MinSizePartitioner: Partitioner that allocates a minimum size per shard. class Partitioner: Partitioner base class: all partitiners inherit from this class.
tf.distribute.experimental.partitioners.FixedShardsPartitioner Partitioner that allocates a fixed number of shards. Inherits From: Partitioner 
tf.distribute.experimental.partitioners.FixedShardsPartitioner(
    num_shards
)
 Examples: 
# standalone usage:
partitioner = FixedShardsPartitioner(num_shards=2)
partitions = partitioner(tf.TensorShape([10, 3]), tf.float32)
[2, 1]

# use in ParameterServerStrategy
# strategy = tf.distribute.experimental.ParameterServerStrategy(
#   cluster_resolver=cluster_resolver, variable_partitioner=partitioner)

 


 Args
  num_shards   int, number of shards to partition.    Methods __call__ View source 
__call__(
    shape, dtype, axis=0
)
 Partitions the given shape and returns the partition results. Examples of a partitioner that allocates a fixed number of shards: partitioner = FixedShardsPartitioner(num_shards=2)
partitions = partitioner(tf.TensorShape([10, 3], tf.float32), axis=0)
print(partitions) # [2, 0]

 


 Args
  shape   a tf.TensorShape, the shape to partition.  
  dtype   a tf.dtypes.Dtype indicating the type of the partition value.  
  axis   The axis to partition along. Default: outermost axis.   
 


 Returns   A list of integers representing the number of partitions on each axis, where i-th value correponds to i-th axis.
tf.distribute.experimental.partitioners.MaxSizePartitioner Partitioner that keeps shards below max_shard_bytes. Inherits From: Partitioner 
tf.distribute.experimental.partitioners.MaxSizePartitioner(
    max_shard_bytes, max_shards=None, bytes_per_string=16
)
 This partitioner ensures each shard has at most max_shard_bytes, and tries to allocate as few shards as possible, i.e., keeping shard size as large as possible. If the partitioner hits the max_shards limit, then each shard may end up larger than max_shard_bytes. By default max_shards equals None and no limit on the number of shards is enforced. Examples: 
partitioner = MaxSizePartitioner(max_shard_bytes=4)
partitions = partitioner(tf.TensorShape([6, 1]), tf.float32)
[6, 1]
partitioner = MaxSizePartitioner(max_shard_bytes=4, max_shards=2)
partitions = partitioner(tf.TensorShape([6, 1]), tf.float32)
[2, 1]
partitioner = MaxSizePartitioner(max_shard_bytes=1024)
partitions = partitioner(tf.TensorShape([6, 1]), tf.float32)
[1, 1]

# use in ParameterServerStrategy
# strategy = tf.distribute.experimental.ParameterServerStrategy(
#   cluster_resolver=cluster_resolver, variable_partitioner=partitioner)

 


 Args
  max_shard_bytes   The maximum size any given shard is allowed to be.  
  max_shards   The maximum number of shards in int created taking precedence over max_shard_bytes.  
  bytes_per_string   If the partition value is of type string, this provides an estimate of how large each string is.    Methods __call__ View source 
__call__(
    shape, dtype, axis=0
)
 Partitions the given shape and returns the partition results. Examples of a partitioner that allocates a fixed number of shards: partitioner = FixedShardsPartitioner(num_shards=2)
partitions = partitioner(tf.TensorShape([10, 3], tf.float32), axis=0)
print(partitions) # [2, 0]

 


 Args
  shape   a tf.TensorShape, the shape to partition.  
  dtype   a tf.dtypes.Dtype indicating the type of the partition value.  
  axis   The axis to partition along. Default: outermost axis.   
 


 Returns   A list of integers representing the number of partitions on each axis, where i-th value correponds to i-th axis.
tf.distribute.experimental.partitioners.MinSizePartitioner Partitioner that allocates a minimum size per shard. Inherits From: Partitioner 
tf.distribute.experimental.partitioners.MinSizePartitioner(
    min_shard_bytes=(256 << 10), max_shards=1, bytes_per_string=16
)
 This partitioner ensures each shard has at least min_shard_bytes, and tries to allocate as many shards as possible, i.e., keeping shard size as small as possible. The maximum number of such shards (upper bound) is given by max_shards. Examples: 
partitioner = MinSizePartitioner(min_shard_bytes=4, max_shards=2)
partitions = partitioner(tf.TensorShape([6, 1]), tf.float32)
[2, 1]
partitioner = MinSizePartitioner(min_shard_bytes=4, max_shards=10)
partitions = partitioner(tf.TensorShape([6, 1]), tf.float32)
[6, 1]

# use in ParameterServerStrategy
# strategy = tf.distribute.experimental.ParameterServerStrategy(
#   cluster_resolver=cluster_resolver, variable_partitioner=partitioner)

 


 Args
  min_shard_bytes   Minimum bytes of each shard. Defaults to 256K.  
  max_shards   Upper bound on the number of shards. Defaults to 1.  
  bytes_per_string   If the partition value is of type string, this provides an estimate of how large each string is.    Methods __call__ View source 
__call__(
    shape, dtype, axis=0
)
 Partitions the given shape and returns the partition results. Examples of a partitioner that allocates a fixed number of shards: partitioner = FixedShardsPartitioner(num_shards=2)
partitions = partitioner(tf.TensorShape([10, 3], tf.float32), axis=0)
print(partitions) # [2, 0]

 


 Args
  shape   a tf.TensorShape, the shape to partition.  
  dtype   a tf.dtypes.Dtype indicating the type of the partition value.  
  axis   The axis to partition along. Default: outermost axis.   
 


 Returns   A list of integers representing the number of partitions on each axis, where i-th value correponds to i-th axis.
tf.distribute.experimental.partitioners.Partitioner Partitioner base class: all partitiners inherit from this class. Partitioners should implement a __call__ method with the following signature: def __call__(self, shape, dtype, axis=0):
  # Partitions the given `shape` and returns the partition results.
  # See docstring of `__call__` method for the format of partition results.
 Methods __call__ View source 
__call__(
    shape, dtype, axis=0
)
 Partitions the given shape and returns the partition results. Examples of a partitioner that allocates a fixed number of shards: partitioner = FixedShardsPartitioner(num_shards=2)
partitions = partitioner(tf.TensorShape([10, 3], tf.float32), axis=0)
print(partitions) # [2, 0]

 


 Args
  shape   a tf.TensorShape, the shape to partition.  
  dtype   a tf.dtypes.Dtype indicating the type of the partition value.  
  axis   The axis to partition along. Default: outermost axis.   
 


 Returns   A list of integers representing the number of partitions on each axis, where i-th value correponds to i-th axis.
tf.distribute.experimental.ValueContext A class wrapping information needed by a distribute function. 
tf.distribute.experimental.ValueContext(
    replica_id_in_sync_group=0, num_replicas_in_sync=1
)
 This is a context class that is passed to the value_fn in strategy.experimental_distribute_values_from_function and contains information about the compute replicas. The num_replicas_in_sync and replica_id can be used to customize the value on each replica. Example usage:  Directly constructed.  
def value_fn(context):
  return context.replica_id_in_sync_group/context.num_replicas_in_sync
context = tf.distribute.experimental.ValueContext(
  replica_id_in_sync_group=2, num_replicas_in_sync=4)
per_replica_value = value_fn(context)
per_replica_value
0.5
  Passed in by experimental_distribute_values_from_function.  
strategy = tf.distribute.MirroredStrategy(["GPU:0", "GPU:1"])
def value_fn(value_context):
  return value_context.num_replicas_in_sync
distributed_values = (
     strategy.experimental_distribute_values_from_function(
       value_fn))
local_result = strategy.experimental_local_results(distributed_values)
local_result
(2, 2)

 


 Args
  replica_id_in_sync_group   the current replica_id, should be an int in [0,num_replicas_in_sync).  
  num_replicas_in_sync   the number of replicas that are in sync.   
 


 Attributes
  num_replicas_in_sync   Returns the number of compute replicas in sync.  
  replica_id_in_sync_group   Returns the replica ID.
tf.distribute.experimental_set_strategy     View source on GitHub    Set a tf.distribute.Strategy as current without with strategy.scope().  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.distribute.experimental_set_strategy  
tf.distribute.experimental_set_strategy(
    strategy
)
 tf.distribute.experimental_set_strategy(strategy1)
f()
tf.distribute.experimental_set_strategy(strategy2)
g()
tf.distribute.experimental_set_strategy(None)
h()
 is equivalent to: with strategy1.scope():
  f()
with strategy2.scope():
  g()
h()
 In general, you should use the with strategy.scope(): API, but this alternative may be convenient in notebooks where you would have to put each cell in a with strategy.scope(): block. 
Note: This should only be called outside of any TensorFlow scope to avoid improper nesting.

 


 Args
  strategy   A tf.distribute.Strategy object or None.   
 


 Raises
  RuntimeError   If called inside a with strategy.scope():.
tf.distribute.get_replica_context     View source on GitHub    Returns the current tf.distribute.ReplicaContext or None.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.distribute.get_replica_context  
tf.distribute.get_replica_context()
 Returns None if in a cross-replica context. Note that execution:  starts in the default (single-replica) replica context (this function will return the default ReplicaContext object); switches to cross-replica context (in which case this will return None) when entering a with tf.distribute.Strategy.scope(): block; switches to a (non-default) replica context inside strategy.run(fn, ...); if fn calls get_replica_context().merge_call(merge_fn, ...), then inside merge_fn you are back in the cross-replica context (and again this function will return None).  Most tf.distribute.Strategy methods may only be executed in a cross-replica context, in a replica context you should use the API of the tf.distribute.ReplicaContext object returned by this method instead. assert tf.distribute.get_replica_context() is not None  # default
with strategy.scope():
  assert tf.distribute.get_replica_context() is None

  def f():
    replica_context = tf.distribute.get_replica_context()  # for strategy
    assert replica_context is not None
    tf.print("Replica id: ", replica_context.replica_id_in_sync_group,
             " of ", replica_context.num_replicas_in_sync)

  strategy.run(f)

 


 Returns   The current tf.distribute.ReplicaContext object when in a replica context scope, else None. Within a particular block, exactly one of these two things will be true:  
get_replica_context() returns non-None, or 
tf.distribute.is_cross_replica_context() returns True.
tf.distribute.get_strategy     View source on GitHub    Returns the current tf.distribute.Strategy object.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.distribute.get_strategy  
tf.distribute.get_strategy()
 Typically only used in a cross-replica context: if tf.distribute.in_cross_replica_context():
  strategy = tf.distribute.get_strategy()
  ...

 


 Returns   A tf.distribute.Strategy object. Inside a with strategy.scope() block, it returns strategy, otherwise it returns the default (single-replica) tf.distribute.Strategy object.
tf.distribute.has_strategy     View source on GitHub    Return if there is a current non-default tf.distribute.Strategy.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.distribute.has_strategy  
tf.distribute.has_strategy()
 assert not tf.distribute.has_strategy()
with strategy.scope():
  assert tf.distribute.has_strategy()

 


 Returns   True if inside a with strategy.scope():.
tf.distribute.InputContext     View source on GitHub    A class wrapping information needed by an input function.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.distribute.InputContext  
tf.distribute.InputContext(
    num_input_pipelines=1, input_pipeline_id=0, num_replicas_in_sync=1
)
 This is a context class that is passed to the user's input function and contains information about the compute replicas and input pipelines. The number of compute replicas (in sync training) helps compute the local batch size from the desired global batch size for each replica. The input pipeline information can be used to return a different subset of the input in each replica (for e.g. shard the input pipeline, use a different input source etc).
 


 Args
  num_input_pipelines   the number of input pipelines in a cluster.  
  input_pipeline_id   the current input pipeline id, should be an int in [0,num_input_pipelines).  
  num_replicas_in_sync   the number of replicas that are in sync.   
 


 Attributes
  input_pipeline_id   Returns the input pipeline ID.  
  num_input_pipelines   Returns the number of input pipelines.  
  num_replicas_in_sync   Returns the number of compute replicas in sync.    Methods get_per_replica_batch_size View source 
get_per_replica_batch_size(
    global_batch_size
)
 Returns the per-replica batch size.
 


 Args
  global_batch_size   the global batch size which should be divisible by num_replicas_in_sync.   
 


 Returns   the per-replica batch size.  

 


 Raises
  ValueError   if global_batch_size not divisible by num_replicas_in_sync.
tf.distribute.InputOptions Run options for experimental_distribute_dataset(s_from_function). 
tf.distribute.InputOptions(
    experimental_prefetch_to_device=True,
    experimental_replication_mode=tf.distribute.InputReplicationMode.PER_WORKER,
    experimental_place_dataset_on_device=False
)
 This can be used to hold some strategy specific configs. # Setup TPUStrategy
resolver = tf.distribute.cluster_resolver.TPUClusterResolver(tpu='')
tf.config.experimental_connect_to_cluster(resolver)
tf.tpu.experimental.initialize_tpu_system(resolver)
strategy = tf.distribute.TPUStrategy(resolver)

dataset = tf.data.Dataset.range(16)
distributed_dataset_on_host = (
    strategy.experimental_distribute_dataset(
        dataset,
        tf.distribute.InputOptions(
            experimental_replication_mode=
            experimental_replication_mode.PER_WORKER,
            experimental_place_dataset_on_device=False)))

 


 Attributes
  experimental_prefetch_to_device   Boolean. Defaults to True. If True, dataset elements will be prefetched to accelerator device memory. When False, dataset elements are prefetched to host device memory. Must be False when using TPUEmbedding API. experimental_prefetch_to_device can only be used with experimental_replication_mode=PER_WORKER  
  experimental_replication_mode   Replication mode for the input function. Currently, the InputReplicationMode.PER_REPLICA is only supported with tf.distribute.MirroredStrategy. experimental_distribute_datasets_from_function. The default value is InputReplicationMode.PER_WORKER.  
  experimental_place_dataset_on_device   Boolean. Default to False. When True, dataset will be placed on the device, otherwise it will remain on the host. experimental_place_dataset_on_device=True can only be used with experimental_replication_mode=PER_REPLICA
tf.distribute.InputReplicationMode     View source on GitHub    Replication mode for input function.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.distribute.InputReplicationMode   
PER_WORKER: The input function will be called on each worker independently, creating as many input pipelines as number of workers. Replicas will dequeue from the local Dataset on their worker. tf.distribute.Strategy doesn't manage any state sharing between such separate input pipelines. 
PER_REPLICA: The input function will be called on each replica seperately. tf.distribute.Strategy doesn't manage any state sharing between such separate input pipelines. 
 


 Class Variables
  PER_REPLICA   tf.distribute.InputReplicationMode  
  PER_WORKER   tf.distribute.InputReplicationMode
tf.distribute.in_cross_replica_context     View source on GitHub    Returns True if in a cross-replica context.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.distribute.in_cross_replica_context  
tf.distribute.in_cross_replica_context()
 See tf.distribute.get_replica_context for details. assert not tf.distribute.in_cross_replica_context()
with strategy.scope():
  assert tf.distribute.in_cross_replica_context()

  def f():
    assert not tf.distribute.in_cross_replica_context()

  strategy.run(f)

 


 Returns   True if in a cross-replica context (get_replica_context() returns None), or False if in a replica context (get_replica_context() returns non-None).
tf.distribute.ReduceOp     View source on GitHub    Indicates how a set of values should be reduced.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.distribute.ReduceOp   
SUM: Add all the values. 
MEAN: Take the arithmetic mean ("average") of the values. 
 


 Class Variables
  MEAN   tf.distribute.ReduceOp  
  SUM   tf.distribute.ReduceOp
tf.distribute.RunOptions Run options for strategy.run.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.distribute.RunOptions  
tf.distribute.RunOptions(
    experimental_enable_dynamic_batch_size=True,
    experimental_bucketizing_dynamic_shape=False
)
 This can be used to hold some strategy specific configs.
 


 Attributes
  experimental_enable_dynamic_batch_size   Boolean. Only applies to TPUStrategy. Default to True. If True, TPUStrategy will enable dynamic padder to support dynamic batch size for the inputs. Otherwise only static shape inputs are allowed.  
  experimental_bucketizing_dynamic_shape   Boolean. Only applies to TPUStrategy. Default to False. If True, TPUStrategy will automatic bucketize inputs passed into run if the input shape is dynamic. This is a performance optimization to reduce XLA recompilation, which should not have impact on correctness.
Module: tf.dtypes Public API for tf.dtypes namespace. Classes class DType: Represents the type of the elements in a Tensor. Functions as_dtype(...): Converts the given type_value to a DType. cast(...): Casts a tensor to a new type. complex(...): Converts two real numbers to a complex number. saturate_cast(...): Performs a safe saturating cast of value to dtype.
 


 Other Members
  QUANTIZED_DTYPES  
 
  bfloat16   tf.dtypes.DType  
  bool   tf.dtypes.DType  
  complex128   tf.dtypes.DType  
  complex64   tf.dtypes.DType  
  double   tf.dtypes.DType  
  float16   tf.dtypes.DType  
  float32   tf.dtypes.DType  
  float64   tf.dtypes.DType  
  half   tf.dtypes.DType  
  int16   tf.dtypes.DType  
  int32   tf.dtypes.DType  
  int64   tf.dtypes.DType  
  int8   tf.dtypes.DType  
  qint16   tf.dtypes.DType  
  qint32   tf.dtypes.DType  
  qint8   tf.dtypes.DType  
  quint16   tf.dtypes.DType  
  quint8   tf.dtypes.DType  
  resource   tf.dtypes.DType  
  string   tf.dtypes.DType  
  uint16   tf.dtypes.DType  
  uint32   tf.dtypes.DType  
  uint64   tf.dtypes.DType  
  uint8   tf.dtypes.DType  
  variant   tf.dtypes.DType
tf.dtypes.as_dtype     View source on GitHub    Converts the given type_value to a DType.  View aliases  Main aliases 
tf.as_dtype Compat aliases for migration See Migration guide for more details. tf.compat.v1.as_dtype, tf.compat.v1.dtypes.as_dtype  
tf.dtypes.as_dtype(
    type_value
)
 
Note: DType values are interned. When passed a new DType object, as_dtype always returns the interned value.

 


 Args
  type_value   A value that can be converted to a tf.DType object. This may currently be a tf.DType object, a DataType enum, a string type name, or a numpy.dtype.   
 


 Returns   A DType corresponding to type_value.  

 


 Raises
  TypeError   If type_value cannot be converted to a DType.
tf.dtypes.complex       View source on GitHub    Converts two real numbers to a complex number.  View aliases  Main aliases 
tf.complex Compat aliases for migration See Migration guide for more details. tf.compat.v1.complex, tf.compat.v1.dtypes.complex  
tf.dtypes.complex(
    real, imag, name=None
)
 Given a tensor real representing the real part of a complex number, and a tensor imag representing the imaginary part of a complex number, this operation returns complex numbers elementwise of the form \(a + bj\), where a represents the real part and b represents the imag part. The input tensors real and imag must have the same shape. For example: real = tf.constant([2.25, 3.25])
imag = tf.constant([4.75, 5.75])
tf.complex(real, imag)  # [[2.25 + 4.75j], [3.25 + 5.75j]]

 


 Args
  real   A Tensor. Must be one of the following types: float32, float64.  
  imag   A Tensor. Must have the same type as real.  
  name   A name for the operation (optional).   
 


 Returns   A Tensor of type complex64 or complex128.  

 


 Raises
  TypeError   Real and imag must be correct types
tf.dtypes.saturate_cast     View source on GitHub    Performs a safe saturating cast of value to dtype.  View aliases  Main aliases 
tf.saturate_cast Compat aliases for migration See Migration guide for more details. tf.compat.v1.dtypes.saturate_cast, tf.compat.v1.saturate_cast  
tf.dtypes.saturate_cast(
    value, dtype, name=None
)
 This function casts the input to dtype without applying any scaling. If there is a danger that values would over or underflow in the cast, this op applies the appropriate clamping before the cast.
 


 Args
  value   A Tensor.  
  dtype   The desired output DType.  
  name   A name for the operation (optional).   
 


 Returns   value safely cast to dtype.
tf.dynamic_partition Partitions data into num_partitions tensors using indices from partitions.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.dynamic_partition  
tf.dynamic_partition(
    data, partitions, num_partitions, name=None
)
 For each index tuple js of size partitions.ndim, the slice data[js, ...] becomes part of outputs[partitions[js]]. The slices with partitions[js] = i are placed in outputs[i] in lexicographic order of js, and the first dimension of outputs[i] is the number of entries in partitions equal to i. In detail, outputs[i].shape = [sum(partitions == i)] + data.shape[partitions.ndim:]

outputs[i] = pack([data[js, ...] for js if partitions[js] == i])
 data.shape must start with partitions.shape. For example: # Scalar partitions.
partitions = 1
num_partitions = 2
data = [10, 20]
outputs[0] = []  # Empty with shape [0, 2]
outputs[1] = [[10, 20]]

# Vector partitions.
partitions = [0, 0, 1, 1, 0]
num_partitions = 2
data = [10, 20, 30, 40, 50]
outputs[0] = [10, 20, 50]
outputs[1] = [30, 40]
 See dynamic_stitch for an example on how to merge partitions back.   
 


 Args
  data   A Tensor.  
  partitions   A Tensor of type int32. Any shape. Indices in the range [0, num_partitions).  
  num_partitions   An int that is >= 1. The number of partitions to output.  
  name   A name for the operation (optional).   
 


 Returns   A list of num_partitions Tensor objects with the same type as data.