neulab/docprompting-conala huggingface dataset:
tf.keras.metrics.Accuracy     View source on GitHub    Calculates how often predictions equal labels. Inherits From: Mean, Metric, Layer, Module  View aliases  Main aliases 
tf.metrics.Accuracy Compat aliases for migration See Migration guide for more details. tf.compat.v1.keras.metrics.Accuracy  
tf.keras.metrics.Accuracy(
    name='accuracy', dtype=None
)
 This metric creates two local variables, total and count that are used to compute the frequency with which y_pred matches y_true. This frequency is ultimately returned as binary accuracy: an idempotent operation that simply divides total by count. If sample_weight is None, weights default to 1. Use sample_weight of 0 to mask values.
 


 Args
  name   (Optional) string name of the metric instance.  
  dtype   (Optional) data type of the metric result.    Standalone usage: 
m = tf.keras.metrics.Accuracy()
m.update_state([[1], [2], [3], [4]], [[0], [2], [3], [4]])
m.result().numpy()
0.75
 
m.reset_states()
m.update_state([[1], [2], [3], [4]], [[0], [2], [3], [4]],
               sample_weight=[1, 1, 0, 0])
m.result().numpy()
0.5
 Usage with compile() API: model.compile(optimizer='sgd',
              loss='mse',
              metrics=[tf.keras.metrics.Accuracy()])
 Methods reset_states View source 
reset_states()
 Resets all of the metric state variables. This function is called between epochs/steps, when a metric is evaluated during training. result View source 
result()
 Computes and returns the metric value tensor. Result computation is an idempotent operation that simply calculates the metric value using the state variables. update_state View source 
update_state(
    y_true, y_pred, sample_weight=None
)
 Accumulates metric statistics. y_true and y_pred should have the same shape.
 


 Args
  y_true   Ground truth values. shape = [batch_size, d0, .. dN].  
  y_pred   The predicted values. shape = [batch_size, d0, .. dN].  
  sample_weight   Optional sample_weight acts as a coefficient for the metric. If a scalar is provided, then the metric is simply scaled by the given value. If sample_weight is a tensor of size [batch_size], then the metric for each sample of the batch is rescaled by the corresponding element in the sample_weight vector. If the shape of sample_weight is [batch_size, d0, .. dN-1] (or can be broadcasted to this shape), then each metric element of y_pred is scaled by the corresponding value of sample_weight. (Note on dN-1: all metric functions reduce by 1 dimension, usually the last axis (-1)).   
 


 Returns   Update op.
tf.keras.metrics.BinaryAccuracy     View source on GitHub    Calculates how often predictions match binary labels. Inherits From: Mean, Metric, Layer, Module  View aliases  Main aliases 
tf.metrics.BinaryAccuracy Compat aliases for migration See Migration guide for more details. tf.compat.v1.keras.metrics.BinaryAccuracy  
tf.keras.metrics.BinaryAccuracy(
    name='binary_accuracy', dtype=None, threshold=0.5
)
 This metric creates two local variables, total and count that are used to compute the frequency with which y_pred matches y_true. This frequency is ultimately returned as binary accuracy: an idempotent operation that simply divides total by count. If sample_weight is None, weights default to 1. Use sample_weight of 0 to mask values.
 


 Args
  name   (Optional) string name of the metric instance.  
  dtype   (Optional) data type of the metric result.  
  threshold   (Optional) Float representing the threshold for deciding whether prediction values are 1 or 0.    Standalone usage: 
m = tf.keras.metrics.BinaryAccuracy()
m.update_state([[1], [1], [0], [0]], [[0.98], [1], [0], [0.6]])
m.result().numpy()
0.75
 
m.reset_states()
m.update_state([[1], [1], [0], [0]], [[0.98], [1], [0], [0.6]],
               sample_weight=[1, 0, 0, 1])
m.result().numpy()
0.5
 Usage with compile() API: model.compile(optimizer='sgd',
              loss='mse',
              metrics=[tf.keras.metrics.BinaryAccuracy()])
 Methods reset_states View source 
reset_states()
 Resets all of the metric state variables. This function is called between epochs/steps, when a metric is evaluated during training. result View source 
result()
 Computes and returns the metric value tensor. Result computation is an idempotent operation that simply calculates the metric value using the state variables. update_state View source 
update_state(
    y_true, y_pred, sample_weight=None
)
 Accumulates metric statistics. y_true and y_pred should have the same shape.
 


 Args
  y_true   Ground truth values. shape = [batch_size, d0, .. dN].  
  y_pred   The predicted values. shape = [batch_size, d0, .. dN].  
  sample_weight   Optional sample_weight acts as a coefficient for the metric. If a scalar is provided, then the metric is simply scaled by the given value. If sample_weight is a tensor of size [batch_size], then the metric for each sample of the batch is rescaled by the corresponding element in the sample_weight vector. If the shape of sample_weight is [batch_size, d0, .. dN-1] (or can be broadcasted to this shape), then each metric element of y_pred is scaled by the corresponding value of sample_weight. (Note on dN-1: all metric functions reduce by 1 dimension, usually the last axis (-1)).   
 


 Returns   Update op.
tf.keras.metrics.BinaryCrossentropy     View source on GitHub    Computes the crossentropy metric between the labels and predictions. Inherits From: Mean, Metric, Layer, Module  View aliases  Main aliases 
tf.metrics.BinaryCrossentropy Compat aliases for migration See Migration guide for more details. tf.compat.v1.keras.metrics.BinaryCrossentropy  
tf.keras.metrics.BinaryCrossentropy(
    name='binary_crossentropy', dtype=None, from_logits=False,
    label_smoothing=0
)
 This is the crossentropy metric class to be used when there are only two label classes (0 and 1).
 


 Args
  name   (Optional) string name of the metric instance.  
  dtype   (Optional) data type of the metric result.  
  from_logits   (Optional )Whether output is expected to be a logits tensor. By default, we consider that output encodes a probability distribution.  
  label_smoothing   (Optional) Float in [0, 1]. When > 0, label values are smoothed, meaning the confidence on label values are relaxed. e.g. label_smoothing=0.2 means that we will use a value of 0.1 for label 0 and 0.9 for label 1".    Standalone usage: 
m = tf.keras.metrics.BinaryCrossentropy()
m.update_state([[0, 1], [0, 0]], [[0.6, 0.4], [0.4, 0.6]])
m.result().numpy()
0.81492424
 
m.reset_states()
m.update_state([[0, 1], [0, 0]], [[0.6, 0.4], [0.4, 0.6]],
               sample_weight=[1, 0])
m.result().numpy()
0.9162905
 Usage with compile() API: model.compile(
    optimizer='sgd',
    loss='mse',
    metrics=[tf.keras.metrics.BinaryCrossentropy()])
 Methods reset_states View source 
reset_states()
 Resets all of the metric state variables. This function is called between epochs/steps, when a metric is evaluated during training. result View source 
result()
 Computes and returns the metric value tensor. Result computation is an idempotent operation that simply calculates the metric value using the state variables. update_state View source 
update_state(
    y_true, y_pred, sample_weight=None
)
 Accumulates metric statistics. y_true and y_pred should have the same shape.
 


 Args
  y_true   Ground truth values. shape = [batch_size, d0, .. dN].  
  y_pred   The predicted values. shape = [batch_size, d0, .. dN].  
  sample_weight   Optional sample_weight acts as a coefficient for the metric. If a scalar is provided, then the metric is simply scaled by the given value. If sample_weight is a tensor of size [batch_size], then the metric for each sample of the batch is rescaled by the corresponding element in the sample_weight vector. If the shape of sample_weight is [batch_size, d0, .. dN-1] (or can be broadcasted to this shape), then each metric element of y_pred is scaled by the corresponding value of sample_weight. (Note on dN-1: all metric functions reduce by 1 dimension, usually the last axis (-1)).   
 


 Returns   Update op.
tf.keras.metrics.binary_accuracy     View source on GitHub    Calculates how often predictions matches binary labels.  View aliases  Main aliases 
tf.metrics.binary_accuracy Compat aliases for migration See Migration guide for more details. tf.compat.v1.keras.metrics.binary_accuracy  
tf.keras.metrics.binary_accuracy(
    y_true, y_pred, threshold=0.5
)
 Standalone usage: 
y_true = [[1], [1], [0], [0]]
y_pred = [[1], [1], [0], [0]]
m = tf.keras.metrics.binary_accuracy(y_true, y_pred)
assert m.shape == (4,)
m.numpy()
array([1., 1., 1., 1.], dtype=float32)

 


 Args
  y_true   Ground truth values. shape = [batch_size, d0, .. dN].  
  y_pred   The predicted values. shape = [batch_size, d0, .. dN].  
  threshold   (Optional) Float representing the threshold for deciding whether prediction values are 1 or 0.   
 


 Returns   Binary accuracy values. shape = [batch_size, d0, .. dN-1]
tf.keras.metrics.CategoricalAccuracy     View source on GitHub    Calculates how often predictions matches one-hot labels. Inherits From: Mean, Metric, Layer, Module  View aliases  Main aliases 
tf.metrics.CategoricalAccuracy Compat aliases for migration See Migration guide for more details. tf.compat.v1.keras.metrics.CategoricalAccuracy  
tf.keras.metrics.CategoricalAccuracy(
    name='categorical_accuracy', dtype=None
)
 You can provide logits of classes as y_pred, since argmax of logits and probabilities are same. This metric creates two local variables, total and count that are used to compute the frequency with which y_pred matches y_true. This frequency is ultimately returned as categorical accuracy: an idempotent operation that simply divides total by count. y_pred and y_true should be passed in as vectors of probabilities, rather than as labels. If necessary, use tf.one_hot to expand y_true as a vector. If sample_weight is None, weights default to 1. Use sample_weight of 0 to mask values.
 


 Args
  name   (Optional) string name of the metric instance.  
  dtype   (Optional) data type of the metric result.    Standalone usage: 
m = tf.keras.metrics.CategoricalAccuracy()
m.update_state([[0, 0, 1], [0, 1, 0]], [[0.1, 0.9, 0.8],
                [0.05, 0.95, 0]])
m.result().numpy()
0.5
 
m.reset_states()
m.update_state([[0, 0, 1], [0, 1, 0]], [[0.1, 0.9, 0.8],
                [0.05, 0.95, 0]],
               sample_weight=[0.7, 0.3])
m.result().numpy()
0.3
 Usage with compile() API: model.compile(
  optimizer='sgd',
  loss='mse',
  metrics=[tf.keras.metrics.CategoricalAccuracy()])
 Methods reset_states View source 
reset_states()
 Resets all of the metric state variables. This function is called between epochs/steps, when a metric is evaluated during training. result View source 
result()
 Computes and returns the metric value tensor. Result computation is an idempotent operation that simply calculates the metric value using the state variables. update_state View source 
update_state(
    y_true, y_pred, sample_weight=None
)
 Accumulates metric statistics. y_true and y_pred should have the same shape.
 


 Args
  y_true   Ground truth values. shape = [batch_size, d0, .. dN].  
  y_pred   The predicted values. shape = [batch_size, d0, .. dN].  
  sample_weight   Optional sample_weight acts as a coefficient for the metric. If a scalar is provided, then the metric is simply scaled by the given value. If sample_weight is a tensor of size [batch_size], then the metric for each sample of the batch is rescaled by the corresponding element in the sample_weight vector. If the shape of sample_weight is [batch_size, d0, .. dN-1] (or can be broadcasted to this shape), then each metric element of y_pred is scaled by the corresponding value of sample_weight. (Note on dN-1: all metric functions reduce by 1 dimension, usually the last axis (-1)).   
 


 Returns   Update op.
tf.keras.metrics.CategoricalHinge     View source on GitHub    Computes the categorical hinge metric between y_true and y_pred. Inherits From: Mean, Metric, Layer, Module  View aliases  Main aliases 
tf.metrics.CategoricalHinge Compat aliases for migration See Migration guide for more details. tf.compat.v1.keras.metrics.CategoricalHinge  
tf.keras.metrics.CategoricalHinge(
    name='categorical_hinge', dtype=None
)

 


 Args
  name   (Optional) string name of the metric instance.  
  dtype   (Optional) data type of the metric result.    Standalone usage: 
m = tf.keras.metrics.CategoricalHinge()
m.update_state([[0, 1], [0, 0]], [[0.6, 0.4], [0.4, 0.6]])
m.result().numpy()
1.4000001
 
m.reset_states()
m.update_state([[0, 1], [0, 0]], [[0.6, 0.4], [0.4, 0.6]],
               sample_weight=[1, 0])
m.result().numpy()
1.2
 Usage with compile() API: model.compile(
    optimizer='sgd',
    loss='mse',
    metrics=[tf.keras.metrics.CategoricalHinge()])
 Methods reset_states View source 
reset_states()
 Resets all of the metric state variables. This function is called between epochs/steps, when a metric is evaluated during training. result View source 
result()
 Computes and returns the metric value tensor. Result computation is an idempotent operation that simply calculates the metric value using the state variables. update_state View source 
update_state(
    y_true, y_pred, sample_weight=None
)
 Accumulates metric statistics. y_true and y_pred should have the same shape.
 


 Args
  y_true   Ground truth values. shape = [batch_size, d0, .. dN].  
  y_pred   The predicted values. shape = [batch_size, d0, .. dN].  
  sample_weight   Optional sample_weight acts as a coefficient for the metric. If a scalar is provided, then the metric is simply scaled by the given value. If sample_weight is a tensor of size [batch_size], then the metric for each sample of the batch is rescaled by the corresponding element in the sample_weight vector. If the shape of sample_weight is [batch_size, d0, .. dN-1] (or can be broadcasted to this shape), then each metric element of y_pred is scaled by the corresponding value of sample_weight. (Note on dN-1: all metric functions reduce by 1 dimension, usually the last axis (-1)).   
 


 Returns   Update op.
tf.keras.metrics.categorical_accuracy     View source on GitHub    Calculates how often predictions matches one-hot labels.  View aliases  Main aliases 
tf.metrics.categorical_accuracy Compat aliases for migration See Migration guide for more details. tf.compat.v1.keras.metrics.categorical_accuracy  
tf.keras.metrics.categorical_accuracy(
    y_true, y_pred
)
 Standalone usage: 
y_true = [[0, 0, 1], [0, 1, 0]]
y_pred = [[0.1, 0.9, 0.8], [0.05, 0.95, 0]]
m = tf.keras.metrics.categorical_accuracy(y_true, y_pred)
assert m.shape == (2,)
m.numpy()
array([0., 1.], dtype=float32)
 You can provide logits of classes as y_pred, since argmax of logits and probabilities are same.
 


 Args
  y_true   One-hot ground truth values.  
  y_pred   The prediction values.   
 


 Returns   Categorical accuracy values.
tf.keras.metrics.CosineSimilarity     View source on GitHub    Computes the cosine similarity between the labels and predictions. Inherits From: Mean, Metric, Layer, Module  View aliases  Main aliases 
tf.metrics.CosineSimilarity Compat aliases for migration See Migration guide for more details. tf.compat.v1.keras.metrics.CosineSimilarity  
tf.keras.metrics.CosineSimilarity(
    name='cosine_similarity', dtype=None, axis=-1
)
 cosine similarity = (a . b) / ||a|| ||b|| See: Cosine Similarity. This metric keeps the average cosine similarity between predictions and labels over a stream of data.
 


 Args
  name   (Optional) string name of the metric instance.  
  dtype   (Optional) data type of the metric result.  
  axis   (Optional) Defaults to -1. The dimension along which the cosine similarity is computed.    Standalone usage: 
# l2_norm(y_true) = [[0., 1.], [1./1.414], 1./1.414]]]
# l2_norm(y_pred) = [[1., 0.], [1./1.414], 1./1.414]]]
# l2_norm(y_true) . l2_norm(y_pred) = [[0., 0.], [0.5, 0.5]]
# result = mean(sum(l2_norm(y_true) . l2_norm(y_pred), axis=1))
#        = ((0. + 0.) +  (0.5 + 0.5)) / 2
m = tf.keras.metrics.CosineSimilarity(axis=1)
m.update_state([[0., 1.], [1., 1.]], [[1., 0.], [1., 1.]])
m.result().numpy()
0.49999997
 
m.reset_states()
m.update_state([[0., 1.], [1., 1.]], [[1., 0.], [1., 1.]],
               sample_weight=[0.3, 0.7])
m.result().numpy()
0.6999999
 Usage with compile() API: model.compile(
    optimizer='sgd',
    loss='mse',
    metrics=[tf.keras.metrics.CosineSimilarity(axis=1)])
 Methods reset_states View source 
reset_states()
 Resets all of the metric state variables. This function is called between epochs/steps, when a metric is evaluated during training. result View source 
result()
 Computes and returns the metric value tensor. Result computation is an idempotent operation that simply calculates the metric value using the state variables. update_state View source 
update_state(
    y_true, y_pred, sample_weight=None
)
 Accumulates metric statistics. y_true and y_pred should have the same shape.
 


 Args
  y_true   Ground truth values. shape = [batch_size, d0, .. dN].  
  y_pred   The predicted values. shape = [batch_size, d0, .. dN].  
  sample_weight   Optional sample_weight acts as a coefficient for the metric. If a scalar is provided, then the metric is simply scaled by the given value. If sample_weight is a tensor of size [batch_size], then the metric for each sample of the batch is rescaled by the corresponding element in the sample_weight vector. If the shape of sample_weight is [batch_size, d0, .. dN-1] (or can be broadcasted to this shape), then each metric element of y_pred is scaled by the corresponding value of sample_weight. (Note on dN-1: all metric functions reduce by 1 dimension, usually the last axis (-1)).   
 


 Returns   Update op.
tf.keras.metrics.deserialize     View source on GitHub    Deserializes a serialized metric class/function instance.  View aliases  Main aliases 
tf.metrics.deserialize Compat aliases for migration See Migration guide for more details. tf.compat.v1.keras.metrics.deserialize  
tf.keras.metrics.deserialize(
    config, custom_objects=None
)

 


 Arguments
  config   Metric configuration.  
  custom_objects   Optional dictionary mapping names (strings) to custom objects (classes and functions) to be considered during deserialization.   
 


 Returns   A Keras Metric instance or a metric function.
tf.keras.metrics.FalseNegatives     View source on GitHub    Calculates the number of false negatives. Inherits From: Metric, Layer, Module  View aliases  Main aliases 
tf.metrics.FalseNegatives Compat aliases for migration See Migration guide for more details. tf.compat.v1.keras.metrics.FalseNegatives  
tf.keras.metrics.FalseNegatives(
    thresholds=None, name=None, dtype=None
)
 If sample_weight is given, calculates the sum of the weights of false negatives. This metric creates one local variable, accumulator that is used to keep track of the number of false negatives. If sample_weight is None, weights default to 1. Use sample_weight of 0 to mask values.
 


 Args
  thresholds   (Optional) Defaults to 0.5. A float value or a python list/tuple of float threshold values in [0, 1]. A threshold is compared with prediction values to determine the truth value of predictions (i.e., above the threshold is true, below is false). One metric value is generated for each threshold value.  
  name   (Optional) string name of the metric instance.  
  dtype   (Optional) data type of the metric result.    Standalone usage: 
m = tf.keras.metrics.FalseNegatives()
m.update_state([0, 1, 1, 1], [0, 1, 0, 0])
m.result().numpy()
2.0
 
m.reset_states()
m.update_state([0, 1, 1, 1], [0, 1, 0, 0], sample_weight=[0, 0, 1, 0])
m.result().numpy()
1.0
 Usage with compile() API: model.compile(optimizer='sgd',
              loss='mse',
              metrics=[tf.keras.metrics.FalseNegatives()])
 Methods reset_states View source 
reset_states()
 Resets all of the metric state variables. This function is called between epochs/steps, when a metric is evaluated during training. result View source 
result()
 Computes and returns the metric value tensor. Result computation is an idempotent operation that simply calculates the metric value using the state variables. update_state View source 
update_state(
    y_true, y_pred, sample_weight=None
)
 Accumulates the metric statistics.
 


 Args
  y_true   The ground truth values.  
  y_pred   The predicted values.  
  sample_weight   Optional weighting of each example. Defaults to 1. Can be a Tensor whose rank is either 0, or the same rank as y_true, and must be broadcastable to y_true.   
 


 Returns   Update op.
tf.keras.metrics.FalsePositives     View source on GitHub    Calculates the number of false positives. Inherits From: Metric, Layer, Module  View aliases  Main aliases 
tf.metrics.FalsePositives Compat aliases for migration See Migration guide for more details. tf.compat.v1.keras.metrics.FalsePositives  
tf.keras.metrics.FalsePositives(
    thresholds=None, name=None, dtype=None
)
 If sample_weight is given, calculates the sum of the weights of false positives. This metric creates one local variable, accumulator that is used to keep track of the number of false positives. If sample_weight is None, weights default to 1. Use sample_weight of 0 to mask values.
 


 Args
  thresholds   (Optional) Defaults to 0.5. A float value or a python list/tuple of float threshold values in [0, 1]. A threshold is compared with prediction values to determine the truth value of predictions (i.e., above the threshold is true, below is false). One metric value is generated for each threshold value.  
  name   (Optional) string name of the metric instance.  
  dtype   (Optional) data type of the metric result.    Standalone usage: 
m = tf.keras.metrics.FalsePositives()
m.update_state([0, 1, 0, 0], [0, 0, 1, 1])
m.result().numpy()
2.0
 
m.reset_states()
m.update_state([0, 1, 0, 0], [0, 0, 1, 1], sample_weight=[0, 0, 1, 0])
m.result().numpy()
1.0
 Usage with compile() API: model.compile(optimizer='sgd',
              loss='mse',
              metrics=[tf.keras.metrics.FalsePositives()])
 Methods reset_states View source 
reset_states()
 Resets all of the metric state variables. This function is called between epochs/steps, when a metric is evaluated during training. result View source 
result()
 Computes and returns the metric value tensor. Result computation is an idempotent operation that simply calculates the metric value using the state variables. update_state View source 
update_state(
    y_true, y_pred, sample_weight=None
)
 Accumulates the metric statistics.
 


 Args
  y_true   The ground truth values.  
  y_pred   The predicted values.  
  sample_weight   Optional weighting of each example. Defaults to 1. Can be a Tensor whose rank is either 0, or the same rank as y_true, and must be broadcastable to y_true.   
 


 Returns   Update op.
tf.keras.metrics.get     View source on GitHub    Retrieves a Keras metric as a function/Metric class instance.  View aliases  Main aliases 
tf.metrics.get Compat aliases for migration See Migration guide for more details. tf.compat.v1.keras.metrics.get  
tf.keras.metrics.get(
    identifier
)
 The identifier may be the string name of a metric function or class. 
metric = tf.keras.metrics.get("categorical_crossentropy")
type(metric)
<class 'function'>
metric = tf.keras.metrics.get("CategoricalCrossentropy")
type(metric)
<class '...tensorflow.python.keras.metrics.CategoricalCrossentropy'>
 You can also specify config of the metric to this function by passing dict containing class_name and config as an identifier. Also note that the class_name must map to a Metric class 
identifier = {"class_name": "CategoricalCrossentropy",
              "config": {"from_logits": True} }
metric = tf.keras.metrics.get(identifier)
type(metric)
<class '...tensorflow.python.keras.metrics.CategoricalCrossentropy'>

 


 Arguments
  identifier   A metric identifier. One of None or string name of a metric function/class or metric configuration dictionary or a metric function or a metric class instance   
 


 Returns   A Keras metric as a function/ Metric class instance.  

 


 Raises
  ValueError   If identifier cannot be interpreted.