neulab/docprompting-conala huggingface dataset:
tf.feature_column.bucketized_column     View source on GitHub    Represents discretized dense input bucketed by boundaries.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.feature_column.bucketized_column  
tf.feature_column.bucketized_column(
    source_column, boundaries
)
 Buckets include the left boundary, and exclude the right boundary. Namely, boundaries=[0., 1., 2.] generates buckets (-inf, 0.), [0., 1.), [1., 2.), and [2., +inf). For example, if the inputs are boundaries = [0, 10, 100]
input tensor = [[-5, 10000]
                [150,   10]
                [5,    100]]
 then the output will be output = [[0, 3]
          [3, 2]
          [1, 3]]
 Example: price = tf.feature_column.numeric_column('price')
bucketized_price = tf.feature_column.bucketized_column(
    price, boundaries=[...])
columns = [bucketized_price, ...]
features = tf.io.parse_example(
    ..., features=tf.feature_column.make_parse_example_spec(columns))
dense_tensor = tf.keras.layers.DenseFeatures(columns)(features)
 A bucketized_column can also be crossed with another categorical column using crossed_column: price = tf.feature_column.numeric_column('price')
# bucketized_column converts numerical feature to a categorical one.
bucketized_price = tf.feature_column.bucketized_column(
    price, boundaries=[...])
# 'keywords' is a string feature.
price_x_keywords = tf.feature_column.crossed_column(
    [bucketized_price, 'keywords'], 50K)
columns = [price_x_keywords, ...]
features = tf.io.parse_example(
    ..., features=tf.feature_column.make_parse_example_spec(columns))
dense_tensor = tf.keras.layers.DenseFeatures(columns)(features)
linear_model = tf.keras.experimental.LinearModel(units=...)(dense_tensor)

 


 Args
  source_column   A one-dimensional dense column which is generated with numeric_column.  
  boundaries   A sorted list or tuple of floats specifying the boundaries.   
 


 Returns   A BucketizedColumn.  

 


 Raises
  ValueError   If source_column is not a numeric column, or if it is not one-dimensional.  
  ValueError   If boundaries is not a sorted list or tuple.
tf.feature_column.categorical_column_with_hash_bucket     View source on GitHub    Represents sparse feature where ids are set by hashing.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.feature_column.categorical_column_with_hash_bucket  
tf.feature_column.categorical_column_with_hash_bucket(
    key, hash_bucket_size, dtype=tf.dtypes.string
)
 Use this when your sparse features are in string or integer format, and you want to distribute your inputs into a finite number of buckets by hashing. output_id = Hash(input_feature_string) % bucket_size for string type input. For int type input, the value is converted to its string representation first and then hashed by the same formula. For input dictionary features, features[key] is either Tensor or SparseTensor. If Tensor, missing values can be represented by -1 for int and '' for string, which will be dropped by this feature column. Example: keywords = categorical_column_with_hash_bucket("keywords", 10K)
columns = [keywords, ...]
features = tf.io.parse_example(..., features=make_parse_example_spec(columns))
linear_prediction = linear_model(features, columns)

# or
keywords_embedded = embedding_column(keywords, 16)
columns = [keywords_embedded, ...]
features = tf.io.parse_example(..., features=make_parse_example_spec(columns))
dense_tensor = input_layer(features, columns)

 


 Args
  key   A unique string identifying the input feature. It is used as the column name and the dictionary key for feature parsing configs, feature Tensor objects, and feature columns.  
  hash_bucket_size   An int > 1. The number of buckets.  
  dtype   The type of features. Only string and integer types are supported.   
 


 Returns   A HashedCategoricalColumn.  

 


 Raises
  ValueError   hash_bucket_size is not greater than 1.  
  ValueError   dtype is neither string nor integer.
tf.feature_column.categorical_column_with_identity     View source on GitHub    A CategoricalColumn that returns identity values.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.feature_column.categorical_column_with_identity  
tf.feature_column.categorical_column_with_identity(
    key, num_buckets, default_value=None
)
 Use this when your inputs are integers in the range [0, num_buckets), and you want to use the input value itself as the categorical ID. Values outside this range will result in default_value if specified, otherwise it will fail. Typically, this is used for contiguous ranges of integer indexes, but it doesn't have to be. This might be inefficient, however, if many of IDs are unused. Consider categorical_column_with_hash_bucket in that case. For input dictionary features, features[key] is either Tensor or SparseTensor. If Tensor, missing values can be represented by -1 for int and '' for string, which will be dropped by this feature column. In the following examples, each input in the range [0, 1000000) is assigned the same value. All other inputs are assigned default_value 0. Note that a literal 0 in inputs will result in the same default ID. Linear model: video_id = categorical_column_with_identity(
    key='video_id', num_buckets=1000000, default_value=0)
columns = [video_id, ...]
features = tf.io.parse_example(..., features=make_parse_example_spec(columns))
linear_prediction, _, _ = linear_model(features, columns)
 Embedding for a DNN model: columns = [embedding_column(video_id, 9),...]
features = tf.io.parse_example(..., features=make_parse_example_spec(columns))
dense_tensor = input_layer(features, columns)

 


 Args
  key   A unique string identifying the input feature. It is used as the column name and the dictionary key for feature parsing configs, feature Tensor objects, and feature columns.  
  num_buckets   Range of inputs and outputs is [0, num_buckets).  
  default_value   If set, values outside of range [0, num_buckets) will be replaced with this value. If not set, values >= num_buckets will cause a failure while values < 0 will be dropped.   
 


 Returns   A CategoricalColumn that returns identity values.  

 


 Raises
  ValueError   if num_buckets is less than one.  
  ValueError   if default_value is not in range [0, num_buckets).
tf.feature_column.indicator_column     View source on GitHub    Represents multi-hot representation of given categorical column.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.feature_column.indicator_column  
tf.feature_column.indicator_column(
    categorical_column
)
  For DNN model, indicator_column can be used to wrap any categorical_column_* (e.g., to feed to DNN). Consider to Use embedding_column if the number of buckets/unique(values) are large. For Wide (aka linear) model, indicator_column is the internal representation for categorical column when passing categorical column directly (as any element in feature_columns) to linear_model. See linear_model for details.  name = indicator_column(categorical_column_with_vocabulary_list(
    'name', ['bob', 'george', 'wanda']))
columns = [name, ...]
features = tf.io.parse_example(..., features=make_parse_example_spec(columns))
dense_tensor = input_layer(features, columns)

dense_tensor == [[1, 0, 0]]  # If "name" bytes_list is ["bob"]
dense_tensor == [[1, 0, 1]]  # If "name" bytes_list is ["bob", "wanda"]
dense_tensor == [[2, 0, 0]]  # If "name" bytes_list is ["bob", "bob"]

 


 Args
  categorical_column   A CategoricalColumn which is created by categorical_column_with_* or crossed_column functions.   
 


 Returns   An IndicatorColumn.  

 


 Raises
  ValueError   If categorical_column is not CategoricalColumn type.
tf.feature_column.make_parse_example_spec     View source on GitHub    Creates parsing spec dictionary from input feature_columns. 
tf.feature_column.make_parse_example_spec(
    feature_columns
)
 The returned dictionary can be used as arg 'features' in tf.io.parse_example. Typical usage example: # Define features and transformations
feature_a = tf.feature_column.categorical_column_with_vocabulary_file(...)
feature_b = tf.feature_column.numeric_column(...)
feature_c_bucketized = tf.feature_column.bucketized_column(
    tf.feature_column.numeric_column("feature_c"), ...)
feature_a_x_feature_c = tf.feature_column.crossed_column(
    columns=["feature_a", feature_c_bucketized], ...)

feature_columns = set(
    [feature_b, feature_c_bucketized, feature_a_x_feature_c])
features = tf.io.parse_example(
    serialized=serialized_examples,
    features=tf.feature_column.make_parse_example_spec(feature_columns))
 For the above example, make_parse_example_spec would return the dict: {
    "feature_a": parsing_ops.VarLenFeature(tf.string),
    "feature_b": parsing_ops.FixedLenFeature([1], dtype=tf.float32),
    "feature_c": parsing_ops.FixedLenFeature([1], dtype=tf.float32)
}

 


 Args
  feature_columns   An iterable containing all feature columns. All items should be instances of classes derived from FeatureColumn.   
 


 Returns   A dict mapping each feature key to a FixedLenFeature or VarLenFeature value.  

 


 Raises
  ValueError   If any of the given feature_columns is not a FeatureColumn instance.
tf.feature_column.numeric_column     View source on GitHub    Represents real valued or numerical features.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.feature_column.numeric_column  
tf.feature_column.numeric_column(
    key, shape=(1,), default_value=None, dtype=tf.dtypes.float32, normalizer_fn=None
)
 Example: price = numeric_column('price')
columns = [price, ...]
features = tf.io.parse_example(..., features=make_parse_example_spec(columns))
dense_tensor = input_layer(features, columns)

# or
bucketized_price = bucketized_column(price, boundaries=[...])
columns = [bucketized_price, ...]
features = tf.io.parse_example(..., features=make_parse_example_spec(columns))
linear_prediction = linear_model(features, columns)

 


 Args
  key   A unique string identifying the input feature. It is used as the column name and the dictionary key for feature parsing configs, feature Tensor objects, and feature columns.  
  shape   An iterable of integers specifies the shape of the Tensor. An integer can be given which means a single dimension Tensor with given width. The Tensor representing the column will have the shape of [batch_size] + shape.  
  default_value   A single value compatible with dtype or an iterable of values compatible with dtype which the column takes on during tf.Example parsing if data is missing. A default value of None will cause tf.io.parse_example to fail if an example does not contain this column. If a single value is provided, the same value will be applied as the default value for every item. If an iterable of values is provided, the shape of the default_value should be equal to the given shape.  
  dtype   defines the type of values. Default value is tf.float32. Must be a non-quantized, real integer or floating point type.  
  normalizer_fn   If not None, a function that can be used to normalize the value of the tensor after default_value is applied for parsing. Normalizer function takes the input Tensor as its argument, and returns the output Tensor. (e.g. lambda x: (x - 3.0) / 4.2). Please note that even though the most common use case of this function is normalization, it can be used for any kind of Tensorflow transformations.   
 


 Returns   A NumericColumn.  

 


 Raises
  TypeError   if any dimension in shape is not an int  
  ValueError   if any dimension in shape is not a positive integer  
  TypeError   if default_value is an iterable but not compatible with shape  
  TypeError   if default_value is not compatible with dtype.  
  ValueError   if dtype is not convertible to tf.float32.
tf.feature_column.sequence_categorical_column_with_hash_bucket     View source on GitHub    A sequence of categorical terms where ids are set by hashing.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.feature_column.sequence_categorical_column_with_hash_bucket  
tf.feature_column.sequence_categorical_column_with_hash_bucket(
    key, hash_bucket_size, dtype=tf.dtypes.string
)
 Pass this to embedding_column or indicator_column to convert sequence categorical data into dense representation for input to sequence NN, such as RNN. Example: tokens = sequence_categorical_column_with_hash_bucket(
    'tokens', hash_bucket_size=1000)
tokens_embedding = embedding_column(tokens, dimension=10)
columns = [tokens_embedding]

features = tf.io.parse_example(..., features=make_parse_example_spec(columns))
sequence_feature_layer = SequenceFeatures(columns)
sequence_input, sequence_length = sequence_feature_layer(features)
sequence_length_mask = tf.sequence_mask(sequence_length)

rnn_cell = tf.keras.layers.SimpleRNNCell(hidden_size)
rnn_layer = tf.keras.layers.RNN(rnn_cell)
outputs, state = rnn_layer(sequence_input, mask=sequence_length_mask)

 


 Args
  key   A unique string identifying the input feature.  
  hash_bucket_size   An int > 1. The number of buckets.  
  dtype   The type of features. Only string and integer types are supported.   
 


 Returns   A SequenceCategoricalColumn.  

 


 Raises
  ValueError   hash_bucket_size is not greater than 1.  
  ValueError   dtype is neither string nor integer.
tf.feature_column.sequence_categorical_column_with_identity     View source on GitHub    Returns a feature column that represents sequences of integers.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.feature_column.sequence_categorical_column_with_identity  
tf.feature_column.sequence_categorical_column_with_identity(
    key, num_buckets, default_value=None
)
 Pass this to embedding_column or indicator_column to convert sequence categorical data into dense representation for input to sequence NN, such as RNN. Example: watches = sequence_categorical_column_with_identity(
    'watches', num_buckets=1000)
watches_embedding = embedding_column(watches, dimension=10)
columns = [watches_embedding]

features = tf.io.parse_example(..., features=make_parse_example_spec(columns))
sequence_feature_layer = SequenceFeatures(columns)
sequence_input, sequence_length = sequence_feature_layer(features)
sequence_length_mask = tf.sequence_mask(sequence_length)

rnn_cell = tf.keras.layers.SimpleRNNCell(hidden_size)
rnn_layer = tf.keras.layers.RNN(rnn_cell)
outputs, state = rnn_layer(sequence_input, mask=sequence_length_mask)

 


 Args
  key   A unique string identifying the input feature.  
  num_buckets   Range of inputs. Namely, inputs are expected to be in the range [0, num_buckets).  
  default_value   If None, this column's graph operations will fail for out-of-range inputs. Otherwise, this value must be in the range [0, num_buckets), and will replace out-of-range inputs.   
 


 Returns   A SequenceCategoricalColumn.  

 


 Raises
  ValueError   if num_buckets is less than one.  
  ValueError   if default_value is not in range [0, num_buckets).
tf.feature_column.sequence_categorical_column_with_vocabulary_file     View source on GitHub    A sequence of categorical terms where ids use a vocabulary file.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.feature_column.sequence_categorical_column_with_vocabulary_file  
tf.feature_column.sequence_categorical_column_with_vocabulary_file(
    key, vocabulary_file, vocabulary_size=None, num_oov_buckets=0,
    default_value=None, dtype=tf.dtypes.string
)
 Pass this to embedding_column or indicator_column to convert sequence categorical data into dense representation for input to sequence NN, such as RNN. Example: states = sequence_categorical_column_with_vocabulary_file(
    key='states', vocabulary_file='/us/states.txt', vocabulary_size=50,
    num_oov_buckets=5)
states_embedding = embedding_column(states, dimension=10)
columns = [states_embedding]

features = tf.io.parse_example(..., features=make_parse_example_spec(columns))
sequence_feature_layer = SequenceFeatures(columns)
sequence_input, sequence_length = sequence_feature_layer(features)
sequence_length_mask = tf.sequence_mask(sequence_length)

rnn_cell = tf.keras.layers.SimpleRNNCell(hidden_size)
rnn_layer = tf.keras.layers.RNN(rnn_cell)
outputs, state = rnn_layer(sequence_input, mask=sequence_length_mask)

 


 Args
  key   A unique string identifying the input feature.  
  vocabulary_file   The vocabulary file name.  
  vocabulary_size   Number of the elements in the vocabulary. This must be no greater than length of vocabulary_file, if less than length, later values are ignored. If None, it is set to the length of vocabulary_file.  
  num_oov_buckets   Non-negative integer, the number of out-of-vocabulary buckets. All out-of-vocabulary inputs will be assigned IDs in the range [vocabulary_size, vocabulary_size+num_oov_buckets) based on a hash of the input value. A positive num_oov_buckets can not be specified with default_value.  
  default_value   The integer ID value to return for out-of-vocabulary feature values, defaults to -1. This can not be specified with a positive num_oov_buckets.  
  dtype   The type of features. Only string and integer types are supported.   
 


 Returns   A SequenceCategoricalColumn.  

 


 Raises
  ValueError   vocabulary_file is missing or cannot be opened.  
  ValueError   vocabulary_size is missing or < 1.  
  ValueError   num_oov_buckets is a negative integer.  
  ValueError   num_oov_buckets and default_value are both specified.  
  ValueError   dtype is neither string nor integer.
tf.feature_column.sequence_categorical_column_with_vocabulary_list     View source on GitHub    A sequence of categorical terms where ids use an in-memory list.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.feature_column.sequence_categorical_column_with_vocabulary_list  
tf.feature_column.sequence_categorical_column_with_vocabulary_list(
    key, vocabulary_list, dtype=None, default_value=-1, num_oov_buckets=0
)
 Pass this to embedding_column or indicator_column to convert sequence categorical data into dense representation for input to sequence NN, such as RNN. Example: colors = sequence_categorical_column_with_vocabulary_list(
    key='colors', vocabulary_list=('R', 'G', 'B', 'Y'),
    num_oov_buckets=2)
colors_embedding = embedding_column(colors, dimension=3)
columns = [colors_embedding]

features = tf.io.parse_example(..., features=make_parse_example_spec(columns))
sequence_feature_layer = SequenceFeatures(columns)
sequence_input, sequence_length = sequence_feature_layer(features)
sequence_length_mask = tf.sequence_mask(sequence_length)

rnn_cell = tf.keras.layers.SimpleRNNCell(hidden_size)
rnn_layer = tf.keras.layers.RNN(rnn_cell)
outputs, state = rnn_layer(sequence_input, mask=sequence_length_mask)

 


 Args
  key   A unique string identifying the input feature.  
  vocabulary_list   An ordered iterable defining the vocabulary. Each feature is mapped to the index of its value (if present) in vocabulary_list. Must be castable to dtype.  
  dtype   The type of features. Only string and integer types are supported. If None, it will be inferred from vocabulary_list.  
  default_value   The integer ID value to return for out-of-vocabulary feature values, defaults to -1. This can not be specified with a positive num_oov_buckets.  
  num_oov_buckets   Non-negative integer, the number of out-of-vocabulary buckets. All out-of-vocabulary inputs will be assigned IDs in the range [len(vocabulary_list), len(vocabulary_list)+num_oov_buckets) based on a hash of the input value. A positive num_oov_buckets can not be specified with default_value.   
 


 Returns   A SequenceCategoricalColumn.  

 


 Raises
  ValueError   if vocabulary_list is empty, or contains duplicate keys.  
  ValueError   num_oov_buckets is a negative integer.  
  ValueError   num_oov_buckets and default_value are both specified.  
  ValueError   if dtype is not integer or string.
tf.feature_column.sequence_numeric_column     View source on GitHub    Returns a feature column that represents sequences of numeric data.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.feature_column.sequence_numeric_column  
tf.feature_column.sequence_numeric_column(
    key, shape=(1,), default_value=0.0, dtype=tf.dtypes.float32, normalizer_fn=None
)
 Example: temperature = sequence_numeric_column('temperature')
columns = [temperature]

features = tf.io.parse_example(..., features=make_parse_example_spec(columns))
sequence_feature_layer = SequenceFeatures(columns)
sequence_input, sequence_length = sequence_feature_layer(features)
sequence_length_mask = tf.sequence_mask(sequence_length)

rnn_cell = tf.keras.layers.SimpleRNNCell(hidden_size)
rnn_layer = tf.keras.layers.RNN(rnn_cell)
outputs, state = rnn_layer(sequence_input, mask=sequence_length_mask)

 


 Args
  key   A unique string identifying the input features.  
  shape   The shape of the input data per sequence id. E.g. if shape=(2,), each example must contain 2 * sequence_length values.  
  default_value   A single value compatible with dtype that is used for padding the sparse data into a dense Tensor.  
  dtype   The type of values.  
  normalizer_fn   If not None, a function that can be used to normalize the value of the tensor after default_value is applied for parsing. Normalizer function takes the input Tensor as its argument, and returns the output Tensor. (e.g. lambda x: (x - 3.0) / 4.2). Please note that even though the most common use case of this function is normalization, it can be used for any kind of Tensorflow transformations.   
 


 Returns   A SequenceNumericColumn.  

 


 Raises
  TypeError   if any dimension in shape is not an int.  
  ValueError   if any dimension in shape is not a positive integer.  
  ValueError   if dtype is not convertible to tf.float32.
tf.feature_column.weighted_categorical_column     View source on GitHub    Applies weight values to a CategoricalColumn.  View aliases  Compat aliases for migration 
See Migration guide for more details. tf.compat.v1.feature_column.weighted_categorical_column  
tf.feature_column.weighted_categorical_column(
    categorical_column, weight_feature_key, dtype=tf.dtypes.float32
)
 Use this when each of your sparse inputs has both an ID and a value. For example, if you're representing text documents as a collection of word frequencies, you can provide 2 parallel sparse input features ('terms' and 'frequencies' below). Example: Input tf.Example objects: [
  features {
    feature {
      key: "terms"
      value {bytes_list {value: "very" value: "model"} }
    }
    feature {
      key: "frequencies"
      value {float_list {value: 0.3 value: 0.1} }
    }
  },
  features {
    feature {
      key: "terms"
      value {bytes_list {value: "when" value: "course" value: "human"} }
    }
    feature {
      key: "frequencies"
      value {float_list {value: 0.4 value: 0.1 value: 0.2} }
    }
  }
]

categorical_column = categorical_column_with_hash_bucket(
    column_name='terms', hash_bucket_size=1000)
weighted_column = weighted_categorical_column(
    categorical_column=categorical_column, weight_feature_key='frequencies')
columns = [weighted_column, ...]
features = tf.io.parse_example(..., features=make_parse_example_spec(columns))
linear_prediction, _, _ = linear_model(features, columns)
 This assumes the input dictionary contains a SparseTensor for key 'terms', and a SparseTensor for key 'frequencies'. These 2 tensors must have the same indices and dense shape.
 


 Args
  categorical_column   A CategoricalColumn created by categorical_column_with_* functions.  
  weight_feature_key   String key for weight values.  
  dtype   Type of weights, such as tf.float32. Only float and integer weights are supported.   
 


 Returns   A CategoricalColumn composed of two sparse features: one represents id, the other represents weight (value) of the id feature in that example.  

 


 Raises
  ValueError   if dtype is not convertible to float.