tf.keras.layers.experimental.preprocessing.TextVectorization Text vectorization layer. Inherits From: PreprocessingLayer, Layer, Module
tf.keras.layers.experimental.preprocessing.TextVectorization(
    max_tokens=None, standardize=LOWER_AND_STRIP_PUNCTUATION,
    split=SPLIT_ON_WHITESPACE, ngrams=None, output_mode=INT,
    output_sequence_length=None, pad_to_max_tokens=True, vocabulary=None, **kwargs
)
 This layer has basic options for managing text in a Keras model. It transforms a batch of strings (one sample = one string) into either a list of token indices (one sample = 1D tensor of integer token indices) or a dense representation (one sample = 1D tensor of float values representing data about the sample's tokens). If desired, the user can call this layer's adapt() method on a dataset. When this layer is adapted, it will analyze the dataset, determine the frequency of individual string values, and create a 'vocabulary' from them. This vocabulary can have unlimited size or be capped, depending on the configuration options for this layer; if there are more unique values in the input than the maximum vocabulary size, the most frequent terms will be used to create the vocabulary. The processing of each sample contains the following steps:  standardize each sample (usually lowercasing + punctuation stripping) split each sample into substrings (usually words) recombine substrings into tokens (usually ngrams) index tokens (associate a unique int value with each token) transform each sample using this index, either into a vector of ints or a dense float vector.  Some notes on passing Callables to customize splitting and normalization for this layer:  Any callable can be passed to this Layer, but if you want to serialize this object you should only pass functions that are registered Keras serializables (see tf.keras.utils.register_keras_serializable for more details). When using a custom callable for standardize, the data received by the callable will be exactly as passed to this layer. The callable should return a tensor of the same shape as the input. When using a custom callable for split, the data received by the callable will have the 1st dimension squeezed out - instead of [["string to split"], ["another string to split"]], the Callable will see ["string to split", "another string to split"]. The callable should return a Tensor with the first dimension containing the split tokens - in this example, we should see something like [["string", "to", "split], ["another", "string", "to", "split"]]. This makes the callable site natively compatible with tf.strings.split().  Example: This example instantiates a TextVectorization layer that lowercases text, splits on whitespace, strips punctuation, and outputs integer vocab indices.
text_dataset = tf.data.Dataset.from_tensor_slices(["foo", "bar", "baz"])
max_features = 5000  # Maximum vocab size.
max_len = 4  # Sequence length to pad the outputs to.
embedding_dims = 2

# Create the layer.
vectorize_layer = TextVectorization(
 max_tokens=max_features,
 output_mode='int',
 output_sequence_length=max_len)

# Now that the vocab layer has been created, call `adapt` on the text-only
# dataset to create the vocabulary. You don't have to batch, but for large
# datasets this means we're not keeping spare copies of the dataset.
vectorize_layer.adapt(text_dataset.batch(64))

# Create the model that uses the vectorize text layer
model = tf.keras.models.Sequential()

# Start by creating an explicit input layer. It needs to have a shape of
# (1,) (because we need to guarantee that there is exactly one string
# input per batch), and the dtype needs to be 'string'.
model.add(tf.keras.Input(shape=(1,), dtype=tf.string))

# The first layer in our model is the vectorization layer. After this
# layer, we have a tensor of shape (batch_size, max_len) containing vocab
# indices.
model.add(vectorize_layer)

# Now, the model can map strings to integers, and you can add an embedding
# layer to map these integers to learned embeddings.
input_data = [["foo qux bar"], ["qux baz"]]
model.predict(input_data)
array([[2, 1, 4, 0],
       [1, 3, 0, 0]])
 Example: This example instantiates a TextVectorization layer by passing a list of vocabulary terms to the layer's init method. input_array = np.array([["earth", "wind", "and", "fire"], ["fire", "and", "earth", "michigan"]]) expected_output = [[2, 3, 4, 5], [5, 4, 2, 1]] input_data = keras.Input(shape=(None,), dtype=dtypes.string) layer = get_layer_class()( max_tokens=None, standardize=None, split=None, output_mode=text_vectorization.INT, vocabulary=vocab_data) int_data = layer(input_data) model = keras.Model(inputs=input_data, outputs=int_data) output_dataset = model.predict(input_array) >>> vocab_data = ["earth", "wind", "and", "fire"]
>>> max_len = 4  # Sequence length to pad the outputs to.
>>>
>>> # Create the layer, passing the vocab directly. You can also pass the
>>> # vocabulary arg a path to a file containing one vocabulary word per
>>> # line.
>>> vectorize_layer = TextVectorization(
...  max_tokens=max_features,
...  output_mode='int',
...  output_sequence_length=max_len,
...  vocabulary=vocab_data)
>>>
>>> # Because we've passed the vocabulary directly, we don't need to adapt
>>> # the layer - the vocabulary is already set. The vocabulary contains the
>>> # padding token ('') and OOV token ('[UNK]') as well as the passed tokens.
>>> vectorize_layer.get_vocabulary()
['', '[UNK]', 'earth', 'wind', 'and', 'fire']

 Attributes
  max_tokens   The maximum size of the vocabulary for this layer. If None, there is no cap on the size of the vocabulary. Note that this vocabulary contains 1 OOV token, so the effective number of tokens is (max_tokens - 1 - (1 if output == "int" else 0)).
  standardize   Optional specification for standardization to apply to the input text. Values can be None (no standardization), 'lower_and_strip_punctuation' (lowercase and remove punctuation) or a Callable. Default is 'lower_and_strip_punctuation'.
  split   Optional specification for splitting the input text. Values can be None (no splitting), 'whitespace' (split on ASCII whitespace), or a Callable. The default is 'whitespace'.
  ngrams   Optional specification for ngrams to create from the possibly-split input text. Values can be None, an integer or tuple of integers; passing an integer will create ngrams up to that integer, and passing a tuple of integers will create ngrams for the specified values in the tuple. Passing None means that no ngrams will be created.
  output_mode   Optional specification for the output of the layer. Values can be "int", "binary", "count" or "tf-idf", configuring the layer as follows: "int": Outputs integer indices, one integer index per split string token. When output == "int", 0 is reserved for masked locations; this reduces the vocab size to max_tokens-2 instead of max_tokens-1 "binary": Outputs a single int array per batch, of either vocab_size or max_tokens size, containing 1s in all elements where the token mapped to that index exists at least once in the batch item. "count": As "binary", but the int array contains a count of the number of times the token at that index appeared in the batch item. "tf-idf": As "binary", but the TF-IDF algorithm is applied to find the value in each token slot.
  output_sequence_length   Only valid in INT mode. If set, the output will have its time dimension padded or truncated to exactly output_sequence_length values, resulting in a tensor of shape [batch_size, output_sequence_length] regardless of how many tokens resulted from the splitting step. Defaults to None.
  pad_to_max_tokens   Only valid in "binary", "count", and "tf-idf" modes. If True, the output will have its feature axis padded to max_tokens even if the number of unique tokens in the vocabulary is less than max_tokens, resulting in a tensor of shape [batch_size, max_tokens] regardless of vocabulary size. Defaults to True.
  vocabulary   An optional list of vocabulary terms, or a path to a text file containing a vocabulary to load into this layer. The file should contain one token per line. If the list or file contains the same token multiple times, an error will be thrown.    Methods adapt View source
adapt(
    data, reset_state=True
)
 Fits the state of the preprocessing layer to the dataset. Overrides the default adapt method to apply relevant preprocessing to the inputs before passing to the combiner.

 Arguments
  data   The data to train on. It can be passed either as a tf.data Dataset, as a NumPy array, a string tensor, or as a list of texts.
  reset_state   Optional argument specifying whether to clear the state of the layer at the start of the call to adapt. This must be True for this layer, which does not support repeated calls to adapt.    get_vocabulary View source
get_vocabulary()
 set_vocabulary View source
set_vocabulary(
    vocab, df_data=None, oov_df_value=None
)
 Sets vocabulary (and optionally document frequency) data for this layer. This method sets the vocabulary and DF data for this layer directly, instead of analyzing a dataset through 'adapt'. It should be used whenever the vocab (and optionally document frequency) information is already known. If vocabulary data is already present in the layer, this method will replace it.

 Arguments
  vocab   An array of string tokens.
  df_data   An array of document frequency data. Only necessary if the layer output_mode is TFIDF.
  oov_df_value   The document frequency of the OOV token. Only necessary if output_mode is TFIDF.

 Raises
  ValueError   If there are too many inputs, the inputs do not match, or input data is missing.
  RuntimeError   If the vocabulary cannot be set when this function is called. This happens when "binary", "count", and "tfidf" modes, if "pad_to_max_tokens" is False and the layer itself has already been called.


tf.keras.layers.experimental.RandomFourierFeatures Layer that projects its inputs into a random feature space. Inherits From: Layer, Module  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.keras.layers.experimental.RandomFourierFeatures
tf.keras.layers.experimental.RandomFourierFeatures(
    output_dim, kernel_initializer='gaussian', scale=None,
    trainable=False, name=None, **kwargs
)
 This layer implements a mapping from input space to a space with output_dim dimensions, which approximates shift-invariant kernels. A kernel function K(x, y) is shift-invariant if K(x, y) == k(x - y) for some function k. Many popular Radial Basis Functions (RBF), including Gaussian and Laplacian kernels, are shift-invariant. The implementation of this layer is based on the following paper: "Random Features for Large-Scale Kernel Machines" by Ali Rahimi and Ben Recht. The distribution from which the parameters of the random features map (layer) are sampled determines which shift-invariant kernel the layer approximates (see paper for more details). You can use the distribution of your choice. The layer supports out-of-the-box approximation sof the following two RBF kernels:  Gaussian: K(x, y) == exp(- square(x - y) / (2 * square(scale)))
 Laplacian: K(x, y) = exp(-abs(x - y) / scale))

Note: Unlike what is described in the paper and unlike what is used in the Scikit-Learn implementation, the output of this layer does not apply the sqrt(2 / D) normalization factor.
 Usage: Typically, this layer is used to "kernelize" linear models by applying a non-linear transformation (this layer) to the input features and then training a linear model on top of the transformed features. Depending on the loss function of the linear model, the composition of this layer and the linear model results to models that are equivalent (up to approximation) to kernel SVMs (for hinge loss), kernel logistic regression (for logistic loss), kernel linear regression (for squared loss), etc. Examples: A kernel multinomial logistic regression model with Gaussian kernel for MNIST: model = keras.Sequential([
  keras.Input(shape=(784,)),
  RandomFourierFeatures(
      output_dim=4096,
      scale=10.,
      kernel_initializer='gaussian'),
  layers.Dense(units=10, activation='softmax'),
])
model.compile(
    optimizer='adam',
    loss='categorical_crossentropy',
    metrics=['categorical_accuracy']
)
 A quasi-SVM classifier for MNIST: model = keras.Sequential([
  keras.Input(shape=(784,)),
  RandomFourierFeatures(
      output_dim=4096,
      scale=10.,
      kernel_initializer='gaussian'),
  layers.Dense(units=10),
])
model.compile(
    optimizer='adam',
    loss='hinge',
    metrics=['categorical_accuracy']
)
 To use another kernel, just replace the layer creation line with: random_features_layer = RandomFourierFeatures(
    output_dim=500,
    kernel_initializer=<my_initializer>,
    scale=...,
    ...)

 Arguments
  output_dim   Positive integer, the dimension of the layer's output, i.e., the number of random features used to approximate the kernel.
  kernel_initializer   Determines the distribution of the parameters of the random features map (and therefore the kernel approximated by the layer). It can be either a string identifier or a Keras Initializer instance. Currently only 'gaussian' and 'laplacian' are supported string identifiers (case insensitive). Note that the kernel matrix is not trainable.
  scale   For Gaussian and Laplacian kernels, this corresponds to a scaling factor of the corresponding kernel approximated by the layer (see concrete definitions above). When provided, it should be a positive float. If None, a default value is used: if the kernel initializer is set to "gaussian", scale defaults to sqrt(input_dim / 2), otherwise, it defaults to 1.0. Both the approximation error of the kernel and the classification quality are sensitive to this parameter. If trainable is set to True, this parameter is learned end-to-end during training and the provided value serves as the initial value. Note: When features from this layer are fed to a linear model, by making scale trainable, the resulting optimization problem is no longer convex (even if the loss function used by the linear model is convex).
  trainable   Whether the scaling parameter of the layer should be trainable. Defaults to False.
  name   String, name to use for this layer.


tf.keras.layers.experimental.SyncBatchNormalization Normalize and scale inputs or activations synchronously across replicas. Inherits From: Layer, Module
tf.keras.layers.experimental.SyncBatchNormalization(
    axis=-1, momentum=0.99, epsilon=0.001, center=True, scale=True,
    beta_initializer='zeros', gamma_initializer='ones',
    moving_mean_initializer='zeros',
    moving_variance_initializer='ones', beta_regularizer=None,
    gamma_regularizer=None, beta_constraint=None, gamma_constraint=None,
    renorm=False, renorm_clipping=None, renorm_momentum=0.99, trainable=True,
    adjustment=None, name=None, **kwargs
)
 Applies batch normalization to activations of the previous layer at each batch by synchronizing the global batch statistics across all devices that are training the model. For specific details about batch normalization please refer to the tf.keras.layers.BatchNormalization layer docs. If this layer is used when using tf.distribute strategy to train models across devices/workers, there will be an allreduce call to aggregate batch statistics across all replicas at every training step. Without tf.distribute strategy, this layer behaves as a regular tf.keras.layers.BatchNormalization layer. Example usage: strategy = tf.distribute.MirroredStrategy()

with strategy.scope():
  model = tf.keras.Sequential()
  model.add(tf.keras.layers.Dense(16))
  model.add(tf.keras.layers.experimental.SyncBatchNormalization())

 Arguments
  axis   Integer, the axis that should be normalized (typically the features axis). For instance, after a Conv2D layer with data_format="channels_first", set axis=1 in BatchNormalization.
  momentum   Momentum for the moving average.
  epsilon   Small float added to variance to avoid dividing by zero.
  center   If True, add offset of beta to normalized tensor. If False, beta is ignored.
  scale   If True, multiply by gamma. If False, gamma is not used. When the next layer is linear (also e.g. nn.relu), this can be disabled since the scaling will be done by the next layer.
  beta_initializer   Initializer for the beta weight.
  gamma_initializer   Initializer for the gamma weight.
  moving_mean_initializer   Initializer for the moving mean.
  moving_variance_initializer   Initializer for the moving variance.
  beta_regularizer   Optional regularizer for the beta weight.
  gamma_regularizer   Optional regularizer for the gamma weight.
  beta_constraint   Optional constraint for the beta weight.
  gamma_constraint   Optional constraint for the gamma weight.
  renorm   Whether to use Batch Renormalization. This adds extra variables during training. The inference is the same for either value of this parameter.
  renorm_clipping   A dictionary that may map keys 'rmax', 'rmin', 'dmax' to scalar Tensors used to clip the renorm correction. The correction (r, d) is used as corrected_value = normalized_value * r + d, with r clipped to [rmin, rmax], and d to [-dmax, dmax]. Missing rmax, rmin, dmax are set to inf, 0, inf, respectively.
  renorm_momentum   Momentum used to update the moving means and standard deviations with renorm. Unlike momentum, this affects training and should be neither too small (which would add noise) nor too large (which would give stale estimates). Note that momentum is still applied to get the means and variances for inference.
  trainable   Boolean, if True the variables will be marked as trainable.    Call arguments:
inputs: Input tensor (of any rank).
training: Python boolean indicating whether the layer should behave in training mode or in inference mode.
training=True: The layer will normalize its inputs using the mean and variance of the current batch of inputs.
training=False: The layer will normalize its inputs using the mean and variance of its moving statistics, learned during training.
  Input shape: Arbitrary. Use the keyword argument input_shape (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model. Output shape: Same shape as input.


tf.keras.layers.Flatten     View source on GitHub    Flattens the input. Does not affect the batch size. Inherits From: Layer, Module  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.keras.layers.Flatten
tf.keras.layers.Flatten(
    data_format=None, **kwargs
)

Note: If inputs are shaped (batch,) without a feature axis, then flattening adds an extra channel dimension and output shape is (batch, 1).

 Arguments
  data_format   A string, one of channels_last (default) or channels_first. The ordering of the dimensions in the inputs. channels_last corresponds to inputs with shape (batch, ..., channels) while channels_first corresponds to inputs with shape (batch, channels, ...). It defaults to the image_data_format value found in your Keras config file at ~/.keras/keras.json. If you never set it, then it will be "channels_last".    Example:
model = tf.keras.Sequential()
model.add(tf.keras.layers.Conv2D(64, 3, 3, input_shape=(3, 32, 32)))
model.output_shape
(None, 1, 10, 64)

model.add(Flatten())
model.output_shape
(None, 640)


tf.keras.layers.GaussianDropout     View source on GitHub    Apply multiplicative 1-centered Gaussian noise. Inherits From: Layer, Module  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.keras.layers.GaussianDropout
tf.keras.layers.GaussianDropout(
    rate, **kwargs
)
 As it is a regularization layer, it is only active at training time.

 Arguments
  rate   Float, drop probability (as with Dropout). The multiplicative noise will have standard deviation sqrt(rate / (1 - rate)).    Call arguments:
inputs: Input tensor (of any rank).
training: Python boolean indicating whether the layer should behave in training mode (adding dropout) or in inference mode (doing nothing).  Input shape: Arbitrary. Use the keyword argument input_shape (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model. Output shape: Same shape as input.


tf.keras.layers.GaussianNoise     View source on GitHub    Apply additive zero-centered Gaussian noise. Inherits From: Layer, Module  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.keras.layers.GaussianNoise
tf.keras.layers.GaussianNoise(
    stddev, **kwargs
)
 This is useful to mitigate overfitting (you could see it as a form of random data augmentation). Gaussian Noise (GS) is a natural choice as corruption process for real valued inputs. As it is a regularization layer, it is only active at training time.

 Arguments
  stddev   Float, standard deviation of the noise distribution.    Call arguments:
inputs: Input tensor (of any rank).
training: Python boolean indicating whether the layer should behave in training mode (adding noise) or in inference mode (doing nothing).  Input shape: Arbitrary. Use the keyword argument input_shape (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model. Output shape: Same shape as input.


tf.keras.layers.GlobalAveragePooling1D     View source on GitHub    Global average pooling operation for temporal data. Inherits From: Layer, Module  View aliases  Main aliases
tf.keras.layers.GlobalAvgPool1D Compat aliases for migration See Migration guide for more details. tf.compat.v1.keras.layers.GlobalAveragePooling1D, tf.compat.v1.keras.layers.GlobalAvgPool1D
tf.keras.layers.GlobalAveragePooling1D(
    data_format='channels_last', **kwargs
)
 Examples:
input_shape = (2, 3, 4)
x = tf.random.normal(input_shape)
y = tf.keras.layers.GlobalAveragePooling1D()(x)
print(y.shape)
(2, 4)

 Arguments
  data_format   A string, one of channels_last (default) or channels_first. The ordering of the dimensions in the inputs. channels_last corresponds to inputs with shape (batch, steps, features) while channels_first corresponds to inputs with shape (batch, features, steps).    Call arguments:
inputs: A 3D tensor.
mask: Binary tensor of shape (batch_size, steps) indicating whether a given step should be masked (excluded from the average).  Input shape:  If data_format='channels_last': 3D tensor with shape: (batch_size, steps, features)
 If data_format='channels_first': 3D tensor with shape: (batch_size, features, steps)
  Output shape: 2D tensor with shape (batch_size, features).


tf.keras.layers.GlobalAveragePooling2D     View source on GitHub    Global average pooling operation for spatial data. Inherits From: Layer, Module  View aliases  Main aliases
tf.keras.layers.GlobalAvgPool2D Compat aliases for migration See Migration guide for more details. tf.compat.v1.keras.layers.GlobalAveragePooling2D, tf.compat.v1.keras.layers.GlobalAvgPool2D
tf.keras.layers.GlobalAveragePooling2D(
    data_format=None, **kwargs
)
 Examples:
input_shape = (2, 4, 5, 3)
x = tf.random.normal(input_shape)
y = tf.keras.layers.GlobalAveragePooling2D()(x)
print(y.shape)
(2, 3)

 Arguments
  data_format   A string, one of channels_last (default) or channels_first. The ordering of the dimensions in the inputs. channels_last corresponds to inputs with shape (batch, height, width, channels) while channels_first corresponds to inputs with shape (batch, channels, height, width). It defaults to the image_data_format value found in your Keras config file at ~/.keras/keras.json. If you never set it, then it will be "channels_last".    Input shape:  If data_format='channels_last': 4D tensor with shape (batch_size, rows, cols, channels). If data_format='channels_first': 4D tensor with shape (batch_size, channels, rows, cols).  Output shape: 2D tensor with shape (batch_size, channels).


tf.keras.layers.GlobalAveragePooling3D     View source on GitHub    Global Average pooling operation for 3D data. Inherits From: Layer, Module  View aliases  Main aliases
tf.keras.layers.GlobalAvgPool3D Compat aliases for migration See Migration guide for more details. tf.compat.v1.keras.layers.GlobalAveragePooling3D, tf.compat.v1.keras.layers.GlobalAvgPool3D
tf.keras.layers.GlobalAveragePooling3D(
    data_format=None, **kwargs
)

 Arguments
  data_format   A string, one of channels_last (default) or channels_first. The ordering of the dimensions in the inputs. channels_last corresponds to inputs with shape (batch, spatial_dim1, spatial_dim2, spatial_dim3, channels) while channels_first corresponds to inputs with shape (batch, channels, spatial_dim1, spatial_dim2, spatial_dim3). It defaults to the image_data_format value found in your Keras config file at ~/.keras/keras.json. If you never set it, then it will be "channels_last".    Input shape:  If data_format='channels_last': 5D tensor with shape: (batch_size, spatial_dim1, spatial_dim2, spatial_dim3, channels)
 If data_format='channels_first': 5D tensor with shape: (batch_size, channels, spatial_dim1, spatial_dim2, spatial_dim3)
  Output shape: 2D tensor with shape (batch_size, channels).


tf.keras.layers.GlobalMaxPool1D     View source on GitHub    Global max pooling operation for 1D temporal data. Inherits From: Layer, Module  View aliases  Main aliases
tf.keras.layers.GlobalMaxPooling1D Compat aliases for migration See Migration guide for more details. tf.compat.v1.keras.layers.GlobalMaxPool1D, tf.compat.v1.keras.layers.GlobalMaxPooling1D
tf.keras.layers.GlobalMaxPool1D(
    data_format='channels_last', **kwargs
)
 Downsamples the input representation by taking the maximum value over the time dimension. For example:
x = tf.constant([[1., 2., 3.], [4., 5., 6.], [7., 8., 9.]])
x = tf.reshape(x, [3, 3, 1])
x
<tf.Tensor: shape=(3, 3, 1), dtype=float32, numpy=
array([[[1.], [2.], [3.]],
       [[4.], [5.], [6.]],
       [[7.], [8.], [9.]]], dtype=float32)>
max_pool_1d = tf.keras.layers.GlobalMaxPooling1D()
max_pool_1d(x)
<tf.Tensor: shape=(3, 1), dtype=float32, numpy=
array([[3.],
       [6.],
       [9.], dtype=float32)>

 Arguments
  data_format   A string, one of channels_last (default) or channels_first. The ordering of the dimensions in the inputs. channels_last corresponds to inputs with shape (batch, steps, features) while channels_first corresponds to inputs with shape (batch, features, steps).    Input shape:  If data_format='channels_last': 3D tensor with shape: (batch_size, steps, features)
 If data_format='channels_first': 3D tensor with shape: (batch_size, features, steps)
  Output shape: 2D tensor with shape (batch_size, features).


tf.keras.layers.GlobalMaxPool2D     View source on GitHub    Global max pooling operation for spatial data. Inherits From: Layer, Module  View aliases  Main aliases
tf.keras.layers.GlobalMaxPooling2D Compat aliases for migration See Migration guide for more details. tf.compat.v1.keras.layers.GlobalMaxPool2D, tf.compat.v1.keras.layers.GlobalMaxPooling2D
tf.keras.layers.GlobalMaxPool2D(
    data_format=None, **kwargs
)
 Examples:
input_shape = (2, 4, 5, 3)
x = tf.random.normal(input_shape)
y = tf.keras.layers.GlobalMaxPool2D()(x)
print(y.shape)
(2, 3)

 Arguments
  data_format   A string, one of channels_last (default) or channels_first. The ordering of the dimensions in the inputs. channels_last corresponds to inputs with shape (batch, height, width, channels) while channels_first corresponds to inputs with shape (batch, channels, height, width). It defaults to the image_data_format value found in your Keras config file at ~/.keras/keras.json. If you never set it, then it will be "channels_last".    Input shape:  If data_format='channels_last': 4D tensor with shape (batch_size, rows, cols, channels). If data_format='channels_first': 4D tensor with shape (batch_size, channels, rows, cols).  Output shape: 2D tensor with shape (batch_size, channels).


tf.keras.layers.GlobalMaxPool3D     View source on GitHub    Global Max pooling operation for 3D data. Inherits From: Layer, Module  View aliases  Main aliases
tf.keras.layers.GlobalMaxPooling3D Compat aliases for migration See Migration guide for more details. tf.compat.v1.keras.layers.GlobalMaxPool3D, tf.compat.v1.keras.layers.GlobalMaxPooling3D
tf.keras.layers.GlobalMaxPool3D(
    data_format=None, **kwargs
)

 Arguments
  data_format   A string, one of channels_last (default) or channels_first. The ordering of the dimensions in the inputs. channels_last corresponds to inputs with shape (batch, spatial_dim1, spatial_dim2, spatial_dim3, channels) while channels_first corresponds to inputs with shape (batch, channels, spatial_dim1, spatial_dim2, spatial_dim3). It defaults to the image_data_format value found in your Keras config file at ~/.keras/keras.json. If you never set it, then it will be "channels_last".    Input shape:  If data_format='channels_last': 5D tensor with shape: (batch_size, spatial_dim1, spatial_dim2, spatial_dim3, channels)
 If data_format='channels_first': 5D tensor with shape: (batch_size, channels, spatial_dim1, spatial_dim2, spatial_dim3)
  Output shape: 2D tensor with shape (batch_size, channels).


tf.keras.layers.GRU     View source on GitHub    Gated Recurrent Unit - Cho et al. 2014. Inherits From: GRU, RNN, Layer, Module
tf.keras.layers.GRU(
    units, activation='tanh', recurrent_activation='sigmoid',
    use_bias=True, kernel_initializer='glorot_uniform',
    recurrent_initializer='orthogonal',
    bias_initializer='zeros', kernel_regularizer=None,
    recurrent_regularizer=None, bias_regularizer=None, activity_regularizer=None,
    kernel_constraint=None, recurrent_constraint=None, bias_constraint=None,
    dropout=0.0, recurrent_dropout=0.0, return_sequences=False, return_state=False,
    go_backwards=False, stateful=False, unroll=False, time_major=False,
    reset_after=True, **kwargs
)
 See the Keras RNN API guide for details about the usage of RNN API. Based on available runtime hardware and constraints, this layer will choose different implementations (cuDNN-based or pure-TensorFlow) to maximize the performance. If a GPU is available and all the arguments to the layer meet the requirement of the CuDNN kernel (see below for details), the layer will use a fast cuDNN implementation. The requirements to use the cuDNN implementation are:
activation == tanh

recurrent_activation == sigmoid

recurrent_dropout == 0
unroll is False

use_bias is True

reset_after is True
 Inputs, if use masking, are strictly right-padded. Eager execution is enabled in the outermost context.  There are two variants of the GRU implementation. The default one is based on v3 and has reset gate applied to hidden state before matrix multiplication. The other one is based on original and has the order reversed. The second variant is compatible with CuDNNGRU (GPU-only) and allows inference on CPU. Thus it has separate biases for kernel and recurrent_kernel. To use this variant, set 'reset_after'=True and recurrent_activation='sigmoid'. For example:
inputs = tf.random.normal([32, 10, 8])
gru = tf.keras.layers.GRU(4)
output = gru(inputs)
print(output.shape)
(32, 4)
gru = tf.keras.layers.GRU(4, return_sequences=True, return_state=True)
whole_sequence_output, final_state = gru(inputs)
print(whole_sequence_output.shape)
(32, 10, 4)
print(final_state.shape)
(32, 4)

 Arguments
  units   Positive integer, dimensionality of the output space.
  activation   Activation function to use. Default: hyperbolic tangent (tanh). If you pass None, no activation is applied (ie. "linear" activation: a(x) = x).
  recurrent_activation   Activation function to use for the recurrent step. Default: sigmoid (sigmoid). If you pass None, no activation is applied (ie. "linear" activation: a(x) = x).
  use_bias   Boolean, (default True), whether the layer uses a bias vector.
  kernel_initializer   Initializer for the kernel weights matrix, used for the linear transformation of the inputs. Default: glorot_uniform.
  recurrent_initializer   Initializer for the recurrent_kernel weights matrix, used for the linear transformation of the recurrent state. Default: orthogonal.
  bias_initializer   Initializer for the bias vector. Default: zeros.
  kernel_regularizer   Regularizer function applied to the kernel weights matrix. Default: None.
  recurrent_regularizer   Regularizer function applied to the recurrent_kernel weights matrix. Default: None.
  bias_regularizer   Regularizer function applied to the bias vector. Default: None.
  activity_regularizer   Regularizer function applied to the output of the layer (its "activation"). Default: None.
  kernel_constraint   Constraint function applied to the kernel weights matrix. Default: None.
  recurrent_constraint   Constraint function applied to the recurrent_kernel weights matrix. Default: None.
  bias_constraint   Constraint function applied to the bias vector. Default: None.
  dropout   Float between 0 and 1. Fraction of the units to drop for the linear transformation of the inputs. Default: 0.
  recurrent_dropout   Float between 0 and 1. Fraction of the units to drop for the linear transformation of the recurrent state. Default: 0.
  return_sequences   Boolean. Whether to return the last output in the output sequence, or the full sequence. Default: False.
  return_state   Boolean. Whether to return the last state in addition to the output. Default: False.
  go_backwards   Boolean (default False). If True, process the input sequence backwards and return the reversed sequence.
  stateful   Boolean (default False). If True, the last state for each sample at index i in a batch will be used as initial state for the sample of index i in the following batch.
  unroll   Boolean (default False). If True, the network will be unrolled, else a symbolic loop will be used. Unrolling can speed-up a RNN, although it tends to be more memory-intensive. Unrolling is only suitable for short sequences.
  time_major   The shape format of the inputs and outputs tensors. If True, the inputs and outputs will be in shape [timesteps, batch, feature], whereas in the False case, it will be [batch, timesteps, feature]. Using time_major = True is a bit more efficient because it avoids transposes at the beginning and end of the RNN calculation. However, most TensorFlow data is batch-major, so by default this function accepts input and emits output in batch-major form.
  reset_after   GRU convention (whether to apply reset gate after or before matrix multiplication). False = "before", True = "after" (default and CuDNN compatible).    Call arguments:
inputs: A 3D tensor, with shape [batch, timesteps, feature].
mask: Binary tensor of shape [samples, timesteps] indicating whether a given timestep should be masked (optional, defaults to None).
training: Python boolean indicating whether the layer should behave in training mode or in inference mode. This argument is passed to the cell when calling it. This is only relevant if dropout or recurrent_dropout is used (optional, defaults to None).
initial_state: List of initial state tensors to be passed to the first call of the cell (optional, defaults to None which causes creation of zero-filled initial state tensors).

 Attributes
  activation

  bias_constraint

  bias_initializer

  bias_regularizer

  dropout

  implementation

  kernel_constraint

  kernel_initializer

  kernel_regularizer

  recurrent_activation

  recurrent_constraint

  recurrent_dropout

  recurrent_initializer

  recurrent_regularizer

  reset_after

  states

  units

  use_bias
   Methods get_dropout_mask_for_cell View source
get_dropout_mask_for_cell(
    inputs, training, count=1
)
 Get the dropout mask for RNN cell's input. It will create mask based on context if there isn't any existing cached mask. If a new mask is generated, it will update the cache in the cell.

 Args
  inputs   The input tensor whose shape will be used to generate dropout mask.
  training   Boolean tensor, whether its in training mode, dropout will be ignored in non-training mode.
  count   Int, how many dropout mask will be generated. It is useful for cell that has internal weights fused together.

 Returns   List of mask tensor, generated or cached mask based on context.
 get_recurrent_dropout_mask_for_cell View source
get_recurrent_dropout_mask_for_cell(
    inputs, training, count=1
)
 Get the recurrent dropout mask for RNN cell. It will create mask based on context if there isn't any existing cached mask. If a new mask is generated, it will update the cache in the cell.

 Args
  inputs   The input tensor whose shape will be used to generate dropout mask.
  training   Boolean tensor, whether its in training mode, dropout will be ignored in non-training mode.
  count   Int, how many dropout mask will be generated. It is useful for cell that has internal weights fused together.

 Returns   List of mask tensor, generated or cached mask based on context.
 reset_dropout_mask View source
reset_dropout_mask()
 Reset the cached dropout masks if any. This is important for the RNN layer to invoke this in it call() method so that the cached mask is cleared before calling the cell.call(). The mask should be cached across the timestep within the same batch, but shouldn't be cached between batches. Otherwise it will introduce unreasonable bias against certain index of data within the batch. reset_recurrent_dropout_mask View source
reset_recurrent_dropout_mask()
 Reset the cached recurrent dropout masks if any. This is important for the RNN layer to invoke this in it call() method so that the cached mask is cleared before calling the cell.call(). The mask should be cached across the timestep within the same batch, but shouldn't be cached between batches. Otherwise it will introduce unreasonable bias against certain index of data within the batch. reset_states View source
reset_states(
    states=None
)
 Reset the recorded states for the stateful RNN layer. Can only be used when RNN layer is constructed with stateful = True. Args: states: Numpy arrays that contains the value for the initial state, which will be feed to cell at the first time step. When the value is None, zero filled numpy array will be created based on the cell state size.

 Raises
  AttributeError   When the RNN layer is not stateful.
  ValueError   When the batch size of the RNN layer is unknown.
  ValueError   When the input numpy array is not compatible with the RNN layer state, either size wise or dtype wise.


tf.keras.layers.GRUCell     View source on GitHub    Cell class for the GRU layer. Inherits From: GRUCell, Layer, Module
tf.keras.layers.GRUCell(
    units, activation='tanh', recurrent_activation='sigmoid',
    use_bias=True, kernel_initializer='glorot_uniform',
    recurrent_initializer='orthogonal',
    bias_initializer='zeros', kernel_regularizer=None,
    recurrent_regularizer=None, bias_regularizer=None, kernel_constraint=None,
    recurrent_constraint=None, bias_constraint=None, dropout=0.0,
    recurrent_dropout=0.0, reset_after=True, **kwargs
)
 See the Keras RNN API guide for details about the usage of RNN API. This class processes one step within the whole time sequence input, whereas tf.keras.layer.GRU processes the whole sequence. For example:
inputs = tf.random.normal([32, 10, 8])
rnn = tf.keras.layers.RNN(tf.keras.layers.GRUCell(4))
output = rnn(inputs)
print(output.shape)
(32, 4)
rnn = tf.keras.layers.RNN(
   tf.keras.layers.GRUCell(4),
   return_sequences=True,
   return_state=True)
whole_sequence_output, final_state = rnn(inputs)
print(whole_sequence_output.shape)
(32, 10, 4)
print(final_state.shape)
(32, 4)

 Arguments
  units   Positive integer, dimensionality of the output space.
  activation   Activation function to use. Default: hyperbolic tangent (tanh). If you pass None, no activation is applied (ie. "linear" activation: a(x) = x).
  recurrent_activation   Activation function to use for the recurrent step. Default: sigmoid (sigmoid). If you pass None, no activation is applied (ie. "linear" activation: a(x) = x).
  use_bias   Boolean, (default True), whether the layer uses a bias vector.
  kernel_initializer   Initializer for the kernel weights matrix, used for the linear transformation of the inputs. Default: glorot_uniform.
  recurrent_initializer   Initializer for the recurrent_kernel weights matrix, used for the linear transformation of the recurrent state. Default: orthogonal.
  bias_initializer   Initializer for the bias vector. Default: zeros.
  kernel_regularizer   Regularizer function applied to the kernel weights matrix. Default: None.
  recurrent_regularizer   Regularizer function applied to the recurrent_kernel weights matrix. Default: None.
  bias_regularizer   Regularizer function applied to the bias vector. Default: None.
  kernel_constraint   Constraint function applied to the kernel weights matrix. Default: None.
  recurrent_constraint   Constraint function applied to the recurrent_kernel weights matrix. Default: None.
  bias_constraint   Constraint function applied to the bias vector. Default: None.
  dropout   Float between 0 and 1. Fraction of the units to drop for the linear transformation of the inputs. Default: 0.
  recurrent_dropout   Float between 0 and 1. Fraction of the units to drop for the linear transformation of the recurrent state. Default: 0.
  reset_after   GRU convention (whether to apply reset gate after or before matrix multiplication). False = "before", True = "after" (default and CuDNN compatible).    Call arguments:
inputs: A 2D tensor, with shape of [batch, feature].
states: A 2D tensor with shape of [batch, units], which is the state from the previous time step. For timestep 0, the initial state provided by user will be feed to cell.
training: Python boolean indicating whether the layer should behave in training mode or in inference mode. Only relevant when dropout or recurrent_dropout is used.  Methods get_dropout_mask_for_cell View source
get_dropout_mask_for_cell(
    inputs, training, count=1
)
 Get the dropout mask for RNN cell's input. It will create mask based on context if there isn't any existing cached mask. If a new mask is generated, it will update the cache in the cell.

 Args
  inputs   The input tensor whose shape will be used to generate dropout mask.
  training   Boolean tensor, whether its in training mode, dropout will be ignored in non-training mode.
  count   Int, how many dropout mask will be generated. It is useful for cell that has internal weights fused together.

 Returns   List of mask tensor, generated or cached mask based on context.
 get_initial_state View source
get_initial_state(
    inputs=None, batch_size=None, dtype=None
)
 get_recurrent_dropout_mask_for_cell View source
get_recurrent_dropout_mask_for_cell(
    inputs, training, count=1
)
 Get the recurrent dropout mask for RNN cell. It will create mask based on context if there isn't any existing cached mask. If a new mask is generated, it will update the cache in the cell.

 Args
  inputs   The input tensor whose shape will be used to generate dropout mask.
  training   Boolean tensor, whether its in training mode, dropout will be ignored in non-training mode.
  count   Int, how many dropout mask will be generated. It is useful for cell that has internal weights fused together.

 Returns   List of mask tensor, generated or cached mask based on context.
 reset_dropout_mask View source
reset_dropout_mask()
 Reset the cached dropout masks if any. This is important for the RNN layer to invoke this in it call() method so that the cached mask is cleared before calling the cell.call(). The mask should be cached across the timestep within the same batch, but shouldn't be cached between batches. Otherwise it will introduce unreasonable bias against certain index of data within the batch. reset_recurrent_dropout_mask View source
reset_recurrent_dropout_mask()
 Reset the cached recurrent dropout masks if any. This is important for the RNN layer to invoke this in it call() method so that the cached mask is cleared before calling the cell.call(). The mask should be cached across the timestep within the same batch, but shouldn't be cached between batches. Otherwise it will introduce unreasonable bias against certain index of data within the batch.


tf.keras.layers.InputLayer     View source on GitHub    Layer to be used as an entry point into a Network (a graph of layers). Inherits From: Layer, Module  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.keras.layers.InputLayer
tf.keras.layers.InputLayer(
    input_shape=None, batch_size=None, dtype=None, input_tensor=None, sparse=False,
    name=None, ragged=False, **kwargs
)
 It can either wrap an existing tensor (pass an input_tensor argument) or create a placeholder tensor (pass arguments input_shape, and optionally, dtype). It is generally recommend to use the functional layer API via Input, (which creates an InputLayer) without directly using InputLayer. When using InputLayer with Keras Sequential model, it can be skipped by moving the input_shape parameter to the first layer after the InputLayer. This class can create placeholders for tf.Tensors, tf.SparseTensors, and tf.RaggedTensors by choosing 'sparse=True' or 'ragged=True'. Note that 'sparse' and 'ragged' can't be configured to True at same time. Usage: # With explicit InputLayer.
model = tf.keras.Sequential([
  tf.keras.layers.InputLayer(input_shape=(4,)),
  tf.keras.layers.Dense(8)])
model.compile(tf.optimizers.RMSprop(0.001), loss='mse')
model.fit(np.zeros((10, 4)),
          np.ones((10, 8)))

# Without InputLayer and let the first layer to have the input_shape.
# Keras will add a input for the model behind the scene.
model = tf.keras.Sequential([
  tf.keras.layers.Dense(8, input_shape=(4,))])
model.compile(tf.optimizers.RMSprop(0.001), loss='mse')
model.fit(np.zeros((10, 4)),
          np.ones((10, 8)))

 Arguments
  input_shape   Shape tuple (not including the batch axis), or TensorShape instance (not including the batch axis).
  batch_size   Optional input batch size (integer or None).
  dtype   Optional datatype of the input. When not provided, the Keras default float type will be used.
  input_tensor   Optional tensor to use as layer input. If set, the layer will use the tf.TypeSpec of this tensor rather than creating a new placeholder tensor.
  sparse   Boolean, whether the placeholder created is meant to be sparse. Default to False.
  ragged   Boolean, whether the placeholder created is meant to be ragged. In this case, values of 'None' in the 'shape' argument represent ragged dimensions. For more information about RaggedTensors, see this guide. Default to False.
  name   Optional name of the layer (string).


tf.keras.layers.InputSpec Specifies the rank, dtype and shape of every input to a layer.  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.keras.layers.InputSpec, tf.compat.v1.layers.InputSpec
tf.keras.layers.InputSpec(
    dtype=None, shape=None, ndim=None, max_ndim=None, min_ndim=None, axes=None,
    allow_last_axis_squeeze=False, name=None
)
 Layers can expose (if appropriate) an input_spec attribute: an instance of InputSpec, or a nested structure of InputSpec instances (one per input tensor). These objects enable the layer to run input compatibility checks for input structure, input rank, input shape, and input dtype. A None entry in a shape is compatible with any dimension, a None shape is compatible with any shape.

 Arguments
  dtype   Expected DataType of the input.
  shape   Shape tuple, expected shape of the input (may include None for unchecked axes). Includes the batch size.
  ndim   Integer, expected rank of the input.
  max_ndim   Integer, maximum rank of the input.
  min_ndim   Integer, minimum rank of the input.
  axes   Dictionary mapping integer axes to a specific dimension value.
  allow_last_axis_squeeze   If True, then allow inputs of rank N+1 as long as the last axis of the input is 1, as well as inputs of rank N-1 as long as the last axis of the spec is 1.
  name   Expected key corresponding to this input when passing data as a dictionary.    Example: class MyLayer(Layer):
    def __init__(self):
        super(MyLayer, self).__init__()
        # The layer will accept inputs with shape (?, 28, 28) & (?, 28, 28, 1)
        # and raise an appropriate error message otherwise.
        self.input_spec = InputSpec(
            shape=(None, 28, 28, 1),
            allow_last_axis_squeeze=True)
 Methods from_config View source
@classmethod
from_config(
    config
)
 get_config View source
get_config()


tf.keras.layers.Lambda     View source on GitHub    Wraps arbitrary expressions as a Layer object. Inherits From: Layer, Module  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.keras.layers.Lambda
tf.keras.layers.Lambda(
    function, output_shape=None, mask=None, arguments=None, **kwargs
)
 The Lambda layer exists so that arbitrary TensorFlow functions can be used when constructing Sequential and Functional API models. Lambda layers are best suited for simple operations or quick experimentation. For more advanced use cases, follow this guide for subclassing tf.keras.layers.Layer. The main reason to subclass tf.keras.layers.Layer instead of using a Lambda layer is saving and inspecting a Model. Lambda layers are saved by serializing the Python bytecode, which is fundamentally non-portable. They should only be loaded in the same environment where they were saved. Subclassed layers can be saved in a more portable way by overriding their get_config method. Models that rely on subclassed Layers are also often easier to visualize and reason about. Examples: # add a x -> x^2 layer
model.add(Lambda(lambda x: x ** 2))

# add a layer that returns the concatenation
# of the positive part of the input and
# the opposite of the negative part

def antirectifier(x):
    x -= K.mean(x, axis=1, keepdims=True)
    x = K.l2_normalize(x, axis=1)
    pos = K.relu(x)
    neg = K.relu(-x)
    return K.concatenate([pos, neg], axis=1)

model.add(Lambda(antirectifier))
 Variables: While it is possible to use Variables with Lambda layers, this practice is discouraged as it can easily lead to bugs. For instance, consider the following layer: scale = tf.Variable(1.)
scale_layer = tf.keras.layers.Lambda(lambda x: x * scale)
 Because scale_layer does not directly track the scale variable, it will not appear in scale_layer.trainable_weights and will therefore not be trained if scale_layer is used in a Model. A better pattern is to write a subclassed Layer: class ScaleLayer(tf.keras.layers.Layer):
  def __init__(self):
    super(ScaleLayer, self).__init__()
    self.scale = tf.Variable(1.)

  def call(self, inputs):
    return inputs * self.scale
 In general, Lambda layers can be convenient for simple stateless computation, but anything more complex should use a subclass Layer instead.

 Arguments
  function   The function to be evaluated. Takes input tensor as first argument.
  output_shape   Expected output shape from function. This argument can be inferred if not explicitly provided. Can be a tuple or function. If a tuple, it only specifies the first dimension onward; sample dimension is assumed either the same as the input: output_shape = (input_shape[0], ) + output_shape or, the input is None and the sample dimension is also None: output_shape = (None, ) + output_shape If a function, it specifies the entire shape as a function of the input shape: output_shape = f(input_shape)
  mask   Either None (indicating no masking) or a callable with the same signature as the compute_mask layer method, or a tensor that will be returned as output mask regardless of what the input is.
  arguments   Optional dictionary of keyword arguments to be passed to the function.    Input shape: Arbitrary. Use the keyword argument input_shape (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model. Output shape: Specified by output_shape argument


tf.keras.layers.Layer     View source on GitHub    This is the class from which all layers inherit. Inherits From: Module  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.keras.layers.Layer
tf.keras.layers.Layer(
    trainable=True, name=None, dtype=None, dynamic=False, **kwargs
)
 A layer is a callable object that takes as input one or more tensors and that outputs one or more tensors. It involves computation, defined in the call() method, and a state (weight variables), defined either in the constructor __init__() or in the build() method. Users will just instantiate a layer and then treat it as a callable.

 Arguments
  trainable   Boolean, whether the layer's variables should be trainable.
  name   String name of the layer.
  dtype   The dtype of the layer's computations and weights. Can also be a tf.keras.mixed_precision.Policy, which allows the computation and weight dtype to differ. Default of None means to use tf.keras.mixed_precision.global_policy(), which is a float32 policy unless set to different value.
  dynamic   Set this to True if your layer should only be run eagerly, and should not be used to generate a static computation graph. This would be the case for a Tree-RNN or a recursive network, for example, or generally for any layer that manipulates tensors using Python control flow. If False, we assume that the layer can safely be used to generate a static computation graph.    We recommend that descendants of Layer implement the following methods:
__init__(): Defines custom layer attributes, and creates layer state variables that do not depend on input shapes, using add_weight().
build(self, input_shape): This method can be used to create weights that depend on the shape(s) of the input(s), using add_weight(). __call__() will automatically build the layer (if it has not been built yet) by calling build().
call(self, *args, **kwargs): Called in __call__ after making sure build() has been called. call() performs the logic of applying the layer to the input tensors (which should be passed in as argument). Two reserved keyword arguments you can optionally use in call() are:
training (boolean, whether the call is in inference mode or training mode)
mask (boolean tensor encoding masked timesteps in the input, used in RNN layers)

get_config(self): Returns a dictionary containing the configuration used to initialize this layer. If the keys differ from the arguments in __init__, then override from_config(self) as well. This method is used when saving the layer or a model that contains this layer.  Examples: Here's a basic example: a layer with two variables, w and b, that returns y = w . x + b. It shows how to implement build() and call(). Variables set as attributes of a layer are tracked as weights of the layers (in layer.weights). class SimpleDense(Layer):

  def __init__(self, units=32):
      super(SimpleDense, self).__init__()
      self.units = units

  def build(self, input_shape):  # Create the state of the layer (weights)
    w_init = tf.random_normal_initializer()
    self.w = tf.Variable(
        initial_value=w_init(shape=(input_shape[-1], self.units),
                             dtype='float32'),
        trainable=True)
    b_init = tf.zeros_initializer()
    self.b = tf.Variable(
        initial_value=b_init(shape=(self.units,), dtype='float32'),
        trainable=True)

  def call(self, inputs):  # Defines the computation from inputs to outputs
      return tf.matmul(inputs, self.w) + self.b

# Instantiates the layer.
linear_layer = SimpleDense(4)

# This will also call `build(input_shape)` and create the weights.
y = linear_layer(tf.ones((2, 2)))
assert len(linear_layer.weights) == 2

# These weights are trainable, so they're listed in `trainable_weights`:
assert len(linear_layer.trainable_weights) == 2
 Note that the method add_weight() offers a shortcut to create weights: class SimpleDense(Layer):

  def __init__(self, units=32):
      super(SimpleDense, self).__init__()
      self.units = units

  def build(self, input_shape):
      self.w = self.add_weight(shape=(input_shape[-1], self.units),
                               initializer='random_normal',
                               trainable=True)
      self.b = self.add_weight(shape=(self.units,),
                               initializer='random_normal',
                               trainable=True)

  def call(self, inputs):
      return tf.matmul(inputs, self.w) + self.b
 Besides trainable weights, updated via backpropagation during training, layers can also have non-trainable weights. These weights are meant to be updated manually during call(). Here's a example layer that computes the running sum of its inputs: class ComputeSum(Layer):

  def __init__(self, input_dim):
      super(ComputeSum, self).__init__()
      # Create a non-trainable weight.
      self.total = tf.Variable(initial_value=tf.zeros((input_dim,)),
                               trainable=False)

  def call(self, inputs):
      self.total.assign_add(tf.reduce_sum(inputs, axis=0))
      return self.total

my_sum = ComputeSum(2)
x = tf.ones((2, 2))

y = my_sum(x)
print(y.numpy())  # [2. 2.]

y = my_sum(x)
print(y.numpy())  # [4. 4.]

assert my_sum.weights == [my_sum.total]
assert my_sum.non_trainable_weights == [my_sum.total]
assert my_sum.trainable_weights == []
 For more information about creating layers, see the guide Writing custom layers and models with Keras

 Attributes
  name   The name of the layer (string).
  dtype   The dtype of the layer's weights.
  variable_dtype   Alias of dtype.
  compute_dtype   The dtype of the layer's computations. Layers automatically cast inputs to this dtype which causes the computations and output to also be in this dtype. When mixed precision is used with a tf.keras.mixed_precision.Policy, this will be different than variable_dtype.
  dtype_policy   The layer's dtype policy. See the tf.keras.mixed_precision.Policy documentation for details.
  trainable_weights   List of variables to be included in backprop.
  non_trainable_weights   List of variables that should not be included in backprop.
  weights   The concatenation of the lists trainable_weights and non_trainable_weights (in this order).
  trainable   Whether the layer should be trained (boolean), i.e. whether its potentially-trainable weights should be returned as part of layer.trainable_weights.
  input_spec   Optional (list of) InputSpec object(s) specifying the constraints on inputs that can be accepted by the layer.
  activity_regularizer   Optional regularizer function for the output of this layer.
  dynamic   Whether the layer is dynamic (eager-only); set in the constructor.
  input   Retrieves the input tensor(s) of a layer. Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer.

  losses   List of losses added using the add_loss() API. Variable regularization tensors are created when this property is accessed, so it is eager safe: accessing losses under a tf.GradientTape will propagate gradients back to the corresponding variables.
class MyLayer(tf.keras.layers.Layer):
  def call(self, inputs):
    self.add_loss(tf.abs(tf.reduce_mean(inputs)))
    return inputs
l = MyLayer()
l(np.ones((10, 1)))
l.losses
[1.0]

inputs = tf.keras.Input(shape=(10,))
x = tf.keras.layers.Dense(10)(inputs)
outputs = tf.keras.layers.Dense(1)(x)
model = tf.keras.Model(inputs, outputs)
# Activity regularization.
len(model.losses)
0
model.add_loss(tf.abs(tf.reduce_mean(x)))
len(model.losses)
1

inputs = tf.keras.Input(shape=(10,))
d = tf.keras.layers.Dense(10, kernel_initializer='ones')
x = d(inputs)
outputs = tf.keras.layers.Dense(1)(x)
model = tf.keras.Model(inputs, outputs)
# Weight regularization.
model.add_loss(lambda: tf.reduce_mean(d.kernel))
model.losses
[<tf.Tensor: shape=(), dtype=float32, numpy=1.0>]

  metrics   List of metrics added using the add_metric() API.
input = tf.keras.layers.Input(shape=(3,))
d = tf.keras.layers.Dense(2)
output = d(input)
d.add_metric(tf.reduce_max(output), name='max')
d.add_metric(tf.reduce_min(output), name='min')
[m.name for m in d.metrics]
['max', 'min']

  output   Retrieves the output tensor(s) of a layer. Only applicable if the layer has exactly one output, i.e. if it is connected to one incoming layer.

  supports_masking   Whether this layer supports computing a mask using compute_mask.    Methods add_loss View source
add_loss(
    losses, **kwargs
)
 Add loss tensor(s), potentially dependent on layer inputs. Some losses (for instance, activity regularization losses) may be dependent on the inputs passed when calling a layer. Hence, when reusing the same layer on different inputs a and b, some entries in layer.losses may be dependent on a and some on b. This method automatically keeps track of dependencies. This method can be used inside a subclassed layer or model's call function, in which case losses should be a Tensor or list of Tensors. Example: class MyLayer(tf.keras.layers.Layer):
  def call(self, inputs):
    self.add_loss(tf.abs(tf.reduce_mean(inputs)))
    return inputs
 This method can also be called directly on a Functional Model during construction. In this case, any loss Tensors passed to this Model must be symbolic and be able to be traced back to the model's Inputs. These losses become part of the model's topology and are tracked in get_config. Example: inputs = tf.keras.Input(shape=(10,))
x = tf.keras.layers.Dense(10)(inputs)
outputs = tf.keras.layers.Dense(1)(x)
model = tf.keras.Model(inputs, outputs)
# Activity regularization.
model.add_loss(tf.abs(tf.reduce_mean(x)))
 If this is not the case for your loss (if, for example, your loss references a Variable of one of the model's layers), you can wrap your loss in a zero-argument lambda. These losses are not tracked as part of the model's topology since they can't be serialized. Example: inputs = tf.keras.Input(shape=(10,))
d = tf.keras.layers.Dense(10)
x = d(inputs)
outputs = tf.keras.layers.Dense(1)(x)
model = tf.keras.Model(inputs, outputs)
# Weight regularization.
model.add_loss(lambda: tf.reduce_mean(d.kernel))

 Arguments
  losses   Loss tensor, or list/tuple of tensors. Rather than tensors, losses may also be zero-argument callables which create a loss tensor.
  **kwargs   Additional keyword arguments for backward compatibility. Accepted values: inputs - Deprecated, will be automatically inferred.    add_metric View source
add_metric(
    value, name=None, **kwargs
)
 Adds metric tensor to the layer. This method can be used inside the call() method of a subclassed layer or model. class MyMetricLayer(tf.keras.layers.Layer):
  def __init__(self):
    super(MyMetricLayer, self).__init__(name='my_metric_layer')
    self.mean = tf.keras.metrics.Mean(name='metric_1')

  def call(self, inputs):
    self.add_metric(self.mean(x))
    self.add_metric(tf.reduce_sum(x), name='metric_2')
    return inputs
 This method can also be called directly on a Functional Model during construction. In this case, any tensor passed to this Model must be symbolic and be able to be traced back to the model's Inputs. These metrics become part of the model's topology and are tracked when you save the model via save(). inputs = tf.keras.Input(shape=(10,))
x = tf.keras.layers.Dense(10)(inputs)
outputs = tf.keras.layers.Dense(1)(x)
model = tf.keras.Model(inputs, outputs)
model.add_metric(math_ops.reduce_sum(x), name='metric_1')

Note: Calling add_metric() with the result of a metric object on a Functional Model, as shown in the example below, is not supported. This is because we cannot trace the metric result tensor back to the model's inputs.

inputs = tf.keras.Input(shape=(10,))
x = tf.keras.layers.Dense(10)(inputs)
outputs = tf.keras.layers.Dense(1)(x)
model = tf.keras.Model(inputs, outputs)
model.add_metric(tf.keras.metrics.Mean()(x), name='metric_1')

 Args
  value   Metric tensor.
  name   String metric name.
  **kwargs   Additional keyword arguments for backward compatibility. Accepted values: aggregation - When the value tensor provided is not the result of calling a keras.Metric instance, it will be aggregated by default using a keras.Metric.Mean.    add_weight View source
add_weight(
    name=None, shape=None, dtype=None, initializer=None, regularizer=None,
    trainable=None, constraint=None, use_resource=None,
    synchronization=tf.VariableSynchronization.AUTO,
    aggregation=tf.compat.v1.VariableAggregation.NONE, **kwargs
)
 Adds a new variable to the layer.

 Arguments
  name   Variable name.
  shape   Variable shape. Defaults to scalar if unspecified.
  dtype   The type of the variable. Defaults to self.dtype.
  initializer   Initializer instance (callable).
  regularizer   Regularizer instance (callable).
  trainable   Boolean, whether the variable should be part of the layer's "trainable_variables" (e.g. variables, biases) or "non_trainable_variables" (e.g. BatchNorm mean and variance). Note that trainable cannot be True if synchronization is set to ON_READ.
  constraint   Constraint instance (callable).
  use_resource   Whether to use ResourceVariable.
  synchronization   Indicates when a distributed a variable will be aggregated. Accepted values are constants defined in the class tf.VariableSynchronization. By default the synchronization is set to AUTO and the current DistributionStrategy chooses when to synchronize. If synchronization is set to ON_READ, trainable must not be set to True.
  aggregation   Indicates how a distributed variable will be aggregated. Accepted values are constants defined in the class tf.VariableAggregation.
  **kwargs   Additional keyword arguments. Accepted values are getter, collections, experimental_autocast and caching_device.

 Returns   The variable created.

 Raises
  ValueError   When giving unsupported dtype and no initializer or when trainable has been set to True with synchronization set as ON_READ.    build View source
build(
    input_shape
)
 Creates the variables of the layer (optional, for subclass implementers). This is a method that implementers of subclasses of Layer or Model can override if they need a state-creation step in-between layer instantiation and layer call. This is typically used to create the weights of Layer subclasses.

 Arguments
  input_shape   Instance of TensorShape, or list of instances of TensorShape if the layer expects a list of inputs (one instance per input).    call View source
call(
    inputs, **kwargs
)
 This is where the layer's logic lives. Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.

 Arguments
  inputs   Input tensor, or list/tuple of input tensors.
  **kwargs   Additional keyword arguments. Currently unused.

 Returns   A tensor or list/tuple of tensors.
 compute_mask View source
compute_mask(
    inputs, mask=None
)
 Computes an output mask tensor.

 Arguments
  inputs   Tensor or list of tensors.
  mask   Tensor or list of tensors.

 Returns   None or a tensor (or list of tensors, one per output tensor of the layer).
 compute_output_shape View source
compute_output_shape(
    input_shape
)
 Computes the output shape of the layer. If the layer has not been built, this method will call build on the layer. This assumes that the layer will later be used with inputs that match the input shape provided here.

 Arguments
  input_shape   Shape tuple (tuple of integers) or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.

 Returns   An input shape tuple.
 compute_output_signature View source
compute_output_signature(
    input_signature
)
 Compute the output tensor signature of the layer based on the inputs. Unlike a TensorShape object, a TensorSpec object contains both shape and dtype information for a tensor. This method allows layers to provide output dtype information if it is different from the input dtype. For any layer that doesn't implement this function, the framework will fall back to use compute_output_shape, and will assume that the output dtype matches the input dtype.

 Args
  input_signature   Single TensorSpec or nested structure of TensorSpec objects, describing a candidate input for the layer.

 Returns   Single TensorSpec or nested structure of TensorSpec objects, describing how the layer would transform the provided input.

 Raises
  TypeError   If input_signature contains a non-TensorSpec object.    count_params View source
count_params()
 Count the total number of scalars composing the weights.

 Returns   An integer count.

 Raises
  ValueError   if the layer isn't yet built (in which case its weights aren't yet defined).    from_config View source
@classmethod
from_config(
    config
)
 Creates a layer from its config. This method is the reverse of get_config, capable of instantiating the same layer from the config dictionary. It does not handle layer connectivity (handled by Network), nor weights (handled by set_weights).

 Arguments
  config   A Python dictionary, typically the output of get_config.

 Returns   A layer instance.
 get_config View source
get_config()
 Returns the config of the layer. A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration. The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).

 Returns   Python dictionary.
 get_weights View source
get_weights()
 Returns the current weights of the layer. The weights of a layer represent the state of the layer. This function returns both trainable and non-trainable weight values associated with this layer as a list of Numpy arrays, which can in turn be used to load state into similarly parameterized layers. For example, a Dense layer returns a list of two values-- per-output weights and the bias value. These can be used to set the weights of another Dense layer:
a = tf.keras.layers.Dense(1,
  kernel_initializer=tf.constant_initializer(1.))
a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
b = tf.keras.layers.Dense(1,
  kernel_initializer=tf.constant_initializer(2.))
b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
b.set_weights(a.get_weights())
b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

 Returns   Weights values as a list of numpy arrays.
 set_weights View source
set_weights(
    weights
)
 Sets the weights of the layer, from Numpy arrays. The weights of a layer represent the state of the layer. This function sets the weight values from numpy arrays. The weight values should be passed in the order they are created by the layer. Note that the layer's weights must be instantiated before calling this function by calling the layer. For example, a Dense layer returns a list of two values-- per-output weights and the bias value. These can be used to set the weights of another Dense layer:
a = tf.keras.layers.Dense(1,
  kernel_initializer=tf.constant_initializer(1.))
a_out = a(tf.convert_to_tensor([[1., 2., 3.]]))
a.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]
b = tf.keras.layers.Dense(1,
  kernel_initializer=tf.constant_initializer(2.))
b_out = b(tf.convert_to_tensor([[10., 20., 30.]]))
b.get_weights()
[array([[2.],
       [2.],
       [2.]], dtype=float32), array([0.], dtype=float32)]
b.set_weights(a.get_weights())
b.get_weights()
[array([[1.],
       [1.],
       [1.]], dtype=float32), array([0.], dtype=float32)]

 Arguments
  weights   a list of Numpy arrays. The number of arrays and their shape must match number of the dimensions of the weights of the layer (i.e. it should match the output of get_weights).

 Raises
  ValueError   If the provided weights list does not match the layer's specifications.    __call__ View source
__call__(
    *args, **kwargs
)
 Wraps call, applying pre- and post-processing steps.

 Arguments
  *args   Positional arguments to be passed to self.call.
  **kwargs   Keyword arguments to be passed to self.call.

 Returns   Output tensor(s).
 Note:  The following optional keyword arguments are reserved for specific uses:
training: Boolean scalar tensor of Python boolean indicating whether the call is meant for training or inference.
mask: Boolean input mask.
 If the layer's call method takes a mask argument (as some Keras layers do), its default value will be set to the mask generated for inputs by the previous layer (if input did come from a layer that generated a corresponding mask, i.e. if it came from a Keras layer with masking support.

 Raises
  ValueError   if the layer's call method returns None (an invalid value).
  RuntimeError   if super().__init__() was not called in the constructor.


tf.keras.layers.LayerNormalization     View source on GitHub    Layer normalization layer (Ba et al., 2016). Inherits From: Layer, Module  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.keras.layers.LayerNormalization
tf.keras.layers.LayerNormalization(
    axis=-1, epsilon=0.001, center=True, scale=True,
    beta_initializer='zeros', gamma_initializer='ones',
    beta_regularizer=None, gamma_regularizer=None, beta_constraint=None,
    gamma_constraint=None, trainable=True, name=None, **kwargs
)
 Normalize the activations of the previous layer for each given example in a batch independently, rather than across a batch like Batch Normalization. i.e. applies a transformation that maintains the mean activation within each example close to 0 and the activation standard deviation close to 1. Given a tensor inputs, moments are calculated and normalization is performed across the axes specified in axis. Example:
data = tf.constant(np.arange(10).reshape(5, 2) * 10, dtype=tf.float32)
print(data)
tf.Tensor(
[[ 0. 10.]
 [20. 30.]
 [40. 50.]
 [60. 70.]
 [80. 90.]], shape=(5, 2), dtype=float32)

layer = tf.keras.layers.LayerNormalization(axis=1)
output = layer(data)
print(output)
tf.Tensor(
[[-1. 1.]
 [-1. 1.]
 [-1. 1.]
 [-1. 1.]
 [-1. 1.]], shape=(5, 2), dtype=float32)
 Notice that with Layer Normalization the normalization happens across the axes within each example, rather than across different examples in the batch. If scale or center are enabled, the layer will scale the normalized outputs by broadcasting them with a trainable variable gamma, and center the outputs by broadcasting with a trainable variable beta. gamma will default to a ones tensor and beta will default to a zeros tensor, so that centering and scaling are no-ops before training has begun. So, with scaling and centering enabled the normalization equations are as follows: Let the intermediate activations for a mini-batch to be the inputs. For each sample x_i in inputs with k features, we compute the mean and variance of the sample: mean_i = sum(x_i[j] for j in range(k)) / k
var_i = sum((x_i[j] - mean_i) ** 2 for j in range(k)) / k
 and then compute a normalized x_i_normalized, including a small factor epsilon for numerical stability. x_i_normalized = (x_i - mean_i) / sqrt(var_i + epsilon)
 And finally x_i_normalized is linearly transformed by gamma and beta, which are learned parameters: output_i = x_i_normalized * gamma + beta
 gamma and beta will span the axes of inputs specified in axis, and this part of the inputs' shape must be fully defined. For example:
layer = tf.keras.layers.LayerNormalization(axis=[1, 2, 3])
layer.build([5, 20, 30, 40])
print(layer.beta.shape)
(20, 30, 40)
print(layer.gamma.shape)
(20, 30, 40)
 Note that other implementations of layer normalization may choose to define gamma and beta over a separate set of axes from the axes being normalized across. For example, Group Normalization (Wu et al. 2018) with group size of 1 corresponds to a Layer Normalization that normalizes across height, width, and channel and has gamma and beta span only the channel dimension. So, this Layer Normalization implementation will not match a Group Normalization layer with group size set to 1.

 Arguments
  axis   Integer or List/Tuple. The axis or axes to normalize across. Typically this is the features axis/axes. The left-out axes are typically the batch axis/axes. This argument defaults to -1, the last dimension in the input.
  epsilon   Small float added to variance to avoid dividing by zero. Defaults to 1e-3
  center   If True, add offset of beta to normalized tensor. If False, beta is ignored. Defaults to True.
  scale   If True, multiply by gamma. If False, gamma is not used. Defaults to True. When the next layer is linear (also e.g. nn.relu), this can be disabled since the scaling will be done by the next layer.
  beta_initializer   Initializer for the beta weight. Defaults to zeros.
  gamma_initializer   Initializer for the gamma weight. Defaults to ones.
  beta_regularizer   Optional regularizer for the beta weight. None by default.
  gamma_regularizer   Optional regularizer for the gamma weight. None by default.
  beta_constraint   Optional constraint for the beta weight. None by default.
  gamma_constraint   Optional constraint for the gamma weight. None by default.
  trainable   Boolean, if True the variables will be marked as trainable. Defaults to True.    Input shape: Arbitrary. Use the keyword argument input_shape (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model. Output shape: Same shape as input. Reference:
Lei Ba et al., 2016.


tf.keras.layers.LeakyReLU     View source on GitHub    Leaky version of a Rectified Linear Unit. Inherits From: Layer, Module  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.keras.layers.LeakyReLU
tf.keras.layers.LeakyReLU(
    alpha=0.3, **kwargs
)
 It allows a small gradient when the unit is not active: f(x) = alpha * x if x < 0
f(x) = x if x >= 0
 Usage:
layer = tf.keras.layers.LeakyReLU()
output = layer([-3.0, -1.0, 0.0, 2.0])
list(output.numpy())
[-0.9, -0.3, 0.0, 2.0]
layer = tf.keras.layers.LeakyReLU(alpha=0.1)
output = layer([-3.0, -1.0, 0.0, 2.0])
list(output.numpy())
[-0.3, -0.1, 0.0, 2.0]
 Input shape: Arbitrary. Use the keyword argument input_shape (tuple of integers, does not include the batch axis) when using this layer as the first layer in a model. Output shape: Same shape as the input.

 Arguments
  alpha   Float >= 0. Negative slope coefficient. Default to 0.3.


tf.keras.layers.LocallyConnected1D     View source on GitHub    Locally-connected layer for 1D inputs. Inherits From: Layer, Module  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.keras.layers.LocallyConnected1D
tf.keras.layers.LocallyConnected1D(
    filters, kernel_size, strides=1, padding='valid', data_format=None,
    activation=None, use_bias=True, kernel_initializer='glorot_uniform',
    bias_initializer='zeros', kernel_regularizer=None,
    bias_regularizer=None, activity_regularizer=None, kernel_constraint=None,
    bias_constraint=None, implementation=1, **kwargs
)
 The LocallyConnected1D layer works similarly to the Conv1D layer, except that weights are unshared, that is, a different set of filters is applied at each different patch of the input.
Note: layer attributes cannot be modified after the layer has been called once (except the trainable attribute).
 Example: # apply a unshared weight convolution 1d of length 3 to a sequence with
# 10 timesteps, with 64 output filters
model = Sequential()
model.add(LocallyConnected1D(64, 3, input_shape=(10, 32)))
# now model.output_shape == (None, 8, 64)
# add a new conv1d on top
model.add(LocallyConnected1D(32, 3))
# now model.output_shape == (None, 6, 32)

 Arguments
  filters   Integer, the dimensionality of the output space (i.e. the number of output filters in the convolution).
  kernel_size   An integer or tuple/list of a single integer, specifying the length of the 1D convolution window.
  strides   An integer or tuple/list of a single integer, specifying the stride length of the convolution. Specifying any stride value != 1 is incompatible with specifying any dilation_rate value != 1.
  padding   Currently only supports "valid" (case-insensitive). "same" may be supported in the future. "valid" means no padding.
  data_format   A string, one of channels_last (default) or channels_first. The ordering of the dimensions in the inputs. channels_last corresponds to inputs with shape (batch, length, channels) while channels_first corresponds to inputs with shape (batch, channels, length). It defaults to the image_data_format value found in your Keras config file at ~/.keras/keras.json. If you never set it, then it will be "channels_last".
  activation   Activation function to use. If you don't specify anything, no activation is applied (ie. "linear" activation: a(x) = x).
  use_bias   Boolean, whether the layer uses a bias vector.
  kernel_initializer   Initializer for the kernel weights matrix.
  bias_initializer   Initializer for the bias vector.
  kernel_regularizer   Regularizer function applied to the kernel weights matrix.
  bias_regularizer   Regularizer function applied to the bias vector.
  activity_regularizer   Regularizer function applied to the output of the layer (its "activation")..
  kernel_constraint   Constraint function applied to the kernel matrix.
  bias_constraint   Constraint function applied to the bias vector.
  implementation   implementation mode, either 1, 2, or 3. 1 loops over input spatial locations to perform the forward pass. It is memory-efficient but performs a lot of (small) ops. 2 stores layer weights in a dense but sparsely-populated 2D matrix and implements the forward pass as a single matrix-multiply. It uses a lot of RAM but performs few (large) ops. 3 stores layer weights in a sparse tensor and implements the forward pass as a single sparse matrix-multiply. How to choose: 1: large, dense models, 2: small models, 3: large, sparse models, where "large" stands for large input/output activations (i.e. many filters, input_filters, large input_size, output_size), and "sparse" stands for few connections between inputs and outputs, i.e. small ratio filters * input_filters * kernel_size / (input_size * strides), where inputs to and outputs of the layer are assumed to have shapes (input_size, input_filters), (output_size, filters) respectively. It is recommended to benchmark each in the setting of interest to pick the most efficient one (in terms of speed and memory usage). Correct choice of implementation can lead to dramatic speed improvements (e.g. 50X), potentially at the expense of RAM. Also, only padding="valid" is supported by implementation=1.
   Input shape: 3D tensor with shape: (batch_size, steps, input_dim) Output shape: 3D tensor with shape: (batch_size, new_steps, filters) steps value might have changed due to padding or strides.


tf.keras.layers.LocallyConnected2D     View source on GitHub    Locally-connected layer for 2D inputs. Inherits From: Layer, Module  View aliases  Compat aliases for migration
See Migration guide for more details. tf.compat.v1.keras.layers.LocallyConnected2D
tf.keras.layers.LocallyConnected2D(
    filters, kernel_size, strides=(1, 1), padding='valid',
    data_format=None, activation=None, use_bias=True,
    kernel_initializer='glorot_uniform',
    bias_initializer='zeros', kernel_regularizer=None,
    bias_regularizer=None, activity_regularizer=None, kernel_constraint=None,
    bias_constraint=None, implementation=1, **kwargs
)
 The LocallyConnected2D layer works similarly to the Conv2D layer, except that weights are unshared, that is, a different set of filters is applied at each different patch of the input.
Note: layer attributes cannot be modified after the layer has been called once (except the trainable attribute).
 Examples: # apply a 3x3 unshared weights convolution with 64 output filters on a
32x32 image
# with `data_format="channels_last"`:
model = Sequential()
model.add(LocallyConnected2D(64, (3, 3), input_shape=(32, 32, 3)))
# now model.output_shape == (None, 30, 30, 64)
# notice that this layer will consume (30*30)*(3*3*3*64) + (30*30)*64
parameters

# add a 3x3 unshared weights convolution on top, with 32 output filters:
model.add(LocallyConnected2D(32, (3, 3)))
# now model.output_shape == (None, 28, 28, 32)

 Arguments
  filters   Integer, the dimensionality of the output space (i.e. the number of output filters in the convolution).
  kernel_size   An integer or tuple/list of 2 integers, specifying the width and height of the 2D convolution window. Can be a single integer to specify the same value for all spatial dimensions.
  strides   An integer or tuple/list of 2 integers, specifying the strides of the convolution along the width and height. Can be a single integer to specify the same value for all spatial dimensions.
  padding   Currently only support "valid" (case-insensitive). "same" will be supported in future. "valid" means no padding.
  data_format   A string, one of channels_last (default) or channels_first. The ordering of the dimensions in the inputs. channels_last corresponds to inputs with shape (batch, height, width, channels) while channels_first corresponds to inputs with shape (batch, channels, height, width). It defaults to the image_data_format value found in your Keras config file at ~/.keras/keras.json. If you never set it, then it will be "channels_last".
  activation   Activation function to use. If you don't specify anything, no activation is applied (ie. "linear" activation: a(x) = x).
  use_bias   Boolean, whether the layer uses a bias vector.
  kernel_initializer   Initializer for the kernel weights matrix.
  bias_initializer   Initializer for the bias vector.
  kernel_regularizer   Regularizer function applied to the kernel weights matrix.
  bias_regularizer   Regularizer function applied to the bias vector.
  activity_regularizer   Regularizer function applied to the output of the layer (its "activation").
  kernel_constraint   Constraint function applied to the kernel matrix.
  bias_constraint   Constraint function applied to the bias vector.
  implementation   implementation mode, either 1, 2, or 3. 1 loops over input spatial locations to perform the forward pass. It is memory-efficient but performs a lot of (small) ops. 2 stores layer weights in a dense but sparsely-populated 2D matrix and implements the forward pass as a single matrix-multiply. It uses a lot of RAM but performs few (large) ops. 3 stores layer weights in a sparse tensor and implements the forward pass as a single sparse matrix-multiply. How to choose: 1: large, dense models, 2: small models, 3: large, sparse models, where "large" stands for large input/output activations (i.e. many filters, input_filters, large np.prod(input_size), np.prod(output_size)), and "sparse" stands for few connections between inputs and outputs, i.e. small ratio `filters * input_filters * np.prod(kernel_size) / (np.prod(input_size)  np.prod(strides)), where inputs to and outputs of the layer are assumed to have shapesinput_size + (input_filters,),output_size + (filters,)` respectively.  It is recommended to benchmark each in the setting of interest to pick the most efficient one (in terms of speed and memory usage). Correct choice of implementation can lead to dramatic speed improvements (e.g. 50X), potentially at the expense of RAM. Also, only padding="valid" is supported by implementation=1.
   Input shape: 4D tensor with shape: (samples, channels, rows, cols) if data_format='channels_first' or 4D tensor with shape: (samples, rows, cols, channels) if data_format='channels_last'. Output shape: 4D tensor with shape: (samples, filters, new_rows, new_cols) if data_format='channels_first' or 4D tensor with shape: (samples, new_rows, new_cols, filters) if data_format='channels_last'. rows and cols values might have changed due to padding.


tf.keras.layers.LSTM     View source on GitHub    Long Short-Term Memory layer - Hochreiter 1997. Inherits From: LSTM, RNN, Layer, Module
tf.keras.layers.LSTM(
    units, activation='tanh', recurrent_activation='sigmoid',
    use_bias=True, kernel_initializer='glorot_uniform',
    recurrent_initializer='orthogonal',
    bias_initializer='zeros', unit_forget_bias=True,
    kernel_regularizer=None, recurrent_regularizer=None, bias_regularizer=None,
    activity_regularizer=None, kernel_constraint=None, recurrent_constraint=None,
    bias_constraint=None, dropout=0.0, recurrent_dropout=0.0,
    return_sequences=False, return_state=False, go_backwards=False, stateful=False,
    time_major=False, unroll=False, **kwargs
)
 See the Keras RNN API guide for details about the usage of RNN API. Based on available runtime hardware and constraints, this layer will choose different implementations (cuDNN-based or pure-TensorFlow) to maximize the performance. If a GPU is available and all the arguments to the layer meet the requirement of the CuDNN kernel (see below for details), the layer will use a fast cuDNN implementation. The requirements to use the cuDNN implementation are:
activation == tanh

recurrent_activation == sigmoid

recurrent_dropout == 0
unroll is False

use_bias is True
 Inputs, if use masking, are strictly right-padded. Eager execution is enabled in the outermost context.  For example:
inputs = tf.random.normal([32, 10, 8])
lstm = tf.keras.layers.LSTM(4)
output = lstm(inputs)
print(output.shape)
(32, 4)
lstm = tf.keras.layers.LSTM(4, return_sequences=True, return_state=True)
whole_seq_output, final_memory_state, final_carry_state = lstm(inputs)
print(whole_seq_output.shape)
(32, 10, 4)
print(final_memory_state.shape)
(32, 4)
print(final_carry_state.shape)
(32, 4)

 Arguments
  units   Positive integer, dimensionality of the output space.
  activation   Activation function to use. Default: hyperbolic tangent (tanh). If you pass None, no activation is applied (ie. "linear" activation: a(x) = x).
  recurrent_activation   Activation function to use for the recurrent step. Default: sigmoid (sigmoid). If you pass None, no activation is applied (ie. "linear" activation: a(x) = x).
  use_bias   Boolean (default True), whether the layer uses a bias vector.
  kernel_initializer   Initializer for the kernel weights matrix, used for the linear transformation of the inputs. Default: glorot_uniform.
  recurrent_initializer   Initializer for the recurrent_kernel weights matrix, used for the linear transformation of the recurrent state. Default: orthogonal.
  bias_initializer   Initializer for the bias vector. Default: zeros.
  unit_forget_bias   Boolean (default True). If True, add 1 to the bias of the forget gate at initialization. Setting it to true will also force bias_initializer="zeros". This is recommended in Jozefowicz et al..
  kernel_regularizer   Regularizer function applied to the kernel weights matrix. Default: None.
  recurrent_regularizer   Regularizer function applied to the recurrent_kernel weights matrix. Default: None.
  bias_regularizer   Regularizer function applied to the bias vector. Default: None.
  activity_regularizer   Regularizer function applied to the output of the layer (its "activation"). Default: None.
  kernel_constraint   Constraint function applied to the kernel weights matrix. Default: None.
  recurrent_constraint   Constraint function applied to the recurrent_kernel weights matrix. Default: None.
  bias_constraint   Constraint function applied to the bias vector. Default: None.
  dropout   Float between 0 and 1. Fraction of the units to drop for the linear transformation of the inputs. Default: 0.
  recurrent_dropout   Float between 0 and 1. Fraction of the units to drop for the linear transformation of the recurrent state. Default: 0.
  return_sequences   Boolean. Whether to return the last output. in the output sequence, or the full sequence. Default: False.
  return_state   Boolean. Whether to return the last state in addition to the output. Default: False.
  go_backwards   Boolean (default False). If True, process the input sequence backwards and return the reversed sequence.
  stateful   Boolean (default False). If True, the last state for each sample at index i in a batch will be used as initial state for the sample of index i in the following batch.
  time_major   The shape format of the inputs and outputs tensors. If True, the inputs and outputs will be in shape [timesteps, batch, feature], whereas in the False case, it will be [batch, timesteps, feature]. Using time_major = True is a bit more efficient because it avoids transposes at the beginning and end of the RNN calculation. However, most TensorFlow data is batch-major, so by default this function accepts input and emits output in batch-major form.
  unroll   Boolean (default False). If True, the network will be unrolled, else a symbolic loop will be used. Unrolling can speed-up a RNN, although it tends to be more memory-intensive. Unrolling is only suitable for short sequences.    Call arguments:
inputs: A 3D tensor with shape [batch, timesteps, feature].
mask: Binary tensor of shape [batch, timesteps] indicating whether a given timestep should be masked (optional, defaults to None).
training: Python boolean indicating whether the layer should behave in training mode or in inference mode. This argument is passed to the cell when calling it. This is only relevant if dropout or recurrent_dropout is used (optional, defaults to None).
initial_state: List of initial state tensors to be passed to the first call of the cell (optional, defaults to None which causes creation of zero-filled initial state tensors).

 Attributes
  activation

  bias_constraint

  bias_initializer

  bias_regularizer

  dropout

  implementation

  kernel_constraint

  kernel_initializer

  kernel_regularizer

  recurrent_activation

  recurrent_constraint

  recurrent_dropout

  recurrent_initializer

  recurrent_regularizer

  states

  unit_forget_bias

  units

  use_bias
   Methods get_dropout_mask_for_cell View source
get_dropout_mask_for_cell(
    inputs, training, count=1
)
 Get the dropout mask for RNN cell's input. It will create mask based on context if there isn't any existing cached mask. If a new mask is generated, it will update the cache in the cell.

 Args
  inputs   The input tensor whose shape will be used to generate dropout mask.
  training   Boolean tensor, whether its in training mode, dropout will be ignored in non-training mode.
  count   Int, how many dropout mask will be generated. It is useful for cell that has internal weights fused together.

 Returns   List of mask tensor, generated or cached mask based on context.
 get_recurrent_dropout_mask_for_cell View source
get_recurrent_dropout_mask_for_cell(
    inputs, training, count=1
)
 Get the recurrent dropout mask for RNN cell. It will create mask based on context if there isn't any existing cached mask. If a new mask is generated, it will update the cache in the cell.

 Args
  inputs   The input tensor whose shape will be used to generate dropout mask.
  training   Boolean tensor, whether its in training mode, dropout will be ignored in non-training mode.
  count   Int, how many dropout mask will be generated. It is useful for cell that has internal weights fused together.

 Returns   List of mask tensor, generated or cached mask based on context.
 reset_dropout_mask View source
reset_dropout_mask()
 Reset the cached dropout masks if any. This is important for the RNN layer to invoke this in it call() method so that the cached mask is cleared before calling the cell.call(). The mask should be cached across the timestep within the same batch, but shouldn't be cached between batches. Otherwise it will introduce unreasonable bias against certain index of data within the batch. reset_recurrent_dropout_mask View source
reset_recurrent_dropout_mask()
 Reset the cached recurrent dropout masks if any. This is important for the RNN layer to invoke this in it call() method so that the cached mask is cleared before calling the cell.call(). The mask should be cached across the timestep within the same batch, but shouldn't be cached between batches. Otherwise it will introduce unreasonable bias against certain index of data within the batch. reset_states View source
reset_states(
    states=None
)
 Reset the recorded states for the stateful RNN layer. Can only be used when RNN layer is constructed with stateful = True. Args: states: Numpy arrays that contains the value for the initial state, which will be feed to cell at the first time step. When the value is None, zero filled numpy array will be created based on the cell state size.

 Raises
  AttributeError   When the RNN layer is not stateful.
  ValueError   When the batch size of the RNN layer is unknown.
  ValueError   When the input numpy array is not compatible with the RNN layer state, either size wise or dtype wise.