Repository: keras-team/keras-nlp
Pull request #670 (2023-01-17 07:57:54)

add f_net_classifier and f_net_classifier_test
Add FNetClassifier #650

Here I add two script at f_net folder.

But we need to improve the f_net_classifier file, because if this model similar to [BertClassifier](https://github.com/keras-team/keras-nlp/blob/master/keras_nlp/models/bert/bert_classifier.py), then the f_net_presets.py file is require that is not present at f_net folder. 

@abheesht17 @mattdangerw 

mattdangerw: @ADITYADAS1999 looks like there are some testing failures, could you take a look at those first? You'll probably want an environment in which you can reproduce the failure locally, as described in our [contributing guide](https://github.com/keras-team/keras-nlp/blob/master/CONTRIBUTING.md).
ADITYADAS1999: > @ADITYADAS1999 looks like there are some testing failures, could you take a look at those first? You'll probably want an environment in which you can reproduce the failure locally, as described in our [contributing guide](https://github.com/keras-team/keras-nlp/blob/master/CONTRIBUTING.md).

okk I test the file locally 
ADITYADAS1999: > > @ADITYADAS1999 looks like there are some testing failures, could you take a look at those first? You'll probably want an environment in which you can reproduce the failure locally, as described in our [contributing guide](https://github.com/keras-team/keras-nlp/blob/master/CONTRIBUTING.md).
> 
> okk I test the file locally


![testone](https://user-images.githubusercontent.com/58718316/213361402-8f2148b5-95d9-41fb-89d9-0ece7cc22eae.PNG)


when I use this commands for windows it show this errors in my pycharm editor


isort --sp "setup.cfg" --sl <name-of-file>
black --line-length 80 <name-of-file>
flake8 --config "setup.cfg" --max-line-length=200 <name-of-file>

python version 3.7
OS : windows10
abheesht17: @ADITYADAS1999, pass the complete path from the root of the repository.

Instead of `f_net_classifier.py`, pass this: `keras_nlp/models/f_net/f_net_classifier.py`.

So, your commands will be:

```
isort --sp "setup.cfg" --sl keras_nlp/models/f_net/f_net_classifier.py
black --line-length 80 keras_nlp/models/f_net/f_net_classifier.py
flake8 --config "setup.cfg" --max-line-length=200 keras_nlp/models/f_net/f_net_classifier.py

isort --sp "setup.cfg" --sl keras_nlp/models/f_net/f_net_classifier_test.py
black --line-length 80 keras_nlp/models/f_net/f_net_classifier_test.py
flake8 --config "setup.cfg" --max-line-length=200 keras_nlp/models/f_net/f_net_classifier_test.py
```

Hope this solves the issue! :)
ADITYADAS1999: > @ADITYADAS1999, pass the complete path from the root of the repository.
> 
> Instead of `f_net_classifier.py`, pass this: `keras_nlp/models/f_net/f_net_classifier.py`.
> 
> So, your commands will be:
> 
> ```
> isort --sp "setup.cfg" --sl keras_nlp/models/f_net/f_net_classifier.py
> black --line-length 80 keras_nlp/models/f_net/f_net_classifier.py
> flake8 --config "setup.cfg" --max-line-length=200 keras_nlp/models/f_net/f_net_classifier.py
> 
> isort --sp "setup.cfg" --sl keras_nlp/models/f_net/f_net_classifier_test.py
> black --line-length 80 keras_nlp/models/f_net/f_net_classifier_test.py
> flake8 --config "setup.cfg" --max-line-length=200 keras_nlp/models/f_net/f_net_classifier_test.py
> ```
> 
> Hope this solves the issue! :)

thanks [abheesht17](https://github.com/abheesht17) try this commands
abheesht17: @ADITYADAS1999 - it is recommended that if you are using Windows, use WSL. Makes life very easy! You can follow this guide: https://learn.microsoft.com/en-us/windows/wsl/install.
ADITYADAS1999: > > @ADITYADAS1999, pass the complete path from the root of the repository.
> > Instead of `f_net_classifier.py`, pass this: `keras_nlp/models/f_net/f_net_classifier.py`.
> > So, your commands will be:
> > ```
> > isort --sp "setup.cfg" --sl keras_nlp/models/f_net/f_net_classifier.py
> > black --line-length 80 keras_nlp/models/f_net/f_net_classifier.py
> > flake8 --config "setup.cfg" --max-line-length=200 keras_nlp/models/f_net/f_net_classifier.py
> > 
> > isort --sp "setup.cfg" --sl keras_nlp/models/f_net/f_net_classifier_test.py
> > black --line-length 80 keras_nlp/models/f_net/f_net_classifier_test.py
> > flake8 --config "setup.cfg" --max-line-length=200 keras_nlp/models/f_net/f_net_classifier_test.py
> > ```
> > 
> > 
> >     
> >       
> >     
> > 
> >       
> >     
> > 
> >     
> >   
> > Hope this solves the issue! :)
> 
> thanks [abheesht17](https://github.com/abheesht17), I will try this commands

![codeone](https://user-images.githubusercontent.com/58718316/213402610-b91ef618-ea7c-490c-a175-d938c6864ac9.PNG)

It works, I will make the changes soon 😀
ADITYADAS1999: > @ADITYADAS1999 - it is recommended that if you are using Windows, use WSL. Makes life very easy! You can follow this guide: https://learn.microsoft.com/en-us/windows/wsl/install.

I need to install this environment,  thanks [abheesht17](https://github.com/abheesht17)
ADITYADAS1999: > @ADITYADAS1999 - it is recommended that if you are using Windows, use WSL. Makes life very easy! You can follow this guide: https://learn.microsoft.com/en-us/windows/wsl/install.


installed successfully 
![linux](https://user-images.githubusercontent.com/58718316/213467741-025063c1-491b-448c-9181-469ffedd6ce3.PNG)
 


abheesht17: @ADITYADAS1999, yep! The commands are correct. Do you have the correct black and flake version though? Please follow the pip install commands given here: https://github.com/keras-team/keras-nlp/blob/master/CONTRIBUTING.md#formatting-code.
ADITYADAS1999: > @ADITYADAS1999, yep! The commands are correct. Do you have the correct black and flake version though? Please follow the pip install commands given here: https://github.com/keras-team/keras-nlp/blob/master/CONTRIBUTING.md#formatting-code.

yes I install both 
![78454](https://user-images.githubusercontent.com/58718316/213706292-efbe7ac1-78f8-4549-a9f4-51409ec03f16.PNG)


ADITYADAS1999: hey @mattdangerw can you review this one, If necessary changes are require , All tests are now passed. 

Patch:
diff --git a/keras_nlp/models/albert/albert_backbone.py b/keras_nlp/models/albert/albert_backbone.py
index 17b61b21..fdd3a85c 100644
--- a/keras_nlp/models/albert/albert_backbone.py
+++ b/keras_nlp/models/albert/albert_backbone.py
@@ -271,6 +271,10 @@ class AlbertBackbone(Backbone):
         )
         return config
 
+    @property
+    def token_embedding(self):
+        return self.get_layer("token_embedding")
+
     @classproperty
     def presets(cls):
         return copy.deepcopy(backbone_presets)
diff --git a/keras_nlp/models/albert/albert_tokenizer.py b/keras_nlp/models/albert/albert_tokenizer.py
index e2647c30..35dc7ec8 100644
--- a/keras_nlp/models/albert/albert_tokenizer.py
+++ b/keras_nlp/models/albert/albert_tokenizer.py
@@ -15,14 +15,12 @@
 """ALBERT tokenizer."""
 
 import copy
-import os
 
 from tensorflow import keras
 
 from keras_nlp.models.albert.albert_presets import backbone_presets
 from keras_nlp.tokenizers.sentence_piece_tokenizer import SentencePieceTokenizer
 from keras_nlp.utils.python_utils import classproperty
-from keras_nlp.utils.python_utils import format_docstring
 
 
 @keras.utils.register_keras_serializable(package="keras_nlp")
@@ -89,52 +87,3 @@ class AlbertTokenizer(SentencePieceTokenizer):
     @classproperty
     def presets(cls):
         return copy.deepcopy(backbone_presets)
-
-    @classmethod
-    @format_docstring(names=", ".join(backbone_presets))
-    def from_preset(
-        cls,
-        preset,
-        **kwargs,
-    ):
-        """Instantiate an ALBERT tokenizer from preset vocabulary.
-
-        Args:
-            preset: string. Must be one of {{names}}.
-
-        Examples:
-        ```python
-        # Load a preset tokenizer.
-        tokenizer = keras_nlp.models.AlbertTokenizer.from_preset(
-            "albert_base_en_uncased",
-        )
-
-        # Tokenize some input.
-        tokenizer("The quick brown fox tripped.")
-
-        # Detokenize some input.
-        tokenizer.detokenize([5, 6, 7, 8, 9])
-        ```
-        """
-        if preset not in cls.presets:
-            raise ValueError(
-                "`preset` must be one of "
-                f"""{", ".join(cls.presets)}. Received: {preset}."""
-            )
-        metadata = cls.presets[preset]
-
-        spm_proto = keras.utils.get_file(
-            "vocab.spm",
-            metadata["spm_proto_url"],
-            cache_subdir=os.path.join("models", preset),
-            file_hash=metadata["spm_proto_hash"],
-        )
-
-        config = metadata["preprocessor_config"]
-        config.update(
-            {
-                "proto": spm_proto,
-            },
-        )
-
-        return cls.from_config({**config, **kwargs})
diff --git a/keras_nlp/models/backbone.py b/keras_nlp/models/backbone.py
index 7cdc7cd3..647da9cf 100644
--- a/keras_nlp/models/backbone.py
+++ b/keras_nlp/models/backbone.py
@@ -27,6 +27,11 @@ class Backbone(keras.Model):
     def __init__(self, *args, **kwargs):
         super().__init__(*args, **kwargs)
 
+    @property
+    def token_embedding(self):
+        """A `keras.layers.Embedding` instance for embedding token ids."""
+        raise NotImplementedError
+
     def get_config(self):
         # Don't chain to super here. The default `get_config()` for functional
         # models is nested and cannot be passed to our Backbone constructors.
diff --git a/keras_nlp/models/bart/bart_backbone.py b/keras_nlp/models/bart/bart_backbone.py
index 475f0b2e..7a5cc5d7 100644
--- a/keras_nlp/models/bart/bart_backbone.py
+++ b/keras_nlp/models/bart/bart_backbone.py
@@ -244,3 +244,7 @@ class BartBackbone(Backbone):
             "name": self.name,
             "trainable": self.trainable,
         }
+
+    @property
+    def token_embedding(self):
+        return self.get_layer("token_embedding")
diff --git a/keras_nlp/models/bert/bert_backbone.py b/keras_nlp/models/bert/bert_backbone.py
index d59fc11e..3e7d4673 100644
--- a/keras_nlp/models/bert/bert_backbone.py
+++ b/keras_nlp/models/bert/bert_backbone.py
@@ -190,16 +190,16 @@ class BertBackbone(Backbone):
             },
             **kwargs,
         )
+
         # All references to `self` below this line
         self.vocabulary_size = vocabulary_size
-        self.hidden_dim = hidden_dim
-        self.intermediate_dim = intermediate_dim
         self.num_layers = num_layers
         self.num_heads = num_heads
+        self.hidden_dim = hidden_dim
+        self.intermediate_dim = intermediate_dim
+        self.dropout = dropout
         self.max_sequence_length = max_sequence_length
         self.num_segments = num_segments
-        self.dropout = dropout
-        self.token_embedding = token_embedding_layer
         self.cls_token_index = cls_token_index
 
     def get_config(self):
@@ -207,17 +207,21 @@ class BertBackbone(Backbone):
         config.update(
             {
                 "vocabulary_size": self.vocabulary_size,
-                "hidden_dim": self.hidden_dim,
-                "intermediate_dim": self.intermediate_dim,
                 "num_layers": self.num_layers,
                 "num_heads": self.num_heads,
+                "hidden_dim": self.hidden_dim,
+                "intermediate_dim": self.intermediate_dim,
+                "dropout": self.dropout,
                 "max_sequence_length": self.max_sequence_length,
                 "num_segments": self.num_segments,
-                "dropout": self.dropout,
             }
         )
         return config
 
+    @property
+    def token_embedding(self):
+        return self.get_layer("token_embedding")
+
     @classproperty
     def presets(cls):
         return copy.deepcopy(backbone_presets)
diff --git a/keras_nlp/models/bert/bert_tokenizer.py b/keras_nlp/models/bert/bert_tokenizer.py
index 8f1db180..629e2a3c 100644
--- a/keras_nlp/models/bert/bert_tokenizer.py
+++ b/keras_nlp/models/bert/bert_tokenizer.py
@@ -14,7 +14,6 @@
 """BERT tokenizer."""
 
 import copy
-import os
 
 from tensorflow import keras
 
@@ -22,7 +21,6 @@ from keras_nlp.models.bert.bert_presets import backbone_presets
 from keras_nlp.models.bert.bert_presets import classifier_presets
 from keras_nlp.tokenizers.word_piece_tokenizer import WordPieceTokenizer
 from keras_nlp.utils.python_utils import classproperty
-from keras_nlp.utils.python_utils import format_docstring
 
 PRESET_NAMES = ", ".join(list(backbone_presets) + list(classifier_presets))
 
@@ -112,52 +110,3 @@ class BertTokenizer(WordPieceTokenizer):
     @classproperty
     def presets(cls):
         return copy.deepcopy({**backbone_presets, **classifier_presets})
-
-    @classmethod
-    @format_docstring(names=PRESET_NAMES)
-    def from_preset(
-        cls,
-        preset,
-        **kwargs,
-    ):
-        """Instantiate a BERT tokenizer from preset vocabulary.
-
-        Args:
-            preset: string. Must be one of {{names}}.
-
-        Examples:
-        ```python
-        # Load a preset tokenizer.
-        tokenizer = keras_nlp.models.BertTokenizer.from_preset(
-            "bert_base_en_uncased",
-        )
-
-        # Tokenize some input.
-        tokenizer("The quick brown fox tripped.")
-
-        # Detokenize some input.
-        tokenizer.detokenize([5, 6, 7, 8, 9])
-        ```
-        """
-        if preset not in cls.presets:
-            raise ValueError(
-                "`preset` must be one of "
-                f"""{", ".join(cls.presets)}. Received: {preset}."""
-            )
-        metadata = cls.presets[preset]
-
-        vocabulary = keras.utils.get_file(
-            "vocab.txt",
-            metadata["vocabulary_url"],
-            cache_subdir=os.path.join("models", preset),
-            file_hash=metadata["vocabulary_hash"],
-        )
-
-        config = metadata["preprocessor_config"]
-        config.update(
-            {
-                "vocabulary": vocabulary,
-            },
-        )
-
-        return cls.from_config({**config, **kwargs})
diff --git a/keras_nlp/models/deberta_v3/deberta_v3_backbone.py b/keras_nlp/models/deberta_v3/deberta_v3_backbone.py
index 4473ebcb..e9152903 100644
--- a/keras_nlp/models/deberta_v3/deberta_v3_backbone.py
+++ b/keras_nlp/models/deberta_v3/deberta_v3_backbone.py
@@ -200,6 +200,10 @@ class DebertaV3Backbone(Backbone):
         )
         return config
 
+    @property
+    def token_embedding(self):
+        return self.get_layer("token_embedding")
+
     @classproperty
     def presets(cls):
         return copy.deepcopy(backbone_presets)
diff --git a/keras_nlp/models/deberta_v3/deberta_v3_tokenizer.py b/keras_nlp/models/deberta_v3/deberta_v3_tokenizer.py
index aad3092c..dfd6ba83 100644
--- a/keras_nlp/models/deberta_v3/deberta_v3_tokenizer.py
+++ b/keras_nlp/models/deberta_v3/deberta_v3_tokenizer.py
@@ -15,14 +15,12 @@
 """DeBERTa tokenizer."""
 
 import copy
-import os
 
 from tensorflow import keras
 
 from keras_nlp.models.deberta_v3.deberta_v3_presets import backbone_presets
 from keras_nlp.tokenizers.sentence_piece_tokenizer import SentencePieceTokenizer
 from keras_nlp.utils.python_utils import classproperty
-from keras_nlp.utils.python_utils import format_docstring
 
 
 @keras.utils.register_keras_serializable(package="keras_nlp")
@@ -89,52 +87,3 @@ class DebertaV3Tokenizer(SentencePieceTokenizer):
     @classproperty
     def presets(cls):
         return copy.deepcopy(backbone_presets)
-
-    @classmethod
-    @format_docstring(names=", ".join(backbone_presets))
-    def from_preset(
-        cls,
-        preset,
-        **kwargs,
-    ):
-        """Instantiate a DeBERTa tokenizer from preset vocabulary.
-
-        Args:
-            preset: string. Must be one of {{names}}.
-
-        Examples:
-        ```python
-        # Load a preset tokenizer.
-        tokenizer = keras_nlp.models.DebertaV3Tokenizer.from_preset(
-            "deberta_v3_base_en",
-        )
-
-        # Tokenize some input.
-        tokenizer("The quick brown fox tripped.")
-
-        # Detokenize some input.
-        tokenizer.detokenize([5, 6, 7, 8, 9])
-        ```
-        """
-        if preset not in cls.presets:
-            raise ValueError(
-                "`preset` must be one of "
-                f"""{", ".join(cls.presets)}. Received: {preset}."""
-            )
-        metadata = cls.presets[preset]
-
-        spm_proto = keras.utils.get_file(
-            "vocab.spm",
-            metadata["spm_proto_url"],
-            cache_subdir=os.path.join("models", preset),
-            file_hash=metadata["spm_proto_hash"],
-        )
-
-        config = metadata["preprocessor_config"]
-        config.update(
-            {
-                "proto": spm_proto,
-            },
-        )
-
-        return cls.from_config({**config, **kwargs})
diff --git a/keras_nlp/models/distil_bert/distil_bert_backbone.py b/keras_nlp/models/distil_bert/distil_bert_backbone.py
index 451b646d..4920083c 100644
--- a/keras_nlp/models/distil_bert/distil_bert_backbone.py
+++ b/keras_nlp/models/distil_bert/distil_bert_backbone.py
@@ -180,6 +180,10 @@ class DistilBertBackbone(Backbone):
         )
         return config
 
+    @property
+    def token_embedding(self):
+        return self.get_layer("token_and_position_embedding").token_embedding
+
     @classproperty
     def presets(cls):
         return copy.deepcopy(backbone_presets)
diff --git a/keras_nlp/models/distil_bert/distil_bert_tokenizer.py b/keras_nlp/models/distil_bert/distil_bert_tokenizer.py
index 103f015c..3d0dd7d7 100644
--- a/keras_nlp/models/distil_bert/distil_bert_tokenizer.py
+++ b/keras_nlp/models/distil_bert/distil_bert_tokenizer.py
@@ -14,14 +14,12 @@
 """DistilBERT tokenizer."""
 
 import copy
-import os
 
 from tensorflow import keras
 
 from keras_nlp.models.distil_bert.distil_bert_presets import backbone_presets
 from keras_nlp.tokenizers.word_piece_tokenizer import WordPieceTokenizer
 from keras_nlp.utils.python_utils import classproperty
-from keras_nlp.utils.python_utils import format_docstring
 
 
 @keras.utils.register_keras_serializable(package="keras_nlp")
@@ -109,52 +107,3 @@ class DistilBertTokenizer(WordPieceTokenizer):
     @classproperty
     def presets(cls):
         return copy.deepcopy(backbone_presets)
-
-    @classmethod
-    @format_docstring(names=", ".join(backbone_presets))
-    def from_preset(
-        cls,
-        preset,
-        **kwargs,
-    ):
-        """Instantiate a DistilBERT tokenizer from preset vocabulary.
-
-        Args:
-            preset: string. Must be one of {{names}}.
-
-        Examples:
-        ```python
-        # Load a preset tokenizer.
-        tokenizer = keras_nlp.models.DistilBertTokenizer.from_preset(
-            "distil_bert_base_en_uncased",
-        )
-
-        # Tokenize some input.
-        tokenizer("The quick brown fox tripped.")
-
-        # Detokenize some input.
-        tokenizer.detokenize([5, 6, 7, 8, 9])
-        ```
-        """
-        if preset not in cls.presets:
-            raise ValueError(
-                "`preset` must be one of "
-                f"""{", ".join(cls.presets)}. Received: {preset}."""
-            )
-        metadata = cls.presets[preset]
-
-        vocabulary = keras.utils.get_file(
-            "vocab.txt",
-            metadata["vocabulary_url"],
-            cache_subdir=os.path.join("models", preset),
-            file_hash=metadata["vocabulary_hash"],
-        )
-
-        config = metadata["preprocessor_config"]
-        config.update(
-            {
-                "vocabulary": vocabulary,
-            },
-        )
-
-        return cls.from_config({**config, **kwargs})
diff --git a/keras_nlp/models/f_net/f_net_backbone.py b/keras_nlp/models/f_net/f_net_backbone.py
index 83acd9e2..2e6dc6dd 100644
--- a/keras_nlp/models/f_net/f_net_backbone.py
+++ b/keras_nlp/models/f_net/f_net_backbone.py
@@ -216,6 +216,10 @@ class FNetBackbone(Backbone):
         )
         return config
 
+    @property
+    def token_embedding(self):
+        return self.get_layer("token_embedding")
+
     @classproperty
     def presets(cls):
         return copy.deepcopy(backbone_presets)
diff --git a/keras_nlp/models/f_net/f_net_classifier.py b/keras_nlp/models/f_net/f_net_classifier.py
new file mode 100644
index 00000000..04b1fef9
--- /dev/null
+++ b/keras_nlp/models/f_net/f_net_classifier.py
@@ -0,0 +1,144 @@
+# Copyright 2022 The KerasNLP Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""FNet classification model."""
+
+import copy
+
+from tensorflow import keras
+
+from keras_nlp.models.f_net.f_net_backbone import FNetBackbone
+from keras_nlp.models.f_net.f_net_backbone import f_net_kernel_initializer
+from keras_nlp.models.f_net.f_net_preprocessor import FNetPreprocessor
+from keras_nlp.models.f_net.f_net_presets import backbone_presets
+from keras_nlp.models.task import Task
+from keras_nlp.utils.python_utils import classproperty
+
+
+@keras.utils.register_keras_serializable(package="keras_nlp")
+class FNetClassifier(Task):
+    """An end-to-end f_net model for classification tasks.
+
+    This model attaches a classification head to a
+    `keras_nlp.model.FNetBackbone` model, mapping from the backbone
+    outputs to logit output suitable for a classification task. For usage of
+    this model with pre-trained weights, see the `from_preset()` method.
+
+    This model can optionally be configured with a `preprocessor` layer, in
+    which case it will automatically apply preprocessing to raw inputs during
+    `fit()`, `predict()`, and `evaluate()`. This is done by default when
+    creating the model with `from_preset()`.
+
+    Disclaimer: Pre-trained models are provided on an "as is" basis, without
+    warranties or conditions of any kind.
+
+    Args:
+        backbone: A `keras_nlp.models.FNetBackbone` instance.
+        num_classes: int. Number of classes to predict.
+        hidden_dim: int. The size of the pooler layer.
+        dropout: float. The dropout probability value, applied after the dense
+            layer.
+        preprocessor: A `keras_nlp.models.FNetPreprocessor` or `None`. If
+            `None`, this model will not apply preprocessing, and inputs should
+            be preprocessed before calling the model.
+
+    Example usage:
+    ```python
+    preprocessed_features = {
+        "token_ids": tf.ones(shape=(2, 12), dtype=tf.int64),
+        "segment_ids": tf.constant(
+            [[0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0]] * 2, shape=(2, 12)
+        ),
+        "padding_mask": tf.constant(
+            [[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]] * 2, shape=(2, 12)
+        ),
+    }
+    labels = [0, 3]
+
+    # Randomly initialize a Fnet backbone
+    backbone = keras_nlp.models.FNetBackbone(
+        vocabulary_size=32000,
+        num_layers=12,
+        num_heads=12,
+        hidden_dim=768,
+        intermediate_dim=3072,
+        max_sequence_length=12,
+    )
+
+    # Create a Fnet classifier and fit your data.
+    classifier = keras_nlp.models.FnetClassifier(
+        backbone,
+        num_classes=4,
+        preprocessor=None,
+    )
+    classifier.compile(
+        loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
+    )
+    classifier.fit(x=preprocessed_features, y=labels, batch_size=2)
+
+    # Access backbone programatically (e.g., to change `trainable`)
+    classifier.backbone.trainable = False
+    ```
+    """
+
+    def __init__(
+        self,
+        backbone,
+        num_classes=2,
+        dropout=0.1,
+        preprocessor=None,
+        **kwargs,
+    ):
+        inputs = backbone.input
+        pooled = backbone(inputs)["pooled_output"]
+        pooled = keras.layers.Dropout(dropout)(pooled)
+        outputs = keras.layers.Dense(
+            num_classes,
+            kernel_initializer=f_net_kernel_initializer(),
+            name="logits",
+        )(pooled)
+        # Instantiate using Functional API Model constructor
+        super().__init__(
+            inputs=inputs,
+            outputs=outputs,
+            include_preprocessing=preprocessor is not None,
+            **kwargs,
+        )
+        # All references to `self` below this line
+        self._backbone = backbone
+        self._preprocessor = preprocessor
+        self.num_classes = num_classes
+        self.dropout = dropout
+
+    def get_config(self):
+        config = super().get_config()
+        config.update(
+            {
+                "num_classes": self.num_classes,
+                "dropout": self.dropout,
+            }
+        )
+        return config
+
+    @classproperty
+    def backbone_cls(cls):
+        return FNetBackbone
+
+    @classproperty
+    def preprocessor_cls(cls):
+        return FNetPreprocessor
+
+    @classproperty
+    def presets(cls):
+        return copy.deepcopy(backbone_presets)
diff --git a/keras_nlp/models/gpt2/gpt2_backbone.py b/keras_nlp/models/gpt2/gpt2_backbone.py
index 17aec139..b52778ae 100644
--- a/keras_nlp/models/gpt2/gpt2_backbone.py
+++ b/keras_nlp/models/gpt2/gpt2_backbone.py
@@ -187,6 +187,10 @@ class GPT2Backbone(Backbone):
         )
         return config
 
+    @property
+    def token_embedding(self):
+        return self.get_layer("token_embedding")
+
     @classproperty
     def presets(cls):
         return copy.deepcopy(backbone_presets)
diff --git a/keras_nlp/models/gpt2/gpt2_tokenizer.py b/keras_nlp/models/gpt2/gpt2_tokenizer.py
index d6ce5ab6..9665391a 100644
--- a/keras_nlp/models/gpt2/gpt2_tokenizer.py
+++ b/keras_nlp/models/gpt2/gpt2_tokenizer.py
@@ -14,14 +14,12 @@
 """GPT-2 preprocessing layers."""
 
 import copy
-import os
 
 from tensorflow import keras
 
 from keras_nlp.models.gpt2.gpt2_presets import backbone_presets
 from keras_nlp.tokenizers.byte_pair_tokenizer import BytePairTokenizer
 from keras_nlp.utils.python_utils import classproperty
-from keras_nlp.utils.python_utils import format_docstring
 
 
 @keras.utils.register_keras_serializable(package="keras_nlp")
@@ -118,58 +116,3 @@ class GPT2Tokenizer(BytePairTokenizer):
     @classproperty
     def presets(cls):
         return copy.deepcopy(backbone_presets)
-
-    @classmethod
-    @format_docstring(names=", ".join(backbone_presets))
-    def from_preset(
-        cls,
-        preset,
-        **kwargs,
-    ):
-        """Instantiate a GPT-2 tokenizer from preset vocabulary and merge rules.
-
-        Args:
-            preset: string. Must be one of {{names}}.
-
-        Examples:
-        ```python
-        # Load a preset tokenizer.
-        tokenizer = keras_nlp.models.GPT2Tokenizer.from_preset(
-            "gpt2_base_en",
-        )
-        # Tokenize some input.
-        tokenizer("The quick brown fox tripped.")
-        # Detokenize some input.
-        tokenizer.detokenize([5, 6, 7, 8, 9])
-        ```
-        """
-
-        if preset not in cls.presets:
-            raise ValueError(
-                "`preset` must be one of "
-                f"""{", ".join(cls.presets)}. Received: {preset}."""
-            )
-        metadata = cls.presets[preset]
-
-        vocabulary = keras.utils.get_file(
-            "vocab.json",
-            metadata["vocabulary_url"],
-            cache_subdir=os.path.join("models", preset),
-            file_hash=metadata["vocabulary_hash"],
-        )
-        merges = keras.utils.get_file(
-            "merges.txt",
-            metadata["merges_url"],
-            cache_subdir=os.path.join("models", preset),
-            file_hash=metadata["merges_hash"],
-        )
-
-        config = metadata["preprocessor_config"]
-        config.update(
-            {
-                "vocabulary": vocabulary,
-                "merges": merges,
-            },
-        )
-
-        return cls.from_config({**config, **kwargs})
diff --git a/keras_nlp/models/roberta/roberta_backbone.py b/keras_nlp/models/roberta/roberta_backbone.py
index 5b06e0fc..ebeb9842 100644
--- a/keras_nlp/models/roberta/roberta_backbone.py
+++ b/keras_nlp/models/roberta/roberta_backbone.py
@@ -174,6 +174,10 @@ class RobertaBackbone(Backbone):
         )
         return config
 
+    @property
+    def token_embedding(self):
+        return self.get_layer("embeddings").token_embedding
+
     @classproperty
     def presets(cls):
         return copy.deepcopy(backbone_presets)
diff --git a/keras_nlp/models/roberta/roberta_preprocessor.py b/keras_nlp/models/roberta/roberta_preprocessor.py
index 9ae8ca8a..4d4203ce 100644
--- a/keras_nlp/models/roberta/roberta_preprocessor.py
+++ b/keras_nlp/models/roberta/roberta_preprocessor.py
@@ -17,7 +17,6 @@
 import copy
 
 import tensorflow as tf
-import tensorflow_text as tf_text
 from tensorflow import keras
 
 from keras_nlp.models.preprocessor import Preprocessor
@@ -28,6 +27,12 @@ from keras_nlp.utils.keras_utils import (
 )
 from keras_nlp.utils.keras_utils import pack_x_y_sample_weight
 from keras_nlp.utils.python_utils import classproperty
+from keras_nlp.utils.tf_utils import assert_tf_text_installed
+
+try:
+    import tensorflow_text as tf_text
+except ImportError:
+    tf_text = None
 
 
 @keras.utils.register_keras_serializable(package="keras_nlp")
@@ -236,6 +241,8 @@ class RobertaMultiSegmentPacker(keras.layers.Layer):
         truncate="round_robin",
         **kwargs,
     ):
+        assert_tf_text_installed(self.__class__.__name__)
+
         super().__init__(**kwargs)
         self.sequence_length = sequence_length
         if truncate not in ("round_robin", "waterfall"):
diff --git a/keras_nlp/models/roberta/roberta_tokenizer.py b/keras_nlp/models/roberta/roberta_tokenizer.py
index 6778bd42..933a28fc 100644
--- a/keras_nlp/models/roberta/roberta_tokenizer.py
+++ b/keras_nlp/models/roberta/roberta_tokenizer.py
@@ -15,14 +15,12 @@
 """RoBERTa tokenizer."""
 
 import copy
-import os
 
 from tensorflow import keras
 
 from keras_nlp.models.roberta.roberta_presets import backbone_presets
 from keras_nlp.tokenizers.byte_pair_tokenizer import BytePairTokenizer
 from keras_nlp.utils.python_utils import classproperty
-from keras_nlp.utils.python_utils import format_docstring
 
 
 @keras.utils.register_keras_serializable(package="keras_nlp")
@@ -126,58 +124,3 @@ class RobertaTokenizer(BytePairTokenizer):
     @classproperty
     def presets(cls):
         return copy.deepcopy(backbone_presets)
-
-    @classmethod
-    @format_docstring(names=", ".join(backbone_presets))
-    def from_preset(
-        cls,
-        preset,
-        **kwargs,
-    ):
-        """Instantiate a RoBERTa tokenizer from preset vocabulary and merge rules.
-
-        Args:
-            preset: string. Must be one of {{names}}.
-
-        Examples:
-        ```python
-        # Load a preset tokenizer.
-        tokenizer = keras_nlp.models.RobertaTokenizer.from_preset(
-            "roberta_base_en",
-        )
-        # Tokenize some input.
-        tokenizer("The quick brown fox tripped.")
-        # Detokenize some input.
-        tokenizer.detokenize([5, 6, 7, 8, 9])
-        ```
-        """
-
-        if preset not in cls.presets:
-            raise ValueError(
-                "`preset` must be one of "
-                f"""{", ".join(cls.presets)}. Received: {preset}."""
-            )
-        metadata = cls.presets[preset]
-
-        vocabulary = keras.utils.get_file(
-            "vocab.json",
-            metadata["vocabulary_url"],
-            cache_subdir=os.path.join("models", preset),
-            file_hash=metadata["vocabulary_hash"],
-        )
-        merges = keras.utils.get_file(
-            "merges.txt",
-            metadata["merges_url"],
-            cache_subdir=os.path.join("models", preset),
-            file_hash=metadata["merges_hash"],
-        )
-
-        config = metadata["preprocessor_config"]
-        config.update(
-            {
-                "vocabulary": vocabulary,
-                "merges": merges,
-            },
-        )
-
-        return cls.from_config({**config, **kwargs})
diff --git a/keras_nlp/models/xlm_roberta/xlm_roberta_tokenizer.py b/keras_nlp/models/xlm_roberta/xlm_roberta_tokenizer.py
index 3851e1db..a675e555 100644
--- a/keras_nlp/models/xlm_roberta/xlm_roberta_tokenizer.py
+++ b/keras_nlp/models/xlm_roberta/xlm_roberta_tokenizer.py
@@ -15,7 +15,6 @@
 """XLM-RoBERTa tokenizer."""
 
 import copy
-import os
 
 import tensorflow as tf
 from tensorflow import keras
@@ -23,7 +22,6 @@ from tensorflow import keras
 from keras_nlp.models.xlm_roberta.xlm_roberta_presets import backbone_presets
 from keras_nlp.tokenizers.sentence_piece_tokenizer import SentencePieceTokenizer
 from keras_nlp.utils.python_utils import classproperty
-from keras_nlp.utils.python_utils import format_docstring
 from keras_nlp.utils.tf_utils import tensor_to_string_list
 
 
@@ -162,52 +160,3 @@ class XLMRobertaTokenizer(SentencePieceTokenizer):
     @classproperty
     def presets(cls):
         return copy.deepcopy(backbone_presets)
-
-    @classmethod
-    @format_docstring(names=", ".join(backbone_presets))
-    def from_preset(
-        cls,
-        preset,
-        **kwargs,
-    ):
-        """Instantiate an XLM-RoBERTa tokenizer from preset vocabulary.
-
-        Args:
-            preset: string. Must be one of {{names}}.
-
-        Examples:
-        ```python
-        # Load a preset tokenizer.
-        tokenizer = keras_nlp.models.XLMRobertaTokenizer.from_preset(
-            "xlm_roberta_base_multi",
-        )
-
-        # Tokenize some input.
-        tokenizer("The quick brown fox tripped.")
-
-        # Detokenize some input.
-        tokenizer.detokenize(tf.constant([581, 63773, 119455, 6, 147797]))
-        ```
-        """
-        if preset not in cls.presets:
-            raise ValueError(
-                "`preset` must be one of "
-                f"""{", ".join(cls.presets)}. Received: {preset}."""
-            )
-        metadata = cls.presets[preset]
-
-        spm_proto = keras.utils.get_file(
-            "vocab.spm",
-            metadata["spm_proto_url"],
-            cache_subdir=os.path.join("models", preset),
-            file_hash=metadata["spm_proto_hash"],
-        )
-
-        config = metadata["preprocessor_config"]
-        config.update(
-            {
-                "proto": spm_proto,
-            },
-        )
-
-        return cls.from_config({**config, **kwargs})
diff --git a/keras_nlp/tokenizers/byte_pair_tokenizer.py b/keras_nlp/tokenizers/byte_pair_tokenizer.py
index 3b61007a..8d7e886c 100644
--- a/keras_nlp/tokenizers/byte_pair_tokenizer.py
+++ b/keras_nlp/tokenizers/byte_pair_tokenizer.py
@@ -20,6 +20,7 @@ but is TF graph compatible.
 """
 
 import json
+import os
 from typing import Iterable
 from typing import List
 
@@ -27,6 +28,8 @@ import tensorflow as tf
 from tensorflow import keras
 
 from keras_nlp.tokenizers import tokenizer
+from keras_nlp.utils.python_utils import classproperty
+from keras_nlp.utils.python_utils import format_docstring
 from keras_nlp.utils.tf_utils import assert_tf_text_installed
 
 try:
@@ -536,3 +539,90 @@ class BytePairTokenizer(tokenizer.Tokenizer):
             tokenized_words, axis=1, separator=" "
         )
         self.cache.insert(tokens, tokenized_words)
+
+    @classproperty
+    def presets(cls):
+        return {}
+
+    @classmethod
+    def from_preset(
+        cls,
+        preset,
+        **kwargs,
+    ):
+        """Instantiate {{model_name}} tokenizer from preset vocabulary.
+
+        Args:
+            preset: string. Must be one of "{{preset_names}}".
+
+        Examples:
+        ```python
+        # Load a preset tokenizer.
+        tokenizer = {{model_name}}.from_preset("{{example_preset_name}}")
+
+        # Tokenize some input.
+        tokenizer("The quick brown fox tripped.")
+
+        # Detokenize some input.
+        tokenizer.detokenize([5, 6, 7, 8, 9])
+        ```
+        """
+
+        if not cls.presets:
+            raise NotImplementedError(
+                "No presets have been created for this class"
+            )
+
+        if preset not in cls.presets:
+            raise ValueError(
+                "`preset` must be one of "
+                f"""{", ".join(cls.presets)}. Received: {preset}."""
+            )
+        metadata = cls.presets[preset]
+
+        vocabulary = keras.utils.get_file(
+            "vocab.json",
+            metadata["vocabulary_url"],
+            cache_subdir=os.path.join("models", preset),
+            file_hash=metadata["vocabulary_hash"],
+        )
+        merges = keras.utils.get_file(
+            "merges.txt",
+            metadata["merges_url"],
+            cache_subdir=os.path.join("models", preset),
+            file_hash=metadata["merges_hash"],
+        )
+
+        config = metadata["preprocessor_config"]
+        config.update(
+            {
+                "vocabulary": vocabulary,
+                "merges": merges,
+            },
+        )
+
+        return cls.from_config({**config, **kwargs})
+
+    def __init_subclass__(cls, **kwargs):
+        # Use __init_subclass__ to setup a correct docstring for from_preset.
+        super().__init_subclass__(**kwargs)
+
+        # If the subclass does not define from_preset, assign a wrapper so that
+        # each class can have an distinct docstring.
+        if "from_preset" not in cls.__dict__:
+
+            def from_preset(calling_cls, *args, **kwargs):
+                return super(cls, calling_cls).from_preset(*args, **kwargs)
+
+            cls.from_preset = classmethod(from_preset)
+
+        # Format and assign the docstring unless the subclass has overridden it.
+        if cls.from_preset.__doc__ is None:
+            cls.from_preset.__func__.__doc__ = (
+                BytePairTokenizer.from_preset.__doc__
+            )
+            format_docstring(
+                model_name=cls.__name__,
+                example_preset_name=next(iter(cls.presets), ""),
+                preset_names='", "'.join(cls.presets),
+            )(cls.from_preset.__func__)
diff --git a/keras_nlp/tokenizers/sentence_piece_tokenizer.py b/keras_nlp/tokenizers/sentence_piece_tokenizer.py
index a14276cd..fda70d4f 100644
--- a/keras_nlp/tokenizers/sentence_piece_tokenizer.py
+++ b/keras_nlp/tokenizers/sentence_piece_tokenizer.py
@@ -14,12 +14,15 @@
 
 import base64
 import binascii
+import os
 from typing import List
 
 import tensorflow as tf
 from tensorflow import keras
 
 from keras_nlp.tokenizers import tokenizer
+from keras_nlp.utils.python_utils import classproperty
+from keras_nlp.utils.python_utils import format_docstring
 from keras_nlp.utils.tf_utils import assert_tf_text_installed
 from keras_nlp.utils.tf_utils import tensor_to_string_list
 
@@ -206,3 +209,83 @@ class SentencePieceTokenizer(tokenizer.Tokenizer):
 
     def detokenize(self, inputs):
         return self._sentence_piece.detokenize(inputs)
+
+    @classproperty
+    def presets(cls):
+        return {}
+
+    @classmethod
+    def from_preset(
+        cls,
+        preset,
+        **kwargs,
+    ):
+        """Instantiate {{model_name}} tokenizer from preset vocabulary.
+
+        Args:
+            preset: string. Must be one of "{{preset_names}}".
+
+        Examples:
+        ```python
+        # Load a preset tokenizer.
+        tokenizer = {{model_name}}.from_preset("{{example_preset_name}}")
+
+        # Tokenize some input.
+        tokenizer("The quick brown fox tripped.")
+
+        # Detokenize some input.
+        tokenizer.detokenize([5, 6, 7, 8, 9])
+        ```
+        """
+
+        if not cls.presets:
+            raise NotImplementedError(
+                "No presets have been created for this class"
+            )
+
+        if preset not in cls.presets:
+            raise ValueError(
+                "`preset` must be one of "
+                f"""{", ".join(cls.presets)}. Received: {preset}."""
+            )
+        metadata = cls.presets[preset]
+
+        spm_proto = keras.utils.get_file(
+            "vocab.spm",
+            metadata["spm_proto_url"],
+            cache_subdir=os.path.join("models", preset),
+            file_hash=metadata["spm_proto_hash"],
+        )
+
+        config = metadata["preprocessor_config"]
+        config.update(
+            {
+                "proto": spm_proto,
+            },
+        )
+
+        return cls.from_config({**config, **kwargs})
+
+    def __init_subclass__(cls, **kwargs):
+        # Use __init_subclass__ to setup a correct docstring for from_preset.
+        super().__init_subclass__(**kwargs)
+
+        # If the subclass does not define from_preset, assign a wrapper so that
+        # each class can have an distinct docstring.
+        if "from_preset" not in cls.__dict__:
+
+            def from_preset(calling_cls, *args, **kwargs):
+                return super(cls, calling_cls).from_preset(*args, **kwargs)
+
+            cls.from_preset = classmethod(from_preset)
+
+        # Format and assign the docstring unless the subclass has overridden it.
+        if cls.from_preset.__doc__ is None:
+            cls.from_preset.__func__.__doc__ = (
+                SentencePieceTokenizer.from_preset.__doc__
+            )
+            format_docstring(
+                model_name=cls.__name__,
+                example_preset_name=next(iter(cls.presets), ""),
+                preset_names='", "'.join(cls.presets),
+            )(cls.from_preset.__func__)
diff --git a/keras_nlp/tokenizers/word_piece_tokenizer.py b/keras_nlp/tokenizers/word_piece_tokenizer.py
index 00ffccc6..ed593a32 100644
--- a/keras_nlp/tokenizers/word_piece_tokenizer.py
+++ b/keras_nlp/tokenizers/word_piece_tokenizer.py
@@ -12,6 +12,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+import os
 from typing import Iterable
 from typing import List
 
@@ -19,6 +20,8 @@ import tensorflow as tf
 from tensorflow import keras
 
 from keras_nlp.tokenizers import tokenizer
+from keras_nlp.utils.python_utils import classproperty
+from keras_nlp.utils.python_utils import format_docstring
 from keras_nlp.utils.tf_utils import assert_tf_text_installed
 
 try:
@@ -416,3 +419,83 @@ class WordPieceTokenizer(tokenizer.Tokenizer):
 
     def detokenize(self, inputs):
         return self._fast_word_piece.detokenize(inputs)
+
+    @classproperty
+    def presets(cls):
+        return {}
+
+    @classmethod
+    def from_preset(
+        cls,
+        preset,
+        **kwargs,
+    ):
+        """Instantiate {{model_name}} tokenizer from preset vocabulary.
+
+        Args:
+            preset: string. Must be one of "{{preset_names}}".
+
+        Examples:
+        ```python
+        # Load a preset tokenizer.
+        tokenizer = {{model_name}}.from_preset("{{example_preset_name}}")
+
+        # Tokenize some input.
+        tokenizer("The quick brown fox tripped.")
+
+        # Detokenize some input.
+        tokenizer.detokenize([5, 6, 7, 8, 9])
+        ```
+        """
+
+        if not cls.presets:
+            raise NotImplementedError(
+                "No presets have been created for this class"
+            )
+
+        if preset not in cls.presets:
+            raise ValueError(
+                "`preset` must be one of "
+                f"""{", ".join(cls.presets)}. Received: {preset}."""
+            )
+        metadata = cls.presets[preset]
+
+        vocabulary = keras.utils.get_file(
+            "vocab.txt",
+            metadata["vocabulary_url"],
+            cache_subdir=os.path.join("models", preset),
+            file_hash=metadata["vocabulary_hash"],
+        )
+
+        config = metadata["preprocessor_config"]
+        config.update(
+            {
+                "vocabulary": vocabulary,
+            },
+        )
+
+        return cls.from_config({**config, **kwargs})
+
+    def __init_subclass__(cls, **kwargs):
+        # Use __init_subclass__ to setup a correct docstring for from_preset.
+        super().__init_subclass__(**kwargs)
+
+        # If the subclass does not define from_preset, assign a wrapper so that
+        # each class can have an distinct docstring.
+        if "from_preset" not in cls.__dict__:
+
+            def from_preset(calling_cls, *args, **kwargs):
+                return super(cls, calling_cls).from_preset(*args, **kwargs)
+
+            cls.from_preset = classmethod(from_preset)
+
+        # Format and assign the docstring unless the subclass has overridden it.
+        if cls.from_preset.__doc__ is None:
+            cls.from_preset.__func__.__doc__ = (
+                WordPieceTokenizer.from_preset.__doc__
+            )
+            format_docstring(
+                model_name=cls.__name__,
+                example_preset_name=next(iter(cls.presets), ""),
+                preset_names='", "'.join(cls.presets),
+            )(cls.from_preset.__func__)


Test patch:
diff --git a/keras_nlp/models/f_net/f_net_classifier_test.py b/keras_nlp/models/f_net/f_net_classifier_test.py
new file mode 100644
index 00000000..cf4120fe
--- /dev/null
+++ b/keras_nlp/models/f_net/f_net_classifier_test.py
@@ -0,0 +1,145 @@
+# Copyright 2023 The KerasNLP Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Tests for FNet classification model."""
+
+import io
+import os
+
+import sentencepiece
+import tensorflow as tf
+from absl.testing import parameterized
+from tensorflow import keras
+
+from keras_nlp.models.f_net.f_net_backbone import FNetBackbone
+from keras_nlp.models.f_net.f_net_classifier import FNetClassifier
+from keras_nlp.models.f_net.f_net_preprocessor import FNetPreprocessor
+from keras_nlp.models.f_net.f_net_tokenizer import FNetTokenizer
+
+
+class FNetClassifierTest(tf.test.TestCase, parameterized.TestCase):
+    def setUp(self):
+        self.backbone = FNetBackbone(
+            vocabulary_size=1000,
+            num_layers=2,
+            hidden_dim=64,
+            intermediate_dim=128,
+            max_sequence_length=128,
+            name="encoder",
+        )
+
+        bytes_io = io.BytesIO()
+        vocab_data = tf.data.Dataset.from_tensor_slices(
+            ["the quick brown fox", "the earth is round"]
+        )
+
+        sentencepiece.SentencePieceTrainer.train(
+            sentence_iterator=vocab_data.as_numpy_iterator(),
+            model_writer=bytes_io,
+            vocab_size=10,
+            model_type="WORD",
+            pad_id=3,
+            unk_id=0,
+            bos_id=4,
+            eos_id=5,
+            pad_piece="<pad>",
+            unk_piece="<unk>",
+            bos_piece="[CLS]",
+            eos_piece="[SEP]",
+        )
+
+        self.proto = bytes_io.getvalue()
+
+        self.preprocessor = FNetPreprocessor(
+            tokenizer=FNetTokenizer(proto=self.proto),
+            sequence_length=12,
+        )
+
+        self.classifier = FNetClassifier(
+            self.backbone,
+            4,
+            preprocessor=self.preprocessor,
+        )
+        self.classifier_no_preprocessing = FNetClassifier(
+            self.backbone,
+            4,
+            preprocessor=None,
+        )
+
+        self.raw_batch = tf.constant(
+            [
+                "the quick brown fox.",
+                "the slow brown fox.",
+                "the smelly brown fox.",
+                "the old brown fox.",
+            ]
+        )
+        self.preprocessed_batch = self.preprocessor(self.raw_batch)
+        self.raw_dataset = tf.data.Dataset.from_tensor_slices(
+            (self.raw_batch, tf.ones((4,)))
+        ).batch(2)
+        self.preprocessed_dataset = self.raw_dataset.map(self.preprocessor)
+
+    def test_valid_call_classifier(self):
+        self.classifier(self.preprocessed_batch)
+
+    @parameterized.named_parameters(
+        ("jit_compile_false", False), ("jit_compile_true", True)
+    )
+    def test_fnet_classifier_predict(self, jit_compile):
+        self.classifier.compile(jit_compile=jit_compile)
+        self.classifier.predict(self.raw_batch)
+
+    @parameterized.named_parameters(
+        ("jit_compile_false", False), ("jit_compile_true", True)
+    )
+    def test_fnet_classifier_predict_no_preprocessing(self, jit_compile):
+        self.classifier_no_preprocessing.compile(jit_compile=jit_compile)
+        self.classifier_no_preprocessing.predict(self.preprocessed_batch)
+
+    @parameterized.named_parameters(
+        ("jit_compile_false", False), ("jit_compile_true", True)
+    )
+    def test_fnet_classifier_fit(self, jit_compile):
+        self.classifier.compile(
+            loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
+            jit_compile=jit_compile,
+        )
+        self.classifier.fit(self.raw_dataset)
+
+    @parameterized.named_parameters(
+        ("jit_compile_false", False), ("jit_compile_true", True)
+    )
+    def test_fnet_classifier_fit_no_preprocessing(self, jit_compile):
+        self.classifier_no_preprocessing.compile(
+            loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
+            jit_compile=jit_compile,
+        )
+        self.classifier_no_preprocessing.fit(self.preprocessed_dataset)
+
+    @parameterized.named_parameters(
+        ("tf_format", "tf", "model"),
+        ("keras_format", "keras_v3", "model.keras"),
+    )
+    def test_saved_model(self, save_format, filename):
+        model_output = self.classifier.predict(self.raw_batch)
+        save_path = os.path.join(self.get_temp_dir(), filename)
+        self.classifier.save(save_path, save_format=save_format)
+        restored_model = keras.models.load_model(save_path)
+
+        # Check we got the real object back.
+        self.assertIsInstance(restored_model, FNetClassifier)
+
+        # Check that output matches.
+        restored_output = restored_model.predict(self.raw_batch)
+        self.assertAllClose(model_output, restored_output)
diff --git a/keras_nlp/models/f_net/f_net_presets_test.py b/keras_nlp/models/f_net/f_net_presets_test.py
index b73b3d22..e92bd6d2 100644
--- a/keras_nlp/models/f_net/f_net_presets_test.py
+++ b/keras_nlp/models/f_net/f_net_presets_test.py
@@ -18,6 +18,7 @@ import tensorflow as tf
 from absl.testing import parameterized
 
 from keras_nlp.models.f_net.f_net_backbone import FNetBackbone
+from keras_nlp.models.f_net.f_net_classifier import FNetClassifier
 from keras_nlp.models.f_net.f_net_preprocessor import FNetPreprocessor
 from keras_nlp.models.f_net.f_net_tokenizer import FNetTokenizer
 
@@ -49,26 +50,46 @@ class FNetPresetSmokeTest(tf.test.TestCase, parameterized.TestCase):
         self.assertAllEqual(outputs, expected_outputs)
 
     @parameterized.named_parameters(
-        ("preset_weights", True), ("random_weights", False)
+        ("load_weights", True), ("no_load_weights", False)
     )
     def test_backbone_output(self, load_weights):
         input_data = {
-            "token_ids": tf.constant([[4, 97, 1467, 5]]),
+            "token_ids": tf.constant([[101, 1996, 4248, 102]]),
             "segment_ids": tf.constant([[0, 0, 0, 0]]),
+            "padding_mask": tf.constant([[1, 1, 1, 1]]),
         }
         model = FNetBackbone.from_preset(
             "f_net_base_en", load_weights=load_weights
         )
-        outputs = model(input_data)
+        outputs = model(input_data)["sequence_output"]
         if load_weights:
-            outputs = outputs["sequence_output"][0, 0, :5]
-            expected = [4.182479, -0.072181, -0.138097, -0.036582, -0.521765]
+            # The forward pass from a preset should be stable!
+            # This test should catch cases where we unintentionally change our
+            # network code in a way that would invalidate our preset weights.
+            # We should only update these numbers if we are updating a weights
+            # file, or have found a discrepancy with the upstream source.
+            outputs = outputs[0, 0, :5]
+            expected = [4.157282, -0.096616, -0.244943, -0.068104, -0.559592]
+            # Keep a high tolerance, so we are robust to different hardware.
             self.assertAllClose(outputs, expected, atol=0.01, rtol=0.01)
 
+    @parameterized.named_parameters(
+        ("load_weights", True), ("no_load_weights", False)
+    )
+    def test_classifier_output(self, load_weights):
+        input_data = tf.constant(["The quick brown fox."])
+        model = FNetClassifier.from_preset(
+            "f_net_base_en",
+            load_weights=load_weights,
+        )
+        # We don't assert output values, as the head weights are random.
+        model.predict(input_data)
+
     @parameterized.named_parameters(
         ("f_net_tokenizer", FNetTokenizer),
         ("f_net_preprocessor", FNetPreprocessor),
         ("f_net", FNetBackbone),
+        ("f_net_classifier", FNetClassifier),
     )
     def test_preset_docstring(self, cls):
         """Check we did our docstring formatting correctly."""
@@ -79,6 +100,7 @@ class FNetPresetSmokeTest(tf.test.TestCase, parameterized.TestCase):
         ("f_net_tokenizer", FNetTokenizer),
         ("f_net_preprocessor", FNetPreprocessor),
         ("f_net", FNetBackbone),
+        ("f_net_classifier", FNetClassifier),
     )
     def test_unknown_preset_error(self, cls):
         # Not a preset name
@@ -112,6 +134,41 @@ class FNetPresetFullTest(tf.test.TestCase, parameterized.TestCase):
             }
             model(input_data)
 
+    @parameterized.named_parameters(
+        ("load_weights", True), ("no_load_weights", False)
+    )
+    def test_load_fnet_classifier(self, load_weights):
+        for preset in FNetClassifier.presets:
+            classifier = FNetClassifier.from_preset(
+                preset,
+                load_weights=load_weights,
+            )
+            input_data = tf.constant(["This quick brown fox"])
+            classifier.predict(input_data)
+
+    @parameterized.named_parameters(
+        ("load_weights", True), ("no_load_weights", False)
+    )
+    def test_load_fnet_classifier_without_preprocessing(self, load_weights):
+        for preset in FNetClassifier.presets:
+            classifier = FNetClassifier.from_preset(
+                preset,
+                preprocessor=None,
+                load_weights=load_weights,
+            )
+            input_data = {
+                "token_ids": tf.random.uniform(
+                    shape=(1, 512),
+                    dtype=tf.int64,
+                    maxval=classifier.backbone.vocabulary_size,
+                ),
+                "segment_ids": tf.constant(
+                    [0] * 200 + [1] * 312, shape=(1, 512)
+                ),
+                "padding_mask": tf.constant([1] * 512, shape=(1, 512)),
+            }
+            classifier.predict(input_data)
+
     def test_load_tokenizers(self):
         for preset in FNetTokenizer.presets:
             tokenizer = FNetTokenizer.from_preset(preset)