Commit
•
148ea95
1
Parent(s):
dedf948
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,5 +1,11 @@
|
|
| 1 |
---
|
| 2 |
library_name: keras-hub
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
| 4 |
## Model Overview
|
| 5 |
FNet is a set of language models published by Google as part of the paper [FNet: Mixing Tokens with Fourier Transforms](https://arxiv.org/abs/2105.03824). FNet replaces the self-attention of BERT with an unparameterized fourier transform, dramatically lowering the number of trainable parameters in the model. FNet achieves training at 92-97% accuracy of BERT counterparts on GLUE benchmark, with faster training and much smaller saved checkpoints.
|
|
@@ -131,4 +137,4 @@ classifier = keras_hub.models.FNetClassifier.from_preset(
|
|
| 131 |
preprocessor=None,
|
| 132 |
)
|
| 133 |
classifier.fit(x=features, y=labels, batch_size=2)
|
| 134 |
-
```
|
|
|
|
| 1 |
---
|
| 2 |
library_name: keras-hub
|
| 3 |
+
license: apache-2.0
|
| 4 |
+
language:
|
| 5 |
+
- en
|
| 6 |
+
tags:
|
| 7 |
+
- text-classification
|
| 8 |
+
- keras
|
| 9 |
---
|
| 10 |
## Model Overview
|
| 11 |
FNet is a set of language models published by Google as part of the paper [FNet: Mixing Tokens with Fourier Transforms](https://arxiv.org/abs/2105.03824). FNet replaces the self-attention of BERT with an unparameterized fourier transform, dramatically lowering the number of trainable parameters in the model. FNet achieves training at 92-97% accuracy of BERT counterparts on GLUE benchmark, with faster training and much smaller saved checkpoints.
|
|
|
|
| 137 |
preprocessor=None,
|
| 138 |
)
|
| 139 |
classifier.fit(x=features, y=labels, batch_size=2)
|
| 140 |
+
```
|