File size: 5,240 Bytes
8b40026 0e3e017 66836c3 8b40026 b973c6f 330b099 3ad5821 330b099 b973c6f 330b099 0e3e017 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 |
---
library_name: keras-hub
license: apache-2.0
language:
- en
tags:
- text-classification
pipeline_tag: text-classification
---
## Model Overview
BERT (Bidirectional Encoder Representations from Transformers) is a set of language models published by Google. They are intended for classification and embedding of text, not for text-generation. See the model card below for benchmarks, data sources, and intended use cases.
Weights and Keras model code are released under the [Apache 2 License](https://github.com/keras-team/keras-hub/blob/master/LICENSE).
## Links
* [Bert Quickstart Notebook](https://www.kaggle.com/code/matthewdwatson/bert-quickstart)
* [Bert API Documentation](https://keras.io/api/keras_hub/models/bert/)
* [Bert Model Card](https://github.com/google-research/bert/blob/master/README.md)
* [KerasHub Beginner Guide](https://keras.io/guides/keras_hub/getting_started/)
* [KerasHub Model Publishing Guide](https://keras.io/guides/keras_hub/upload/)
## Installation
Keras and KerasHub can be installed with:
```
pip install -U -q keras-hub
pip install -U -q keras>=3
```
Jax, TensorFlow, and Torch come preinstalled in Kaggle Notebooks. For instruction on installing them in another environment see the [Keras Getting Started](https://keras.io/getting_started/) page.
## Presets
The following model checkpoints are provided by the Keras team. Full code examples for each are available below.
| Preset name | Parameters | Description |
|------------------------|------------|-------------------------------------------------------------------------------------------------|
| `bert_tiny_en_uncased` | 4.39M | 2-layer BERT model where all input is lowercased. |
| `bert_small_en_uncased` | 28.76M | 4-layer BERT model where all input is lowercased. |
| `bert_medium_en_uncased` | 41.37M | 8-layer BERT model where all input is lowercased. |
| `bert_base_en_uncased` | 109.48M | 12-layer BERT model where all input is lowercased. |
| `bert_base_en` | 108.31M | 12-layer BERT model where case is maintained. |
| `bert_base_zh` | 102.27M | 12-layer BERT model. Trained on Chinese Wikipedia. |
| `bert_base_multi` | 177.85M | 12-layer BERT model where case is maintained. |
| `bert_large_en_uncased` | 335.14M | 24-layer BERT model where all input is lowercased. |
| `bert_large_en` | 333.58M | 24-layer BERT model where case is maintained. |
## Example Usage
```python
import keras
import keras_hub
import numpy as np
```
Raw string data.
```python
features = ["The quick brown fox jumped.", "I forgot my homework."]
labels = [0, 3]
# Pretrained classifier.
classifier = keras_hub.models.BertClassifier.from_preset(
"bert_base_en_uncased",
num_classes=4,
)
classifier.fit(x=features, y=labels, batch_size=2)
classifier.predict(x=features, batch_size=2)
# Re-compile (e.g., with a new learning rate).
classifier.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=keras.optimizers.Adam(5e-5),
jit_compile=True,
)
# Access backbone programmatically (e.g., to change `trainable`).
classifier.backbone.trainable = False
# Fit again.
classifier.fit(x=features, y=labels, batch_size=2)
```
Preprocessed integer data.
```python
features = {
"token_ids": np.ones(shape=(2, 12), dtype="int32"),
"segment_ids": np.array([[0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0]] * 2),
"padding_mask": np.array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]] * 2),
}
labels = [0, 3]
# Pretrained classifier without preprocessing.
classifier = keras_hub.models.BertClassifier.from_preset(
"bert_base_en_uncased",
num_classes=4,
preprocessor=None,
)
classifier.fit(x=features, y=labels, batch_size=2)
```
## Example Usage with Hugging Face URI
```python
import keras
import keras_hub
import numpy as np
```
Raw string data.
```python
features = ["The quick brown fox jumped.", "I forgot my homework."]
labels = [0, 3]
# Pretrained classifier.
classifier = keras_hub.models.BertClassifier.from_preset(
"hf://keras/bert_base_en_uncased",
num_classes=4,
)
classifier.fit(x=features, y=labels, batch_size=2)
classifier.predict(x=features, batch_size=2)
# Re-compile (e.g., with a new learning rate).
classifier.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=keras.optimizers.Adam(5e-5),
jit_compile=True,
)
# Access backbone programmatically (e.g., to change `trainable`).
classifier.backbone.trainable = False
# Fit again.
classifier.fit(x=features, y=labels, batch_size=2)
```
Preprocessed integer data.
```python
features = {
"token_ids": np.ones(shape=(2, 12), dtype="int32"),
"segment_ids": np.array([[0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0]] * 2),
"padding_mask": np.array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]] * 2),
}
labels = [0, 3]
# Pretrained classifier without preprocessing.
classifier = keras_hub.models.BertClassifier.from_preset(
"hf://keras/bert_base_en_uncased",
num_classes=4,
preprocessor=None,
)
classifier.fit(x=features, y=labels, batch_size=2)
``` |