The tokenizer is not custom (#1)
Browse files- The tokenizer is not custom (5278bf9953718797ce8674541d6c59e89ae3438f)
Co-authored-by: Sylvain Gugger <sgugger@users.noreply.huggingface.co>
README.md
CHANGED
|
@@ -11,8 +11,6 @@ This is the [distilbert-base-uncased-finetuned-sst-2-english](https://huggingfac
|
|
| 11 |
|
| 12 |
The source code is taken from Apple's [ml-ane-transformers](https://github.com/apple/ml-ane-transformers) GitHub repo, modified slightly to make it usable from the 🤗 Transformers library.
|
| 13 |
|
| 14 |
-
For more details about DistilBERT, we encourage users to check out [this model card](https://huggingface.co/distilbert-base-uncased).
|
| 15 |
-
|
| 16 |
## How to use
|
| 17 |
|
| 18 |
Usage example:
|
|
@@ -22,7 +20,7 @@ import torch
|
|
| 22 |
from transformers import AutoModelForSequenceClassification, AutoTokenizer
|
| 23 |
|
| 24 |
model_checkpoint = "apple/ane-distilbert-base-uncased-finetuned-sst-2-english"
|
| 25 |
-
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint
|
| 26 |
model = AutoModelForSequenceClassification.from_pretrained(
|
| 27 |
model_checkpoint, trust_remote_code=True, return_dict=False,
|
| 28 |
)
|
|
|
|
| 11 |
|
| 12 |
The source code is taken from Apple's [ml-ane-transformers](https://github.com/apple/ml-ane-transformers) GitHub repo, modified slightly to make it usable from the 🤗 Transformers library.
|
| 13 |
|
|
|
|
|
|
|
| 14 |
## How to use
|
| 15 |
|
| 16 |
Usage example:
|
|
|
|
| 20 |
from transformers import AutoModelForSequenceClassification, AutoTokenizer
|
| 21 |
|
| 22 |
model_checkpoint = "apple/ane-distilbert-base-uncased-finetuned-sst-2-english"
|
| 23 |
+
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
|
| 24 |
model = AutoModelForSequenceClassification.from_pretrained(
|
| 25 |
model_checkpoint, trust_remote_code=True, return_dict=False,
|
| 26 |
)
|