Added truncation fallback
Browse files
README.md
CHANGED
|
@@ -20,6 +20,8 @@ Generate Answer: '["solid-phase microextraction"]'<br></br>
|
|
| 20 |
Tokenizer is same as the Google Flan-T5 base.<br></br>
|
| 21 |
```tokenizer = AutoTokenizer.from_pretrained("google/flan-t5-base")```
|
| 22 |
|
|
|
|
|
|
|
| 23 |
Pls refer to the <a href="https://huggingface.co/datasets/Universal-NER/Pile-NER-type">Universal-NER</a> for Licencing terms for the dataset.
|
| 24 |
|
| 25 |
# License
|
|
|
|
| 20 |
Tokenizer is same as the Google Flan-T5 base.<br></br>
|
| 21 |
```tokenizer = AutoTokenizer.from_pretrained("google/flan-t5-base")```
|
| 22 |
|
| 23 |
+
If text is getting truncated, increase max_new_tokens keyword argument in the model generate function.
|
| 24 |
+
|
| 25 |
Pls refer to the <a href="https://huggingface.co/datasets/Universal-NER/Pile-NER-type">Universal-NER</a> for Licencing terms for the dataset.
|
| 26 |
|
| 27 |
# License
|