Update README.md
Browse files
README.md
CHANGED
|
@@ -97,6 +97,12 @@ Official results with the [model evaluator](https://huggingface.co/spaces/autoev
|
|
| 97 |
|
| 98 |
lol
|
| 99 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 100 |
---
|
| 101 |
|
| 102 |
## Training procedure
|
|
|
|
| 97 |
|
| 98 |
lol
|
| 99 |
|
| 100 |
+
### How to run inference over a very long (30k+ tokens) document in batches?
|
| 101 |
+
|
| 102 |
+
See `summarize.py` in [the code for my hf space Document Summarization](https://huggingface.co/spaces/pszemraj/document-summarization/blob/main/summarize.py) :)
|
| 103 |
+
|
| 104 |
+
You can also use the same code to split a document into batches of 4096, etc., and run over those with the model. This is useful in situations where CUDA memory is limited.
|
| 105 |
+
|
| 106 |
---
|
| 107 |
|
| 108 |
## Training procedure
|