Clarify transformer-style usage for custom D3PM runtime
Browse files
README.md
CHANGED
|
@@ -39,7 +39,7 @@ print(predict("dharmo rakṣati rakṣitaḥ")["output"])
|
|
| 39 |
|
| 40 |
## Transformer-Style Usage (Recommended)
|
| 41 |
|
| 42 |
-
Use this model
|
| 43 |
|
| 44 |
```python
|
| 45 |
import torch
|
|
@@ -68,9 +68,16 @@ def generate(text: str):
|
|
| 68 |
print(generate("yadā mano nivarteta viṣayebhyaḥ svabhāvataḥ"))
|
| 69 |
```
|
| 70 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 71 |
## About `transformers` Compatibility
|
| 72 |
|
| 73 |
- This repo does not expose `config.json` + `model.safetensors` in `transformers` format.
|
|
|
|
| 74 |
- If you want full `AutoModel`/`pipeline` compatibility, you must create a wrapper architecture and export weights into HF Transformers conventions.
|
| 75 |
- For production today, use:
|
| 76 |
- `inference_api.py` for Python apps
|
|
|
|
| 39 |
|
| 40 |
## Transformer-Style Usage (Recommended)
|
| 41 |
|
| 42 |
+
Use this model like a transformer pipeline pattern: load once, call `generate(text)` many times.
|
| 43 |
|
| 44 |
```python
|
| 45 |
import torch
|
|
|
|
| 68 |
print(generate("yadā mano nivarteta viṣayebhyaḥ svabhāvataḥ"))
|
| 69 |
```
|
| 70 |
|
| 71 |
+
### Minimal 3-Step Pattern
|
| 72 |
+
|
| 73 |
+
1. `load_model(...)` once at app startup
|
| 74 |
+
2. `encode -> model.generate(...) -> decode` for each request
|
| 75 |
+
3. Reuse loaded model/tokenizers for all requests
|
| 76 |
+
|
| 77 |
## About `transformers` Compatibility
|
| 78 |
|
| 79 |
- This repo does not expose `config.json` + `model.safetensors` in `transformers` format.
|
| 80 |
+
- This is not a PEFT/LoRA adapter repository.
|
| 81 |
- If you want full `AutoModel`/`pipeline` compatibility, you must create a wrapper architecture and export weights into HF Transformers conventions.
|
| 82 |
- For production today, use:
|
| 83 |
- `inference_api.py` for Python apps
|