File size: 421 Bytes
9633f0e 51848bb 5cdcc89 51848bb 5cdcc89 51848bb 5cdcc89 51848bb 5cdcc89 51848bb 9633f0e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
---
license: apache-2.0
---
```
#llama3_70b
#trained no_robot dataset
#using fsdp_qlora 8 GPU cluster
#git clone https://github.com/bigsnarfdude/fsdp-qlora
import transformers
import torch
model_id = "vincentoh/llama3_70b_no_robot_fsdp_qlora"
pipeline = transformers.pipeline("text-generation", model=model_id, model_kwargs={"torch_dtype": torch.bfloat16}, device_map="auto")
pipeline("Why is the sky blue?")
``` |