|
--- |
|
pipeline_tag: text-generation |
|
tags: |
|
- text-generation-inference |
|
- backpack |
|
- backpackmodel |
|
library_name: transformers |
|
license: apache-2.0 |
|
datasets: |
|
- openwebtext |
|
language: |
|
- en |
|
--- |
|
|
|
# How to Get Started with the Model |
|
|
|
Please install `transformers`, `safetensors` and `torch` to use this model. |
|
|
|
```bash |
|
pip install transformers safetensors torch |
|
``` |
|
|
|
Run the following Python code: |
|
|
|
```python |
|
import torch |
|
import transformers |
|
from transformers import AutoModelForCausalLM |
|
|
|
|
|
model_id = "ivanzhouyq/levanter-backpack-1b-100k" |
|
config = transformers.AutoConfig.from_pretrained(model_id, trust_remote_code=True) |
|
torch_model = AutoModelForCausalLM.from_pretrained( |
|
model_id, |
|
config=config, |
|
trust_remote_code=True |
|
) |
|
torch_model.eval() |
|
|
|
input = torch.randint(0, 50264, (1, 512), dtype=torch.long) |
|
torch_out = torch_model(input, position_ids=None,) |
|
torch_out = torch.nn.functional.softmax(torch_out.logits, dim=-1) |
|
print(torch_out.shape) |
|
``` |
