stanford-crfm/levanter-backpacks-test

Levanter Backpacks

Example:

import torch
import transformers
from transformers import AutoModelForCausalLM


model_id = "ivanzhouyq/levanter-backpacks"
config = transformers.AutoConfig.from_pretrained(model_id, trust_remote_code=True)
torch_model = AutoModelForCausalLM.from_pretrained(model_id, config=config, trust_remote_code=True)
torch_model.eval()

input = torch.randint(0, 50264, (1, 512), dtype=torch.long)
torch_out = torch_model(
    input,
    position_ids=None,
)
torch_out = torch.nn.functional.softmax(torch_out.logits, dim=-1)

Note that you need to install safetensors together with transformers and torch to make it work.