bigscience
/

tr1-13B-checkpoints

Model card Files Files and versions Community

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

YAML Metadata Warning: empty or missing yaml metadata in repo card (https://huggingface.co/docs/hub/model-cards#model-card-metadata)

160 intermediary checkpoints from the tr1-13B training

these models have a bug in them. While we are fixing things if you try to use any of these please run it through this script:

python -c '
import sys, torch
f=sys.argv[1]
sd=torch.load(f)
d=2048
for k in sd.keys():
    if k.endswith(".attn.bias"):
        sd[k] = torch.tril(torch.ones((d, d), dtype=torch.float16)).view(1, 1, d, d)
torch.save(sd, f)
' global_step594/pytorch_model.bin

Downloads last month: 0

Unable to determine this model's library. Check the docs .