WesScivetti/random_100M_short_1_10_20

Custom GPT-BERT checkpoint from the NPN filtered-corpus training experiments.

This repository includes custom architecture code for Transformers 5.x. Install transformers>=5,<6 and torch>=2.6 before loading the .bin checkpoint. Example:

from transformers import AutoModelForMaskedLM, AutoTokenizer

repo_id = "WesScivetti/random_100M_short_1_10_20"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForMaskedLM.from_pretrained(repo_id, trust_remote_code=True)
model.eval()