ANLP Assignment 2, Part 2: cautious

Decoder-only transformer pretrained on the human/AI parallel English corpus with a hand-written implementation of this optimizer.

Details

Optimizer: cautious
Learning rate: 0.0003
Parameters: 35.27M
Training tokens: 38.93M
Validation loss: 4.1639
Test BLEU: 2.06
Dataset: browndw/human-ai-parallel-corpus

Checkpoint is a plain torch.save payload with keys model, state and config. Load it with torch.load(..., weights_only=False) and feed model to the transformer defined in the accompanying repository.

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support