YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
scratch-model
This is a scratch transformer model created using the Incremental Model Trainer.
Model Configuration
- Architecture: Transformer decoder
- Parameters: 9.3M
- Hidden Size: 256
- Layers: 8
- Attention Heads: 4
- FFN Dimension: 512
- Vocabulary Size: 8000
- Max Sequence Length: 4096
- Dropout: 0.1
Usage
from trainer.scratch_model import ScratchModelConfig, ScratchTransformer
config = ScratchModelConfig.from_dict(json.load(open("config.json")))
model = ScratchTransformer.from_pretrained(".", config)
Created with Incremental Model Trainer
- Downloads last month
- 9
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support