NTransformer β Neuratal's open-source PyTorch Transformer
NTransformer is not a pretrained model; this page documents the project, not model weights.
NTransformer is an independently developed, PyTorch-based, decoder-only Transformer research/engineering framework by Neuratal (https://neuratal.com).
- Not pretrained. No weights are distributed with the repository; outputs from an untrained preset are gibberish.
- NTokenizer-native. Vocab 262,144; token IDs consumed verbatim; SHA-256 fingerprint-verified checkpoints against the NTokenizer v0.3 artifact.
- Apache-2.0. NTokenizer is separate (CC-BY-SA-4.0); NTransformer is Apache-2.0.
- Full NTokenizer integration: the entire pipeline β token IDs consumed verbatim, SHA-256 fingerprint bound to checkpoints, no BOS/EOS injection, correct special-token map. Designed to operate as the language model for NTokenizer v0.3-polished.
- Source + docs: https://github.com/Neuratal/NTransformer
- README: https://github.com/Neuratal/NTransformer/blob/master/README.md
- Latest release: https://github.com/Neuratal/NTransformer/releases/tag/v1.4.0
What it is
A research/engineering framework covering architecture, inference, training, and checkpointing β everything you need to train, evaluate and iterate on a decoder-only Transformer without vendoring a tokenizer, model hub, or hosted inference service. NTransformer is not a pretrained model; no trained weights are distributed. No quality claims are made. Use as a research tool, not as a production ML stack.
- Architecture block: decoder-only pre-norm; GQA, RoPE, SwiGLU, RMSNorm, factorized tied embeddings β all LLaMA-style.
- Long-context paths for 16K/32K sequences with gradient checkpointing.
- Tested: 460 tests, real release, honest release notes.
Why this is here
NTransformer needs a presence on Hugging Face so that the project card, tags and references live where serious ML practitioners browse. This page intentionally does not host a model; it points to the open-source repository.
Links
- GitHub repository: https://github.com/Neuratal/NTransformer
- NTokenizer (CC-BY-SA-4.0): https://huggingface.co/Neuratal/ntokenizer
- About Neuratal: https://neuratal.com/about
- License: Apache-2.0
Limitations (honest)
- No trained weights.
- Not production support; research-oriented.
- Multi-GPU/NCCL not verified on the development hardware.
- Benchmarks are development-machine numbers; do not treat as broad-hardware measurements.