Transformers
PyTorch
Safetensors
English
maba
maba-v1
maba-v1.1
architecture
recurrent
gated-deltanet
gdn
gdn-2
linear-attention
linear-recurrence
state-space-model
ssm
gqa
grouped-query-attention
swiglu
rmsnorm
rope
speculative-decoding
mtp
multi-token-prediction
scaling
100m
1b
3b
7b
30b
cpp
avx2
Instructions to use AndrewThompson1233/maba-v1-architecture with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use AndrewThompson1233/maba-v1-architecture with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("AndrewThompson1233/maba-v1-architecture", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Factorized embeddings intervention for GPT-2
17
#1 opened about 1 month ago
by
gpjt