---
license: mit
tags:
- mamba
- pytorch
- Test Generation
- research abstract
datasets: pt-sk/research_papers_short
metrics: CrossEntropyLoss
---
This model uses Mamba Architecture trained on a research abstract dataset. 

* Optimizer: AdamW
* Leanring Rate: 0.001



Import the scripts from the code folder
```
from model import Mamba, ModelArgs
```


Loading Model
```
mamba_model = Mamba.from_pretrained("pt-sk/mamba").to("cuda")
```


Loading Tokenizer
```
tokenizer = AutoTokenizer.from_pretrained('pt-sk/mamba')
```

mamba_reserach file contains the state dict of optimizer and the model.