---
license: apache-2.0
language:
- en
---

## StripedHyena-Hessian-7B (SH-7B)


### Model Architecture

The architecture of StripedHyena-Hessian-7B is quite different from traditional decoder-only Transformers. 

StripedHyena is a hybrid architecture composed of multi-head, grouped-query attention and gated convolutions arranged in [Hyena](https://arxiv.org/abs/2302.10866) blocks.