---
license: apache-2.0
tags:
- merge
language:
- en
---
![image/png](https://cdn-uploads.huggingface.co/production/uploads/6589d7e6586088fd2784a12c/IPxdU9JFGcrEmqzOdV46Z.png)
# It will run...but on what hardware?

## 512 experts in one MoE

[A giant merge of all the same model](https://huggingface.co/ahxt/LiteLlama-460M-1T)