Update README.md
Browse files
README.md
CHANGED
@@ -16,7 +16,7 @@ This insight opens up new avenues for inference speed, akin to MoE's selective a
|
|
16 |
By dynamically choosing model parameters for computation, we can substantially boost inference speed.
|
17 |
|
18 |
However, the widespread adoption of ReLU-based models in the LLM field remains limited.
|
19 |
-
Here we introduce a new 7B ReLU-based LLM, Bamboo (Github link:[https://github.com/SJTU-IPADS/Bamboo](https://github.com/SJTU-IPADS/Bamboo)),
|
20 |
which boasts nearly 85% sparsity and performance levels on par with [Mistral-7B](https://huggingface.co/mistralai/Mistral-7B-v0.1).
|
21 |
|
22 |
## Model Architecture
|
|
|
16 |
By dynamically choosing model parameters for computation, we can substantially boost inference speed.
|
17 |
|
18 |
However, the widespread adoption of ReLU-based models in the LLM field remains limited.
|
19 |
+
Here we introduce a new 7B ReLU-based LLM, Bamboo (Github link: [https://github.com/SJTU-IPADS/Bamboo](https://github.com/SJTU-IPADS/Bamboo)),
|
20 |
which boasts nearly 85% sparsity and performance levels on par with [Mistral-7B](https://huggingface.co/mistralai/Mistral-7B-v0.1).
|
21 |
|
22 |
## Model Architecture
|