johnrachwanpruna
commited on
Commit
•
b27a30e
1
Parent(s):
828b54a
Create README.md
Browse files
README.md
ADDED
@@ -0,0 +1,50 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
library_name: pruna-engine
|
3 |
+
thumbnail: "https://assets-global.website-files.com/646b351987a8d8ce158d1940/64ec9e96b4334c0e1ac41504_Logo%20with%20white%20text.svg"
|
4 |
+
metrics:
|
5 |
+
- memory_disk
|
6 |
+
- memory_inference
|
7 |
+
- inference_latency
|
8 |
+
- inference_throughput
|
9 |
+
- inference_CO2_emissions
|
10 |
+
- inference_energy_consumption
|
11 |
+
---
|
12 |
+
<!-- header start -->
|
13 |
+
<!-- 200823 -->
|
14 |
+
<div style="width: auto; margin-left: auto; margin-right: auto">
|
15 |
+
<a href="https://www.pruna.ai/" target="_blank" rel="noopener noreferrer">
|
16 |
+
<img src="https://i.imgur.com/eDAlcgk.png" alt="PrunaAI" style="width: 100%; min-width: 400px; display: block; margin: auto;">
|
17 |
+
</a>
|
18 |
+
</div>
|
19 |
+
<!-- header end -->
|
20 |
+
|
21 |
+
[![Twitter](https://img.shields.io/twitter/follow/PrunaAI?style=social)](https://twitter.com/PrunaAI)
|
22 |
+
[![GitHub](https://img.shields.io/github/followers/PrunaAI?label=Follow%20%40PrunaAI&style=social)](https://github.com/PrunaAI)
|
23 |
+
[![LinkedIn](https://img.shields.io/badge/LinkedIn-Connect-blue)](https://www.linkedin.com/company/93832878/admin/feed/posts/?feedType=following)
|
24 |
+
[![Discord](https://img.shields.io/badge/Discord-Join%20Us-blue?style=social&logo=discord)](https://discord.gg/CP4VSgck)
|
25 |
+
|
26 |
+
# Simply make AI models cheaper, smaller, faster, and greener!
|
27 |
+
|
28 |
+
- Give a thumbs up if you like this model!
|
29 |
+
- Contact us and tell us which model to compress next [here](https://www.pruna.ai/contact).
|
30 |
+
- Request access to easily compress your *own* AI models [here](https://z0halsaff74.typeform.com/pruna-access?typeform-source=www.pruna.ai).
|
31 |
+
- Read the documentations to know more [here](https://pruna-ai-pruna.readthedocs-hosted.com/en/latest/)
|
32 |
+
- Join Pruna AI community on Discord [here](https://discord.gg/CP4VSgck) to share feedback/suggestions or get help.
|
33 |
+
|
34 |
+
**Frequently Asked Questions**
|
35 |
+
- ***How does the compression work?*** The model is compressed by using bitsandbytes.
|
36 |
+
- ***How does the model quality change?*** The quality of the model output will slightly degrade.
|
37 |
+
- ***What is the model format?*** We the standard safetensors format.
|
38 |
+
- ***How to compress my own models?*** You can request premium access to more compression methods and tech support for your specific use-cases [here](https://z0halsaff74.typeform.com/pruna-access?typeform-source=www.pruna.ai).
|
39 |
+
|
40 |
+
## Usage
|
41 |
+
|
42 |
+
|
43 |
+
## Credits & License
|
44 |
+
|
45 |
+
The license of the smashed model follows the license of the original model. Please check the license of the original model mattshumer/Hermes-2-Pro-11B before using this model which provided the base model. The license of the `pruna-engine` is [here](https://pypi.org/project/pruna-engine/) on Pypi.
|
46 |
+
|
47 |
+
## Want to compress other models?
|
48 |
+
|
49 |
+
- Contact us and tell us which model to compress next [here](https://www.pruna.ai/contact).
|
50 |
+
- Request access to easily compress your own AI models [here](https://z0halsaff74.typeform.com/pruna-access?typeform-source=www.pruna.ai).
|