Add zipnn to readme
Browse files
README.md
CHANGED
@@ -1,15 +1,58 @@
|
|
1 |
---
|
2 |
library_name: transformers
|
3 |
pipeline_tag: image-text-to-text
|
|
|
|
|
|
|
4 |
---
|
5 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
6 |
# Model Card for Model ID
|
7 |
Transformers compatible pixtral checkpoints. Make sure to install from source or wait for v4.45!
|
8 |
|
9 |
```python
|
10 |
from PIL import Image
|
11 |
from transformers import AutoProcessor, LlavaForConditionalGeneration
|
12 |
-
|
|
|
|
|
|
|
|
|
13 |
model = LlavaForConditionalGeneration.from_pretrained(model_id)
|
14 |
processor = AutoProcessor.from_pretrained(model_id)
|
15 |
|
@@ -61,7 +104,11 @@ Here's an example with text and multiple images interleaved in the same message:
|
|
61 |
```python
|
62 |
from PIL import Image
|
63 |
from transformers import AutoProcessor, LlavaForConditionalGeneration
|
64 |
-
|
|
|
|
|
|
|
|
|
65 |
model = LlavaForConditionalGeneration.from_pretrained(model_id)
|
66 |
processor = AutoProcessor.from_pretrained(model_id)
|
67 |
|
@@ -107,5 +154,4 @@ Would you like more information on any specific aspect?
|
|
107 |
```
|
108 |
|
109 |
While it may appear that spacing in the input is disrupted, this is caused by us skipping special tokens for display, and actually "Can this animal" and "live here" are
|
110 |
-
correctly separated by image tokens. Try decoding with special tokens included to see exactly what the model sees!
|
111 |
-
|
|
|
1 |
---
|
2 |
library_name: transformers
|
3 |
pipeline_tag: image-text-to-text
|
4 |
+
license: apache-2.0
|
5 |
+
base_model:
|
6 |
+
- mistral-community/pixtral-12b
|
7 |
---
|
8 |
|
9 |
+
# Disclaimer and Requirements
|
10 |
+
|
11 |
+
This model is a clone of [**mistral-community/pixtral-12b**](https://huggingface.co/mistral-community/pixtral-12b) compressed using ZipNN. Compressed losslessly to 67% its original size, ZipNN saved ~9GB in storage and potentially ~113TB in data transfer **monthly**.
|
12 |
+
|
13 |
+
### Requirement
|
14 |
+
|
15 |
+
In order to use the model, ZipNN is necessary:
|
16 |
+
```bash
|
17 |
+
pip install zipnn
|
18 |
+
```
|
19 |
+
### Use This Model
|
20 |
+
```python
|
21 |
+
# Load model directly
|
22 |
+
from transformers import AutoProcessor, AutoModelForPreTraining
|
23 |
+
from zipnn import zipnn_hf
|
24 |
+
|
25 |
+
zipnn_hf()
|
26 |
+
|
27 |
+
processor = AutoProcessor.from_pretrained("royleibov/pixtral-12b-ZipNN-Compressed")
|
28 |
+
model = AutoModelForPreTraining.from_pretrained("royleibov/pixtral-12b-ZipNN-Compressed")
|
29 |
+
```
|
30 |
+
### ZipNN
|
31 |
+
ZipNN also allows you to seemlessly save local disk space in your cache after the model is downloaded.
|
32 |
+
|
33 |
+
To compress the cached model, simply run:
|
34 |
+
```bash
|
35 |
+
python zipnn_compress_path.py safetensors --model royleibov/pixtral-12b-ZipNN-Compressed --hf_cache
|
36 |
+
```
|
37 |
+
|
38 |
+
The model will be decompressed automatically and safely as long as `zipnn_hf()` is added at the top of the file like in the [example above](#use-this-model).
|
39 |
+
|
40 |
+
To decompress manualy, simply run:
|
41 |
+
```bash
|
42 |
+
python zipnn_decompress_path.py --model royleibov/pixtral-12b-ZipNN-Compressed --hf_cache
|
43 |
+
```
|
44 |
+
|
45 |
# Model Card for Model ID
|
46 |
Transformers compatible pixtral checkpoints. Make sure to install from source or wait for v4.45!
|
47 |
|
48 |
```python
|
49 |
from PIL import Image
|
50 |
from transformers import AutoProcessor, LlavaForConditionalGeneration
|
51 |
+
from zipnn import zipnn_hf
|
52 |
+
|
53 |
+
zipnn_hf()
|
54 |
+
|
55 |
+
model_id = "royleibov/pixtral-12b-ZipNN-Compressed"
|
56 |
model = LlavaForConditionalGeneration.from_pretrained(model_id)
|
57 |
processor = AutoProcessor.from_pretrained(model_id)
|
58 |
|
|
|
104 |
```python
|
105 |
from PIL import Image
|
106 |
from transformers import AutoProcessor, LlavaForConditionalGeneration
|
107 |
+
from zipnn import zipnn_hf
|
108 |
+
|
109 |
+
zipnn_hf()
|
110 |
+
|
111 |
+
model_id = "royleibov/pixtral-12b-ZipNN-Compressed"
|
112 |
model = LlavaForConditionalGeneration.from_pretrained(model_id)
|
113 |
processor = AutoProcessor.from_pretrained(model_id)
|
114 |
|
|
|
154 |
```
|
155 |
|
156 |
While it may appear that spacing in the input is disrupted, this is caused by us skipping special tokens for display, and actually "Can this animal" and "live here" are
|
157 |
+
correctly separated by image tokens. Try decoding with special tokens included to see exactly what the model sees!
|
|