royleibov commited on
Commit
33e78ff
1 Parent(s): 6bc66ab

Add zipnn to readme

Browse files
Files changed (1) hide show
  1. README.md +50 -4
README.md CHANGED
@@ -1,15 +1,58 @@
1
  ---
2
  library_name: transformers
3
  pipeline_tag: image-text-to-text
 
 
 
4
  ---
5
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6
  # Model Card for Model ID
7
  Transformers compatible pixtral checkpoints. Make sure to install from source or wait for v4.45!
8
 
9
  ```python
10
  from PIL import Image
11
  from transformers import AutoProcessor, LlavaForConditionalGeneration
12
- model_id = "mistral-community/pixtral-12b"
 
 
 
 
13
  model = LlavaForConditionalGeneration.from_pretrained(model_id)
14
  processor = AutoProcessor.from_pretrained(model_id)
15
 
@@ -61,7 +104,11 @@ Here's an example with text and multiple images interleaved in the same message:
61
  ```python
62
  from PIL import Image
63
  from transformers import AutoProcessor, LlavaForConditionalGeneration
64
- model_id = "mistral-community/pixtral-12b"
 
 
 
 
65
  model = LlavaForConditionalGeneration.from_pretrained(model_id)
66
  processor = AutoProcessor.from_pretrained(model_id)
67
 
@@ -107,5 +154,4 @@ Would you like more information on any specific aspect?
107
  ```
108
 
109
  While it may appear that spacing in the input is disrupted, this is caused by us skipping special tokens for display, and actually "Can this animal" and "live here" are
110
- correctly separated by image tokens. Try decoding with special tokens included to see exactly what the model sees!
111
-
 
1
  ---
2
  library_name: transformers
3
  pipeline_tag: image-text-to-text
4
+ license: apache-2.0
5
+ base_model:
6
+ - mistral-community/pixtral-12b
7
  ---
8
 
9
+ # Disclaimer and Requirements
10
+
11
+ This model is a clone of [**mistral-community/pixtral-12b**](https://huggingface.co/mistral-community/pixtral-12b) compressed using ZipNN. Compressed losslessly to 67% its original size, ZipNN saved ~9GB in storage and potentially ~113TB in data transfer **monthly**.
12
+
13
+ ### Requirement
14
+
15
+ In order to use the model, ZipNN is necessary:
16
+ ```bash
17
+ pip install zipnn
18
+ ```
19
+ ### Use This Model
20
+ ```python
21
+ # Load model directly
22
+ from transformers import AutoProcessor, AutoModelForPreTraining
23
+ from zipnn import zipnn_hf
24
+
25
+ zipnn_hf()
26
+
27
+ processor = AutoProcessor.from_pretrained("royleibov/pixtral-12b-ZipNN-Compressed")
28
+ model = AutoModelForPreTraining.from_pretrained("royleibov/pixtral-12b-ZipNN-Compressed")
29
+ ```
30
+ ### ZipNN
31
+ ZipNN also allows you to seemlessly save local disk space in your cache after the model is downloaded.
32
+
33
+ To compress the cached model, simply run:
34
+ ```bash
35
+ python zipnn_compress_path.py safetensors --model royleibov/pixtral-12b-ZipNN-Compressed --hf_cache
36
+ ```
37
+
38
+ The model will be decompressed automatically and safely as long as `zipnn_hf()` is added at the top of the file like in the [example above](#use-this-model).
39
+
40
+ To decompress manualy, simply run:
41
+ ```bash
42
+ python zipnn_decompress_path.py --model royleibov/pixtral-12b-ZipNN-Compressed --hf_cache
43
+ ```
44
+
45
  # Model Card for Model ID
46
  Transformers compatible pixtral checkpoints. Make sure to install from source or wait for v4.45!
47
 
48
  ```python
49
  from PIL import Image
50
  from transformers import AutoProcessor, LlavaForConditionalGeneration
51
+ from zipnn import zipnn_hf
52
+
53
+ zipnn_hf()
54
+
55
+ model_id = "royleibov/pixtral-12b-ZipNN-Compressed"
56
  model = LlavaForConditionalGeneration.from_pretrained(model_id)
57
  processor = AutoProcessor.from_pretrained(model_id)
58
 
 
104
  ```python
105
  from PIL import Image
106
  from transformers import AutoProcessor, LlavaForConditionalGeneration
107
+ from zipnn import zipnn_hf
108
+
109
+ zipnn_hf()
110
+
111
+ model_id = "royleibov/pixtral-12b-ZipNN-Compressed"
112
  model = LlavaForConditionalGeneration.from_pretrained(model_id)
113
  processor = AutoProcessor.from_pretrained(model_id)
114
 
 
154
  ```
155
 
156
  While it may appear that spacing in the input is disrupted, this is caused by us skipping special tokens for display, and actually "Can this animal" and "live here" are
157
+ correctly separated by image tokens. Try decoding with special tokens included to see exactly what the model sees!