Echo9Zulu commited on
Commit
84f505c
·
verified ·
1 Parent(s): 7573377

Update app.py

Browse files
Files changed (1) hide show
  1. app.py +53 -50
app.py CHANGED
@@ -3,59 +3,11 @@ import gradio as gr
3
  class ConversionTool:
4
  def __init__(self):
5
  # Initialize widgets
6
- self.intro = gr.Markdown("""
7
 
8
- ```
9
- ### # Optimum CLI Export Tool.. tool
10
-
11
- This tool helps organize the process of creating conversion commands when using Intel Optimum for Transformers to convert models outside of a script..
12
-
13
-
14
- My goal was to make it easier to construct commands for the [Optimum CLI conversion tool](https://huggingface.co/docs/optimum/main/en/intel/openvino/export) which enables converting models to the OpenVINO Intermediate Representation
15
- outside of the from.pretrained method used in Transformers with the OpenVINO related classes like OVModelForCausalLM, OVModelForSeq2SeqLM, OVModelForQuestionAnswering, etc.
16
-
17
- ## Usage
18
- Here I'm assuming you have followed the instructions in the documentation and have all your dependencies in order.
19
-
20
- Run to to get the latest.
21
- ```
22
- pip install --upgrade --upgrade-strategy eager optimum[openvino]
23
- ```
24
-
25
- Intended workflow:
26
- -Select parameters.
27
- -Copy command.
28
- -Execute in your environment.
29
-
30
- Note: Converstion can take a while.
31
- Expect slow performance and rejoice. After all, OpenVINO supports Intel CPUs from 6th gen forward, so you can
32
- squeeze performance
33
-
34
- ## Discussion
35
-
36
- Leveraging hardware acceleration from OpenVINO requires converting a model into an Intermediate format derived from ONNX. Basically the command we execute rebuilds the model graph from it's source to be optimized for how OpenVINO uses this graph in memory.
37
-
38
- Using OpenVINO effectively requires considering facts about what hardware the Intel PC running the code has available to it alongside details about the model you're working with.
39
-
40
-
41
-
42
-
43
- So here are some questions you should be able to answer before using this tool;
44
- - What data types does my CPU support?
45
- - What instruction sets?
46
- - How will I be using the model?
47
- - Do I have enough system memory for this task?
48
-
49
- Visit the [Intel Ark ]([Intel® Processors for PC, Laptops, Servers, and AI | Intel®](https://www.intel.com/content/www/us/en/products/details/processors.html)) product database to find this information. It's *the* ground truth on these sorts of specs. Even so, when testing with different model architecture
50
-
51
-
52
 
53
 
54
 
55
-
56
- """)
57
-
58
-
59
  self.model_input = gr.Textbox(
60
  label='Model',
61
  placeholder='Model ID on huggingface.co or path on disk',
@@ -301,4 +253,55 @@ Visit the [Intel Ark ]([Intel® Processors for PC, Laptops, Servers, and AI | I
301
  if __name__ == "__main__":
302
  tool = ConversionTool()
303
  app = tool.gradio_app()
304
- app.launch(share = False)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  class ConversionTool:
4
  def __init__(self):
5
  # Initialize widgets
6
+ self.intro = gr.Markdown(INTRODUCTION)
7
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8
 
9
 
10
 
 
 
 
 
11
  self.model_input = gr.Textbox(
12
  label='Model',
13
  placeholder='Model ID on huggingface.co or path on disk',
 
253
  if __name__ == "__main__":
254
  tool = ConversionTool()
255
  app = tool.gradio_app()
256
+ app.launch(share = False)
257
+
258
+
259
+
260
+ INTRODUCTION="""
261
+
262
+ ### # Optimum CLI Export Tool.. tool
263
+
264
+ This tool helps organize conversion commands when using Intel Optimum for Transformers and respects the order of positional arguments. Otherwise these commands can get quite nuanced to keep track of.
265
+
266
+ My goal was to make it easier to construct commands for the [Optimum CLI conversion tool](https://huggingface.co/docs/optimum/main/en/intel/openvino/export) which enables converting models to the OpenVINO Intermediate Representation
267
+ outside of the from.pretrained method used in Transformers with OpenVINO related classes like OVModelForCausalLM, OVModelForSeq2SeqLM, OVModelForQuestionAnswering, etc, which interface with the OpenVINO runtime.
268
+
269
+ ## Usage
270
+ Here I'm assuming you have followed the instructions in the documentation and have all your dependencies in order.
271
+
272
+ Run to to get the latest version of the neccessary extension for optimum:
273
+ ```
274
+ pip install --upgrade --upgrade-strategy eager optimum[openvino]
275
+ ```
276
+
277
+ Intended workflow:
278
+ -Select conversion parameters.
279
+ -Hit "Submit"
280
+ -Copy command.
281
+ -Execute in your environment.
282
+
283
+ Note: Converstion can take a while and will be resource intensive.
284
+
285
+
286
+ OpenVINO supports Intel CPUs from 6th gen forward, so you can squeeze performance out of older hardware with
287
+ different accuracy/performance tradeoffs than the popular quants of GGUFs.
288
+
289
+ ## Discussion
290
+
291
+ Leveraging CPU, GPU and NPU hardware acceleration from OpenVINO requires converting a model into an Intermediate format derived from ONNX.
292
+ The command we execute rebuilds the model graph from it's source to be optimized for how OpenVINO uses this graph in memory.
293
+
294
+ Using OpenVINO effectively requires considering facts about your Intel hardware. Visit the [Intel Ark ]([Intel® Processors for PC, Laptops, Servers, and AI | Intel®](https://www.intel.com/content/www/us/en/products/details/processors.html)) product database to find this information.
295
+
296
+ Here are some hardware questions you should be able to answer before using this tool;
297
+
298
+ - What data types does my CPU support?
299
+ - What instruction sets?
300
+ - How will I be using the model?
301
+ - Do I have enough system memory for this task?
302
+
303
+
304
+
305
+ It's *the* ground truth for Intel Hardware specs. Even so, when testing with different model architectures
306
+
307
+ """