avsolatorio commited on
Commit
68c46a7
1 Parent(s): cc320bc

Add DEVELOP.md as guide for adding onnx models

Browse files

Signed-off-by: Aivin V. Solatorio <avsolatorio@gmail.com>

Files changed (1) hide show
  1. DEVELOP.md +35 -0
DEVELOP.md ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ONNX models generation
2
+
3
+ The transformers.js library uses ONNX models. To convert a Hugging Face model to ONNX, you can use the `optimum-cli` tool. The following commands show how to convert a Hugging Face model to ONNX and quantize it using the `optimum-cli` tool.
4
+
5
+ ```bash
6
+ MODEL_NAME=GIST-all-MiniLM-L6-v2
7
+
8
+ poetry run optimum-cli export onnx -m avsolatorio/${MODEL_NAME} ${MODEL_NAME}_onnx/
9
+ poetry run optimum-cli onnxruntime quantize \
10
+ --avx512 \
11
+ --onnx_model ${MODEL_NAME}_onnx -o ${MODEL_NAME}_onnx_quantized
12
+ ```
13
+
14
+ We then upload this to the huggingface model repository. The onnx models must be in the `onnx/` folder in the repository.
15
+
16
+ ```bash
17
+ git clone https://huggingface.co/avsolatorio/${MODEL_NAME}
18
+ cd ${MODEL_NAME}
19
+
20
+ if [ ! -d "onnx" ]; then
21
+ mkdir onnx && cd onnx
22
+
23
+ # Copy the onnx model
24
+ rsync -avP ../../${MODEL_NAME}_onnx/model.onnx onnx/
25
+
26
+ # Copy the quantized onnx model
27
+ rsync -avP ../../${MODEL_NAME}_onnx_quantized/model_quantized.onnx onnx/
28
+ rsync -avP ../../${MODEL_NAME}_onnx_quantized/ort_config.json onnx/
29
+
30
+ # Commit and push
31
+ git add onnx
32
+ git commit -m "Add onnx models"
33
+ git push origin main
34
+ fi
35
+ ```